In winter 2014/15 privacyIDEA was migrated – or should I say rewritten – based on Flask. At this point it was also possible to restructure the library layer resulting in an unmangling of the old Pylons-based code. Thus it resulted in the fact, that the database layer, the library layer and the API layer could run totally independent from each other. I.e. each database function and each library function could be tested, without starting up a Flask instance.
This also results in the nice side effect, that the tool pi-manage.py can be used to create realms and configurations, without the server running, as pi-manage.py can directly access the library and database level. In addition policies where implemented using python decorators.
Unit Tests
All this resulted in an easy way to write simple modular tests, which you can find here, and in a great test coverage of the server code of privacyIDEA. Unit tests are run at travis-ci and you can see the results here.
Code Coverage
During the test run on Travis CI a code coverage report is generated. You can see the code coverage at Coveralls.io.
It is good to have a reliable server component. Well, but this was only the server code.
UI Tests
During the update from version 2.2 to 2.3 a nasty bug occurred. Due to a new API call, which was only accessible by users with the role “admin”, non-admin users were not able to enroll tokens anymore. The bug is nasty, since it could have been easily avoided.
So I took action to setup a better UI testing. As privacyIDEA is an open source project and aims to give you maximum transparency, we now created a workflow based on Circle CI, Heroku and Ghost Inspector.
After a successful test at Circle CI (the built of Ubuntu packages is tested), the current code gets uploaded to a dyno at Heroku.
- "[[ ! -s \"$(git rev-parse – git-dir)/shallow\" ]] || git fetch – unshallow" - git push git@heroku.com:privacyidea-test.git $CIRCLE_SHA1:refs/heads/master - heroku pg:reset DATABASE_URL – confirm privacyidea-test - heroku run python ./pi-manage.py createdb – app privacyidea-test - heroku run – app privacyidea-test – python ./pi-manage.py admin add -p 'Test1234!' admin admin@localhost
- The database is resettet.
- The database tables are created and
- a default admin is created
Then we have a running Heroku dyno with the current privacyIDEA server code.
Finally we start up our UI tests at Chost Inspector.
We hope all this results in better code to suit your needs. But as important as tests is your feedback, which helps to improve code and features.
Thanks a lot!
