C3DL Test Automation

My main assignment for this summer will be to add features to and improve the performance of C3DL. I believe that improving performance is impossible without an automatic and consistent way to gather evidence about what the performance actually is. It's also necessary to test that the engine is still rendering images correctly, so that we know immediately if something breaks.

In order to do both of these goals my first objective is to establish an automated testing suite. Because CDOT has given me a machine with a public facing IP address I can set up some sort of WIMP stack on it that can receive test results and store them in a database. Then I can set up some javascript test scripts that will run tests and deliver results to that stack.

The image rendering tests will be rendering a static image, and can be used in something like testswarm as a result. This will allow us to ensure cross-browser and operating system compatibility.

The performance tests will need to be run on machines that are guaranteed to be not doing anything else when they are run, but they don't need to be run on nearly as many machines or operating systems as the image tests because increases in the performance of our engine should be seen across all platforms when we make those increases. Eventually we may get into the area of coding specific enhancements for platform/browser/graphics cards, and I very much don't want to be working on this project if that happens, so I don't need to worry about that right now.

I'll be making a battery of tests that check very specific features and are designed for a machine to test them, unlike the human-oriented tests that we currently have. These tests will need to be designed in a way that allows some javascript code to automatically redirect between them once they have finished and the results have been delivered.

By storing the results of all the tests in a database we'll be able to make a timeline of rendering/performance increases across the lifespan of the project, showing quantitatively how much work we've been doing on it.

We can also show the results of our tests to the browser developers if they reveal bugs in the browsers.

Should be an exciting process! Although enabling the actual wimp stack itself is really boring right now.