Printable View
Hey all,
I'm designing an automated unit test system for my engine and it has rapidly become clear that there is a nearly unlimited number of ways to do it. I was wondering if anyone around here had implemented something like this and how your strategy worked.
Which OS/Compiler ?
Developing on gcc and fc5, but robust to msvc 8.0 as well.
A couple of ideas - as you say, the possibilities are endless.
- Use http://www.swig.org/ to expose key parts of your API to various scripting languages.
- Using your gcc/fc5 platform, make lots of use of 'gcov' to keep an eye on the test coverage. Use this to guide you in creating new tests for the code other tests cannot reach.
- Create a 'smoke test' application around your engine, designed to exercise most of the mainline code. It should be a short test (a few seconds), so it can be run often. It's used to spot early any major breakage.
How are you going to measure "success"? Does your engine produce text files which can be compared with 'diff', or is it more complicated because you have lots of images?
If you want a good unit testing infrastructure, jake JUnit and copy the idea.
You can also use something like Bicycle to automate the test running and generate pretty graphs.
The scripting way seems good to me. Using my python interface this is what I have implemented since last night:
The makefile copies the test driver file over to the active directory and runs it with the directory to search for test files in (../src) and the search pattern for those files (test_*.py), then when it's done it cleans up and comes back. From there, the test driver searches out all of the files that match that pattern in that directory and runs them. In each test file, I inherit a class from UnitTest and run the test. UnitTest.start searches for all the functions in the inherited class that start with a prefix (test_) and runs them, watching for failed assertions and exceptions, then reports the results.
This is pretty handy because it means I can just put a test_whatever.py file next to the source file I'm testing, then put a class in there inherited from UnitTest with a bunch of test_thisThing functions and they will automatically run when I run make test.
Boost.Test is a unit testing framework for C++. Whether it lends itself well to testing a game engine is a different question, but at least parts - e.g. math routines - could be tested.
My impression is that ichijoji is already using a unit test framework. If that is so, what framework are you using, ichijoji?
Just the one I described before. It boils down to a simple unit testing system, but I'm not sure how I'm going to go about higher level testing.