Software Testing, references & bibliography

[Mario F.]

I'd like to do some software testing on my current project. It's a fairly complex set of code running on the hundredths of thousands of lines, OOP based, including both multi-threaded and single-threaded portions, and with its fair share of dependencies (boost, sqlite, ncurses).

I'd like for the first time to approach this in a better way, instead of the ad-hoc attitude I've always took towards software testing. However I lack the background and would appreciate some references and bibliography anyone could give me on the subject of Unit Testing in particular, and Software Testing in general. I'd also appreciate some suggestions on the matter of tools I could use. But most important for now is indeed learning about the workflow(s) and processes involved in software testing.

[cpjust]

That's what I get paid for.

The first thing I would do is come up with a set of Test Cases. I just use a spread sheet to enter my test cases into. I use these columns:

ID | Pre-condition setup | Test Steps | Expected Results | Pass/Fail

Give each test a unique ID (not required, but useful for organizing hundreds of test cases), list any steps you need to do before you run the tests in the Pre-condition Setup, put the explicit steps required in the Test Steps, put what should happen in Expected Results, and when you run the tests you can put a P or F in the Pass/Fail column. Only put one test per row, don't try to do too much in a single test; and don't make the test steps or expected results ambiguous in the slightest bit, otherwise it might affect the reproducibility of the test later.

Once you have a good set of manual tests you can run, you can try adding automated tests. Try looking into a framework like cppunit to organize your automated tests. These usually test individual functions or classes by passing valid & invalid values to them, as well as doing bounds checking (i.e. pass the minimumum, maximum, min - 1 & max + 1 values to see if they are handled properly), pass null pointers, empty strings...

[VirtualAce]

I've use both cppunit and EZunit. I both love and hate them. I find EZunit has too many macros but it is effective ..but very mundane programming.

I would also recommend using a code coverage tool such as Bullseye when running your unit tests. This will give you a fair indication of how much coverage you are getting and how you can tweak your unit tests to force errors. Most of the uncovered areas will be exception and error handlers. You may then have to force errors in order to exercise those sections of code. Generally 80% code coverage is what I shoot for with more being even better.

I do not have a lot of experience with automated testing so I cannot give any advice about that. There are several tools available which will do automated testing so you might check into some of those and see if they will work for you. There are also tools that plug right into Visual Studio that can simulate faults and put a load on the system which are very useful for testing.

[Dino]

I have a signed copy of this one:
Amazon.com: Software Testing: Testing Across the Entire Software Development Life Cycle (9780471793717): Gerald D. Everett, Raymond McLeod Jr.: Books

My buddy, Dr. Jerry Everett, is one of the authors.

[Mario F.]

Thanks folks. I should take it from here. Your pointers will get me there.

[VirtualAce]

Let me know what you come up with. I'm always looking for new code analysis and testing tools so if you come upon one that you really like I would appreciate if you would let me know either here or in a PM.