How to test the tests?
Posted
by
Ryszard Szopa
on Programmers
See other posts from Programmers
or by Ryszard Szopa
Published on 2010-10-12T23:04:46Z
Indexed on
2011/02/20
23:32 UTC
Read the original article
Hit count: 283
We test our code to make it more correct (actually, less likely to be incorrect). However, the tests are also code -- they can also contain errors. And if your tests are buggy, they hardly make your code better.
I can think of three possible types of errors in tests:
Logical errors, when the programmer misunderstood the task at hand, and the tests do what he thought they should do, which is wrong;
Errors in the underlying testing framework (eg. a leaky mocking abstraction);
Bugs in the tests: the test is doing slightly different than what the programmer thinks it is.
Type (1) errors seem to be impossible to prevent (unless the programmer just... gets smarter). However, (2) and (3) may be tractable. How do you deal with these types of errors? Do you have any special strategies to avoid them? For example, do you write some special "empty" tests, that only check the test author's presuppositions? Also, how do you approach debugging a broken test case?
© Programmers or respective owner