Unit Test Coverage

S.Lott writes in his blog about unit test code-coverage: how much is enough?

Effective tests should account not only for code paths, but also input values and other application state or external environment that may affect the behavior.

For example, it may be easy to get 100% code coverage from tests for a function like the following:

divide(x, y) { return x/y; }

But unless you test for division-by-zero (when the parameter y is zero), you haven’t tested sufficiently.

The code-coverage metric doesn’t reveal when you’ve tested a good variety of input values. It only tests if your tests have visited the given lines of code, not what values were in each variable at the time.

Likewise for other application state besides input parameters. Values in other application objects, the contents of databases or files, or the operating system environment can all affect the proper functioning of a class or function that you’re testing. These variations are not measured by code-coverage metrics.

It could be argued that if you’re testing for external state, you aren’t doing unit testing by its strict definition; you’re doing functional or system testing. Nevertheless, most people rely chiefly on unit testing tools, because automated unit testing tools that generate code-coverage metrics are pretty easy to use.

While it’s a worthwhile goal to try to get high code-coverage in your unit-testing, a score of 100% doesn’t guarantee that you’ve tested enough. Likewise, a score below 100% isn’t necessarily an indication of inadequate testing. Code-coverage is therefore not a goal in itself; it’s one way of measuring one type of testing.