[home] - [up] (toc) - [page 2]
What are some things missing from C++ that some other OO languages enjoy?
Technologies such as: unit test, contract, module, immutability, constraints, code coverage.
This series of blog posts will focus on unit test, and will touch upon code coverage.
To make up for lack of having unit test as part of the core language, there are things like Catch2 (for C++11 and later), Google Test, and Boost Test, and quite a few others.
All those fill the gap for C++ not having unit test built into the core language. The three I cited are all ones I recommend.
But because unit test isn’t part of the core language, none of these unit test frameworks are interchangeable. A program that opts into using one has quite a bit of work to switch to a different one.
Not a lot of C++ programmers use unit tests. And some C++ programmers do not know what a unit testing is all about. Before we can get to that, we need to establish a definition of a unit test.
“What you call unit testing is not how my company uses the term.”
Yes, and that just adds to the confusion. My apologies, but not my fault. Unfortunately, the software development industry has overloaded the term unit test to mean different things to different people in different contexts. Some use unit test to mean what I’d call an acceptance test, or an integration test, or something else.
In this blog post, I am using unit test strictly in the Test-Driven Design sense of the term.
A unit test exercises a single aspect of a single method or function. It has no dependencies, no branches or loops, and can be run in parallel (multi-threaded). They run very quickly, such that a large project with millions of lines of code can have a unit test suite that runs in a couple seconds. They follow the “arrange, act, assert” pattern. If a unit test fails, the point of failure in the code should be very obvious.
There is a counterpoint to DRY principle called WET principle. DRY is “Don't Repeat Yourself”, which encourages developers to coalesce behavior so that it is not copied or cut-n-pasted all over the source code. Having multiple copies is a maintenance nightmare. But for unit testing, WET is “Write Expressive Tests”, which is guidance that it is better for your unit tests to be complete unto themselves, not to have a communal setup/teardown. And definitely worth avoiding order dependency of the unit tests, or any shared global state. As a side benefit, avoiding global state bleeds into the source code under test, since the source cannot use mutable global state. A virtuous cycle, since mutable global state makes the source code difficult to reason about.
One misconception is that unit testing is primarily about testing. After all, it has the word “test” in there!
I’ve run into this misconception from other developers, quality engineers, and managers.
For Test-Driven Design (TDD), a unit test guides the implementation design of the program. It acts as programmer pressure to abide by many of the “good OO” principles such as SOLID, YAGNI, KISS, GRASP, and Package principles.
Why do OO languages have those “good OO” principles? To shore up the inadequacies and shortcomings of OO. TDD helps OO programmers do the right thing.
The value proposition of unit testing, in my estimation, is threefold:
To take an existing C++ code base that does not have unit tests, and to write up unit tests for that code base, will run into two problems.
Problem 1. Code not designed for unit testing will be nigh impossible to create unit tests. Refactoring the code so as to be able to create the unit tests means aggressive refactoring, but because there are no existing unit tests, it is refactoring blindly. Which is risky, and likely to introduce bugs.
Problem 2. The time taken to add unit tests is already after-the-fact. The return on investment is poor, because where the unit test has the most value is when designing the implementation.
In addition, a unit test suite is analogous to accounting’s double-entry bookkeeping. Does the code do what you intended? Tomorrow, does the code still do what you intended? A year from now, does the code still do what you intended? As code evolves, the basic correctness assumptions may get violated. Joe Rainsburger stated something along the lines that the vast majority of programming bugs are due to violating basic correctness.
The big differences between unit tests and other kinds of testing (integration, system, performance, stress, smoke, acceptance, security, et cetera):
Unit tests are not a substitute for other kinds of tests, nor vice versa. The Venn diagrams do not overlap.
Another misconception is TDD means you don’t have to do architecture, TDD will create the program organically. No, that is incorrect. If the team does not do architectural design the results will not meet requirements. As James Coplien argued, using TDD to make a banking system will wind up making a calculator.
At a prior company, we were using unit testing to help us with our project. The application was written in C♯ for .NET platform, in Visual Studio, using NUnit framework for unit tests, and NCrunch as the unit test runner.
We had about 70% code coverage. NCrunch is so wonderfully amazing that it would be running the unit tests continuously for instant feedback. The experience was incredible. Unit testing in that environment was not merely useful, but it was fun. Actually, bona fide fun.
Now, I am on a C++ project. C++ is not as amenable to unit testing compared to C♯ and all the supporting technologies.
One of my colleagues, Matt Fuerch, has taken it upon himself to bring unit testing to our project. Awesome! But many developers on the team are “Well, so what? What’s this unit testing all about? How does it impact me?”
To help demystify unit testing, I thought a very simple step-by-step example would show that unit testing is very, very easy. Nothing to be scared of, or daunted by. So easy that my example will not use any of the aforementioned unit testing frameworks. Instead, I will just roll my own and do it yourself.
I’ll also take the worst case scenario of retrofitting unit tests on a contrived example, which I will incrementally change as the example progresses. I will be relying on: C++17, Xcode 11.2, Clang (-fprofile-instr-generate -fcoverage-mapping), and LLVM (llvm-profdata, llvm-cov).
The same general principles apply regardless if you are using Clang, Visual Studio, GCC, or whatever.
Question? “Should I use a unit testing framework, or should I just do-it-yourself?”
Answer! Just for the sake of explaining I am rolling my own. I recommend using a framework. For smaller projects, Catch2 has everything you need, but you could roll your own. For larger projects, Google Test or Boost Test will be better.
An application with three routines that check the parameter and print out what category that number belongs.
I’m cheating a bit, because all these routines are leaf routines, and do not depend on other code. As such, I do not need to utilize mocks or fakes as stand-ins for other parts of the code.
In unit testing, you avoid having class instances depend on other class instances. Because then it isn't unit testing, it is integration testing. They can depend on the interfaces to those classes, and instead of using an instance of the other class, instead an instance of a mock is used. Making a mock in C++ may sound like a lot of work, but <sarcasm>rest assured</sarcasm>, since C++ does not have reflection, it is a lot of work.
Also, I’m starting with (contrived) pre-existing code, and adding in unit tests in arrears. Poor ROI as I mentioned above. I’m not trying to demonstrate how to do TDD here. (Note to self: future blog post on how to do TDD.)
Continue onto [page 2] - the starting source code.