At SparkPost, we put a lot of time and effort into testing our code. Our platform is written in C, and recently I researched integrating with a unit testing framework called “CppUTest”, which provides xUnit-style testing for C/C++. This framework is robust, feature-rich, and under active development, which makes it a great choice. It also provides a C integration layer which made it easy to use with our platform C code even though most of the framework is C++. This tutorial covers how to get started with CppUTest on your own projects.

Downloading CppUTest

The CppUTest project page is available here, and the repository is on github. It’s also included in the package management repositories for many linux distros, as well as homebrew on Mac OS. The examples that follow were executed on Mac OS X, but they’re derived from code written for Red Hat, the OS our platform runs on.

The basics are well documented on CppUTest’s home page. We’re going to breeze through that and get to some of the more interesting features.

Laying the Foundation

First things first, let’s write some code!

Our test project will have a ‘main’ file and will include a utility library called ‘code’. The library will provide a simple function that returns 1 (for now). The files will be laid out like this:

├── src │ ├── code │ │ ├── code.cpp │ │ └── code.h │ └── main.cpp └── t ├── main.cpp └── test.cpp

Let’s start by writing the src/ files

// src/main.cpp #include <stdlib.h> #include <stdio.h> #include "code.h" int main(void) { test_func(); printf("hello world!\n"); exit(0); }

// src/code/code.cpp #include <stdlib.h> #include "code.h" int test_func () { return 1; }

// src/code/code.h #ifndef __code_h__ #define __code_h__ int test_func (); #endif

Now, let’s do the tests, which will live in the t/ directory.  The first thing to do is to set up a test runner which will run our test files. This is also the ‘main’  function that will execute once this is all compiled:

// t/main.cpp #include "CppUTest/CommandLineTestRunner.h" int main(int ac, char** av) { return CommandLineTestRunner::RunAllTests(ac, av); }

Now we can write our first test module:

// t/test.cpp #include "CppUTest/TestHarness.h" #include "code.h" TEST_GROUP(AwesomeExamples) { }; TEST(AwesomeExamples, FirstExample) { int x = test_func(); CHECK_EQUAL(1, x); }

Next, we need to write makefiles.  We’ll need two: one for the project files under src/, and one for the tests.

Project Makefile

The project makefile will be at the same level as the ‘src’ and ‘t’ directories at the root of the project. It should look like this:

# Makefile SRC_DIR=./src CODE_DIR=$(SRC_DIR)/code OUT=example TEST_DIR=t test: make -C $(TEST_DIR) test_clean: make -C $(TEST_DIR) clean code.o: gcc -c -I$(CODE_DIR) $(CODE_DIR)/code.cpp -o $(CODE_DIR)/code.o main: code.o gcc -I$(CODE_DIR) $(CODE_DIR)/code.o $(SRC_DIR)/main.cpp -o $(OUT) all: test main clean: test_clean rm $(SRC_DIR)/*.o $(CODE_DIR)/*.o $(OUT)

Note that this uses ‘make -C’  for the test targets – meaning that it will call ‘make’  again using the makefile in the test directory.

At this point we can compile the ‘src’ code with the makefile and see that it works:

[]$ make main gcc -c -I./src/code ./src/code/code.cpp -o ./src/code/code.o gcc -I./src/code ./src/code/code.o ./src/main.cpp -o example []$ ./example hello world!

Tests Makefile

For the tests, things are a little more involved since we need to properly load and integrate with the CppUTest library.

The CppUTest repository provides a file called “MakefileWorker.mk”. It provides a lot of functionality which makes building with CppUTest simple. The file lives under the “build” directory in the git repository. For this tutorial we’re going to assume it’s been copied to the ‘t/’ directory. It can be used as follows:

# we don’t want to use relative paths, so we set these variables PROJECT_DIR=/path/to/project SRC_DIR=$(PROJECT_DIR)/src TEST_DIR=$(PROJECT_DIR)/t # specify where the source code and includes are located INCLUDE_DIRS=$(SRC_DIR)/code SRC_DIRS=$(SRC_DIR)/code # specify where the test code is located TEST_SRC_DIRS = $(TEST_DIR) # what to call the test binary TEST_TARGET=example # where the cpputest library is located CPPUTEST_HOME=/usr/local # run MakefileWorker.mk with the variables defined here include MakefileWorker.mk

Note that CPPUTEST_HOME must be set to wherever CppUTest was installed. If you’ve installed a distro package, this will typically be under /usr/local on a linux/mac system. If you’ve checked out the repo on your own, it’s wherever that checkout is.

These options are all documented in MakefileWorker.mk.

MakefileWorker.mk also adds a few makefile targets, including the following:

1. all – builds the tests indicated by the makefile

2. clean – removes all of the object and gcov files generated for the tests

3. realclean – removes any object or gcov files in the entire directory tree

4. flags – lists all of the configured flags used to compile the tests

5. debug – lists all the source files, objects, dependencies, and ‘stuff to clean’

Code Coverage

Unit testing would not be complete without a coverage report. The go-to tool for this for projects using gcc is gcov, available as part of the standard suite of gcc utilities. Cpputest integrates easily with gcov, all you need to do is add this line to the makefile:

CPPUTEST_USE_GCOV=Y

Next, we need to make sure that the filterGcov.sh script from this repo is in ‘/scripts/filterGcov.sh’ relative to wherever you set ‘CPPUTEST_HOME’ to be. It also needs to have execute perms.

In the example Makefile, it would be deployed to ‘/usr/local/scripts/filterGcov.sh’. If you’re running CppUTest from a repo checkout, everything should work without modification.

With that in place, you can simply run ‘make gcov’ and the analysis will be generated for you. In our case we’ll need to ‘make -B’ to rebuild the object files with gcov enabled:

[]$ make -B gcov < compilation output > for d in /Users/ykuperman/code/blogpost/qa/src/code ; do \ FILES=`ls $d/*.c $d/*.cc $d/*.cpp 2> /dev/null` ; \ gcov --object-directory objs/$d $FILES >> gcov_output.txt 2>>gcov_error.txt ; \ done for f in ; do \ gcov --object-directory objs/$f $f >> gcov_output.txt 2>>gcov_error.txt ; \ done /usr/local/scripts/filterGcov.sh gcov_output.txt gcov_error.txt gcov_report.txt example.txt cat gcov_report.txt 100.00% /Users/ykuperman/code/blogpost/qa/src/code/code.cpp mkdir -p gcov mv *.gcov gcov mv gcov_* gcov See gcov directory for details

This will output a number of files to a new ‘gcov’ directory. These are:

1. code.cpp.gcov – the actual ‘gcov’ file for the code being tested

2. gcov_error.txt – an error report (in our case, it should be empty)

3. gcov_output.txt – the actual output of the gcov command that was run

4. gcov_report.txt – a summary of the coverage for each file under test

5. gcov_report.txt.html – an html version of gcov_report

Cpputest Memory Leak Detection

Cpputest allows you to automatically detect leaked memory by redefining the standard “malloc/free” family of functions to use its own wrappers instead. This allows it to quickly catch leaks and report them for each test execution. This is enabled by default in MakefileWorker.mk, so it’s already on with the steps outlined so far.

To illustrate, let’s leak some memory in test_func() !

Going back to code.c, we add a malloc()  to the function, like so:

int test_func() { malloc(1); return 1; }

Now, after recompiling, the following error is produced:

test.cpp:9: error: Failure in TEST(AwesomeExamples, FirstExample) Memory leak(s) found. Alloc num (4) Leak size: 1 Allocated at: ./code.c and line: 6. Type: "malloc" Memory: <0x7fc9e94056d0> Content: 0000: 00 |.| Total number of leaks: 1

This shows which test caused the leak, where the leak happened in the source code, and what was in the leaked memory. Very helpful!

There are a couple of caveats with this feature:

1. Cpputest is using preprocesser macros to dynamically redefine all calls to the standard memory management functions. That means it will only work for calls in the source code under test since that’s what’s compiled in with CppUTest’s overrides. Leaks in linked libraries will not be caught.

2. Sometimes memory which is allocated for the entire life of the process is not meant to be freed. This can make a lot of spammy errors if you’re testing a module with this behavior. To disable the leak detection, you can do this:

CPPUTEST_USE_MEM_LEAK_DETECTION=N

Interested in More?

This is just the tip of the iceberg when it comes to all the features contained in this tool. Besides the basics discussed here, it also has a mocking framework, a direct C integration layer, and a plugin framework, to name a few significant ones. The repo also contains a whole directory of helper scripts that can help automate some of the routine parts of working with the framework.

I hope the information here helps you improve your C/C++ code quality with this great tool!