Skip to content

11. Testing with Catch2

What & why

Tests let you change code without fear. In C++, where a small mistake can corrupt memory silently, they matter even more — especially when run under sanitizers. Catch2 is a popular, easy-to-read testing framework. (GoogleTest is the other common choice; the concepts transfer directly.)

Setup with CMake

tests/CMakeLists.txt:

include(FetchContent)
FetchContent_Declare(
  Catch2
  GIT_REPOSITORY https://github.com/catchorg/Catch2.git
  GIT_TAG        v3.7.1
)
FetchContent_MakeAvailable(Catch2)

add_executable(tests test_stats.cpp)
target_link_libraries(tests PRIVATE stats_lib Catch2::Catch2WithMain)

include(Catch)
catch_discover_tests(tests)      # registers each TEST_CASE with CTest

Catch2WithMain supplies main() for you.

cmake -S . -B build
cmake --build build
ctest --test-dir build --output-on-failure
# or run the test binary directly for Catch2's own output:
./build/tests/tests

Your first tests

Say stats.hpp declares:

#pragma once
#include <optional>
#include <vector>

std::optional<double> mean(const std::vector<double>& values);
double median(std::vector<double> values);   // throws std::invalid_argument on empty

test_stats.cpp:

#include <catch2/catch_test_macros.hpp>
#include <catch2/matchers/catch_matchers_floating_point.hpp>

#include <stdexcept>

#include "stats/stats.hpp"

using Catch::Matchers::WithinAbs;

TEST_CASE("mean of an empty vector is empty") {
    REQUIRE_FALSE(mean({}).has_value());
}

TEST_CASE("mean of values") {
    auto m = mean({1.0, 2.0, 3.0, 4.0});
    REQUIRE(m.has_value());
    REQUIRE_THAT(*m, WithinAbs(2.5, 1e-12));
}

TEST_CASE("median", "[median]") {
    CHECK(median({3, 1, 2}) == 2);
    CHECK(median({4, 1, 3, 2}) == 2.5);
    CHECK_THROWS_AS(median({}), std::invalid_argument);
}

REQUIRE vs. CHECK

  • REQUIRE(expr) — on failure, stops this test case.
  • CHECK(expr) — on failure, records it and keeps going.

Use REQUIRE when later lines depend on it (e.g. has_value() before *m); CHECK otherwise.

On failure Catch2 shows the expanded values:

test_stats.cpp:22: FAILED:
  CHECK( median({4, 1, 3, 2}) == 2.5 )
with expansion:
  2.0 == 2.5

Other assertions

REQUIRE_FALSE(expr);
REQUIRE_THROWS(expr);
REQUIRE_THROWS_AS(expr, ExceptionType);
REQUIRE_THROWS_WITH(expr, "message");
REQUIRE_NOTHROW(expr);
REQUIRE_THAT(value, matcher);        // floating point, strings, ranges...

Never compare floating-point results with == unless they're exactly representable (like 2.5). Use WithinAbs / WithinRel.

Sections: shared setup

Each SECTION runs the test case from the top, so every section gets fresh setup:

TEST_CASE("Stack behavior") {
    Stack<int> s;
    s.push(1);
    s.push(2);

    SECTION("pop returns the last pushed value") {
        REQUIRE(s.pop() == 2);
        REQUIRE(s.size() == 1);
    }

    SECTION("popping everything empties it") {
        s.pop();
        s.pop();
        REQUIRE(s.empty());
        REQUIRE_THROWS_AS(s.pop(), std::out_of_range);
    }
}

Data-driven tests: generators

#include <catch2/generators/catch_generators.hpp>

TEST_CASE("is_prime") {
    auto [n, expected] = GENERATE(table<int, bool>({
        {0, false}, {1, false}, {2, true}, {3, true},
        {4, false}, {17, true}, {21, false}, {97, true},
    }));
    CAPTURE(n);                 // printed if the assertion fails
    REQUIRE(is_prime(n) == expected);
}

Tags and filtering

TEST_CASE("parses big files", "[parser][slow]") { ... }

./tests "[median]"         # only tests tagged median
./tests "~[slow]"          # everything except slow
./tests "mean*"            # by name wildcard
./tests --list-tests

Testing code with side effects

Design for testability: keep logic in pure functions, and push I/O to the edges.

Inject dependencies

Instead of a function that reads std::cin and writes std::cout, pass streams:

// hard to test
void run();

// easy to test
void run(std::istream& in, std::ostream& out);

TEST_CASE("run echoes uppercase") {
    std::istringstream in{"hello\n"};
    std::ostringstream out;
    run(in, out);
    REQUIRE(out.str() == "HELLO\n");
}

Files: use a temporary directory

#include <filesystem>
namespace fs = std::filesystem;

struct TempDir {
    fs::path path = fs::temp_directory_path() / ("test-" + std::to_string(std::rand()));
    TempDir() { fs::create_directories(path); }
    ~TempDir() { fs::remove_all(path); }     // RAII cleanup
};

TEST_CASE("save and load round-trip") {
    TempDir dir;
    auto file = dir.path / "tasks.txt";
    save_tasks(file, {{1, "Buy milk"}});
    auto loaded = load_tasks(file);
    REQUIRE(loaded.size() == 1);
    REQUIRE(loaded[0].text == "Buy milk");
}

Interfaces for the outside world

For clocks, networks, and random numbers, depend on an interface (lesson 7) or template parameter, and pass a fake in tests.

Run tests under sanitizers

The best thing you can do for C++ test quality:

cmake -S . -B build-asan -DCMAKE_BUILD_TYPE=Debug \
      -DCMAKE_CXX_FLAGS="-fsanitize=address,undefined -fno-omit-frame-pointer"
cmake --build build-asan
ctest --test-dir build-asan --output-on-failure

A test that "passes" while reading freed memory is a time bomb; AddressSanitizer turns it into a clear failure.

Coverage (GCC/Clang)

cmake -S . -B build-cov -DCMAKE_CXX_FLAGS="--coverage"
cmake --build build-cov && ctest --test-dir build-cov
gcovr -r . build-cov          # pip install gcovr

Coverage shows what isn't tested; high coverage doesn't prove correctness.

What to test

  • Normal cases — the typical inputs.
  • Edge cases — empty, one element, maximum size, zero, negative, duplicates.
  • Error cases — invalid input throws or returns an error.
  • Regressions — every bug you fix gets a test that would have caught it.

Good tests are fast, isolated (no shared state, no network), deterministic (seed your RNGs), and readable (the test name says what behavior is expected).

Common mistakes

  • Putting logic in main.cpp so tests can't reach it.
  • == on floating-point results.
  • Tests that depend on each other or on execution order.
  • Tests that touch real files in the working directory, the network, or the wall clock.
  • Only testing the happy path.
  • Never running tests under sanitizers.

Exercises

  1. Add Catch2 to your CMake project from lesson 10 and write tests for mean, median, and stddev, including edge cases.
  2. Write tests for the Fraction class: normalization, arithmetic, comparisons, and zero-denominator errors. Use GENERATE for arithmetic cases.
  3. Refactor the beginner capstone so the command functions take std::ostream& and a data-file path, then write tests for add, done, and remove in a temporary directory.
  4. Introduce a use-after-free bug on purpose, confirm the normal test run passes, then confirm the ASan build fails.
  5. Measure coverage with gcovr and add tests until every branch in Fraction is covered.

Recap

  • Catch2: TEST_CASE, SECTION, REQUIRE/CHECK, matchers, GENERATE, tags.
  • Wire tests into CMake with FetchContent + catch_discover_tests, run with ctest.
  • Design for testability: pure functions, injected streams/paths/interfaces.
  • Test normal, edge, error, and regression cases.
  • Always run the suite under AddressSanitizer + UBSan too.

Next: Capstone: link checker →