7. Performance & Profiling¶

What & why¶

People choose C++ for performance — then write slow C++ by optimizing the wrong things. Modern hardware rewards predictable memory access far more than clever arithmetic, and your intuition about where time goes is usually wrong. This lesson is about measuring first, understanding the machine, and applying the optimizations that actually pay off.

The rules¶

Don't guess. Profile. Measure where the time goes before changing anything.
Optimize the bottleneck. 90% of time is usually in <10% of the code.
Algorithm beats micro-optimization. O(n²) → O(n log n) dwarfs any constant-factor trick.
Memory access patterns beat instruction counts on modern CPUs.
Measure the change, in an optimized build, on realistic data.

Build for measurement¶

Profile optimized builds with debug info and frame pointers:

cmake -S . -B build-prof -DCMAKE_BUILD_TYPE=RelWithDebInfo \
      -DCMAKE_CXX_FLAGS="-fno-omit-frame-pointer"

Benchmarking a Debug (-O0) build tells you nothing — it can be 10–50× slower, with a completely different profile.

Useful flags:

Flag	Effect
`-O2`	solid optimization (default for Release in many setups)
`-O3`	more aggressive inlining and vectorization; measure, not always faster
`-march=native`	use every instruction your CPU supports (binary won't run on older CPUs)
`-flto`	link-time optimization: inline across translation units
`-fprofile-generate` / `-fprofile-use`	profile-guided optimization (PGO): often 10–20% free

Timing¶

#include <chrono>

auto start = std::chrono::steady_clock::now();
do_work();
auto elapsed = std::chrono::steady_clock::now() - start;
std::cout << std::chrono::duration<double, std::milli>(elapsed).count() << " ms\n";

Use steady_clock (monotonic), never system_clock, for durations.

Micro-benchmarks with Google Benchmark¶

Hand-rolled timing loops are easy to get wrong: the optimizer deletes unused results, caches are warm, runs are noisy. Use a framework:

#include <benchmark/benchmark.h>
#include <algorithm>
#include <random>
#include <vector>

static std::vector<int> random_ints(std::size_t n) {
    std::mt19937 gen{42};
    std::uniform_int_distribution<int> dist;
    std::vector<int> v(n);
    std::ranges::generate(v, [&] { return dist(gen); });
    return v;
}

static void BM_Sort(benchmark::State& state) {
    auto data = random_ints(state.range(0));
    for (auto _ : state) {
        state.PauseTiming();
        auto copy = data;
        state.ResumeTiming();
        std::ranges::sort(copy);
        benchmark::DoNotOptimize(copy.data());
    }
    state.SetComplexityN(state.range(0));
}
BENCHMARK(BM_Sort)->RangeMultiplier(4)->Range(1 << 10, 1 << 20)->Complexity();

BENCHMARK_MAIN();

FetchContent_Declare(benchmark GIT_REPOSITORY https://github.com/google/benchmark.git GIT_TAG v1.9.1)
set(BENCHMARK_ENABLE_TESTING OFF)
FetchContent_MakeAvailable(benchmark)
target_link_libraries(bench PRIVATE benchmark::benchmark)

benchmark::DoNotOptimize and benchmark::ClobberMemory stop the optimizer from deleting the work you're measuring.

Profilers¶

`perf` (Linux) — sampling, low overhead¶

perf stat ./app                      # cycles, instructions, cache misses, branch misses
perf record -g ./app                 # sample call stacks
perf report                          # interactive: hottest functions first

Visualize with flame graphs (Brendan Gregg's scripts or hotspot): the width of each box is the time spent in that function and everything it calls.

Others¶

macOS: Instruments (Time Profiler), xctrace.
Windows: Visual Studio Profiler, Windows Performance Analyzer, Superluminal.
Cross-platform: Intel VTune, AMD uProf, Tracy (frame profiler for games/real-time), Valgrind's callgrind (slow but exact) + KCachegrind.
Heap profilers: heaptrack, Valgrind massif — find allocation hot spots.

The memory hierarchy¶

Approximate latencies on a modern CPU:

Access	Latency
L1 cache	~1 ns (4 cycles)
L2 cache	~4 ns
L3 cache	~15 ns
Main memory (DRAM)	~80–100 ns

A cache miss to DRAM costs as much as hundreds of arithmetic instructions. Memory is fetched in 64-byte cache lines, and hardware prefetchers love sequential access.

Consequences:

std::vector beats std::list for nearly everything, even middle insertion at moderate sizes — contiguous memory wins.
Iterate in memory order. For a row-major 2D array, the inner loop should walk columns of a row:

// Fast: sequential
for (std::size_t r = 0; r < rows; ++r)
    for (std::size_t c = 0; c < cols; ++c)
        sum += grid[r * cols + c];

// Slow: strided — a cache miss per access for large grids
for (std::size_t c = 0; c < cols; ++c)
    for (std::size_t r = 0; r < rows; ++r)
        sum += grid[r * cols + c];

Pointer chasing is slow: std::map, linked lists, and vectors of unique_ptr scatter data across the heap. Consider flat alternatives (std::flat_map in C++23, sorted vectors, open-addressing hash maps like absl::flat_hash_map / boost::unordered_flat_map).

Data-oriented design¶

Lay data out for how it's processed, not how it's conceptually grouped:

// Array of Structs (AoS): updating positions drags names and colors through the cache
struct Particle { Vec3 pos; Vec3 vel; std::string name; Color color; };
std::vector<Particle> particles;

// Struct of Arrays (SoA): the hot loop touches only what it needs, contiguously
struct Particles {
    std::vector<Vec3> pos;
    std::vector<Vec3> vel;
    std::vector<std::string> name;
    std::vector<Color> color;
};

void step(Particles& p, float dt) {
    for (std::size_t i = 0; i < p.pos.size(); ++i) {
        p.pos[i] += p.vel[i] * dt;      // compilers can vectorize this (SIMD)
    }
}

Allocation¶

Heap allocation is expensive (tens to hundreds of nanoseconds, plus contention in multithreaded code) and scatters data.

reserve() vectors when you know the size.
Reuse buffers across loop iterations: clear() keeps capacity.
Avoid needless copies: pass by const&, move, use std::string_view / std::span.
Small string/buffer optimization: short std::strings (≤15 chars in libstdc++) don't allocate.
std::pmr (polymorphic memory resources) for arena/pool allocation:

#include <memory_resource>

std::array<std::byte, 64 * 1024> buffer;
std::pmr::monotonic_buffer_resource arena{buffer.data(), buffer.size()};
std::pmr::vector<std::pmr::string> words{&arena};   // allocations come from the stack buffer

Branches¶

Mispredicted branches cost ~15–20 cycles. For unpredictable data in hot loops:

Sort data first so branches become predictable, or
make code branchless: sum += (x > 0) * x;, or std::min/std::max (often compiled to conditional moves).
[[likely]] / [[unlikely]] hint rare paths (e.g. error handling). Measure — they rarely matter.

Letting the compiler help¶

Inlining needs visible definitions: small functions in headers, or LTO.
Vectorization (SIMD): simple loops over contiguous arrays with no aliasing and no early exits. Check with -fopt-info-vec (GCC) or -Rpass=loop-vectorize (Clang). Compiler Explorer shows the generated assembly.
Aliasing: if the compiler can't prove two pointers don't overlap, it reloads memory. Local copies of values used in loops help.
const and noexcept rarely change codegen directly, but noexcept moves enable faster container operations.

A realistic optimization workflow¶

Write a benchmark that reproduces the slow case with realistic data.
Profile it; find the top one or two hot spots.
Check algorithmic complexity first; then data structures and memory layout; then allocation; only then micro-optimizations.
Make one change at a time; re-measure; keep it only if it's significantly faster.
Keep tests passing — fast and wrong is worthless.

Common mistakes¶

Benchmarking Debug builds.
Optimizing without profiling.
Benchmarks the optimizer deletes — results unused, inputs constant.
Measuring once — noise, CPU frequency scaling, and warm-up distort single runs.
std::endl in hot output loops — flushes every line. Use '\n'.
std::list / std::map by default where a vector or hash map is faster.
Premature micro-optimization that makes code unreadable for no measurable gain.

Exercises¶

Benchmark std::vector vs std::list for inserting 100,000 random ints in sorted position (linear search for the position in both). Surprised?
Benchmark row-major vs column-major traversal of a 4096×4096 std::vector<int>. Use perf stat -e cache-misses to explain the difference.
Take the intermediate link-checker capstone, generate a large docs tree, and profile it with perf record. What's hot? (Hint: std::regex.) Replace the hottest regex with a hand-written scanner and measure.
Convert an AoS particle simulation to SoA and benchmark step() for 1 million particles.
Try PGO on a program of your choice: build with -fprofile-generate, run a representative workload, rebuild with -fprofile-use, compare.
Paste a simple summation loop into Compiler Explorer and find the vectorized instructions at -O3. Add a data-dependent break and watch vectorization disappear.

Recap¶

Measure first: optimized builds with debug info, proper benchmarks, sampling profilers, flame graphs.
Algorithms first, then memory layout, then allocations, then micro-tweaks.
Caches dominate: contiguous, sequential data wins; pointer chasing loses.
Data-oriented design (SoA) and fewer allocations (reserve, reuse, pmr) pay off in hot loops.
Let the compiler help: inlining, vectorization, LTO, PGO — and check its output.

Next: Undefined behavior & sanitizers →