# Build Systems Reference Makefile patterns, compiler flags, and build acceleration tools for C/systems developers. --- ## 1. Auto-Dependency Generation Compiler generates `.d` files listing what each `.o` depends on. On the next `make` run, those files are included and stale objects are rebuilt automatically. ```makefile DEPDIR := .deps DEPFLAGS = -MMD -MP -MF $(DEPDIR)/$*.d SRCS := $(wildcard src/*.c) OBJS := $(patsubst src/%.c, build/%.o, $(SRCS)) build/%.o: src/%.c | $(DEPDIR) $(CC) $(DEPFLAGS) $(CFLAGS) -c $< -o $@ $(DEPDIR): mkdir -p $@ -include $(patsubst src/%.c, $(DEPDIR)/%.d, $(SRCS)) ``` Flag breakdown: | Flag | Effect | |------|--------| | `-MMD` | Write a `.d` file with user-header deps (omits system headers) | | `-MP` | Add phony targets for each dep so deleted headers don't error | | `-MF ` | Write the dependency file to this path instead of stdout | | `-MD` | Like `-MMD` but includes system headers | The leading `-` on `-include` suppresses errors when `.d` files don't exist yet (first build). --- ## 2. GNU Make Automatic Variables | Variable | Expands to | |----------|-----------| | `$@` | The target filename | | `$<` | The first prerequisite | | `$^` | All prerequisites, deduplicated | | `$+` | All prerequisites, with duplicates preserved | | `$?` | Prerequisites newer than the target | | `$*` | The stem matched by `%` in a pattern rule | | `$(@D)` | Directory part of `$@` | | `$(@F)` | File part of `$@` | | `$( version.h ``` --- ## 5. Parallel Builds ```bash make -j$(nproc) # use all logical CPUs make -j$(nproc) -Otarget # serialize output per target (readable logs) make -j8 # fixed job count ``` Output sync options for `-O`: | Value | Behavior | |-------|----------| | `none` | No synchronization (default) | | `line` | Output one line at a time | | `target` | Buffer all output per target, print when done | | `recurse` | Like target but for recursive make | Limiting parallelism for specific targets: ```makefile # These two targets must not run simultaneously link: .NOTPARALLEL link: $(OBJS) $(CC) $^ -o $@ ``` Or use ordering prerequisites: ```makefile b: a # b waits for a, even with -j a: sleep 1 && echo a b: a echo b ``` --- ## 6. Non-Recursive Make with Includes Recursive make (calling `$(MAKE)` in subdirectories) loses dependency information across directories and is slow. The alternative is a single top-level Makefile that includes per-module `module.mk` files. **Top-level Makefile:** ```makefile # Reset before each include SRCS := CFLAGS_EXTRA := include src/lib/module.mk include src/app/module.mk include src/test/module.mk OBJS := $(patsubst %.c,build/%.o,$(SRCS)) ``` **src/lib/module.mk:** ```makefile # Paths relative to project root SRCS += src/lib/alloc.c \ src/lib/buf.c \ src/lib/hash.c # Module-specific flags appended to global compile rule CFLAGS_EXTRA += -Isrc/lib/include ``` **Pattern rule handles all modules:** ```makefile build/%.o: %.c @mkdir -p $(@D) $(CC) $(CFLAGS) $(CFLAGS_EXTRA) $(DEPFLAGS) -c $< -o $@ ``` Benefits: single pass, full cross-directory dependency tracking, faster than recursive invocations. --- ## 7. ccache ccache intercepts compiler calls, hashes inputs, and returns cached object files on hits. Typical speedup on a warm cache: 5-100x. ### Install ```bash # Arch sudo pacman -S ccache # Debian/Ubuntu sudo apt install ccache # macOS brew install ccache ``` ### Use in Makefile ```makefile CC := ccache gcc CXX := ccache g++ ``` Or prefix via environment without changing the Makefile: ```bash export CC="ccache gcc" export CXX="ccache g++" make -j$(nproc) ``` Or symlink method (intercepts without changing any config): ```bash export PATH="/usr/lib/ccache:$PATH" # /usr/lib/ccache contains gcc, g++, cc, c++ symlinks pointing to ccache ``` ### Key config commands ```bash ccache --show-stats # hit rate, cache size, cache dir ccache --show-config # effective config ccache --zero-stats # reset counters ccache --clear # purge all cached objects # Config (persistent) ccache --set-config max_size=10G ccache --set-config cache_dir=~/.cache/ccache ccache --set-config compression=true ccache --set-config compression_level=6 # Useful env overrides export CCACHE_DIR=~/.cache/ccache export CCACHE_MAXSIZE=10G export CCACHE_COMPRESS=1 export CCACHE_SLOPPINESS=pch_defines,time_macros # needed for PCH caching ``` ### Cache miss causes to eliminate - `__DATE__` / `__TIME__` macros (use `-DVERSION` instead) - Absolute paths baked in by `-I` or `__FILE__` (use `-ffile-prefix-map=old=new`) - Changing `umask` between builds --- ## 8. distcc distcc distributes compilation across a cluster. Each remote machine compiles preprocessed source sent by the local machine. Works transparently as a compiler wrapper. ### Install ```bash sudo pacman -S distcc # Arch sudo apt install distcc # Debian ``` ### Daemon setup (on each remote build host) ```bash # Edit /etc/conf.d/distccd or /etc/default/distcc DISTCC_ARGS="--allow 192.168.1.0/24 --jobs 8 --log-stderr" sudo systemctl enable --now distccd ``` ### Local configuration ```bash export DISTCC_HOSTS="localhost/4 192.168.1.10/8 192.168.1.11/8" # Format: host/max_jobs ``` ### Makefile integration ```makefile CC := distcc gcc CXX := distcc g++ ``` ### Pump mode (faster preprocessing) Pump mode offloads the preprocessor step to remotes too, reducing data sent over the wire: ```bash pump make -j32 CC="distcc gcc" ``` ### Combine distcc + ccache ```makefile CC := ccache distcc gcc CXX := ccache distcc g++ ``` ccache checks locally first; on a miss, distcc handles compilation remotely. ### Monitor ```bash distccmon-text 1 # refresh every 1 second distccmon-gnome # GUI monitor if installed ``` --- ## 9. bear for compile_commands.json `compile_commands.json` is required by clangd, clang-tidy, include-what-you-use, and most editors. `bear` intercepts the compiler via `LD_PRELOAD` and records every compilation command. ```bash sudo pacman -S bear # Arch sudo apt install bear # Debian ``` ### Generate ```bash bear -- make -j$(nproc) # Writes compile_commands.json in the current directory ``` ### Incremental update (bear 3.x) ```bash bear --append -- make changed_module.o ``` ### Alternatives ```bash # cmake generates it natively cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON .. # intercept-build (part of scan-build/clang-tools) intercept-build make # compiledb (Python, wraps make) pip install compiledb compiledb make ``` --- ## 10. mold Linker mold is a modern linker written in C++. On large projects it links 10-20x faster than GNU ld and 2-5x faster than lld. ### Install ```bash sudo pacman -S mold # Arch sudo apt install mold # Debian 12+ # Or from source git clone https://github.com/rui314/mold cd mold && cmake -DCMAKE_BUILD_TYPE=Release -B build && cmake --build build -j$(nproc) sudo cmake --install build ``` ### Use ```bash # Pass to compiler driver gcc -fuse-ld=mold -o prog $(OBJS) g++ -fuse-ld=mold -o prog $(OBJS) # Or set in Makefile LDFLAGS += -fuse-ld=mold ``` ### Makefile pattern ```makefile LD := $(CC) LDFLAGS := -fuse-ld=mold -Wl,--as-needed $(TARGET): $(OBJS) $(LD) $(LDFLAGS) $^ -o $@ ``` ### Notes - `-fuse-ld=mold` requires GCC 12+ or Clang 12+; on older compilers use `-B /usr/libexec/mold` - Not for static libraries (those use `ar`, not a linker) - ThinLTO is supported: add `-flto=thin` to both `CFLAGS` and `LDFLAGS` --- ## 11. Precompiled Headers Precompiling a large or stable header (like a project-wide `common.h` or a bundled `stb.h`) eliminates repeated parse time for every translation unit that includes it. ### Create a PCH ```makefile PCH_SRC := src/precompiled.h PCH_OUT := build/precompiled.h.gch $(PCH_OUT): $(PCH_SRC) | build $(CC) $(CFLAGS) -x c-header $< -o $@ ``` ### Use the PCH GCC looks for a `.gch` file alongside the original header automatically: ```makefile build/%.o: src/%.c $(PCH_OUT) $(CC) $(CFLAGS) -include src/precompiled.h -c $< -o $@ ``` The `-include` flag forces inclusion before anything in the source file. If `build/precompiled.h.gch` exists and was compiled with the same flags, GCC uses it automatically. ### Clang Clang requires explicit `-include-pch`: ```makefile $(PCH_OUT): $(PCH_SRC) $(CC) $(CFLAGS) -x c-header $< -o $@ build/%.o: src/%.c $(PCH_OUT) $(CC) $(CFLAGS) -include-pch $(PCH_OUT) -c $< -o $@ ``` ### ccache + PCH ```bash export CCACHE_SLOPPINESS=pch_defines,time_macros ``` Without this, ccache misses on PCH users because it is conservative about flag differences. --- ## 12. Unity Builds A unity build concatenates all `.c` files into one and compiles the single translation unit. The linker sees one object. Benefits: no repeated header parsing, better inlining across TUs, smaller binaries. Drawback: all-or-nothing rebuilds. ### Manual unity file ```c /* unity.c */ #include "src/alloc.c" #include "src/buf.c" #include "src/hash.c" #include "src/io.c" ``` ```makefile unity: build/unity.o $(CC) build/unity.o -o $@ build/unity.o: unity.c $(CC) $(CFLAGS) -c $< -o $@ ``` ### Generated unity file ```makefile SRCS := $(wildcard src/*.c) build/unity.c: $(SRCS) @mkdir -p $(@D) $(file > $@,/* generated */) $(foreach s,$^,$(file >> $@,#include "../$(s)")) build/unity.o: build/unity.c $(CC) $(CFLAGS) -c $< -o $@ ``` ### Hybrid: per-module unity + parallel compilation ```makefile # Each module gets its own unity file; modules compile in parallel MODULES := net io fs crypto define unity_rule build/$(1)-unity.c: $$(wildcard src/$(1)/*.c) $$(file > $$@,/* $(1) */) $$(foreach s,$$^,$$(file >> $$@,#include "../../$$s")) build/$(1)-unity.o: build/$(1)-unity.c $$(CC) $$(CFLAGS) -c $$< -o $$@ endef $(foreach m,$(MODULES),$(eval $(call unity_rule,$(m)))) ``` --- ## 13. Make Debugging ### Dry run and tracing ```bash make -n # print commands without running them make -n --always-make # -n but pretend everything is out of date make --trace # print why each target is rebuilt make --debug=a # verbose: all decisions, including up-to-date make --debug=v # verbose: variable values make --debug=i # verbose: implicit rule search make --warn-undefined-variables # warn whenever $(UNDEFINED) expands to empty ``` ### Print database ```bash make -p # dump all rules, variables, and implicit rules make -p -f /dev/null # dump builtins without running any Makefile ``` ### In-Makefile diagnostics ```makefile # $(info ...) prints at parse time, no error $(info SRCS = $(SRCS)) $(info CC = $(CC)) # $(warning ...) prints with filename:line prefix $(warning Building with flags: $(CFLAGS)) # $(error ...) prints and immediately aborts ifeq ($(CC),) $(error CC is not set) endif # origin tells you where a variable was defined $(info origin of CC: $(origin CC)) # default, environment, file, command line, override, automatic ``` ### Inspecting a specific target ```bash make --trace target make -d target 2>&1 | less # very verbose, full decision log ``` --- ## 14. Compiler Flags Reference ### Debug build ```makefile CFLAGS_DEBUG := \ -O0 \ # no optimization; fast compilation -g3 \ # max debug info including macros -fno-omit-frame-pointer \ -fsanitize=address,undefined \ -fstack-protector-strong \ -DDEBUG ``` ### Release build ```makefile CFLAGS_RELEASE := \ -O2 \ # safe optimizations -DNDEBUG \ -ffunction-sections \ # enables --gc-sections to strip dead code -fdata-sections \ -flto \ # link-time optimization -march=native # tune for the build machine (not for distributed binaries) LDFLAGS_RELEASE := \ -Wl,--gc-sections \ -Wl,-O1 \ -flto ``` ### Profiling build ```makefile CFLAGS_PROF := \ -O2 \ -g \ # keep symbols for perf/gprof -pg \ # gprof instrumentation -fno-omit-frame-pointer \ -DNDEBUG ``` For `perf` you do not need `-pg`, just `-g -fno-omit-frame-pointer`. ### Full warnings ```makefile WARN_FLAGS := \ -Wall \ # common warnings -Wextra \ # more warnings -Wpedantic \ # strict ISO C -Wshadow \ -Wconversion \ -Wsign-conversion \ -Wdouble-promotion \ -Wformat=2 \ -Wundef \ -fno-common \ -Wunused \ -Wmissing-prototypes \ # C only -Wstrict-prototypes # C only ``` Promote warnings to errors in CI: ```makefile CFLAGS += -Werror ``` ### Hardening flags (production binaries) ```makefile CFLAGS_HARDEN := \ -D_FORTIFY_SOURCE=2 \ -fstack-protector-strong \ -fPIE LDFLAGS_HARDEN := \ -pie \ -Wl,-z,relro \ -Wl,-z,now ``` --- ## 15. mk (Plan 9 Make) `mk` is the Plan 9 equivalent of `make`, designed to be simpler and cleaner. ### Key differences from GNU make | Feature | GNU make | mk | |---------|----------|----| | Shell | Uses `/bin/sh` per recipe line | Uses `rc` shell (Plan 9) or shell of choice | | Parallel flag | `-j N` | `-n N` | | Automatic variables | `$@`, `$<`, `$^` | `$target`, `$prereq`, `$newprereq` | | Pattern rules | `%.o: %.c` | `%.$O: %.c` | | All prerequisites | `$^` | `$prereq` | | Variable export | Manual | Automatic via `MKFLAGS` | | Dependency files | Manual `-include` | Native `<| mkdepend` integration | ### mk syntax ```mk CC = gcc CFLAGS = -O2 -Wall %.o: %.c $CC $CFLAGS -c $prereq -o $target prog: main.o util.o $CC $prereq -o $target ``` ### Install on Linux ```bash # 9base (Plan 9 userland port) sudo pacman -S 9base # Arch AUR # or build from suckless.org/tools/9base ``` ### Advantages - Simpler rule syntax - No recursive variable expansion surprises - `mkfile` can be used with the 9front toolchain directly - Recipe is a single string passed to a shell, so `.ONESHELL` is the default behavior --- ## 16. Ninja Build System Ninja is a low-level build system focused purely on speed. It does no rule inference; higher-level tools (CMake, Meson, GN) generate `.ninja` files. ### Install ```bash sudo pacman -S ninja # Arch sudo apt install ninja-build # Debian ``` ### Key speed advantages over make - Parses its input files in parallel - Uses a dependency log (`.ninja_deps`, `.ninja_log`) instead of re-stat-ing every file - Avoids shell overhead: single process, no fork per recipe line - Implicit dep scanning built in (no need for `-MMD` tricks) ### Basic ninja file syntax ```ninja cc = gcc cflags = -O2 -Wall rule cc command = $cc $cflags -MMD -MF $out.d -c $in -o $out depfile = $out.d deps = gcc rule link command = $cc $in -o $out build build/main.o: cc src/main.c build build/util.o: cc src/util.c build prog: link build/main.o build/util.o default prog ``` ### Run ```bash ninja # build default target ninja -j$(nproc) # explicit parallelism (default is all CPUs) ninja -v # verbose (show full commands) ninja -n # dry run ninja -d explain # why is each target rebuilt ``` ### Generating ninja files ```bash # CMake cmake -GNinja -DCMAKE_BUILD_TYPE=Release .. ninja # Meson meson setup build cd build && ninja # GN (Chromium ecosystem) gn gen out/Release ninja -C out/Release ``` --- ## 17. Complete Production Makefile Template ```makefile # ------------------------------------------------------------ # Project config # ------------------------------------------------------------ TARGET := prog SRCDIR := src BLDDIR := build DEPDIR := $(BLDDIR)/.deps CC := ccache gcc CXX := ccache g++ AR := ar LD := $(CC) # ------------------------------------------------------------ # Sources and objects # ------------------------------------------------------------ SRCS := $(shell find $(SRCDIR) -name '*.c') OBJS := $(patsubst $(SRCDIR)/%.c, $(BLDDIR)/%.o, $(SRCS)) # ------------------------------------------------------------ # Flags # ------------------------------------------------------------ CFLAGS := -std=c11 -Wall -Wextra -Wpedantic -Wshadow \ -ffunction-sections -fdata-sections LDFLAGS := -fuse-ld=mold -Wl,--gc-sections -Wl,--as-needed LIBS := DEPFLAGS = -MMD -MP -MF $(DEPDIR)/$*.d ifeq ($(BUILD),debug) CFLAGS += -O0 -g3 -fsanitize=address,undefined -fno-omit-frame-pointer -DDEBUG LDFLAGS += -fsanitize=address,undefined else ifeq ($(BUILD),release) CFLAGS += -O2 -DNDEBUG -flto -march=native LDFLAGS += -flto else CFLAGS += -Og -g -fno-omit-frame-pointer endif # ------------------------------------------------------------ # Special targets # ------------------------------------------------------------ .DELETE_ON_ERROR: .PHONY: all clean distclean test compile_commands # ------------------------------------------------------------ # Default target # ------------------------------------------------------------ all: $(BLDDIR)/$(TARGET) # ------------------------------------------------------------ # Link # ------------------------------------------------------------ $(BLDDIR)/$(TARGET): $(OBJS) $(LD) $(LDFLAGS) $^ $(LIBS) -o $@ # ------------------------------------------------------------ # Compile # ------------------------------------------------------------ $(BLDDIR)/%.o: $(SRCDIR)/%.c | $(DEPDIR) @mkdir -p $(@D) $(dir $(DEPDIR)/$*.d) $(CC) $(CFLAGS) $(DEPFLAGS) -c $< -o $@ # ------------------------------------------------------------ # Directories # ------------------------------------------------------------ $(BLDDIR) $(DEPDIR): mkdir -p $@ # ------------------------------------------------------------ # Dependencies # ------------------------------------------------------------ -include $(wildcard $(DEPDIR)/**/*.d $(DEPDIR)/*.d) # ------------------------------------------------------------ # compile_commands.json # ------------------------------------------------------------ compile_commands: bear -- $(MAKE) clean all # ------------------------------------------------------------ # Clean # ------------------------------------------------------------ clean: rm -rf $(BLDDIR) distclean: clean rm -f compile_commands.json # ------------------------------------------------------------ # Test # ------------------------------------------------------------ test: $(BLDDIR)/$(TARGET) ./tests/run.sh $< # ------------------------------------------------------------ # Debug helpers # ------------------------------------------------------------ print-%: @echo '$* = $($*)' # Usage: make print-CFLAGS ``` ### Usage ```bash make # debug-ish build (Og) make BUILD=release -j$(nproc) make BUILD=debug -j$(nproc) make compile_commands make print-CFLAGS ``` --- ## 18. Incremental Linking with -r and module.ro Partial linking (`-r`) combines multiple object files into a single relocatable object without resolving symbols. The result can be linked again later. Useful for large projects with stable subsystems. ### Create a partial object ```makefile build/net.ro: $(NET_OBJS) $(LD) -r $^ -o $@ build/io.ro: $(IO_OBJS) $(LD) -r $^ -o $@ build/crypto.ro: $(CRYPTO_OBJS) $(LD) -r $^ -o $@ ``` ### Final link using partial objects ```makefile $(TARGET): build/net.ro build/io.ro build/crypto.ro build/main.o $(LD) $(LDFLAGS) $^ $(LIBS) -o $@ ``` ### Advantages - Subsystem objects can be compiled independently and cached - Only the changed subsystem's `.ro` is rebuilt; others are reused - The final link step only sees a handful of `.ro` files instead of hundreds of `.o` files, making it faster - Can hide internal symbols with `--localize-hidden` on partial objects ### Strip internal symbols after partial link ```makefile build/net.ro: $(NET_OBJS) $(LD) -r $^ -o $@.tmp objcopy --localize-hidden $@.tmp $@ rm $@.tmp ``` This enforces subsystem boundaries: symbols marked `__attribute__((visibility("hidden")))` inside `net/` become local after the partial link and cannot be referenced by other modules. --- ## Highest Impact Optimizations Ranked by Time Saved Ranked by observed wall-clock build time reduction on a typical C project (1-500k SLOC). Your numbers will vary based on hardware and project structure. | Rank | Optimization | Typical Speedup | Notes | |------|-------------|----------------|-------| | 1 | `-j$(nproc)` parallel make | 4-16x | Free; limited by the critical path and I/O | | 2 | ccache (warm cache) | 5-100x | Near-instant for unchanged files across branches | | 3 | mold linker | 10-20x (link step) | Link is often the serializing bottleneck | | 4 | Precompiled headers | 2-5x | High value when headers are large and stable | | 5 | Unity builds | 2-10x | Eliminates repeated header parsing; best for CI | | 6 | distcc (cluster) | Linear with machines | Needs fast network and homogeneous compilers | | 7 | Non-recursive make | 1.2-2x | Eliminates repeated `make` invocations and shell forks | | 8 | Auto-dependency generation | Correct > fast | Avoids unnecessary full rebuilds on header changes | | 9 | `-O0` in debug builds | 1.5-3x compile | Slower binary, but much faster compilation | | 10 | Ninja over make | 1.1-1.5x | Parse overhead matters most on projects with thousands of targets | | 11 | Incremental linking (`-r`) | 1.2-2x (link) | Effective when only one subsystem changes at a time | | 12 | `-flto=thin` | Smaller binary, slower build | Worth it for release; use `lld` or `mold` to parallelize | ### Combining for maximum effect ```bash # Development inner loop (full acceleration) CC="ccache gcc" make -j$(nproc) -Otarget BUILD=debug # CI clean build (unity + parallel + mold) make unity -j$(nproc) LDFLAGS="-fuse-ld=mold" # Release (LTO + mold + ccache) CC="ccache gcc" make -j$(nproc) BUILD=release LDFLAGS="-fuse-ld=mold -flto" ``` The single highest-ROI change for most projects starting from a naive serial `make`: add `-j$(nproc)` and set `CC := ccache gcc`. Takes 30 seconds and cuts iteration time dramatically.