A Look at Build Systems (2): Ninja

This is not a tutorial! It is a collection of Ninja’s strengths and weaknesses. Part 1 was about Make.

Overview

Evan Martin was on Google’s Chrome team during the initial fork from Chromium. He noticed that even very minor code changes took several minutes to compile. He set out to develop an improved build system – with its syntax and usage roughly like Make’s, but faster.

He soon realized that this would be achieved by trading Make’s complicated macros for a very simple syntax with more repeating statements. Since humans are bad at writing repeating statements, high-level tasks – e.g. listing all files for compilation – were moved to a generator, like CMake or Meson. Its excellent manual provides an overview of design goals and a comparison to Make.

Ninja has since built Chrome (on all platforms) and reached version 1.0.0 in 2012. It is now CMake’s default backend and has found wide adoption.

A simple .ninja file to compile a C program could look like this:

# build.ninja

rule c
  command = gcc -c $in -o $out

rule link
  command = gcc -o $out $in

build main.o:  c main.c
build foo.o:   c foo.c
build program: link main.o foo.o

Calling ninja program will use GCC to compile the C source files to object files, eventually linking them to program. You can do the same with a simple shell script, but the points of Ninja are:

  1. It does only the least work necessary. For example, it doesn’t re-compile foo.o if foo.c hasn’t changed since the last build.

  2. Due to its dependency analysis, Make can parallelize the build process by compiling foo.c and main.c concurrently.

Ninja is a general-purpose build system: It can be used to build any language and any kind of target. You can even use the result of one build step as a compiler for a later build step (if you build your own tools).

Getting Ninja

Pre-built binaries are available for Linux, Windows, and Mac. Porting to other platforms is easy because Ninja is open source and written in portable C.

If you use Windows with Visual Studio, Ninja is pre-installed and can be used from the Visual Studio Command Prompt.

In contrast to Make, there are no different flavors or competing forks.

The Windows executable being 500 KiB small, Ninja is even smaller than Make.

Improvements over Make

Performance

Ninja is probably the fastest build system. Chrome is built from several tens of thousands of source files, and one-file-changes were famously clocked at 10–20 seconds with Make, and at less than a second with Ninja.

Target Directories

Make is famously hard with out-of-source builds because it does not automatically create directories for its targets. You can create the directories manually via complicated rules, but this leads to other problems with parallel builds (the presence of a directory cannot easily be used as a dependency).

Ninja just creates all directories automatically for you before building the files inside.

C/C++ Header Dependencies

Ninja departs from the simplicity-first philosophy in one (fortunate) point.

Did you know that handling C/C++ header dependencies with Make requires writing a complicated feedback loop? Ninja wouldn’t have any of that. It can parse compiler output on header dependencies directly via the deps directive. Header information is not written to files the classic way, but instead stored in a database for rapidly improved performance.

Bootstrapping

Ninja supports order-only dependencies as easy means to bootstrap an environment.

Problems with Oversimplification

These problems can mostly be solved by using a generator instead of manually poking around in Ninja files.

Ninja’s simplicity can sometimes be undesirable. It does not even provide simple if/else statements, thus making a few tasks overly complicated. In particular:

Configurations

You often want to build your projects in two configurations – one for debugging (maximal logging, quick rebuild cycle), and one for deployment (maximal performance). In this case, Ninja explicitly states that your generator should generate two build scripts:

[…] even build-time decisions like which compiler flags should I use? or should I build a debug or release-mode binary? belong in the .ninja file generator.

However, I’ve found a way around this. You should decide for yourself whether it’s worth the additional complexity.

Shell Dependency

Command lines in rules are sent directly to the shell on Linux, but not on Windows (instead, CreateProcess() is invoked directly). This obviously leads to the situation that simple shell commands like cd or setting an environment variable are the easiest things on Linux, but tremendously hard on Windows.

The only solution I’ve found so far: Stuff those things into scripts and select the proper script for your current platform via the configuration trick.

This makes simple problems like copy a file require 50 lines of code or even more, if you need to solve them in a platform-independent way.

Conclusion

Ninja takes the best parts of Make, throws away the prehistoric macro syntax, and adds a few comfort features. Its syntax is well-structured and just complex enough to perform fundamental build tasks. It is extremely fast.

With a generator like CMake or Meson, Ninja will just work. If you write your Ninja files manually, however, expect lots of repetititive statements and some frustration working around its limitations.