Makefile

Short names for the two main directories

K and U are variables holding the directory names. Because they are one letter long they can be used as $K and $U without parentheses, which keeps the long file lists below readable: $K/vm.o means kernel/vm.o.

The kernel's object files

OBJS lists every object file (.o) the kernel is linked from, one per source file in kernel/. Each \ at the end of a line continues the list onto the next line.

You will not find rules here for building each .o. For a .S file, the pattern rule on line 98 does it. For a .c file there is no rule in this Makefile at all: make uses its built-in rule for C, which runs $(CC) $(CFLAGS) -c, so the CFLAGS set below apply. Click any entry to open the source file it is built from.

$K/entry.o comes first. Order matters to the linker: object files are laid out in this order, and kernel/entry.S must be at the very start of the kernel. (The linker script also makes sure of that; see kernel/kernel.ld:13.)

Find the RISC-V toolchain

The compiler, assembler and linker for RISC-V are installed under different names on different systems: riscv64-unknown-elf-gcc, riscv64-elf-gcc (Homebrew), riscv64-linux-gnu-gcc (Debian/Ubuntu), and so on. TOOLPREFIX holds the part before gcc, and every tool name below is built from it.

If you did not set TOOLPREFIX yourself (ifndef), lines 39–52 run a shell command, once, while the Makefile is read: for each candidate prefix it runs that objdump -i, which lists the file formats the tool understands, and checks the list for elf64-big. A RISC-V objdump lists elf64-bigriscv. The first prefix that passes is used. If none does, the command prints the error on lines 49–51 to the terminal. Note that make ignores the failure of a $(shell ...) command: it leaves TOOLPREFIX empty and carries on with plain gcc, so the build then fails with confusing compiler errors. If you see them, look for the *** message above them.

Lines 33–35 are comments: a hint about where the tools might live, and a commented-out line you could edit to force a prefix.

The programs make will run

QEMU names the emulator and MIN_QEMU_VERSION the oldest version known to work (checked by the target on line 196). CC, LD, OBJCOPY and OBJDUMP are the C compiler, linker, object-file copier and disassembler, each with the prefix found above. For example, with Homebrew’s toolchain $(CC) becomes riscv64-elf-gcc. (OBJCOPY is defined but not used in this Makefile.)

Reproducible builds

DETFLAGS holds one compiler option, -ffile-prefix-map. CURDIR is the directory make is running in. The option replaces that directory’s full path with . in everything the compiler records (debug information), so two people building the same source in different directories get identical output.

How every C file is compiled

CFLAGS collects the options passed to the C compiler. It is built up over several lines with +=, one theme per line. Hover over any option in the code to see its glossary entry. In groups:

• Warnings and debugging (line 66): stop on any warning (-Wall -Werror), optimize lightly (-O), and include debug information and frame pointers so that gdb can show source lines and stack traces.
• Target (68–71): generate 64-bit RISC-V code (-march=rv64gc) that can be placed anywhere in memory (-mcmodel=medany, needed because the kernel lives at 0x80000000); write a .d dependency file next to each .o (-MD).
• No operating system underneath (72–79): the kernel and the user programs are freestanding (vs. hosted) C C. There is no C library, so the compiler must not assume one: -nostdlib, and -fno-builtin-X for each function name xv6 writes itself (memset, printf, malloc…), so the compiler never swaps in its own idea of what they do. (-ffreestanding already implies this for every function, and -nostdlib only matters when gcc does the linking, which it never does here; both lists are belt-and-braces safeguards.)
• Headers (80): -I. lets source files #include "kernel/types.h" from the top directory.
• Stack protector (81): disabled if the compiler supports disabling it.

The same CFLAGS are used for both the kernel and the user programs.

Turn off position-independent code where needed

Some compilers (the comment mentions Ubuntu 16.10’s) produce position-independent executables (PIE) by default. xv6 is linked to fixed addresses, so PIE must be off. Each test asks the compiler to print its built-in configuration (-dumpspecs) and looks for a mention of the option no-pie (or the older spelling nopie); if the compiler knows the option, the flags to turn PIE off are added. [^f] in the pattern skips the text fno-pie, so only a standalone no-pie counts.

Linker options

LDFLAGS holds -z max-page-size=4096: the linker aligns the program’s loadable segments to 4096-byte pages. That is already the default for the RISC-V ld (linking without it produces an identical kernel), so the option only guards against a toolchain with a larger default.

Link the kernel: kernel/kernel

The rule for the kernel itself. Because it is the first ordinary rule in the file, it is make’s default: plain make builds kernel/kernel and nothing else.

The prerequisites are every object file in OBJS plus the linker script, so the kernel is relinked whenever any of them changes. The recipe:

1. links the objects with ld, using kernel/kernel.ld for the memory layout;
2. writes kernel/kernel.asm, a disassembly of the whole kernel with the source lines mixed in, useful for seeing what any C line compiled into;
3. writes kernel/kernel.sym, a list of every symbol and its address.

Assemble the kernel's .S files

A pattern rule: kernel/X.o can be built from kernel/X.S (% matches the X). It runs gcc rather than the assembler directly, because files ending in capital .S must go through the C preprocessor first (for #include and #define), and gcc does that automatically. $@ is the target and $< the source (automatic variables).

Only a few options are needed: the instruction set, debug info (-g) and the reproducible-path option. The preprocessor finds #include "riscv.h" in kernel/trampoline.S because quoted includes are looked up in the including file’s own directory first.

An editor index (optional)

make tags builds a TAGS file with etags, which lets the Emacs editor jump to definitions. It is not needed to build or run xv6 (though, because its prerequisites are OBJS, it compiles the kernel’s object files first).

The user library

ULIB lists the object files linked into every user program: user/ulib.c (string functions, gets, stat…), user/usys.o (the system-call stubs from user/usys.S), user/printf.c and user/umalloc.c (malloc/free). This is all the “C library” xv6 programs get.

Link a user program: user/_X

A pattern rule for every user program. The target user/_cat is built from user/cat.o plus ULIB, laid out by user/user.ld, which starts each program at address 0.

When the pattern (_%) contains no /, make matches only the file-name part, so for user/_cat the stem $* is user/cat, and the prerequisite %.o becomes user/cat.o.

The program files are named with a leading _ so that, on your computer, user/_cat can never be mistaken for the real cat. mkfs/mkfs.c removes the _ when it copies the program into the disk image, so inside xv6 it is /cat.

Like the kernel rule, it also writes a disassembly (user/cat.asm) and a symbol list (user/cat.sym).

Generate and assemble the system-call stubs

User programs call fork(), write() and so on as if they were ordinary functions. Each is a three-instruction stub in assembly that puts the system-call number in a7 and executes ecall. Rather than writing them by hand, the Perl script user/usys.pl prints them all; line 112 saves its output as user/usys.S, and line 115 assembles that. The stubs need kernel/syscall.h, where the numbers are defined, which is found through -I. in CFLAGS.

forktest is linked differently

This explicit rule overrides the pattern rule above for user/_forktest alone. user/forktest.c creates processes until the process table is full, so the program is kept as small as possible: it is linked with only ulib.o and usys.o (no printf, no malloc). -N (sections not page-aligned), -e main (start at main) and -Ttext 0 (code at address 0) replace the user linker script.

Lines 118–119 start with a tab, so make treats them as part of the recipe and passes them to the shell, which ignores them as comments. As a side effect make prints them while building.

Build mkfs, a program for your computer

mkfs/mkfs.c creates the disk image. It runs on your computer during the build, not inside xv6, so it is compiled with your computer’s ordinary C compiler, plain gcc, not the RISC-V cross-compiler. It shares the on-disk format with the kernel by including kernel/fs.h and kernel/param.h, which is why those headers are prerequisites.

Keep the .o files

When make builds user/_cat from user/cat.c through the in-between file user/cat.o, it normally deletes cat.o afterwards, as an “intermediate” file. .PRECIOUS keeps every %.o. As the comment explains, that stops make from rebuilding user/_cat, and with it fs.img, the next time you run it, which would wipe any files you created inside xv6.

The list of user programs

UPROGS lists every program that goes into the disk image. To add your own program to xv6, you write user/myprog.c and add $U/_myprog\ to this list; the pattern rule on line 106 knows how to build it.

Build the disk image: fs.img

mkfs creates fs.img containing the README file and every program in UPROGS. The prerequisites guarantee that mkfs and every program are built (or rebuilt) first.

Rebuild when a header changes

Because of -MD, compiling kernel/vm.c also wrote kernel/vm.d, a one-rule makefile listing vm.c and every header it included as prerequisites of vm.o. This line reads all of those files (-include), so editing, say, kernel/riscv.h rebuilds exactly the object files that include it. The - makes make skip the line silently before the first build, when no .d files exist yet.

make clean

Deletes everything the build produced: object and dependency files, disassemblies, symbol lists, the kernel, the disk image, mkfs, the generated .gdbinit and usys.S, and the user programs. The first line also removes files from typesetting the xv6 book with TeX (*.tex, *.dvi…), probably left over from older versions that kept the book in the same repository.

Settings for debugging with gdb

make qemu-gdb (line 188) lets the gdb debugger control the emulated machine over a network port. On a shared machine, several students could do this at once, so GDBPORT derives a port number from your user ID: uid % 5000 + 25000, a number between 25000 and 29999.

QEMUGDB holds the QEMU option that opens the port. Very old QEMU versions (before 0.11) spelled it -s -p PORT instead of -gdb tcp::PORT, so the shell command checks which one this QEMU’s help text mentions.

How many CPUs

CPUS is the number of harts QEMU emulates, 3 unless you choose otherwise: make qemu CPUS=2 starts xv6 on two harts. A value given on the command line wins over the assignment here anyway; the ifndef also lets you set CPUS as an environment variable. The kernel supports up to NCPU (8).

The emulated machine

QEMUOPTS describes the computer QEMU pretends to be:

make qemu

The command you use most. It first checks the QEMU version, builds the kernel and the disk image if anything changed, then runs QEMU with the options above. xv6 boots in your terminal. To quit QEMU, press Ctrl-A and then X.

Generate .gdbinit

gdb reads commands from a file named .gdbinit when it starts. The template .gdbinit.tmpl-riscv contains the default port 1234; sed replaces it with GDBPORT. $^ is the template and $@ the output file.

make qemu-gdb

Like make qemu, but QEMU starts with every hart paused (-S) and waits for a debugger on GDBPORT. You then run gdb in a second terminal; it reads .gdbinit, connects, and you can set breakpoints anywhere in the kernel, even on _entry, before the kernel’s first instruction runs. (Recent gdb versions only read a project’s .gdbinit if you allow it, e.g. with add-auto-load-safe-path in ~/.gdbinit; gdb’s warning message shows the exact line to add.) The @ before echo stops make from printing the command itself.

make print-gdbport

Prints the port number, for scripts that need it.

Check the QEMU version

Line 195 runs qemu-system-riscv64 --version once and extracts the “major.minor” part from its first line (QEMU emulator version 11.0.0 becomes 11.0). The target check-qemu-version compares it with MIN_QEMU_VERSION using bc, a command-line calculator that prints 1 when 11.0 >= 7.2 is true and 0 when it is false. If it is false, the build stops with an error before QEMU is started.

make fmt

Reformats all C sources in place with clang-format, following the style in .clang-format. .PHONY marks fmt as an action, not a file name. wildcard expands to the list of existing .c and .h files.