RISC-V privilege and CSRs · xv6, line by line

1warm-upChoose one

start runs in machine mode and wants main to run in supervisor mode. Why does it write mstatus.MPP and mepc and then execute mret, instead of switching modes directly?

kernel/start.c

14void
15start()
16{
17  // set M Previous Privilege mode to Supervisor, for mret.
18  unsigned long x = r_mstatus();
19  x &= ~MSTATUS_MPP_MASK;
20  x |= MSTATUS_MPP_S;
21  w_mstatus(x);
22
23  // set M Exception Program Counter to main, for mret.
24  // requires gcc -mcmodel=medany
25  w_mepc((uint64)main);

RISC-V has no instruction that simply lowers the privilege mode; the only way down is to return from a trap, and mret takes the new mode from MPP and the new pc from mepc.mepc holds the address of the machine-mode trap handler, which mret installs.mret is needed to switch to the supervisor’s own stack pointer.There is such an instruction, but mret is used because it also jumps, saving one instruction.

2warm-upClick the line

Click the line in start where the hart actually stops running in machine mode.

kernel/start.c

14void
15start()
16{
17  // set M Previous Privilege mode to Supervisor, for mret.
18  unsigned long x = r_mstatus();
19  x &= ~MSTATUS_MPP_MASK;
20  x |= MSTATUS_MPP_S;
21  w_mstatus(x);
22
23  // set M Exception Program Counter to main, for mret.
24  // requires gcc -mcmodel=medany
25  w_mepc((uint64)main);
26
27  // disable paging for now.
28  w_satp(0);
29
30  // delegate all interrupts and exceptions to supervisor mode.
31  w_medeleg(0xffff);
32  w_mideleg(0xffff);
33  w_sie(r_sie() | SIE_SEIE | SIE_STIE);
34
35  // configure Physical Memory Protection to give supervisor mode
36  // access to all of physical memory.
37  w_pmpaddr0(0x3fffffffffffffull);
38  w_pmpcfg0(0xf);
39
40  // enable hardware updates of page table A and D bits
41  w_menvcfg(r_menvcfg() | MENVCFG_ADUE);
42
43  // ask for clock interrupts.
44  timerinit();
45
46  // keep each CPU's hartid in its tp register, for cpuid().
47  int id = r_mhartid();
48  w_tp(id);
49
50  // switch to supervisor mode and jump to main().
51  asm volatile("mret");
52}

Your pick: none yet (click a line in the code)

3warm-upTrue or false, and why

True or false: when a user program executes ecall, the hardware switches to the process’s kernel stack and the kernel page table as part of the trap.

kernel/trampoline.S

22uservec:    
#
      # trap.c sets stvec to point here, so
      # traps from user space start here,
      # in supervisor mode, but with a
      # user page table.
      #
29
      # save user a0 in sscratch so
      # a0 can be used to get at TRAPFRAME.
      csrw sscratch, a0
33
      # each process has a separate p->trapframe memory area,
      # but it's mapped to the same virtual address
      # (TRAPFRAME) in every process's user page table.
      li a0, TRAPFRAME

TrueFalse

Why?

The hardware switches the stack but not the page table, using the kernel_sp field of the trapframe.The hardware switches the page table but not the stack; uservec only needs to set sp.The trap leaves sp and satp untouched; uservec switches them itself, with ld sp, 8(a0) (line 76) and csrw satp, t1 (line 92).

4warm-upChoose one

Just before mret, start copies the hart ID into the ordinary register tp. From then on cpuid returns tp. Why not just read mhartid whenever the ID is needed?

kernel/start.c

// keep each CPU's hartid in its tp register, for cpuid().
int id = r_mhartid();
w_tp(id);
49
// switch to supervisor mode and jump to main().
asm volatile("mret");

mhartid changes when a process moves to another hart, so it cannot be trusted.mret clears mhartid, so it must be saved first.mhartid is a machine-mode CSR; once the hart runs in supervisor mode, reading it raises an illegal-instruction exception.Reading a CSR is too slow to do on every mycpu() call.

5warm-upMatch the pairs

Match each supervisor CSR with what it holds in xv6.

stvecsepcscausesatpsscratchstimecmp

6warm-upChoose all that apply

A user program executes ecall. Which of these does the hardware change as part of taking the trap? Choose all that apply.

7warm-upChoose one

Lines 31–32 write 0xffff to medeleg and mideleg. What would go wrong without them?

kernel/start.c

// delegate all interrupts and exceptions to supervisor mode.
w_medeleg(0xffff);
w_mideleg(0xffff);
w_sie(r_sie() | SIE_SEIE | SIE_STIE);

Interrupts would stay disabled, because delegation is what enables them.User programs would trap straight into user-mode handlers instead of the kernel.Traps would go to machine mode, where xv6 has no handler at all; system calls, page faults and timer interrupts would never reach the kernel’s supervisor-mode code.Supervisor mode could not read the time CSR.

8warm-upType a number

usertrap recognizes a system call by the value of scause. What value is it?

kernel/trap.c

p->trapframe->epc = r_sepc();
53
if (r_scause() == 8) {
  // system call
56
  if (killed(p))
    kexit(-1);

decimal, 0x hex or 0b binary

9solidDecode the bits

After line 33 of start, gdb reads sie = 0x220 on every hart. Decode it.

kernel/start.c

// delegate all interrupts and exceptions to supervisor mode.
w_medeleg(0xffff);
w_mideleg(0xffff);
w_sie(r_sie() | SIE_SEIE | SIE_STIE);

Value: 0x220

SEIE (bit 9): supervisor external interruptsSTIE (bit 5): supervisor timer interruptsSSIE (bit 1): supervisor software interruptsCan a timer interrupt be taken in main right after mret?

10solidDecode the bits

After kvminithart, gdb reads satp = 0x8000000000087fff on every hart. Decode it as an Sv39 satp (MODE in bits 63–60, ASID in bits 59–44, PPN in bits 43–0).

kernel/riscv.h

245// use riscv's sv39 page table scheme.
246#define SATP_SV39 (8L << 60)
247
248#define MAKE_SATP(pagetable) (SATP_SV39 | (((uint64)pagetable) >> 12))

Value: 0x8000000000087fff

MODE (bits 63-60)ASID (bits 59-44)Physical address of the root page table

11solidFill in the machine state

A user program has just executed ecall. The hart is about to run the first instruction of uservec, line 32. Fill in its state. (For the stack, count it only if the code running here may push onto it; see The stacks of xv6.)

kernel/trampoline.S

22uservec:    
#
      # trap.c sets stvec to point here, so
      # traps from user space start here,
      # in supervisor mode, but with a
      # user page table.
      #
29
      # save user a0 in sscratch so
      # a0 can be used to get at TRAPFRAME.
      csrw sscratch, a0
33
      # each process has a separate p->trapframe memory area,
      # but it's mapped to the same virtual address
      # (TRAPFRAME) in every process's user page table.
      li a0, TRAPFRAME

Privilege modeActive stackPage table (satp)Interrupts (sstatus.SIE)

12solidFill in the machine state

Hart 2 has just executed the mret at the end of start and is at the first instruction of main. Fill in its state.

kernel/main.c

9// start() jumps here in supervisor mode on all CPUs.
10void
11main()
12{
13  if (cpuid() == 0) {

Privilege modeActive stackPage table (satp)Interrupts (sstatus.SIE)noff

13solidType a number

On entry to start, gdb reads mstatus = 0xa00000000. What value is written to mstatus on line 21? Answer in hex.

kernel/riscv.h

14#define MSTATUS_MPP_MASK (3L << 11) // previous mode.
15#define MSTATUS_MPP_M    (3L << 11)
16#define MSTATUS_MPP_S    (1L << 11)
17#define MSTATUS_MPP_U    (0L << 11)

decimal, 0x hex or 0b binary

14solidType a number

How many places in the kernel’s source write satp (count each w_satp(...) call and each csrw satp instruction, not the helper’s definition in riscv.h)?

decimal, 0x hex or 0b binary

15solidChoose all that apply

Hart 0 executes each of these. Which ones can affect what hart 1 sees or does? Choose all that apply.

w_stimecmp in clockintrkfree pushing a page onto kmem.freelistsfence.vma in kvminithartcsrw satp in kvminithartw_stvec in trapinithartplicinit writing the UART’s priority register in the PLIC

16solidChoose one

usertrap panics on line 43 if the trap did not come from user mode. How does it know where the trap came from?

kernel/trap.c

37uint64
38usertrap(void)
39{
40  int which_dev = 0;
41
42  if ((r_sstatus() & SSTATUS_SPP) != 0)
43    panic("usertrap: not from user mode");
44
45  // send interrupts and exceptions to kerneltrap(),
46  // since we're now in the kernel.
47  w_stvec((uint64)kernelvec); //DOC: kernelvec

It reads a CSR that holds the current privilege mode.It checks whether sepc is a user address (below MAXVA).It reads scause, whose value says which mode the trap came from.It reads sstatus.SPP, the previous mode that the hardware recorded when it took the trap: 0 means user, 1 means supervisor.

17solidPut in order

Put the steps of a return to user mode in order, from prepare_return to the sret.

stvec = uservec in the trampoline
intr_off(): clear sstatus.SIE
fill the trapframe’s kernel_satp, kernel_sp, kernel_trap, kernel_hartid
write sstatus with SPP = 0 and SPIE = 1
sepc = p->trapframe->epc
ld sp, 48(a0): the user’s sp
sret
csrw satp, a0: user page table

18solidChoose one

timerinit asks for the first timer interrupt 0.1 s after boot. Hart 0 spends about 1.5 s in main building the kernel. When is hart 0’s first timer interrupt actually taken?

kernel/start.c

54// ask each hart to generate timer interrupts.
55void
56timerinit()
57{
58  // enable the sstc extension (i.e. stimecmp).
59  w_menvcfg(r_menvcfg() | MENVCFG_STCE);
60
61  // allow supervisor to use stimecmp and time.
62  w_mcounteren(r_mcounteren() | 2);
63
64  // ask for the very first timer interrupt.
65  w_stimecmp(r_time() + 1000000);
66}

In the middle of main, 0.1 s after timerinit, because line 33 enabled timer interrupts in sie.At the scheduler’s first intr_on() (proc.c line 441); until then it stays pending.Never; a timer interrupt that arrives while interrupts are off is lost, and clockintr must re-arm it.Immediately, while still in start, because machine mode always takes interrupts.

19solidChoose one

For a system call, usertrap adds 4 to the saved epc (line 62). For a page fault that vmfault fixes, it does not. Why the difference?

kernel/trap.c

// save user program counter.
p->trapframe->epc = r_sepc();
53
if (r_scause() == 8) {
  // system call
56
  if (killed(p))
    kexit(-1);
59
  // sepc points to the ecall instruction,
  // but we want to return to the next instruction.
  p->trapframe->epc += 4;
63
  // an interrupt will change sepc, scause, and sstatus,
  // so enable only now that we're done with those registers.
  intr_on();
67
  syscall();
} else if ((which_dev = devintr()) != 0) {
  // ok
} else if ((r_scause() == 15 || r_scause() == 13) &&
           vmfault(p->pagetable, p->sz, r_stval(),
                   (r_scause() == 13) ? 1 : 0) != 0) {
  // page fault on lazily-allocated page

Page faults are interrupts, and interrupts never need epc adjusted.For a page fault the hardware has already advanced sepc, but for ecall it has not.The 4 skips the ret in the system-call stub, which the kernel performs on the program’s behalf.sepc holds the address of the instruction that trapped. After a page fault the program must retry that same load or store; after an ecall it must continue with the next instruction, 4 bytes later.

20deepChoose one

xv6 has no machine-mode trap handler. timerinit therefore enables the Sstc extension (menvcfg.STCE) and uses stimecmp. Why couldn’t xv6 just use the classic machine timer (mtimecmp) and rely on mideleg = 0xffff?

kernel/start.c

54// ask each hart to generate timer interrupts.
55void
56timerinit()
57{
58  // enable the sstc extension (i.e. stimecmp).
59  w_menvcfg(r_menvcfg() | MENVCFG_STCE);
60
61  // allow supervisor to use stimecmp and time.
62  w_mcounteren(r_mcounteren() | 2);
63
64  // ask for the very first timer interrupt.
65  w_stimecmp(r_time() + 1000000);
66}

mtimecmp does not exist on QEMU’s virt machine.It could; mideleg = 0xffff delegates the machine timer too, and Sstc only saves a few instructions.Without Sstc, supervisor mode cannot read the time CSR.The machine timer raises a machine-level interrupt (MTIP), which is taken in machine mode; its mideleg bit does not stick on this machine (gdb reads mideleg = 0x3666). Sstc’s stimecmp raises a supervisor-level interrupt (STIP), which is delegated.

21deepChoose one

In a scratch copy of xv6, line 62 (w_mcounteren(r_mcounteren() | 2)) is deleted. The kernel finishes main on hart 0 and enters the scheduler. What happens next?

kernel/start.c

54// ask each hart to generate timer interrupts.
55void
56timerinit()
57{
58  // enable the sstc extension (i.e. stimecmp).
59  w_menvcfg(r_menvcfg() | MENVCFG_STCE);
60
61  // allow supervisor to use stimecmp and time.
62  w_mcounteren(r_mcounteren() | 2);
63
64  // ask for the very first timer interrupt.
65  w_stimecmp(r_time() + 1000000);
66}

timerinit itself traps on r_time() and the boot hangs in machine mode.No timer interrupt is ever raised, so the kernel runs without preemption.At the first intr_on(), the pending timer interrupt reaches clockintr, whose read of time is an illegal instruction in supervisor mode: kerneltrap prints scause=0x2 and panics.Nothing changes; time is a user-readable counter that any mode can read.

22deepChoose one

xv6’s PTEs never set the A (accessed) and D (dirty) bits. Line 41 sets menvcfg.ADUE (QEMU happens to set it already at reset). On a machine where it starts at 0, what would happen if line 41 were missing?

kernel/start.c

40 // enable hardware updates of page table A and D bits

41 w_menvcfg(r_menvcfg() | MENVCFG_ADUE);

Only user programs would fault, on their first access to each page; the kernel would boot normally.Right after kvminithart turns on paging, the next instruction fetch raises a page fault (A = 0), and since stvec is not set yet the hart traps to address 0 and keeps faulting, with no message.Nothing; A and D are only hints that xv6 ignores.The kernel would panic with kerneltrap and print scause.

23deepTrue or false, and why

True or false: if prepare_return left sstatus.SPIE at 0 (so that sret sets SIE to 0), a user program spinning in an infinite loop could never be preempted by the timer.

kernel/trap.c

// set S Previous Privilege mode to User.
unsigned long x = r_sstatus();
x &= ~SSTATUS_SPP; // clear SPP to 0 for user mode
x |= SSTATUS_SPIE; // enable interrupts in user mode
w_sstatus(x);

TrueFalse

Why?

usertrap calls intr_on() on every trap, so SIE comes back anyway.While the hart runs in user mode, supervisor-level interrupts that are enabled in sie are taken whatever sstatus.SIE says; SIE only gates interrupts while the hart is in supervisor mode.The timer interrupt goes to machine mode, which re-enables SIE before passing it on.

24deepChoose one

A trap from user mode always sets sstatus.SPP to 0. Yet prepare_return clears SPP explicitly on line 126 before every return to user mode. Why is that necessary?

kernel/trap.c

// set up the registers that trampoline.S's sret will use
// to get to user space.
123
// set S Previous Privilege mode to User.
unsigned long x = r_sstatus();
x &= ~SSTATUS_SPP; // clear SPP to 0 for user mode
x |= SSTATUS_SPIE; // enable interrupts in user mode
w_sstatus(x);
129
// set S Exception Program Counter to the saved user pc.
w_sepc(p->trapframe->epc);

Because sret ignores SPP unless it was written by software since the last trap.Because intr_on() in usertrap sets SPP back to 1.It is not necessary; the line is only documentation of the intended value.sstatus belongs to the hart, not the process. Since this process’s trap, the hart may have run other threads, one of which yielded from inside a kernel trap (leaving SPP = 1); and a new process in forkret never trapped from user mode at all.

25deepChoose all that apply

A kernel thread is preempted by a timer interrupt (it calls yield from kerneltrap) and is later resumed by a scheduler, possibly on another hart. When it runs again, which of these may hold a different value from the moment before it yielded? Choose all that apply.