Traps and system calls · xv6, line by line

1warm-upChoose one

echo calls write(1, "hi", 2). What is write in user space?

user/usys.S

global write
29write:
li a7, SYS_write
ecall
ret

A C function in user/ulib.c that copies the bytes into the kernel’s memory.A jump to a fixed kernel address that user programs are allowed to call.A three-instruction stub, generated by user/usys.pl, that puts the system-call number in a7 and executes ecall.The kernel’s sys_write, linked into every user program.

2warm-upChoose one

How does the kernel know which system call the program asked for, and where does syscall read that information?

kernel/syscall.c

136void
137syscall(void)
138{
139  int num;
140  struct proc *p = myproc();
141
142  num = p->trapframe->a7;
143  if (num > 0 && num < NELEM(syscalls) && syscalls[num]) {
144    // Use num to lookup the system call function for num, call it,
145    // and store its return value in p->trapframe->a0
146    p->trapframe->a0 = syscalls[num]();
147  } else {
148    printk("%d %s: unknown sys call %d\n", p->pid, p->name, num);
149    p->trapframe->a0 = -1;
150  }
151}

The stub put the number in a7; syscall() reads the saved copy, p->trapframe->a7.The stub pushes the number on the user stack, and the kernel pops it.ecall writes the number into scause.The number is in a0, the first argument register.

3warm-upType a number

In our gdb run, echo’s ecall for write sits at user address 0x354. When the system call finishes, at what address does echo continue? (Answer in hex.)

kernel/trap.c

49  struct proc *p = myproc();
50
51  // save user program counter.
52  p->trapframe->epc = r_sepc();
53
54  if (r_scause() == 8) {
55    // system call
56
57    if (killed(p))
58      kexit(-1);
59
60    // sepc points to the ecall instruction,
61    // but we want to return to the next instruction.
62    p->trapframe->epc += 4;
63

decimal, 0x hex or 0b binary

4warm-upTrue or false, and why

True or false: when a user program executes ecall, the hardware switches sp to the process’s kernel stack.

TrueFalse

Why?

ecall loads sp from the trapframe’s kernel_sp field.ecall changes only the mode, the pc and a few CSRs (sepc, scause, stval, sstatus); the stack switch is an ordinary load, ld sp, 8(a0) in uservec.ecall loads sp from sscratch.

5warm-upChoose one

Why does usertrap add 4 to p->trapframe->epc for a system call?

kernel/trap.c

struct proc *p = myproc();
50
// save user program counter.
p->trapframe->epc = r_sepc();
53
if (r_scause() == 8) {
  // system call
56
  if (killed(p))
    kexit(-1);
59
  // sepc points to the ecall instruction,
  // but we want to return to the next instruction.
  p->trapframe->epc += 4;
63
  // an interrupt will change sepc, scause, and sstatus,
  // so enable only now that we're done with those registers.
  intr_on();
67
  syscall();
} else if ((which_dev = devintr()) != 0) {

Because epc is counted in 4-byte words, not bytes.To skip the stub’s ret, which the kernel has already performed.sepc holds the address of the ecall itself; without the +4, the process would return to the ecall and make the same system call again, forever.To skip the li a7 instruction in front of the ecall.

6warm-upChoose one

The first thing usertrap does after its SPP check is point stvec at kernelvec (line 47). Why?

kernel/trap.c

37uint64
38usertrap(void)
39{
40  int which_dev = 0;
41
42  if ((r_sstatus() & SSTATUS_SPP) != 0)
43    panic("usertrap: not from user mode");
44
45  // send interrupts and exceptions to kerneltrap(),
46  // since we're now in the kernel.
47  w_stvec((uint64)kernelvec); //DOC: kernelvec
48
49  struct proc *p = myproc();
50

So that this process’s next system call reaches the kernel faster.From now on the hart runs kernel code; a trap here must go to the handler for traps from supervisor mode, because uservec assumes the user page table and saves through TRAPFRAME, which the kernel page table does not map.So that interrupts on the other harts go to kernelvec.Because ecall cleared stvec, and it has to be given a valid value again.

7solidMatch the pairs

Match each scause value with what it means. These are the values usertrap, kerneltrap and devintr compare against.

8131520x80000000000000050x8000000000000009

8solidChoose all that apply

Which of these does the hardware itself change when a user program executes ecall?

9solidChoose one

For a system call, usertrap turns interrupts on at line 66, but only after line 52 and the scause test. Why not earlier?

kernel/trap.c

struct proc *p = myproc();
50
// save user program counter.
p->trapframe->epc = r_sepc();
53
if (r_scause() == 8) {
  // system call
56
  if (killed(p))
    kexit(-1);
59
  // sepc points to the ecall instruction,
  // but we want to return to the next instruction.
  p->trapframe->epc += 4;
63
  // an interrupt will change sepc, scause, and sstatus,
  // so enable only now that we're done with those registers.
  intr_on();
67
  syscall();
} else if ((which_dev = devintr()) != 0) {

To keep other harts from modifying this process’s trapframe until it is complete.Any interrupt overwrites sepc, scause and sstatus on this hart; by line 66 sepc is safe in the trapframe and scause has been read.The kernel stack only becomes usable at line 66.killed at line 57 takes p->lock, and a spinlock must not be acquired with interrupts on.

10warm-upClick the line

Click the line that delivers sys_write’s result to the user program.

kernel/syscall.c

136void
137syscall(void)
138{
139  int num;
140  struct proc *p = myproc();
141
142  num = p->trapframe->a7;
143  if (num > 0 && num < NELEM(syscalls) && syscalls[num]) {
144    // Use num to lookup the system call function for num, call it,
145    // and store its return value in p->trapframe->a0
146    p->trapframe->a0 = syscalls[num]();
147  } else {
148    printk("%d %s: unknown sys call %d\n", p->pid, p->name, num);
149    p->trapframe->a0 = -1;
150  }
151}

Your pick: none yet (click a line in the code)

11solidChoose one

A buggy program puts 99 in a7 and executes ecall. What happens in this kernel?

kernel/syscall.c

136void
137syscall(void)
138{
139  int num;
140  struct proc *p = myproc();
141
142  num = p->trapframe->a7;
143  if (num > 0 && num < NELEM(syscalls) && syscalls[num]) {
144    // Use num to lookup the system call function for num, call it,
145    // and store its return value in p->trapframe->a0
146    p->trapframe->a0 = syscalls[num]();
147  } else {
148    printk("%d %s: unknown sys call %d\n", p->pid, p->name, num);
149    p->trapframe->a0 = -1;
150  }
151}

Whatever word lies 99 entries past the start of syscalls[] is called as a function.The kernel panics: index 99 is outside syscalls[].The kernel prints <pid> <name>: unknown sys call 99, the ecall returns -1 in a0, and the program keeps running.The process is killed, like one that executes an illegal instruction.

12solidType a number

What is NELEM(syscalls), the number of elements in the table?

kernel/syscall.c

107// An array mapping syscall numbers from syscall.h
108// to the function that handles the system call.
109static uint64 (*syscalls[])(void) = {
// clang-format off
[SYS_fork]    = sys_fork,
[SYS_exit]    = sys_exit,
[SYS_wait]    = sys_wait,
[SYS_pipe]    = sys_pipe,
[SYS_read]    = sys_read,
[SYS_kill]    = sys_kill,
[SYS_exec]    = sys_exec,
[SYS_fstat]   = sys_fstat,
[SYS_chdir]   = sys_chdir,
[SYS_dup]     = sys_dup,
[SYS_getpid]  = sys_getpid,
[SYS_sbrk]    = sys_sbrk,
[SYS_pause]   = sys_pause,
[SYS_uptime]  = sys_uptime,
[SYS_open]    = sys_open,
[SYS_write]   = sys_write,
[SYS_mknod]   = sys_mknod,
[SYS_unlink]  = sys_unlink,
[SYS_link]    = sys_link,
[SYS_mkdir]   = sys_mkdir,
[SYS_close]   = sys_close,
[SYS_sync]    = sys_sync,
// clang-format on
134};

decimal, 0x hex or 0b binary

13warm-upType a number

At most how many arguments can a system call handler fetch with argint, argaddr or argstr?

kernel/syscall.c

34static uint64
35argraw(int n)
36{
struct proc *p = myproc();
switch (n) {
case 0:
  return p->trapframe->a0;
case 1:
  return p->trapframe->a1;
case 2:
  return p->trapframe->a2;
case 3:
  return p->trapframe->a3;
case 4:
  return p->trapframe->a4;
case 5:
  return p->trapframe->a5;
}
panic("argraw");
return -1;
54}

decimal, 0x hex or 0b binary

14solidChoose all that apply

A timer interrupt arrives while cat runs in user mode (it has not been killed). Which of these does usertrap do for this trap?

kernel/trap.c

37uint64
38usertrap(void)
39{
40  int which_dev = 0;
41
42  if ((r_sstatus() & SSTATUS_SPP) != 0)
43    panic("usertrap: not from user mode");
44
45  // send interrupts and exceptions to kerneltrap(),
46  // since we're now in the kernel.
47  w_stvec((uint64)kernelvec); //DOC: kernelvec
48
49  struct proc *p = myproc();
50
51  // save user program counter.
52  p->trapframe->epc = r_sepc();
53
54  if (r_scause() == 8) {
55    // system call
56
57    if (killed(p))
58      kexit(-1);
59
60    // sepc points to the ecall instruction,
61    // but we want to return to the next instruction.
62    p->trapframe->epc += 4;
63
64    // an interrupt will change sepc, scause, and sstatus,
65    // so enable only now that we're done with those registers.
66    intr_on();
67
68    syscall();
69  } else if ((which_dev = devintr()) != 0) {
70    // ok
71  } else if ((r_scause() == 15 || r_scause() == 13) &&
72             vmfault(p->pagetable, p->sz, r_stval(),
73                     (r_scause() == 13) ? 1 : 0) != 0) {
74    // page fault on lazily-allocated page
75  } else {
76    printk("usertrap(): unexpected scause 0x%lx pid=%d\n", r_scause(), p->pid);
77    printk("            sepc=0x%lx stval=0x%lx\n", r_sepc(), r_stval());
78    setkilled(p);
79  }
80
81  if (killed(p))
82    kexit(-1);
83
84  // give up the CPU if this is a timer interrupt.
85  if (which_dev == 2)
86    yield();
87
88  prepare_return();
89
90  // the user page table to switch to, for trampoline.S
91  uint64 satp = MAKE_SATP(p->pagetable);
92
93  // return to trampoline.S; satp value in a0.
94  return satp;
95}

15deepFill in the machine state

A user program stores to a lazily allocated heap page and takes a store page fault (scause 15). usertrap calls vmfault. What is the state of the hart at line 463, before kalloc has been called?

kernel/vm.c

458uint64
459vmfault(pagetable_t pagetable, uint64 psz, uint64 va, int read)
460{
uint64 mem;
462
if (va >= psz)
  return 0;
va = PGROUNDDOWN(va);
if (ismapped(pagetable, va)) {
  return 0;
}
mem = (uint64)kalloc();
if (mem == 0)
  return 0;
memset((void *)mem, 0, PGSIZE);
if (mappages(pagetable, va, PGSIZE, mem, PTE_W | PTE_U | PTE_R) != 0) {
  kfree((void *)mem);
  return 0;
}
return mem;
478}

Privilege modeActive stackPage table (satp)Interrupts (sstatus.SIE)noffintena

16solidDecode the bits

Inside usertrap, scause reads 0x8000000000000009. Decode it.

Value: 0x8000000000000009

Bit 63Code (the low bits)MeaningWhat devintr returnsWill usertrap call yield() for this trap?

17solidTrue or false, and why

True or false: if a user program sets sp to 0 just before its ecall, it can crash the kernel.

TrueFalse

Why?

uservec only stores the user’s sp in the trapframe (line 41) and then loads the kernel stack from kernel_sp (line 76); the kernel never pushes through the user’s sp.ecall replaces a bad sp with the kernel stack.The kernel checks sp with copyin before it uses it.

18solidPut in order

Put these steps of one system call (one that does not sleep or yield) in the order they happen.

syscall runs the handler and stores its result in trapframe->a0
uservec saves the user registers into the trapframe
uservec loads the kernel stack and switches satp to the kernel page table
userret switches to the user page table, restores the registers and executes sret
usertrap turns interrupts on
ecall: mode U to S, sepc = the ecall’s address, pc = stvec
prepare_return turns interrupts off and points stvec at uservec
usertrap copies sepc into p->trapframe->epc

19deepChoose one

Why must kerneltrap write sepc and sstatus back (lines 162–163) before returning to kernelvec?

kernel/trap.c

134// interrupts and exceptions from kernel code go here via kernelvec,
135// on whatever the current kernel stack is.
136void
137kerneltrap()
138{
139  int which_dev = 0;
140  uint64 sepc = r_sepc();
141  uint64 sstatus = r_sstatus();
142  uint64 scause = r_scause();
143
144  if ((sstatus & SSTATUS_SPP) == 0)
145    panic("kerneltrap: not from supervisor mode");
146  if (intr_get() != 0)
147    panic("kerneltrap: interrupts enabled");
148
149  if ((which_dev = devintr()) == 0) {
150    // interrupt or trap from an unknown source
151    printk("scause=0x%lx sepc=0x%lx stval=0x%lx\n", scause, r_sepc(),
152           r_stval());
153    panic("kerneltrap");
154  }
155
156  // give up the CPU if this is a timer interrupt.
157  if (which_dev == 2 && myproc() != 0)
158    yield();
159
160  // the yield() may have caused some traps to occur,
161  // so restore trap registers for use by kernelvec.S's sepc instruction.
162  w_sepc(sepc);
163  w_sstatus(sstatus);
164}

kernelvec saved them on the stack, and these lines copy them back.To return to user mode: the saved sstatus has SPP = 0.yield() may have run other threads on this hart, or moved this thread to another hart; their traps and returns overwrote sepc and sstatus, and kernelvec’s sret needs this trap’s values.sret clears sepc, so it has to be rewritten before every sret.

20deepChoose one

kernelvec restores every register it saved except tp; it does not even save it. Why?

kernel/kernelvec.S

      # restore registers.
      ld ra, 0(sp)
      # ld sp, 8(sp)
      ld gp, 16(sp)
      # not tp (contains hartid), in case we moved CPUs
      ld t0, 32(sp)
      ld t1, 40(sp)
      ld t2, 48(sp)
      ld a0, 72(sp)
      ld a1, 80(sp)
      ld a2, 88(sp)
      ld a3, 96(sp)
      ld a4, 104(sp)
      ld a5, 112(sp)
      ld a6, 120(sp)
      ld a7, 128(sp)
      ld t3, 216(sp)
      ld t4, 224(sp)
      ld t5, 232(sp)
      ld t6, 240(sp)
60
      addi sp, sp, 256
62
      # return to whatever we were doing in the kernel.
      sret

tp holds the user’s thread pointer, which the kernel must not modify.The interrupted thread may have yielded and resumed on another hart; tp must hold the ID of the hart the thread is on now, which it already does.The 256-byte frame has no slot for tp.tp is a callee-saved register, so kerneltrap preserves it anyway.

21deepChoose one

Suppose usertrap called intr_on() right after line 47 (so before line 52), and a timer interrupt was already pending when echo executed its ecall for write. What would happen?

kernel/trap.c

struct proc *p = myproc();
50
// save user program counter.
p->trapframe->epc = r_sepc();
53
if (r_scause() == 8) {
  // system call
56
  if (killed(p))
    kexit(-1);
59
  // sepc points to the ecall instruction,
  // but we want to return to the next instruction.
  p->trapframe->epc += 4;
63
  // an interrupt will change sepc, scause, and sstatus,
  // so enable only now that we're done with those registers.
  intr_on();
67
  syscall();
} else if ((which_dev = devintr()) != 0) {

The kernel panics with kerneltrap.The timer is taken at once through kernelvec; afterwards sepc holds a kernel address and scause says “timer”, so usertrap saves the kernel address as epc, never calls syscall(), and echo returns to that address in user mode, faults and is killed.Nothing changes: kernelvec saves and restores sepc and scause.The write runs twice.

22deepChoose all that apply

In this kernel, which of these traps from user mode end with usertrap killing the process?

kernel/trap.c

} else if ((which_dev = devintr()) != 0) {
  // ok
} else if ((r_scause() == 15 || r_scause() == 13) &&
           vmfault(p->pagetable, p->sz, r_stval(),
                   (r_scause() == 13) ? 1 : 0) != 0) {
  // page fault on lazily-allocated page
} else {
  printk("usertrap(): unexpected scause 0x%lx pid=%d\n", r_scause(), p->pid);
  printk("            sepc=0x%lx stval=0x%lx\n", r_sepc(), r_stval());
  setkilled(p);
}
80
if (killed(p))
  kexit(-1);

an illegal instruction, such as csrr of a machine-mode CSR (scause 2)a load from a lazily allocated page below p->sz that was never touched (scause 13)an ecall with a7 = 99a store to the program’s own read-only text page (scause 15)a jump into a lazily allocated heap page that was never touched (instruction page fault, scause 12)a timer interrupt

23deepPut in order

cat is inside copyout (in a system call, interrupts on) when hart 2’s timer fires. Put the steps in order.

kerneltrap copies sepc and sstatus into local variables
The hardware sets sepc to the interrupted instruction, scause to the timer code, clears SIE, and jumps to stvec, which is kernelvec
kernelvec pushes a 256-byte frame on cat’s kernel stack and saves the caller-saved registers
devintr calls clockintr, which writes a new stimecmp
kernelvec reloads the registers, pops its frame and executes sret
yield switches away; later cat resumes, perhaps on another hart
kerneltrap writes the saved sepc and sstatus back

24solidChoose one

Hart 1 has nothing to run. Its scheduler briefly enables interrupts at line 441, and a timer interrupt is taken there. What does kerneltrap do?

kernel/trap.c

if ((which_dev = devintr()) == 0) {
  // interrupt or trap from an unknown source
  printk("scause=0x%lx sepc=0x%lx stval=0x%lx\n", scause, r_sepc(),
         r_stval());
  panic("kerneltrap");
}
155
// give up the CPU if this is a timer interrupt.
if (which_dev == 2 && myproc() != 0)
  yield();
159
// the yield() may have caused some traps to occur,
// so restore trap registers for use by kernelvec.S's sepc instruction.
w_sepc(sepc);
w_sstatus(sstatus);
164}

clockintr re-arms hart 1’s timer, and that is all: myproc() is 0, so there is no yield, and kernelvec returns to the scheduler loop.It increments ticks and wakes sleepers.It calls yield so the scheduler can pick a process.It panics, because traps are not allowed on the scheduler stack.

25solidType a number

During one system call that neither sleeps nor yields, how many times is stvec written, from the ecall to the sret?

decimal, 0x hex or 0b binary