xv6, line by line
lab 5

Extension labs · lab 5 · Traps and control flow · ★★★☆☆

sigalarm and sigreturn: calling a user handler from a timer

You add two system calls. sigalarm(n, handler) asks the kernel to call handler in user space after every n timer ticks of CPU time the process uses; sigreturn(), called at the end of the handler, puts the process back exactly where the timer interrupted it, with every register as it was. sigalarm(0, 0) turns the alarm off. It is a small version of what Unix calls a signal: the kernel makes a running program jump to a function it never called, then makes it carry on as if nothing had happened.

About sixty lines of kernel code (not counting comments), and almost every one of them touches something central. You will decide what “a tick of this process” means when three harts each take their own timer interrupt but only hart 0 advances ticks. You will find out where the hart decides which user instruction comes next, and whether you can change it from usertrap. You will decide what to save so that the interrupted code notices nothing, and where to keep it. Along the way: is user address 0 a valid handler? What happens to a0 when sigreturn, itself a system call, returns a value? And what if the handler runs longer than its interval?

Read first: Tour 5: Life of a system call, Tour 7: The trampoline and the trapframe, Tour 11: From a timer tick to a context switch, Tour 22: exec, Tour 43: A system call, CSR by CSR, Tour 44: One interrupt, three landing sites · The stacks of xv6, Locks and interrupt state, Mode, stack and page table: the master question

What this lab teaches

  • How a timer interrupt reaches usertrap on every hart, why the global ticks counts only hart 0’s interrupts (kernel/trap.c:169), and how to charge a tick to the process that was running on the hart that took it.
  • How the return to user space chooses where to land, and which piece of state you must change (a register, or something in memory?) to send the process somewhere else.
  • What the trapframe holds (31 registers and the program counter), what an ordinary C function called from nowhere changes (sp, ra, s0 and its scratch registers), and why the copy has to live somewhere the kernel controls.
  • How the ordinary system-call path (usertrap and syscall) treats the trapframe before and after a handler runs, and what that means for a system call whose job is to restore registers.
  • What happens when a timer tick lands inside a handler that is still running, and what one per-process fact prevents it; the clinic shows the real failure.
  • Why all of this needs no lock on three harts: every field is used only by the process that owns it, and a process runs on one hart at a time even when it moves between harts mid-handler.

The reference branch

ext/05-sigalarm in ShowMeTheStack/xv6-riscv-labs, branched from the frozen commit 06aad25; 8 commits.

git clone https://github.com/ShowMeTheStack/xv6-riscv-labs
cd xv6-riscv-labs
git checkout -b my-sigalarm 06aad25   # start your own
git diff 06aad25 origin/ext/05-sigalarm   # only when you want the answer

1. The spec

The system calls.

int sigalarm(int ticks, void (*handler)());
int sigreturn(void);

Constraints. Processes that never call sigalarm behave exactly as before; usertests -q must still print ALL TESTS PASSED on three harts. Existing system call numbers do not change (sigalarm is 23, sigreturn 24).

The test. alarmtest runs six checks and prints OK or FAIL for each. Everything it reports, it checks itself; nothing needs to be compared by eye.

$ alarmtest
alarmtest: periodic is at address 0x0000000000000000
alarmtest: 5 calls; 0 more in 10 ticks after sigalarm(0, 0)
alarmtest: handler called, then stopped: OK
alarmtest: sigreturn outside a handler fails: OK
alarmtest: the handler ran during 10 fillspin runs (10 calls)
alarmtest: registers preserved: OK
alarmtest: slow ran 3 times, nested at most 1 deep
alarmtest: no re-entry: OK
alarmtest: the handler ran during 10 a0spin runs (10 calls)
alarmtest: sigreturn restores a0: OK
alarmtest: handler calls in the forked child: 0, after exec: 0
alarmtest: fork and exec start with the alarm off: OK
alarmtest: 6 of 6 checks OK

A second program, alarmrate INTERVAL SECONDS NPROC, measures how often handlers run; it checks nothing and prints numbers (see Measure).

2. Think first

Answer each question in your head (or on paper) before opening a hint. Hints get more specific; the reference answer comes last.

1What does the kernel need to remember, and what means "off"?

sigalarm(n, handler) must record something in the kernel for later. List the per-process state you need. Then decide: how does the rest of the kernel tell whether the alarm is on? Commit to one test, for example “the handler is not null” or “the interval is not 0”, and justify it.

Check yourself

1warm-upChoose one

A learner writes if (p->alarmhandler == 0) return; as the “alarm is off” test. alarmtest calls sigalarm(2, periodic), and periodic is at user address 0. What happens?

2What counts as one tick of this process?

The alarm should fire after the process has used n ticks of CPU time. The kernel already counts ticks. On three harts, decide exactly which event is “one tick of this process”, and find the place in the kernel where that event happens and you know which process it belongs to. Is reading ticks enough?

Check yourself

1solidChoose one

A process spins in user mode on hart 2 for one second. Nothing else is runnable. About how many timer interrupts does hart 2 take, and by how much does ticks advance?

2solidFill in the machine state

A timer interrupt has arrived while alarmtest (pid 3) was spinning in user mode on hart 2. usertrap has called devintr(), which returned 2, and is about to count the tick. Fill in the machine state of hart 2.

3How do you make the process land in the handler?

The count has reached the interval. The next user instruction this process executes must be the first instruction of the handler, not the one the timer interrupted. The hart will leave the kernel through sret. Where does sret get its target, and what is the last place where you can change it?

Check yourself

1solidClick the line

Click the line that decides where sret sends the process.

kernel/trap.c
103 struct proc *p = myproc();
105 // we're about to switch the destination of traps from
106 // kerneltrap() to usertrap(). because a trap from kernel
107 // code to usertrap would be a disaster, turn off interrupts.
110 // send syscalls, interrupts, and exceptions to uservec in trampoline.S
114 // set up trapframe values that uservec will need when
115 // the process next traps into the kernel.
116 p->trapframe->kernel_satp = r_satp(); // kernel page table
117 p->trapframe->kernel_sp = p->kstack + PGSIZE; // process's kernel stack
119 p->trapframe->kernel_hartid = r_tp(); // hartid for cpuid()
121 // set up the registers that trampoline.S's sret will use
122 // to get to user space.
124 // set S Previous Privilege mode to User.
125 unsigned long x = r_sstatus();
126 x &= ~SSTATUS_SPP; // clear SPP to 0 for user mode
127 x |= SSTATUS_SPIE; // enable interrupts in user mode
130 // set S Exception Program Counter to the saved user pc.

Your pick: none yet (click a line in the code)

2solidChoose one

A learner skips the trapframe and writes w_sepc(p->alarmhandler); in usertrap just before yield(). What happens?

4What must you save, and where?

The handler is an ordinary C function. It was never called, so it has no caller to return to, and it will run with the interrupted code’s registers and then change some of them. Which registers can it change? What exactly must you keep so that the interrupted code can continue as if nothing happened, and where will you keep it?

Check yourself

1solidChoose all that apply

periodic is count++; sigreturn();, compiled as addi sp,sp,-16; sd ra,8(sp); sd s0,0(sp); addi s0,sp,16; then auipc a5 / lw a5 / addiw a5 / auipc a4 / sw a5, then jal sigreturn, whose stub is li a7,24; ecall. When the ecall executes, which registers may differ from their values at the moment of the timer interrupt?

5How does sigreturn get back, and what happens to a0?

sigreturn() is itself a system call. Trace what happens to the trapframe’s epc and a0 from the handler’s ecall to the moment the interrupted code runs again. Then write the body of sys_sigreturn: what does it copy, and what does it return?

Check yourself

1solidType a number

sys_sigreturn restores the trapframe correctly but ends with return 0;. The interrupted code had a0 = 0x0a0a0a0a0a0a0a0a. What value is in a0 when the interrupted code runs again?

decimal, 0x hex or 0b binary
2deepPut in order

Put these events in order, from the handler’s call to sigreturn to the interrupted code running again.

  1. prepare_return copies trapframe->epc into sepc
  2. userret reloads a0 from the trapframe and executes sret
  3. sys_sigreturn copies alarmframe over the trapframe
  4. uservec saves the handler’s registers in the trapframe
  5. syscall() stores the return value in trapframe->a0
  6. usertrap adds 4 to trapframe->epc

6What if the handler is still running at the next tick?

Take sigalarm(1, slow) where slow runs for three ticks before calling sigreturn. With the design so far, what happens at the first tick that lands while slow is running? Follow it through the saved frame. Then decide what the kernel should do with ticks that arrive while a handler runs.

Check yourself

1deepChoose one

With no “handler running” flag, alarmtest’s re-entry test (sigalarm(1, slow), slow spins for 3 ticks) was run on three harts. What did the run show?

7Does any of this need a lock on three harts?

List every place that reads or writes the alarm fields: sys_sigalarm, the timer path in usertrap, sys_sigreturn, and (coming next) exec and freeproc. Can two of them ever run at the same time for the same process? Can an interrupt land in the middle of one of them and run another? Decide whether you need a lock.

Check yourself

1solidTrue or false, and why

True or false: sys_sigalarm must hold p->lock while it sets the three fields, because a timer interrupt could run the counting code in the middle.

Why?

8What should fork and exec do with an alarm?

A process with an alarm calls fork, or calls exec. For each, decide whether the alarm should survive, and which function must change (if any). Think about what the handler’s address means in each case.

Check yourself

1solidChoose one

kexec is left unchanged. A process sets sigalarm(1, handler) and execs ls. What is the most accurate prediction?

3. Build it

Start from the frozen commit, in your own clone of xv6 (not this site’s xv6/):

git checkout -b my-sigalarm 06aad25

Work in this order; each milestone can be tested before the next. Run make clean whenever you change kernel/proc.h (or after switching between commits): the Makefile’s dependency files usually catch it, but a kernel with two layouts of struct proc in different object files fails in baffling ways. (A stale object file can make even a correct solution fail its own test.)

  1. The plumbing. Numbers 23 and 24 in kernel/syscall.h, two entry(...) lines in user/usys.pl, two prototypes in user/user.h, two externs and two table entries in kernel/syscall.c. Three fields in struct proc. sys_sigalarm stores them; sys_sigreturn returns -1 for now. Check with a tiny program that sigalarm(2, f) returns 0.
  2. Count. In usertrap's which_dev == 2 branch, before yield(), count a tick if the interval is not 0. To see it work, temporarily printk the count every 10 ticks.
  3. Deliver. When the count reaches the interval, save the trapframe and set epc to the handler. A handler that only increments a counter and loops forever should now run. (Without sigreturn it cannot get back yet.)
  4. Return. sys_sigreturn restores the frame and returns the restored a0. Now a handler that ends with sigreturn() works: try test_called from alarmtest.
  5. No re-entry. The alarmactive flag, set in delivery, cleared in sigreturn.
  6. exec and freeproc. Turn the alarm off in kexec, clear the fields in freeproc.
  7. The test. alarmtest, then usertests -q, then alarmtest again.

Debugging. Most bugs here show up in user space, as wrong registers or a process lost in a loop, so the kernel never panics. TOOLPREFIX is your RISC-V toolchain’s prefix, the same one xv6’s Makefile detects (riscv64-unknown-elf-, riscv64-linux-gnu- or riscv64-elf-); set it with export TOOLPREFIX=riscv64-unknown-elf- or whichever you have. On Debian/Ubuntu/WSL, gdb-multiarch also works as the debugger. In gdb (make qemu-gdb, then ${TOOLPREFIX}gdb kernel/kernel and target remote), break on the line that changes epc and on sys_sigreturn, and print *p->trapframe and p->alarmframe side by side: epc, sp, ra and a0 tell the story. For a hang, interrupt and look at each hart (info threads, thread 2, bt); for a process stuck in user space, print its trapframe from proc[i].trapframe. gdb reads memory through the page table that is installed at the moment it stops, so to read user memory, stop while that process’s page table is in satp, or walk the page table yourself.

4. Debugging clinic

Each of these bugs was put into the reference solution on purpose and run on three harts. The symptom is exactly what happened. Try to explain it before revealing why.

1Saved and restored only the program counter

The learner reasons that the handler only needs to “come back to where it was”:

-  struct trapframe alarmframe; // user registers when the handler was called
+  uint64 alarmepc;     // user pc when the handler was called
...
-  p->alarmframe = *p->trapframe;
+  p->alarmepc = p->trapframe->epc;
...
-  *p->trapframe = p->alarmframe;
+  p->trapframe->epc = p->alarmepc;

What happened when we ran it

$ alarmtest
alarmtest: periodic is at address 0x0000000000000000
alarmtest: 5 calls; 0 more in 10 ticks after sigalarm(0, 0)
alarmtest: handler called, then stopped: OK
alarmtest: sigreturn outside a handler fails: OK

(no more output: no prompt after 90 seconds. gdb then attached six times, about two
seconds apart, reading memory physically; first sample:)
--- attach 1 16:31:56.814508084
s_sstatus (x=2) at kernel/riscv.h:67
warning: 67	kernel/riscv.h: No such file or directory
hart 0 pc=0x80001e10 sepc=0x80001e14 scause=0x8000000000000005 satp=0x8000000000087fff(kernel)
hart 1 pc=0x80001e10 sepc=0x80001e14 scause=0x8000000000000005 satp=0x8000000000087fff(kernel)
hart 2 pc=0x80000bbc sepc=0xd76 scause=0x8 satp=0x8000000000087fff(kernel) cpus[2].proc=alarmtest/3
proc init/1 SLEEPING satp=0x8000000000087f52 interval=0 aticks=0 active=0 tf.epc=0x380 tf.sp=0x3fb0 tf.ra=0x4a tf.a0=0x0 tf.a7=0x3
proc sh/2 SLEEPING satp=0x8000000000087f41 interval=0 aticks=0 active=0 tf.epc=0xc94 tf.sp=0x4f90 tf.ra=0x938 tf.a0=0x0 tf.a7=0x3
proc alarmtest/3 RUNNING satp=0x8000000000087f25 interval=1 aticks=0 active=0 tf.epc=0xd7a tf.sp=0x4f20 tf.ra=0x1e tf.a0=0xffffffffffffffff tf.a7=0x18
  user 0x4f20: 0x0101010101010101
  user 0x4f28: 0x0000000000000000
  user 0x4f30: 0x0000000000004fb0
  user 0x4f38: 0x000000000000068e
  user 0x4f40: 0x0000000000000012
  user 0x4f48: 0x0505050505050505
[...]
--- attach 2 16:31:58.922623975
[...]
hart 1 pc=0x800025de sepc=0xd76 scause=0x8 satp=0x8000000000087fff(kernel) cpus[1].proc=alarmtest/3
[...]
proc alarmtest/3 RUNNING satp=0x8000000000087f25 interval=1 aticks=0 active=0 tf.epc=0xd7a tf.sp=0x4f20 tf.ra=0x1e tf.a0=0xffffffffffffffff tf.a7=0x18

2No check for a handler that is already running

The flag exists and sigreturn clears it, but the timer path does not look at it:

 alarmtick(struct proc *p)
 {
-  if (p->alarminterval == 0 || p->alarmactive)
+  if (p->alarminterval == 0)
     return;

What happened when we ran it

$ alarmtest
alarmtest: periodic is at address 0x0000000000000000
alarmtest: 5 calls; 0 more in 10 ticks after sigalarm(0, 0)
alarmtest: handler called, then stopped: OK
alarmtest: sigreturn outside a handler fails: OK
alarmtest: the handler ran during 10 fillspin runs (10 calls)
alarmtest: registers preserved: OK
alarmtest: slow entered while already running
alarmtest: slow entered while already running
[...]
alarmtest: slow entered while already running
usertrap(): unexpected scause 0xf pid=3
            sepc=0xd7e stval=0x3fe8
$

(56 "slow entered while already running" lines in all.)

3Used handler == 0 to mean “off”

 alarmtick(struct proc *p)
 {
-  if (p->alarminterval == 0 || p->alarmactive)
+  if (p->alarmhandler == 0 || p->alarmactive)
     return;

What happened when we ran it

$ alarmtest
alarmtest: periodic is at address 0x0000000000000000
alarmtest: 0 calls; 0 more in 10 ticks after sigalarm(0, 0)
alarmtest: handler called, then stopped: FAIL
alarmtest: sigreturn outside a handler fails: OK
alarmtest: the handler ran during 0 fillspin runs (0 calls)
alarmtest: registers preserved: FAIL
alarmtest: slow ran 3 times, nested at most 1 deep
alarmtest: no re-entry: OK
alarmtest: the handler ran during 0 a0spin runs (0 calls)
alarmtest: sigreturn restores a0: FAIL
alarmtest: handler calls in the forked child: 0, after exec: 0
alarmtest: fork and exec start with the alarm off: OK
alarmtest: 3 of 6 checks FAILED
$

4sigreturn returned 0

   *p->trapframe = p->alarmframe;
-  // syscall() stores the return value in trapframe->a0:
-  // return the interrupted code's a0, not 0.
-  return p->trapframe->a0;
+  return 0;
 }

What happened when we ran it

$ alarmtest
alarmtest: periodic is at address 0x0000000000000000
alarmtest: 5 calls; 0 more in 10 ticks after sigalarm(0, 0)
alarmtest: handler called, then stopped: OK
alarmtest: sigreturn outside a handler fails: OK
alarmtest: the handler ran during 10 fillspin runs (10 calls)
alarmtest: registers preserved: OK
alarmtest: slow ran 3 times, nested at most 1 deep
alarmtest: no re-entry: OK
alarmtest: a0 is 0x0 after sigreturn
[...]
alarmtest: a0 is 0x0 after sigreturn
alarmtest: the handler ran during 10 a0spin runs (10 calls)
alarmtest: sigreturn restores a0: FAIL
alarmtest: handler calls in the forked child: 0, after exec: 0
alarmtest: fork and exec start with the alarm off: OK
alarmtest: 1 of 6 checks FAILED
$

(ten "a0 is 0x0" lines in all, one per interrupted run.)

5Counted ticks next to ticks++ in clockintr

The learner looks for “where the tick happens” and finds ticks++:

   if (cpuid() == 0) {
     acquire(&tickslock);
     ticks++;
     wakeup(&ticks);
     release(&tickslock);
+    // one more tick for the running process's alarm.
+    struct proc *p = myproc();
+    if (p != 0 && p->alarminterval != 0 && !p->alarmactive)
+      p->alarmticks++;
   }

and alarmtick no longer increments the count; it only checks it and delivers.

What happened when we ran it

$ alarmtest
alarmtest: periodic is at address 0x0000000000000000
alarmtest: 5 calls; 0 more in 10 ticks after sigalarm(0, 0)
alarmtest: handler called, then stopped: OK
[...]
alarmtest: 6 of 6 checks OK
$ alarmrate 1 10 1
alarmrate: pid 7: 28 handler calls
alarmrate: a process with a hart to itself for 100 ticks would make about 100
$ alarmrate 1 10 3
alarmrate: pid 9: 7 handler calls
alarmrate: pid 10: 0 handler calls
alarmrate: pid 11: 89 handler calls
alarmrate: a process with a hart to itself for 100 ticks would make about 100
$ alarmrate 1 10 1
alarmrate: pid 13: 83 handler calls
alarmrate: a process with a hart to itself for 100 ticks would make about 100

(a second boot of the same kernel, gdb attaching 8 times about two seconds apart
while this ran:)
$ alarmrate 1 30 3
alarmrate: pid 4: 6 handler calls
alarmrate: pid 6: 251 handler calls
alarmrate: pid 5: 23 handler calls
alarmrate: a process with a hart to itself for 300 ticks would make about 300

(gdb, first sample:)
--- attach 1 16:31:00.452078225
hart 0 pc=0x11c sepc=0x49a scause=0x8 satp=0x8000000000087f33(alarmrate/6) cpus[0].proc=alarmrate/6
hart 1 pc=0x11c sepc=0x49a scause=0x8 satp=0x8000000000087f29(alarmrate/5) cpus[1].proc=alarmrate/5
hart 2 pc=0x11c sepc=0x49a scause=0x8 satp=0x8000000000087f24(alarmrate/4) cpus[2].proc=alarmrate/4
[...]

5. The reference solution

Take the guided tour through the reference solution, one commit at a time, with the machine state at every step:

Open the reveal tour →

Or read the commits

  1. 7f077bf Add the sigalarm and sigreturn system calls

    kernel/proc.h

    @@ -100,5 +100,10 @@ struct proc {
    100100 struct context context; // swtch() here to run process
    101101 struct file *ofile[NOFILE]; // Open files
    102102 struct inode *cwd; // Current directory
    103103 char name[16]; // Process name (debugging)
    104
    105 // sigalarm state, also private to the process.
    106 int alarminterval; // ticks between handler calls; 0 = off
    107 int alarmticks; // ticks counted since the last call
    108 uint64 alarmhandler; // user address of the handler (0 is valid)
    104109};

    kernel/syscall.c

    @@ -102,8 +102,10 @@ extern uint64 sys_unlink(void);
    102102extern uint64 sys_link(void);
    103103extern uint64 sys_mkdir(void);
    104104extern uint64 sys_close(void);
    105105extern uint64 sys_sync(void);
    106extern uint64 sys_sigalarm(void);
    107extern uint64 sys_sigreturn(void);
    106108
    107109// An array mapping syscall numbers from syscall.h
    108110// to the function that handles the system call.
    109111static uint64 (*syscalls[])(void) = {
    @@ -129,8 +131,10 @@ static uint64 (*syscalls[])(void) = {
    129131 [SYS_link] = sys_link,
    130132 [SYS_mkdir] = sys_mkdir,
    131133 [SYS_close] = sys_close,
    132134 [SYS_sync] = sys_sync,
    135 [SYS_sigalarm] = sys_sigalarm,
    136 [SYS_sigreturn] = sys_sigreturn,
    133137 // clang-format on
    134138};
    135139
    136140void

    kernel/syscall.h

    @@ -20,4 +20,6 @@
    2020#define SYS_link 19
    2121#define SYS_mkdir 20
    2222#define SYS_close 21
    2323#define SYS_sync 22
    24#define SYS_sigalarm 23
    25#define SYS_sigreturn 24

    kernel/sysproc.c

    @@ -109,4 +109,31 @@ sys_uptime(void)
    109109 xticks = ticks;
    110110 release(&tickslock);
    111111 return xticks;
    112112}
    113
    114// sigalarm(ticks, handler): call handler after every ticks
    115// timer ticks of user CPU time. sigalarm(0, 0) turns it off.
    116uint64
    117sys_sigalarm(void)
    118{
    119 int interval;
    120 uint64 handler;
    121 struct proc *p = myproc();
    122
    123 argint(0, &interval);
    124 argaddr(1, &handler);
    125 if (interval < 0)
    126 return -1;
    127 p->alarminterval = interval;
    128 p->alarmhandler = handler;
    129 p->alarmticks = 0;
    130 return 0;
    131}
    132
    133// sigreturn(): return from an alarm handler to the code it
    134// interrupted. nothing to return from yet.
    135uint64
    136sys_sigreturn(void)
    137{
    138 return -1;
    139}

    user/user.h

    @@ -24,8 +24,10 @@ int getpid(void);
    2424char *sys_sbrk(int, int);
    2525int pause(int);
    2626int uptime(void);
    2727int sync(void);
    28int sigalarm(int ticks, void (*handler)());
    29int sigreturn(void);
    2830
    2931// ulib.c
    3032int stat(const char *, struct stat *);
    3133char *strcpy(char *, const char *);

    user/usys.pl

    @@ -42,4 +42,6 @@ entry("getpid");
    4242entry("sbrk");
    4343entry("pause");
    4444entry("uptime");
    4545entry("sync");
    46entry("sigalarm");
    47entry("sigreturn");
  2. 7f7b105 Count a process's timer ticks in usertrap

    kernel/trap.c

    @@ -14,8 +14,9 @@ extern char trampoline[], uservec[];
    1414// in kernelvec.S, calls kerneltrap().
    1515void kernelvec();
    1616
    1717extern int devintr();
    18static void alarmtick(struct proc *);
    1819
    1920void
    2021trapinit(void)
    2122{
    @@ -81,10 +82,12 @@ usertrap(void)
    8182 if (killed(p))
    8283 kexit(-1);
    8384
    8485 // give up the CPU if this is a timer interrupt.
    85 if (which_dev == 2)
    86 if (which_dev == 2) {
    87 alarmtick(p);
    8688 yield();
    89 }
    8790
    8992
    9093 // the user page table to switch to, for trampoline.S
    @@ -93,8 +96,19 @@ usertrap(void)
    9396 // return to trampoline.S; satp value in a0.
    9497 return satp;
    9598}
    9699
    100// the timer interrupted p while it ran in user mode, on this hart.
    101// every hart takes its own timer interrupts, so this counts p's
    102// user CPU time on all harts, not just hart 0's ticks.
    103static void
    104alarmtick(struct proc *p)
    105{
    106 if (p->alarminterval == 0)
    107 return;
    108 p->alarmticks++;
    109}
    110
    97111//
    98112// set up trapframe and control registers for a return to user space
    99113//
    100114void
  3. 73aab91 Call the alarm handler when the interval runs out

    kernel/proc.h

    @@ -105,5 +105,6 @@ struct proc {
    105105 // sigalarm state, also private to the process.
    106106 int alarminterval; // ticks between handler calls; 0 = off
    107107 int alarmticks; // ticks counted since the last call
    108108 uint64 alarmhandler; // user address of the handler (0 is valid)
    109 struct trapframe alarmframe; // user registers when the handler was called
    109110};

    kernel/trap.c

    @@ -105,8 +105,17 @@ alarmtick(struct proc *p)
    105105{
    106106 if (p->alarminterval == 0)
    107107 return;
    108108 p->alarmticks++;
    109 if (p->alarmticks < p->alarminterval)
    110 return;
    111
    112 // call the handler: keep a copy of every user register,
    113 // then make the return to user space jump to the handler.
    114 // prepare_return copies epc into sepc for userret's sret.
    115 p->alarmticks = 0;
    116 p->alarmframe = *p->trapframe;
    117 p->trapframe->epc = p->alarmhandler;
    109118}
    110119
    111120//
    112121// set up trapframe and control registers for a return to user space
  4. ec3efd0 Restore the interrupted registers in sigreturn

    kernel/sysproc.c

    @@ -130,10 +130,16 @@ sys_sigalarm(void)
    130130 return 0;
    131131}
    132132
    133133// sigreturn(): return from an alarm handler to the code it
    134// interrupted. nothing to return from yet.
    134// interrupted, with every register as it was.
    135135uint64
    136136sys_sigreturn(void)
    137137{
    138 return -1;
    138 struct proc *p = myproc();
    139
    140 // overwrites the epc that usertrap advanced past the ecall.
    141 *p->trapframe = p->alarmframe;
    142 // syscall() stores the return value in trapframe->a0:
    143 // return the interrupted code's a0, not 0.
    144 return p->trapframe->a0;
    139145}
  5. 733f7ac Do not re-enter a running alarm handler

    kernel/proc.h

    @@ -105,6 +105,7 @@ struct proc {
    105105 // sigalarm state, also private to the process.
    106106 int alarminterval; // ticks between handler calls; 0 = off
    107107 int alarmticks; // ticks counted since the last call
    108108 uint64 alarmhandler; // user address of the handler (0 is valid)
    109 int alarmactive; // the handler is running: don't call it again
    109110 struct trapframe alarmframe; // user registers when the handler was called
    110111};

    kernel/sysproc.c

    @@ -136,8 +136,12 @@ uint64
    136136sys_sigreturn(void)
    137137{
    138138 struct proc *p = myproc();
    139139
    140 if (!p->alarmactive)
    141 return -1;
    142 p->alarmactive = 0;
    143
    140144 // overwrites the epc that usertrap advanced past the ecall.
    141145 *p->trapframe = p->alarmframe;
    142146 // syscall() stores the return value in trapframe->a0:
    143147 // return the interrupted code's a0, not 0.

    kernel/trap.c

    @@ -102,9 +102,9 @@ usertrap(void)
    102102// user CPU time on all harts, not just hart 0's ticks.
    103103static void
    104104alarmtick(struct proc *p)
    105105{
    106 if (p->alarminterval == 0)
    106 if (p->alarminterval == 0 || p->alarmactive)
    107107 return;
    108108 p->alarmticks++;
    109109 if (p->alarmticks < p->alarminterval)
    110110 return;
    @@ -112,8 +112,9 @@ alarmtick(struct proc *p)
    112112 // call the handler: keep a copy of every user register,
    113113 // then make the return to user space jump to the handler.
    114114 // prepare_return copies epc into sepc for userret's sret.
    115115 p->alarmticks = 0;
    116 p->alarmactive = 1;
    116117 p->alarmframe = *p->trapframe;
    117118 p->trapframe->epc = p->alarmhandler;
    118119}
    119120
  6. c877bd6 Turn the alarm off in exec and in freeproc

    kernel/exec.c

    @@ -136,8 +136,12 @@ kexec(char *path, char **argv)
    136136 p->trapframe->epc = elf.entry; // initial program counter = ulib.c:start()
    137137 p->trapframe->sp = sp; // initial stack pointer
    138138 proc_freepagetable(oldpagetable, oldsz);
    139139
    140 // the handler's address meant something only in the old image.
    141 p->alarminterval = 0;
    142 p->alarmactive = 0;
    143
    140144 return argc; // this ends up in a0, the first argument to main(argc, argv)
    141145
    142146bad:
    143147 if (pagetable)

    kernel/proc.c

    @@ -166,8 +166,12 @@ freeproc(struct proc *p)
    166166 p->name[0] = 0;
    167167 p->chan = 0;
    168168 p->killed = 0;
    169169 p->xstate = 0;
    170 p->alarminterval = 0;
    171 p->alarmticks = 0;
    172 p->alarmhandler = 0;
    173 p->alarmactive = 0;
    170174 p->state = UNUSED;
    171175}
    172176
    173177// Create a user page table for a given process, with no user memory,
  7. 5319e19 Add alarmtest, a test program for sigalarm

    Makefile

    @@ -145,8 +145,9 @@ UPROGS=\
    145145 $U/_usertests\
    146146 $U/_grind\
    147147 $U/_wc\
    148148 $U/_zombie\
    149 $U/_alarmtest\
    149150 $U/_logstress\
    150151 $U/_forphan\
    151152 $U/_dorphan\
    152153 $U/_sync\

    user/alarmtest.c

    @@ -0,0 +1,260 @@
    1// alarmtest: tests for sigalarm and sigreturn.
    2
    3#include "kernel/types.h"
    4#include "kernel/stat.h"
    5#include "user/user.h"
    6
    7volatile int count; // calls of periodic
    8
    9// the first function in this file: in this build it sits at
    10// user address 0, so a kernel that treats a handler of 0 as
    11// "no handler" never calls it.
    12void
    13periodic()
    14{
    15 count++;
    16 sigreturn();
    17}
    18
    19volatile int depth, maxdepth, slowcalls;
    20
    21// a handler that runs for 3 ticks or more, longer than its interval.
    22void
    23slow()
    24{
    25 int t0;
    26
    27 depth++;
    28 if (depth > maxdepth)
    29 maxdepth = depth;
    30 if (depth > 1)
    31 printf("alarmtest: slow entered while already running\n");
    32 t0 = uptime();
    33 while (uptime() - t0 < 3)
    34 for (volatile int i = 0; i < 100000; i++)
    35 ;
    36 slowcalls++;
    37 depth--;
    38 sigreturn();
    39}
    40
    41// burn user CPU time until *c reaches n or t ticks pass.
    42// returns 1 if *c reached n.
    43int
    44waitfor(volatile int *c, int n, int t)
    45{
    46 int t0 = uptime();
    47
    48 while (*c < n) {
    49 if (uptime() - t0 >= t)
    50 return 0;
    51 for (volatile int i = 0; i < 100000; i++)
    52 ;
    53 }
    54 return 1;
    55}
    56
    57int
    58check(char *what, int ok)
    59{
    60 printf("alarmtest: %s: %s\n", what, ok ? "OK" : "FAIL");
    61 return ok;
    62}
    63
    64// the handler runs every 2 ticks; sigalarm(0, 0) stops it.
    65int
    66test_called(void)
    67{
    68 int ok, n;
    69
    70 printf("alarmtest: periodic is at address %p\n", periodic);
    71 count = 0;
    72 sigalarm(2, periodic);
    73 ok = waitfor(&count, 5, 300);
    74 sigalarm(0, 0);
    75 n = count;
    76 waitfor(&count, n + 1, 10);
    77 printf("alarmtest: %d calls; %d more in 10 ticks after sigalarm(0, 0)\n", n,
    78 count - n);
    79 return check("handler called, then stopped", ok && count == n) +
    80 check("sigreturn outside a handler fails", sigreturn() == -1);
    81}
    82
    83// load register r with eight copies of the byte 0xb.
    84#define FILL(r, b) "li " #r ", 0x" #b #b #b #b #b #b #b #b "\n"
    85// regs[i] = the register (i is its number, in decimal)
    86#define SAVE(r, i) "sd " #r ", " #i "*8(a0)\n"
    87
    88uint64 regs[32];
    89
    90// fill 26 registers, spin with the counter in a0, save them all.
    91void
    92fillspin(void)
    93{
    94 asm volatile(FILL(ra, 01) FILL(t0, 05) FILL(t1, 06) FILL(t2, 07)
    95 FILL(s1, 09) FILL(a1, 11) FILL(a2, 12) FILL(a3, 13)
    96 FILL(a4, 14) FILL(a5, 15) FILL(a6, 16) FILL(a7, 17)
    97 FILL(s2, 18) FILL(s3, 19) FILL(s4, 20) FILL(s5, 21)
    98 FILL(s6, 22) FILL(s7, 23) FILL(s8, 24) FILL(s9, 25)
    99 FILL(s10, 26) FILL(s11, 27) FILL(t3, 28) FILL(t4, 29)
    100 FILL(t5, 30) FILL(t6, 31)
    101 "li a0, 20000000\n"
    102 "1: addi a0, a0, -1\n"
    103 "bgtz a0, 1b\n"
    104 "la a0, regs\n"
    105 SAVE(ra, 1) SAVE(t0, 5) SAVE(t1, 6) SAVE(t2, 7)
    106 SAVE(s1, 9) SAVE(a1, 11) SAVE(a2, 12) SAVE(a3, 13)
    107 SAVE(a4, 14) SAVE(a5, 15) SAVE(a6, 16) SAVE(a7, 17)
    108 SAVE(s2, 18) SAVE(s3, 19) SAVE(s4, 20) SAVE(s5, 21)
    109 SAVE(s6, 22) SAVE(s7, 23) SAVE(s8, 24) SAVE(s9, 25)
    110 SAVE(s10, 26) SAVE(s11, 27) SAVE(t3, 28) SAVE(t4, 29)
    111 SAVE(t5, 30) SAVE(t6, 31)
    112 :
    113 :
    114 : "ra", "t0", "t1", "t2", "s1", "a0", "a1", "a2", "a3", "a4",
    115 "a5", "a6", "a7", "s2", "s3", "s4", "s5", "s6", "s7", "s8",
    116 "s9", "s10", "s11", "t3", "t4", "t5", "t6", "memory");
    117}
    118
    119// the registers fillspin checks, by number. FILL writes the number's
    120// decimal digits as hex: x11 holds 0x1111111111111111.
    121int regno[] = {1, 5, 6, 7, 9, 11, 12, 13, 14, 15, 16, 17, 18,
    122 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31};
    123
    124// the handler interrupts fillspin; every register must survive.
    125int
    126test_registers(void)
    127{
    128 int c0, hits = 0, bad = 0, t0 = uptime();
    129
    130 count = 0;
    131 sigalarm(1, periodic);
    132 while (hits < 10 && uptime() - t0 < 300) {
    133 c0 = count;
    134 fillspin();
    135 if (count != c0)
    136 hits++;
    137 for (int i = 0; i < sizeof(regno) / sizeof(regno[0]); i++) {
    138 int n = regno[i];
    139 uint64 want = 0x0101010101010101ULL * (n / 10 * 16 + n % 10);
    140 if (regs[n] != want) {
    141 printf("alarmtest: x%d is 0x%lx, not 0x%lx\n", n, regs[n], want);
    142 bad++;
    143 }
    144 }
    145 }
    146 sigalarm(0, 0);
    147 printf("alarmtest: the handler ran during %d fillspin runs (%d calls)\n",
    148 hits, count);
    149 return check("registers preserved", hits == 10 && bad == 0);
    150}
    151
    152// a handler that runs longer than the interval is not entered again.
    153int
    154test_reentry(void)
    155{
    156 int ok;
    157
    158 depth = maxdepth = slowcalls = 0;
    159 sigalarm(1, slow);
    160 ok = waitfor(&slowcalls, 3, 300);
    161 sigalarm(0, 0);
    162 printf("alarmtest: slow ran %d times, nested at most %d deep\n", slowcalls,
    163 maxdepth);
    164 return check("no re-entry", ok && maxdepth == 1);
    165}
    166
    167// spin with a0 holding 0x0a0a0a0a0a0a0a0a; return a0.
    168uint64
    169a0spin(void)
    170{
    171 uint64 a0;
    172
    173 asm volatile("li a0, 0x0a0a0a0a0a0a0a0a\n"
    174 "li t0, 20000000\n"
    175 "1: addi t0, t0, -1\n"
    176 "bgtz t0, 1b\n"
    177 "mv %0, a0\n"
    178 : "=r"(a0)
    179 :
    180 : "a0", "t0");
    181 return a0;
    182}
    183
    184// sigreturn is a system call, and syscall() stores its return
    185// value in a0: the interrupted code's a0 must still survive.
    186int
    187test_a0(void)
    188{
    189 int c0, hits = 0, bad = 0, t0 = uptime();
    190 uint64 a0;
    191
    192 count = 0;
    193 sigalarm(1, periodic);
    194 while (hits < 10 && uptime() - t0 < 300) {
    195 c0 = count;
    196 a0 = a0spin();
    197 if (count != c0)
    198 hits++;
    199 if (a0 != 0x0a0a0a0a0a0a0a0aULL) {
    200 printf("alarmtest: a0 is 0x%lx after sigreturn\n", a0);
    201 bad++;
    202 }
    203 }
    204 sigalarm(0, 0);
    205 printf("alarmtest: the handler ran during %d a0spin runs (%d calls)\n", hits,
    206 count);
    207 return check("sigreturn restores a0", hits == 10 && bad == 0);
    208}
    209
    210// a forked child, and an exec'd program, start with no alarm.
    211int
    212test_forkexec(void)
    213{
    214 int pid, st1, st2;
    215 char *argv[] = {"alarmtest", "exec", 0};
    216
    217 sigalarm(1, periodic);
    218 pid = fork();
    219 if (pid == 0) {
    220 count = 0;
    221 waitfor(&count, 1, 10);
    222 exit(count);
    223 }
    224 wait(&st1);
    225 pid = fork();
    226 if (pid == 0) {
    227 // turn the alarm on in this process, then replace its program.
    228 sigalarm(1, periodic);
    229 exec("alarmtest", argv);
    230 exit(-1);
    231 }
    232 wait(&st2);
    233 sigalarm(0, 0);
    234 printf("alarmtest: handler calls in the forked child: %d, after exec: %d\n",
    235 st1, st2);
    236 return check("fork and exec start with the alarm off", st1 == 0 && st2 == 0);
    237}
    238
    239int
    240main(int argc, char *argv[])
    241{
    242 int ok = 0;
    243
    244 if (argc == 2 && strcmp(argv[1], "exec") == 0) {
    245 // the exec'd half of test_forkexec: periodic is at the same
    246 // address in this new image, so a surviving alarm would call it.
    247 waitfor(&count, 1, 10);
    248 exit(count);
    249 }
    250 ok += test_called();
    251 ok += test_registers();
    252 ok += test_reentry();
    253 ok += test_a0();
    254 ok += test_forkexec();
    255 if (ok == 6)
    256 printf("alarmtest: 6 of 6 checks OK\n");
    257 else
    258 printf("alarmtest: %d of 6 checks FAILED\n", 6 - ok);
    259 exit(0);
    260}
  8. 0da1305 Add alarmrate, to measure how often handlers run

    Makefile

    @@ -146,8 +146,9 @@ UPROGS=\
    146146 $U/_grind\
    147147 $U/_wc\
    148148 $U/_zombie\
    149149 $U/_alarmtest\
    150 $U/_alarmrate\
    150151 $U/_logstress\
    151152 $U/_forphan\
    152153 $U/_dorphan\
    153154 $U/_sync\

    user/alarmrate.c

    @@ -0,0 +1,56 @@
    1// alarmrate INTERVAL SECONDS NPROC: NPROC processes each burn user
    2// CPU time for SECONDS of wall-clock time with sigalarm(INTERVAL)
    3// on, and report how often their handlers ran.
    4
    5#include "kernel/types.h"
    6#include "kernel/stat.h"
    7#include "user/user.h"
    8
    9volatile int calls;
    10
    11void
    12tick()
    13{
    14 calls++;
    15 sigreturn();
    16}
    17
    18int
    19main(int argc, char *argv[])
    20{
    21 int interval, ticks, nproc, pid, st;
    22
    23 if (argc != 4) {
    24 fprintf(2, "usage: alarmrate interval seconds nproc\n");
    25 exit(1);
    26 }
    27 interval = atoi(argv[1]);
    28 ticks = atoi(argv[2]) * 10; // uptime() counts 10 ticks a second
    29 nproc = atoi(argv[3]);
    30 if (interval < 1 || ticks < 1 || nproc < 1) {
    31 fprintf(2, "alarmrate: bad arguments\n");
    32 exit(1);
    33 }
    34
    35 for (int i = 0; i < nproc; i++) {
    36 if (fork() == 0) {
    37 int t0 = uptime();
    38 sigalarm(interval, tick);
    39 while (uptime() - t0 < ticks)
    40 for (volatile int j = 0; j < 100000; j++)
    41 ;
    42 sigalarm(0, 0);
    43 exit(calls);
    44 }
    45 }
    46 // the children report through their exit status, so their
    47 // lines cannot interleave.
    48 for (int i = 0; i < nproc; i++) {
    49 pid = wait(&st);
    50 printf("alarmrate: pid %d: %d handler calls\n", pid, st);
    51 }
    52 printf("alarmrate: a process with a hart to itself for %d ticks "
    53 "would make about %d\n",
    54 ticks, ticks / interval);
    55 exit(0);
    56}

6. Verify and measure

On the branch head, on three harts (QEMU -smp 3), alarmtest passes, usertests -q passes, and alarmtest passes again afterwards:

$ alarmtest
alarmtest: periodic is at address 0x0000000000000000
alarmtest: 5 calls; 0 more in 10 ticks after sigalarm(0, 0)
alarmtest: handler called, then stopped: OK
alarmtest: sigreturn outside a handler fails: OK
alarmtest: the handler ran during 10 fillspin runs (10 calls)
alarmtest: registers preserved: OK
alarmtest: slow ran 3 times, nested at most 1 deep
alarmtest: no re-entry: OK
alarmtest: the handler ran during 10 a0spin runs (10 calls)
alarmtest: sigreturn restores a0: OK
alarmtest: handler calls in the forked child: 0, after exec: 0
alarmtest: fork and exec start with the alarm off: OK
alarmtest: 6 of 6 checks OK
$ usertests -q
usertests starting
[...]
test lazy_sbrk: OK
test partial_write: OK
test unlinkcwd: OK
ALL TESTS PASSED
$ alarmtest
alarmtest: periodic is at address 0x0000000000000000
[...]
alarmtest: 6 of 6 checks OK

What this shows: the handler is called and stopped; all 26 compared registers and a0 survive ten interrupted runs each; a long handler is never nested; a forked child and an exec’d program start with the alarm off. No program in usertests calls sigalarm, so for every other process the only cost is one load and compare per user-mode timer interrupt (alarminterval == 0), and 312 more bytes in each of the 64 struct proc slots (struct proc grows from 360 to 672 bytes in this build).

Every commit on the branch was built from clean (make clean; make kernel/kernel fs.img).

Handler calls versus the requested interval, on three harts. alarmrate INTERVAL 10 NPROC (each process spins for 100 ticks of ticks, ten seconds), reference kernel, one boot:

interval processes handler calls per process with a hart to itself
1 1 97 about 100
1 3 91, 96, 94 about 100
1 6 45, 46, 49, 45, 49, 47 about 100
5 1 18 about 20
5 6 10, 9, 9, 9, 9, 9 about 20
1 1 (again, at the end) 89 about 100

The same measurement with ticks counted next to ticks++ (clinic 5), same machine: one process, interval 1, 10 seconds: 28 calls on one run and 83 on another; three processes: 89, 7 and 0. In a 30-second run with gdb sampling the harts, pid 6 was on hart 0 in all 8 samples and got 251 calls; pids 4 and 5, on harts 1 and 2, got 6 and 23. The reference kernel gave the three processes 91, 96 and 94. The difference is the whole point of the second think question: a process’s ticks must be counted on the hart it runs on, and only hart 0 advances ticks.

Cost of a delivery. One extra trip through the kernel (the sigreturn system call; the timer interrupt that redirects would have happened anyway), plus two 288-byte copies.

7. Go further