Interrupts, the PLIC and the timer

1warm-upChoose one

A supervisor timer interrupt is pending on hart 1. In this kernel, what makes it stop being pending?

kernel/trap.c

166void
167clockintr()
168{
169  if (cpuid() == 0) {
170    acquire(&tickslock);
171    ticks++;
172    wakeup(&ticks);
173    release(&tickslock);
174  }
175
176  // ask for the next timer interrupt. this also clears
177  // the interrupt request. 1000000 is about a tenth
178  // of a second.
179  w_stimecmp(r_time() + 1000000);
180}

clockintr writes a new, later value into stimecmp (line 179).devintr calls plic_complete for the timer.Taking the trap clears the pending bit automatically.A machine-mode timer handler clears mip.MTIP and forwards a software interrupt to S-mode.

2warm-upChoose one

In devintr, the external-interrupt branch asks the PLIC who interrupted with plic_claim, but the timer branch does not. Why?

kernel/trap.c

187int
188devintr()
189{
uint64 scause = r_scause();
191
if (scause == 0x8000000000000009L) {
  // this is a supervisor external interrupt, via PLIC.
194
  // irq indicates which device interrupted.
  int irq = plic_claim();
197
  if (irq == UART0_IRQ) {
    uartintr();
  } else if (irq == VIRTIO0_IRQ) {
    virtio_disk_intr();
  } else if (irq) {
    printk("unexpected interrupt irq=%d\n", irq);
  }
205
  // the PLIC allows each device to raise at most one
  // interrupt at a time; tell the PLIC the device is
  // now allowed to interrupt again.
  if (irq)
    plic_complete(irq);
211
  return 1;
} else if (scause == 0x8000000000000005L) {
  // timer interrupt.
  clockintr();
  return 2;
} else {
  return 0;
}
220}

Claiming is only needed when several harts might take the same interrupt, and only hart 0 takes timer interrupts.clockintr claims the timer interrupt itself, inside its own code.The timer is part of each hart, not a device behind the PLIC; its own scause code (5) already identifies it.The timer has PLIC priority 0, so a claim would always return 0.

3solidType a number

plicinithart writes one 32-bit value into this hart’s S-mode enable register. What is that value? (Hex like 0x10 or decimal are both accepted.)

kernel/plic.c

19void
20plicinithart(void)
21{
22  int hart = cpuid();
23
24  // set enable bits for this hart's S-mode
25  // for the uart and virtio disk.
26  *(uint32 *)PLIC_SENABLE(hart) = (1 << UART0_IRQ) | (1 << VIRTIO0_IRQ);
27
28  // set this hart's S-mode priority threshold to 0.
29  *(uint32 *)PLIC_SPRIORITY(hart) = 0;
30}

decimal, 0x hex or 0b binary

4solidType a number

What is the address of hart 2’s S-mode claim/complete register, the one plic_claim reads on hart 2? Give it in hex.

kernel/memlayout.h

32// qemu puts platform-level interrupt controller (PLIC) here.
33#define PLIC                 0x0c000000L
34#define PLIC_PRIORITY        (PLIC + 0x0)
35#define PLIC_PENDING         (PLIC + 0x1000)
36#define PLIC_SENABLE(hart)   (PLIC + 0x2080 + (hart) * 0x100)
37#define PLIC_SPRIORITY(hart) (PLIC + 0x201000 + (hart) * 0x2000)
38#define PLIC_SCLAIM(hart)    (PLIC + 0x201004 + (hart) * 0x2000)

decimal, 0x hex or 0b binary

5warm-upType a number

The machine has three harts, and QEMU’s time counter runs at 10,000,000 per second. Approximately how many times per second does the kernel’s global ticks counter increase? (Give the nearest whole number.)

kernel/trap.c

166void
167clockintr()
168{
169  if (cpuid() == 0) {
170    acquire(&tickslock);
171    ticks++;
172    wakeup(&ticks);
173    release(&tickslock);
174  }
175
176  // ask for the next timer interrupt. this also clears
177  // the interrupt request. 1000000 is about a tenth
178  // of a second.
179  w_stimecmp(r_time() + 1000000);
180}

decimal, 0x hex or 0b binary

6solidChoose all that apply

A hart is running kernel code in supervisor mode. Which of these does the hardware check before it takes a supervisor timer interrupt?

7warm-upDecode the bits

kerneltrap reads scause = 0x8000000000000009. Decode it.

Value: 0x8000000000000009

Bit 63Cause code (low bits)What devintr does nextdevintr’s return value

8solidDecode the bits

Under gdb, hart 0 at the first instruction of scheduler shows sie = 0x220. Decode it.

Value: 0x220

STIE (bit 5)SEIE (bit 9)SSIE (bit 1)Will hart 0 take a pending timer interrupt right now?

9solidPut in order

Hart 2 is idle in its scheduler when the disk finishes a read. Put the events in order, from the device to the moment the PLIC may forward the disk’s interrupts again.

Hart 2 executes intr_on() in its scheduler and takes the trap at stvec, which is kernelvec
plic_claim reads hart 2’s claim register and gets 1
virtio_disk_intr acknowledges the device, sets b->disk = 0 and calls wakeup
kerneltrap calls devintr, which sees scause 0x8000000000000009
plic_complete(1) writes 1 back to the claim register
The disk raises interrupt source 1; the PLIC marks it pending and signals the harts (SEIP)

10warm-upClick the line

In clockintr, click the line that acknowledges the timer interrupt, so that it stops being pending.

kernel/trap.c

166void
167clockintr()
168{
169  if (cpuid() == 0) {
170    acquire(&tickslock);
171    ticks++;
172    wakeup(&ticks);
173    release(&tickslock);
174  }
175
176  // ask for the next timer interrupt. this also clears
177  // the interrupt request. 1000000 is about a tenth
178  // of a second.
179  w_stimecmp(r_time() + 1000000);
180}

Your pick: none yet (click a line in the code)

11solidClick the line

Two harts trap for the same disk interrupt. Hart 2 claims it; hart 1’s claim returns 0. In devintr, click the line that keeps hart 1 from sending a completion to the PLIC.

kernel/trap.c

187int
188devintr()
189{
uint64 scause = r_scause();
191
if (scause == 0x8000000000000009L) {
  // this is a supervisor external interrupt, via PLIC.
194
  // irq indicates which device interrupted.
  int irq = plic_claim();
197
  if (irq == UART0_IRQ) {
    uartintr();
  } else if (irq == VIRTIO0_IRQ) {
    virtio_disk_intr();
  } else if (irq) {
    printk("unexpected interrupt irq=%d\n", irq);
  }
205
  // the PLIC allows each device to raise at most one
  // interrupt at a time; tell the PLIC the device is
  // now allowed to interrupt again.
  if (irq)
    plic_complete(irq);
211
  return 1;
} else if (scause == 0x8000000000000005L) {
  // timer interrupt.
  clockintr();
  return 2;
} else {
  return 0;
}
220}

Your pick: none yet (click a line in the code)

12deepChoose one

Hart 1 is idle in its scheduler and traps for a disk interrupt that hart 2 has already claimed, so hart 1’s plic_claim returns 0. What happens next on hart 1?

kernel/trap.c

137kerneltrap()
138{
int which_dev = 0;
uint64 sepc = r_sepc();
uint64 sstatus = r_sstatus();
uint64 scause = r_scause();
143
if ((sstatus & SSTATUS_SPP) == 0)
  panic("kerneltrap: not from supervisor mode");
if (intr_get() != 0)
  panic("kerneltrap: interrupts enabled");
148
if ((which_dev = devintr()) == 0) {
  // interrupt or trap from an unknown source
  printk("scause=0x%lx sepc=0x%lx stval=0x%lx\n", scause, r_sepc(),
         r_stval());
  panic("kerneltrap");
}
155
// give up the CPU if this is a timer interrupt.
if (which_dev == 2 && myproc() != 0)
  yield();
159
// the yield() may have caused some traps to occur,
// so restore trap registers for use by kernelvec.S's sepc instruction.
w_sepc(sepc);
w_sstatus(sstatus);
164}

devintr calls plic_complete(0) to tell the PLIC it saw nothing.devintr returns 1, so kerneltrap neither panics nor yields; it restores sepc and sstatus and returns into the scheduler.devintr returns 0, which kerneltrap treats as harmless.devintr returns 0, and kerneltrap panics with “kerneltrap”.

13solidTrue or false, and why

True or false: while a hart is executing user code, a pending supervisor timer interrupt (with sie.STIE set) is taken even if sstatus.SIE happened to be 0.

TrueFalse

Why?

Because prepare_return sets SPIE and sret copies it into SIE, so SIE is never 0 in user mode.RISC-V treats interrupts for a more privileged mode than the current one as always globally enabled; SIE matters only while the hart runs in S-mode.Because the timer bypasses every enable bit, including sie.STIE.

14warm-upChoose one

clockintr increments ticks only when cpuid() == 0. What is the reason?

kernel/trap.c

166void
167clockintr()
168{
169  if (cpuid() == 0) {
170    acquire(&tickslock);
171    ticks++;
172    wakeup(&ticks);
173    release(&tickslock);
174  }
175
176  // ask for the next timer interrupt. this also clears
177  // the interrupt request. 1000000 is about a tenth
178  // of a second.
179  w_stimecmp(r_time() + 1000000);
180}

Only hart 0 receives timer interrupts.To avoid deadlock between harts that take tickslock at the same time.Only hart 0 is allowed to acquire tickslock.So that ticks advances at one hart’s rate (about 10 per second) no matter how many harts there are.

15deepChoose one

Suppose clockintr were changed so that only hart 0 executes line 179 (w_stimecmp(...)), on the theory that only hart 0’s ticks matter. What happens to hart 1 after its first timer interrupt?

kernel/trap.c

166void
167clockintr()
168{
169  if (cpuid() == 0) {
170    acquire(&tickslock);
171    ticks++;
172    wakeup(&ticks);
173    release(&tickslock);
174  }
175
176  // ask for the next timer interrupt. this also clears
177  // the interrupt request. 1000000 is about a tenth
178  // of a second.
179  w_stimecmp(r_time() + 1000000);
180}

Nothing changes, because stimecmp is a single shared register that hart 0 re-arms for everyone.ticks stops advancing.Its timer interrupt stays pending, so it traps again as soon as it returns to user mode or opens its scheduler window; no user instruction ever runs on hart 1 again.Hart 1 gets no more timer interrupts, so programs on it can no longer be preempted.

16deepFill in the machine state

Hart 2 was idle in its scheduler when a disk interrupt arrived. It is now in virtio_disk_intr and has just executed line 303, acquire(&disk.vdisk_lock). What is the state of hart 2?

kernel/virtio_disk.c

300void
301virtio_disk_intr()
302{
303  acquire(&disk.vdisk_lock);
304
305  // the device won't raise another interrupt until we tell it
306  // we've seen this interrupt, which the following line does.
307  // this may race with the device writing new entries to
308  // the "used" ring, in which case we may process the new
309  // completion entries in this interrupt, and have nothing to do
310  // in the next interrupt, which is harmless.
311  *R(VIRTIO_MMIO_INTERRUPT_ACK) = *R(VIRTIO_MMIO_INTERRUPT_STATUS) & 0x3;
312
313  io_fence();

Privilege modeActive stackPage table (satp)Interrupts (sstatus.SIE)noffintena

17solidMatch the pairs

Match each register with its role in taking an interrupt.

stimecmpsiesstatus.SIEstvecscausethe PLIC claim register

18solidChoose one

When its scan finds nothing to run, the scheduler executes wfi at line 467 with interrupts off (line 442 turned them off). Why doesn’t the hart sleep through the interrupt that should wake it?

kernel/proc.c

for (;;) {
  // The most recent process to run may have had interrupts
  // turned off; enable them to avoid a deadlock if all
  // processes are waiting. Then turn them back off
  // to avoid a possible race between an interrupt
  // and wfi.
  intr_on();
  intr_off();
443
  int found = 0;
  for (p = proc; p < &proc[NPROC]; p++) {
    acquire(&p->lock);
    if (p->state == RUNNABLE) {
      // Switch to chosen process.  It is the process's job
      // to release its lock and then reacquire it
      // before jumping back to us.
      p->state = RUNNING;
      c->proc = p;
      swtch(&c->context, &p->context);
454
      // Don't re-enable interrupts on release.
      mycpu()->intena = 0;
457
      // Process is done running for now.
      // It should have changed its p->state before coming back.
      c->proc = 0;
      found = 1;
    }
    release(&p->lock);
  }
  if (found == 0) {
    // nothing to run; stop running on this core until an interrupt.
    asm volatile("wfi");
  }
}

The scheduler turns interrupts back on right before wfi.The interrupt is taken by machine mode, which wakes the hart.Another hart sends an inter-processor interrupt when a process becomes runnable.wfi resumes when an interrupt enabled in sie becomes pending, even if sstatus.SIE is 0; the loop then reaches intr_on() and the interrupt is taken there.

19deepChoose one

The scheduler turns interrupts on at line 441 and straight back off at line 442, and then scans the table and possibly executes wfi with interrupts off. What would go wrong if line 442 were removed, so interrupts stayed on during the scan and wfi?

kernel/proc.c

for (;;) {
  // The most recent process to run may have had interrupts
  // turned off; enable them to avoid a deadlock if all
  // processes are waiting. Then turn them back off
  // to avoid a possible race between an interrupt
  // and wfi.
  intr_on();
  intr_off();
443
  int found = 0;
  for (p = proc; p < &proc[NPROC]; p++) {
    acquire(&p->lock);
    if (p->state == RUNNABLE) {
      // Switch to chosen process.  It is the process's job
      // to release its lock and then reacquire it
      // before jumping back to us.
      p->state = RUNNING;
      c->proc = p;
      swtch(&c->context, &p->context);
454
      // Don't re-enable interrupts on release.
      mycpu()->intena = 0;
457
      // Process is done running for now.
      // It should have changed its p->state before coming back.
      c->proc = 0;
      found = 1;
    }
    release(&p->lock);
  }
  if (found == 0) {
    // nothing to run; stop running on this core until an interrupt.
    asm volatile("wfi");
  }
}

An interrupt during the scan would deadlock, because the scheduler holds a p->lock.An interrupt could arrive after the scan found nothing but before wfi, make a process runnable, and be fully handled; the hart would then sleep in wfi with work to do until some later interrupt.Nothing: interrupts are off during the scan anyway, because each p->lock is held.The scheduler would never take an interrupt at all.

20solidType a number

A process calls pause(5) just after a tick. Assume nothing else wakes or kills it, and that after each wakeup some scheduler runs it again before the next tick. How many times does sys_pause call sleep() (line 85)?

kernel/sysproc.c

67uint64
68sys_pause(void)
69{
int n;
uint ticks0;
72
argint(0, &n);
if (n < 0)
  n = 0;
acquire(&tickslock);
ticks0 = ticks;
while (ticks - ticks0 < n) {
  if (killed(myproc())) {
    release(&tickslock);
    return -1;
  }
  sleep_prepare(&ticks);
  release(&tickslock);
  sleep();
  acquire(&tickslock);
}
release(&tickslock);
return 0;
90}

decimal, 0x hex or 0b binary

21warm-upChoose one

Hart 0 started a disk read and its process went to sleep. When the disk finishes, which hart runs virtio_disk_intr?

The hart that started the read, since the request belongs to it.Always hart 0, because hart 0 ran plicinit.Whichever hart claims the interrupt first from the PLIC; it can be any hart with interrupts on.The PLIC picks harts in round-robin order.

22solidChoose one

virtio_disk_intr writes the device’s INTERRUPT_ACK register (line 311), and later devintr calls plic_complete. What would happen if xv6 called plic_complete without acknowledging the device?

kernel/virtio_disk.c

300void
301virtio_disk_intr()
302{
303  acquire(&disk.vdisk_lock);
304
305  // the device won't raise another interrupt until we tell it
306  // we've seen this interrupt, which the following line does.
307  // this may race with the device writing new entries to
308  // the "used" ring, in which case we may process the new
309  // completion entries in this interrupt, and have nothing to do
310  // in the next interrupt, which is harmless.
311  *R(VIRTIO_MMIO_INTERRUPT_ACK) = *R(VIRTIO_MMIO_INTERRUPT_STATUS) & 0x3;
312
313  io_fence();

The data in b->data would be lost.The PLIC would never forward another disk interrupt.The device would keep its interrupt line raised, so as soon as the PLIC was re-armed it would interrupt a hart again for work already done.Nothing: plic_complete acknowledges the device too.

23warm-upTrue or false, and why

True or false: if hart 1 has interrupts off when its timer fires, hart 0 may handle that timer interrupt instead.

TrueFalse

Why?

Each hart has its own stimecmp and its own pending timer bit; hart 1’s interrupt waits, pending, until hart 1 turns interrupts on.The PLIC routes timer interrupts to hart 0 only.Only hart 0 has a timer, so the question cannot arise.

24deepChoose all that apply

Which of these always run with interrupts off (sstatus.SIE = 0) on the hart executing them?

syscall dispatching a read system callmain on hart 0 while it runs kinit, kvminit and the other initializersclockintr, wherever it is called fromA user program’s codeusertrap handling a timer interrupt from user mode, on that thread, from its first line to its return

25deepChoose all that apply

Hypothetically, hart 0 runs for 0.35 seconds with interrupts off (say, a buggy driver spinning). Harts 1 and 2 run normally. Which statements are true?

Hart 1 notices that hart 0 is late and increments ticks instead.Hart 0’s timer interrupt stays pending and is taken once, when hart 0 turns interrupts back on.clockintr later adds the three missed ticks in a burst.ticks does not advance during the stretch, and advances by only one when interrupts come back on.Harts 1 and 2 keep taking their own timer interrupts.