# Kernel Stack Trampoline Fix — riscv64 **Date**: 2026-08-02 **Branch**: `master` **Commit**: `7366275` --- ## Problem The riscv64 kernel crashed partway through VM bootstrap during the 2026-08-01 DoE campaign run, well after amd64 and aarch64 had both booted cleanly to `ok>` in the same session. The crash was a wild jump: execution landed at an address nowhere inside the kernel's own linked image, then almost immediately faulted trying to load from a small, near-null address. ``` Registering FORTH-79 Standard word set... Registering FORTH-79 arithmetic words... FORTH-79 Standard word set registration complete *** EXCEPTION (riscv64) *** scause = 0x000000000000000d sepc = 0x00000000be27c3c2 stval = 0x00000000000074d8 ``` `scause = 0xd` is a supervisor-mode load page fault. `sepc` (~0xbe27c3c2) sat near the top of QEMU's 1 GB RAM window — nowhere close to the kernel's own `.text`/`.data`/`.bss` footprint (`0x400000`–`0xcdf9a0`, confirmed via `nm`/ `readelf` on the exact build that produced the log). `stval` (0x74d8, ~30 KB) stayed fixed across rebuilds even as `sepc` drifted — the signature of a corrupted return address rather than a fixed logic bug at one source line. ## Investigation Two dead ends were ruled out before the real cause surfaced: 1. **ELF relocation / buffer overlap in the split-build loader.** The codebase has a `!MONOLITHIC_BUILD` path (`elf_load_kernel()` in `elf_loader.c`) that loads `kernel.elf` from the ESP into a UEFI `AllocateAnyPages` buffer. This looked like a plausible overlap candidate at first, but `Makefile.starkernel` defaults to `MONOLITHIC=1`, meaning `kernel_main` is linked directly into the loader and that code path never runs. Confirmed by checking the built binary for the investigation's own diagnostic strings — absent, because the compilation unit was built with `-DMONOLITHIC_BUILD` and the code was preprocessed out. 2. **`RAW_LOG()` silently no-ops on every non-amd64 arch.** All of `load_kernel_from_esp()`'s narration (and an early ConOut-based diagnostic) never reached the captured serial log, on riscv64, in any build. `RAW_LOG` was defined only for `ARCH_AMD64`; everywhere else it expanded to `((void)0)`. This made straightforward diagnosis impossible until a real raw-UART writer was added for riscv64 (see Secondary Fix). With real output available, `console_println()` checkpoints were bisected through `vm_init_with_host()` and `sk_vm_bootstrap_parity()`. Every checkpoint inside `vm_init_with_host()` printed successfully, including the very last line of the function — but the checkpoint immediately after its *return* never appeared. Re-running with a trivial, unrelated local-variable addition shifted the crash earlier and changed the exception type entirely (from a load fault to an instruction-fetch fault at `sepc == stval`, i.e. the CPU tried to *execute* an unmapped address). A crash whose location and even category shift under unrelated code changes is the standard signature of stack corruption, not a fixed bug at a specific call site. ## Root Cause `kernel_main.c` documented the gap directly, once read closely: > On amd64, `kernel_entry.S` switches the stack from UEFI's default to a > 2 MiB zero-initialised BSS stack and tail-calls this function as > `kernel_main_impl`. On aarch64 and riscv64 the assembly trampoline is not > yet implemented and the UEFI loader calls `kernel_main` directly. amd64 has carried a dedicated stack-switch trampoline (`src/starkernel/arch/amd64/kernel_entry.S`) since early on, with its own comment explaining why: *"UEFI hands control to kernel_main() running on a small UEFI-provided stack (typically 128 KB–256 KB). The FORTH interpreter + DOE experiment loop can easily exceed that depth."* riscv64 never had the equivalent. It ran the entire VM bootstrap — 27 chained word-registration modules, physics/SSM init, Tripod capsule birth — directly on whatever small stack EDK2's riscv64 firmware handed the loader, overflowing it and corrupting a return address. aarch64 has the same theoretical exposure but happens not to trip it, apparently because its firmware's default boot stack is larger — incidental, not a guarantee. ## Fix **`src/starkernel/arch/riscv64/kernel_entry.S`** (new) — RISC-V stack-switch trampoline mirroring amd64's: ```asm .section .bss .align 4 .global g_kernel_stack g_kernel_stack: .space 0x200000 g_kernel_stack_top: .global g_kernel_stack_top .section .text .extern kernel_main_impl .global kernel_main kernel_main: ld t0, BOOT_INFO_KERNEL_STACK_BASE_OFFSET(a0) bnez t0, .Ldynamic_stack la t0, g_kernel_stack_top j .Lstack_ready .Ldynamic_stack: ld t1, BOOT_INFO_KERNEL_STACK_SIZE_OFFSET(a0) add t0, t0, t1 .Lstack_ready: andi t0, t0, -16 mv sp, t0 mv s0, zero tail kernel_main_impl ``` **`src/starkernel/kernel_main.c`** — riscv64 now builds `kernel_main_impl` (invoked via the trampoline) instead of `kernel_main` directly, same pattern as amd64: ```c #if defined(__x86_64__) || defined(__riscv) void kernel_main_impl(BootInfo *boot_info) { #else void kernel_main(BootInfo *boot_info) { #endif ``` **`Makefile.starkernel`** — wires the new file into the riscv64 loader build: ```makefile ifeq ($(ARCH),riscv64) LOADER_ASM += $(KERNEL_SRC)/arch/$(ARCH)/kernel_entry.S endif ``` ## Secondary Fix `RAW_LOG()` in `src/starkernel/boot/uefi_loader.c` was a no-op on every arch except amd64, silencing all of the loader's own diagnostic messages on riscv64 (and aarch64). Added a real raw-UART writer for riscv64, targeting QEMU virt's 16550-compatible `uart8250` at MMIO `0x10000000` (matches `Domain0 Region03` in the OpenSBI boot banner) — the same byte-polling protocol as the existing amd64 port-I/O version, just MMIO instead of `outb`/ `inb`. Existing and future `RAW_LOG()` call sites in the loader now actually produce output on riscv64. ## Result | Criterion | Before | After | |---|---|---| | `sepc` after word registration | Wild address outside kernel image (`0xbe27c3c2`-ish, drifting per build) | N/A — no longer occurs | | Exception | Load or instruction-fetch page fault, non-deterministic between builds | None | | `[Hera] ok>` (riscv64) | Never reached | ✅ reached, Tripod fleet (Hera/Hermes/Artemis) runs to completion | | `PASS: E2E msg flow` | Never logged | ✅ | | `PARITY:MAMA_INIT` / `Init: Mama birth OK` | Never logged | ✅ | | Loader `RAW_LOG()` output on riscv64 | Silent no-op | ✅ visible in serial log | ## Acceptance Testing All three architectures must boot to `ok>`, run the Tripod fleet, and reach `[Hera] ok>`. Per repo convention, run in this exact order, one QEMU instance at a time, always `clean` before `qemu`: ```bash make -f Makefile.starkernel ARCH=amd64 clean qemu make -f Makefile.starkernel ARCH=aarch64 clean qemu make -f Makefile.starkernel ARCH=riscv64 clean qemu ``` Verify each log: ```bash grep "ok>" logs///qemu--*.log grep "PASS: E2E msg flow" logs///qemu--*.log grep "PARITY:MAMA_INIT" logs///qemu--*.log grep "EXCEPTION" logs///qemu--*.log # must be empty ``` All three architectures confirmed clean on 2026-08-02 (see `logs/20260802-064050/riscv64/`, `logs/20260802-064424/amd64/`, `logs/20260802-064656/aarch64/`, committed alongside this fix).