FABRIC.md: item 4.5d Finding 4 -- deep-traced, not yet root-caused
Computed the loader's true runtime relocation delta to correctly correlate fault addresses (the running image is starkernel_loader.efi, a relocated PE, not the separately-linked starkernel_kernel.elf assumed at first). Fault RIP decodes to log_message()'s entry -- coincidental, not causal, since it's called on nearly every HADES dispatch during word registration. Used QEMU's monitor for -d exec,int tracing. Late-start tracing (stop right before the danger zone to keep trace size down) failed twice -- the window between a detectable checkpoint and the crash is shorter than host-side reaction latency. Fell back to full-boot tracing from -S (~2.7GB per attempt, not committed). That trace shows an unremarkable, normal-looking repeating three-block loop immediately before the fault, then "Servicing hardware INT=0x20" (APIC_TIMER_VECTOR) with IDT already showing limit=0 at that instant. Ruled out a second illegitimate lidt call (only one call site exists anywhere, one-time M4 boot setup; searched the trace for any later execution of that address range and found none). Not yet established: the actual corrupting write. Documented two remaining explanations (earlier silent corruption vs. a genuine TCG artifact) and that pinpointing the exact instruction needs GDB-level single-stepping, a bigger tooling step than attempted this session. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
1c5e926c3f
commit
065ab50240
@@ -5692,7 +5692,64 @@ document and committing that amendment as its own item.*
|
||||
> Same cascade shape as before (`#GP` → `#GP` → double fault → triple fault → silent QEMU
|
||||
> exit), so this was very likely mis-diagnosed as "part of the same hang" before tracing
|
||||
> distinguished the two. `-O2` reverted; `muldiv64()`'s fix is kept (real, independently
|
||||
> verified). 4.5d cannot complete until Finding 4 is resolved.
|
||||
> verified).
|
||||
>
|
||||
> **Follow-up, 2026-08-11 — deeper tracing, not yet root-caused.** Computed the loader's
|
||||
> true runtime-vs-link-time relocation delta (printed `&arch_interrupts_init` at runtime,
|
||||
> diffed against its `nm` link-time address: delta `0x3d5b9000`) to correlate fault
|
||||
> addresses against the actual binary — necessary because the running image is
|
||||
> `starkernel_loader.efi` (a PE, `MONOLITHIC_BUILD`-embedded, genuinely relocated at UEFI
|
||||
> load time), not the separately-linked `starkernel_kernel.elf` first assumed; that
|
||||
> separate `starkernel_loader.elf` debug-symbol target (`Makefile.starkernel:688`) fails to
|
||||
> link (`R_X86_64_32S` relocation error) and isn't part of `all` — a pre-existing gap, not
|
||||
> touched.
|
||||
>
|
||||
> With the delta in hand, the fault RIP (`0x3d5cc0e0`/`0x3d5cc0e0`-ish across runs) decodes
|
||||
> to link address `0x130e0` — the very entry of `log_message()` (`vm/host/shim.c`). This is
|
||||
> almost certainly coincidental, not causal: `log_message()` is called on essentially every
|
||||
> HADES/ECW dispatch during word registration, so an async 100 Hz timer tick landing exactly
|
||||
> at its entry is unsurprising and doesn't implicate the function itself.
|
||||
>
|
||||
> Used QEMU's monitor to enable `-d exec,int` execution+exception tracing (the earlier
|
||||
> `-d int`-only session had already established the fault; this pass wanted the
|
||||
> instruction-by-instruction lead-up). Two attempts to start tracing late (right before the
|
||||
> danger zone, via the monitor's `stop`/`log`/`cont` sequence, to keep the trace small)
|
||||
> **both failed** — QEMU reached the crash and exited before the host-side script could
|
||||
> connect, meaning the window between a detectable serial checkpoint and the actual fault is
|
||||
> shorter than the host's reaction latency even under `-S` synchronization races. Fell back
|
||||
> to tracing from cold boot with `-S` + immediate `log exec,int` before any `cont` (reliable,
|
||||
> but expensive: ~2.7 GB / ~39M lines per attempt; both trace files deleted after use, not
|
||||
> committed).
|
||||
>
|
||||
> That full trace shows the CPU executing a small, tight, **three-block repeating loop**
|
||||
> (`0x3d5bd120` → `0x3d5bcad9` → `0x3d5bcd05`) many times immediately before the fault — a
|
||||
> completely unremarkable pattern consistent with normal per-word dictionary-registration
|
||||
> work, not a wild jump in progress. Immediately after: `Stopped execution of TB chain`,
|
||||
> `Servicing hardware INT=0x20` (`APIC_TIMER_VECTOR`), and then the `#GP` — with `IDT=` already
|
||||
> showing `limit=0` at that exact instant. Re-examined the error code (`e=0102`) in this
|
||||
> light: `IDT=1` (bit 1), index `0x102>>3=32` — consistent with *either* "descriptor 32
|
||||
> specifically is bad" *or* "the table's limit is exceeded for any index," and an
|
||||
> x86-64 IDT-limit violation reports the attempted vector number in the error code either
|
||||
> way, so this doesn't distinguish between "one corrupted descriptor" and "the whole table
|
||||
> became limit=0" — both remain live explanations.
|
||||
>
|
||||
> **Ruled out:** a second, illegitimate `lidt` call — confirmed only one call site in the
|
||||
> entire codebase (`interrupts.c:476`, one-time M4 boot setup), and searched the exec trace
|
||||
> for any *later* execution of `arch_interrupts_init()`'s address range — found none; the
|
||||
> only occurrences are the single legitimate boot-time cluster.
|
||||
> **Not yet established:** the actual corrupting write/instruction. The repeating loop
|
||||
> immediately preceding the fault looks unremarkable in the trace, which means either (a)
|
||||
> the corruption happened earlier still and IDTR/the IDT table sat silently wrong for a
|
||||
> while before the next timer tick exposed it (most likely, given nothing in the visible
|
||||
> lead-up looks like a wild jump), or (b) it's a genuine TCG emulation artifact rather than a
|
||||
> guest-code bug at all (not tested — would need a different accelerator or QEMU version to
|
||||
> rule in/out, and this project's methodology is TCG-only by design).
|
||||
>
|
||||
> **Where this leaves it:** pinpointing the exact corrupting instruction from here needs
|
||||
> either GDB-level single-stepping (QEMU's `-s -S` gdb stub + a matching cross-gdb, watching
|
||||
> the `idt[]` array and IDTR directly across the whole pre-fault window) or a fundamentally
|
||||
> different narrowing strategy — a bigger tooling step than the tracing done so far, and one
|
||||
> this session did not attempt. 4.5d cannot complete until Finding 4 is resolved.
|
||||
|
||||
- [ ] **4.5e — Three-arch acceptance boot with optimization enabled.** Depends on 4.5d. This
|
||||
is the actual gate, per CLAUDE.md's non-negotiable acceptance criteria — a kernel that has
|
||||
|
||||
Reference in New Issue
Block a user