ACL-TTL overhead: paired 18-cell measurement complete, FABRIC-2.md Section P
Full 18-cell paired campaign (9 arch/seed pairs x ACL disabled/enabled) complete: 8,640/8,640 rows, all 16 cfg values x 30 each in every cell, zero errors. Confirmed capsules/zuse.4th's ACL-ZUSE-BOOT self-pin bug (self-pin placed inside its own colon definition, causing a genuine forward-reference failure) is isolated from the core ACL enforcement mechanism -- verified via live VM state query on multiple cells that ACL-INIT-PRIMITIVES and EXEC/BYE pinning both complete correctly regardless. Result: +5.30% pooled overhead, +4.42% unweighted mean across the 9 pairs (sd 6.77%), paired t=2.043 (df=8) -- not significant at p<0.05. Documented honestly as a real positive trend that doesn't establish a precise percentage with confidence, given wall-clock timing's noise floor is comparable to the effect size -- unlike the original ACL-RWT campaign's VM-internal tick-counter methodology. A tighter measurement (more replications, or reading a VM-internal counter directly) is scoped as a next step, not attempted here. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
6a71f503a4
commit
6e7323b3b8
+64
-2
@@ -2123,5 +2123,67 @@ mechanism could never have executed. Not re-litigated or corrected in the origin
|
||||
**Naming decision for any future paired ACL-enabled/disabled measurement, ruled 2026-08-20:**
|
||||
use an accurate name (e.g. "ACL-TTL overhead") rather than continuing to call it "ACL-RWT" —
|
||||
the RWT mechanism no longer exists in the codebase at all, live or dead, so the name would
|
||||
describe nothing real going forward. The paired measurement itself (item 5.1/F.3's own
|
||||
remaining open item) is not started as of this entry.
|
||||
describe nothing real going forward.
|
||||
|
||||
## P. ACL-TTL overhead — paired 18-cell measurement, 2026-08-20
|
||||
|
||||
Full paired campaign: 9 (arch, seed) cells × {ACL disabled, ACL enabled} = 18 cells, all in
|
||||
one continuous sitting, fully randomized order (not blocked by ACL state or architecture —
|
||||
same rigor as Sections M/O, learned the hard way earlier this session), independent `clean`
|
||||
build per cell. All 18 cells: 480/480 rows, all 16 `cfg` values × 30 each, zero errors —
|
||||
8,640/8,640 rows total. Data: `experiments/bare_metal/runs/acl-ttl-overhead-20260820/`.
|
||||
|
||||
**A real, separate bug confirmed harmless before trusting any "enabled" cell's data.**
|
||||
`capsules/zuse.4th`'s `ACL-ZUSE-BOOT` places its own self-pin (`['] ACL-ZUSE-BOOT ACL-PIN`)
|
||||
*inside* its own colon-definition body instead of after the closing `;`, causing a genuine
|
||||
forward-reference failure at capsule-load time (`[CAPSULE][DEFER]` reports it, doesn't resolve
|
||||
it) — reproduced identically on every ACL-enabled `riscv64`/`aarch64` cell. Verified via live
|
||||
VM state query (`' EXEC ACL-MODE@ .` → `1`, `' DOE-WORK ACL-MODE@ .` → `0`) on multiple cells
|
||||
that `ACL-BOOT` itself completes correctly regardless — `ACL-INIT-PRIMITIVES` stamps the whole
|
||||
dictionary, `EXEC`/`BYE` correctly pinned `STRICT` — so this bug is fully isolated from the
|
||||
core enforcement mechanism being measured. Not fixed, flagged only.
|
||||
|
||||
**Methodology.** Overhead measured as host wall-clock time from the moment `EXEC-DOE` was
|
||||
injected (via the same `socat`-against-QEMU-serial-socket tooling built for Sections M/O) to
|
||||
the moment `DOE: complete` appeared in the log — bracketing done in this session, not read
|
||||
from any VM-internal counter. This is a real methodological limitation, not glossed over: the
|
||||
original ACL-RWT campaign (`+0.0054%–+0.0088%`, `.claude/CLAUDE.md`) used the VM's own
|
||||
heartbeat tick counter, immune to host-side timing noise; this measurement's noise floor
|
||||
(injection round-trip latency, host scheduling jitter, TCG's own known-noisy wall-clock
|
||||
behavior) is plausibly comparable in magnitude to the true ACL overhead itself. One cell
|
||||
(`aarch64`/13579/disabled) has a further caveat: a session-usage-limit pause landed inside its
|
||||
timing window, contaminating the direct stopwatch reading; the log file's own last-write mtime
|
||||
was substituted as a corrected proxy (`274.30s`, second-granularity rather than sub-second).
|
||||
|
||||
**Result:**
|
||||
|
||||
| arch | seed | disabled (s) | enabled (s) | overhead |
|
||||
|---------|-------|--------------|-------------|----------|
|
||||
| aarch64 | 12345 | 283.04 | 310.50 | +9.70% |
|
||||
| aarch64 | 13579 | 274.30\* | 317.06 | +15.59% |
|
||||
| aarch64 | 67890 | 280.10 | 288.43 | +2.97% |
|
||||
| amd64 | 12345 | 189.13 | 199.26 | +5.35% |
|
||||
| amd64 | 13579 | 191.72 | 200.52 | +4.59% |
|
||||
| amd64 | 67890 | 186.82 | 200.64 | +7.40% |
|
||||
| riscv64 | 12345 | 172.44 | 158.12 | −8.31% |
|
||||
| riscv64 | 13579 | 157.06 | 163.99 | +4.41% |
|
||||
| riscv64 | 67890 | 166.43 | 163.23 | −1.92% |
|
||||
|
||||
\* mtime-based, see caveat above.
|
||||
|
||||
Pooled totals: 1901.04s disabled, 2001.75s enabled, **+5.30% overall**. Unweighted mean across
|
||||
the 9 pairs: **+4.42% (sd 6.77%)**. Paired t-test: $t = 2.043$, $df = 8$ — **not significant**
|
||||
at $p < 0.05$ (two-tailed critical $t_{8} = 2.306$; this result falls short of it). `riscv64`
|
||||
shows negative overhead for 2 of its 3 seeds — the smallest-magnitude, fastest-completing
|
||||
architecture, where sub-second host-side jitter has the largest relative effect on a percentage
|
||||
computed from a ~150–170s baseline. `aarch64` shows the most consistent positive signal (all 3
|
||||
seeds positive, 3.0%–15.6%) and also has the longest per-cell runtime (~275–320s), diluting
|
||||
host-jitter's relative contribution the most.
|
||||
|
||||
**Honest conclusion: this measurement shows a real, positive trend consistent with ACL
|
||||
enforcement costing something, but does not by itself establish a precise overhead percentage
|
||||
with statistical confidence** — the wall-clock methodology's noise floor is too close to the
|
||||
effect size. A tighter measurement would need either (a) many more replications per cell to
|
||||
shrink the standard error, or (b) reading a VM-internal tick/cycle counter directly (matching
|
||||
the original campaign's own method) rather than host-side wall-clock bracketing. Neither
|
||||
attempted here; scoped as a natural next step, not undertaken in this pass.
|
||||
|
||||
Reference in New Issue
Block a user