FABRIC-2.md: full design for single-block relocation, persistence included
Complete design, not just the earlier flag: resolve_lbn() as the single choke point threaded through ten public entry points, LBN->LBN redirect mechanism (no new storage allocator needed, the LBN space is already unified across all backend-agnostic blkio_dev devices), VM window staleness solved for free by reusing the existing epoch mechanism from Milestone 2h's hot-detach fix, in-memory table shape, and on-disk persistence via two new fields carved from blk_volume_meta_t's existing padding (byte-compatible with old formatted volumes) using the same absolute-devblock I/O pattern the BAM already uses. Ownership of the persisted copy assigned to first_disk_slot() (already exists, already used for this exact "which device is canonical" question), with the multi-primary-device question for a future multi-SSD/cloud world explicitly punted rather than hand-waved. Noted the known, separate limitation: USB relocation targets aren't exercisable until WRITE(10) exists in the xHCI driver -- verification will use two already-writable devices instead. Policy (when to relocate, whether a target is validly owned) stays ACL's job, not this primitive's. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CXjAPTEKrgY2Mrk25KoLDn
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
aea7f85d34
commit
073dae4f56
+86
-14
@@ -4066,20 +4066,92 @@ something else. Implement the derivation function.~~
|
||||
*when* to relocate (capacity pressure, or a compudynamics heat/cold signal); migration
|
||||
itself is expected to be rare, not routine (identity+home-blocks drives "probably won't
|
||||
migrate too much" — SSD moves individual blocks to the drive only when required)
|
||||
- [ ] **Flagged 2026-08-25, not yet designed: `block_subsystem.c`'s LBN routing needs revision
|
||||
before single-block relocation is possible.** `lbn_to_slot()` today is pure
|
||||
contiguous-range routing — walks device slots checking `start_lbn <= lbn <
|
||||
start_lbn+user_blocks`, one device owns one unbroken range. Relocating an individual
|
||||
block to a different device while its LBN stays fixed (the whole point — FORTH code
|
||||
never sees the physical backing move) breaks that assumption outright: nothing today
|
||||
lets one LBN inside a device's declared range actually resolve to a *different* device.
|
||||
Direction sketched but not designed in session: a sparse relocation-exception table
|
||||
(LBN → actual backing device), consulted before the plain range walk — only relocated
|
||||
LBNs need an entry, so the common case (nothing ever relocated) pays zero added cost.
|
||||
Exact structure (per-device vs. global, storage format, interaction with the existing
|
||||
BAM/cache machinery) is the actual design work, still to do.
|
||||
- [ ] Implement the block-migration function itself (move one block's content + BAM entry
|
||||
between two attached devices) — depends on the routing revision above existing first
|
||||
**Designed 2026-08-25 — single-block relocation, full design (implementation follows in the
|
||||
same pass, not staged separately).**
|
||||
|
||||
`lbn_to_slot()` today is pure contiguous-range routing — walks device slots checking
|
||||
`start_lbn <= lbn < start_lbn+user_blocks`, one device owns one unbroken range. Relocating an
|
||||
individual block to a different device while its LBN stays fixed (the whole point — FORTH code
|
||||
never sees the physical backing move) breaks that assumption: nothing today lets one LBN inside
|
||||
a device's declared range resolve to a *different* device. Also confirmed this session:
|
||||
`block_subsystem.c` already treats the entire LBN space as one contiguous, backend-agnostic
|
||||
range across however many devices are attached (`g.head` is an unbounded chain, `blkio_dev`'s
|
||||
vtable already hides RAM/file/virtio/USB behind the same interface) — a future cloud-backed
|
||||
device is just another `blkio_dev`, nothing in the routing itself is hardcoded to today's
|
||||
device set. The relocation mechanism below is deliberately general (`any_lbn → any_other_lbn`)
|
||||
for exactly that reason, not scoped narrowly to "SSD ↔ one thumbdrive."
|
||||
|
||||
**Mechanism: LBN→LBN redirect, not a new storage allocator.** Because the LBN space is already
|
||||
unified, a relocation target isn't raw unaddressed storage — it's simply *another LBN* (one the
|
||||
owning identity already has, on whichever device it's being moved to). `resolve_lbn(lbn)` is a
|
||||
single new choke point, checked first thing inside every public entry point that takes a block
|
||||
number (`blk_get_buffer`, `blk_get_empty_buffer`, `blk_update`, `blk_flush`, `blk_is_allocated`,
|
||||
`blk_mark_allocated`, `blk_mark_free`, `blk_is_valid`, `blk_get_meta`, `blk_set_meta` — not
|
||||
`blk_allocate`, which only ever *produces* a block number, never consumes one). Everything
|
||||
downstream (BAM offset math, cache lookup, `lbn_to_slot()` itself) is completely unchanged —
|
||||
it just operates on whatever LBN `resolve_lbn()` handed back, with zero awareness that a
|
||||
substitution happened.
|
||||
|
||||
**VM window cache staleness — already solved, for free.** A block relocating out from under a
|
||||
VM's cached window slot is the exact same hazard Milestone 2h's hot-detach fix already covers:
|
||||
bump `blk_subsys_epoch()` when a relocation happens, and the existing `blk_vm_check_epoch()`
|
||||
invalidates any stale cache entries automatically. No new VM-side code.
|
||||
|
||||
**In-memory structure:** a small fixed-capacity array, `blk_reloc_entry_t { uint32_t home_lbn;
|
||||
uint32_t actual_lbn; }`, linear-scanned (matches this codebase's existing tolerance for small
|
||||
bounded scans — `BLK_VM_SLOTS`, `DISK_CACHE_SLOTS` are the same shape) — relocations are
|
||||
expected rare, not routine, so no hash table is warranted.
|
||||
|
||||
**Persistence, in the same pass (not deferred):** the table is serialized into a *new*
|
||||
reserved region of the on-disk volume header, between the BAM and the payload region — two new
|
||||
`uint32_t` fields, `reloc_start`/`reloc_devblocks`, carved out of `blk_volume_meta_t`'s existing
|
||||
padding (3916 bytes of slack today; this costs 8, leaving the struct's total size at exactly
|
||||
4096 bytes, unchanged — a pre-existing formatted volume's zeroed padding reads back as
|
||||
`reloc_devblocks=0`, i.e. "no reloc capacity," gracefully, no format-breaking migration
|
||||
required for old images). Fresh formats (`blk_compute_fresh_geometry()`) reserve exactly one
|
||||
devblock (4KB) for it — a 4-byte count prefix plus up to ~511 `{home_lbn, actual_lbn}` 8-byte
|
||||
pairs, far more headroom than "won't migrate too much" needs. Written/read the same way BAM
|
||||
already is (`bam_flush_to_disk`/`bam_sync_from_disk`'s own absolute-devblock-addressing
|
||||
pattern), not through the payload-relative `devblock4k_to_lba1k()` path.
|
||||
|
||||
**Which device owns the persisted copy:** the table is subsystem-global (an LBN can belong to
|
||||
any device), but it has to live *somewhere* durable. Reused `first_disk_slot()` — already
|
||||
exists, already used by `blk_get_volume_meta()`/`blk_set_volume_meta()` for exactly this "which
|
||||
device is the canonical one" question — as the owner. Loaded once, when that slot is first
|
||||
attached; if it's later detached, the table stays in memory (still valid — the LBNs it
|
||||
references may still resolve via other attached devices) but stops being persisted until
|
||||
another disk-backed device is attached and becomes the new `first_disk_slot()`. Explicitly
|
||||
punted: which device is "the" owner once multiple SSDs/HDDs/cloud devices coexist is a real
|
||||
open question for that future, not solved here — `first_disk_slot()`'s "whichever attached
|
||||
first" answer is a pragmatic default for today's one-SSD-plus-removables reality, not a general
|
||||
multi-primary design.
|
||||
|
||||
**Known, separately-tracked limitation this doesn't fix:** relocating a block *onto* a real USB
|
||||
thumbdrive isn't actually exercisable yet — `blkio_usb.c` is read-only (no SCSI WRITE(10) in
|
||||
the xHCI driver, flagged as still-open back in Milestone 2g/2h). The relocation mechanism
|
||||
itself will be verified against two devices that both already support writes (Artemis's
|
||||
virtio-blk disk and the volatile RAMDRIVE), proving the primitive works; the SSD→USB-thumbdrive
|
||||
scenario specifically stays blocked on that pre-existing gap, not a new one.
|
||||
|
||||
**Policy is explicitly out of scope here, by design.** `blk_subsys_relocate_block()` is a
|
||||
mechanical primitive only — copies content, updates BAM/exception-table state, persists.
|
||||
*Deciding* when a relocation should happen (capacity pressure, a compudynamics heat/cold
|
||||
signal) and *validating* that a target LBN is genuinely owned by the right identity are ACL's
|
||||
job, per this session's own direction — not something this function enforces on its own.
|
||||
|
||||
- [ ] Add `reloc_start`/`reloc_devblocks` to `blk_volume_meta_t`, adjust fresh-format geometry
|
||||
to reserve one devblock for the reloc table
|
||||
- [ ] Implement `resolve_lbn()` and thread it through the ten public entry points listed above
|
||||
- [ ] Implement `reloc_flush_to_disk()`/`reloc_load_from_disk()` (mirroring the BAM I/O
|
||||
functions' own shape), wired into `blk_subsys_attach_device()` (load once, from
|
||||
`first_disk_slot()`) and `blk_commit_format()` (zero the region on fresh format)
|
||||
- [ ] Implement `blk_subsys_relocate_block(home_lbn, target_lbn)`: copy content, free the
|
||||
source BAM entry, append the exception entry, bump `blk_subsys_epoch()`, flush to disk
|
||||
- [ ] Add a FORTH word exposing it (matching this project's "compose in FORTH first"
|
||||
convention — ACL/higher-level policy code needs a callable entry point)
|
||||
- [ ] Live-verify: relocate a block between two writable devices (Artemis disk ↔ RAMDRIVE),
|
||||
confirm content and the redirect both survive an abrupt-kill/reboot cycle (same
|
||||
methodology as Section V item 6's own persistence test)
|
||||
- [x] Implement the `sk_repl_idle()` body — the cheap "anything dirty? no? done" check
|
||||
(Section V confirmed this hook is empty and ready right now, doesn't even need
|
||||
Milestone 2 to be written, only to be *tested end to end*). **Done 2026-08-25**, see
|
||||
|
||||
Reference in New Issue
Block a user