Add FENCE word (SDK v1.9.0 scoping); fix severe pre-existing FORGET use-after-free
FENCE ( -- ) exposes the dict_fence_latest/dict_fence_here state FORGET already honored internally, letting callers (e.g. a future SDK capsule) raise the boundary after loading their own content -- no new VM fields, no policy logic beyond exposing existing state. Writing a direct test for it surfaced a real, severe, pre-existing bug in FORGET's relink logic, unrelated to FENCE itself and reproducible with the original boot-time fence alone: - Forgetting the single newest word incorrectly destroyed every other word back to the fence too, not just the target. - Forgetting an older word (correctly cascading to remove newer words too, per FORTH-79 semantics) crashed with SIGSEGV. Root cause: the relink code's target_prev pointer was, by construction, always inside the range the preceding loop had just freed whenever target wasn't vm->latest -- so writing through it was a use-after-free every time that branch executed. Fixed by removing the target_prev tracking and both branches entirely; vm->latest unconditionally becomes target_next (target's own captured, still-valid link) after the free loop, correct in every case. Added a FENCE test suite to dictionary_manipulation_words_test.c (Module 14) including the exact regression case (forgetting the newest word must not disturb an older one). Verified zero warnings and identical POST/dict_hash results across all three kernel architectures. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
6d8b48f100
commit
4e7dcdf889
+50
@@ -1752,3 +1752,53 @@ Block range for `capsules/sdk.4th`, whenever it's written: `5200+` (clear of `tu
|
||||
once scoped: three-arch boot to `zuse)ok>`, POST green, `mkcapsule --lint` clean, HOWTOs
|
||||
present for everything shipped — matching every other acceptance bar in this document, nothing
|
||||
new invented for this specific release.
|
||||
|
||||
**Decisions made 2026-08-19 (user's calls, per the open questions above):** SDK vocabulary
|
||||
re-exports the cookbook (`turtle.4th` + `doe.4th`) plus `FENCE`; load model is REPL-only
|
||||
(`S" sdk.4th" EXEC`, matching `turtle.4th`'s own precedent, not wired into `init.4th`);
|
||||
kernel-only is explicitly accepted (the SDK is a kernel-side developer surface, hosted builds
|
||||
remain compile-sanity-only). Version string and `turtle.4th`'s unconfirmed rendering remain
|
||||
open, not decided here.
|
||||
|
||||
**`FENCE ( -- )` implemented, and a real, severe pre-existing bug in `FORGET` found and fixed
|
||||
in the process — not a design bug in the new word, a bug in code this session didn't touch
|
||||
until now.** Added `dictionary_word_fence()` right next to `FORGET` in `defining_words.c`
|
||||
(registered alongside it) — it does exactly what the scoping above proposed: raises
|
||||
`dict_fence_latest`/`dict_fence_here` to the current dictionary top, nothing more. While
|
||||
writing a direct reproduction test for it (three plain `CREATE`d words, forget the newest,
|
||||
confirm the older one survives), the older one did *not* survive — `UNKNOWN WORD` — with no
|
||||
`FENCE` call involved at all, using only the original boot-time fence. Traced to
|
||||
`defining_words.c`'s `FORGET` relink logic (`~653-660`, pre-fix): when the target being
|
||||
forgotten is `vm->latest` itself (the single newest word), the code branches on
|
||||
`target_prev == NULL` and — reading that as "nothing exists between `vm->latest` and the
|
||||
fence" — resets `vm->latest = vm->dict_fence_latest` directly, discarding *everything* back to
|
||||
the fence instead of just the one targeted word. That reasoning was simply wrong:
|
||||
`target_prev == NULL` only means target is newest, not that it's adjacent to the fence.
|
||||
|
||||
**Escalated on a second reproduction, past the first (narrower) framing — this is a
|
||||
use-after-free, not just an off-by-one.** Testing the *other* direction (forget the *oldest*
|
||||
of three words, which per FORTH-79 "target and newer" semantics should legitimately remove all
|
||||
three) produced a **SIGSEGV**, not wrong output. Root cause: the preceding free loop already
|
||||
frees every entry from `vm->latest` down through target inclusive — which includes
|
||||
`target_prev` (by construction, the search loop's `prev` pointer is always the entry
|
||||
immediately newer than target, and thus always inside that just-freed range whenever target
|
||||
isn't `vm->latest` itself). The `if (target_prev) { target_prev->link = ...; vm->latest =
|
||||
target_prev; }` branch was therefore *always* operating on already-freed memory when it ran at
|
||||
all — the `target_prev == NULL` branch (discussed above) merely hid this by taking the *other*
|
||||
path instead, in the one case where it happened not to matter. Fix: deleted the `target_prev`
|
||||
tracking and both branches entirely; `vm->latest` unconditionally becomes `target_next`
|
||||
(target's own captured `->link`, valid and outside the freed range) after the free loop —
|
||||
correct whether target was latest, oldest, or in the middle, and correct whether it was the
|
||||
only survivor-adjacent entry or the fence itself. Verified this collapses cleanly to the
|
||||
already-correct fence-boundary-rejection case too (`FORGET DUP` against the original boot
|
||||
fence still correctly refuses, no crash, dictionary intact afterward).
|
||||
|
||||
Added a `FENCE` `WordTestSuite` entry to `dictionary_manipulation_words_test.c` right after
|
||||
`FORGET`'s own (Module 14) — three cases: forgetting a post-`FENCE` word succeeds, forgetting a
|
||||
pre-`FENCE` word is refused, and `forget_latest_keeps_predecessor` — the exact regression this
|
||||
fix addresses, encoded as a permanent POST case so it can't silently regress again. Verified:
|
||||
zero build warnings hosted and on all three kernel architectures; hosted POST 1009→1012 (+3,
|
||||
matching the new test count exactly), 0 failed, 0 errors; three-arch kernel boot identical
|
||||
(1012/0/0 on amd64/aarch64/riscv64), `dict_hash` matches exactly across all three (changed from
|
||||
the pre-`FENCE` baseline as expected — a new C word legitimately changes the dictionary hash;
|
||||
cross-arch agreement is what's being checked, not stability against the prior baseline).
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# Capsule Block Manifest — Auto-generated
|
||||
<!-- Generated by mkcapsule --manifest 2026-08-19T05:50:05Z -->
|
||||
<!-- Generated by mkcapsule --manifest 2026-08-19T09:54:15Z -->
|
||||
<!-- DO NOT EDIT — re-run mkcapsule --manifest to refresh. -->
|
||||
<!-- Hand-written justifications and immutability notes live -->
|
||||
<!-- in MANIFEST.md alongside this auto-generated index. -->
|
||||
|
||||
Binary file not shown.
+20909
-20692
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
Binary file not shown.
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -70,6 +70,34 @@ static WordTestSuite dict_manip_word_suites[] = {
|
||||
3, {0}
|
||||
},
|
||||
|
||||
{
|
||||
/* FENCE ( -- ): raises the FORGET boundary to the current dictionary
|
||||
* top. Added alongside a real bug fix in FORGET itself, caught while
|
||||
* building this word -- forgetting the single newest word used to
|
||||
* incorrectly destroy every other word back to the fence too (a
|
||||
* use-after-free in the relink logic, not just an off-by-one), a
|
||||
* SIGSEGV in the worst case. "forget_latest_keeps_predecessor" below
|
||||
* is exactly the regression case that fix addresses. */
|
||||
"FENCE", {
|
||||
{
|
||||
"basic", "CREATE fnc1 FENCE CREATE fnc2 FORGET fnc2",
|
||||
"Should forget a word defined after FENCE", TEST_NORMAL, 0, 1, {0}
|
||||
},
|
||||
{
|
||||
"protects_older", "CREATE fnc3 CREATE fnc4 FENCE FORGET fnc3",
|
||||
"Should refuse to forget a word defined before FENCE", TEST_ERROR_CASE, 1, 1, {0}
|
||||
},
|
||||
{
|
||||
"forget_latest_keeps_predecessor",
|
||||
"CREATE fnc5 FENCE CREATE fnc6 CREATE fnc7 FORGET fnc7 fnc6 DROP",
|
||||
"Forgetting the newest word must not disturb an older one still above FENCE",
|
||||
TEST_NORMAL, 0, 1, {0}
|
||||
},
|
||||
{NULL, NULL, NULL, TEST_NORMAL, 0, 0, {0}}
|
||||
},
|
||||
3, {0}
|
||||
},
|
||||
|
||||
{
|
||||
"IMMEDIATE", {
|
||||
{"basic", ": test3 42 ; IMMEDIATE test3 . CR", "Should execute immediately", TEST_NORMAL, 0, 1, {0}},
|
||||
|
||||
@@ -76,6 +76,7 @@ static void defining_word_literal(VM * vm);
|
||||
static void defining_word_does(VM * vm);
|
||||
static void defining_word_immediate(VM * vm);
|
||||
static void dictionary_word_forget(VM * vm);
|
||||
static void dictionary_word_fence(VM * vm);
|
||||
|
||||
/* ───────────────────────────── Runtimes ───────────────────────────── */
|
||||
|
||||
@@ -579,18 +580,16 @@ static void dictionary_word_forget(VM *vm) {
|
||||
int forget_failed = 0;
|
||||
sf_mutex_lock(&vm->dict_lock);
|
||||
|
||||
/* Find target and its previous entry */
|
||||
DictEntry *prev = NULL, *e = vm->latest;
|
||||
DictEntry *target = NULL, *target_prev = NULL;
|
||||
/* Find target */
|
||||
DictEntry *e = vm->latest;
|
||||
DictEntry *target = NULL;
|
||||
while (e) {
|
||||
if (!(e->flags & WORD_HIDDEN) &&
|
||||
e->name_len == (uint8_t) nlen &&
|
||||
memcmp(e->name, namebuf, (size_t) nlen) == 0) {
|
||||
target = e;
|
||||
target_prev = prev;
|
||||
break;
|
||||
}
|
||||
prev = e;
|
||||
e = e->link;
|
||||
}
|
||||
if (!target) {
|
||||
@@ -649,14 +648,17 @@ static void dictionary_word_forget(VM *vm) {
|
||||
e = next;
|
||||
}
|
||||
|
||||
/* Relink: if target_prev exists, point it past the freed chain */
|
||||
if (target_prev) {
|
||||
target_prev->link = target_next; /* Skip freed entries */
|
||||
vm->latest = target_prev; /* Latest is now the entry before target */
|
||||
} else {
|
||||
/* No prev means target was latest - reset to fence */
|
||||
vm->latest = vm->dict_fence_latest;
|
||||
}
|
||||
/* Relink: FORGET removes target and everything newer (closer to
|
||||
* vm->latest) -- the free loop above just freed every entry from
|
||||
* vm->latest through target inclusive, which by definition includes
|
||||
* any entry between them. There is nothing left to "skip past" --
|
||||
* target_next (target's own surviving link) is always the correct
|
||||
* new head, whether or not target was vm->latest itself. (Previously
|
||||
* this branched on whether an entry existed between vm->latest and
|
||||
* target, and relinked through it -- but that entry was always inside
|
||||
* the just-freed range, making it a use-after-free on every FORGET of
|
||||
* a non-latest word.) */
|
||||
vm->latest = target_next;
|
||||
|
||||
vm->here = (size_t) new_here;
|
||||
|
||||
@@ -669,6 +671,21 @@ forget_exit:
|
||||
return;
|
||||
}
|
||||
|
||||
/* FENCE ( -- ) raise the FORGET boundary to the current dictionary top.
|
||||
* dict_fence_latest/dict_fence_here already exist and are already honored
|
||||
* by FORGET (set once at bootstrap, protecting the base wordset); this is
|
||||
* the first word exposing that state so callers can advance it further --
|
||||
* e.g. an SDK capsule calling FENCE after loading protects its own
|
||||
* definitions the same way the base wordset is already protected. */
|
||||
static void dictionary_word_fence(VM *vm) {
|
||||
if (!vm) return;
|
||||
sf_mutex_lock(&vm->dict_lock);
|
||||
vm->dict_fence_latest = vm->latest;
|
||||
vm->dict_fence_here = vm->here;
|
||||
sf_mutex_unlock(&vm->dict_lock);
|
||||
log_message(LOG_DEBUG, "FENCE: raised to HERE=%ld", (long) vm->dict_fence_here);
|
||||
}
|
||||
|
||||
/* DOES> — IMMEDIATE: finalize the defining word’s create-part and compile DOES>-body */
|
||||
static void defining_word_does(VM *vm) {
|
||||
if (!vm) return;
|
||||
@@ -755,6 +772,7 @@ void register_defining_words(VM *vm) {
|
||||
|
||||
/* Dictionary management */
|
||||
register_word(vm, "FORGET", dictionary_word_forget);
|
||||
register_word(vm, "FENCE", dictionary_word_fence);
|
||||
|
||||
/* Compile helpers — immediate */
|
||||
register_word(vm, "COMPILE", defining_word_compile);
|
||||
|
||||
Reference in New Issue
Block a user