fabric: UTF-8 decoder (DECODE-UTF8)
Punch list §25 item 4.3.6a complete. capsules/fabric.4th blocks 4918-4920: UTF8-SEQ-LEN/UTF8-CONT? verbatim from §27.6.2, DECODE-UTF8 dispatching to four UTF8-ASSEMBLE-N helpers (1/2/3/4-byte). Factored into helpers after the single-word draft hit mkcapsule's real 64-char x 16-line block limit. Verified live on amd64: a 6-byte test buffer (ASCII + deg-sign + em-dash) decoded to 65/176/8212 across three sequential calls, exact match against hand-computed codepoints, buffer fully consumed. Three-arch acceptance boot clean, Stadium conservation unaffected. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
82e948afe9
commit
24d6b9e20e
@@ -4426,7 +4426,7 @@ document and committing that amendment as its own item.*
|
||||
> unaffected by this item, as expected): amd64 (`logs/20260809-184959`), aarch64
|
||||
> (`logs/20260809-185037`), riscv64 (`logs/20260809-185127`).
|
||||
|
||||
- [ ] **4.3.6a — UTF-8 decoder.** `DECODE-UTF8`, composed in FORTH per the
|
||||
- [x] **4.3.6a — UTF-8 decoder.** `DECODE-UTF8`, composed in FORTH per the
|
||||
compose-in-FORTH-first rule — no existing UTF-8 decoding anywhere in this tree (checked).
|
||||
Must handle 1-byte and 2-byte sequences (ASCII, Latin-1) *and* 3-byte sequences (General
|
||||
Punctuation — confirmed load-bearing for v1, not future-i18n-only, since smart
|
||||
@@ -4435,6 +4435,36 @@ document and committing that amendment as its own item.*
|
||||
`—` U+2014) sequences, verified live against hand-computed expected codepoints. *Refs:*
|
||||
§27.6.2.
|
||||
|
||||
> **Done, 2026-08-09.** Blocks 4918–4920 in `capsules/fabric.4th` (lint-clean).
|
||||
> `UTF8-SEQ-LEN`/`UTF8-CONT?` verbatim from §27.6.2. `DECODE-UTF8 ( addr u -- codepoint
|
||||
> addr' u' )` dispatches to one of four `UTF8-ASSEMBLE-N` helper words (1/2/3/4-byte,
|
||||
> all four formulas implemented since the spec text lists all four as what the word "must
|
||||
> compute," even though only 1/2/3 are required by this item's own acceptance) — factored
|
||||
> out because the single-word draft hit `mkcapsule`'s real limit, discovered here: **64
|
||||
> chars × 16 lines per block**, not the looser 1024-byte framing used elsewhere in this
|
||||
> document. Same factoring pattern 4.3.3b already used for `LINE`/`LINE-SETUP`/etc.
|
||||
> Invalid lead bytes fall to `U+FFFD` (replacement character) advancing 1 byte, avoiding
|
||||
> an infinite loop on malformed input — not specified by §27.6.2, a minimal safety default
|
||||
> consistent with the word's own contract always returning a valid `(codepoint addr' u')`.
|
||||
>
|
||||
> Hand-verified the 2-byte and 3-byte formulas against the required test characters before
|
||||
> writing code: `°` U+00B0=176 via `0xC2 0xB0` (`(0xC2 AND 31) LSHIFT 6 OR (0xB0 AND 63)` =
|
||||
> `128 OR 48` = `176`); `—` U+2014=8212 via `0xE2 0x80 0x94` (`(0xE2 AND 15) LSHIFT 12 OR
|
||||
> (0x80 AND 63) LSHIFT 6 OR (0x94 AND 63)` = `8192 OR 0 OR 20` = `8212`) — both exact.
|
||||
>
|
||||
> **Verified live on amd64**, same temporary-probe pattern as 4.3.6 (built, run once,
|
||||
> reverted): a 6-byte test buffer (`'A' 0xC2 0xB0 0xE2 0x80 0x94`) decoded via three
|
||||
> sequential `DECODE-UTF8` calls, each printing its codepoint and re-feeding the returned
|
||||
> `(addr' u')` into the next call. Results: `65` (ASCII `A`), `176` (`°`), `8212` (`—`) —
|
||||
> exact match on all three, and the buffer's remaining length hit exactly `0` after the
|
||||
> third call, confirming the byte-consumption bookkeeping is correct, not just the
|
||||
> codepoint arithmetic.
|
||||
>
|
||||
> Three-architecture acceptance boot clean, Stadium conservation unchanged from item
|
||||
> 4.3.6's baseline (`resident_sum=43691 reservoir=21845 sum=65536`): amd64
|
||||
> (`logs/20260809-185758`), aarch64 (`logs/20260809-185835`), riscv64
|
||||
> (`logs/20260809-185925`).
|
||||
|
||||
- [ ] **4.3.6b — Codepoint → glyph dispatch.** Bucketed `CASE`/`OF`/`ENDOF` chain
|
||||
(`DISPATCH-DIGIT`/`-UPPER`/`-LOWER`/`-ASCII-PUNCT`/`-LATIN1`/`-GENPUNCT`) routed by
|
||||
`DISPATCH-GLYPH` via `WITHIN` range checks, per §27.6.3 — chosen over a flat xt-table
|
||||
|
||||
Reference in New Issue
Block a user