// Moved from docs/src/architecture-internals/INIT_SYSTEM.adoc to docs/working/scratch/src/architecture-internals/INIT_SYSTEM.adoc on 2026-06-16 (docs reorg Phase 2) == StarForth INIT System :toc: left :toc-title: Contents :toclevels: 3 xref:../README.adoc[← Back to Documentation Index] === Overview StarForth features a deterministic initialization system that loads foundational Forth definitions from `+./conf/init.4th+` at startup. This system provides a clean separation between boot-time initialization and runtime operation, with automatic cleanup and protection mechanisms. === Architecture ==== Components [arabic] . *`+./conf/init.4th+`* - Version-controlled initialization file . *`+INIT+` word* - Core initialization primitive (starforth_words.c) . *`+(-+` comment word* - Metadata marker for init blocks . *Dictionary fence* - Protection mechanism for init words . *Block subsystem* - Transient block storage for init code ==== Design Principles * *Deterministic boot* - Same init.4th always produces same system state * *Transient initialization* - Init blocks execute then vanish (zeroed) * *Protected dictionary* - Init words cannot be FORGOTten * *No dependencies* - Runtime has no file dependencies after boot * *Platform agnostic* - Works with both filesystem (Linux) and ROMFS (L4Re) === Init.4th Format ==== File Structure [source,forth] ---- Block (- ) Block (- ) ---- ==== Example [source,forth] ---- Block 2048 (- Large Letter F ) : STAR 42 EMIT ; : STARS 0 DO STAR LOOP ; : MARGIN CR 30 SPACES ; : BLIP MARGIN STAR ; : BAR MARGIN 5 STARS ; : F BAR BLIP BAR BLIP BLIP ; Block 3000 (- Add Me to init.4th ) : E BAR BLIP BAR BLIP BAR ; Block 3001 (- Add Me to init.4th ) E F CR ---- ==== Format Rules [arabic] . *Block headers* - Each block starts with `+Block +` . *Metadata comments* - Each block should have a `+(-+` comment describing its purpose . *Sequential loading* - Blocks are loaded in file order (1, 2, 3…) regardless of original numbers . *LOAD rewriting* - References like `+2048 LOAD+` are automatically remapped to sequential numbers . *Newline preservation* - Newlines in blocks are preserved for proper parsing . *No `+-->+` in init* - Use separate blocks instead of `+-->+` for clarity === The INIT Word ==== Execution Flow .... 1. Read ./conf/init.4th (or ROMFS in L4Re) 2. Parse Block headers and build mapping table - Block 2048 → Sequential block 1 - Block 3000 → Sequential block 2 - Block 3001 → Sequential block 3 3. Copy block content sequentially - Rewrite LOAD references (2048 LOAD → 1 LOAD) - Preserve newlines for proper parsing 4. Execute blocks via LOAD (1 LOAD, 2 LOAD, 3 LOAD...) 5. Switch to FORTH vocabulary context 6. Zero all init blocks (1-N) for userspace 7. Set dictionary fence (main.c) .... ==== Stack Effect [source,forth] ---- INIT ( -- ) ---- ==== Error Handling * If init.4th cannot be opened → VM halts with error * If block allocation fails → VM halts with error * If block execution fails → VM halts with error * On any error, system cannot start REPL ==== Platform Support ===== Linux (IDE Mode) [source,c] ---- FILE *fp = fopen("./conf/init.4th", "r"); // Read, parse, execute ---- ===== L4Re (Production Mode) [source,c] ---- #ifdef L4RE_TARGET // TODO: Read from ROMFS // Defined in modules.list #endif ---- === The (- Comment Word ==== Purpose The `+(-+` word serves dual purposes: [arabic] . *Runtime* - Standard Forth comment (like `+(+` but with dash) . *Tooling* - Metadata marker for extraction/documentation tools ==== Syntax [source,forth] ---- (- ) ---- ==== Behavior * Consumes input from `+(-+` to the first unmatched `+)+` * Handles nested parentheses correctly * No stack effect * Logs at DEBUG level for tracing ==== Usage in init.4th [source,forth] ---- Block 2048 (- Large Letter F ) \ Metadata for tooling : STAR 42 EMIT ; \ Actual code ---- === Dictionary Fence Protection ==== Problem After init completes, all init words (F, E, STAR, BAR, etc.) are in the dictionary. If a user issues `+FORGET F+`, it would: [arabic] . Free F and all words defined after it . Leave E with broken references to BAR and BLIP . Cause segfault when E executes ==== Solution After INIT completes, main.c sets the dictionary fence: [source,c] ---- /* Set dictionary fence after INIT to protect foundational words from FORGET */ vm.dict_fence_latest = vm.latest; vm.dict_fence_here = vm.here; log_message(LOG_INFO, "Dictionary fence set - init words protected from FORGET"); ---- This prevents FORGET from removing any init words, ensuring system stability. ==== Effect [source,forth] ---- ok> FORGET F \ Silently fails - F is protected ok> F \ F still works - no crash ---- === Block Lifecycle ==== During INIT .... Blocks 1-N: [init code from init.4th] - Block 1: STAR, STARS, MARGIN, BLIP, BAR, F - Block 2: E - Block 3: E F CR .... ==== After INIT .... Blocks 1-N: [all zeros - ready for userspace] Dictionary: F, E, STAR, BAR, BLIP, MARGIN, STARS, INIT, (- ^ All protected by fence .... ==== Runtime .... Blocks 1-992: [available for userspace programs] - No init dependency - Clean slate for applications - Can be loaded, edited, saved .... === Startup Sequence ==== Full Boot Process .... 1. Parse command line arguments 2. Open block device (FILE or RAM backend) 3. Initialize VM (vm_init) 4. Initialize block subsystem (1MB RAM blocks 0-1023) 5. Register all FORTH-79 words 6. Run INIT word - Load init.4th - Execute blocks - Clean up 7. Set dictionary fence 8. Start REPL .... ==== Logging Example .... [08:30:26] INFO: Running system initialization (INIT)... [08:30:26] INFO: INIT: Starting system initialization from init.4th [08:30:26] DEBUG: INIT: Block mapping: 2048 -> 1 [08:30:26] DEBUG: INIT: Block mapping: 3000 -> 2 [08:30:26] DEBUG: INIT: Block mapping: 3001 -> 3 [08:30:26] INFO: INIT: Copied block content to block 1 (173 bytes) [08:30:26] INFO: INIT: Rewrote 2048 LOAD -> 1 LOAD [08:30:26] INFO: INIT: Copied block content to block 2 (68 bytes) [08:30:26] INFO: INIT: Copied block content to block 3 (7 bytes) [08:30:26] INFO: INIT: Loaded 3 blocks from init.4th [08:30:26] INFO: INIT: Executing initialization blocks... [08:30:26] DEBUG: INIT: Executing block 1 (LOAD) [08:30:26] DEBUG: INIT: Executing block 2 (LOAD) [08:30:26] DEBUG: INIT: Executing block 3 (LOAD) [08:30:26] INFO: INIT: Switching to FORTH vocabulary [08:30:26] INFO: INIT: Zeroing 3 init blocks for userspace use [08:30:26] DEBUG: INIT: Zeroed block 1 [08:30:26] DEBUG: INIT: Zeroed block 2 [08:30:26] DEBUG: INIT: Zeroed block 3 [08:30:26] INFO: INIT: System initialization complete - blocks freed, FORTH context active [08:30:26] INFO: Dictionary fence set - init words protected from FORGET [08:30:26] INFO: Starting Forth REPL .... === Development Workflow ==== IDE Mode (Linux) [arabic] . *Edit init.4th* - Modify `+./conf/init.4th+` directly . *Test* - Run `+./build/starforth+` to see results . *Commit* - Version control tracks plain text changes . *PR Review* - Reviewers see diffs of init.4th ==== Disk Image Workflow (Future) [arabic] . *Work in disk images* - Edit Forth code in `+./disks/*.img+` . *Mark blocks* - Use `+(-+` comments to mark blocks for extraction . *Extract* - Tools in `+./tools/+` scan images and update init.4th . *Review* - init.4th changes show up in git diff . *Commit* - Both disk images (LFS) and init.4th (git) are tracked ==== L4Re Deployment [arabic] . *Build* - init.4th is packed into ROMFS at build time . *modules.list* - Defines init.4th as a module . *Boot* - L4Re loads ROMFS, INIT reads from ROMFS . *Runtime* - No filesystem dependency === Implementation Details ==== LOAD Rewriting Algorithm [source,c] ---- // First pass: Build mapping table Block 2048 -> Sequential 1 Block 3000 -> Sequential 2 Block 3001 -> Sequential 3 // Second pass: Copy with rewriting while (copying block content) { if (found "NNNN LOAD" pattern) { lookup NNNN in mapping replace with mapped number write "M LOAD" to block } } ---- ==== Memory Layout .... VM Memory (5MB): ├─ Dictionary (grows up) │ ├─ System words (protected by fence) │ ├─ Init words (F, E, STAR... - protected by fence) │ └─ User words (can be FORGOTten) └─ HERE (current allocation point) Block RAM (1MB): ├─ Block 0: Reserved (volume metadata) ├─ Blocks 1-992: User blocks (zeroed after INIT) └─ Blocks 993-1023: Reserved External Disk (if attached): └─ Blocks 1024+: Persistent storage .... ==== Performance * *Init time*: <10ms typical (depends on init.4th size) * *Memory overhead*: Transient (blocks zeroed after init) * *Dictionary size*: Proportional to init.4th definitions * *No runtime cost*: Init happens once at boot === Best Practices ==== Writing init.4th [arabic] . *Keep it minimal* - Only foundational definitions . *Document blocks* - Always use `+(-+` comments . *Avoid cross-references* - Define dependencies first . *Test incrementally* - Add blocks one at a time . *No side effects* - Init should define words, not execute actions (except testing in last block) ==== Managing Dependencies [source,forth] ---- \ GOOD: Dependencies defined first Block 1 (- Foundation words ) : HELPER1 ... ; : HELPER2 ... ; Block 2 (- Uses helpers ) : MAIN HELPER1 HELPER2 ; \ BAD: Forward reference Block 1 (- Uses undefined word ) : MAIN HELPER1 ; \ HELPER1 not defined yet! Block 2 (- Define helper ) : HELPER1 ... ; ---- ==== Avoiding `+-->+` [source,forth] ---- \ GOOD: Separate blocks Block 1 (- First part ) : FOO ... ; Block 2 (- Second part ) : BAR FOO ; \ BAD: Using --> in init Block 1 (- Spanning definition ) : BIGWORD part1 --> Block 2 part2 ; \ Problem: INIT loads blocks sequentially, --> causes block 2 to execute twice ---- === Troubleshooting ==== "`INIT: Failed to open ./conf/init.4th`" *Cause*: File missing or wrong working directory *Fix*: Ensure init.4th exists and run from project root ==== "`INIT: Error executing block N`" *Cause*: Syntax error or undefined word in block N *Fix*: Check block N content, ensure dependencies are defined first ==== "`UNKNOWN WORD: '`FOO`'`" *Cause*: FOO referenced before definition *Fix*: Reorder blocks so FOO is defined before use ==== Segfault after FORGET *Cause*: Attempted to FORGET init word (shouldn’t be possible after fence) *Fix*: Verify fence is set in main.c after INIT ==== Words missing after INIT *Cause*: LOAD not executing properly, or blocks zeroed too early *Fix*: Check LOAD implementation, verify zeroing happens after execution === Code References ==== INIT Word Implementation *File*: `+src/word_source/starforth_words.c:214-495+` Key sections: * Line 234-240: Open init.4th file * Line 272-305: First pass - build block mapping table * Line 307-418: Second pass - copy blocks with LOAD rewriting * Line 336-398: LOAD reference rewriting logic * Line 447-465: Execute blocks via LOAD * Line 468-481: Switch to FORTH vocabulary * Line 484-493: Zero init blocks ==== Dictionary Fence Setting *File*: `+src/main.c:488-491+` [source,c] ---- /* Set dictionary fence after INIT to protect foundational words from FORGET */ vm.dict_fence_latest = vm.latest; vm.dict_fence_here = vm.here; log_message(LOG_INFO, "Dictionary fence set - init words protected from FORGET"); ---- ==== FORGET Implementation *File*: `+src/word_source/defining_words.c:530-612+` Key section: * Line 559-571: Check if target is newer than fence (prevent forgetting protected words) * Line 586-606: Free entries and relink dictionary chain ==== LOAD Implementation *File*: `+src/word_source/block_words.c:191-213+` [source,c] ---- void block_word_load(VM *vm) { // ... get block buffer ... /* Interpret the block content as Forth source */ char block_text[1025]; memcpy(block_text, buf, 1024); block_text[1024] = '\0'; /* Interpret the block content */ vm_interpret(vm, block_text); } ---- ==== `+-->+` Implementation *File*: `+src/word_source/block_words.c:289-309+` Continues interpretation to next sequential block (SCR + 1). === Future Enhancements ==== Planned Features [arabic] . *Tool: extract-init* - Extract init.4th from disk images . *Tool: validate-init* - Syntax check init.4th . *Tool: merge-init* - Merge multiple init.4th files . *INIT options* - `+INIT-VERBOSE+`, `+INIT-DEBUG+` flags . *Hot reload* - Reload init.4th without restart (dev mode) . *Init profiles* - Multiple init files for different environments ==== L4Re Integration [source,c] ---- // Planned L4Re ROMFS support #ifdef L4RE_TARGET L4Re::Env::env()->get_cap("init.4th"); // Map dataspace, read content #endif ---- === References * FORTH-79 Standard (LOAD, BLOCK, LIST) * Block Storage Guide (docs/BLOCK_STORAGE_GUIDE.md) * L4Re Integration (docs/L4RE_INTEGRATION.md) * StarForth Architecture (docs/ARCHITECTURE.md) === See Also * `+src/word_source/starforth_words.c+` - INIT and (- implementation * `+src/word_source/block_words.c+` - LOAD, –>, THRU implementation * `+src/main.c+` - Startup sequence and fence setting * `+conf/init.4th+` - Example initialization file