545 lines
13 KiB
Plaintext
545 lines
13 KiB
Plaintext
// Moved from docs/src/architecture-internals/INIT_SYSTEM.adoc to docs/working/scratch/src/architecture-internals/INIT_SYSTEM.adoc on 2026-06-16 (docs reorg Phase 2)
|
||
== StarForth INIT System
|
||
:toc: left
|
||
:toc-title: Contents
|
||
:toclevels: 3
|
||
xref:../README.adoc[← Back to Documentation Index]
|
||
|
||
|
||
|
||
=== Overview
|
||
|
||
StarForth features a deterministic initialization system that loads
|
||
foundational Forth definitions from `+./conf/init.4th+` at startup. This
|
||
system provides a clean separation between boot-time initialization and
|
||
runtime operation, with automatic cleanup and protection mechanisms.
|
||
|
||
=== Architecture
|
||
|
||
==== Components
|
||
|
||
[arabic]
|
||
. *`+./conf/init.4th+`* - Version-controlled initialization file
|
||
. *`+INIT+` word* - Core initialization primitive (starforth_words.c)
|
||
. *`+(-+` comment word* - Metadata marker for init blocks
|
||
. *Dictionary fence* - Protection mechanism for init words
|
||
. *Block subsystem* - Transient block storage for init code
|
||
|
||
==== Design Principles
|
||
|
||
* *Deterministic boot* - Same init.4th always produces same system state
|
||
* *Transient initialization* - Init blocks execute then vanish (zeroed)
|
||
* *Protected dictionary* - Init words cannot be FORGOTten
|
||
* *No dependencies* - Runtime has no file dependencies after boot
|
||
* *Platform agnostic* - Works with both filesystem (Linux) and ROMFS
|
||
(L4Re)
|
||
|
||
=== Init.4th Format
|
||
|
||
==== File Structure
|
||
|
||
[source,forth]
|
||
----
|
||
Block <original-number>
|
||
(- <description> )
|
||
<forth-source-code>
|
||
|
||
Block <original-number>
|
||
(- <description> )
|
||
<forth-source-code>
|
||
----
|
||
|
||
==== Example
|
||
|
||
[source,forth]
|
||
----
|
||
Block 2048
|
||
(- Large Letter F )
|
||
: STAR 42 EMIT ;
|
||
: STARS 0 DO STAR LOOP ;
|
||
: MARGIN CR 30 SPACES ;
|
||
: BLIP MARGIN STAR ;
|
||
: BAR MARGIN 5 STARS ;
|
||
: F BAR BLIP BAR BLIP BLIP ;
|
||
|
||
Block 3000
|
||
(- Add Me to init.4th )
|
||
: E BAR BLIP BAR BLIP BAR ;
|
||
|
||
Block 3001
|
||
(- Add Me to init.4th )
|
||
E F CR
|
||
----
|
||
|
||
==== Format Rules
|
||
|
||
[arabic]
|
||
. *Block headers* - Each block starts with `+Block <number>+`
|
||
. *Metadata comments* - Each block should have a `+(-+` comment
|
||
describing its purpose
|
||
. *Sequential loading* - Blocks are loaded in file order (1, 2, 3…)
|
||
regardless of original numbers
|
||
. *LOAD rewriting* - References like `+2048 LOAD+` are automatically
|
||
remapped to sequential numbers
|
||
. *Newline preservation* - Newlines in blocks are preserved for proper
|
||
parsing
|
||
. *No `+-->+` in init* - Use separate blocks instead of `+-->+` for
|
||
clarity
|
||
|
||
=== The INIT Word
|
||
|
||
==== Execution Flow
|
||
|
||
....
|
||
1. Read ./conf/init.4th (or ROMFS in L4Re)
|
||
2. Parse Block headers and build mapping table
|
||
- Block 2048 → Sequential block 1
|
||
- Block 3000 → Sequential block 2
|
||
- Block 3001 → Sequential block 3
|
||
3. Copy block content sequentially
|
||
- Rewrite LOAD references (2048 LOAD → 1 LOAD)
|
||
- Preserve newlines for proper parsing
|
||
4. Execute blocks via LOAD (1 LOAD, 2 LOAD, 3 LOAD...)
|
||
5. Switch to FORTH vocabulary context
|
||
6. Zero all init blocks (1-N) for userspace
|
||
7. Set dictionary fence (main.c)
|
||
....
|
||
|
||
==== Stack Effect
|
||
|
||
[source,forth]
|
||
----
|
||
INIT ( -- )
|
||
----
|
||
|
||
==== Error Handling
|
||
|
||
* If init.4th cannot be opened → VM halts with error
|
||
* If block allocation fails → VM halts with error
|
||
* If block execution fails → VM halts with error
|
||
* On any error, system cannot start REPL
|
||
|
||
==== Platform Support
|
||
|
||
===== Linux (IDE Mode)
|
||
|
||
[source,c]
|
||
----
|
||
FILE *fp = fopen("./conf/init.4th", "r");
|
||
// Read, parse, execute
|
||
----
|
||
|
||
===== L4Re (Production Mode)
|
||
|
||
[source,c]
|
||
----
|
||
#ifdef L4RE_TARGET
|
||
// TODO: Read from ROMFS
|
||
// Defined in modules.list
|
||
#endif
|
||
----
|
||
|
||
=== The (- Comment Word
|
||
|
||
==== Purpose
|
||
|
||
The `+(-+` word serves dual purposes:
|
||
|
||
[arabic]
|
||
. *Runtime* - Standard Forth comment (like `+(+` but with dash)
|
||
. *Tooling* - Metadata marker for extraction/documentation tools
|
||
|
||
==== Syntax
|
||
|
||
[source,forth]
|
||
----
|
||
(- <text until closing paren> )
|
||
----
|
||
|
||
==== Behavior
|
||
|
||
* Consumes input from `+(-+` to the first unmatched `+)+`
|
||
* Handles nested parentheses correctly
|
||
* No stack effect
|
||
* Logs at DEBUG level for tracing
|
||
|
||
==== Usage in init.4th
|
||
|
||
[source,forth]
|
||
----
|
||
Block 2048
|
||
(- Large Letter F ) \ Metadata for tooling
|
||
: STAR 42 EMIT ; \ Actual code
|
||
----
|
||
|
||
=== Dictionary Fence Protection
|
||
|
||
==== Problem
|
||
|
||
After init completes, all init words (F, E, STAR, BAR, etc.) are in the
|
||
dictionary. If a user issues `+FORGET F+`, it would:
|
||
|
||
[arabic]
|
||
. Free F and all words defined after it
|
||
. Leave E with broken references to BAR and BLIP
|
||
. Cause segfault when E executes
|
||
|
||
==== Solution
|
||
|
||
After INIT completes, main.c sets the dictionary fence:
|
||
|
||
[source,c]
|
||
----
|
||
/* Set dictionary fence after INIT to protect foundational words from FORGET */
|
||
vm.dict_fence_latest = vm.latest;
|
||
vm.dict_fence_here = vm.here;
|
||
log_message(LOG_INFO, "Dictionary fence set - init words protected from FORGET");
|
||
----
|
||
|
||
This prevents FORGET from removing any init words, ensuring system
|
||
stability.
|
||
|
||
==== Effect
|
||
|
||
[source,forth]
|
||
----
|
||
ok> FORGET F
|
||
\ Silently fails - F is protected
|
||
ok> F
|
||
\ F still works - no crash
|
||
----
|
||
|
||
=== Block Lifecycle
|
||
|
||
==== During INIT
|
||
|
||
....
|
||
Blocks 1-N: [init code from init.4th]
|
||
- Block 1: STAR, STARS, MARGIN, BLIP, BAR, F
|
||
- Block 2: E
|
||
- Block 3: E F CR
|
||
....
|
||
|
||
==== After INIT
|
||
|
||
....
|
||
Blocks 1-N: [all zeros - ready for userspace]
|
||
Dictionary: F, E, STAR, BAR, BLIP, MARGIN, STARS, INIT, (-
|
||
^ All protected by fence
|
||
....
|
||
|
||
==== Runtime
|
||
|
||
....
|
||
Blocks 1-992: [available for userspace programs]
|
||
- No init dependency
|
||
- Clean slate for applications
|
||
- Can be loaded, edited, saved
|
||
....
|
||
|
||
=== Startup Sequence
|
||
|
||
==== Full Boot Process
|
||
|
||
....
|
||
1. Parse command line arguments
|
||
2. Open block device (FILE or RAM backend)
|
||
3. Initialize VM (vm_init)
|
||
4. Initialize block subsystem (1MB RAM blocks 0-1023)
|
||
5. Register all FORTH-79 words
|
||
6. Run INIT word
|
||
- Load init.4th
|
||
- Execute blocks
|
||
- Clean up
|
||
7. Set dictionary fence
|
||
8. Start REPL
|
||
....
|
||
|
||
==== Logging Example
|
||
|
||
....
|
||
[08:30:26] INFO: Running system initialization (INIT)...
|
||
[08:30:26] INFO: INIT: Starting system initialization from init.4th
|
||
[08:30:26] DEBUG: INIT: Block mapping: 2048 -> 1
|
||
[08:30:26] DEBUG: INIT: Block mapping: 3000 -> 2
|
||
[08:30:26] DEBUG: INIT: Block mapping: 3001 -> 3
|
||
[08:30:26] INFO: INIT: Copied block content to block 1 (173 bytes)
|
||
[08:30:26] INFO: INIT: Rewrote 2048 LOAD -> 1 LOAD
|
||
[08:30:26] INFO: INIT: Copied block content to block 2 (68 bytes)
|
||
[08:30:26] INFO: INIT: Copied block content to block 3 (7 bytes)
|
||
[08:30:26] INFO: INIT: Loaded 3 blocks from init.4th
|
||
[08:30:26] INFO: INIT: Executing initialization blocks...
|
||
[08:30:26] DEBUG: INIT: Executing block 1 (LOAD)
|
||
[08:30:26] DEBUG: INIT: Executing block 2 (LOAD)
|
||
[08:30:26] DEBUG: INIT: Executing block 3 (LOAD)
|
||
[08:30:26] INFO: INIT: Switching to FORTH vocabulary
|
||
[08:30:26] INFO: INIT: Zeroing 3 init blocks for userspace use
|
||
[08:30:26] DEBUG: INIT: Zeroed block 1
|
||
[08:30:26] DEBUG: INIT: Zeroed block 2
|
||
[08:30:26] DEBUG: INIT: Zeroed block 3
|
||
[08:30:26] INFO: INIT: System initialization complete - blocks freed, FORTH context active
|
||
[08:30:26] INFO: Dictionary fence set - init words protected from FORGET
|
||
[08:30:26] INFO: Starting Forth REPL
|
||
....
|
||
|
||
=== Development Workflow
|
||
|
||
==== IDE Mode (Linux)
|
||
|
||
[arabic]
|
||
. *Edit init.4th* - Modify `+./conf/init.4th+` directly
|
||
. *Test* - Run `+./build/starforth+` to see results
|
||
. *Commit* - Version control tracks plain text changes
|
||
. *PR Review* - Reviewers see diffs of init.4th
|
||
|
||
==== Disk Image Workflow (Future)
|
||
|
||
[arabic]
|
||
. *Work in disk images* - Edit Forth code in `+./disks/*.img+`
|
||
. *Mark blocks* - Use `+(-+` comments to mark blocks for extraction
|
||
. *Extract* - Tools in `+./tools/+` scan images and update init.4th
|
||
. *Review* - init.4th changes show up in git diff
|
||
. *Commit* - Both disk images (LFS) and init.4th (git) are tracked
|
||
|
||
==== L4Re Deployment
|
||
|
||
[arabic]
|
||
. *Build* - init.4th is packed into ROMFS at build time
|
||
. *modules.list* - Defines init.4th as a module
|
||
. *Boot* - L4Re loads ROMFS, INIT reads from ROMFS
|
||
. *Runtime* - No filesystem dependency
|
||
|
||
=== Implementation Details
|
||
|
||
==== LOAD Rewriting Algorithm
|
||
|
||
[source,c]
|
||
----
|
||
// First pass: Build mapping table
|
||
Block 2048 -> Sequential 1
|
||
Block 3000 -> Sequential 2
|
||
Block 3001 -> Sequential 3
|
||
|
||
// Second pass: Copy with rewriting
|
||
while (copying block content) {
|
||
if (found "NNNN LOAD" pattern) {
|
||
lookup NNNN in mapping
|
||
replace with mapped number
|
||
write "M LOAD" to block
|
||
}
|
||
}
|
||
----
|
||
|
||
==== Memory Layout
|
||
|
||
....
|
||
VM Memory (5MB):
|
||
├─ Dictionary (grows up)
|
||
│ ├─ System words (protected by fence)
|
||
│ ├─ Init words (F, E, STAR... - protected by fence)
|
||
│ └─ User words (can be FORGOTten)
|
||
└─ HERE (current allocation point)
|
||
|
||
Block RAM (1MB):
|
||
├─ Block 0: Reserved (volume metadata)
|
||
├─ Blocks 1-992: User blocks (zeroed after INIT)
|
||
└─ Blocks 993-1023: Reserved
|
||
|
||
External Disk (if attached):
|
||
└─ Blocks 1024+: Persistent storage
|
||
....
|
||
|
||
==== Performance
|
||
|
||
* *Init time*: <10ms typical (depends on init.4th size)
|
||
* *Memory overhead*: Transient (blocks zeroed after init)
|
||
* *Dictionary size*: Proportional to init.4th definitions
|
||
* *No runtime cost*: Init happens once at boot
|
||
|
||
=== Best Practices
|
||
|
||
==== Writing init.4th
|
||
|
||
[arabic]
|
||
. *Keep it minimal* - Only foundational definitions
|
||
. *Document blocks* - Always use `+(-+` comments
|
||
. *Avoid cross-references* - Define dependencies first
|
||
. *Test incrementally* - Add blocks one at a time
|
||
. *No side effects* - Init should define words, not execute actions
|
||
(except testing in last block)
|
||
|
||
==== Managing Dependencies
|
||
|
||
[source,forth]
|
||
----
|
||
\ GOOD: Dependencies defined first
|
||
Block 1
|
||
(- Foundation words )
|
||
: HELPER1 ... ;
|
||
: HELPER2 ... ;
|
||
|
||
Block 2
|
||
(- Uses helpers )
|
||
: MAIN HELPER1 HELPER2 ;
|
||
|
||
\ BAD: Forward reference
|
||
Block 1
|
||
(- Uses undefined word )
|
||
: MAIN HELPER1 ; \ HELPER1 not defined yet!
|
||
|
||
Block 2
|
||
(- Define helper )
|
||
: HELPER1 ... ;
|
||
----
|
||
|
||
==== Avoiding `+-->+`
|
||
|
||
[source,forth]
|
||
----
|
||
\ GOOD: Separate blocks
|
||
Block 1
|
||
(- First part )
|
||
: FOO ... ;
|
||
|
||
Block 2
|
||
(- Second part )
|
||
: BAR FOO ;
|
||
|
||
\ BAD: Using --> in init
|
||
Block 1
|
||
(- Spanning definition )
|
||
: BIGWORD
|
||
part1
|
||
-->
|
||
Block 2
|
||
part2 ;
|
||
\ Problem: INIT loads blocks sequentially, --> causes block 2 to execute twice
|
||
----
|
||
|
||
=== Troubleshooting
|
||
|
||
==== "`INIT: Failed to open ./conf/init.4th`"
|
||
|
||
*Cause*: File missing or wrong working directory *Fix*: Ensure init.4th
|
||
exists and run from project root
|
||
|
||
==== "`INIT: Error executing block N`"
|
||
|
||
*Cause*: Syntax error or undefined word in block N *Fix*: Check block N
|
||
content, ensure dependencies are defined first
|
||
|
||
==== "`UNKNOWN WORD: '`FOO`'`"
|
||
|
||
*Cause*: FOO referenced before definition *Fix*: Reorder blocks so FOO
|
||
is defined before use
|
||
|
||
==== Segfault after FORGET
|
||
|
||
*Cause*: Attempted to FORGET init word (shouldn’t be possible after
|
||
fence) *Fix*: Verify fence is set in main.c after INIT
|
||
|
||
==== Words missing after INIT
|
||
|
||
*Cause*: LOAD not executing properly, or blocks zeroed too early *Fix*:
|
||
Check LOAD implementation, verify zeroing happens after execution
|
||
|
||
=== Code References
|
||
|
||
==== INIT Word Implementation
|
||
|
||
*File*: `+src/word_source/starforth_words.c:214-495+`
|
||
|
||
Key sections:
|
||
|
||
* Line 234-240: Open init.4th file
|
||
* Line 272-305: First pass - build block mapping table
|
||
* Line 307-418: Second pass - copy blocks with LOAD rewriting
|
||
* Line 336-398: LOAD reference rewriting logic
|
||
* Line 447-465: Execute blocks via LOAD
|
||
* Line 468-481: Switch to FORTH vocabulary
|
||
* Line 484-493: Zero init blocks
|
||
|
||
==== Dictionary Fence Setting
|
||
|
||
*File*: `+src/main.c:488-491+`
|
||
|
||
[source,c]
|
||
----
|
||
/* Set dictionary fence after INIT to protect foundational words from FORGET */
|
||
vm.dict_fence_latest = vm.latest;
|
||
vm.dict_fence_here = vm.here;
|
||
log_message(LOG_INFO, "Dictionary fence set - init words protected from FORGET");
|
||
----
|
||
|
||
==== FORGET Implementation
|
||
|
||
*File*: `+src/word_source/defining_words.c:530-612+`
|
||
|
||
Key section:
|
||
|
||
* Line 559-571: Check if target is newer than fence (prevent forgetting
|
||
protected words)
|
||
* Line 586-606: Free entries and relink dictionary chain
|
||
|
||
==== LOAD Implementation
|
||
|
||
*File*: `+src/word_source/block_words.c:191-213+`
|
||
|
||
[source,c]
|
||
----
|
||
void block_word_load(VM *vm) {
|
||
// ... get block buffer ...
|
||
|
||
/* Interpret the block content as Forth source */
|
||
char block_text[1025];
|
||
memcpy(block_text, buf, 1024);
|
||
block_text[1024] = '\0';
|
||
|
||
/* Interpret the block content */
|
||
vm_interpret(vm, block_text);
|
||
}
|
||
----
|
||
|
||
==== `+-->+` Implementation
|
||
|
||
*File*: `+src/word_source/block_words.c:289-309+`
|
||
|
||
Continues interpretation to next sequential block (SCR + 1).
|
||
|
||
=== Future Enhancements
|
||
|
||
==== Planned Features
|
||
|
||
[arabic]
|
||
. *Tool: extract-init* - Extract init.4th from disk images
|
||
. *Tool: validate-init* - Syntax check init.4th
|
||
. *Tool: merge-init* - Merge multiple init.4th files
|
||
. *INIT options* - `+INIT-VERBOSE+`, `+INIT-DEBUG+` flags
|
||
. *Hot reload* - Reload init.4th without restart (dev mode)
|
||
. *Init profiles* - Multiple init files for different environments
|
||
|
||
==== L4Re Integration
|
||
|
||
[source,c]
|
||
----
|
||
// Planned L4Re ROMFS support
|
||
#ifdef L4RE_TARGET
|
||
L4Re::Env::env()->get_cap<L4Re::Dataspace>("init.4th");
|
||
// Map dataspace, read content
|
||
#endif
|
||
----
|
||
|
||
=== References
|
||
|
||
* FORTH-79 Standard (LOAD, BLOCK, LIST)
|
||
* Block Storage Guide (docs/BLOCK_STORAGE_GUIDE.md)
|
||
* L4Re Integration (docs/L4RE_INTEGRATION.md)
|
||
* StarForth Architecture (docs/ARCHITECTURE.md)
|
||
|
||
=== See Also
|
||
|
||
* `+src/word_source/starforth_words.c+` - INIT and (- implementation
|
||
* `+src/word_source/block_words.c+` - LOAD, –>, THRU implementation
|
||
* `+src/main.c+` - Startup sequence and fence setting
|
||
* `+conf/init.4th+` - Example initialization file
|