202 lines
6.0 KiB
Plaintext
202 lines
6.0 KiB
Plaintext
// Moved from docs/src/platform-integration/L4RE_DICTIONARY_ALLOCATION.adoc to docs/working/scratch/src/platform-integration/L4RE_DICTIONARY_ALLOCATION.adoc on 2026-06-16 (docs reorg Phase 2)
|
||
== Dictionary Allocation Strategy for L4Re
|
||
:toc: left
|
||
:toc-title: Contents
|
||
:toclevels: 3
|
||
xref:../README.adoc[← Back to Documentation Index]
|
||
|
||
|
||
|
||
=== Current Implementation
|
||
|
||
StarForth currently uses `+malloc()+` and `+free()+` from the host C
|
||
library for dictionary entry allocation:
|
||
|
||
* *Location*: `+src/dictionary_management.c:228+`
|
||
* *Function*: `+vm_create_word()+` uses `+malloc(total)+` to allocate
|
||
DictEntry structures
|
||
* *Cleanup*: `+FORGET+` uses `+free(e)+` to deallocate entries (line 554
|
||
in defining_words.c)
|
||
|
||
This works fine for hosted environments but creates a dependency on libc
|
||
heap allocator.
|
||
|
||
=== Why This Matters for L4Re/StarshipOS
|
||
|
||
==== Issues with Current Approach:
|
||
|
||
[arabic]
|
||
. *Heap Dependency*: Requires full malloc implementation in L4Re
|
||
environment
|
||
. *Non-Deterministic*: Heap fragmentation and unpredictable allocation
|
||
behavior
|
||
. *Separate Address Space*: Dictionary entries live outside VM memory
|
||
model
|
||
. *Hard to Introspect*: Can’t dump entire VM state as single memory
|
||
region
|
||
|
||
==== Benefits of VM-Based Allocation:
|
||
|
||
[arabic]
|
||
. *Zero External Dependencies*: No libc malloc needed
|
||
. *Deterministic*: Simple bump allocator, predictable behavior
|
||
. *Unified Address Space*: Everything in one 5MB VM memory region
|
||
. *Easy Snapshots*: Can serialize entire VM state
|
||
. *Better for Microkernel*: Aligns with L4Re dataspace model
|
||
|
||
=== Recommended Approach for L4Re
|
||
|
||
==== Strategy: Conditional Compilation
|
||
|
||
Add a compile-time flag to switch between malloc and VM-based
|
||
allocation:
|
||
|
||
[source,c]
|
||
----
|
||
// In dictionary_management.c
|
||
#ifdef STARFORTH_VM_DICT_ALLOC
|
||
/* VM-based allocation (for L4Re/embedded) */
|
||
DictEntry *entry = dict_alloc_from_vm(vm, total);
|
||
#else
|
||
/* Standard malloc (for hosted environments) */
|
||
DictEntry *entry = (DictEntry *) malloc(total);
|
||
#endif
|
||
----
|
||
|
||
==== Memory Layout (VM-Based Mode):
|
||
|
||
....
|
||
vm->memory[0...VM_MEMORY_SIZE] (5MB):
|
||
┌──────────────────────────────────────────┐
|
||
│ Dictionary Headers (256KB) │ ← dict_headers_used grows up
|
||
│ - DictEntry structures │
|
||
│ - Allocated via dict_alloc() │
|
||
├──────────────────────────────────────────┤
|
||
│ User Data & Code (variable) │ ← HERE grows up
|
||
│ - Compiled definitions │
|
||
│ - Variables, constants │
|
||
│ - String literals │
|
||
├──────────────────────────────────────────┤
|
||
│ Block Storage (remainder) │
|
||
│ - 1KB blocks for Forth block system │
|
||
└──────────────────────────────────────────┘
|
||
....
|
||
|
||
==== Implementation Steps:
|
||
|
||
[arabic]
|
||
. *Add Dictionary Allocator* (256KB bump allocator at `+memory[0]+`):
|
||
|
||
[source,c]
|
||
----
|
||
static DictEntry *dict_alloc_from_vm(VM *vm, size_t total_bytes) {
|
||
/* Align to cell_t boundary */
|
||
size_t align = sizeof(cell_t);
|
||
total_bytes = (total_bytes + align - 1) & ~(align - 1);
|
||
|
||
/* Check space */
|
||
if (vm->dict_headers_used + total_bytes > DICT_HEADERS_MAX) {
|
||
vm->error = 1;
|
||
return NULL;
|
||
}
|
||
|
||
/* Allocate from VM memory */
|
||
DictEntry *entry = (DictEntry *)(vm->memory + vm->dict_headers_used);
|
||
vm->dict_headers_used += total_bytes;
|
||
return entry;
|
||
}
|
||
----
|
||
|
||
[arabic, start=2]
|
||
. *Update FORGET* (no individual frees, just rewind):
|
||
|
||
[source,c]
|
||
----
|
||
/* Instead of free(e) loop, rewind allocator */
|
||
size_t target_offset = (uint8_t*)target - vm->memory;
|
||
vm->dict_headers_used = target_offset;
|
||
----
|
||
|
||
[arabic, start=3]
|
||
. *Adjust VM Initialization*:
|
||
|
||
[source,c]
|
||
----
|
||
vm->dict_headers_used = 0; /* Headers start at memory[0] */
|
||
vm->here = DICT_HEADERS_MAX; /* User data starts at 256KB */
|
||
----
|
||
|
||
[arabic, start=4]
|
||
. *No Changes to Fast-Lookup Cache*: Cache buckets still use malloc for
|
||
the pointer arrays, but entries themselves are in VM memory.
|
||
|
||
==== Testing Strategy:
|
||
|
||
[source,bash]
|
||
----
|
||
# Test with standard malloc (current):
|
||
make clean && make test
|
||
|
||
# Test with VM-based allocation:
|
||
make clean
|
||
CFLAGS="-DSTARFORTH_VM_DICT_ALLOC" make test
|
||
|
||
# Test L4Re minimal build:
|
||
make clean
|
||
CFLAGS="-DSTARFORTH_MINIMAL -DSTARFORTH_VM_DICT_ALLOC" make rpi4-cross
|
||
----
|
||
|
||
=== Current Status
|
||
|
||
*Decision*: Keep using `+malloc()+` for now, document the abstraction
|
||
point clearly.
|
||
|
||
*Rationale*:
|
||
|
||
* Current implementation is stable and well-tested
|
||
* L4Re port can implement the switch when needed
|
||
* Abstraction layer is simple and well-defined
|
||
* No performance difference between approaches
|
||
|
||
*Action Items for L4Re Port*:
|
||
|
||
[arabic]
|
||
. Define `+STARFORTH_VM_DICT_ALLOC+` in L4Re build
|
||
. Implement `+dict_alloc_from_vm()+` function
|
||
. Update `+FORGET+` to use rewind instead of free loop
|
||
. Test with full test suite
|
||
. Verify no memory leaks in microkernel environment
|
||
|
||
=== Minimal Code Changes Needed
|
||
|
||
The switch to VM-based allocation requires changes to only 3 functions:
|
||
|
||
[arabic]
|
||
. *`+vm_create_word()+`* - Use `+dict_alloc_from_vm()+` instead of
|
||
`+malloc()+`
|
||
. *`+dictionary_word_forget()+`* - Rewind `+dict_headers_used+` instead
|
||
of `+free()+` loop
|
||
. *`+vm_init()+`* - Initialize `+dict_headers_used+` and offset `+HERE+`
|
||
|
||
Add to `+VM+` struct in `+include/vm.h+`:
|
||
|
||
[source,c]
|
||
----
|
||
size_t dict_headers_used; /* Bytes used for dictionary headers (VM mode only) */
|
||
----
|
||
|
||
*Estimated effort*: 4 hours to implement + 2 hours testing = *6 hours
|
||
total*
|
||
|
||
=== References
|
||
|
||
* Current malloc usage: `+src/dictionary_management.c:228+`
|
||
* Current free usage: `+src/word_source/defining_words.c:554+`
|
||
* Fast-lookup cache: `+src/dictionary_management.c:43-67+` (unchanged)
|
||
* L4Re dataspace docs: `+docs/L4RE_INTEGRATION.md+`
|
||
|
||
'''''
|
||
|
||
*Author*: R. A. James *Date*: 2025-10-01 *Status*: Documented,
|
||
implementation deferred to L4Re port
|