345 lines
14 KiB
Markdown
345 lines
14 KiB
Markdown
# Hardware Abstraction Layer (HAL) Architecture
|
|
|
|
> **⚠️ Important Note**: This document describes the **target HAL architecture** with standardized `hal_*` function naming. The current codebase uses `sf_time_backend_t` and similar abstractions with different naming conventions. See [migration-plan.md](migration-plan.md) for the refactoring steps to migrate existing platform code to the HAL interfaces described here.
|
|
>
|
|
> **HISTORICAL — L4Re/Fiasco.OC**: this document's L4Re references (platform comparison table, microkernel discussion, etc.) describe a target that was a supported platform through mid-2026. L4Re support has since been removed as an active target — `src/platform/l4re/time.c` and related `#ifdef __l4__` code are retained for reference but no longer wired into any build. Treat every L4Re mention below as historical design context, not a currently buildable platform.
|
|
|
|
## Executive Summary
|
|
|
|
The Hardware Abstraction Layer (HAL) is the architectural foundation that enables StarForth to evolve from a hosted VM into LithosAnanke while preserving the physics-driven adaptive runtime's deterministic behavior across all platforms.
|
|
|
|
**Critical insight:** The HAL is not an afterthought—it's the linchpin that makes LithosAnanke → StarshipOS possible without compromising StarForth's experimental integrity.
|
|
|
|
---
|
|
|
|
## The Problem
|
|
|
|
StarForth currently runs on multiple platforms:
|
|
- **Linux** (POSIX, hosted)
|
|
- **L4Re/Fiasco.OC** (microkernel)
|
|
- **Bare metal** (limited, experimental)
|
|
|
|
Each platform has its own:
|
|
- Timing mechanisms (POSIX timers vs. hardware timers)
|
|
- Interrupt handling (signals vs. IDT/APIC)
|
|
- Memory allocation (malloc vs. physical page frames)
|
|
- I/O mechanisms (stdin/stdout vs. UART/framebuffer)
|
|
|
|
**Without a HAL:** Platform-specific code bleeds into the VM core, physics subsystems, and word implementations. This creates:
|
|
- ❌ Fragile #ifdef PLATFORM_X conditionals throughout codebase
|
|
- ❌ Platform-specific bugs in supposedly portable code
|
|
- ❌ Inability to test kernel code on hosted platforms
|
|
- ❌ Risk to deterministic behavior guarantees (0% algorithmic variance)
|
|
|
|
**With a HAL:** Clean separation between VM logic and platform implementation:
|
|
- ✅ VM and physics subsystems are platform-agnostic
|
|
- ✅ Platform code is isolated and testable
|
|
- ✅ New platforms (LithosAnanke) can be added without touching VM core
|
|
- ✅ Deterministic behavior guaranteed by HAL contract, not platform quirks
|
|
|
|
---
|
|
|
|
## Architecture Overview
|
|
|
|
```
|
|
┌────────────────────────────────────────────────────────────┐
|
|
│ StarForth VM Core + Physics Subsystems │
|
|
│ • Interpreter loop (vm.c) │
|
|
│ • Execution heat tracking │
|
|
│ • Rolling window of truth │
|
|
│ • Hot-words cache │
|
|
│ • Pipelining metrics │
|
|
│ • Inference engine │
|
|
│ • Heartbeat coordination │
|
|
│ │
|
|
│ ↓ Calls HAL interfaces (platform-agnostic) │
|
|
├────────────────────────────────────────────────────────────┤
|
|
│ Hardware Abstraction Layer (HAL) │
|
|
│ • hal_time.h - Monotonic time, timers, calibration │
|
|
│ • hal_interrupt.h - IRQ management, ISR registration │
|
|
│ • hal_memory.h - Allocation, page mapping, heap │
|
|
│ • hal_console.h - Character I/O (serial, framebuffer) │
|
|
│ • hal_cpu.h - CPU ID, relax/halt, SMP coordination │
|
|
│ │
|
|
│ ↓ Platform-specific implementations │
|
|
├────────────────────────────────────────────────────────────┤
|
|
│ Platform Implementations │
|
|
│ ┌──────────────┬──────────────┬──────────────────────┐ │
|
|
│ │ Linux │ L4Re │ Kernel (LithosAnanke) │ │
|
|
│ │ (POSIX) │ (microkernel)│ (freestanding) │ │
|
|
│ ├──────────────┼──────────────┼──────────────────────┤ │
|
|
│ │ clock_gettime│ L4Re::Clock │ TSC + HPET + APIC │ │
|
|
│ │ timerfd │ L4Re::IrqEoi │ IDT + APIC IRQ │ │
|
|
│ │ malloc/free │ dataspaces │ PMM + VMM + kmalloc │ │
|
|
│ │ stdin/stdout │ L4Re::Console│ UART + framebuffer │ │
|
|
│ │ pthread │ L4Re::Thread │ SMP bring-up │ │
|
|
│ └──────────────┴──────────────┴──────────────────────┘ │
|
|
└────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## Design Principles
|
|
|
|
### 1. **VM Purity**
|
|
The VM core must never know which platform it's running on. All platform awareness lives in HAL implementations.
|
|
|
|
**Anti-pattern (current):**
|
|
```c
|
|
#ifdef PLATFORM_LINUX
|
|
clock_gettime(CLOCK_MONOTONIC, &ts);
|
|
#elif PLATFORM_L4RE
|
|
l4re_kip_clock(kip);
|
|
#elif PLATFORM_KERNEL
|
|
rdtsc();
|
|
#endif
|
|
```
|
|
|
|
**Correct pattern (HAL):**
|
|
```c
|
|
// In VM code (platform-agnostic)
|
|
uint64_t now = hal_time_now_ns();
|
|
|
|
// In platform/linux/hal_time.c
|
|
uint64_t hal_time_now_ns(void) {
|
|
struct timespec ts;
|
|
clock_gettime(CLOCK_MONOTONIC, &ts);
|
|
return (uint64_t)ts.tv_sec * 1000000000ULL + ts.tv_nsec;
|
|
}
|
|
|
|
// In platform/kernel/hal_time.c
|
|
uint64_t hal_time_now_ns(void) {
|
|
return tsc_to_ns(rdtsc());
|
|
}
|
|
```
|
|
|
|
### 2. **Contract-First Design**
|
|
HAL interfaces are **contracts**, not convenience wrappers. Each HAL function has:
|
|
- **Precise semantics** - What it does, guaranteed across all platforms
|
|
- **Error handling** - When it can fail and how
|
|
- **Performance expectations** - Allowed latency/overhead
|
|
- **Concurrency model** - Thread-safe? ISR-safe?
|
|
|
|
### 3. **Testability on Hosted Platforms**
|
|
LithosAnanke code must be developable and testable on Linux/L4Re before deploying to bare metal.
|
|
|
|
**Example:** Heartbeat ISR development
|
|
- **Kernel implementation:** APIC timer interrupt → ISR → ring buffer
|
|
- **Linux test implementation:** timerfd + signal handler → ISR → ring buffer
|
|
- **Same HAL interface, same VM code, different platform layer**
|
|
|
|
### 4. **Zero Overhead When Possible**
|
|
HAL calls should compile to direct hardware access on kernel platforms, not add abstraction tax.
|
|
|
|
**Good:** `hal_time_now_ns()` inlines to `rdtsc()` with `-O2`
|
|
**Bad:** Function pointer indirection adds 5-10 cycles per call
|
|
|
|
### 5. **Fail-Fast Validation**
|
|
HAL implementations validate platform assumptions at init time, not during execution.
|
|
|
|
**Example:**
|
|
```c
|
|
void hal_time_init(void) {
|
|
// Calibrate TSC frequency at boot
|
|
tsc_calibrate_hpet();
|
|
|
|
// Validate monotonicity
|
|
uint64_t t1 = hal_time_now_ns();
|
|
hal_cpu_relax();
|
|
uint64_t t2 = hal_time_now_ns();
|
|
|
|
if (t2 < t1) {
|
|
hal_panic("hal_time: TSC not monotonic!");
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## HAL Subsystems
|
|
|
|
### 1. **Time & Timers** (`hal_time.h`)
|
|
**Purpose:** Monotonic time, periodic/oneshot timers, calibration
|
|
|
|
**Critical for StarForth:** The heartbeat subsystem and physics feedback loops depend on precise, jitter-free timing. HAL must guarantee:
|
|
- Monotonic time (never goes backward)
|
|
- Sub-microsecond resolution
|
|
- Calibrated frequency (for TSC-based timing)
|
|
|
|
**Platform challenges:**
|
|
- **Linux:** `clock_gettime()` is good, but signal-based timers have latency
|
|
- **Kernel:** TSC drift, HPET/PIT fallback, per-core calibration
|
|
|
|
### 2. **Interrupts** (`hal_interrupt.h`)
|
|
**Purpose:** Enable/disable IRQs, register ISRs, query interrupt context
|
|
|
|
**Critical for StarForth:** Heartbeat ISR must run at precise intervals without VM involvement.
|
|
|
|
**Platform challenges:**
|
|
- **Linux:** Signals are "interrupt-like" but not true IRQs
|
|
- **Kernel:** IDT setup, APIC configuration, spurious interrupt handling
|
|
|
|
### 3. **Memory** (`hal_memory.h`)
|
|
**Purpose:** Allocate/free memory, page mapping, heap management
|
|
|
|
**Critical for StarForth:** Dictionary allocation, stack allocation, heap allocator must work identically across platforms.
|
|
|
|
**Platform challenges:**
|
|
- **Linux:** malloc/free are simple
|
|
- **Kernel:** Physical memory manager, virtual memory manager, heap allocator—all from scratch
|
|
|
|
### 4. **Console** (`hal_console.h`)
|
|
**Purpose:** Character I/O for REPL and diagnostics
|
|
|
|
**Critical for StarForth:** The REPL must work on all platforms for interactive experimentation.
|
|
|
|
**Platform challenges:**
|
|
- **Linux:** stdin/stdout are perfect
|
|
- **Kernel:** UART 16550 is well-documented but framebuffer is tricky
|
|
|
|
### 5. **CPU** (`hal_cpu.h`)
|
|
**Purpose:** CPU ID, relax/halt, SMP coordination
|
|
|
|
**Critical for StarForth:** Per-core execution heat tracking (future), SMP scalability (future).
|
|
|
|
**Platform challenges:**
|
|
- **Linux:** pthread local storage
|
|
- **Kernel:** Local APIC, per-core stacks, CPU-local storage
|
|
|
|
---
|
|
|
|
## HAL and the Physics Subsystems
|
|
|
|
The physics-driven adaptive runtime is the **primary beneficiary** of the HAL:
|
|
|
|
| Physics Subsystem | HAL Dependency | Why |
|
|
|-------------------|----------------|-----|
|
|
| **Execution Heat** | None (pure VM state) | Tracks word execution frequency |
|
|
| **Rolling Window** | `hal_time_now_ns()` | Timestamps for window entries |
|
|
| **Hot-Words Cache** | None (pure dictionary state) | Frequency-based reordering |
|
|
| **Pipelining Metrics** | None (pure transition tracking) | Word-to-word prediction |
|
|
| **Inference Engine** | `hal_time_now_ns()` | Calibration timing, ANOVA |
|
|
| **Heartbeat** | `hal_timer_periodic()`, `hal_interrupt.*` | ISR-based sampling |
|
|
|
|
**Key insight:** Only 2 of 6 subsystems touch the HAL directly, and only via clean interfaces. This preserves deterministic behavior while enabling kernel deployment.
|
|
|
|
---
|
|
|
|
## HAL and LithosAnanke
|
|
|
|
LithosAnanke is a **new platform implementation** of the HAL:
|
|
|
|
```
|
|
src/platform/kernel/
|
|
├── boot/
|
|
│ └── uefi_loader.c # UEFI entry point → BootInfo handoff
|
|
├── hal_time.c # TSC + HPET + APIC timer
|
|
├── hal_interrupt.c # IDT + Local APIC + IOAPIC
|
|
├── hal_memory.c # PMM + VMM + kmalloc
|
|
├── hal_console.c # UART 16550 + framebuffer
|
|
├── hal_cpu.c # SMP + CPU-local storage
|
|
└── drivers/ # PCI, AHCI, NVMe, VirtIO, etc.
|
|
```
|
|
|
|
**LithosAnanke does NOT modify:**
|
|
- VM core (`src/vm.c`)
|
|
- Physics subsystems (`src/dictionary_heat_optimization.c`, etc.)
|
|
- Word implementations (`src/word_source/*.c`)
|
|
|
|
**LithosAnanke ONLY implements:**
|
|
- HAL interfaces (`hal_*.c`)
|
|
- UEFI boot loader (`boot/uefi_loader.c`)
|
|
- Device drivers (`drivers/*.c`)
|
|
|
|
This is the **proof** that the HAL abstraction works: LithosAnanke is a pure platform layer, not a VM fork.
|
|
|
|
---
|
|
|
|
## HAL and StarshipOS
|
|
|
|
StarshipOS builds on LithosAnanke by adding:
|
|
- **Process model** - Forth tasks vs. traditional processes
|
|
- **Filesystem** - FAT32, ext2, or log-structured
|
|
- **Networking** - TCP/IP stack, DHCP, DNS
|
|
- **Device model** - Unified block/net/char device interfaces
|
|
- **Security model** - Capabilities, ACL, Forth-based access control
|
|
|
|
All of these **still use the HAL** for low-level access:
|
|
- Filesystem → `hal_memory` for caching, `hal_interrupt` for async I/O
|
|
- Networking → `hal_interrupt` for packet RX, `hal_time` for timeouts
|
|
- Device model → HAL as the common substrate
|
|
|
|
**The HAL is not just a kernel bootstrapping tool—it's the foundation for the entire OS.**
|
|
|
|
---
|
|
|
|
## Migration Strategy
|
|
|
|
Current StarForth codebase must be refactored to introduce the HAL:
|
|
|
|
### Phase 1: Define HAL Interfaces
|
|
- Write `include/hal/*.h` headers with contracts
|
|
- Document semantics, error handling, performance expectations
|
|
- **No implementation yet**
|
|
|
|
### Phase 2: Refactor Existing Platforms
|
|
- Create `src/platform/linux/hal_*.c` implementing HAL
|
|
- Create `src/platform/l4re/hal_*.c` implementing HAL
|
|
- Update VM code to call HAL instead of platform-specific APIs
|
|
- **VM must still build and pass all 936+ tests**
|
|
|
|
### Phase 3: Validate HAL
|
|
- Build and test on Linux
|
|
- Build and test on L4Re (if available)
|
|
- Verify deterministic behavior (0% algorithmic variance) still holds
|
|
- Benchmark: HAL must not add measurable overhead
|
|
|
|
### Phase 4: Implement LithosAnanke Platform
|
|
- Create `src/platform/kernel/` with HAL implementations
|
|
- UEFI boot loader
|
|
- Basic MM, console, timing
|
|
- **Goal:** Boot to `ok` prompt on QEMU/OVMF
|
|
|
|
### Phase 5: Full LithosAnanke
|
|
- Complete HAL implementations (interrupts, SMP, drivers)
|
|
- Heartbeat ISR running at kernel level
|
|
- Physics subsystems operational
|
|
- **Goal:** Reproduce DoE results on bare metal
|
|
|
|
### Phase 6: StarshipOS
|
|
- Process model, filesystem, networking
|
|
- Forth as native control plane
|
|
- **Goal:** Self-hosting OS
|
|
|
|
---
|
|
|
|
## Success Criteria
|
|
|
|
The HAL is successful if:
|
|
|
|
1. ✅ **VM core has zero platform-specific code**
|
|
2. ✅ **All 936+ tests pass on Linux, L4Re, and LithosAnanke**
|
|
3. ✅ **0% algorithmic variance maintained across platforms**
|
|
4. ✅ **No measurable performance regression from HAL abstraction**
|
|
5. ✅ **LithosAnanke boots to `ok` prompt and runs REPL**
|
|
6. ✅ **Heartbeat subsystem works identically on all platforms**
|
|
7. ✅ **New platforms can be added without touching VM code**
|
|
|
|
---
|
|
|
|
## Next Steps
|
|
|
|
See companion documentation:
|
|
- `interfaces.md` - Detailed HAL interface specifications
|
|
- `platform-implementations.md` - How to implement HAL for new platforms
|
|
- `migration-plan.md` - Step-by-step refactoring guide
|
|
- `lithosananke-integration.md` - LithosAnanke-specific implementation details
|
|
|
|
---
|
|
|
|
## References
|
|
|
|
- StarForth VM core: `src/vm.c`
|
|
- Heartbeat system: `docs/03-architecture/heartbeat-system/`
|
|
- Physics feedback loops: `docs/FEEDBACK_LOOPS.md`
|
|
- Platform abstraction (current): `src/platform/` |