322 lines
17 KiB
Markdown
322 lines
17 KiB
Markdown
# Hermes Message/Channel Heat Physics — Design Doc
|
|
**Date:** 2026-07-08 (rev 2026-07-08a)
|
|
**Branch:** `lithosananke`
|
|
**Status:** Design only. Not implemented. **Depends on a sibling
|
|
architectural doc**: `HERMES-MESSAGE-BLOCK-STORAGE-DESIGN-20260708.md`
|
|
(rev c), which plans to move Hermes's message area off its current
|
|
in-memory `CREATE...ALLOT` arena (`MSG-ARENA`/`CH-ARENA`, `MSG-MAX`=32/
|
|
`CH-MAX`=16) onto a dedicated, LBN-backed block range within the kernel
|
|
ramdrive — a Hermes-owned physical BAM over Hermes's own blocks, built
|
|
independently of and with zero coupling to Artemis (Artemis's own
|
|
disk-block model is precedent for the allocator *shape*, not something
|
|
Hermes's storage shares or depends on). That redesign is out of scope for
|
|
this doc — this doc's window-sizing reasoning (section 2) is provisional
|
|
against the *current* in-memory scale and will need revisiting once the
|
|
block-backed redesign lands, since block-backed storage will likely
|
|
change both the arena's real capacity and its churn characteristics.
|
|
Sibling doc to
|
|
`VM-PHYSICS-DYNAMIC-FLEET-DESIGN-20260705.md`'s "Fold-in question" section,
|
|
which settled that messages and blocks get their own independent decay
|
|
mechanisms — not merged into `VMPhysics`, not derived from its slope —
|
|
each fit from its own rolling window, in separate follow-on docs. This is
|
|
that doc for Hermes (messages + channels). See
|
|
`ARTEMIS-BLOCK-PHYSICS-DESIGN-20260708.md` for the block-heat sibling.
|
|
**Author:** Captain Bob / Claude Code
|
|
|
|
**Correction (2026-07-09):** this doc's original Goal section (rev a)
|
|
claimed message/channel heat has no conservation invariant, citing a
|
|
"settled fold-in decision" from `VM-PHYSICS-DYNAMIC-FLEET-DESIGN-
|
|
20260705.md`. That claim was wrong — it was derived without first reading
|
|
`.claude/HERMES.md`, which is authoritative and states plainly: *"Every
|
|
object Hermes manages — messages and channels — participates in K≡1.0.
|
|
Fleet K includes the thermal contribution of in-flight messages and open
|
|
channels."* A full Tripod-docs audit (`TRIPOD.md`/`HERMES.md`/
|
|
`ARTEMIS.md`) confirmed this directly contradicts what's below. The Goal
|
|
section is corrected in place. The rest of this doc's mechanism — an
|
|
independently-fit decay *rate* for messages and channels, its own
|
|
`HermesHeatWindow` struct, its own rolling window, separate from
|
|
word-level and VM-fleet physics — is unaffected: `HERMES.md` requires
|
|
message/channel heat to *participate in* K≡1.0, not that it share
|
|
VM-fleet's inference mechanism. Those are separable, and only the
|
|
conservation claim was wrong.
|
|
|
|
---
|
|
|
|
## Problem
|
|
|
|
`capsules/hermes/init.4th` hardcodes `65208 CONSTANT Q-DECAY` (Block 4100)
|
|
and applies it identically to two different entities via straight
|
|
multiplicative shrink:
|
|
|
|
- `MSG-COOL-ONE`/`MSG-COOL-ALL` (Block 4108): `heat = heat * Q-DECAY` for
|
|
every live message in the 32-slot `MSG-ARENA`.
|
|
- `CH-COOL-ALL` (Block 4114): same formula, walked over the live channel
|
|
linked list (16-slot `CH-ARENA`).
|
|
|
|
This is the exact "before" picture `VM-PHYSICS-DYNAMIC-FLEET-DESIGN`
|
|
already diagnosed for VM heat: `65208` was independently re-declared as
|
|
an identical hand-picked constant in three separate files
|
|
(`compudynamics.4th`, `hermes/init.4th`, `artemis/init.4th`) — copy-pasted,
|
|
uncoordinated, with no regression anywhere. The VM-heat instance of this
|
|
was fixed (`capsule_vm_physics.c`). Hermes's and Artemis's instances
|
|
weren't — this doc is Hermes's fix.
|
|
|
|
No evidence exists that `65208` (≈0.9950 per cool-call, i.e. a message or
|
|
channel's heat halves after roughly 138 `MSG-COOL-ALL` calls) was ever
|
|
tuned against real message/channel churn. It's the same number VM heat
|
|
used, which was itself never tuned against VM touch frequency either.
|
|
|
|
## Goal
|
|
|
|
Replace the hardcoded `Q-DECAY` with a `msg_decay_slope_q48` inferred from
|
|
observed cooling behavior, mirroring the *shape* of the word-level and
|
|
VM-fleet physics engines — same mechanics (rolling window, inferred rate,
|
|
skip-rather-than-substitute-a-default when unwarmed) applied to a third
|
|
kind of entity — without merging into either existing engine's *state* or
|
|
*inference mechanism*.
|
|
|
|
**Message and channel heat DOES participate in K≡1.0** — per
|
|
`.claude/HERMES.md`'s Compudynamic Invariant: *"Fleet K includes the
|
|
thermal contribution of in-flight messages and open channels... Hermes
|
|
reaping a message or channel must rebalance K correctly."* This doc does
|
|
not design that redistribution mechanism — `HERMES.md` already establishes
|
|
it conceptually (ACK → clean death, K redistributed; reap → K rebalanced),
|
|
and `HERMES.md`'s own G8 gap tracks the one known incompleteness (`HERMES-
|
|
K` exists and totals live message/channel heat, but wiring it into Hera's
|
|
`K-FLEET` is deferred pending an async inter-VM return-value model). What
|
|
this doc *does* design is orthogonal to that: **the rate** at which a
|
|
live message or channel's heat decays between creation and reap. Whether
|
|
that decay is later credited back to the fleet correctly is `HERMES.md`'s
|
|
concern and G8's tracked gap, not this doc's.
|
|
|
|
## Why this can't just copy either existing mechanism verbatim
|
|
|
|
**Not word-level's shape (rolling window of `word_id`s + trajectory
|
|
replay against current heat):** words are permanent dictionary entries —
|
|
`extract_heat_trajectory` (`inference_engine.c`) replays a window of
|
|
`word_id`s and looks up each one's *current* heat, exploiting that a word
|
|
never stops existing. Messages and channels are transient: `MSG-REAP`
|
|
frees a message's arena slot back to the free list once its heat cools
|
|
past whatever threshold or it's explicitly consumed, and a freed slot's
|
|
"current heat" is meaningless — it may already belong to an unrelated
|
|
message by the time anyone would replay a window containing its old ID.
|
|
Per-entity trajectory replay does not work here.
|
|
|
|
**Not VM-fleet's shape either, but structurally the closest analog:**
|
|
`capsule_vm_physics.c` (rev o) doesn't curve-fit a trajectory — it
|
|
recovers the transfer rate directly, because the transfer law is known
|
|
exactly (`amount = elapsed_us * slope_q48 >> 16`) and any unclamped touch
|
|
can be inverted for an exact per-sample rate; the fleet-wide
|
|
`fleet_transfer_slope_q48` is the *median* of recovered per-sample rates
|
|
over a 64-deep window of aggregate touch samples, not per-VM. The same
|
|
"known law, recover the rate, aggregate not per-entity" shape applies
|
|
here: `heat_new = heat_old * Q-DECAY` is *also* a fully known multiplicative
|
|
law, so there is nothing to curve-fit — what's actually being inferred is
|
|
not "what is the decay law" (known) but "is the current decay *rate*
|
|
tuned right for observed churn," which is a different question than either
|
|
existing engine answers, addressed below.
|
|
|
|
## The mechanism
|
|
|
|
### 1. What "inferring the rate" actually means here
|
|
|
|
Because the decay law itself is known and fixed-shape (`heat *= rate`),
|
|
there's no unknown functional relationship to recover the way VM-fleet
|
|
recovers `fleet_transfer_slope_q48` from touch samples, or the way Loop #6
|
|
recovers `decay_slope_q48` from a heat trajectory. What *can* legitimately
|
|
vary and be worth observing: **how fast messages/channels are actually
|
|
being reaped relative to how fast they're being created.** If churn is
|
|
high (many `MSG-ALLOC`/`MSG-REAP` cycles per cool-sweep), a slow decay
|
|
rate lets dead-weight messages linger, consuming arena slots that new
|
|
messages need. If churn is low, an aggressive decay rate discards heat
|
|
signal (e.g. NACK-retry priority, per `hermes/init.4th`'s own
|
|
`MSG-REDELIVER-NACKED`) before it's useful.
|
|
|
|
`msg_decay_slope_q48` should therefore be a function of **observed reap
|
|
throughput relative to arena occupancy**, not a re-derivation of the
|
|
`Q-DECAY` law itself. Concretely: a rolling window of samples taken once
|
|
per `MSG-COOL-ALL` call, each recording `(live_count, reaped_since_last)`;
|
|
periodically re-fit `msg_decay_slope_q48` so that, at the observed churn
|
|
rate, an average message crosses the reap threshold within a target
|
|
window (e.g. "most messages should be reaped within N cool-sweeps of
|
|
going idle" — the specific target is an open question, see Resolved
|
|
Items).
|
|
|
|
This is a genuinely different inference problem than either existing
|
|
engine solves, which is why it doesn't reduce to copying `infer_decay_slope_q48`
|
|
or `vm_physics_tick`'s median-of-rates — it's closer in spirit to Loop #5's
|
|
window-width inference (which also asks "is the current parameter well-
|
|
matched to observed dynamics," not "what is the true decay curve").
|
|
|
|
### 2. `HermesHeatWindow` — new struct, two instances (message, channel)
|
|
|
|
```c
|
|
typedef struct {
|
|
uint32_t live_count; /* live entities at this sample */
|
|
uint32_t reaped_since_last; /* entities freed since the prior sample */
|
|
} HermesHeatSample;
|
|
|
|
typedef struct {
|
|
HermesHeatSample samples[HERMES_HEAT_WINDOW_DEPTH];
|
|
uint32_t head;
|
|
uint32_t count; /* saturates at HERMES_HEAT_WINDOW_DEPTH */
|
|
int is_warm; /* count >= HERMES_HEAT_WINDOW_DEPTH */
|
|
} HermesHeatWindow;
|
|
```
|
|
|
|
Two instances — one for messages, one for channels — since they have
|
|
independent arenas, independent churn rates, and are proposed to have
|
|
independent decay slopes as this doc's own conservative default (see
|
|
Open questions). Not one shared window; `MSG-MAX`=32
|
|
and `CH-MAX`=16 are different enough scales that conflating them risks
|
|
exactly the kind of unjustified shared-constant problem this doc exists
|
|
to fix.
|
|
|
|
**`HERMES_HEAT_WINDOW_DEPTH`: proposed 16, not yet settled, and provisional
|
|
against a scale that's about to change.** `MSG-MAX`=32 and `CH-MAX`=16 are
|
|
today's *in-memory arena* limits — far smaller than word-level's millions
|
|
of executions or even VM-fleet's `VM_FLEET_WINDOW_DEPTH`=64, so a window
|
|
deeper than the arena itself provides no additional information at
|
|
*current* scale. But per this doc's Status header, the message area is
|
|
planned to move onto a dedicated block range, which will likely raise
|
|
that capacity substantially (Artemis's equivalent, `ART-DATA-BLKS`, is
|
|
22,998 — three orders of magnitude larger than `MSG-MAX`=32). 16 is a
|
|
starting guess against *today's* scale, not derived from anything, and
|
|
should be treated as provisional until the block-backed redesign's real
|
|
capacity is known (see Resolved Items).
|
|
|
|
### 3. Recording — passive, hooked at the existing cool-sweep call sites
|
|
|
|
`MSG-COOL-ALL` and `CH-COOL-ALL` already sweep their entire arena once per
|
|
call; a new `hermes_heat_sample(HermesHeatWindow*, uint32_t live_count,
|
|
uint32_t reaped_since_last)` call at the end of each sweep is the only new
|
|
call site needed — no new dispatch points, matching both existing engines'
|
|
"passive observer, hooked at what's already there" discipline.
|
|
|
|
**`reaped_since_last` bookkeeping is the one real implementation question
|
|
this doc leaves open.** `MSG-REAP` and channel reaping aren't currently
|
|
instrumented to report a count to the cool-sweep — that's new plumbing,
|
|
not free. Whether it's a shared counter incremented at each reap site and
|
|
read-and-reset by the cool-sweep, or something else, is unresolved (see
|
|
Resolved Items) — flagged rather than guessed at.
|
|
|
|
### 4. Inference — heartbeat-gated, same skip-don't-substitute discipline
|
|
|
|
`hermes_heat_tick()`, called from `HERMES-TICK`'s existing heartbeat path
|
|
(mirroring `vm_physics_heartbeat_tick`'s hook into each VM's own tick):
|
|
when a window is warm, re-fits that window's decay slope; when unwarmed,
|
|
leaves the current slope untouched rather than substituting a default —
|
|
same philosophy both existing engines already use.
|
|
|
|
The actual fit function (how `(live_count, reaped_since_last)` samples
|
|
become a new `msg_decay_slope_q48`) is intentionally not specified in
|
|
concrete formula form here — see Resolved Items. Writing a specific
|
|
formula now, without the real observed-churn data either existing engine's
|
|
design had before it settled on its final shape (VM-fleet went through
|
|
revisions b through o before landing on median-of-rates), would be
|
|
inventing content rather than designing it.
|
|
|
|
### 5. Application — replaces `Q-DECAY Q.*` at both cool-sweep sites
|
|
|
|
`MSG-COOL-ONE` and `CH-COOL-ALL`'s `Q-DECAY Q.*` becomes
|
|
`msg_decay_slope_q48 Q.*` / `ch_decay_slope_q48 Q.*` (or a single
|
|
`msg_decay_slope_q48` if the message/channel split above turns out
|
|
unwarranted after real observation — see Resolved Items). No change to
|
|
the multiplicative decay *shape* itself, only to where the rate comes
|
|
from.
|
|
|
|
## Consumer code migration
|
|
|
|
- **`hermes/init.4th` Block 4100**: `65208 CONSTANT Q-DECAY` deleted.
|
|
- **`hermes/init.4th` Block 4108** (`MSG-COOL-ONE`/`MSG-COOL-ALL`):
|
|
`Q-DECAY` reference replaced with a call to fetch the current
|
|
`msg_decay_slope_q48` (new C primitive, e.g. `MSG-DECAY-SLOPE@`);
|
|
`MSG-COOL-ALL` gains the sample-recording call described in section 3.
|
|
- **`hermes/init.4th` Block 4114** (`CH-COOL-ALL`): same pattern for
|
|
channel heat.
|
|
- New kernel-only file pair, matching `capsule_vm_physics.c`'s own scope
|
|
(Hermes-specific state, no hosted-build equivalent since the hosted
|
|
build has no Hermes):
|
|
- `src/starkernel/capsule/hermes_heat_physics.c`
|
|
- `include/starkernel/hermes_heat_physics.h`
|
|
- New FORTH-visible primitives (backed by the new file): `MSG-DECAY-SLOPE@`,
|
|
`CH-DECAY-SLOPE@` (or one shared accessor if sections above's split is
|
|
resolved differently), plus a status word mirroring `VM-PHYSICS-STATUS`'s
|
|
shape for diagnostics.
|
|
|
|
## What gets deleted
|
|
|
|
- `hermes/init.4th` Block 4100: `65208 CONSTANT Q-DECAY`.
|
|
|
|
Nothing else — `MSG-COOL-ONE`/`MSG-COOL-ALL`/`CH-COOL-ALL` keep their
|
|
existing structure, only the decay-rate source changes.
|
|
|
|
## Verification approach
|
|
|
|
- Three-arch acceptance as usual (kernel-only code,
|
|
`#ifdef __STARKERNEL__` gates per `.claude/CLAUDE.md`).
|
|
- A targeted test: drive real message/channel churn (repeated
|
|
`SEND-BROADCAST-TEST`-style traffic) at two different rates across
|
|
separate boot runs, confirm `msg_decay_slope_q48` converges to
|
|
different values reflecting the different observed churn — the direct
|
|
analog of confirming VM-fleet's `fleet_transfer_slope_q48` actually
|
|
responds to real touch frequency rather than sitting at its seed value.
|
|
- Confirm `HermesHeatWindow`'s dead-entity handling (a message reaped
|
|
mid-window) doesn't fault or corrupt the sample count — same class of
|
|
test as VM-fleet's "kill-during-warm-up" test
|
|
(`VM-PHYSICS-DYNAMIC-FLEET-DESIGN-20260705.md`).
|
|
|
|
## Explicitly out of scope
|
|
|
|
- **Moving Hermes's message/channel storage onto a dedicated LBN-backed
|
|
block range** (mirroring Artemis's `ART-DATA-BLKS`). Flagged in this
|
|
doc's Status header as a real dependency, but the redesign itself —
|
|
which blocks, what the on-disk layout looks like, how it interacts with
|
|
`MSG-ARENA`/`CH-ARENA`'s current in-memory shape — is a separate,
|
|
not-yet-written architectural doc, not part of this heat-decay design.
|
|
- **Artemis block heat** — sibling doc, `ARTEMIS-BLOCK-PHYSICS-DESIGN-20260708.md`.
|
|
- **Designing how K is credited/debited at message and channel alloc/reap**
|
|
— already governed conceptually by `HERMES.md`'s Compudynamic Invariant
|
|
section; the one known incompleteness (wiring `HERMES-K` into Hera's
|
|
`K-FLEET`) is `HERMES.md`'s own tracked G8 gap, not something this doc
|
|
redesigns. This doc's decay-rate inference operates on top of that
|
|
redistribution, not in place of it.
|
|
- **The multi-level DoE rewrite** (word/VM-fleet/message/block as four
|
|
independent metric spaces) — this doc gives message/channel heat its
|
|
mechanism; observing it as part of a real experiment is separate,
|
|
larger work already on the punch list.
|
|
- **Implementation** — this is a design doc. Writing
|
|
`hermes_heat_physics.c` is follow-on work, not part of this pass.
|
|
|
|
## Resolved items
|
|
|
|
None yet — this is a first-pass design doc, not an iterated one. Unlike
|
|
`VM-PHYSICS-DYNAMIC-FLEET-DESIGN-20260705.md`'s extensive revision history
|
|
(rev a through t, each resolving a question found during real
|
|
implementation and three-arch verification), this doc has had no
|
|
implementation pass yet to surface and resolve open questions against.
|
|
|
|
## Open questions (explicitly not resolved — flagged, not guessed at)
|
|
|
|
1. **`HERMES_HEAT_WINDOW_DEPTH` value.** Proposed 16 (matching `CH-MAX`),
|
|
not derived from real observed churn data. Needs a real implementation
|
|
pass with instrumented boot logs (same discovery method VM-fleet used:
|
|
`doe_log.c`'s per-VM heat CSV columns exposed the dead-flat-trajectory
|
|
bug that led to VM-fleet's rev-f seed-value fix) before treating this
|
|
as settled.
|
|
2. **The concrete fit function** for turning `(live_count,
|
|
reaped_since_last)` samples into `msg_decay_slope_q48`. Section 4
|
|
deliberately stops short of a formula. A first candidate worth
|
|
prototyping: target reap-latency-in-cool-sweeps, adjust the slope
|
|
proportionally to the ratio of observed to target latency — but this
|
|
is a starting hypothesis to implement and test against, not a
|
|
specified design.
|
|
3. **`reaped_since_last` bookkeeping mechanism** — where the reap-count
|
|
plumbing lives and how it's reset per sample window.
|
|
4. **Whether messages and channels genuinely need independent slopes**,
|
|
or whether observed data shows one shared `msg_decay_slope_q48`
|
|
suffices — section 2 assumes independence as this doc's own
|
|
conservative default (matching the pattern word-level and VM-fleet
|
|
physics each use their own independently-fit rate), but this should
|
|
be revisited once real data exists.
|