// Moved from docs/src/architecture-internals/MESSAGING.adoc to docs/working/scratch/src/architecture-internals/MESSAGING.adoc on 2026-06-16 (docs reorg Phase 2)
== 1. Overview
:toc: left
:toc-title: Contents
:toclevels: 3
xref:../README.adoc[← Back to Documentation Index]



This document defines the *thermodynamic model for the StarshipOS
Messaging Field*, a formal framework describing the behavior of message
propagation and processing as a physical system.

The messaging subsystem is not treated as a passive bus. Instead, it is
conceptualized as a *field* in which discrete *messages* behave
analogously to quanta (photons), and *words* (the fundamental executable
units of the StarshipOS runtime) behave as *particles* or loci of
interaction.

This model provides a unified basis for:

* *Prioritization & Scheduling*
* *Credit Allocation and Flow Control*
* *Thermal Regulation of System Load*
* *Entropy-Driven Policy and ML Feedback*
* *Instrumentation of Message Dynamics*

'''''

== 2. Conceptual Model

=== 2.1 Message Field

The *Message Field* is the logical substrate mediating all communication
within StarshipOS. It consists of:

* *Endpoints*: Boundaries between word spaces and the field.
* *Channels*: Directed pathways through which messages propagate
(virtual channels, pub/sub topics, RPC routes).
* *Field Properties*: Dynamic parameters such as message density,
propagation latency, and entropy.

The field is *not centralized*. It emerges from the coordinated behavior
of `+sf_msg-srv+` (broker/router), `+sf_msg-drv+` (ABI provider), and
`+libsfmsg+` (participant library), operating over shared memory rings
and control IPC.

'''''

=== 2.2 Messages

Messages are modeled as *energy quanta* with the following core
properties:

[cols=",",options="header",]
|===
|Property |Description
|Energy |Encodes priority, TTL, QoS, and thermodynamic state
|Momentum |Routing vector through the field
|Cross Section |Probability of absorption by a word
|Entropy |Thermodynamic measure of the flow’s state space
|===

Messages can be *absorbed*, *emitted*, or *scattered* by words and
services, analogous to photon–atom interactions.

'''''

=== 2.3 Words as Particles

Each *word* is treated as a *particle* in the field — a locus of
interaction rather than an isolated routine. Words have intrinsic
properties:

[cols=",",options="header",]
|===
|Property |Meaning
|Mass |Computational cost or complexity
|Charge |Degree of side-effect or state mutation
|Cross-Section |Message types and classes to which the word responds
|===

When a message reaches a word, the interaction may *excite* the word
(trigger execution), *alter its state*, or __ __pass through
unabsorbed**. Executed words may **emit new messages**, either
deterministically or in a burst (stimulated emission).

'''''

== 3. Mathematical Formulation

=== 3.1 Maxwellian Entropy

The system adopts a *Maxwell–Boltzmann entropy model*, treating each
endpoint/channel as a thermodynamic micro-system.

For a given flow ( f ):

{empty}[ S_f(t) = k ( 1 + )]

Where:

* ( ^2_\{t}(f) ): Variance of inter-arrival intervals
* ( Q_\{}(f) ): Normalized queue occupancy (0–1)
* ( U_\{}(f) ): Ratio of credits consumed to granted
* ( R_\{}(f) ): Number of delivery paths (fan-out)
* ( T_\{}(f) ): Normalized TTL of messages in the flow
* ( k ): Scaling constant (typically 1)

( S_f ) represents the *thermodynamic entropy* of the flow. High entropy
indicates unpredictable, high-energy flows approaching saturation. Low
entropy indicates stable, structured behavior.

=== 3.2 System Temperature and Pressure

* *Temperature (Θ)* is the average energy of messages within the field,
weighted by density.
* *Pressure (Π)* represents backlog intensity: [ = _f Q_\{}(f) U_\{}(f)]
* *Entropy (Σ)* is the aggregate system entropy: [ = _f S_f]

These quantities provide real-time metrics for system health and are
suitable as inputs for control algorithms.

'''''

== 4. Header and Data Structure Extensions

Messages carry *thermodynamic metadata* in addition to conventional
routing and control information.

=== 4.1 Thermodynamic Header Extension

[source,c]
----
struct sfm_hdr_thermo {
    uint16_t energy_q8;    // Encoded priority/TTL/entropy composite (fixed point)
    uint16_t entropy_mx;   // Maxwellian entropy (Q8.8 fixed-point)
    uint16_t temp_hint;    // Optional producer-side temperature hint
    uint16_t reserved;
};
----

* `+energy_q8+` is a composite scalar derived from QoS class, TTL, and
policy weighting.
* `+entropy_mx+` is computed at the sender or by the broker to
characterize flow state.
* `+temp_hint+` allows producers to indicate expected burstiness or
instability.

=== 4.2 Endpoint State

Endpoints maintain thermodynamic state variables:

[source,c]
----
struct sfm_endpoint_state {
    float entropy_current;
    float temperature;
    float pressure;
    float credit_utilization;
    uint64_t last_refresh_tsc;
};
----

This state is updated continuously during message transfer and
scheduling.

'''''

== 5. Scheduler and Credit System Integration

=== 5.1 Credits as Energy Quanta

Credits represent the *available energy budget* for message emission
along a channel.

* *High-entropy flows* receive smaller, more frequent credit refreshes
(tight regulation).
* *Low-entropy flows* receive larger, batched credits (looser
regulation).

Credit recycling policies (immediate, piggyback, or batched) are
selected dynamically based on the flow’s entropy and temperature.

'''''

=== 5.2 Thermodynamic Scheduling

Scheduling priority is a function of QoS class, deadline, and entropy:

{empty}[ P_\{sched} = f(, , S_f)]

* Low-entropy, real-time flows are scheduled deterministically.
* High-entropy, bulk flows are throttled or coalesced.
* Aging and anti-starvation mechanisms apply within entropy bands.

This approach prevents high-entropy flows from destabilizing the system,
while ensuring low-entropy flows achieve predictable latency.

'''''

== 6. Thermodynamic Monitoring and Control Loop

A *control loop* runs in `+sf_msg-srv+`, maintaining field stability:

[arabic]
. *Sampling* — Entropy, temperature, and pressure are sampled
periodically from endpoints and channels.
. *Aggregation* — System-wide Σ, Θ, Π are computed.
. *Policy Evaluation* — Control laws or ML bandits adjust credit
windows, scheduling weights, and routing based on sampled state.
. *Actuation* — The scheduler and credit allocator apply the updated
parameters.

This loop can operate using fixed thresholds or adaptive policies. ML
components treat entropy as the *order parameter * for optimization.

'''''

== 7. Implications for Forth and StarshipOS Runtime

In StarshipOS, *words are the fundamental execution loci*. By modeling
them as particles interacting via message photons, the runtime gains:

* *A unified abstraction for computation and communication*
* *Fine-grained control over execution dynamics through entropy*
* *A natural basis for prioritization without ad-hoc heuristics*
* *Thermodynamic instrumentation for debugging and analysis*

This model integrates cleanly with the existing Forth execution model,
since words are already explicit entities with well-defined entry
points.

'''''

== 8. Future Work

[arabic]
. *Formal Policy Language* Define a compact language for specifying
entropy-based routing and scheduling policies.
. *Distributed Thermodynamic Fields* Extend the model to multi-node
topologies, where message photons propagate across network transports.
. *Entropy-Driven GC & Memory Tiering* Couple message entropy metrics
with VM memory placement decisions.
. *Visualization* Develop tooling to visualize message flow as a dynamic
field, aiding in debugging and optimization.

'''''

== 9. APPEXDIX

....
/*
* StarshipOS - Messaging Thermodynamics
* File: include/sfm_thermo.h
* Status: Draft / Reference Only
*
* This header defines the public structures and APIs for the
* Maxwellian entropy model applied to StarshipOS messaging flows.
* It is intended as a design reference, not production code.
*
* Language: C99
* License: CC0 1.0 / Public Domain
  */

#ifndef SFM_THERMO_H
#define SFM_THERMO_H

#include <stdint.h>
#include <stddef.h>

#ifdef __cplusplus
extern "C" {
#endif

/* =============================================================
* Fixed-point helpers (Q8.8 and Q16.16 formats)
* =========================================================== */

/** Convert float → Q8.8 fixed point. */
static inline uint16_t sfm_fp_to_q8_8(float x) {
if (x <= 0.0f) return 0u;
float v = x * 256.0f;
if (v > 65535.0f) v = 65535.0f;
return (uint16_t)(v + 0.5f);
}

/** Convert Q8.8 fixed point → float. */
static inline float sfm_q8_8_to_fp(uint16_t q) {
return ((float)q) / 256.0f;
}

/** Convert float → Q16.16 fixed point. */
static inline uint32_t sfm_fp_to_q16_16(float x) {
if (x <= 0.0f) return 0u;
double v = (double)x * 65536.0;
if (v > 4294967295.0) v = 4294967295.0;
return (uint32_t)(v + 0.5);
}

/** Convert Q16.16 fixed point → float. */
static inline float sfm_q16_16_to_fp(uint32_t q) {
return ((float)q) / 65536.0f;
}

/* =============================================================
* Thermodynamic Header Extension
* =========================================================== */

/**
* Additional header fields attached to each message.
* These encode thermodynamic properties derived from the
* flow's state and scheduling context.
*
* All fields are fixed-point Q8.8 unless otherwise stated.
  */
  struct sfm_hdr_thermo {
  uint16_t energy_q8;    /**< Composite of priority, TTL, entropy.       */
  uint16_t entropy_mx;   /**< Maxwellian entropy estimate (per-flow).    */
  uint16_t temp_hint;    /**< Optional producer hint for burstiness.    */
  uint16_t reserved;     /**< Align to 8 bytes.                         */
  };

/* =============================================================
* Per-flow Thermodynamic State
* =========================================================== */

/**
* Credit refresh policy hints returned to allocator.
  */
  typedef enum {
  SFM_CREDIT_IMMEDIATE = 0, /**< Refresh credits on every recycle. */
  SFM_CREDIT_PIGGYBACK = 1, /**< Return credits with next outbound. */
  SFM_CREDIT_BATCHED   = 2  /**< Batch refresh until watermark.    */
  } sfm_credit_mode_t;

/**
* Per-flow thermodynamic state, maintained by sf_msg-srv.
* All quantities are continuously updated during operation.
  */
  typedef struct {
  /* --- Inter-arrival statistics (Welford) --- */
  uint64_t last_arrival_us;
  double   iat_mean;
  double   iat_M2;
  uint64_t iat_count;

  /* --- Queue occupancy EWMA --- */
  double q_occ_ewma;
  double q_alpha;

  /* --- Credit utilization EWMA --- */
  uint64_t credits_granted_total;
  uint64_t credits_consumed_total;
  double   credit_util_ewma;
  double   credit_alpha;

  /* --- Fanout & TTL tracking --- */
  double fanout_ewma;
  double ttl_ms_ewma;

  /* --- Derived thermodynamic quantities --- */
  double entropy_mx;   /**< Maxwellian entropy S_f */
  double temperature;  /**< Average energy proxy Θ */
  double pressure;     /**< Backlog pressure proxy Π */

  /* --- Policy thresholds --- */
  double wm_low;
  double wm_high;
  double entropy_cool;
  double entropy_hot;

  /* --- Last update timestamp --- */
  uint64_t last_update_us;

} sfm_flow_state_t;

/**
* Snapshot of field-wide thermodynamic state.
* Useful for monitoring and control loops.
  */
  typedef struct {
  double system_entropy_sum;
  double system_temperature_sum;
  double system_pressure_sum;
  uint64_t flows_count;
  } sfm_field_snapshot_t;

/**
* Credit allocation decision returned to the scheduler.
  */
  typedef struct {
  sfm_credit_mode_t mode; /**< Suggested refresh mode. */
  uint32_t grant_quanta;  /**< Number of descriptors to grant. */
  uint8_t qos_hint;       /**< Optional QoS class hint (255 = no change). */
  } sfm_thermo_decision_t;

/* =============================================================
* API Prototypes
* =========================================================== */

/**
* Initialize a flow state structure with sane defaults.
*
* @param st             Pointer to the state structure.
* @param q_alpha        EWMA alpha for queue occupancy.
* @param credit_alpha   EWMA alpha for credit utilization.
* @param wm_low         Low watermark for batching (0–1).
* @param wm_high        High watermark for backpressure (0–1).
* @param entropy_cool   Entropy threshold for loosening control.
* @param entropy_hot    Entropy threshold for tightening control.
  */
  void sfm_thermo_init_flow(sfm_flow_state_t *st,
  double q_alpha,
  double credit_alpha,
  double wm_low,
  double wm_high,
  double entropy_cool,
  double entropy_hot);

/**
* Update flow state when a message is enqueued or processed.
*
* @param st         Flow state.
* @param now_us     Current monotonic timestamp in microseconds.
* @param q_norm     Current normalized queue occupancy [0..1].
* @param credits_g  Delta of credits granted since last update.
* @param credits_c  Delta of credits consumed since last update.
* @param fanout     Observed fanout for this message.
* @param ttl_ms     Message TTL in milliseconds.
  */
  void sfm_thermo_on_enq(sfm_flow_state_t *st,
  uint64_t now_us,
  double q_norm,
  uint32_t credits_g,
  uint32_t credits_c,
  double fanout,
  double ttl_ms);

/**
* Sample the current state and compute a scheduling / credit
* allocation decision. This is invoked periodically or on events.
*
* @param st               Flow state.
* @param batch_watermark  Recycle count watermark for batching.
* @param max_grant        Safety cap on credit grants.
* @return                 Suggested decision structure.
  */
  sfm_thermo_decision_t
  sfm_thermo_sample_and_decide(sfm_flow_state_t *st,
  uint32_t batch_watermark,
  uint32_t max_grant);

/**
* Aggregate a set of flow states into a single field snapshot.
*
* @param flows   Array of flow states.
* @param n       Number of elements in `flows`.
* @param out     Output snapshot structure.
  */
  void sfm_thermo_aggregate(const sfm_flow_state_t *flows,
  size_t n,
  sfm_field_snapshot_t *out);

/**
* Populate a thermodynamic header extension for a message.
*
* @param st         Flow state.
* @param h          Pointer to header extension struct.
* @param energy_q8  Precomputed composite energy (Q8.8).
  */
  static inline void
  sfm_thermo_fill_hdr(const sfm_flow_state_t *st,
  struct sfm_hdr_thermo *h,
  uint16_t energy_q8)
  {
  if (!st || !h) return;
  h->energy_q8  = energy_q8;
  h->entropy_mx = sfm_fp_to_q8_8((float)st->entropy_mx);
  h->temp_hint  = sfm_fp_to_q8_8((float)st->temperature);
  h->reserved   = 0;
  }

#ifdef __cplusplus
}
#endif

#endif /* SFM_THERMO_H */
....

== 10. References

* Maxwell, J.C., _Illustrations of the Dynamical Theory of Gases_, Phil.
Mag. 1860.
* Boltzmann, L., _Weitere Studien über das Wärmegleichgewicht unter
Gasmolekülen_, 1872.
* StarshipOS Internal Messaging Architecture Specifications.

'''''

*End of Document*

'''''
