<!-- Moved from docs/03-architecture/adaptive-systems/multivariate-dynamics.md to docs/working/architecture/03-architecture/adaptive-systems/multivariate-dynamics.md on 2026-06-16 (docs reorg Phase 2) -->
# StarForth Multivariate Coupled Dynamics Experiment
## Phase 2: Golden Configuration Discovery via Stability Fingerprinting

**Status**: Design Phase
**Date**: 2025-11-20
**Paradigm**: Multivariate systems analysis (not factorial DOE)

---

## 1. Fundamental Insight

This is **not** a traditional DOE. We are measuring a **coupled dynamical system** where:

Every metric is **simultaneously**:
- A **signal** (measurable outcome)
- A **feedback variable** (influences future state)
- Part of a **field** (high-dimensional attractor landscape)

### The Coupling Web (Per Tick)
```
cache_hit%  ──┐
              ├──> bucket_load ──> heat ──> decay ──> window_width ──┐
bucket_hit% ─┘                                                       │
                                                                      ├──> prediction_accuracy
              ┌──────────────────────────────────────────────────────┘
              │
              ├──> cache_hit% (next tick feedback)
              │
              └──> heartbeat_interval (adaptive response to load)

load_intensity ──> word_access_pattern ──> heat_distribution ──> decay_effectiveness
                                                                      │
                                                                      └──> jitter_envelope
```

Each configuration produces a unique **trajectory** through metric-space. We don't measure "main effects." We measure **the shape of the trajectory**.

---

## 2. What We're Actually Measuring

### Per Tick (Milliseconds)
Each heartbeat tick (~1ms) captures the *current state* of the coupled system:

```c
typedef struct {
    /* === Temporal Anchor === */
    uint32_t tick_number;              /* Tick index within run */
    uint64_t tick_ns;                  /* Actual tick interval (ns) */
    uint64_t elapsed_ns;               /* Total elapsed since run start */

    /* === Metric Wave Functions === */
    uint32_t cache_hits_delta;         /* Cache hits this tick */
    uint32_t bucket_hits_delta;        /* Bucket hits this tick */
    double cache_hit_percent;          /* ψ_cache(t) */
    double bucket_hit_percent;         /* ψ_bucket(t) */

    /* === Feedback Signals === */
    uint64_t hot_word_count;           /* Words above heat threshold */
    double avg_word_heat;              /* Mean execution heat */
    uint32_t decay_estimate;           /* Inferred decay rate */
    uint32_t window_width;             /* Current window size */

    /* === Load Response === */
    uint32_t word_executions_delta;    /* Execution count this tick */
    uint32_t predictions_this_tick;    /* Speculative prefetch attempts */
    uint32_t prefetch_hits_delta;      /* Successful predictions */

    /* === Stability Measures === */
    double tick_jitter_ns;             /* Deviation from mean tick */
    double metric_stability_index;     /* Composite smoothness score */
} HeartbeatTickSnapshot;
```

### Per Run Summary Vector (After 100K+ ticks)
```c
typedef struct {
    /* === Convergence === */
    uint32_t convergence_time_ticks;   /* When steady-state reached */
    double cache_convergence_cv;       /* CV of cache% after convergence */
    double bucket_convergence_cv;      /* CV of bucket% after convergence */
    double decay_convergence_cv;       /* CV of decay estimate */
    double window_convergence_cv;      /* CV of window width */

    /* === Steady State === */
    double cache_hit_percent_steady;   /* Mean in steady state */
    double bucket_hit_percent_steady;  /* Mean in steady state */
    double avg_jitter_ns_steady;       /* Mean tick jitter (steady) */
    double hotword_ratio_steady;       /* Mean hot word count */

    /* === Dynamics === */
    double cache_rise_time_ticks;      /* Time to reach 95% of steady */
    double decay_rise_time_ticks;      /* Decay learning curve */
    double settling_time_ticks;        /* Total settle time */
    double overshoot_percent;          /* Peak overshoot if any */

    /* === Coupling Strength === */
    double load_to_heartbeat_corr;     /* correlation(load, tick_ns) */
    double load_to_cache_corr;         /* correlation(load, cache_hit%) */
    double heartbeat_to_cache_corr;    /* lag-aware correlation */
    double metric_correlation_matrix[7][7];  /* Full cross-correlation */

    /* === Spectral Properties === */
    double dominant_frequency_hz;      /* Primary oscillation frequency */
    double spectral_entropy;           /* Shannon entropy of spectrum */
    double dominant_eigenvalue;        /* Largest eigenvalue of cov matrix */

    /* === Stability Index === */
    double overall_stability_score;    /* 0-100: composite stability fingerprint */
    // Computed as:
    // (1/3)*jitter_score + (1/3)*convergence_score + (1/3)*coupling_score
} RunStabilityFingerprint;
```

### Per Configuration Summary (After 150+ runs)
```c
typedef struct {
    /* === Between-Run Statistics === */
    double stability_score_mean;       /* Mean across runs */
    double stability_score_std;        /* StdDev (low = reproducible) */
    double stability_score_cv;         /* Coefficient of variation */

    /* === Convergence Reliability === */
    double convergence_time_mean_ticks;
    double convergence_time_std_ticks;
    uint32_t convergence_success_rate; /* % of runs that converge */

    /* === Coupling Stability === */
    double load_heartbeat_corr_mean;
    double load_heartbeat_corr_std;
    int coupling_is_stable;            /* 1 if corr reliable across runs */

    /* === Attractor Basin Depth === */
    double eigenmode_decay_rate;       /* How fast perturbations die */
    double basin_depth_estimate;       /* Resistance to noise */

    /* === Jitter Envelope === */
    double tick_jitter_mean_ns;
    double tick_jitter_std_ns;
    double tick_jitter_cv;             /* Coefficient of variation */
    int jitter_is_white;               /* 1 if noise is Gaussian white */

    /* === Noise Propagation === */
    double metric_noise_coupling;      /* How noise spreads across metrics */
    double entropy_growth_rate;        /* d(entropy)/dt */

    /* === Configuration Signature === */
    int config_id;                     /* 1_0_1_1_1_0 encoded */
    char config_name[32];              /* "1_0_1_1_1_0" */
    int loop_1_enabled, loop_2_enabled, ...;
} ConfigurationFingerprint;
```

---

## 3. Experimental Design

### Parameters
```
Configurations:     5 elite (from Stage 1)
Runs per config:    150-200 (statistically significant for correlation)
Ticks per run:      ~100,000 (to measure convergence + steady state)
Total heartbeat ticks: 5 × 150 × 100,000 = 75M ticks (~1.25 hours per config)
Total experiment:   ~6-8 hours wall-clock
```

### Test Matrix
- **Randomized run order** across all 750 runs (avoid temporal confounds)
- **Interleaved configs** to minimize thermal/memory state drift
- **Background tasks minimized** (dedicated CPU for heartbeat)

### Sampling Strategy
- **Every tick captured** (no subsampling—need time-series fidelity)
- **Per-run fingerprint computed** from time-series
- **Per-config fingerprint computed** from 150+ run fingerprints

---

## 4. Analysis Pipeline

### Phase 1: Per-Run Fingerprinting (Immediate)
For each run's 100K-tick time series:

1. **Detect convergence**: When ψ_cache(t) CV drops < 5%
2. **Compute steady-state metrics**: Mean, std, CV after convergence
3. **Extract dynamics**: Rise time, settling time, overshoot
4. **Measure coupling**:
   - correlation(load_intensity, heartbeat_ns) with lag search
   - correlation matrix of all 7 metrics
5. **Spectral analysis**: FFT of cache_hit% to find dominant modes
6. **Compute stability index**: Composite of jitter + convergence + coupling

### Phase 2: Per-Config Fingerprinting (After all runs)
For each config's 150+ fingerprints:

1. **Compute robustness**: std of stability_score across runs
2. **Assess convergence reliability**: % of runs with successful convergence
3. **Evaluate coupling strength**: mean(load↔heartbeat_corr), std
4. **Estimate attractor basin**: Eigenmode decay rate
5. **Characterize noise**: Jitter CV, entropy growth

### Phase 3: Golden Config Selection
Compare 5 configuration fingerprints on:

| Dimension | Weight | Ideal Behavior |
|-----------|--------|--|
| **Stability Score** | 30% | Highest mean, lowest std |
| **Convergence Reliability** | 25% | 100% convergence, low variability |
| **Coupling Strength** | 20% | Strong (0.8+) & stable load↔heartbeat correlation |
| **Attractor Basin Depth** | 15% | Fast eigenmode decay (resilient to noise) |
| **Jitter Smoothness** | 10% | Low CV, white noise (not colored) |

**Golden Config** = argmax(weighted_score) over 5 configs

---

## 5. CSV/Data Format

### Primary Output: Per-Tick Time Series
```csv
run_id,config,tick_number,tick_ns,elapsed_ns,cache_hits_delta,bucket_hits_delta,cache_hit_percent,bucket_hit_percent,hot_word_count,avg_word_heat,decay_estimate,window_width,word_executions_delta,predictions_this_tick,prefetch_hits_delta,tick_jitter_ns,metric_stability_index
1,1_0_1_1_1_0,1,1000000,1000000,45,23,89.2,91.5,8,234.5,0.33,4096,156,12,8,0,0.95
1,1_0_1_1_1_0,2,1001200,2001200,47,25,89.3,91.6,8,234.6,0.33,4096,158,11,9,1200,0.96
...
```
(100K rows per run × 750 runs = 75M rows total)

### Secondary Output: Per-Run Summary
```csv
run_id,config,convergence_time_ticks,cache_convergence_cv,bucket_convergence_cv,cache_hit_percent_steady,...,overall_stability_score
1,1_0_1_1_1_0,5234,0.032,0.028,...,82.4
2,1_0_1_1_1_0,5156,0.031,0.029,...,83.1
...
```

### Tertiary Output: Per-Config Summary
```csv
config,stability_score_mean,stability_score_std,convergence_time_mean_ticks,...,golden_rank
1_0_1_1_1_0,82.7,2.1,5200,...,1
1_0_1_1_1_1,79.3,4.5,6100,...,3
...
```

---

## 6. Implementation Roadmap

### Step 1: Heartbeat Instrumentation
- Extend `HeartbeatWorker` with circular buffer for per-tick snapshots
- Capture 7 key metrics every tick (minimal overhead)
- Stream snapshots to file or in-memory ring buffer

### Step 2: Run Fingerprinting
- Implement per-run analysis: convergence detection, CV computation, coupling measurement
- Generate `RunStabilityFingerprint` struct for each run
- Write R/Python code to compute eigenvalues, spectral properties

### Step 3: Config Fingerprinting
- Aggregate 150+ run fingerprints per config
- Compute robustness statistics
- Generate `ConfigurationFingerprint` for each of 5 configs

### Step 4: Golden Config Decision
- Weight and score 5 config fingerprints
- Identify winner by composite stability index
- Generate report with reasoning

### Step 5: Visualization & Reporting
- Time-series plots per run (cache%, bucket%, tick_ns over time)
- Correlation heatmaps (per config, per run)
- PCA projection of runs into first 3 principal components
- Stability score distributions (violin plots)
- Golden config justification report

---

## 7. Interpretation Guide

### What Each Metric Means

| Metric | Signal | Interpretation |
|--------|--------|---|
| **cache_hit%** | Feedback loop #1 effectiveness | Higher = better prediction |
| **bucket_hit%** | Secondary lookup efficiency | Validates hash function |
| **hot_word_count** | Decay loop output | More = hotter words concentrated |
| **avg_word_heat** | Decay state | Slopes indicate decay strength |
| **tick_ns variation (CV)** | Heartbeat stability | Lower CV = steadier timing |
| **convergence_time** | Inference speed | Faster = learns pattern quicker |
| **load↔heartbeat_corr** | Adaptive responsiveness | 1.0 = perfect coupling, 0 = deaf |
| **dominant_eigenvalue** | Attractor basin strength | Larger = deeper basin, more stable |
| **overall_stability_score** | Composite quality | 0-100, captures all dimensions |

### Interpreting Configuration Fingerprints

A "good" config fingerprint has:
- ✅ High stability_score_mean (>80)
- ✅ Low stability_score_std (<3)
confidence high = reproducible
- ✅ 100% convergence_success_rate (all runs stabilize)
- ✅ 0.8+ load_heartbeat_corr (tight coupling to load)
- ✅ Low tick_jitter_cv (<0.2, <20% variation)
- ✅ Large dominant_eigenvalue (deep attractor)

A "poor" config fingerprint has:
- ❌ Low stability_score_mean (<75)
- ❌ High stability_score_std (>5)
unstable = irreproducible
- ❌ <80% convergence_success_rate (some runs diverge)
- ❌ <0.6 load_heartbeat_corr (loose/noisy coupling)
- ❌ High tick_jitter_cv (>0.35, >35% variation)
- ❌ Small dominant_eigenvalue (shallow attractor, noise-prone)

---

## 8. Next Steps

1. **Instrument heartbeat loop** to capture per-tick snapshots
2. **Implement RunStabilityFingerprint** computation (convergence detection, correlations)
3. **Build fingerprint aggregation** for ConfigurationFingerprint
4. **Write R analysis suite**:
   - Per-run time-series visualization
   - Correlation heatmaps
   - PCA decomposition
   - Stability score violin plots
   - Golden config selection report
5. **Execute experiment** with 150-200 runs per config (5 × 150+ = 750 total)
6. **Analyze results** and select golden config

---

## References

- **Coupled dynamical systems**: Kuramoto model, coupled oscillators
- **Stability analysis**: Lyapunov exponents, attractor basin depth
- **Time-series analysis**: Spectral entropy, eigenmode decomposition
- **Systems biology**: Feedback loops, homeostasis under load
- **Control theory**: Load-to-output coupling, response time, settling time

---

**Captain, this is the right framework. Ready to build the harness.**