87 lines
3.3 KiB
TeX
87 lines
3.3 KiB
TeX
%% SCRAP: experiments/02-experiments/heartbeat-doe/design
|
|
%% SOURCE: docs/working/experiments/02-experiments/heartbeat-doe/design.md
|
|
%% STATUS: WORKING
|
|
%% FITS: experiments/ch-heartbeat
|
|
%% EDITORIAL: lifted — prose rewritten to press voice
|
|
%% NOTE: This is the initial design document, superseded by corrected-design.tex.
|
|
%% Kept as the historical record of Stage 2 design intent.
|
|
|
|
\section{Heartbeat DoE — Initial Design}
|
|
\label{sec:heartbeat-doe-design}
|
|
|
|
\subsection{Overview}
|
|
|
|
The heartbeat design-of-experiments experiment constitutes Stage~2 of the
|
|
StarForth factorial campaign. Where Stage~1 (3{,}200 runs, six factors, no
|
|
heartbeat thread) measured static final-performance metrics, Stage~2
|
|
incorporates heartbeat observability to capture \emph{temporal stability}
|
|
characteristics: convergence speed, jitter envelope, load-response coupling,
|
|
and decay-slope tracking.
|
|
|
|
\subsection{Design Points}
|
|
|
|
Stage~2 focuses on the five elite configurations identified by Stage~1
|
|
analysis:
|
|
|
|
\begin{center}
|
|
\begin{tabular}{llp{6cm}}
|
|
\toprule
|
|
\# & Configuration & Key Characteristics \\
|
|
\midrule
|
|
1 & \texttt{1\_0\_1\_1\_1\_0} & Heat + Decay + Pipelining + Window Inference \\
|
|
2 & \texttt{1\_0\_1\_1\_1\_1} & Config~1 plus Decay Inference \\
|
|
3 & \texttt{1\_1\_0\_1\_1\_1} & Heat + Rolling Window + Pipelining + both Inferences \\
|
|
4 & \texttt{1\_0\_1\_0\_1\_0} & Heat + Decay + Window Inference (lean) \\
|
|
5 & \texttt{0\_1\_1\_0\_1\_1} & Rolling Window + Decay + both Inferences (no heat) \\
|
|
\bottomrule
|
|
\end{tabular}
|
|
\end{center}
|
|
|
|
\subsection{New Metrics}
|
|
|
|
Stage~2 extends the \texttt{DoeMetrics} struct with approximately 20
|
|
heartbeat-specific fields, raising the CSV column count from 38 to
|
|
approximately 60:
|
|
|
|
\begin{center}
|
|
\begin{tabular}{llll}
|
|
\toprule
|
|
Metric & Type & Purpose & Target \\
|
|
\midrule
|
|
\texttt{tick\_interval\_cv} & double & Jitter coefficient of variation & $< 0.15$ \\
|
|
\texttt{tick\_outlier\_ratio} & double & Fraction of ticks $> 3\sigma$ & $< 0.02$ \\
|
|
\texttt{decay\_slope\_convergence\_rate} & double & Ticks to slope convergence & $> 50$ \\
|
|
\texttt{load\_interval\_correlation} & double & Workload--heartbeat coupling & $> 0.75$ \\
|
|
\texttt{settling\_time\_ticks} & double & Ticks to 10\% settling band & $< 1{,}000$ \\
|
|
\texttt{overall\_stability\_score} & double & Composite 0--100 score & $> 75$ \\
|
|
\bottomrule
|
|
\end{tabular}
|
|
\end{center}
|
|
|
|
\subsection{Execution Plan}
|
|
|
|
Five configurations at 50 replicates each yield 250 total runs, randomised
|
|
into a single test matrix to eliminate ordering bias. The experiment is
|
|
designed to complete in 2--4~hours.
|
|
|
|
The composite stability score weights jitter, convergence, and load coupling
|
|
equally:
|
|
|
|
\[
|
|
S = \tfrac{1}{3}\bigl(S_{\text{jitter}} + S_{\text{convergence}}
|
|
+ S_{\text{coupling}}\bigr) \in [0, 100]
|
|
\]
|
|
|
|
The configuration with the highest~$S$ becomes the \emph{golden
|
|
configuration} for use as the MamaForth production baseline.
|
|
|
|
\subsection{Note on Subsequent Correction}
|
|
|
|
This initial design assumed a fixed $\sim$1~ms heartbeat interval and framed
|
|
the primary metric as jitter in a fixed tick. A subsequent design review
|
|
identified a critical gap: the heartbeat tick interval \texttt{tick\_ns} is
|
|
dynamically modulated by the inference engine. The corrected design measures
|
|
load-to-heartrate coupling rather than fixed-interval jitter.
|
|
See Section~\ref{sec:heartbeat-doe-corrected-design}.
|
|
|