33 lines
1.5 KiB
Plaintext
33 lines
1.5 KiB
Plaintext
Artemis surface stress test -- hypothesis tests
|
|
=================================================
|
|
|
|
H1 -- pre-fix failure clustering (Wald-Wolfowitz runs test)
|
|
H0: pass/fail outcomes are independently ordered (no clustering).
|
|
Observed runs = 26 (expected under H0 = 26.00, sd = 3.50)
|
|
z = 0.000, p = 1.00e+00
|
|
-> fail to reject H0 at alpha=0.05: failures are not serially clustered.
|
|
|
|
H1b -- pre-fix outcome vs. trial/write order directly (Wilcoxon)
|
|
H0: trial index is drawn from the same distribution for PASS/FAIL
|
|
(complements H1: the runs test only catches consecutive-value
|
|
clustering, not a general rank-order trend).
|
|
W = 332.0, p = 0.715
|
|
-> fail to reject H0 at alpha=0.05: outcome is not associated with write order.
|
|
|
|
H2 -- pre-fix failure vs. disk location (Wilcoxon rank-sum test)
|
|
H0: logical block number is drawn from the same distribution
|
|
for PASS and FAIL trials (failure does not depend on location).
|
|
W = 505.0, p = 0.000
|
|
-> REJECT H0 at alpha=0.05: failure IS associated with LBN.
|
|
|
|
Secondary check -- sibling position (0/1/2) vs. outcome (chi-squared)
|
|
chi-sq = 0.790, df = 2, p = 0.674
|
|
-> fail to reject H0: outcome is not associated with sibling position.
|
|
|
|
H3 -- did the fix change the pass rate (two-proportion test)
|
|
H0: pre-fix and post-fix trials share the same true pass rate.
|
|
Pre-fix: 25/50 = 50.0% Post-fix (pooled, 3 arches): 150/150 = 100.0%
|
|
chi-sq = 81.20, df = 1, p = 2.04e-19
|
|
95% CI on the difference in pass rate: [-0.652, -0.348]
|
|
-> REJECT H0 at alpha=0.05: the fix DID CHANGE the pass rate.
|