We ran Steriod against the ESA Anomaly Detection Benchmark: two real spacecraft, 1.39 billion telemetry samples, about 500 expert-labelled events. We used only mined, human-readable rules and statistical models: no training, no labels in any loop, re-mining walk-forward year by year, and every score produced by ESA's own published evaluation code.
Self-reported on ESA's public labels and scored with ESA's own evaluation code. Our walk-forward protocol differs from the paper's train/test split; supervised methods in other setups score higher. Dataset: ESA Anomaly Dataset, ESA/ESOC, Airbus DS, KP Labs, CC-BY 4.0, DOI 10.5281/zenodo.12528695.
The benchmark paper lists operator requirements next to the dataset. The top complaint from spacecraft operations engineers was too many false positives, because false alarms decrease adoption and trust. Among the requirements: binary verdicts, a list of affected channels, no repeated alarms for one event, rare nominal events (often commanded) told apart from anomalies, online operation, and a single high-end PC. Steriod is built around exactly that list.
Two missions, 14 years of Mission 1 telemetry (611 million samples) and Mission 2, 1.39 billion samples in total, with telecommands and about 500 labelled anomalies and rare events.
Labels only as the exam, never as input. No detector reads the future. Every attempt counted, every scoring run pre-registered, nine failures documented.
One laptop. No GPU. No language-model tokens in detection: every threshold is a mined, readable number with its derivation.
Event-wise (EW) scores over all labelled events, against the strongest published methods. The full tables, with every method and metric, are in our engineering blog series.
| Metric | Telemanom-ESA | DC-VAE (5) | Telemanom-ESA-Pruned (best published) | Steriod |
|---|---|---|---|---|
| EW precision | .148 | .063 | .999 | .892 |
| EW recall | .894 | .338 | .424 | .539 |
| EW F0.5 | .178 | .075 | .786 | .788 |
| Alarming precision | .868 | .028 | .875 | 1.000 |
| Metric | Telemanom-ESA | DC-VAE (5) | Telemanom-ESA-Pruned (best published) | Steriod |
|---|---|---|---|---|
| EW precision | .007 | .005 | .050 | .275 |
| EW recall | .946 | .804 | .870 | .791 |
| EW F0.5 | .008 | .007 | .061 | .316 |
| Alarming precision | .074 | .017 | .206 | .973 |
| Subsystem F0.5 | .689 | .623 | .436 | .680 |
A real disturbance fragments into many short alarms, and every fragment is billed as a false interval or a redundant alarm. Merging silent gaps under 24 hours (a horizon taken from the alarm stream's own gap histogram, with no label consulted) turned 3,911 spans into 123 episodes. Recall is preserved; alarming precision went from 0.32 to 0.97.
Commanded operations look like anomalies, so we mine command-to-effect links from the archive. In an early frozen run, a link mined years before explained away a real anomaly (id_116): the command had stopped appearing, but the link still acted as a permanent alibi.
Since then, a mined link only annotates the alarm with its proposed explanation. Suppression requires a human-accepted link. Dangerous suppressions are zero by construction, not by luck.
On Mission 2, one subsystem moved to a new operating regime. Our mining refuses to learn an unacknowledged shift, so those channels keep alarming until a person accepts the new baseline. That is the behaviour you want: a detector that adapts silently would absorb a slow real anomaly the same way. In a labelled simulation, three operator "accept new baseline" decisions lifted the full-set F0.5 from 0.307 to 0.579.
The same detector chain now runs as a live pipeline and reproduces the batch results (one fold differed at first and was fixed). The episodes timeline on our monitoring page is that pipeline.
A command-to-response map found two rare events nothing else reached, with only 14 alarm intervals in seven years. Statistical lanes stay advisory unless they earn alarm status, which keeps precision high.
A Fourier noise-band detector with a three-window consensus: we predicted a mean per-fold F0.5 of 0.8469 over the seven walk-forward years and measured 0.8470. That is a mean of per-year scores, not comparable to the pooled scores above.
Six-week Qualified Monitoring pilot: blind replay over one year of your archive, scored against your own anomaly list afterwards, never trained on it.