Methodology

Benchmark methodology

This page is the canonical methods reference for the public Evidence Portal. Numbers on problem pages are produced by the same governed harness used in Resonix lab validation. If a metric is missing, the reason is stated — we do not imply precision we have not measured.

It is also a vendor benchmark. We designed the harness, chose the problems, configured the baselines, and ran everything on our own machines. Publishing the method in this much detail is how we make that self-interest checkable — not a substitute for someone else checking it.

Verification status

Nothing in this portal has been independently verified. There is no third-party audit, no external replication, no peer review, and no certification behind any number on these pages. Everything is self-reported: internally generated evidence, published so it can be interrogated. It is not proof, and it should not be cited as though a neutral party had signed off on it.

What you can check yourself

The honest way to treat a vendor benchmark is to reproduce the part that matters to you:

• Baselines are public. The 13 comparison arms are SciPy 1.14 and standard open-source integrators. You can run them on your own hardware without us.
• Problems are named. Each page carries the literature identity of the system — textbook source, frozen parameters, initial state, horizon — so you can rebuild the ODE and compare against our medians.
• Your problem beats our problem. Profile your own ODE with free SolvScout, then run SolvSRK on it under a 30-day trial. A result you measured yourself outranks anything we publish.
• Ask for the reproduction package. For evaluations under NDA we will provide problem definitions, solver configurations, and per-seed rows for a scoped set of cells so your team can re-run them. We would rather be checked than believed.

Solver arms compared

Every problem is run against 14 solver arms: SolvSRK plus six SciPy standalones and seven external reference integrators. On clean runs all 14 compete; under injected noise the external integrators are excluded (they cannot reproduce the fixed-path noise model), leaving SolvSRK versus the six SciPy arms.

How we determine accuracy

Two questions, two metrics. Do not conflate them.

• Survival — Did the integrator finish the horizon with a finite state? (t_reached / t_target ≥ 0.99). Default headline.
• SCD — Significant correct digits vs an independent high-fidelity reference at terminal time:−log₁₀(‖y − y_ref‖ / ‖y_ref‖). Shown only when a verified reference exists.

Reference independence: the reference run uses a different solver configuration than the arm under test (typically tighter-tolerance CVODE/Radau). SolvSRK is never its own reference.

Cost is reported as median function evaluations (nfev) and median wall time on reference lab hardware. Wall times are indicative — survival and SCD are the primary scientific metrics.

The four objectives

Every (problem, noise) cell is ranked four different ways, because “best solver” depends on what you care about. Each problem page lets you switch between all four:

• Balanced — the overall trade-off of survival, precision, and speed (the default view).
• Survival — ranked purely on completing the integration without blowing up.
• Precision — ranked on significant correct digits (SCD) at the horizon.
• Efficiency — ranked on wall-clock time to reach the horizon.

This is the same objective set SolvScout / SolvTune exposes for free — we do not hide the cases where a different arm wins a given objective.

Noise models

Binding sensor noise is applied on RHS evaluation (not post-hoc on states unless otherwise noted). Every problem is measured clean and across five additive-Gaussian noise levels: σ ∈ {1e−5, 1e−3, 1e−2, 0.1, 1}. SolvSRK's headline advantage appears in the mid-to-high σ band, where several SciPy arms drop to 0% survival.

What the verdicts mean

Each problem carries one verdict. It answers a single question — should you reach for SolvSRK on this class of problem? Generated catalog verdicts compare the balanced objective at σ=0.1 when available, then the next available noisy level, then clean. The comparison is always against the best competing arm, never an absolute grade. Editorially highlighted problems may add clean or objective-specific context in their callout. Dead-zone rows are explicitly scoped to the named regime.

Survival is evaluated first. A gap of at least 10 percentage points is decisive. If survival is tied, a balanced-score gap of at least 0.05 is decisive. Anything inside both thresholds is parity. This prevents a fractional-digit precision difference from being presented as a broad win.

Publication limits (Phase 1)

We publish methodology, problem metadata, and per-arm median survival / SCD / nfev / wall across all four objectives and six noise levels — browsable one problem at a time. We do not offer a bulk ZIP / JSON dump of the corpus, raw per-seed trial rows, RHS definitions, or SolvSRK's internal algorithm. To fingerprint your ODE against the full corpus, register for free SolvScout; to run SolvSRK yourself, start a 30-day trial.

Default seeds: 20 per cell. Freeze: 2026-08-13. Scope: TRL 4–5 · simulation-lab validated.

@misc{resonix_evidence_portal_2026,
  title        = {Resonix Public Evidence Portal: ODE Benchmark Methodology and Results},
  author       = {{Resonix Labs (Canada) Inc.}},
  year         = {2026},
  howpublished = {\url{https://resonix.tech/evidence/methodology}},
  note         = {Self-reported vendor benchmark; internally generated by Resonix Labs and not independently verified. Accessed YYYY-MM-DD. Freeze 2026-08-13; libsolvsrk 2.3.0; SciPy 1.14.}
}