AERO / BENCHMARK REGISTER

What was tested, and what did it prove?

External measurements test physical predictions. Local-model trials test guarded workflow behavior. These are separate claims.

LOCAL AI INFERENCE · SEPTEMBER 24, 2026

MEASURED · LIMITED SCOPE

RTX 3090 vs GB10

Same-model latency, context scaling, practical-model trials and cache controls. Five measured repeats per completed configuration. Includes charts and downloadable aggregates; no actual CFD/FEA performance or hardware purchase winner is claimed.

ALEX BLYTHE · SEPTEMBER 21, 2026 · CURRENT HX-CFD AUDIT

QUALIFICATION STILL FAILED

Acceptance repair and retained-evidence triage completed

40 behavior tests, 36 topology-verified field comparisons, native mass/energy/interface recovery, and explicit remaining numerical and contract blockers. No new CFD; historical evidence remains unchanged.

THERMAL SYSTEMS / PROGRAM REPORT

EVIDENCE RELEASE · VALIDATION INCOMPLETE

Heat-exchanger benchmark program

HX-S01 is a source-informed synthetic CHT study that failed its frozen numerical gates. HX-B01 adds a separately disclosed analytical correlation comparison that failed by 30.86%, and five synthetic companions exercise coupling, fidelity, sensitivity, trade and transfer paths. Blind and structural candidates remain source-gated; no experimental accuracy, structural accuracy, or design readiness is claimed.

EXTERNAL CFD / CHALLENGE 01

FAIL

Blind Validation Challenge 01

NASA TM-4074 Table XIII: 18 angles × 3 grids. Frozen criteria and input-only execution code preceded the production runs. 54 / 54 attempts finished in the published snapshot. Reference-withheld execution; supervisory setup was not fully blinded.

LOCAL-MODEL / OPERATOR HANDOFF

MIXED

Model-to-CFD workflow trials

Seven iterative airfoil attempts and the internal-flow model trials are retained, including rejected proposals. A local 7B model completed a solver run with operator help but failed strict observation; two bounded 14B workflows passed.

This tests proposal and evidence-reading behavior, not agreement with independent CFD measurements.

THERMAL + STRUCTURAL / SOURCE-DERIVED

SYNTHETIC PREBENCHMARK

Samarmad-derived PCHE gate pack

A separate six-point synthetic shakedown of prediction freeze, criteria hashing, curved-interface transfer, generic OpenFOAM CHT, and generic CalculiX gates.

Synthetic pipeline proof only; not the publication-specific run.

THERMAL-HYDRAULIC / PUBLICATION-TARGETED

NUMERICAL FAIL · EXPERIMENT UNSCORED

Samarmad PCHE v1 — independent reconstruction

Three OpenFOAM CHT grids ran on the reconstructed Model 6 cell. Every grid misses the residual target, and medium-to-fine pressure drop shifts by more than 22%. The article was reviewed after freeze, but no exact matching experimental point could be established; v1 is not tuned.

The paper has no structural test. Exploratory CalculiX mapping failed its 100% coverage gate. The prediction hash is verified; external timestamp confirmation remains pending.

THERMAL-HYDRAULIC / SOURCE-INFORMED SYNTHETIC

NUMERICAL FAIL · EXPERIMENT NOT SCORED

HX-S01 — reconstructed counterflow PCHE

A disclosed, source-informed replacement tests first-principles screening and three-grid OpenFOAM CHT after the publication path could not establish a matched pointwise experimental target. Missing inputs are explicit assumptions; no invented measurements are treated as truth.

The frozen 250-iteration baseline remains failed. Same-settings continuations are diagnostic only and cannot retroactively pass it.

CFD / FAILURE INVESTIGATION · HARCC S4 PART II

NUMERICAL FAIL · INVESTIGATION COMPLETE

HARCC S4 methane cooling: solver-control investigation

A frozen v5 residual failure was investigated with bounded pressure-cap and outer-corrector comparisons. A setup mistake is explicitly excluded; the corrected treatment moves two target residuals down and one up. All target gates remain failed.

Reference-withheld initial model diagnosis; sequential follow-ups. No matched experimental validation, mesh independence or design readiness is claimed.

MODEL-BEHAVIOR / FRESH-CONTEXT COMPARISON

COMPLETE · NO SOLVER RUN

HARCC S4: Luna versus Sol investigator diagnosis

Two models independently inspected the same frozen v5 failure packet with later experiments and publication results withheld. Luna scored 17/18 and Sol-light 16/18 on the predeclared evidence-discipline rubric.

This compares bounded investigation behavior on one case. It is not a model-ranking claim and adds no CFD validation.

CONTEXT EFFICIENCY / MODEL-BEHAVIOR QUALIFICATION

INFRASTRUCTURE PASS · MODEL QUALIFICATION OPEN

CEP-01 — context, routing & portability

Revision 1.5 separates six claims—reporting, telemetry narration, engineering explanation, context compression, routing/escalation, and investigation routing—and compares full versus progressive context across seven retained evidence states.

The fixture harness records 42 comparisons and a 93.82% mean estimated payload reduction. It makes no universal model verdict and does not replace CFD/FEA evidence.

SYNTHETIC STUDY · EXPERIMENT NOT SCORED

HX-S01 — reconstructed counterflow PCHE

A disclosed, source-informed replacement tests first-principles screening and three-grid OpenFOAM CHT where the publication path could not establish a matched pointwise experimental target. Missing inputs are explicit assumptions; no invented measurements are treated as truth.

The failed 250-iteration baseline remains frozen. Same-settings continuations are reported separately and cannot retroactively pass it.

THERMAL-HYDRAULIC / BLIND INVESTIGATOR

BLOCKED PENDING PROPERTY RUNTIME

HX-02B — sodium / supercritical-CO₂ PCHE

GPT-5.6 Luna received only the frozen visible packet and chose an analytical-only path. The operating point, idealized measurement planes, and active-area convention are now explicit; the independent supercritical-CO2 property runtime is the remaining gate.

No solver was dispatched and the sealed experimental truth was not revealed. This is a property-runtime stop, not a solver failure.

Register policy

Each future CFD, thermal or structural benchmark receives a versioned entry, source citation, frozen criteria, outcome, and comparison history. Failed and incomplete trials remain listed. Workflow scores are not presented as physical-validation scores.

Return to all evidence →