AERO / BENCHMARK REGISTER

What was tested, and what did it prove?

External measurements test physical predictions. Local-model trials test guarded workflow behavior. These are separate claims.

EXTERNAL CFD / CHALLENGE 01

FAIL

Blind Validation Challenge 01

NASA TM-4074 Table XIII: 18 angles × 3 grids. Frozen criteria and input-only execution code preceded the production runs. 54 / 54 attempts finished in the published snapshot. Reference-withheld execution; supervisory setup was not fully blinded.

LOCAL-MODEL / OPERATOR HANDOFF

MIXED

Model-to-CFD workflow trials

Seven iterative airfoil attempts and the internal-flow model trials are retained, including rejected proposals. A local 7B model completed a solver run with operator help but failed strict observation; two bounded 14B workflows passed.

This tests proposal and evidence-reading behavior, not agreement with independent CFD measurements.

THERMAL + STRUCTURAL / SOURCE-DERIVED

SYNTHETIC PREBENCHMARK

Samarmad-derived PCHE gate pack

A separate six-point synthetic shakedown of prediction freeze, criteria hashing, curved-interface transfer, generic OpenFOAM CHT, and generic CalculiX gates.

Synthetic pipeline proof only; not the publication-specific run.

THERMAL-HYDRAULIC / PUBLICATION-TARGETED

NUMERICAL FAIL · EXPERIMENT UNSCORED

Samarmad PCHE v1 — independent reconstruction

Three OpenFOAM CHT grids ran on the reconstructed Model 6 cell. Every grid misses the residual target, and medium-to-fine pressure drop shifts by more than 22%. The article was reviewed after freeze, but no exact matching experimental point could be established; v1 is not tuned.

The paper has no structural test. Exploratory CalculiX mapping failed its 100% coverage gate. The prediction hash is verified; external timestamp confirmation remains pending.

Register policy

Each future CFD, thermal or structural benchmark receives a versioned entry, source citation, frozen criteria, outcome, and comparison history. Failed and incomplete trials remain listed. Workflow scores are not presented as physical-validation scores.

Return to all evidence →