Geometry → mesh
Nominal NACA 0012 geometry and a projected C-grid. Coarse, medium and fine counts are fixed in advance; meshes, warnings and decks are retained. An exact as-built airfoil is not claimed.
EXTERNAL BENCHMARK / CHALLENGE 01
RESULT: FAILPredict first. Reveal the measurements second.
NACA 0012 at Mach 0.15 and Reynolds number 5.95 million. 18 operating points, three meshes per point, no omitted angles. The reference stays sealed until a complete prediction has been committed.
54 / 54 case attempts finished
Published snapshot: 2026-09-14T14:08:21.197091+00:00. This is not a live counter. Finished attempts can fail numerical checks.
Actual chronology
- 1. Input frozenCOMPLETE
- 2. Criteria frozenCOMPLETE
- 3. SolveCOMPLETE
- 4. Prediction frozenCOMPLETE
- 5. Reference unsealedCOMPLETE
- 6. ComparisonCOMPLETE
Frozen result: FAIL
Numerical adequacy: FAIL. Experimental comparison: FAIL. A coefficient match cannot rescue failed numerical gates.
| Quantity | RMSE | Largest absolute error | Experimental gate |
|---|---|---|---|
| CD | 0.079659 | 0.232675 | FAIL |
| CL | 0.635532 | 1.420456 | FAIL |
The complete polar, including poor agreement.
Inspect convergence, conservation, wall resolution and actual meshes
The problem, without its answers.
NASA TM-4074, Table XIII, first subtable. Localized No. 180 grit strips at x/c = 0.05 on both surfaces. The full series spans −3.99° to 19.27°, including both near-zero points. The 36 targets are lift and drag coefficients across the polar, not a spatial pressure distribution.
What “blind” means here—and what it does not.
The deterministic solver processes cannot access the experimental reference. They receive the frozen generator and a fresh case directory inside network-disabled, non-root containers. They have no RAG, source paper, decryption key or inherited conversation.
The setup supervisor has seen the source and authored the numerical implementation. Source selection and supervisory setup are therefore not fully blinded. A separate input-only model review preceded the work, but it does not make this a double-blind experiment or exclude model pretraining familiarity. This record claims reference-withheld execution, not stronger independence.
The frozen numerical model
Steady RANS → forces
OpenFOAM 10 simpleFoam, incompressible low-Mach approximation, and the Langtry–Menter transition model. All angles receive 2,000 SIMPLE iterations. Iterations are not physical time.
Transition, turbulence and wall-treatment assumptions
A near-surface intermittency constraint approximates transition at 5% chord. It does not resolve grit height, strip drag or exact trip equivalence. Inlet turbulence intensity is assumed at 0.1%, not presented as a measured tunnel value. Fine-grid y+ must satisfy its declared gate; thin-cell warnings remain visible.
The original model review's suggestion that the whole series is physically steady is not adopted. This is a bounded steady-model test. Oscillating forces fail acceptance; a stable RANS fixed point still does not establish physical steadiness.
Acceptance bounds, set before production.
| Check | Frozen requirement |
|---|---|
| Experimental CL error | RMSE ≤ 0.10; maximum absolute error ≤ 0.20 |
| Experimental CD error | RMSE ≤ 0.005; maximum absolute error ≤ 0.010 |
| Medium → fine | |ΔCL| ≤ 0.02; |ΔCD| ≤ 0.002 at every angle |
| Fine wall y+ | 95th percentile ≤ 2; maximum ≤ 5 |
| Stable force history | Last-window means and ranges must both pass; cancelling oscillations cannot pass on averages alone |
| Residuals and conservation | Initial/final residual gates plus boundary-flux imbalance ≤ 0.1% |
| Coverage | All 18 angles and all three meshes retained, including failed cases |
These are declared engineering utility bounds, not experimental confidence intervals. Per-point experimental uncertainty is unknown and cannot be used to widen thresholds after reveal.
Scientific commitments
| Artifact | SHA-256 |
|---|---|
| input.json | 827bb840cce7068cd09c0a292aea4efcd73a8d046c18c7e7eb1751fbefce3c73 |
| reference.enc | bc70021062c16133cef3816ddea98a6e9fe54a60c1908ccdf7d03f60bada0184 |
| preregistration.json | d82c98b1f835a52b88c7f355e738253741cb2aa0ae3216913cdad1993c985c6c |
| blind_prediction.json | 7b5e04d12e2e0be0aaefdec2eb29c1a9728f6f0ddbb90f2d6a72160742638e31 |
| comparison.json | f165ced785f6a23604e579a5b64b71261d80b78fd3c29b51c04befd45f069b46 |
Public commit anchors
- CRITERIA_FROZEN_ANCHORED:
0fb460ae948d0a11ce39ed565d619d5eb1d78622 - PREDICTION_FROZEN_ANCHORED:
b98e123b11a37bf0f0e0dfae916bfe2d5b4e1507 - COMPARED_ANCHORED:
ada06530369cbdd9393cf9432241c0954d916b93
Hashes establish byte identity, not scientific validity. Git provides auditable chronology, not an administrator-proof timestamp service.
Reproduce and audit
Permanent versioned files. The compact reproduction package holds the code, paper, figures and manifests; each full case archive is downloadable separately through the hashed index.
Reproduction instructionsDownload reproduction packagePublished status snapshotFrozen specificationInput-only execution runnerAll case bundles and hashesRelease manifestLaTeX source
Preserved earlier checkpoints and development history
The preparation report and input-freeze checkpoint are historical records, not the current experiment status. Their bytes remain unchanged.
Engineering scope
This tests one nominal configuration, declared assumptions and a steady-model workflow. It does not establish exact transition equivalence, unsteady fidelity, design readiness, universal Aero validation or flight suitability. Numerical and experimental failures remain part of the record.