Known limitations, by layer — with direction of bias
Tag shows which way each limitation pushes the headline: overstates makes concentration/risk look worse than reality, understates the reverse, uncertain either way. Where I’ve already mitigated it, the green line says how.
Trade reconciliation is share-faithful, not level-faithful uncertain
BACI reconciles the two sides of every trade (exporter vs importer report) into one figure; the method is tuned to get shares right, not absolute dollar levels. Concentration metrics (HHI, top-partner share) are the right use; absolute values are indicative.
✓ The whole atlas is built on shares and ranks, not levels — the reconciliation engine is open and CI-tested against BACI.
The 2017 HS-vintage splice overstates
The long series joins HS-2002 codes (2002–2016) to HS-2017 codes (2017–2024). The origin-gap index steps at that join, so part of the “gap widened 15→18pp” is a splice artifact, not real drift.
✓ The step is caveated everywhere and marked on the Trends chart; the robustness page re-tests each trend separately on the two windows — 6 of 9 rising-concentration findings keep their sign in both halves.
Mine and refining shares are static, hand-entered approximations uncertain
The USGS/IEA production and refining shares are fixed values entered per material on mixed vintages — they don’t move year-to-year, and one can go stale (a case-study audit caught a lithium mine share two years out of date).
✓ Caught and corrected via the case-study audit; a periodic USGS-refresh sweep is in the runbook. This is the atlas’s softest data and the first place to check a surprising number.
The network fragility figures are an upper bound overstates
The trade-network graphs are drawn from the top-6 partners per side for legibility. A truncated graph hides real alternative routes, so “removing China breaks X% of routes” is too high.
✓ Quantified on the truncation-sensitivity page: rebuilt fully uncapped, average fragility falls from 34% to 12%. China’s centrality, however, barely moves — that finding survives.
Trend significance rests on 23 annual points overstates
Mann-Kendall on a short, serially-correlated annual series overstates significance. “9 of 32 materials are significantly concentrating” is an exploratory screen, not a confirmatory test.
✓ Re-run with an autocorrelation-robust correction on the robustness page: 7 of 9 survive. The rest are labelled exploratory.
The satellite footprint is all-commodity, not critical-material-specific uncertain
The Maus et al. mine polygons (Sentinel-2, ~2019) carry no reliable per-mineral label, so the country map is total mining intensity (coal, iron, gold included), not critical-material footprint.
✓ Stated loudly on the satellite page; the curated flagship mines carry the material-specific view. Used only as a cross-check, never as a production measure. We went further and quantified the limit: stacking six independent public mine registers (Jasansky 2023 + USGS MRDS + OpenStreetMap + USGS critical-minerals PP1802 + Geoscience Australia + IPIS artisanal-mining sites) labels ~52% of the footprint and ties ~16% to a tracked critical material — up from 17% / 4% on Jasansky alone — and where two primary-commodity sources label the same mine they agree ~86% of the time. Tellingly, adding the dedicated artisanal-mining dataset (IPIS, ~8,000 eastern-DRC sites) barely moved coverage: artisanal mining scarcely overlaps the ~2019 satellite footprint, so it is invisible to the imagery, not merely unlabelled. The residual ~half is non-critical mines, register geographic gaps, and the by-product problem. More data triples coverage; it does not close the gap.
Implied unit values are noisy uncertain
Price = value ÷ tonnage from trade data mixes product grades and qualities. Read the price charts as trends and cross-country contrasts, not exact $/tonne.
Risk weights and disruption odds are assumptions uncertain
The fixed-weight index, the entropy weights (which reward dispersion, not importance), and the Monte-Carlo failure probabilities are transparent assumptions, not estimated parameters. They’re a panel of lenses, not a verdict.
The origin trace is a first-order upper bound overstates
Re-attributing refiner imports to the dominant mine is a single-step approximation that flags likely refiner-fronting — not proof of disguised true origin.
Complexity is within 32 materials, not the economy uncertain
The RCA/relatedness layer measures specialization within this critical-material slice, not economy-wide economic complexity. The eigenvector ECI was dropped as degenerate on so few products.
What would change my mind
Concrete falsifiers for the headline claims. If these came back the other way, the finding is wrong.
This page is the project’s standing invitation to be checked. The code and data are public; the pre-registration is timestamped. If something here is wrong, it can be shown to be wrong — which is the point.