Assessing National Statistics

DQAF, ODIN and NUSAP: three objects of assessment, and FAND's place among them.

Three communities grade official statistics, and each grades a different thing.

A compiler that synthesizes official statistics across countries and decades, as FAND does, needs all three. None of them grades the compiler. FAND inherits the first two from its sources and owes its readers the third. That is what NUSAP-lite is for.

Three objects of assessment

DQAFODINNUSAP
WhoIMF Statistics DepartmentOpen Data WatchFuntowicz and Ravetz (1990); now van der Sluijs and colleagues
GradesThe producing agency, dataset by datasetThe release on the national statistical office’s websiteThe number, and the assumptions behind it
UnitAgency × dataset (national accounts, prices, government finance, money, balance of payments)22 data categories × 197 countriesA value, or an assumption in a model
ScaleEach element Observed, Largely Observed, Largely Not Observed or Not ObservedTen elements, five on coverage and five on openness, each scored 0, ½ or 1; totals out of 100Pedigree criteria scored 0–4, with spread and assessment alongside
LatestGeneric framework July 2003; dataset frameworks May 2012; a 2025 pilot version now used in data ROSCs2024/25 round; the 2026/27 round began in May 2026Applied in EU food safety and product-footprint rules (below)

The DQAF has siblings on the producer side: the European Statistics Code of Practice, with its Quality Assurance Framework; the UN’s National Quality Assurance Frameworks; and the World Bank’s Statistical Performance Indicators, which in 2021 replaced its Statistical Capacity Index with five pillars (data use, services, products, sources and infrastructure).4 All four grade systems and processes. None grades a number.

How the DQAF lines up with NUSAP

The DQAF runs from prerequisites through five dimensions, each broken into elements, indicators and focal issues. Read against NUSAP, each dimension answers one part of the question NUSAP asks of a number.

DQAF dimensionNUSAPWhat FAND does with it
0. Prerequisites (legal and institutional environment, resources, relevance) and 1. Assurances of integrityP, the acceptance side of pedigreeInherits it. FAND cannot audit a ministry; it can record which agency produced a value and when. See the source agency directory.
2. Methodological soundness (in the 2025 pilot, alignment with international statistical standards)P, the method side of pedigreePosts its own Statements of Expectations as Operational Guidelines, and records departures as Exceptions to Expectations.
3. Accuracy and reliability (source data, statistical techniques, validation, revision studies)S and AMeasures one kind of spread today: the gap between a source’s world total and FAND’s sum of countries.
4. Serviceability (periodicity and timeliness, consistency, revision policy)U, read as the time scale of the unitCompletes ragged recent years by a stated rule and marks every completed value: the currentness switch.
5. Accessibility (data, metadata, assistance to users)N and U as publishedPublishes the replication package that rebuilds every figure from public sources.

The mapping shows where the frameworks part. The DQAF asks whether a process can be trusted. NUSAP asks how much a number can carry once it has left the process. An office rated Observed on every element can still publish a number that is wrong for a 75-year comparison across 200 economies: the concepts changed in 1993, the base year moved, the series was never meant to travel that far. And an office never assessed may publish the only number there is.

Revision studies (DQAF element 3.5) are the one place an official framework measures spread from evidence: how far first estimates moved when later ones arrived. They are the closest thing in official statistics to NUSAP’s S.

What ODIN adds, and what it cannot

ODIN counts a dataset’s reference metadata as present when it carries three things: the definition of the indicator, the date the data were published, and the agency responsible. Those three are the minimum of NUSAP’s U and P. In the 2024/25 round, only a quarter of the 14,136 dataset records ODIN examined carried all three. The source was named in 88 percent, the date in 52 percent, the definition in 44 percent. Across the 171 countries assessed in every round from 2018 to 2024, metadata scores rose 4 points while machine readability rose 17 and open licensing 19.5

Open Data Watch’s September 2026 brief argues that AI raises the bar: a chatbot that answers with an official number needs metadata it can read, not just metadata that exists. It points to SDMX for structure, to the 20 percent of ODIN records reachable by an API, and to the five national statistical or data agencies that now run their own MCP servers, the connectors AI assistants use to query an API.5 Per-cell pedigree is that kind of metadata.

ODIN grades availability, not accuracy. A perfect ODIN score means a number can be found and reused. It does not say whether the number is right, or right for the use a reader has in mind.

NUSAP now, especially in Europe

NUSAP has moved from the academy into the annexes of rules where numbers carry legal or market weight.

Two of those threads matter for FAND. The footprint rating’s geographical and time representativeness are FAND’s distinction between the unit of observation and the unit of analysis: a county figure used for a county page is one thing; the same figure inferred from a state total is another. And the ecoinvent route shows how categorical grades can become spread, which is FAND’s largest gap.

Where FAND’s NUSAP-lite stands

FAND is a compiler, not a producer. It has no ROSC and no ODIN row. What it has, by letter:

The “lite” is deliberate. It names what has not been done: spread per value, and the arithmetic that carries grades through sums and ratios.

Directions for further work

Four steps would close most of the distance, each building on the frameworks above rather than replacing them.

  1. Let the source-authority column inherit other people’s grades. Score it from the IMF dissemination standard the reporting country subscribes to for that dataset (SDDS Plus, SDDS, e-GDDS or none), from the most recent data ROSC rating on accuracy and reliability where one exists, and from ODIN’s openness score for the category. Each carries its date. FAND then grades none of its sources on its own authority.
  2. Derive spread from pedigree, then test it. Follow the ecoinvent route: give each pedigree score an uncertainty factor and compute a band for every value. Calibrate the factors on FAND’s own overlaps rather than stipulating them: wherever two sources cover the same cell (IMF IIP and the External Wealth of Nations; a source’s world total and the sum of countries; US national figures and the sum of states or counties), the observed disagreement estimates the factor.
  3. Publish what ODIN asks for. Put the definition, the publication date and the responsible source on every FAND download, machine-readable, under open terms.
  4. Offer the mapping back. The DQAF-to-NUSAP table above, and FAND’s pedigree matrix, are contributions the NUSAP community at nusap.net can test on a domain it has not covered: national balance sheets.

Footnotes

  1. International Monetary Fund, Data Quality Assessment Framework (DQAF): Generic Framework (Washington, DC, July 2003); IMF Statistics Department, Data Quality Assessment Framework for National Accounts Statistics (May 2012). The 2025 pilot version is the basis of recent data ROSCs, e.g. IMF, Cambodia: Report on the Observance of Standards and Codes—Data Module, Country Report No. 26/62 (March 2026). ↩

  2. Open Data Watch, Open Data Inventory 2022/23 Methodology Guide; Open Data Watch, “The 2026-27 ODIN Assessments Begin in May,” opendatawatch.com (2026). ↩

  3. Silvio O. Funtowicz and Jerome R. Ravetz, Uncertainty and Quality in Science for Policy (Dordrecht: Kluwer, 1990); Jeroen P. van der Sluijs et al., “Combining Quantitative and Qualitative Measures of Uncertainty in Model-Based Environmental Assessment: The NUSAP System,” Risk Analysis 25, no. 2 (2005): 481–492. ↩

  4. Eurostat, European Statistics Code of Practice (rev. 2017); United Nations, National Quality Assurance Frameworks Manual for Official Statistics (2019); Hai-Anh Dang et al., “Statistical Performance Indicators and Index: A New Tool to Measure Country Statistical Capacity,” Scientific Data 10 (2023), doi:10.1038/s41597-023-01971-0. ↩

  5. Open Data Watch (Jamison Henninger), “Making Official Statistics Work in an AI-Driven World,” Open Data Insights, September 2026. ↩ ↩2

  6. EFSA Scientific Committee, “The Principles and Methods behind EFSA’s Guidance on Uncertainty Analysis in Scientific Assessment,” EFSA Journal 16, no. 1 (2018): e05122. ↩

  7. SAPEA, Making Sense of Science for Policy under Conditions of Complexity and Uncertainty, Evidence Review Report (Berlin, 2019), section 5.5.3. ↩

  8. Jeroen P. van der Sluijs et al., RIVM/MNP Guidance for Uncertainty Assessment and Communication (Bilthoven: RIVM/MNP, 2003). ↩

  9. Commission Recommendation (EU) 2021/2279 of 15 December 2021 on the use of the Environmental Footprint methods, Official Journal of the European Union L 471 (30 December 2021), Annex I. ↩

  10. Bo P. Weidema and Marianne S. Wesnæs, “Data Quality Management for Life Cycle Inventories: An Example of Using Data Quality Indicators,” Journal of Cleaner Production 4, no. 3–4 (1996): 167–174. ↩

  11. Andreas Ciroth et al., “Empirically Based Uncertainty Factors for the Pedigree Matrix in ecoinvent,” International Journal of Life Cycle Assessment 21, no. 9 (2016): 1338–1348. ↩

  12. Steve Pye et al., “Assessing Qualitative and Quantitative Dimensions of Uncertainty in Energy Modelling for Policy Support in the United Kingdom,” Energy Research & Social Science 46 (2018): 332–344. ↩