{% extends "base.html" %}
{% block title %}Methodology — Myrdal{% endblock %}
{% block content %}
<section class="ark-wrap ark-prose">
  <h1>Methodology</h1>
  <p class="ark-lede">Where each number comes from, how far it really runs, and what is not here. Every claim on this page can be checked against a file named on it.</p>

  <h2>Sources</h2>
  <p>{{ meta.counts.series_published }} series come from {{ meta.sources | length }} distinct source
  strings, grouped into {{ meta.counts.categories }} families. The family is derived mechanically from
  the registry's own <code>source</code> field — no series is hand-tagged, so the family counts cannot
  drift from the provenance.</p>
  <div class="ark-table-wrap" role="region" tabindex="0" aria-labelledby="cap-fam">
  <table data-arktable class="ark-table">
    <caption id="cap-fam">Series per source family</caption>
    <thead><tr><th scope="col">Family</th><th scope="col">Series</th></tr></thead>
    <tbody>
      {% for fam, n in meta.by_category.items() %}
      <tr><td data-label="Family">{{ fam.replace('_', ' ') }}</td><td data-label="Series">{{ n }}</td></tr>
      {% endfor %}
    </tbody>
  </table>
  </div>
  <ul>
    <li><strong>World Bank, World Development Indicators</strong> — population by age band, dependency ratios, fertility, life expectancy, birth and death rates, under-five mortality, net migration, migrant stock, urban share.</li>
    <li><strong>Maddison Project Database 2023</strong> — population and gross domestic product per head, back to year 1 of the common era.</li>
    <li><strong>Clio-Infra</strong> — historical total and urban population, 1500–2000.</li>
    <li><strong>World Inequality Database</strong> — total, female and male population by country from 1800.</li>
    <li><strong>United States Census Bureau, American Community Survey five-year estimates</strong> — the county panel: population, median age, race and ethnicity, nativity, income, labour force, housing.</li>
    <li><strong>United States Census Bureau, International Database</strong> — country population and components, with the publisher's own projections held separately.</li>
    <li><strong>United States Census Bureau, Population Estimates Program</strong> — county age bands, 2023 vintage.</li>
    <li><strong>Internal Revenue Service, Statistics of Income</strong> — county-to-county and state migration flows from tax filings.</li>
  </ul>

  <h2>How liveness is reported</h2>
  <p>Every “through” date on this site is the <strong>last real observation in the file</strong>. It is
  never a declared end year, and it is never a projection.</p>
  <ul>
    <li>The Census International Database series carry a <code>source_year_type</code> column that
      separates historical rows from projections. The projections run to
      {{ meta.by_id['D050'].projection_end }}; the last <em>observation</em> is
      {{ meta.by_id['D050'].last_real_observation }}. The site reports the second and labels the first
      as projections wherever they appear.</li>
    <li>The Statistics of Income migration files key on a filing-year <strong>pair</strong>
      (for example <code>2022-2023</code>), because a migration flow genuinely spans two years. They are
      not silently collapsed to a single year; the period is the axis label.</li>
    <li>Series are only counted as complete years. Nothing on this site reports a part-year figure
      alongside full years.</li>
  </ul>

  <h2>Construction seams</h2>
  <ul>
    <li><strong>D001, long-run world population.</strong> Rows before 1960 come from the Maddison
      Project and rows from 1960 come from the World Development Indicators, summed over real
      countries. The seam is written into the file's own <code>source</code> column. The same file also
      carries the World Bank's published world aggregate, which is slightly larger than the country sum;
      both are kept rather than one being quietly preferred.</li>
    <li><strong>Derived series.</strong> Three series are explicitly derived from others and say so in
      their provenance: <code>XS001</code> from D030–D033, <code>XS002</code> from D004,
      <code>XS003</code> from D008. <code>D058</code> and <code>D059</code> are computed from the
      Population Estimates Program age bands by formula, and the components are shipped in the same
      file so the ratio can be recomputed rather than trusted.</li>
  </ul>

  <h2>What is missing, stated plainly</h2>
  <ul>
    <li><strong>{{ meta.counts.series_empty }} registered series contain no data.</strong>
      {% for s in meta.series if s.kind == 'empty' %}<code>{{ s.series_id }}</code> ({{ s.display_name }}){{ ", " if not loop.last }}{% endfor %}
      ship a header row and nothing else. They are World Development Indicators percentage-of-total
      variants whose extraction returned zero rows. They remain in the registry and in the catalogue,
      labelled empty, because deleting them would make the count look tidier while hiding a real gap.</li>
    <li><strong>{{ meta.counts.series_cross_section }} series have no time axis.</strong>
      {% for s in meta.series if s.kind == 'cross_section' %}<code>{{ s.series_id }}</code>{{ ", " if not loop.last }}{% endfor %}
      are single-vintage cross-sections of United States counties (Population Estimates Program, 2023).
      They are downloadable but not chartable as a time series, and the explorer says so rather than
      drawing a one-point line.</li>
    <li><strong>{{ meta.counts.series_note_stub }} entry is not a series at all.</strong>
      <code>XS001</code>'s file holds a single hand-off note for a sibling project. It is counted in the
      registry and excluded from every observation count.</li>
    <li><strong>The knowledge-base extraction behind this project is preliminary.</strong> Ten documents
      were extracted as text and tables only, not through the full extraction protocol, and the project
      labels them that way. No number on this site depends on them — every value comes from a
      statistical source above.</li>
    <li><strong>Two of Gunnar Myrdal's own books could not be obtained</strong> (<em>Economic Theory and
      Underdeveloped Regions</em>, 1957; <em>Asian Drama</em>, 1968), and <em>An American Dilemma</em>
      is awaiting extraction. This bounds the interpretive material, not the data.</li>
    <li><strong>The construction pipeline is not yet published.</strong> See <a href="/code">Code</a>.</li>
  </ul>

  <h2>Validation</h2>
  <p>The Anu build recorded a schema validation pass, a cross-source reconciliation pass (Maddison
  against the World Development Indicators) and a ledger health of 100 % green on
  {{ meta.generated_at }}. Those verdicts are carried forward from that build and are labelled as such
  in the manifest; this site does not re-run them. What this site <em>does</em> assert, on every
  rebuild, is that the manifest and the disk agree in both directions — every file listed exists, and
  every file present is listed — and that the counts shown on these pages are read from that manifest
  rather than typed.</p>

  <p class="ark-fineprint">Per-series provenance: <a href="/downloads/PROVENANCE.csv">PROVENANCE.csv</a>.
  Full registry: <a href="/metadata/series_registry.json">series_registry.json</a> (metadata, not a data
  download).</p>
</section>
{% endblock %}