{% extends "base.html" %}
{% block title %}Data — Myrdal{% endblock %}
{% block content %}
<section class="ark-wrap ark-prose">
  <h1>Data</h1>
  <p class="ark-lede">Every published series, in three formats, with the dictionary and the provenance record beside them. Nothing on this page is typed by hand — the counts and the bundle size are read from the package's own manifest at render time.</p>

  <h2>What the package contains</h2>
  <p>These are <strong>separate measures</strong> and they are deliberately not all equal. Forcing them
  to match would hide exactly the things a user needs to know.</p>
  <div class="ark-table-wrap" role="region" tabindex="0" aria-labelledby="cap-counts">
  <table data-arktable class="ark-table">
    <caption id="cap-counts">Package counts, read from <code>WEB_MANIFEST.json</code></caption>
    <thead><tr><th scope="col">Measure</th><th scope="col">Count</th><th scope="col">What it means</th></tr></thead>
    <tbody>
      <tr><td data-label="Measure">Series registered</td><td data-label="Count">{{ meta.counts.series_registered }}</td><td data-label="What it means">Entries in the Anu series registry.</td></tr>
      <tr><td data-label="Measure">Series published</td><td data-label="Count">{{ meta.counts.series_published }}</td><td data-label="What it means">Registry entries carrying <code>publish: true</code>. None is withheld.</td></tr>
      <tr><td data-label="Measure">CSV files</td><td data-label="Count">{{ meta.counts.chopped_csv }}</td><td data-label="What it means">One per series.</td></tr>
      <tr><td data-label="Measure">Parquet files</td><td data-label="Count">{{ meta.counts.chopped_parquet }}</td><td data-label="What it means">One per series, same rows.</td></tr>
      <tr><td data-label="Measure">XLSX files</td><td data-label="Count">{{ meta.counts.chopped_xlsx }}</td><td data-label="What it means">One per series, same rows.</td></tr>
      <tr><td data-label="Measure">Series with data</td><td data-label="Count">{{ meta.counts.series_with_data }}</td><td data-label="What it means">Files with at least one data row. {{ meta.counts.series_empty }} ship a header and nothing else — see <a href="/methodology">Methodology</a>.</td></tr>
      <tr><td data-label="Measure">Series that can be charted</td><td data-label="Count">{{ meta.counts.series_chartable }}</td><td data-label="What it means">Lower again: {{ meta.counts.series_cross_section }} are cross-sections with no year column and {{ meta.counts.series_note_stub }} is a hand-off note, not observations.</td></tr>
      <tr><td data-label="Measure">Observations</td><td data-label="Count">{{ "{:,}".format(meta.counts.observations_total) }}</td><td data-label="What it means">Data rows across every series file.</td></tr>
      <tr><td data-label="Measure">Source families</td><td data-label="Count">{{ meta.counts.categories }}</td><td data-label="What it means">Derived from the registry's own <code>source</code> field, never hand-tagged.</td></tr>
    </tbody>
  </table>
  </div>

  <h2>Download everything</h2>
  {% if meta.bundles and meta.bundles.data %}
  <p class="ark-btn-group">
    <a class="ark-btn" href="{{ meta.bundles.data.href }}"
       data-cdf-size="{{ meta.bundles.data.manifest_total_bytes or meta.bundles.data.bytes }}">Download the data bundle (.zip)</a>
    <a class="ark-btn ark-btn-outline" href="{{ meta.bundles.code.href }}">Download the site code (.zip)</a>
  </p>
  <p class="ark-fineprint">{{ "{:,}".format(meta.bundles.data.manifest_total_bytes or meta.bundles.data.bytes) }} bytes. The zip unpacks to one dated folder holding <code>csv/</code>, <code>parquet/</code> and <code>xlsx/</code>, plus the data dictionary, the provenance record, the terms, the citation file and the registry under <code>metadata/</code>. Per-member byte counts and SHA-256 hashes are published in <a href="/downloads/BUNDLE_MANIFEST.csv">BUNDLE_MANIFEST.csv</a>, so the size above can be checked against the file rather than taken on trust.</p>
  {% else %}
  <p class="ark-note">The bundle has not been generated for this deployment. Individual series remain downloadable below.</p>
  {% endif %}

  <h2>Formats</h2>
  <p>CSV, XLSX and Parquet. <strong>JSON is not offered as a data format</strong> — the registry is
  reachable as metadata at <a href="/metadata/series_registry.json">/metadata/series_registry.json</a>,
  which is a description of the data, not the data.</p>
  <ul>
    <li><a href="/downloads/data_dictionary.csv">data_dictionary.csv</a> — {{ meta.counts.data_dictionary_rows }} rows, one per series: name, family, units, source, kind, row count, first and last real observation.</li>
    <li><a href="/downloads/PROVENANCE.csv">PROVENANCE.csv</a> — one row per series: who published it, how it was built, how far it really runs, and its triage verdict.</li>
    <li><a href="/downloads/TERMS.md">TERMS.md</a> — terms of use. Each upstream publisher keeps its own.</li>
    <li><a href="/downloads/CITATION.cff">CITATION.cff</a> — how to cite this dataset.</li>
  </ul>

  <h2>Every series</h2>
  <p class="ark-fineprint">“Through” is the <strong>last real observation in the file</strong>, not a
  declared end year. Where a source also publishes projections they are shown separately and are never
  counted as observations.</p>
  <div class="ark-table-wrap" role="region" tabindex="0" aria-labelledby="cap-series">
  <table data-arktable class="ark-table">
    <caption id="cap-series">All {{ meta.counts.series_published }} published series</caption>
    <thead>
      <tr>
        <th scope="col">ID</th><th scope="col">Series</th><th scope="col">Family</th>
        <th scope="col">Units</th><th scope="col">Rows</th><th scope="col">From</th>
        <th scope="col">Through</th><th scope="col">Projections</th><th scope="col">Download</th>
      </tr>
    </thead>
    <tbody>
      {% for s in meta.series %}
      <tr>
        <td data-label="ID"><code>{{ s.series_id }}</code></td>
        <td data-label="Series">{{ s.display_name }}</td>
        <td data-label="Family">{{ s.category.replace('_', ' ') }}</td>
        <td data-label="Units">{{ s.units }}</td>
        <td data-label="Rows">{{ "{:,}".format(s.rows_data) }}</td>
        <td data-label="From">{{ s.first_observation if s.first_observation is not none else '—' }}</td>
        <td data-label="Through">{% if s.last_real_observation is not none %}{{ s.last_real_observation }}{% elif s.kind == 'empty' %}no data rows{% elif s.kind == 'cross_section' %}single vintage{% else %}not a series{% endif %}</td>
        <td data-label="Projections">{{ s.projection_end if s.projection_end is not none else '—' }}</td>
        <td data-label="Download">
          <a href="/downloads/series/{{ s.series_id }}.csv">CSV</a> ·
          <a href="/downloads/series/{{ s.series_id }}.xlsx">XLSX</a> ·
          <a href="/downloads/series/{{ s.series_id }}.parquet">Parquet</a>
        </td>
      </tr>
      {% endfor %}
    </tbody>
  </table>
  </div>
</section>
{% endblock %}