Reference section

Choosing and using the reference downloads

Use CSV for a straightforward article-link spreadsheet and JSON when you need nested source or relationship fields. Save the matching checksums and preserve each dataset's scope and status when filtering, joining or sharing the records.

Updated 417 words · 2 min read

Work with the reference records

Available files

File What it describes Useful starting task
Article index CSV Original titles, IDs, original URLs and canonical reading URLs Account for all supplied manuscripts
Reference records JSON Current reader records and metadata Build a local reference table
Source register CSV Exact document URLs, dates and source scope Inspect the source list in a spreadsheet
Source register JSON The same document records as structured data Join sources by their IDs
Consolidation JSON Original-to-canonical mappings and actions Follow an older article address
Bibliography JSON Inherited publication leads and metadata status Plan literature reconciliation
Section evidence JSON Bounded mappings for structured summaries Inspect existing section-source links
SHA-256 list Checksums of the current downloadable files Compare saved bytes with the export

Importing the CSV into a spreadsheet

Use the application's text or CSV import function and keep identifiers as text. Confirm the header row, separator and character encoding before starting an analysis. Check a few titles and URLs after import. Long titles and punctuation should remain in their original cells.

Filter the action or canonical URL columns when investigating consolidation. Several original rows may point to one current reference. Removing repeated canonical URLs is useful for a reader-page count, but preserve the source-ID column when the purpose is attachment coverage.

Working with JSON

Inspect the top-level structure before flattening a file. Reference records can contain source lists and relationships that do not fit naturally into one spreadsheet cell. A separate relationship table can preserve those arrays without losing which record each value belongs to.

Join records using explicit identifiers where they are provided. Titles can change during editing and are a weaker key. Keep the source scope and metadata status alongside the fields you use in a new table.

Keep provenance through transformations

If you rename columns, remove rows or combine records, save a brief description of the transformation and the original download. Record whether counts refer to source manuscripts, canonical references or bibliographic leads. Those units are intentionally separate in this repository.

What should accompany a shared extract?

Include the file identity, download date, relevant status fields, source URLs and a description of any filtering. The reuse page explains why a link to a publication is different from permission to reproduce its full text. The data library explains the boundaries of each dataset.