Corpus Extraction — Plan Overview

A human-readable map of the weeks-long extraction effort. Deep detail lives in the two track documents linked below; this page is the entry point.

The one big idea: two separate tracks

The corpus is enriched by two independent pipelines, deliberately kept apart because their aims are genuinely different. Trying to do both in one pass produces a hybrid that’s mediocre at each.

	Historical Track	Conceptual Track
Aim	the factual layer — who existed, who did what, when, where, which tablets were revealed	organizing doctrine — a concept ontology + concepts as first-class entities
Extracts	`person`, `work`/tablet, `place`, `event` + cited biographical facts	`concept` entities + authoritative doctrinal claims
Ordered by	source authority — primary/eyewitness sources before 3rd-party compilations	interpretive authority, top-down — the latest authorized interpreter governs, backwards-facing
Status	building now (extraction upgraded; validating)	design only — a separate later effort
Detail doc	architecture/history-track.md	architecture/conceptual-track.md

They share only the disambiguated text upstream, and link at the graph: a conceptual claim (“the Íqán teaches X”) just references a person/work entity the historical track built. No shared logic, no merging.

Historical Track — ordered by textual rigor / reliability

Ranked by how carefully each text was collected and edited — a factual criterion, not nominal authority. (’Abdu’l-Bahá’s authority is supreme, but Memorials of the Faithful as a text is Sohrab’s notes from oral story-telling rendered into English, so it can carry errors and ranks below the rigorously-edited scholars.) The graph is anchored by the most reliable texts first; looser ones resolve against them.

Tier 1 — God Passes By → The Dawn-Breakers (the Guardian’s rigorous authoritative history + the person-seed)
Tier 2 — current work — the rigorously-edited scholars: all Taherzadeh/Revelation of Bahá’u’lláh, Balyuzi, Momen, Mázindarání’s Ẓuhúru’l-Ḥaqq (the reliable foundation)
Tier 3 — primary but loosely collected — ’Abdu’l-Bahá texts (Memorials, The Secret of Divine Civilization) + eyewitness/pilgrim accounts (oral notes, personal diaries, translation layers → resolve against Tiers 1–2)
Secondary — the general 3rd-party histories

Conceptual Track — ordered by interpretive authority (design)

Authoritative interpretation is backwards-facing: the latest authorized interpreter governs the reading of everything before him. So the seed runs top-down:

Shoghi Effendi → ’Abdu’l-Bahá → Bahá’u’lláh → the Báb → Shaykhí → Imáms → Qur’án → Christianity → Judaism

Within the Guardian tier the internal order is: God Passes By (his only book — the concept skeleton, breaking a work into specific cited assertions) → the Dispensation of Bahá’u’lláh (his testament, the ontology keystone: Manifestation-vs-Interpreter, the stations, the Covenant) → the general letters (ADJ, PDC, WOB…) → his other letters → his translations (before any secondary translation).
Concepts become first-class entities whose development aggregates across the whole corpus, authority-weighted — bound to their authoritative meaning through a cumulative, cited interpretive lexicon (so a Biblical metaphor like “clouds” carries its authoritative conceptual meaning, not just the literal story).
Interfaith by design: concepts link to their analogues across traditions even when the terminology differs entirely (the Primal Will / the Logos / the First Intellect), so a reader can query one idea across religions.

How everything is produced (shared machinery)

DISAMBIGUATE → {HyPE ∥ EXTRACT} → RECONCILE, one gated orchestrator, per document, in priority order. Every stage: multilingual (cheap DeepSeek-flash for English/Arabic/Hebrew, Claude-haiku for Persian where flash fails), self-healing (escalates to a stronger model on failure), and continuation-robust (dense genealogies like Momen’s are captured in full, never truncated). Enrichment is written in English (so one query reaches the whole multilingual corpus), while source text stays searchable in its own language.

Full machinery: architecture/unified-enrichment-pipeline.md. Live progress: extraction-status.md.