Data Architecture: Three Bases and Naming Guide

This document is the canonical naming reference for the wiki's data architecture. It explains the three conceptual data layers ("Bases"), maps each PostgreSQL table to its Base, and clarifies common naming confusions.

The Three Bases

The wiki organizes data into three conceptual layers:

flowchart TB
  subgraph Sources["Source of Truth"]
      YAML["data/entities/*.yaml
data/resources/*.yaml"]
      FB_YAML["packages/factbase/data/things/*.yaml"]
      MDX["content/docs/**/*.mdx"]
  end
  subgraph Build["Build Pipeline"]
      BD["build-data.mjs"]
  end
  subgraph Artifacts["Build Artifacts"]
      DB_JSON["database.json
(TableBase + WikiBase)"]
      FB_JSON["factbase-data.json
(FactBase)"]
  end
  subgraph PG["PostgreSQL (wiki-server)"]
      direction TB
      TB_PG["TableBase tables
(entities, resources,
entity_ids, summaries)"]
      FB_PG["FactBase mirror
(facts table)"]
      WB_PG["WikiBase mirror
(wiki_pages)"]
      UNI["Cross-Base index
(things table)"]
      OPS["Operational tables
(sessions, jobs,
citation_quotes, etc.)"]
  end
  YAML --> BD
  FB_YAML --> BD
  MDX --> BD
  BD --> DB_JSON
  BD --> FB_JSON
  BD --> TB_PG
  BD --> FB_PG
  BD --> WB_PG
  BD --> UNI

PG Tables Grouped by Base

TableBase tables (entity catalog)

These tables mirror the YAML entity/resource catalog. YAML files remain authoritative; these PG tables are queryable read mirrors for the API.

FactBase tables (structured facts)

YAML files in packages/factbase/data/things/ are currently the primary source. They sync to PG via crux wiki-server sync-facts. The PG facts table provides an export endpoint (GET /api/facts/export) for consumers that need PG-backed access. Once the PG schema includes all Fact fields (validEnd, currency, etc.), PG will become the primary source.

WikiBase tables (prose content)

These tables mirror the MDX wiki pages.

Cross-Base index (the things table)

The things table is a cross-base universal index used for search. Every identifiable item in the system (entity, fact, grant, resource, personnel record, division, etc.) gets a single row. This enables:

• Cross-domain search (search everything in one query)
• A single browse UI for all data

Unified Verification tables

All verification data lives in two tables (replacing the previous six). See discussion #2950.

Operational tables

These tables are not part of any Base. They track system operations, CI/CD, and agent activity.

Naming Confusions (and How to Read Them)

"Entity" means different things in different contexts

Key distinction: The YAML/PG entities table uses slug-based IDs (e.g., anthropic), while FactBase entities have their own 10-character alphanumeric IDs (e.g., mK9pX3rQ7n). The factbase-data.json file includes a slugToEntityId mapping to bridge between them.

"Things" means different things in different contexts

Key distinction: The FactBase "things" directory (packages/factbase/data/things/) contains YAML data files that define entities and their facts. The PG things table is a completely separate concept — it is a universal search/browse index that contains rows pointing to entities, facts, grants, resources, personnel, and other record types. They share a name but serve unrelated purposes.

"Facts" means different things in different contexts

Key distinction: The authoritative source for structured facts is the FactBase YAML (packages/factbase/data/things/). The PG facts table is a read mirror for API queries. The old data/facts/*.yaml system is deprecated for FactBase-covered entities (see Data System Authority RulesE1018Defines which data system is authoritative for each entity type, resolving overlaps between KB YAML and old YAML facts.Quality: 47/100).

Code Module Map

Related Documents

• System ArchitectureE734Technical architecture of the Longterm Wiki — data pipeline, clever design patterns, and the ideas that make it workQuality: 73/100 — High-level technical overview
• DB Schema OverviewE1054Complete reference for the wiki-server PostgreSQL schema — tables, relationships, ER diagrams, and migration history0 — Full ER diagrams and migration history
• Data System Authority RulesE1018Defines which data system is authoritative for each entity type, resolving overlaps between KB YAML and old YAML facts.Quality: 47/100 — Which data system is authoritative for each entity
• Fact System StrategyE886Research-backed strategy for the canonical facts system — what to keep, what to prune, and how to make facts genuinely useful for wiki maintenance.Quality: 55/100 — Strategy for the old YAML facts system