Verification & Source-Checking
What this is
Verification is how we keep facts on the wiki correct. There are several overlapping systems:
- Citation verification — does each footnote URL still resolve, and does the cited source actually support the claim?
- Source-checking — for FactBase facts and wiki page claims, does the cited source contain the asserted value?
- Hallucination risk — per-page confidence scoring based on citation density, claim type, and source quality.
- Audits — full-page review pipelines that can propose or apply fixes.
When to start here
A page says "Anthropic raised $4B in 2024" and you want to know whether that's actually true according to its citation? This is the hub.
Live dashboards
| Dashboard | What it shows |
|---|---|
| Entity Source Checks | Verdict counts (confirmed/contradicted/undecidable) per FactBase entity |
| Citation Accuracy | Per-citation accuracy scores from claim-vs-source LLM checks |
| Citation Content | Citation metadata: title/content availability, broken URLs |
| Hallucination Risk | Per-page and wiki-wide risk scores |
| Source Check Coverage | Coverage by record type and entity type |
CLI playbook
# Citation verification (per page or wiki-wide)
pnpm crux w citations verify anthropic # Verify one page
pnpm crux w citations verify --all --limit=20 # Verify top 20 pages
pnpm crux w citations status anthropic # Show sourcing verdicts
pnpm crux w citations report # Wiki-wide summary
pnpm crux w citations report --broken # List broken citations
# Claim accuracy checks (does the source support the claim?)
pnpm crux w citations check-accuracy anthropic # Check one page
pnpm crux w citations check-accuracy --all # Batch
pnpm crux w citations fix-inaccuracies --apply # Apply fix proposals
# Full audit pipeline (extract → verify → propose fixes)
pnpm crux w citations audit anthropic # Audit one page
pnpm crux w citations audit anthropic --apply # Audit + auto-fix
pnpm crux w citations audit-check anthropic --threshold=0.9 # Independent check (no DB writes)
# Wiki-page sourcing (prose claims → cited footnotes)
pnpm crux w sourcing-wiki-pages --page=anthropic
pnpm crux w sourcing-wiki-pages --dry-run # Preview
pnpm crux w sourcing-wiki-pages --budget=5 # With budget cap
# FactBase fact sourcing (structured facts → cited URLs)
pnpm crux fb sourcing --entity=anthropic # Check one entity's facts
pnpm crux fb sourcing --dry-run --limit=5 # Preview
# Hallucination risk
pnpm crux query risk anthropic # Score for one page
pnpm crux query risk --level=high --limit=20 # Top 20 risky pages
When to use what
| I want to... | Run | Or check |
|---|---|---|
| Find broken citation URLs across the wiki | pnpm crux w citations report --broken | Citation Content dashboard |
| See which pages have unsupported claims | pnpm crux w citations check-accuracy --all | Citation Accuracy dashboard |
| Identify pages that need human review | pnpm crux query risk --level=high | Hallucination Risk dashboard |
| Verify a specific page end-to-end | pnpm crux w citations audit <page> | — |
| Verify FactBase facts for one entity | pnpm crux fb sourcing --entity=<id> | Entity Source Checks dashboard |
| Re-check claims that previously failed | Add --recheck to verify/check-accuracy commands | — |
| Auto-apply proposed citation fixes | pnpm crux w citations fix-inaccuracies --apply | — |
| Understand the verification tier model | — | Content Verification Tiers |
Verdict types
Source-check produces three verdicts for each claim/fact:
confirmed— the cited source contains text that supports the claimcontradicted— the cited source contains text that contradicts the claimundecidable— the source doesn't address the claim (irrelevant, paywalled, broken, or ambiguous)
The dashboards group by these verdicts so you can prioritize: contradicted > undecidable > re-verify confirmed.
Architecture & deep dives
- Content Verification Tiers — verification standards by page type (biographical > analytical > exploratory), inspired by Wikipedia BLP
- Citation Architecture — five overlapping citation systems (footnotes, References, CitationOverlay, ResourceLink, KB fact refs)
crux/lib/sourcing/— orchestrator, LLM checker, verdict handler, source parserscrux/citations/— verify-citations, citation-status, check-accuracyapps/wiki-server/src/routes/sourcing/— RPC endpoints for verdict storage