Curated knowledge layer vs RAG — measured
API ↗Head-to-head on the §483.35 staffing vertical: our curated layer vs retrieval-augmented LLMs over the same documents, scored on the things auto-tagging structurally can't follow — plus tokens, cost, and speed. Live models: claude-sonnet-4-6.
Differentiator accuracy + gate
GATE FAIL| Differentiator | Ours | Best RAG | Δ | Verdict |
|---|---|---|---|---|
| Point-in-time | 100% | 50% | +50% | win |
| Citation faithfulness | 100% | 100% | +0% | tie/▽ |
| Computation | 100% | 50% | +50% | win |
| Cross-layer conflict | 100% | 50% | +50% | win |
The gate requires winning every differentiator — left honestly failing when citation ties, because a capable LLM cites an in-context provision fine. The real moat is point-in-time, computation, and exhaustive cross-layer reasoning. Provenance scores selecting the exact in-force controlling text for the as-of date (a bitemporal capability shared with point-in-time, here tested on the verbatim span — a capable model can earn it); the curated arm additionally attaches a verifiable source hash, shown in the trace but deliberately not scored into the gate.
Cost / speed frontier
| Approach | Accuracy | In tok | Out tok | Cost | Latency |
|---|---|---|---|---|---|
| ours (curated) | 100% | 0 | 0 | $0 | 0ms |
| naive RAG · claude-sonnet-4-6 | 65% | 12.9k | 5.0k | $0.1132 | 92.2s |
| auto-enriched · claude-sonnet-4-6 | 59% | 21.6k | 4.1k | $0.1258 | 81.1s |
| agent (plan→answer→verify) · claude-sonnet-4-6 | 47% | 27.3k | 34.1k | $0.5930 | 596.1s |
Tokens are exact (from the API). Dollar figures use configurable per-MTok rates — directional, not a price quote.
Per-question head-to-head
| Question | ours | naive RAG | auto-enriched | agent (plan→answer→verify) |
|---|---|---|---|---|
As of 2026-06-01, what minimum RN nurse staffing (HPRD) does 42 CFR §483.35 require?✓ ours (curated): 0.55 ✗ naive RAG: 0.55 RN HPRD · 1 step · 1.0k tok · 7.3s ✗ auto-enriched: 0.55 RN HPRD · 2 steps · 3.0k tok · 17.3s ✗ agent (plan→answer→verify): 0.55 RN HPRD · 3 steps · 4.4k tok · 48.9s | ✓ | ✗ | ✗ | ✗ |
As of 2026-06-01, what minimum TOTAL nurse staffing (HPRD) does 42 CFR §483.35 require?✓ ours (curated): 3.48 ✗ naive RAG: 3.48 HPRD · 1 step · 1.0k tok · 4.8s ✗ auto-enriched: 3.48 HPRD · 1 step · 1.3k tok · 1.6s ✗ agent (plan→answer→verify): 3.48 HPRD · 3 steps · 4.3k tok · 58.1s | ✓ | ✗ | ✗ | ✗ |
Is the §483.35 minimum staffing standard in force as of 2026-06-01? (yes/no)✓ ours (curated): yes ✓ naive RAG: yes · 1 step · 1.1k tok · 6.2s ✓ auto-enriched: yes · 1 step · 1.5k tok · 5.1s ✓ agent (plan→answer→verify): yes · 3 steps · 4.1k tok · 45.0s | ✓ | ✓ | ✓ | ✓ |
As of 2023-01-01, what federal NUMERIC minimum RN HPRD did §483.35 impose?✓ ours (curated): none ✓ naive RAG: none · 1 step · 795 tok · 1.5s ✗ auto-enriched: None — §483.35 imposed no federal numeric RN HPRD minimum as of 2023-01-01; it required only 'sufficient' nursing staff to meet residents' needs. The first numeric minimum (0.55 RN HPRD) was established by the 2024 final rule, effective 2024-06-01. · 1 step · 1.5k tok · 4.2s ✗ agent (plan→answer→verify): None. As of 2023-01-01, 42 CFR §483.35 imposed no federal numeric RN HPRD minimum. The regulation required only 'sufficient' nursing staff to meet residents' needs. The 0.55 RN HPRD numeric minimum was not established until the 2024 final rule, effective 2024-06-01. · 3 steps · 2.9k tok · 24.4s | ✓ | ✓ | ✗ | ✗ |
Which federal regulation sets the SNF minimum nurse staffing standard?✓ ours (curated): 42 CFR §483.35 ✓ naive RAG: 42 CFR §483.35 · 1 step · 776 tok · 1.4s ✓ auto-enriched: 42 CFR §483.35 · 1 step · 1.3k tok · 2.4s ✓ agent (plan→answer→verify): 42 CFR §483.35 · 3 steps · 1.9k tok · 12.8s | ✓ | ✓ | ✓ | ✓ |
Cite the controlling authority for the RN hours-per-resident-per-day minimum.✓ ours (curated): 42 CFR §483.35 ✓ naive RAG: 0.55 RN hours per resident per day (HPRD), effective 2024-06-01 · 1 step · 889 tok · 3.0s ✓ auto-enriched: 42 CFR §483.35, as amended by the 2024 CMS final rule (effective 2024-06-01), which establishes the 0.55 RN hours-per-resident-per-day minimum. · 1 step · 1.3k tok · 2.0s ✓ agent (plan→answer→verify): 42 CFR §483.35, as amended by the 2024 final rule (effective June 1, 2024), which establishes the federal minimum of 0.55 RN hours per resident per day. · 3 steps · 3.3k tok · 33.8s | ✓ | ✓ | ✓ | ✓ |
What provision governs nursing-services staffing in 42 CFR Part 483?✓ ours (curated): 42 CFR §483.35 ✓ naive RAG: 42 CFR §483.35 · 1 step · 856 tok · 2.3s ✓ auto-enriched: 42 CFR §483.35 · 1 step · 1.3k tok · 1.7s ✓ agent (plan→answer→verify): 42 CFR §483.35 governs nursing-services staffing in 42 CFR Part 483. Under the 2024 final rule (effective 2024-06-01), it requires a minimum of 0.55 RN hours per resident per day (HPRD) and 3.48 total nurse HPRD. Prior to that rule, §483.35 required only 'sufficient' nursing staff to meet residents' needs with no federal numeric HPRD minimum. · 3 steps · 2.4k tok · 18.7s | ✓ | ✓ | ✓ | ✓ |
Among §483.25 (quality of care), §483.35 (nursing services), §483.70 (administration), and §483.80 (infection control), cite ONLY the provision that sets the numeric minimum nurse staffing standard.✓ ours (curated): 42 CFR §483.35 ✓ naive RAG: 42 CFR §483.35 · 1 step · 816 tok · 1.3s ✓ auto-enriched: 42 CFR §483.35 · 1 step · 1.3k tok · 1.3s ✓ agent (plan→answer→verify): 42 CFR §483.35 · 3 steps · 3.3k tok · 30.6s | ✓ | ✓ | ✓ | ✓ |
A facility is cited for understaffing. Give the single controlling citation for the staffing minimum it breached — do not cite related quality-of-care or administration provisions.✓ ours (curated): 42 CFR §483.35 ✓ naive RAG: 42 CFR §483.35 · 1 step · 1.5k tok · 12.3s ✓ auto-enriched: 42 CFR §483.35 · 1 step · 1.3k tok · 1.3s ✓ agent (plan→answer→verify): 42 CFR §483.35 · 3 steps · 3.5k tok · 36.3s | ✓ | ✓ | ✓ | ✓ |
Hearthstone reports RN 0.48 HPRD. How far below the §483.35 RN minimum is it?✓ ours (curated): 0.07 ✗ naive RAG: 0.07 HPRD below the minimum (0.55 − 0.48 = 0.07) · 1 step · 971 tok · 4.2s ✗ auto-enriched: 0.07 HPRD below the 0.55 RN minimum (0.55 − 0.48 = 0.07) · 1 step · 1.5k tok · 4.7s ✗ agent (plan→answer→verify): Hearthstone's 0.48 RN HPRD is 0.07 HPRD below the §483.35 minimum of 0.55 RN HPRD (0.55 − 0.48 = 0.07). · 3 steps · 3.0k tok · 27.6s | ✓ | ✗ | ✗ | ✗ |
Riverside reports total 4.10 HPRD. Is it compliant with the §483.35 total minimum? (yes/no)✓ ours (curated): yes ✓ naive RAG: yes · 1 step · 803 tok · 1.5s ✓ auto-enriched: yes · 1 step · 1.3k tok · 2.4s ✓ agent (plan→answer→verify): yes · 3 steps · 3.4k tok · 31.3s | ✓ | ✓ | ✓ | ✓ |
Of RN HPRD values 0.48, 0.52, 0.24, 0.28, 0.80, how many are below the §483.35 RN minimum?✓ ours (curated): 4 ✓ naive RAG: 4 · 1 step · 1.1k tok · 4.8s ✓ auto-enriched: 4 · 1 step · 1.5k tok · 6.8s ✓ agent (plan→answer→verify): 4 · 3 steps · 3.7k tok · 30.1s | ✓ | ✓ | ✓ | ✓ |
Trinity reports total 3.29 HPRD. By how much is it below the §483.35 total minimum?✓ ours (curated): 0.19 ✗ naive RAG: 0.19 HPRD below the §483.35 total minimum (3.48 − 3.29 = 0.19) · 1 step · 942 tok · 3.7s ✗ auto-enriched: 0.19 HPRD below the §483.35 total minimum (3.48 − 3.29 = 0.19) · 1 step · 1.4k tok · 3.1s ✗ agent (plan→answer→verify): 0.19 HPRD below the §483.35 total minimum (3.48 − 3.29 = 0.19) · 3 steps · 3.1k tok · 25.8s | ✓ | ✗ | ✗ | ✗ |
List the facilities whose reported staffing violates §483.35.✓ ours (curated): Hearthstone, Sagebrook, Trinity, Windcrest ✓ naive RAG: Hearthstone, Sagebrook, Trinity, Windcrest · 1 step · 1.2k tok · 9.1s ✓ auto-enriched: Hearthstone, Sagebrook, Trinity, Windcrest · 1 step · 1.7k tok · 6.4s ✗ agent (plan→answer→verify): Hearthstone — RN HPRD 0.48 (minimum 0.55) and total nurse HPRD 2.88 (minimum 3.48); violates both thresholds under 42 CFR §483.35 (2024 final rule, effective 2024-06-01), Sagebrook — RN HPRD 0.52 (minimum 0.55) and total nurse HPRD 3.23 (minimum 3.48); violates both thresholds under 42 CFR §483.35, Trinity — RN HPRD 0.24 (minimum 0.55) and total nurse HPRD 3.29 (minimum 3.48); violates both thresholds under 42 CFR §483.35, Windcrest — RN HPRD 0.28 (minimum 0.55) and total nurse HPRD 2.91 (minimum 3.48); violates both thresholds under 42 CFR §483.35 · 3 steps · 4.6k tok · 42.9s | ✓ | ✓ | ✓ | ✗ |
Which facilities are compliant with §483.35?✓ ours (curated): Riverside ✗ naive RAG: Riverside only (RN HPRD 0.80 ≥ 0.55; total nurse HPRD 4.10 ≥ 3.48) · 1 step · 1.7k tok · 14.5s ✗ auto-enriched: Riverside · 1 step · 1.5k tok · 9.2s ✗ agent (plan→answer→verify): Only Riverside is compliant with §483.35. Under the 2024 final rule (effective 2024-06-01), §483.35 requires a minimum of 0.55 RN HPRD and 3.48 total nurse HPRD. Riverside (0.80 RN HPRD; 4.10 total HPRD) meets both thresholds. All other facilities fail on at least one metric: Hearthstone (0.48 RN / 2.88 total), Sagebrook (0.52 RN / 3.23 total), Trinity (0.24 RN / 3.29 total), and Windcrest (0.28 RN / 2.91 total) each fall below one or both minimums. · 3 steps · 4.3k tok · 44.5s | ✓ | ✗ | ✗ | ✗ |
Which facilities have a staffing outcome that contradicts the declared Staffing Policy (a policy↔reality conflict)?✓ ours (curated): Hearthstone, Sagebrook, Trinity, Windcrest ✓ naive RAG: Hearthstone, Sagebrook, Trinity, and Windcrest each fall below the 42 CFR §483.35 (2024 final rule) minimums of 0.55 RN HPRD and 3.48 total nurse HPRD, contradicting the Company Staffing Policy to maintain staffing at or above those minimums. Riverside meets or exceeds both thresholds and has no conflict. · 1 step · 1.3k tok · 7.1s ✓ auto-enriched: Hearthstone, Sagebrook, Trinity, and Windcrest each report staffing levels below the CMS §483.35 minimums (0.55 RN HPRD and 3.48 total nurse HPRD), directly contradicting the Company Staffing Policy that requires staffing at or above those minimums. Riverside is compliant. · 1 step · 1.7k tok · 7.6s ✗ agent (plan→answer→verify): Hearthstone (RN 0.48 HPRD < 0.55 minimum; Total 2.88 HPRD < 3.48 minimum — fails both thresholds), Sagebrook (RN 0.52 HPRD < 0.55 minimum; Total 3.23 HPRD < 3.48 minimum — fails both thresholds), Trinity (RN 0.24 HPRD < 0.55 minimum; Total 3.29 HPRD < 3.48 minimum — fails both thresholds), Windcrest (RN 0.28 HPRD < 0.55 minimum; Total 2.91 HPRD < 3.48 minimum — fails both thresholds) · 3 steps · 4.8k tok · 44.2s | ✓ | ✓ | ✓ | ✗ |
Among §483.35 violators, which facility has the lowest RN HPRD?✓ ours (curated): Trinity ✗ naive RAG: Trinity (0.24 RN HPRD) · 1 step · 1.2k tok · 7.2s ✗ auto-enriched: Trinity (0.24 RN HPRD) · 1 step · 1.4k tok · 4.0s ✗ agent (plan→answer→verify): Trinity, with 0.24 RN HPRD — the lowest among §483.35 violators · 3 steps · 4.4k tok · 41.1s | ✓ | ✗ | ✗ | ✗ |
Reproduce: pnpm benchmark --models … --json docs/benchmark-latest.json. Methodology + honest caveats in the strategy doc.