WAHS AI Architecture & Model Strategy

Section 1 · Three Architectural Approaches Under Comparison

Condense into context, retrieve long notes via semantic search, or dynamically query FHIR

Raw FHIR health records are notoriously verbose: a single multi-year synthetic patient chart reaches 18.4 MB (7,099 entries; ~1.35 million raw JSON tokens / 251,641 extracted clinical tokens), and real longitudinal patient exports can reach 100–200 MB with hundreds of free-text clinical notes. The WAHS team is comparing three architectural options—selecting Full-Chart Condensation as the first pilot iteration while running head-to-head evaluations against dynamic FHIR querying and semantic search over long notes.

Option 1 · First Pilot Iteration

Condense FHIR Record into Context

The WAHS engine reads the logged-in patient’s current FHIR data from the HAPI FHIR server that runs as part of the WAHS deployment, shrinks it by 90%–99% into a clean clinical ledger with [N] source links, attaches automated clinical conflict checks, and sends it with the prompt in a single AI call. Because WAHS is a personal health record, this data changes over time: patients download records from their providers’ EHRs (for example via USCDI or TEFCA), enter their own concerns and goals through web forms, and curate what they keep. The condensed ledger must reflect the record as it stands when the question is asked, not a one-time snapshot.

Immediate streaming: Everything happens in ~100 ms inside the engine before a single AI call, so answers stream right away.
No missed LOINC codes: Deterministic clinical rules map terms like “diabetes tests” to HbA1c, glucose, and kidney labs without requiring a small AI model to write raw FHIR queries.
Strict patient boundary: The AI only ever sees the single patient’s pre-scoped bundle.

Option 2 · Unstructured Data & 100–200 MB Charts

Embeddings & RAG for Long Clinical Notes

Free-text clinical notes (such as 4,000- to 14,000-token palliative care, oncology, or hospital discharge notes) carry the most valuable context in a chart, and real patient exports can reach 100–200 MB.

Combines with Option 1: Condense the structured core chart (allergies, active meds, problems, recent labs) into context, and use embeddings / RAG for the long unstructured notes that do not fit.
Question-targeted excerpts: When a question asks about a past hospitalization or specialist note, retrieve the top matching note sections with their [N] source citations.
Single-patient scope: Embeddings and retrieval remain strictly scoped to the authenticated patient’s record.

Option 3 · Under Active Evaluation

Dynamic Agentic FHIR Querying (MCP / API)

For a question like “Have I had any labs for my diabetes?”, the model identifies relevant clinical concepts, translates them into LOINC / SNOMED codes, queries the FHIR endpoint for those observations, and condenses the returned FHIR response.

Terminology challenge for smaller models: Larger frontier models know LOINC/FHIR syntax natively, whereas smaller open-weight models often need an injected knowledge base or a specialist retrieval agent to translate clinical questions into codes accurately.
Retrieved FHIR still needs condensation: Raw FHIR query responses remain token-heavy and benefit from engine-side condensation before synthesis.
Security & multi-agent UX: Requires strict query guardrails so every AI-generated query stays within the logged-in user’s own Patient resources, plus UI progress steps for multi-agent orchestration + retrieval.

How the approaches fit together: These three options are complementary rather than mutually exclusive. Even when records are retrieved dynamically via FHIR queries or semantic search, raw FHIR JSON is costly in tokens and needs condensation. Starting the pilot with Option 1 (Condense what fits into context) while benchmarking Option 2 (Note embeddings) and Option 3 (Dynamic FHIR querying) on smaller models gives WAHS a fast pilot today and a clear path for 100–200 MB records.

Section 2 · Architecture Questions vs. Empirical Evidence

What our live evaluations already answer

Every major technical question raised in our architecture review has been tested against 18 FHIR R4 bundles across 10 patients—including synthetic multi-clinic records up to 18.4 MB (251,641 extracted clinical tokens) and real de-identified hospital/ICU records from PhysioNet MIMIC-IV-FHIR 2.0. The table below links each question to our plain-English findings and source code.

Architecture Question	What Our Live Benchmarks & Code Show	Live Report & Code Links
1. Can lossless & grouped FHIR condensation shrink records by ~90% without losing clinical facts?	Yes — 90% to 99.4% token reduction with 100% safety-core retention. Our deterministic condenser strips FHIR syntax boilerplate, deduplicates repeated lab panels into compact timelines, and prioritizes the HL7 International Patient Summary (IPS) safety core (allergies, active medications, active problems, implants). On an 18.4 MB chart, lossless structural condensation cuts tokens by >90%, and grouped clinical condensation fits the core chart into 8,000 tokens (99.4% reduction) while recording every omitted item in a completeness manifest.	Phase 2 Condensation Report ↗ `condenser.py` on GitHub ↗
2. What happens if a patient asks about a diabetes lab or hospital note that was squeezed out of the condensed summary?	Pre-call overflow lookup wins back +19 to +21 benchmark points. When older labs or long clinical notes are omitted at tighter budgets, the engine’s Full-Chart Overflow Lookup searches the patient’s full uncondensed record for the concepts in the user’s question and attaches the matching records with [N] citations before calling the AI. On compact 1k summaries, this raised Gemma 4 12B scores from 47.0/100 (Direct AI) to 68.2/100 (Med Agent Hub, 0 unsafe answers).	MacBook 96k vs. Hub Report ↗ `wahs_tools.py` on GitHub ↗
3. Smaller models don’t know LOINC codes or subtle drug interactions — how much does injecting a clinical knowledge base help?	Injecting clinical rules helps every model ≥ 4B, and pairing rules with a patient relationship graph helps even more. Across 972 runs on 36 clinical accuracy & abstention scenarios, adding static clinical rules (Direct + KB) improved accuracy and reduced unsafe answers across all three model families. Routing through the full Med Agent Hub (which links each patient’s problems, medications, lab trends, and conflicting clinic notes into a compact graph before generation) delivered an even larger jump — cutting unsafe answers by 43% to 71%.	9-Model × 3-Pathway Report ↗ `clinical_graph.py` on GitHub ↗
4. Why should condensation happen inside the WAHS engine rather than sending giant 64k–128k charts to the GPU?	Exceeding the GPU server’s active memory window silently cuts off the top of the chart. On a shared 24 GB cloud GPU with a 16k active memory window, sending 32k–128k of chart text caused the server to drop the beginning of the prompt (erasing allergies and active medications) and collapse chart-only scores to 0.0–2.8/100. Keeping condensation inside the WAHS engine lets it size the chart to the model’s sweet spot (8k–16k) and dynamically request a larger GPU memory window only when needed.	1k–128k Context Limit Report ↗ Engine Internals Guide ↓
5. How do we prevent the AI from guessing when asked about facts not in the chart?	Grounded abstention and prompt-coercion guardrails. Across benchmark stress tests that ask about undocumented procedures or allergies (or pressure the model to guess), Med Agent Hub requires every claim to cite a source record and explicitly abstain when information is absent — holding Gemma 4 12B to 2/36 unsafe answers and 0 fabricated citations across the suite.	Suite A + Suite B Report ↗

Section 3 · Model & Pathway Selection (972 Live Runs)

Why Google Gemma 4 + Med Agent Hub is the clear choice for a single-model pilot

To choose the best open-weight model for our single-call streaming pilot, we tested nine models across three families — Google Gemma 4 (E4B, 12B, 26B), Alibaba Qwen 3 (4B, 9B, 14B), and Meta Llama 3.x / Hermes 3 (3.2B, 8B, Hermes 3 8B) — across three pathways at an 8k chart budget across 36 clinical accuracy and abstention scenarios (972 total answers, graded in a blinded review by Claude Opus 4.6 and GPT-5.4).

Model Family & Size	1. Direct AI (Chart Only)	2. Direct AI + Clinical KB Rules	3. Full Med Agent Hub (Chart + KB + Graph)	Fabricated Citations (Hub)	Key Takeaway for WAHS Pilot
Google Gemma 4 26B 26B MoE (4B active)	71.3 / 100 8.01 acc · 6/36 unsafe	72.9 / 100 8.08 acc · 4/36 unsafe	82.1 / 100 (#1) 8.62 acc · 3/36 unsafe (−50%)	0 / 36 23 fully supported	Highest overall accuracy & completeness; ideal when running on the shared UW GPU server.
Google Gemma 4 12B 12B dense (Default)	67.2 / 100 7.71 acc · 7/36 unsafe	70.9 / 100 7.72 acc · 4/36 unsafe	80.5 / 100 (#2) 8.53 acc · 2/36 unsafe (−71%)	0 / 36 21 fully supported	Recommended single-model sweet spot: within 1.6 points of 26B, with the lowest unsafe rate (2/36) and lowest confabulations (1/36) in the entire study.
Google Gemma 4 E4B 4B effective parameters	62.8 / 100 7.50 acc · 8/36 unsafe	64.8 / 100 7.50 acc · 5/36 unsafe	70.4 / 100 (#6) 8.01 acc · 4/36 unsafe (−50%)	2 / 36 18 fully supported	Best fast/compact model: at 4B effective size, pairing E4B with Med Agent Hub beats raw Gemma 12B (67.2) and beats Qwen 3 14B + Hub (65.4).
Alibaba Qwen 3 14B 14B dense	53.4 / 100 6.36 acc · 13/36 unsafe	61.8 / 100 6.88 acc · 6/36 unsafe	65.4 / 100 (#8) 7.36 acc · 5/36 unsafe (−62%)	2 / 36 12 fully supported	Strong +12.0 point gain and −62% unsafe reduction from Med Agent Hub, but trails Gemma 4 12B by 15.1 points.
Alibaba Qwen 3.5 9B 8B/9B dense	49.4 / 100 5.88 acc · 12/36 unsafe	54.0 / 100 6.31 acc · 7/36 unsafe	57.1 / 100 (#12) 6.39 acc · 10/36 unsafe	0 / 36 7 fully supported	Consistent accuracy and completeness gains (+7.7 points) with Med Agent Hub.
Meta Llama 3.1 8B 8B dense	50.6 / 100 6.47 acc · 14/36 unsafe	52.6 / 100 6.25 acc · 9/36 unsafe	55.3 / 100 (#13) 6.42 acc · 8/36 unsafe (−43%)	0 / 36 2 fully supported	Hub cuts unsafe answers by 43%, but Llama 3.1 8B frequently omits inline [N] citations on secondary claims.
Alibaba Qwen 3 4B 4B dense	34.7 / 100 5.68 acc · 12/36 unsafe	42.7 / 100 6.36 acc · 8/36 unsafe	43.5 / 100 (#22) 6.38 acc · 8/36 unsafe (−33%)	0 / 36 16 fully supported	Gains +8.8 points and −33% unsafe answers with Hub, but verbose output lowers relevance scores.
Nous Hermes 3 8B Llama 3.1 8B fine-tune	43.3 / 100 5.36 acc · 11/36 unsafe	46.5 / 100 5.58 acc · 10/36 unsafe	43.0 / 100 (#24) 5.31 acc · 8/36 unsafe (−27%)	0 / 36 2 fully supported	Hub reduces unsafe answers from 11/36 down to 8/36, though citation density remains low.
Meta Llama 3.2 3B 3.2B dense	44.2 / 100 5.79 acc · 9/36 unsafe	44.2 / 100 5.62 acc · 9/36 unsafe	33.8 / 100 (#27) 4.68 acc · 15/36 unsafe	0 / 36 1 fully supported	Capability floor identified: 3.2B struggles to synthesize multi-clinic records + clinical rules simultaneously; 4B effective parameters (Gemma E4B) is the practical minimum.

Section 4 · Pilot System Architecture

How the WAHS Frontend, the WAHS Engine, and the UW GPU server fit together

Following the team’s architecture review, the WAHS pilot separates responsibilities across three clean layers so the FHIR store stays standard, the WAHS frontend stays simple, and the GPU server only runs model inference.

[1. WAHS Frontend & Auth Proxy] • Patient logs in; WAHS auth proxy limits FHIR access to the user's own Patient resources • Simple Chat Tab sends OpenAI-format request (conversation history + patient context) │ ▼ (OpenAI REST /v1/chat/completions, stream=true) [2. WAHS Engine (Scoped from Med Agent Hub · Separate Server)] • Reads the patient's current records from the WAHS HAPI FHIR server and condenses them by 90%–99% (condenser.py) • Runs deterministic 4-Layer Clinical KB + Patient Relationship Graph (clinical_graph.py) • Pulls omitted clinical notes / labs matching the user's question (wahs_tools.py) • Keeps [System Prompt + 8k Condensed Chart] as a fixed prefix across follow-up turns (rebuilt when HAPI records change) │ ▼ (OpenAI REST /v1/chat/completions + KV prefix cache reuse) [3. UW CIRG GPU Server (Gemma 4 12B / 26B / E4B)] • Streams tokens back immediately (single-call workflow; < 0.3s follow-up prefill with cache) • Engine attaches [N] → FHIR resource UUID & date mappings for clickable source inspection

Security & Access Control

Enforcing the single-patient boundary before any AI call

Scoped at login: Existing authentication proxies in WAHS enforce OAuth scopes that limit access to the logged-in user’s own records in the WAHS HAPI FHIR server, which span several Patient resources (one created at first login, one per data import, and one per IPS summary), before the WAHS Engine processes them.
No cross-patient query surface: In the pilot, the AI model never writes database queries against a multi-patient FHIR server; all condensation, clinical rule checks, and note lookups run strictly inside the logged-in patient’s isolated bundle.
Patient-controlled sharing: Patients choose which of their records to share by creating an International Patient Summary (IPS) within WAHS and sharing it via a SMART Health Link.

Multi-Turn Memory & Fast Streaming UX

Instant follow-up answers via GPU prefix caching

Multi-turn from day one: The WAHS chat interface passes the conversation history with each turn so patients can ask natural follow-up questions.
Prefix-cached chart memory: By placing the static [1]...[N] condensed patient chart at the very top of the prompt and appending turn-specific rule alerts and chat turns below it, the GPU server reuses its cached memory (KV cache) across follow-up questions for the same patient. When the patient’s records change, the condensed chart is rebuilt so answers reflect the current record.
Transparent progress cues: Single-call streaming starts almost immediately for the pilot; as multi-step retrieval or specialist checks are added later, lightweight status steps (“Checking medication conflicts…”, “Reading discharge note…”) keep the user informed.

Section 5 · Next Steps & Staged Development Plan

Immediate action items and horizon roadmap

With the condensation pipeline, 4-Layer Clinical KB, patient relationship graph, and 9-model benchmark complete, our next development cycle focuses on six concrete deliverables to launch the pilot chat experience in WAHS and evaluate retrieval strategies for larger records.

Step	Workstream	Concrete Deliverable	Status / Reference
1	Share Condensation & Graph Code for Team Review	Share the non-destructive FHIR condenser (condenser.py), clinical entity extractor (extractors.py), and pre-generation clinical graph (clinical_graph.py) alongside the 972-run benchmark report.	Ready Today `server/fhir/` repo ↗
2	Extract Simplified WAHS Engine from Med Agent Hub	Package a clean, standalone WAHS Engine service (free of legacy OpenMRS/Catalyst routes) that accepts OpenAI-format requests from the WAHS frontend, condenses the patient’s current records from the WAHS HAPI FHIR server in the engine, and streams responses from the UW GPU server.	Pilot Iteration 1
3	Build Open-Ended Streaming Chat Tab in WAHS	Build a dedicated, open-ended Chat tab in the WAHS web application (text input + immediate streamed output via OpenAI REST spec) and evaluate pre-built frontend patterns for conversation history persistence and export.	Pilot Iteration 1
4	Align on Session Memory & GPU Prefix Caching	Compare Ollama session IDs, LiteLLM session/caching capabilities, and repeated context with fixed chart prefixes so follow-up questions on the same patient leverage GPU-side caching without duplicating proxy logic. Any session or caching mechanism adopted must meet three requirements. Security: no user’s or patient’s interaction can ever reach another user’s or patient’s data, whether through session IDs, response caches or shared prefix caches. Data timeliness: cached context is discarded when the patient’s records in WAHS change, so answers never draw on data that has since been added, corrected or deleted. Reliability: a cache miss, eviction or restart may slow a response but never changes its content.	Pilot Iteration 1
5	Benchmark Condensation vs. Dynamic FHIR Querying & Note RAG	Run head-to-head evaluations on smaller open-weight models comparing full-chart condensation against dynamic FHIR querying (translating clinical questions like “Have I had any labs for my diabetes?” into LOINC queries and condensing returned observations) and specialist multi-agent retrieval.	Next Evaluation
6	Real-Patient Long-Note (4k–14k Tokens) & 100 MB+ Record Testing	Expand test coverage with fuller real-world patient exports and long (4,000–14,000 token) palliative/specialty notes, combining core chart condensation with single-patient semantic note chunk retrieval.	Phase 2 Expansion

Section 6 · Engine Internals Reference (Med Agent Hub)

Progressive 9-budget condensation, `ClinicalEntityGraph`, and prompt enrichment

For engineers inspecting the implementation in Med Agent Hub (server/fhir/), every condensed chart begins with patient demographics at [1] and a machine-readable Completeness Manifest at [2]. When a KB-guided profile executes (such as wahs-12b-kb or wahs-26b-kb), build_wahs_hub_enrichment_block(...) runs conditional Tier-0 overflow recovery, builds a patient-specific ClinicalEntityGraph, and attaches up to 800 tokens of matched clinical guidelines before calling the model.

Budget Window	Condensation Strategy	Omitted Clinical Items	Tier-0 Overflow Recovery
256k / 128k	Max-window stress-test ceiling expanding multi-decade and ICU charts up to 251,641 extracted clinical tokens.	`omitted_clinical_items = 0`	Skipped (`overflow_triggered: false`)
64k / 32k / 16k	Full longitudinal clinical ledger preserving complete lab timelines, encounters, and decoded UTF-8 clinical notes.	`omitted_clinical_items = 0` on standard charts	Skipped unless massive 7,000-entry export overflows
8k / 4k	Recommended Limit sweet spot: preserves active conditions, active medications, allergies, immunizations, and folded lab trajectories.	`0` on typical charts; non-zero on multi-decade exports	Conditional — runs only when `omitted_clinical_items > 0`
2k / 1k	Strict HL7 IPS Safety Core micro-capsule; older clinical notes and routine historical lab panels overflow the ceiling.	`omitted_clinical_items > 0`	Active — `wahs_tools.py` recovers question-matched notes and labs with sequential `[N]` citations

Inspect ClinicalEntityGraph typed relationship edges & orchestration prompt block

server/fhir/clinical_graph.py links seven FHIR resource categories (Condition, MedicationRequest, Observation, DocumentReference, AllergyIntolerance, Immunization, and Organization) across five typed clinical edges while preserving original [N] citation numbers so 4B–12B models do not have to infer multi-hop cardiorenal or anticoagulation hazards from raw text alone. Paired wahs-*-raw profiles omit this block for zero-hint baseline ablations.

Graph Edge Type	Source → Target Nodes	Clinical Logic & Knowledge Base Grounding
TREATS	`MedicationRequest` → `Condition`	Maps active medications to documented clinical indications via RxNorm class lookup and AHA/ACC/ADA/KDIGO mappings (e.g., `Apixaban [6] --[TREATS]--> Atrial fibrillation [5]`).
CONTRAINDICATED_OR_DOSE_ADJUST_IN	`MedicationRequest` → `Condition` / `Observation` / `AllergyIntolerance`	Flags renal dosing thresholds, hyperkalemia hazards, supratherapeutic INR, NSAID nephrotoxicity/bleeding, and allergy cross-reactivity (e.g., `Spironolactone [7] --[CONTRAINDICATED_OR_DOSE_ADJUST_IN]--> Potassium 5.6 mEq/L [10]`).
MONITORED_BY	`Condition` / `MedicationRequest` → `Observation`	Connects chronic diseases and high-alert therapies to their longitudinal LOINC laboratory and vital trajectories (eGFR, Creatinine, Potassium, HbA1c, INR, NT-proBNP, LDL-C, TSH).
CONTRADICTED_OR_MODIFIED_BY_NOTE	`MedicationRequest` / `Condition` → `DocumentReference`	Detects narrative-vs-structured discrepancies where a clinical note documents self-discontinuation, a hospital hold order, walk-in urgent care changes, or disease stage progression.
REPORTED_BY_ORG	Clinical Node → `Organization`	Preserves multi-facility care provenance (e.g., primary health system vs. outside urgent care or external reference laboratory).

[MED-AGENT-HUB ORCHESTRATION: CLINICAL ENTITY SUBGRAPH, QUESTION KB & TIER-0 OVERFLOW RECOVERY] Hub Verification Directive: Preserve inline [N] citations when referencing linked clinical entities or Tier-0 overflow-recovered records below. 1. Tier-0 Overflow Recovery (ACTIVE: omitted_clinical_items=6 [clinical_note=2, lab_observation=4]; recovered=3 records via wahs_tools): [6] (2026-01-18) Tier-0 Recovered Clinical Note: Outside Urgent Care Walk-In Note — Walk-in visit at UrgentCare Plus Medical Center: Patient states she stopped taking spironolactone 5 days ago... [UUID: note-outside-urgentcare-99] 2. Patient Clinical Entity Subgraph (8 nodes, 7 edges): - [7] Spironolactone 25 MG Oral Tablet --[CONTRAINDICATED_OR_DOSE_ADJUST_IN]--> [10] Potassium (latest 5.6 mEq/L) | RAAS / MRA cardiorenal safety - [7] Spironolactone 25 MG Oral Tablet --[CONTRADICTED_OR_MODIFIED_BY_NOTE]--> [11] Cardiology Walk-In Note | Structured MedicationRequest modified in note (stopped) 3. Question-Conditioned Clinical Knowledge Base (4 matched guidelines): - [KB wahs-kb-rx-hyperkalemia-01 | AHA/ACC/HFSA & KDIGO] Spironolactone & Hyperkalemia in Stage 3B/4 CKD: Hold MRA when serum K+ > 5.0–5.5 mEq/L...

How WAHS connects open-weight AI to patient health records safely and fast.