Make answers traceable
ChartSearchAI shows supporting chart entries. Our tooling records model work and checks so reviewers can inspect what happened.
Explore the transfer options →How our experience can help people review their health records—and what a small pilot could teach us.
AI flags a possible conflict, a person adds context, and they refine a proposed resolution together. Explore that cycle on a few example cases, keeping sources visible and the person in control. Our methods can transfer before choosing shared tools or code.
ChartSearchAI shows supporting chart entries. Our tooling records model work and checks so reviewers can inspect what happened.
Explore the transfer options →Discuss a flagged difference, revise the proposed resolution and record a decision. Unresolved differences can stay visible.
Walk through the interaction →Study missed disagreements, incorrect combinations, useful corrections and whether people understand AI labels.
Explore research questions →ChartSearchAI identifies supporting records. The AI Transparency guide adds a way to share which AI helped produce a result, what it changed and what human involvement was recorded.
Its review-before-action approach can guide WAHS design without using Catalyst code. Decisions about patient records, corrections and sharing still belong to WAHS.
The discussion suggests fewer than a dozen records per person. One record could still contain many entries. Inspect examples before deciding how much information a model can read at once.
The updated discussion covers four patient-facing needs: resolve conflicting records, build a useful sharing summary, find information through questions, and show AI involvement. Smaller models, record preparation and shared computing support those needs.
Finding useful entries → AI history and human review → Models and shared computing →
The first contribution can be a better method or a shared review session. Using OpenClinAI software is an option, not a condition.
Show original sources, distinguish dates, review ambiguous cases and record why a decision was made.
No code reuse neededUse Catalyst’s review-before-action approach to sketch a WAHS workflow that supports correction and unresolved differences.
No code reuse neededUse the evaluation harness and Med Agent Hub to run the same cases and inspect outputs, errors and reviewer decisions.
A small pilot runner is neededConsider record-reading, search or AI-history components once sample data and the intended workflow make the benefit clear.
Compatibility must be tested| Opportunity | Our foundation | How it could help WAHS | First useful step |
|---|---|---|---|
| Answers with sources | Code reviewed ChartSearchAI and Hub retain supporting record references and checks. | Teach and test the habit of showing the exact entry behind a statement.Evidence and limitsA reference helps inspection. It does not guarantee that the answer interprets the record correctly. Source-handling code → | Methods first |
| Ask questions about records | Code reviewed QueryStore prepares and searches OpenMRS entries; ChartSearchAI uses retrieved records to support answers. | Explore a question-and-answer view that opens the supporting entries.What transfers firstFirst test what information must stay visible and whether search misses relevant entries. A WAHS connection is additional work. How the search approach works → | Examples, then search |
| A read-only record view | Code reviewed QueryStore offers patient records, search, dates and version markers through OpenMRS. | Give people and models a stable view of the same entries while keeping the original system authoritative.Where QueryStore fitsDirect use fits an OpenMRS test source. For WAHS exports, start with the same read-only approach; a connector or separate reader would be needed. Explore the read-only option → | Methods or tools |
| Resolving disagreements | Experience to adapt Record-level evidence review and Catalyst’s inspect-before-action approach. | Flag a difference, exchange context and refine a proposed resolution until a person accepts it or leaves it unresolved.What must be designedPatient statements, clinical decisions and sharing permissions need distinct treatment. No WAHS reconciliation engine was verified. Review workflow → | Walk through cases |
| A useful sharing summary | Experience to adapt Answers tied to chart records and review of generated work. | Choose who the summary is for, inspect what it includes and catch important omissions.What remains openThe recipient, purpose and handling of unresolved differences need agreement with WAHS. People and sharing → | Sketch and review |
| Clear AI history | Related foundation Source references and internal records of model work. | Show what AI contributed and what human involvement was recorded. Exchange that history using the FHIR guide.Additional workReading and writing the guide’s labels and linked history is new work. Existing internal evidence does not establish compatibility. Three kinds of evidence → | Inspect one example |
| A repeatable pilot | Code reviewed Harness run records and Hub model calls. | Run small examples, preserve inputs and outputs, and compare them with human judgments.Additional workAdd the WAHS case format, a runner and review notes. The general Hub endpoint does not provide all of the richer Catalyst request records. Pilot ideas → | A small runner |
| Model and context choices | Methods to apply Experience with local models, limited input space and comparative runs. | Measure errors, response time and information needs on the actual task before choosing infrastructure.What evidence is neededPerformance on clinical queries does not establish performance on record reconciliation. Research questions → | One model, then compare |
| An understandable patient view | Research direction Language and participation questions in the research programme; dashboard design experience. | Test wording and correction with intended users. The updated notes place multilingual support in a future discussion; reading, search and translation will each need testing.Evidence still neededNeither translation quality nor patient understanding was measured in this report. Catalyst’s analytics dashboard serves a different purpose. Research scope → | Review with users |
“Code reviewed” means the named behavior was found in the cited source. It does not mean it has been tested with WAHS.
Its most useful lesson is to keep the original record intact. A read-only source view can serve both a person and a model while draft resolutions, comments and accepted decisions are recorded separately.
QueryStore can return a patient’s indexed entries as readable text with source identifiers and clinical dates. It supports full-chart pages and searches. The harness could capture a view and pass selected entries to Hub.
Use the complete available view as the first comparison. Search can be a later experiment.
See the verified interface →A simple file reader can expose entries with their source, date and original content. That may be enough for a small pilot.
The current QueryStore module is built for OpenMRS. Support for arbitrary WAHS FHIR exports would need to be designed and tested.
The notes propose a first version of search-assisted answering, often called RAG. The useful pattern is simple: prepare readable entries, find relevant ones, then answer with their sources visible.
Keep names, values, units, dates and source links intact.
Read a small case in full or search a larger record.
Use the selected entries and show what supports the answer.
Open the source, spot an omission and ask again.
In the reviewed OpenMRS code, QueryStore turns clinical objects into compact readable text with metadata and source identifiers. An embedding is a numerical representation used to find similar text; it does not replace the source record.
The Elasticsearch path stores vectors alongside record content and can search using an embedding of the question. Other search backends also exist. Which embedding model runs depends on configuration.
The notes name E5-base-v2. The reviewed harness configuration names a different default, so the active model must be checked on the chosen server. The E5 model card describes English-only input and a limited input length; it does not establish multilingual support.
Code and model evidence →Use a few fictional exports and write down which fields a person needs to answer each question. Give a model the complete small case first. Add search only when size or usefulness justifies it, and inspect what it leaves out. A missing source can hide a conflict even when an answer sounds convincing.
This exercise can produce a reusable case pack and review method. Using QueryStore directly would additionally require an OpenMRS source or a tested WAHS adapter.
Med Agent Hub offers configured model calls and reusable prompts. The harness helps preserve the inputs, outputs and test conditions. Compare record-finding, answer quality and response time separately.
Some Hub workflows let a model request an available specialist tool. Other steps remain fixed by the application. A WAHS review loop still needs its own task instructions and human decisions.
Verified model and workflow boundaries →The notes describe a planned shared GPU allocation for DIGI. OpenMRS could call a model service over an API while remaining on its existing host.
Availability, access and model performance still need confirmation. A locally hosted model can support private deployment, but operation without internet access needs an end-to-end check.
What is planned and what is verified →Keep multilingual support as a separate follow-up. Test finding records, answering and user understanding in each intended language; translating the screen alone is not enough.
Adapt Catalyst’s iterative review approach: AI and a person work through a flagged conflict together, revising the proposed resolution as context becomes clearer. Keep the original records and the history of decisions.
Proposed workflow · to explore with WAHSAI identifies a possible conflict and shows its sources.
The person adds context, asks questions or corrects an assumption.
AI suggests a resolution and explains what it would change.
The person and AI revise the proposal as often as needed.
The person accepts, edits, rejects or leaves it unresolved.
Return to the sources and discussion whenever a proposal needs more context. Record an accepted resolution separately from the originals, with its explanation and human decision. Unresolved is a valid outcome. Previewing and sharing the resulting view is a separate step.
The difference may be a dose change, a duplicate import, or a real disagreement. A patient can state what they currently take. That statement does not silently replace the prescription. AI might then propose treating the entries as a change over time, if the dates and supporting records justify it. The person can correct that interpretation, request another revision or leave the difference unresolved.
Concept illustration; not a patient record or clinical recommendation.
Which record supports this answer?
Links to chart entries and checks of their use.Which system affected this output?
AI system, degree of involvement, input sources and affected entry or field.Who reviewed or changed it?
A patient statement and a clinician’s review are different kinds of input.In FHIR, this linked history is called Provenance. A model card describes the model and its intended use. Agree the guide version with the receiving system. Profile and diagram
An AI-assisted reconciled view or sharing summary would make WAHS an AI data producer. Link the new output to its sources and recorded AI and human contributions.
For an International Patient Summary, agree how this history is included in the shared FHIR bundle. One reconciled view can still contain several resources and unresolved differences.
As an AI data consumer, show whether AI was involved, how much it contributed and whether human review is recorded. Distinguish a label on a whole entry from one on a single field.
The supplied Scenario 1 criteria ask a reviewer to determine these facts without reading raw data. An optional extension is opening the model card from a Device attachment, Device note or linked DocumentReference. These technical forms should lead to a simple “About this model” action.
Use a partner-provided example and the agreed guide version. This report has not demonstrated those exchange paths.
Scenario criteria →WAHS offers a practical setting for questions already present in the research programme. The contribution could be a method, a set of cases or evidence about a design choice.
A useful pilot can inform this question. A publishable study would also need a literature review, a study plan and suitable participants.
Try: Give the model all entries in a small case. Compare that with selected entries only when size becomes a problem. QueryStore can support a search comparison for OpenMRS sources.
Learn: Whether selection hides a relevant disagreement, and how much time and input space each approach uses.
Research return: Cases that expose the trade-off between a complete view and a focused view.
Related research →Try: Begin with a guided cycle: read, flag, discuss, propose, revise and decide. If a case needs more information, compare it with a model that can request another read or an available specialist tool.
Learn: Whether flexible steps find better evidence, introduce new errors or increase time and cost. Keep the same source cases and human authority in both approaches.
Research return: Evidence about when orchestration helps. Existing Hub tool use is a starting example, not a general WAHS agent system.
Reviewed Hub behavior →Try: Follow a conflict from the initial flag through human input and revised proposals to an accepted resolution or an unresolved decision. Inspect how corrections change the next suggestion.
Learn: Missed disagreements, incorrect resolutions, useful corrections, and whether later suggestions honor the person’s input.
Research return: A review method that distinguishes computer checks, patient knowledge and clinical judgment.
Related research →Try: Reject a suggestion, correct an entry or change the sharing purpose. Check what appears in the next draft.
Learn: Whether rejected or outdated suggestions reappear, whether edits survive a failure, and whether unresolved items stay visible.
Research return: Repeatable examples for studying control over AI-assisted work.
Related research →Try: Ask readers to identify the source, what AI changed and who reviewed it, using a concise display with expandable details.
Learn: What readers interpret correctly, where they become too confident, and which wording helps them correct a mistake.
Research return: Evidence about an understandable way to display AI history. A label alone does not establish understanding.
Design research →Try: After the first workflow is useful, run the same reviewed cases on a second model with the same instructions and inputs.
Learn: Which cases succeed, when a model admits uncertainty, and the time and computing resources needed. Test local and disconnected use explicitly if those are requirements; assess languages separately.
Research return: Task-specific evidence for model choice. A small demonstration cannot establish deployment readiness.
Existing evaluation plan →A short abstract for exploring how AI and people can flag and resolve conflicting records together. Search and AI history are supporting opportunities.
Explore an iterative conversation around a few fictional medication histories. AI flags a possible conflict and shows the source entries. A person adds context or challenges the interpretation; AI proposes and revises a resolution in response. The person can accept, edit, reject or leave it unresolved. Preserve the originals, label patient comments and record the reasoning behind each decision.
What we could learn: Whether the back-and-forth improves resolutions, catches mistaken assumptions and respects human corrections without forcing agreement.
What we bring: ChartSearchAI’s source references, Catalyst’s iterative review approach, and optional use of the harness and Hub to inspect successive proposals and decisions.
Related review approach →Explore a simple question-and-answer view over a few example records. Let a reader open the entries supporting each answer. Compare reading a small case in full with selecting relevant entries when search would be useful.
What we could learn: Whether the right information is found, what search leaves out and whether readers can check an answer.
What we bring: ChartSearchAI and QueryStore retrieval experience, reusable case-review methods and optional shared evaluation tools. Direct QueryStore use fits an OpenMRS source; WAHS data access would need separate work.
Related search approach →Use a partner’s example to explore a display of whether AI was involved, how much it contributed and what human review is recorded. Compare labels on a whole entry and on one field, with model information available on request.
What we could learn: Whether a reader can understand the supplied connectathon facts without inspecting raw data.
What we bring: Experience showing sources and review evidence, plus a clear distinction between supporting records and the FHIR history of how a result was produced. Reading the guide’s exchange formats would be new work.
Related attribution approach →Use the lead abstract to agree a useful example and who will review it. Discuss what counts as a helpful resolution and when a conflict should remain open. Keep effort and implementation choices for that conversation.
A source-linked sharing summary could also be a useful variant. Shared computing, model comparisons and multilingual support can follow the chosen question. These ideas can contribute methods, examples and research findings even when no OpenClinAI code is reused.
See what supports the report, what is proposed and what still needs to be tested.
Most code and standards links are public. Meeting and research links require the relevant team permissions; each has a summary here.
Meeting page version 5, updated 1 September 2026. It records workstreams and open design questions. A generated feature appendix is a discussion aid, not an approved specification. The updated notes supplied on 9 September add search-assisted answering, record preparation, local model assessment, shared GPU plans and a later multilingual discussion. Timing and the meaning of “record” need partner confirmation. Access requires the team’s Confluence permissions.
Meeting notes · team access ↗In the configured Hub path, each supporting record keeps its identity, dates and text; available source details are retained. The engine records checks of how an answer uses those records. This is evidence of code behavior, not a WAHS integration result.
Evidence mapping ↗ · Grounding records ↗The patient-record GET interface delegates to the existing read service. It supports chart pages, searches, source identifiers, clinical dates and version markers. It requires OpenMRS read privileges. Reading the index does not establish that every source entry is present.
This module is tied to OpenMRS; no arbitrary WAHS FHIR import was verified. Index repair operations have separate administrative privileges. The pilot must distinguish access to a patient’s own data from broader staff access.
Read interface and limits ↗ · Request handler ↗The reviewed module prepares readable text and structured metadata, retains source identities, and supports configurable embedding providers. Elasticsearch is one supported search backend. The notes’ “stripped-down JSON” and “numeric identifiers” are shorthand, not a full description of this code.
The E5-base-v2 model card describes English-only support and a 512-token input limit. The notes identify that model, while the harness configuration names MiniLM by default; neither source establishes the active configuration on another server.
Record preparation code ↗ · Embedding providers ↗ · Harness model default ↗ · E5 model card ↗Hub has reusable prompts, configured model calls and bounded specialist tool use. Its application also controls fixed steps, including preparing evidence before that tool loop. This is a useful pattern for collaboration, but a WAHS task and review workflow still need implementation.
Model requests use explicitly supported options. A YAML file does not guarantee that every Ollama or vLLM setting will be forwarded or honored. The reviewed Hub maps a profile’s “dry” setting to “dry_multiplier”; the note’s intended setting and chosen server still need confirmation. Check sampling controls against both Hub and that server.
Prompts ↗ · Tool use ↗ · Supported settings ↗The notes describe provisioning a shared GPU service, with a proposed H200 MIG 1g.18gb allocation for DIGI. NVIDIA documents that profile; this confirms the option exists, not that the allocation is ready or that a particular model will fit the intended workload.
Docker Model Runner and an OpenAI-compatible HTTPS API were discussed as serving options; the endpoint and exact configuration remain open. CNICS data was mentioned as a possible future research direction, not as an available or approved pilot dataset. The first pilot here uses fictional examples.
NVIDIA H200 profiles ↗The supplied excerpt requires a readable display of AI involvement, degree of involvement and recorded human review, with whole-resource and element-level labels distinguished. Opening model information is an optional extension. The scenario page could not be fetched independently.
Connectathon scenario ↗The general /v1/hub/generate endpoint runs a model and returns its text. The separate Catalyst query-role path records the exact request and settings, plus size measurements when the server provides them; missing measurements are reported. A WAHS runner must record its own evidence; it cannot assume the general endpoint returns that richer record.
Request evidence code ↗The harness can record the code version, data version, model provider and test collection for a run. WAHS cases, expected outcomes and reviewer records still need to be added.
Run manifest code ↗Catalyst is a database query and dashboard workbench. The reviewed roadmap defines a generic SQL connection, visible checks and a person choosing the query to run. The public OpenClinAI site also presents a FHIR Data Pipes and Spark demonstration, with a newer implementation account than that roadmap snapshot. This report did not match the running demonstration to a deployed code revision or rerun its comparison. That review approach can inform WAHS, while record editing and sharing require separate design.
Catalyst roadmap ↗ · Public demonstrations ↗FHIR is a standard for exchanging health information. This draft guide links a result to information about AI involvement, source inputs and people involved. A simple “AI involved” label does not say how much changed or whether a clinician reviewed it. Use a recorded guide version when testing.
Requirements and diagram ↗ · AI Provenance profile ↗One shared FHIR document can contain several entries with their history. Corrections to an issued document create a new document instance; the original remains unchanged.
FHIR documents ↗The existing research focuses on adapting WHO SMART Guidelines for HIV care using open models. WAHS could test whether the methods work in another setting. The linked research repository is private; the summary here is sufficient to follow this report.
Dissertation scope ↗ · Paper plan ↗The September research note proposes checking individual steps and running small experiments. This report uses those as research directions; it does not repeat its wider literature claims as established findings.
Research sitrep ↗The research compares giving a model selected information with letting it search in several steps. Small WAHS cases also permit a simpler comparison: give the model all the relevant entries.
Retrieval canvas ↗The relevant question is whether the application honors a person’s current decisions even when the model proposes something else. Wider product and literature claims in the memo were not checked for this report.
Research memo ↗This is a draft study plan for HIV guideline tasks, with choices still open. A WAHS study would need its own questions, measures and review plan.
Protocol draft ↗The recommendation and main comparisons stay visible. Supporting evidence expands in place with descriptive labels.
NN/g · Progressive disclosure ↗The proposed workflow keeps editing, rejection and recovery explicit. Explanation is kept separate from evidence of correctness.
Microsoft HAX · Correction ↗Expandable details keep the overview readable. Keyboard focus and layouts for smaller screens help readers move through the report.
USWDS · Disclosure guidance ↗Prepared by Piotr Mańkowski with AI-assisted research and drafting. Initial report reviewed on 9 September 2026; Phase 2 prototype and evaluation added on 30 October 2026.
To test the pilot ideas on realistic patient exports, we implemented a 5-Tier Progressive FHIR R4 / US Core / IPS Chart Condenser and a 4-Layer Clinical Knowledge Base in Med Agent Hub, evaluated across three synthetic test charts (Byron202_Emard19, Lamar193_Beer512 and Maurice742_Auer97 — 23.6 MB total).
Inspect duplicate labs and Base64 note deduplication, cross-clinic medication and condition reconciliation, longitudinal biomarker trajectories, privacy-segmented sharing (patient_self vs. external_share) and grounded Q&A with clickable [N] → FHIR resourceUuid provenance.
Condenses Maurice’s 84-year, 18.36 MB (7,099-entry) FHIR R4 Bundle into any target context budget (16k down to 1k tokens) while retaining 100% of the IPS clinical safety core and [N] → resourceUuid provenance.
Deterministic normalizers resolve 2,629 / 2,629 clinical instances across SNOMED CT (KDIGO/ADA stage exclusivity), RxNorm (brand↔generic & statin class rules), CDC ACIP (CVX schedules) and LOINC reference ranges.
Passes all 32 evaluation cases across Use Case A (reconciliation), B-1 (encounter & longitudinal summaries, including the Nov 2023 CABG ICU stay), B-2 (0.0% sensitive SDOH leakage in external_share) and B-3 (100% abstention on unrecorded facts).
Why patient-mediated multi-clinic summaries cannot rely on raw FHIR prompting alone—and how health interoperability standards, clinical practice guidelines, privacy law, and human-AI interaction research ground each layer of the WAHS+ architecture.
Empirical evaluations of frontier language models over long clinical documents consistently demonstrate position-bias degradation (“Lost in the Middle”; Liu et al., TACL 2024 ↗) and severe token inflation when serialized FHIR R4 JSON is supplied directly. In our synthetic cohort, an 84-year longitudinal record (Maurice742_Auer97) spans 18.36 MB and ~3.52M raw tokens—17× to 27× larger than 128k–200k context windows—where over 95% of tokens consist of repeated Base64 encounter notes, administrative claims (Claim / ExplanationOfBenefit), and serial laboratory flowsheets.
To prevent ad-hoc truncation from dropping active allergies or cross-clinic medications, our 5-Tier Condenser implements the priority hierarchy of the HL7 International Patient Summary (IPS) and ISO 27269:
DocumentReference and DiagnosticReport.presentedForm[] while retaining unique clinical narratives.HL7 FHIR IPS Implementation Guide ↗ · ISO 27269 IPS Standard ↗ · HL7 US Core (FHIR R4) ↗
When patients aggregate records across unaffiliated clinics via SMART Health Links ↗ or TEFCA patient-access exchange, semantic heterogeneity causes unassisted language models to miss critical safety conflicts: one clinic prescribes brand Plavix 75 mg while another lists generic Clopidogrel 75 mg; primary care retains CKD Stage 1, 2, and 3 simultaneously active while serial eGFR labs demonstrate progression; or retail pharmacy and state registry (WA IIS) exports report overlapping immunizations.
Our 4-Layer Clinical Knowledge Base executes deterministic ontology and guideline checks prior to narrative generation:
2339-0 vs. 2345-7 for glucose).patient_self vs. external_share before condensationModern EHR problem lists and observation flowsheets routinely store sensitive Social Determinants of Health (SDOH), substance-use screening (NIDA DAST-10 ↗), intimate partner violence (SNOMED 706893006), and justice-involvement findings alongside medical diagnoses. If an LLM summarizes the raw bundle first and attempts to filter sensitive topics only in the output prompt, sensitive context can leak into clinical reasoning or citations.
Layer 4 of our architecture implements pre-condensation data segmentation aligned with federal and Washington state privacy frameworks:
patient_self mode, sensitive items remain visible with explicit privacy badges so the patient knows what is in their chart; in external_share mode, sensitive resources are stripped deterministically before Tier 1–4 condensation, guaranteeing 0.0% token or UUID leakage (ONC Privacy Guidance ↗).Clinical decision-support research shows that fluent AI explanations can induce over-reliance unless users can verify source records directly and distinguish automated checks from human clinical sign-off. Our UX and data model synthesize three complementary standards:
AI-Provenance): Distinguishes (1) supporting chart records ([N] → FHIR resourceUuid), (2) the AI system and condensation tier that transformed them, and (3) recorded human review or patient annotation (HL7 AI-Provenance Profile ↗).To guide Phase 3 of the WA Health Summary collaboration, we conducted a critical audit of our Phase 2 demo data, rule engine, and evaluation suite, and synthesized concrete engineering plans across evaluation methodology, messy synthetic/anonymized data generation, GPU server execution, model selection, multi-agent KB orchestration, hybrid full-chart retrieval, cryptographic FHIR privacy, and patient-centered scenario design.
wahs_analyzer.py) tests deterministic Python normalization and structured templates over condensed FHIR ledgers rather than live open-weight LLM generation (Gemma 4, Qwen 3, MedGemma). Thus, the 99.4 / 100 headline score validates our pre-LLM condensation and rule engine, not generative model fidelity. Meanwhile, the 61.2 / 100 Opus 5.5 row is a reconstructed reference baseline from Phase 1 observations over manually sliced JSON rather than a live side-by-side harness run over all 32 prompts.100% by construction.N = 3): Three clean Synthea patients have 100% coded entries (2,629 / 2,629). Real multi-EHR patient exports and Epic EHI dumps contain uncoded free-text problems, local charge codes, OCR/scanned PDF notes, and garbled sig strings where deterministic lookup alone drops sharply.32 / 32) and standalone HTTP harness runner (summary.json, 81 / 96 cells) exposed a field-name mismatch (abstained vs. abstains) and a substring false positive where the redaction banner triggered the forbidden-term check. We also remediated opaque metrics (e.g., “4-Layer KB Code Coverage” → Medical Code Recognition Rate) across the published evaluation report.[N] → FHIR resourceUuid actually entails the claim—catching citation distortion where a model cites a real lab or note while misstating its value or date.Claude Opus 5.5 + GPT-6-Astra with stripped check labels).To stress-test condensation, reconciliation, and LLM reasoning under realistic clinical messiness, Phase 3 expands our test corpus from 3 clean Synthea files to 8 patient personas (25–50 longitudinal bundles) using a 3-prong data pipeline:
code.text (e.g., “T2DM w/o comp”, “lasix 40 po qd”).MedicationRequest.status remains active.DocumentReference attachments.MedicationStatement with missing requester/dates) and raw EHI export tables.We inspected the UW CIRG GPU server configuration (uwcirg/llm-configs#2 ↗ & #3 ↗) running Traefik + LiteLLM + Ollama/vLLM on NVIDIA H200 hardware (141 GB HBM3e full GPU or MIG 1g.18gb / 2g.35gb slices), alongside our local 64 GB unified-memory Mac llama-router setup.
uwcirg/llm-configs: (1) Raise OLLAMA_MAX_LOADED_MODELS from 2 to 3–4 (or pin one model per stage) so concurrent calls to embed/nomic-embed-text + writer + reviewer do not thrash VRAM; (2) explicitly pass num_ctx: 16384 or 32768 per API call so Ollama does not silently truncate condensed charts at its 2k/4k default; (3) enable OLLAMA_FLASH_ATTENTION=1 and OLLAMA_KV_CACHE_TYPE=q8_0 (or deploy vLLM with --enable-prefix-caching so repeated Q&A and rubric checks over the same patient ledger reuse the KV cache at >80% prefill speedup); and (4) enforce turn_off_message_logging: true in LiteLLM for zero PHI persistence.| Tier & Target Hardware | Candidate Open-Weight Models | VRAM Footprint (Quantization) | Primary Role in WAHS+ Architecture |
|---|---|---|---|
Tier S · Edge / Browser / Mobile0.3B – 4B · WebGPU / Laptop / CPU |
MedGemma 1.5 4B-IT · Gemma 4 E4B · Qwen 3 1.7B / 4B · Llama 3.2 3B · MiniCheck-Flan-T5-Large (770M) |
1.5 – 4.5 GB (Q4_K_M / Q8_0) |
Zero-server client-side summarization over Tier 3/4 condensed capsules (≤ 4k tokens); fast claim-level NLI entailment verification (MiniCheck); query intent routing. |
Tier M · Single MIG Slice / Workhorse7B – 14B · H200 MIG 1g.18gb (18 GB) |
Gemma 4 12B-IT (Q8_0) · Qwen 3.5 9B (Q8_0) · IBM Granite 4.2 8B · Ministral 3 14B · DeepSeek-R1-Distill-Qwen-14B |
6.5 – 14.5 GB (fits in 18 GB slice with 16k KV cache) |
Primary production workhorse for Tier 2/3 chart summarization, cross-clinic conflict explanation, and cross-family label-only review (single-12b-xcheck). |
Tier L · Dual-MIG / Half-GPU Specialist24B – 35B · MIG 2g.35gb / 3g.71gb |
Gemma 4 31B-IT (Q4_K_M) · MedGemma 27B-Text-IT · Qwen 3.8 27B · Qwen 3.6 35B-A3B (MoE, 3B active) · Nemotron 3 Nano 30B-A3B |
16 – 22 GB weights + KV cache |
High-accuracy multi-source reconciliation, complex ICU/longitudinal narrative synthesis, and fast MoE specialist verification (3B active parameters per token). |
Tier XL · Full H200 Server Ceiling / Judge70B – 120B+ · Full H200 (141 GB HBM3e) |
Llama 3.3 70B-Instruct · Qwen 2.5/3 72B-Instruct · OpenBioLLM-70B |
42 – 75 GB (Q4_K_M / FP8) |
Local open-weight reference ceiling, gold-standard synthetic perturbation authoring, and local LLM-as-a-Judge calibration. |
3-Stage Screening Funnel: Stage 1 (Deterministic Gate, N=10 cases): reject any model with <95% citation formatting compliance, >0 SDOH privacy leaks, or <90% abstention accuracy. Stage 2 (Throughput & Tier Sensitivity, N=32 cases × 3 budgets): measure TTFT, tokens/sec, and recall degradation from 8k → 2k tokens. Stage 3 (Full Multi-Persona Blind-Judged Benchmark, N=100+ cases): dual-judge HealthBench + VeriFact scoring.
[2023-11-20 · 2y 11m ago]) so 4B–14B models never perform raw calendar arithmetic.[CLINICAL RULE ALERTS] block above the condensed ledger so the LLM explains verified findings rather than memorizing ontology tables.[N] Bracket Citation Grammar: Require every clinical assertion to carry an inline [N] index from the condensed ledger or explicitly state “Not documented in the provided health summary.”dry_multiplier = 0, temperature = 0.0–0.2) that corrupt medication lists.Drawing on our med-agent-hub findings (specs/checked-answers.md), letting a second agent blindly rewrite a draft often deletes valid clinical details (verifier over-rejection). Instead, Phase 3 tests four controlled architectures on KB-heavy queries:
single-12b-xcheck): Writer (Gemma 4 12B) drafts the answer; a cross-family Reviewer (Qwen 3.5 9B or Granite 4.2 8B) and narrow Entailment Grounder check each claim against the KB and cited FHIR records, attaching verification badges or triggering a single targeted repair pass.team-wahs-kb): A lightweight Router dispatches parallel domain specialists—Medication & RxNorm Specialist, Lab & KDIGO/ADA Trend Specialist, and ACIP Immunization & Care-Gap Specialist—followed by a Synthesizer and Privacy/Provenance Gatekeeper.Condensing an 18.4 MB chart down to 2k–8k tokens guarantees that the IPS Safety Core (active allergies, active meds, active problems, recent labs) is always in context, but aggressive condensation (Tier 3/4) intentionally drops older encounters, historical stopped medications, and full free-text clinical notes. To evaluate information loss across tiers and keep 100% of the full chart queryable even for small-context models, Phase 3 introduces a Two-Tier Hybrid Memory Architecture:
1k–4k tokens)Every prompt includes the Tier 3/4 condensed IPS summary so the model never misses an active allergy, active drug conflict, or chronic condition.
When a question asks about historical details omitted from the condensed view (e.g., “Why was Furosemide stopped in 2022?” or “What did my 2019 cardiology note say?”), the model calls read-only FHIR retrieval tools: search_chart_entries(query, resource_types, date_range), get_lab_history(loinc_code), and read_clinical_note(document_uuid).
Indexes Tier 0/1 overflow entries in an ephemeral in-memory vector index using nomic-embed-text (on the CIRG LiteLLM endpoint) or browser-side Transformers.js, returning top-k historical FHIR entries with exact [N] → resourceUuid provenance.
We will evaluate a 5 (Condensation Configurations: Tier 1 16k, Tier 2 8k, Tier 3 4k, Tier 4 2k static, and Tier 4 2k + On-Demand Retrieval) × 4 (Model Classes: 4B, 9B–12B, 27B–31B, 35B-A3B MoE) factorial matrix across four question horizons:
4B–8B models), whereas Hybrid Full-Chart Queryability requires reliable tool-calling (favoring Qwen 3 8B/27B, Gemma 4 12B/31B, or Granite 4.2 8B).How does WAHS+’s privacy segmentation connect to prior cryptographic FHIR privacy work across UW DIGI and CIRG? Our review links three complementary mechanisms—including Ian Bacher’s proof-of-concept in DIGI-UW/ips-selective-disclosures (sd-fhir) ↗ and uwcirg/shl-ips ↗—into a unified Three-Layer FHIR Privacy & Selective-Disclosure Architecture:
sd-fhir (SD-JWT) and SHL (JWE)DIGI-UW/ips-selective-disclosures): In Ian Bacher’s sd-fhir package, any sensitive FHIR resource in Bundle.entry[] or primitive field (e.g., Patient.address, Patient.telecom, or a sensitive Condition) marked with extension http://smart.who.int/icvp/StructureDefinition/SelectiveDisclosure is extracted by buildDisclosureFrame(). In the shared FHIR JSON, the sensitive element is replaced in-place by a salted SHA-256/SHA-512 cryptographic digest stored inside an http://smart.who.int/icvp/Extension/sd-hash extension (valueBase64Binary), while the underlying [salt, key, value] disclosures travel separately in a companion fhirR4.Parameters resource. This keeps the redacted FHIR Bundle 100% schema-valid and allows an issuer’s signature to be verified without revealing the obfuscated entries!uwcirg/shl-ips): Uses JSON Web Encryption (JWE, RFC 7516, A256GCM) where the 256-bit decryption key is carried in the client-side URL fragment (shlink:/...) and never sent to the hosting server. High-sensitivity resources can also be wrapped in a separately encrypted JWE sub-bundle or _extension unlocked only with a secondary recipient key or patient passcode.meta.security labels for 42 CFR Part 2, SDOH, and WA MHMDA).Automatically tag sensitive FHIR resources and fields (SDOH, IPV, substance use, behavioral/reproductive health) with meta.security and SelectiveDisclosure extensions.
In external_share mode, strip withheld resources before Tier 1–4 condensation and LLM prompting so sensitive facts never enter model context or embeddings.
sd-fhir)Replace withheld Bundle.entry[] resources or fields with salted sd-hash extensions so the exported IPS Bundle remains FHIR-valid and cryptographically verifiable without exposing the hidden text.
Package the selective-disclosure IPS Bundle into a JWE-encrypted SMART Health Link (uwcirg/shl-ips) with recipient-scoped disclosure parameters.
Grounded in OpenNotes / patient-portal research and the WAHS team’s design discussions, our scenarios and interactive examples prioritize what patients and caregivers actually need to understand, verify, and ask when navigating care across clinics—treating clinician/informatics framing as a secondary toggle:
| Patient Need Category | Primary Patient-Centered Question (Default UX Framing) | What the Patient Needs in the Answer | Secondary Provider / Clinical View (Toggle) |
|---|---|---|---|
| 1 · Medication Safety & Pill-Bottle Clarity | “Two different clinics gave me pill lists—am I accidentally taking two blood thinners or two cholesterol pills that do the same thing?” | Plain-language brand↔generic matching (Plavix = Clopidogrel), clear warning not to stop a heart medication without calling the prescriber, and the exact clinic/date of each prescription. | RxNorm ingredient/class conflict alert, P2Y12 / HMG-CoA duplication rule ID, and source MedicationRequest UUIDs. |
| 2 · Understanding Lab Trends & New Diagnoses | “My kidney lab numbers (eGFR) have been changing over the last two years—what do they mean, and why does my chart list three different stages of kidney disease?” | Simple explanation of what eGFR measures, visual trend over time, why old problem-list labels stick around, and what to ask at the next visit. | KDIGO 2024 G-stage exclusivity check, serial LOINC trajectory table, and uncoded G4 progression flag. |
| 3 · Making Sense of a Hospital / ICU Stay | “What actually happened during my November 2023 heart surgery and ICU stay, and what changed in my care afterward?” | Clear timeline of why they were admitted, what surgery was performed, what devices (stent/valve) were placed, and what medications changed at discharge. | Episode-grouped inpatient encounter summary with SNOMED procedure codes, ICU diagnoses, and discharge reconciliation. |
| 4 · Preparing for an Upcoming Appointment | “I’m seeing a new specialist next week—what are the top 3 questions or chart errors I should bring up with them?” | Actionable, printable checklist of unresolved medication discrepancies, missing emergency prescriptions (e.g., EpiPen), or overdue follow-ups. | Pre-visit clinical gap & reconciliation brief with linked FHIR provenance. |
| 5 · School, Camp & Immunization Readiness | “Is my child up to date on vaccines for school, and why does the state registry show a problem with one of his shots?” | Clear vaccine status table, plain explanation of why a dose given too early may need to be repeated, and which clinic recorded each shot. | CDC ACIP CVX minimum-interval validation and WA IIS deduplication log. |
| 6 · Sharing Safely & Protecting Personal Privacy | “If I share this summary link with a physical therapist or school nurse, what personal or social history is hidden, and what stays visible?” | Side-by-side preview of what is kept vs. removed (with sd-hash selective disclosure), reassuring the patient that active allergies and meds remain intact. |
42 CFR Part 2 / WA MHMDA / DS4P segmentation manifest and cryptographic sd-hash verification. |