04 · AI Engineering & Post-Training Laboratory

System-1 clinical verifiers and the 5-rung post-training ladder

Clinical deployments in public-sector hospitals and low-resource clinics rarely have multi-GPU datacenter clusters. Our experimental lab tests two tightly coupled questions: how to replace slow second-pass LLM verification calls with sub-50ms non-autoregressive encoders (Track A), and how far 1.7B–4B open-weight generators can advance up a 5-rung post-training ladder when supervised by deterministic clinical oracles (Track B).

Track A: System-1 Decision Layer Track B: 5-Rung Training Ladder Browse Published Benchmark Reports

Track A · Non-Autoregressive System-1 Decision Layer (DECISION-LAYER-2026-10)

Sub-50ms context admission and claim verification with Laya (~0.4B)

In a standard multi-stage clinical pipeline, invoking a second generative LLM pass to review draft answers against source records adds 1.5–4.0 seconds (~25% of turn wall-clock latency) and contends for KV-cache slots on a shared GPU. We evaluate compact, single-forward-pass decision encoders (Laya ~0.4B vs. the Jev-1.13 reference architecture) that emit calibrated Choice, Boolean, and continuous Score outputs in a single forward pass (~25–50 ms).

Phase 0a · Context Admission

Recall-First Record Filtering

Evaluates 5-fold Leave-One-Patient-Out (LOPO) cross-validation across 7,840 clinical record–topic pairs (1,120 longitudinal FHIR/OpenMRS records × 7 clinical query archetypes).

Safety Gate: Operating threshold (τ_admit) is anchored at ≥99.0% recall on gold must-include clinical records before measuring prompt token reduction.
Multi-Hop Risk Audit: Explicitly audits whether single-record admission drops bridge records needed for multi-hop reasoning (e.g., linking an active medication to a historic renal lab).

Phase 0b · Claim Grounding

Entailment & Review-Error Gating

Tests whether a single-pass encoder can classify each generated clinical claim against its cited source record as supported, partly, or unsupported and gate when the heavy generative reviewer is needed.

Latency Compression: Cuts verification latency from ~2,100 ms (autoregressive 12B review pass) to <50 ms on local Apple Silicon or CPU.
Hybrid Gate: Clean claims pass immediately; only flagged or ambiguous claims trigger a targeted rewrite or abstention warning.

Calibration · Temperature Scaling

Fixing Sharp Logit Overconfidence

Compact cross-encoders trained with margin or cross-entropy losses produce sharply peaked logits that are poorly calibrated as raw probabilities.

Post-Hoc Temperature Scaling: Fitting a single validation scalar (T = 3.297) on held-out clinical pairs reduces Expected Calibration Error (ECE) from 0.466 down to 0.081 and Brier score from 0.312 to 0.164.
Rank Preservation: Monotonic scaling preserves exact AUROC and PR-AUC while making probability thresholds interpretable across clinical tasks.

Verification / Admission Approach	Execution Mode	Typical Latency	Calibration (ECE)	Role in Med Agent Hub & Harness
Deterministic Rules (R1–R10 & UUID Oracles)	Programmatic regex, AST & timestamp checks	<5 ms	Exact (binary rule)	Layer 1 hard gate on citation existence, date windows, and SQL/CQL syntax
Laya (~0.4B) System-1 Encoder (Calibrated)	Single forward pass (`Choice / Boolean / Score`)	25–50 ms	0.081 (post-T scaling)	Layer 2 semantic claim-to-record entailment, abstention check, and review gating
2nd-Call Generative LLM Reviewer (12B)	Autoregressive token generation (200–400 tokens)	1,500–4,000 ms	Unscored text verdict	Conditional fallback when Layer 1 or Layer 2 flags an unsupported or temporally drifted claim

Track B · Generative Post-Training Ladder

Climbing the 5-rung post-training ladder for 1.7B–4B clinical models

Before modifying model weights, we establish strict inference-time baselines on identical patient cohorts. Each rung of the ladder is evaluated against the previous rung using the 4-Layer Clinical Verification Stack so we never mistake stylistic imitation for genuine clinical accuracy.

Rung 1 · Inference Scaling

Constrained Decoding & Best-of-N

Enforces valid citation brackets, SQL ASTs, and JSON schemas at token-sampling time (XGrammar / Outlines) and reranks N candidate completions using Layer 1 deterministic checks + Layer 2 Laya entailment scores.

Rung 2 · Supervised SFT

LoRA Without Regret

Systematic LoRA rank (r ∈ {8, 16, 32, 64}) and all-linear target-module sweeps on verified clinical chart Q&A, calibrated abstention, and FHIR/SQL synthesis, paired with general medical retention probes to prevent catastrophic forgetting.

Rung 3 · Distillation

On-Policy Teacher→Student

Distills verified reasoning and citation trajectories from 12B–31B teachers (Gemma 4 12B/26B/31B, MedGemma 27B) into compact 1.7B–4B students (Gemma 4 E2B/E4B, Qwen3 1.7B/4B) using student-generated on-policy rollouts to eliminate exposure bias.

Rung 4 · Verifiable RL

RLVR / GRPO with Clinical Oracles

Group Relative Policy Optimization driven by deterministic execution rewards: PostgreSQL row-set equivalence for Catalyst SQL and cqf-ruler CQL execution against WHO SMART Guidelines, guarded by reward-hacking stress probes.

Rung 5 · Edge Deployment

Quantization & Tokenizer Fertility

Evaluates BF16 → Q8_0 → Q4_K_M / AWQ clinical safety degradation alongside LMIC multilingual tokenizer fertility (English, French, Swahili, Spanish, Portuguese) and roofline memory-bandwidth (GB/s vs. TPOT) scaling.

Post-Training Rung	Primary Training / Inference Signal	Target Failure Mode Addressed	Reward-Hacking / Regression Guard
Rung 1: Constrained Decoding & Best-of-N	CFG/JSON grammar masks + Layer 1/2 verifier reranking	Malformed UUID citations, broken SQL syntax, single-sample stochastic slips	Wall-clock TTFT/TPOT budget cap at N ≤ 4 on single 24 GB GPU
Rung 2: LoRA SFT ("LoRA Without Regret")	Verified gold clinical trajectories & abstention examples	Poor adherence to temporal rules (`R1–R10`) and missed multi-hop chart synthesis	Held-out patient LOPO splits + out-of-distribution clinical vignette retention probes
Rung 3: On-Policy Distillation	Teacher reverse-KL / logit matching on student rollouts	Distribution shift when a 2B–4B student diverges from 27B teacher prefixes	Layer 1 UUID provenance check on every distilled trajectory
Rung 4: RLVR / GRPO	Composite deterministic oracle reward (SQL row-set match, CQL MeasureReport, UUID binding)	Plausible-looking SQL/CQL or clinical claims that fail execution or temporal bounds	Explicit penalties for citation stuffing, strategic over-abstention, and verbosity gaming
Rung 5: PTQ / QAT Quantization	Activation-aware calibration on clinical chart contexts	4-bit quantization collapse on rare drug contraindications and numeric lab trends	Zero-harm floor & `Safety-Adjusted Benchmark` parity against `Q8_0` baseline

From Lab to Production

How experimental checkpoints graduate into Med Agent Hub

Experiments run in isolated workstreams before any model weight, calibration scalar, or verifier policy is promoted into production client profiles.

Step 01 · Isolated Lab Sweep

Train & calibrate on dev splits

Candidate adapters, distilled checkpoints, and Laya temperature scalars are trained and swept in the laboratory against deterministic oracles and LOPO cross-validation splits.

Step 02 · Held-Out Harness Gate

Run locked 4-layer benchmark

Promising candidates are registered as lab-laya-* or lab-exp-* profiles in Med Agent Hub and evaluated on held-out patients in the Validation Harness.

Step 03 · Public Evidence Packet

Publish report & promote profile

Only candidates that improve or maintain the Safety-Adjusted Benchmark with zero severe harm and faster/equal latency are published to reports.openclinai.org and promoted to default profiles.