Clinical Note Co-pilot — case study

Clinical notes are messy — abbreviations, dictation disfluencies, semi-structured forms, consult letters that ignore SOAP entirely. Coders and reviewers spend disproportionate time on extraction tasks that look mechanical but require judgment.

LLM structured outputs are tempting and unreliable. Free-text JSON parsing breaks; the model invents codes it shouldn’t; confidence is opaque. The fix is not “trust the model” but a tight loop: forced-tool-use schema, source-span citations, visible confidence, and a UI that treats reviewer edits as the point — not an afterthought.

One Next.js app, one MCP server, shared lib/: Zod schemas, prompts, and the Anthropic client. The schemas compile to JSON Schema for tool-use forced output; the same Zod types validate the result before it reaches React. The MCP server imports the same module so Claude Desktop / Cursor / Code get an identical contract.

For the deployed demo, sample extractions are pre-computed and shipped as static fixtures in data/precomputed/. The headline path costs zero API tokens. “Paste your own” is gated behind a user-supplied API key (sessionStorage only, never sent to the server) with rate-limit + cache + cost-badge wrappers.

The extraction prompt forces a single tool call against the SOAPNote schema. The system prompt is short, with two non-negotiable rules at the top:

1. GROUND EVERYTHING. Every section's content must be derivable from
   literal text in the source. Inventing findings is the worst possible
   failure mode.

2. PREFER LOW CONFIDENCE OVER INVENTION. If the source is ambiguous, set
   the section's confidence to "low" and explain in reasoning. "Low
   confidence" is a feature, not a defect.

One concrete iteration. The first pass had Claude returning “medium” confidence on phone-consult notes that contained no objective exam, simply because the section was non-empty (it would say “no exam performed”). That’s wrong — the absence of an exam is a real limitation, not a medium-confidence finding. The fix was to add an explicit calibration rubric (high / medium / low examples) to the system prompt and to require the reasoning field to address “source quality.” After the change, phone consults correctly mark Objective as low-confidence with reasoning that names the absence as the issue.

The redaction layer is hybrid. Regex catches structured patterns (phone, email, MRN, DOB) cheaply and deterministically. The LLM (Haiku, small schema) sees the regex hits as candidates, then verifies, drops false positives, and adds names + addresses that regex would never get right.

Why neither alone works. Regex on names is hopeless — too much variance and proper-noun overlap with non-PHI (“the patient lives in New York” vs. “New York-Presbyterian”). LLM-only would re-discover phone formats from scratch on every call and cost more. Hybrid runs Haiku once, gets the LLM’s judgment on the hard cases, and accepts that synthetic PHI placeholders (“Jane D.”, “DOB 01/01/19XX”) are still PHI for the redaction layer’s purposes.

10-note golden set (the 5 fixtures + 5 variants — pediatric ED, atrial fib, empty non-clinical input, psych follow-up, diabetes intake). The runner runs the entire extraction pipeline live and scores against hand-written expectations.

• Section overlap: fraction of expected keywords present in the extracted section (case-insensitive substring match).
• ICD precision/recall: matched against an “acceptable” code list with fuzzy root-3 matching to forgive unspecified-vs-specific variation.
• PHI type recall: what fraction of the expected PHI categories were detected.
• Calibration: for each (section, confidence) pair, did high-confidence sections actually score >= 0.5 overlap?

Numbers shown are from a representative run on the 10-note set. Results vary across model versions; the runner is at /api/eval behind a typed-confirmation gate.

• Better calibration evals. The current “correct” threshold is keyword overlap; a real evaluation would use rubric-graded LLM judging or human labels.
• Coder-in-the-loop UX research. The reviewer surface is informed by looking at coding tools, not by sitting with coders. Real users would change the field hierarchy.
• Multi-pass extraction. One pass is enough for clean notes; messy dictations would benefit from a normalization pass before SOAP extraction.
• Diff-aware editing. A “what did I change vs. the model” export view would help with audit and with iterating on the prompt.
• Evaluation against retired ICD codes, not just current. The current eval set rewards modern ICD-10; legacy notes would punish it.