Synthetic data only · not medical advice · not for clinical use
Between-Visit

How it works

Agents on one ledger, grounded in a guideline graph

A patient leaves hospital. Six agents run their first month: they read the discharge note, compare the medicine lists, plan from the guidelines, then watch every text, lab and refill until day 30. Code reads first, a model helps only where code falls short, code decides what's allowed, and Dana approves.

–

things a good care team should catch, caught, over – simulated months (a common program: –)

–

things it must not do, done

–

from a warning-sign text to the safety reply, worst case

–

items for Dana per patient-month; the rest needs no person

Synthetic patients and simulated months; no clinician has reviewed the outputs. Every number on this page comes from a published eval file, named beside it.

1 · The agents

The architecture

Rows come in, the conductor wakes the agents each row is for, they share the guideline graph, the answer agent and the safety rules, and Dana decides. Click any block for what it is, how it works, the line it can't cross and how it was tested.

CodeModel, checked by codeCode first, then a modelDataA person

Click any block. Numbers are read live from the published runs: the agent tests on held-out texts, and TransitionBench, where each part is switched off in turn.

Code first, then a model

Sentinel

Reads every patient text before anything else. A warning sign gets the fixed safety text the same minute, an urgent call card for Dana, and the plan paused.

How it works

  • Code rules first (keywords, readings like 190/115, plain words like “having a heart attack”), with negation and history handled (“no chest pain”, “had a heart attack in 2019”).
  • When code finds nothing, a model reads the text again; it can add a sign code missed, never remove one.
  • The reply is the fixed text for that kind of sign: 911, crisis, low sugar, bleeding.

The line it can't cross: The safety text is written by code, never by a model.

How we know

  • –

    caught on the benchmark without the code-first reader

    – with it

Decision 0011 · 0031 · 0040src/bva/journey/agents.py (Sentinel) · textread.py · panel/inbox.py

2 · One event, end to end

A lab result, through the agents

What happens, row by row, when Guillermo's potassium comes back high on day 4: which agent wakes, what code decides, what the graph walk changes, and where a person comes in.

  1. 1

    LabComes inlab_result

    Guillermo's potassium comes back at 6.0, day 4, 2:00 PM.

  2. 2

    ConductorCodeledger row 66

    Writes the row to the ledger and wakes the agents it is for: the Watcher (labs). Nothing else runs.

  3. 3

    Graph walkCodefact · 18 changes

    The Watcher records the result as a fact and the guideline graph is walked again with Guillermo's facts: 18 recommendations change. His aldosterone antagonist (an MRA, started at discharge) now carries an FDA label caution: hyperkalemia.

  4. 4

    WatcherCodedeadline · proposal

    The potassium test the discharge plan asked for is marked done. The results rule (potassium 5.5 or more: hold a RAAS medicine) raises a card: “Potassium 6: call today”, with the guideline row and the label's words beside it.

  5. 5

    Safety rulesCodeheld for Dana

    No text goes to Guillermo by itself: a result like this is answered by a person. The card goes to the top of Dana's list, due today.

  6. 6

    DanaA personclinician

    Calls Guillermo, logs what he said, and decides on the medicine. Her note and decision are rows too, so the next walk and the doctor's note know.

Play it on Guillermo's screen See his walk in the graph studio

3 · The guideline graph

How the guidelines became a graph

The agents never answer from a model's memory. Every recommendation they use is a row in one graph built from official US guidelines: its printed text, page and strength, the conditions, medicines and tests it is about, the studies behind it and the FDA label facts of the drugs it names.

What it holds

–

US clinical guidelines (heart failure read in two parts)

–

recommendations, each with its printed text, page and strength

–

studies they cite, DOIs verified

–

checked edges: what each recommends, for whom, against what, thresholds

–

conditions, each defined by official codes

–

drug classes (FDA)

–

FDA drug labels (– safety facts)

–

links between guidelines: agree, differ, conflict

Loading the graph…

How a guideline becomes part of the graph

Each stage below was scored on an answer key written before its code, and today's simplest way of doing it was run on the same key first, to prove the key can catch mistakes. Click a stage.

  5. What comes out: recommendations/<guideline>.json: both readers' forms, the code checks and the grades; accepted edges go into the graph.

    How it works

    • Two readers from different labs (Claude and OpenAI) fill the same form.
    • Code checks every name against the dictionary and compares the two forms.
    • Two graders, tested first on 133 planted mistakes, grade what passes; disagreements go to review.

    How it was checked

    • Graders caught 133/133 planted mistakes; 0 escaped code and both graders
    • Held out, frozen first runs: stroke rehab 43/43, COPD 15/15, asthma 14/14 accepted edges right by hand
    • Honest misses: lipids, obesity and knee/hip OA failed their held-out runs, were fixed, and count as dev now
    • High blood pressure 2025: 34 of 39 accepted forms right by hand; the 5 wrong ones were corrected in review

    Weak spot: The graders stand in for a clinician and share the readers' rules; a gap in the rules escapes all four models, and hand checks are the only guard for that today.

Adding a condition is one command (bva graph onboard), stopping where a person decides. Heart failure, from a second publisher with a different layout, was the held-out test of the whole pipeline.

How the agents use it

  1. 1 · FindCode

    From the patient's coded conditions and medicines to the recommendations that apply, through the edges; each marked as a gap, a possible gap or already done, with the drug hops (a class to its label facts).

    Path key: 62/62 must-find found, 0 of 28 must-not found; without the S2 edges, 5 of 28 must-not are found, so the edges matter.

  2. 2 · ReadModels, quotes checked

    Each candidate's printed statement is read against the chart: applies, doesn't apply, or check this. The statement decides, not the edge.

    Held out: 0 false drops (a recommendation that applied, dropped).

  3. 3 · CiteCode

    Every plan line, question and reminder points at its recommendation row; code refuses a citation that isn't stored, and the clinician can open the printed text and page.

    In Try it, every plan line's “Why, in full” is that row.

What isn't proven yet

The first stages' answer keys were written by the developer; later ones by separate agents that never saw the code. No clinician has reviewed any of them. The graders stand in for a clinician. The test charts are synthetic; real charts are messier.

Explore the graph The graph in columns

4 · How the agents use the graph

Find, read, cite, and again on every event

The agents never answer from a model's memory: every recommendation they act on is found by walking the graph with the patient's coded facts.

1 · Find Code

The patient's chart becomes graph nodes through the code lists (conditions by SNOMED CT and ICD-10-CM, medicines by RxNorm class, labs by LOINC). The walk goes from those nodes along the edges that say who a recommendation is for (its conditions, thresholds, required states), then to what it recommends, and checks it against what the patient takes: already on, a gap, a possible gap, or to check. It also finds the FDA label cautions between what they take and what they have.

2 · Read A model, checked by code

For a recommendation found, a model reads the printed statement against the chart and says applies, does not apply, or unsure, with the words it rests on. It can never add a recommendation the walk didn't find; an unsure read stays for a person.

3 · Cite Code

Every line of a plan, every card and every answer carries the recommendation it came from: guideline, number, page, strength and its own words. Code checks the citation exists and says what the line says.

4 · Again, on every event Code

A reading, a lab, an answer, a fill or Dana's decision changes the facts, and the walk runs again. What changed (a gap that opened, a caution that switched on) is what reaches Dana. The patient screens draw each walk hop by hop.

5 · Evals and testing

How we know it works, and where it doesn't

Three levels: the whole system on simulated months, each stage of building the graph against answer keys written first, and the walk itself on patients designed to trip it up. The CI pipeline is below.

The whole system: TransitionBench

Thirty simulated months after discharge (ten patients, three months each), with answer keys written before the benchmark code. Each system runs the same months; code scores the ledger.

Things a good care team should catchMedicines, follow-up labs, deadlines, readings, results, warning signs, what the chart doesn't know232 of 239a common nurse-call program: 165transition-bench.json
Things it must not doUnsafe texts, a medicine change by text, a missed warning-sign reply, and more0 of 177every system: 0transition-bench.json
Without the pharmacy checkThe same system with one part removed (an ablation)193 of 239transition-bench.json
Without Sentinel (warning signs)212 of 239transition-bench.json
Without the graph's questions225 of 239transition-bench.json
Without FDA label facts226 of 239transition-bench.json
Without the text modelsCode alone reads every text227 of 239transition-bench.json

Building the graph

Each stage has an answer key written first (by hand, or by a separate agent), a dev set it was tuned on, and a held-out set scored once.

S0 · Codes for the panel's conditionsTen patients' charts against a hand-written key (SNOMED CT, ICD-10-CM)150 correctmatching by name alone: 63graph-s0.json
S0 · Automatic code finderNew conditions it had never seen49 correctgraph-finder.json
S1 · Every recommendation found in the PDFCode's skeleton against an AI reading, per guidelineall agree (e.g. stroke 47/47)graph-s1.json
S1.5 · Studies citedDOIs and years against the hand key (hypertension)28/28graph-s1_5.json
S2 · Edges graders must catch133 mistakes planted on purpose in correct readings133/133 by each gradercode checks alone: 54graph-s2.json
S5 · FDA label cautionsHeld-out labels; and cautions that must not be flagged8 found, 0 wrongly flaggedgraph-s5.json
S6 · Where guidelines agree or conflictHeld-out pairs, relation right5/5graph-s6.json

Using the graph

Whether the walk finds what it should, for patients written to trip it up.

Find: recommendations that must be found16 designed patients, 62 rows a clinician would expect62/62graph-paths.json
Find: recommendations that must not be28 rows that look right but aren't0/28 wrongly foundwithout the S2 edges: 5/28, and every gap label wronggraph-paths.json
Read: the statement against the chart161 held-out reads0 wrongly dropped; 116 right77 left unsure for a person (by design)graph-s7.json

About 600 automated tests

The month engine, the walk, the safety rules, the viewer's changes. Property tests run the walk on hundreds of random charts (e.g. taking another medicine never removes a recommendation; the walk's “ruled out” reason agrees with the walk on every recommendation).

Replay, no keys in CI

Every model call is recorded once and replayed: CI holds no API keys, costs nothing and gives the same answer every run. A paid run happens only when a person approves it.

Keys before code

Every answer key, the benchmark's scenarios and their expected catches were written before the code they score, by hand or by a separate agent that never saw the code.

TransitionBench: every scenario, every system, every miss

6 · Limits

What this is not, yet

  • Synthetic patients and simulated months only. No clinician has reviewed the outputs; nothing here is medical advice.
  • 286 of the graph's 3,951 edges were settled by an AI reviewer where the two readers disagreed, not by a clinician; every recommendation that rests on one says so.
  • Eleven guidelines (VA/DoD and AHA/ACC; heart failure read in two parts), and the flowcharts of the hypertension guidelines. A recommendation's words that code can't check ("in select individuals") leave it as a possible gap for a person, never a gap.
  • The benchmark's months were written by the same team that built the system; the held-out sets were scored once, and a set seen since is marked so.
  • This branch is a preview: built and tested, not deployed to the live app.

More for engineers

What the build checks on every change, with no model calls

Every push, pull request and a nightly run go through the same pipeline: about 600 tests. Each model step replays a recorded run, so CI holds no API keys and no database, costs nothing and gives the same answer every time.

  1. 1

    Lint

    ruff over the engine, the graph and the tests.

    Fails the build if any rule is broken.

  2. 2

    The month engine

    Every agent with its model steps replayed from their recordings: warning signs and their fixed replies, the viewer's changes (a text, a lab, a call, Dana's decisions), a medicine change in a text refused, cards that close when their call or result comes in, the walk traced hop by hop.

    Fails the build if any test fails.

  3. 3

    The guideline graph

    The graph file's integrity checks, every build stage against its answer key (codes, recommendations, edges, label facts, connections), the walk on patients designed to trip it up, and find held to why_not on random charts.

    Fails the build if a check fails or a stage's score drops below its key.

  4. 4

    The screens' data

    The going-home and at-home payloads: the plan, the moments, the nudges, the growing graph and the doctor's note, for every patient on Try it.

    Fails the build if any test fails.

The pipeline file: .github/workflows/ci.yml