Health AI Central
The problem
Health-AI information is scattered: research literature, trial registries, regulatory databases, company announcements, and specialist reporting each tell part of a story, and nothing connects the story to the evidence. I wanted a publication that reads like specialist journalism and cites like a systematic review, and I wanted it to run every day without a newsroom.
The design
More than forty agents do the daily editorial, research, and promotion work; the monitoring registry tracks 66 scheduled jobs in all, most on a staggered overnight cron. The model split follows one rule: Sonnet for judgment-heavy steps, Haiku for high-volume steps. Classifying a thousand headlines is volume; deciding what deserves 1,500 words is judgment.
The map below shows one slice, the editorial pipeline, the most visible of the agent families:
A night runs like this. Ingestion agents pull from news lanes covering 1,000+ companies, 25 RSS feeds, and primary sources (PubMed, ClinicalTrials.gov, FDA databases, NIH RePORTER), most without any model at all. A story listener (Sonnet) scores what came in; the content manager sets the day’s editorial brief; the medical researcher picks 10–12 stories and fetches each source’s full text; the technical writer (Sonnet) drafts news briefs of 400–700 words and deep dives of 1,200–2,000; the editor (Sonnet) reviews and calls the visual artist, which writes an image brief in Sonnet, renders it with Gemini 3 Pro Image, and then scores its own output.
Two other families work alongside the editorial pipeline. Research-intelligence agents screen the incoming data, link records across databases (a trial registration to its sponsor, its publications, and its regulatory record), and generate the intelligence products users come for: company profiles, clinical-area briefs, and classifications of why trials stopped. Back-office agents run the business itself: customer service and inbox triage, traffic analytics, SEO, security monitoring, advertiser operations, and the site's own help assistant. The same rules apply to every family: one gateway, per-agent keys, and a monitor that watches every job.
Hand-offs are files and database states, not vibes. The pipeline is a state machine with an explicit list of allowed transitions; an illegal transition raises an error instead of publishing a mistake. Every model call goes through one gateway with a per-agent key, so each agent’s spend and behavior are individually visible.
Evaluation and guardrails:
- Every article must link a primary record: a DOI, PMID, trial number, FDA clearance number, or NIH project number.
- A source-fidelity reviewer (Sonnet) checks the draft against its sources; a duplicate-story guard checks it against everything already published.
- AI-generated images must score 80 or better in review; source images must pass a provenance check.
- At most two automated revisions, then a human hold.
- For research classification, the evaluation went further: a blind three-vendor judge panel decided disagreements by strict majority, and clinicians checked a subsample. Sealed result: F1 0.953.
And the guardrail that matters most: nothing publishes without me. The pipeline’s last automated act is an email. I approve, reject, or hold every article at Board Review, and my rejections are fed back into the next day’s story selection as negative examples. I reject about a quarter of what reaches me, which is the system working, not failing.
The numbers
| Figure | Value |
|---|---|
| Agents in the daily pipeline | ~40 (66 scheduled jobs in the monitor registry) |
| Articles published since launch | 500+ |
| Drafted per day | ~10–11 |
| Published per day | 5.3 → 6.9 (month over month, Oct 2026) |
| Quality-hold rate | 34.7% → 14.5% after the contract change |
| My rejection rate at Board Review | ~25% |
| Monthly model spend | ≈ $200 via the gateway |
Media
[TODO: 60–120 second screen recording of the pipeline triaging a day's health-AI news]
What went wrong and what I changed
The mailbox agents read free-text email, which made them a prompt-injection surface: anyone who could send mail could try to steer an agent. I paused them the day the risk was clear.
Changed: Replaced free-text mail handling with fixed label-only classifiers that can output at most five tokens (a label, never an action). Tested against 20 real emails including two deliberate injection attempts: 20/20 classified correctly.
More than a third of drafted articles (34.7%) were stalling in quality holds, written and then stuck waiting on a gate instead of a decision.
Changed: Tightened the quality contract (at most two automated revisions, then a hold that emails me with keep / archive / cancel) and worked the holds queue down. A month later the hold rate was 14.5%.
The weekly spend report quietly returned “N/A” after an API key change. The pipeline that watches the money wasn’t being watched itself.
Changed: The monitors joined the registry: the agent monitor now tracks the reporting jobs like any other agent and pushes failures to the status page.
What I’d do next
Expose one read-only agent from the pipeline on this page, so a visitor can ask about a recent health-AI development and see sourced answers.