Health AI Central

The problem

Health-AI information is scattered: research literature, trial registries, regulatory databases, company announcements, and specialist reporting each tell part of a story, and nothing connects the story to the evidence. I wanted a publication that reads like specialist journalism and cites like a systematic review, and I wanted it to run every day without a newsroom.

The design

More than forty agents do the daily editorial, research, and promotion work; the monitoring registry tracks 66 scheduled jobs in all, most on a staggered overnight cron. The model split follows one rule: Sonnet for judgment-heavy steps, Haiku for high-volume steps. Classifying a thousand headlines is volume; deciding what deserves 1,500 words is judgment.

The map below shows one slice, the editorial pipeline, the most visible of the agent families:

Health AI Central agent map The nightly editorial pipeline: ingestion from news and primary sources, overnight intelligence agents, a content manager, a medical researcher, a technical writer, an editor and visual artist, automated quality gates, Billy's human Board Review decision, then publication and promotion. Sources news lanes across 1,000+ companies · 25 RSS feeds · PubMed · ClinicalTrials.gov · FDA · NIH (ingested without an LLM) Blog scraper ~670 blogs · no LLM News monitor Haiku · high volume Story listener Sonnet · judgment Audience intel social metrics Content manager writes the day’s editorial brief Medical researcher picks 10–12 stories · links every primary record Technical writer · Sonnet news briefs 400–700 words · deep dives 1,200–2,000 Editor · Sonnet reviews · requests the image Visual artist · Sonnet brief → render → self-review Quality gates source fidelity (Sonnet) · image ≥ 80 · release-integrity supervisor · max 2 revisions revise (max 2) Board Review: Billy decides human approval on every article · ~25% rejected rejections steer tomorrow’s selection Campaign executor publishes to healthaicentral.com Social media manager · Sonnet platform-specific promotion 66 scheduled jobs · every model call goes through one gateway with a per-agent key · monitored every 5 minutes Accent box = a human decision. Nothing publishes without one.

A night runs like this. Ingestion agents pull from news lanes covering 1,000+ companies, 25 RSS feeds, and primary sources (PubMed, ClinicalTrials.gov, FDA databases, NIH RePORTER), most without any model at all. A story listener (Sonnet) scores what came in; the content manager sets the day’s editorial brief; the medical researcher picks 10–12 stories and fetches each source’s full text; the technical writer (Sonnet) drafts news briefs of 400–700 words and deep dives of 1,200–2,000; the editor (Sonnet) reviews and calls the visual artist, which writes an image brief in Sonnet, renders it with Gemini 3 Pro Image, and then scores its own output.

Two other families work alongside the editorial pipeline. Research-intelligence agents screen the incoming data, link records across databases (a trial registration to its sponsor, its publications, and its regulatory record), and generate the intelligence products users come for: company profiles, clinical-area briefs, and classifications of why trials stopped. Back-office agents run the business itself: customer service and inbox triage, traffic analytics, SEO, security monitoring, advertiser operations, and the site's own help assistant. The same rules apply to every family: one gateway, per-agent keys, and a monitor that watches every job.

Hand-offs are files and database states, not vibes. The pipeline is a state machine with an explicit list of allowed transitions; an illegal transition raises an error instead of publishing a mistake. Every model call goes through one gateway with a per-agent key, so each agent’s spend and behavior are individually visible.

Evaluation and guardrails:

  • Every article must link a primary record: a DOI, PMID, trial number, FDA clearance number, or NIH project number.
  • A source-fidelity reviewer (Sonnet) checks the draft against its sources; a duplicate-story guard checks it against everything already published.
  • AI-generated images must score 80 or better in review; source images must pass a provenance check.
  • At most two automated revisions, then a human hold.
  • For research classification, the evaluation went further: a blind three-vendor judge panel decided disagreements by strict majority, and clinicians checked a subsample. Sealed result: F1 0.953.

And the guardrail that matters most: nothing publishes without me. The pipeline’s last automated act is an email. I approve, reject, or hold every article at Board Review, and my rejections are fed back into the next day’s story selection as negative examples. I reject about a quarter of what reaches me, which is the system working, not failing.

The numbers

Figure Value
Agents in the daily pipeline ~40 (66 scheduled jobs in the monitor registry)
Articles published since launch 500+
Drafted per day ~10–11
Published per day 5.3 → 6.9 (month over month, Oct 2026)
Quality-hold rate 34.7% → 14.5% after the contract change
My rejection rate at Board Review ~25%
Monthly model spend ≈ $200 via the gateway

Media

[TODO: 60–120 second screen recording of the pipeline triaging a day's health-AI news]

What went wrong and what I changed

  • October 9, 2026

    The mailbox agents read free-text email, which made them a prompt-injection surface: anyone who could send mail could try to steer an agent. I paused them the day the risk was clear.

    Changed: Replaced free-text mail handling with fixed label-only classifiers that can output at most five tokens (a label, never an action). Tested against 20 real emails including two deliberate injection attempts: 20/20 classified correctly.

  • September–October 2026

    More than a third of drafted articles (34.7%) were stalling in quality holds, written and then stuck waiting on a gate instead of a decision.

    Changed: Tightened the quality contract (at most two automated revisions, then a hold that emails me with keep / archive / cancel) and worked the holds queue down. A month later the hold rate was 14.5%.

  • October 2026

    The weekly spend report quietly returned “N/A” after an API key change. The pipeline that watches the money wasn’t being watched itself.

    Changed: The monitors joined the registry: the agent monitor now tracks the reporting jobs like any other agent and pushes failures to the status page.

What I’d do next

Expose one read-only agent from the pipeline on this page, so a visitor can ask about a recent health-AI development and see sourced answers.