Billy·Oglesby

Journal of Health AI

The problem

A new international, peer-reviewed, gold open-access journal on the deployment, evaluation and governance of AI across the health landscape needs editorial operations that scale without losing rigor, and it has to hold itself to the standard it publishes about. A journal about health AI that was casual about its own AI use would deserve the reviews it got.

The design

This page is written for a board director, not an engineer, because the journal is a governance exhibit as much as an editorial one.

The pipeline is a fixed sequence of manuscript states, and the automation is the platform (tracking, notifications, deadlines), not the judgment:

Journal of Health AI editorial pipeline Manuscripts flow from submission through editorial check, peer review, and decision to production and publication. Every editorial judgment (triage, reviewer selection, the final decision) is made by humans, shown in the accent color. The only planned AI step is an optional author-side preflight, currently off. Author preflight: optional AI check (planned, off) findings go to the author first · zero-data-retention · no editorial role Submitted Editorial check: an editor triages intake pauses at 10 untriaged submissions Under review: human reviewers single-blind · at least 3 completed reviews · reviewers may not use AI Decision: Editor in Chief only accept · revise · reject revision requested Accepted In production Published gold open access human decision point (every editorial judgment is one) planned, currently off

Three design decisions carry the governance weight:

  • Every editorial judgment is human. Triage is an editor's decision; review is single-blind with at least three completed human reviews per manuscript; the accept, revise, or reject decision belongs to the Editor in Chief alone. Intake pauses when ten submissions sit untriaged, so speed never outruns attention.
  • The AI-use policy is published, not implied. AI tools cannot be authors. Authors must disclose AI use, down to model names and versions. Reviewers may not use AI on manuscripts at all, a confidentiality line rather than a style preference.
  • AI enters, when it enters, on the author's side. Four author-visible preflight agents (declaration and metadata reconciliation, checklist-gap cues, reference-format normalization, and clarity suggestions) are in pre-deployment testing. Findings go to the author first, routed through zero-data-retention providers, with no AI role in any editorial decision. The governing policy took effect August 11, 2026; nothing goes live until it meets that policy. The policy is public: journalofhealthai.org/policies.

The numbers

Figure Value
Reviews required per manuscript ≥ 3 completed, single-blind
Editorial decisions made by AI 0 (by policy)
Author-side preflight agents in pre-deployment testing 4
Production agents (formatting, DOI minting) 0 (human-run, by design)
Intake pause threshold 10 untriaged submissions
AI-use policy in force since August 11, 2026
Submissions and days to first decision to come with the first issue

What went wrong and what I changed

  • September 17, 2026

    A malware incident reached the journal's server through a third-party analytics dependency.

    Changed: Took the server down and rebuilt it from scratch rather than cleaning it in place, and tightened what third-party code the platform is allowed to load.

What I’d do next

Publish the first articles and, with them, the journal's first public throughput numbers.