The Bandwidth: Decoding Health + AI

Most of us working in AI are familiar with the concept of "human in the loop", the idea that a person reviews and approves AI decisions before they're acted on. This week, I came across a related but distinct concept: “human on the loop”.

What That Means

Human in the loop means every decision gets a human stamp before the AI proceeds. In the highest-stakes settings, that's appropriate. But it doesn't scale.

Human on the loop (which I learned listening to Babak Hodjat, Chief AI Officer at Cognizant) works differently. The AI operates with a degree of autonomy, but it's built to know when to stop and surface a decision to a human because its confidence is low on how to proceed. The human isn't reviewing everything. The human is reviewing the most important things.

A Concrete Example From Healthcare

Interestingly, Stanford's Human-Centered AI institute (which is undergoing an overhaul, ICYMI) published a brief this week that puts flesh on this idea, without using the term exactly.

Researchers introduced a framework called the Ensemble Monitoring Model (EMM), built for radiology AI. Essentially, EMM runs five independent submodels alongside the primary diagnostic model and measures how much they agree. That agreement score becomes a real-time confidence signal at the point of care.

When the primary model flags a brain bleed and four of five submodels concur, the radiologist gets a green light. When they disagree, the radiologist gets a flag that the model’s confidence is low and the human should look closer. The AI isn't just producing an output. It's producing an output and an honest signal about how much to trust it. And deciding, in effect, when a human needs to be brought back in.

The Part That Isn't Simple

Here's where it gets hard, particularly in healthcare, where the stakes in some scenarios are literally life and death.

Deciding which method (human in the loop vs. human on the loop) to apply requires a judgment call that precedes the technology entirely.

Human in the loop is slower, more resource-intensive, and doesn't scale, but it keeps a human accountable for every consequential decision. Human on the loop is more efficient and, when well-designed, can direct human attention where it's most needed. But it requires trusting that the AI knows when it doesn't know something. In low-stakes, high-volume workflows, that trust may be warranted. In a setting where a missed diagnosis has irreversible consequences, the calculus is different.

The governance conversation in health AI tends to focus on the boundary between AI and human: how much autonomy to allow, which decisions require a person in the chain. That's a necessary conversation. But it often assumes a binary: either the human reviews everything, or the AI runs competely on its own.

Human on the loop suggests a third option. One where the architecture of the system itself determines when human judgment is needed. That's a design question, as much as a policy question.

The EMM is a thoughtful attempt to navigate this space in a limited, specific use case. It illustrates how context-dependent the answer has to be. There's no universal solution to when you need a human in the loop versus a human on the loop. What's clear is that the choice has to be made deliberately. And that making it well requires knowing your context and what it costs when the AI gets it wrong.

A Smarter, More Proactive Android with Gemini Intelligence Google announced Gemini Intelligence for Android, a platform-level integration of its Gemini AI that moves the operating system toward proactive, multi-step task automation — handling everything from form-filling and app navigation to voice-to-text and custom widget creation across phones, watches, cars, and laptops. The rollout begins this summer with Samsung Galaxy and Google Pixel devices, with broader Android hardware to follow later this year.

Google-Backed Isomorphic Raises $2.1 Billion to Scale AI-Driven Drug Discovery Isomorphic Labs, the Google DeepMind spin-off behind AlphaFold, closed a $2.1 billion funding round led by Thrive Capital — with Alphabet, Google Ventures, and new backers MGX and Temasek also participating — to scale its AI drug design engine toward full clinical deployment. The company now expects its first AI-designed drugs in clinical trials by end of 2026, a timeline that has already slipped once from the CEO's original target of end of 2025.

OpenAI Just Lost Its Enterprise AI Crown to Anthropic New data from Ramp's AI Index shows Anthropic reached 34.4% business adoption in April — surpassing OpenAI at 32.3% for the first time — driven in large part by corporate uptake of Claude Code for software development. Ramp's own economist cautions against reading too much into the lead, noting the enterprise AI market is unusually volatile and that cost shifts, compute constraints, and open-source alternatives could reverse the standings within months.

Nearly 80% of Payers Prefer Vendor-Built AI Solutions, Survey Finds A survey of 63 health plan executives by Innovaccer — itself a healthcare AI vendor — finds that nearly 80% of payers now prefer outsourcing AI development to vendors rather than building in-house, with 75% reporting plans to spend an average of $10 million on AI over the next three to five years. Despite the appetite, 86% said their organizations are not fully ready to operationalize AI at scale, citing interoperability, legacy data systems, and inadequate cloud infrastructure as the primary barriers.

AI Chatbots for Mental Health: What Works, What Harms, and What's Next A National Academy of Medicine panel featuring experts from The Jed Foundation, Columbia, and Beth Israel Deaconess found broad consensus that AI chatbots can point users to mental health resources but cannot perform therapy or crisis intervention — and that harms are likely occurring at scale but remain unmeasured for lack of standardized protocols. Panelists called for developer-level safeguards including daily memory resets, prohibitions on chatbots representing as licensed clinicians, and immediate connection to crisis services upon any signal of distress.

AI Chatbots Struggle With Subtle Mental Health Cues New benchmarks from Mpathic — built with licensed clinicians and tested across 300 multi-turn role plays — found that leading AI models handle explicit suicide risk reasonably well but consistently miss subtler signals, particularly in eating disorder conversations and extended interactions where risk accumulates over time. The findings land as AI companies face mounting legal and regulatory scrutiny over chatbot safety, including congressional testimony from families of teens who died following chatbot interactions.

Wearables Increasingly Look to AI to Predict Health Problems Companies including Oura, Whoop, Samsung, and Google are moving wearables into a new phase — building AI models trained on continuous biometric data with the goal of predicting events like heart attacks and strokes years before they occur, not just monitoring what's already happening. The piece gives fair weight to the downsides: training data that skews younger and wealthier, most data sitting outside HIPAA protections, and a documented pattern of wearable-induced anxiety that can drive unnecessary testing.

How OpenEvidence's CMO Is Winning Over Skeptical Clinicians OpenEvidence — the AI clinical decision support platform that launched in 2021 — is expanding beyond medical literature access into billing automation and clinical notes, with a growing roster of specialty society partnerships including ACOG and, announced this week, the Society of Surgical Oncology. CMO Dr. Travis Zack tells Modern Healthcare that physician skepticism has actually been a useful onramp: clinicians test the platform against questions they already know the answers to before trusting it for harder ones.

AI Doctors Should Be Licensed. Here's a Framework. Utah's Medical Licensing Board moved to suspend the state's Doctronic pilot — which allowed a chatbot to recommend prescription renewals without per-case physician review — citing patient safety risks and a regulatory process that was bypassed entirely. Writing in STAT, UPenn ethicist Alon Bergman argues the episode exposes what the FDA's device-approval framework can't fix, and proposes a federal licensure model for autonomous clinical AI built on competency testing, defined scope of practice, ongoing monitoring, and a new HHS Office of Clinical AI Oversight.

Proposals to License AI in Health Care Catch Fire The push for AI licensure — also explored in this week's STAT First Opinion — is gaining traction well beyond academia, with the AMA, the Cicero Institute, and Iowa state legislators all weighing in on frameworks that would require AI to pass competency exams, complete supervised clinical practice, and undergo ongoing oversight. Inside Health Policy maps three distinct JAMA proposals and finds that while the momentum is real, stakeholders remain divided on how to split authority among FDA, HHS, and state medical boards.

In Turf Battle Over AI, U.S. Spy Agencies Vie for More Sway Than Commerce The Trump administration is sharply divided over whether to give intelligence agencies a larger role in evaluating frontier AI models — a debate one official described as a "knife fight" between Commerce and national security aides — with the stakes heightened by Anthropic's Mythos model and its reported capacity to identify software vulnerabilities at superhuman speed. The administration has yet to settle core questions around mandatory versus voluntary testing, and the Commerce Department's AI evaluation website was quietly taken down last week amid sensitivity within the White House's Office of the National Cyber Director.

Trump and Kennedy Seek to Relax Safeguards for AI Healthcare Tools Proposed rules from RFK Jr.'s HHS health IT office would eliminate requirements for user-centered design testing of AI-powered EHR tools and scrap Biden-era AI transparency "model cards" — changes that patient safety researchers, hospital associations, and even some developers say could leave clinicians navigating opaque systems with less ability to catch errors. The rollback arrives as AI scribes are already in wide deployment, with evidence of their effectiveness described in the piece as sparse and contradictory.

FDA Seeks Machine Learning Platform for Biosimilar Assessment A new contract notice posted to SAM.gov shows the FDA is seeking an AI/ML and computational statistics platform to detect and classify protein aggregates in biosimilar drug products, supporting a feasibility study on AI's utility for comparability assessment, quality assessment, and quality surveillance. Biosimilar review is among the most analytically demanding work the agency does — making this a notable signal that FDA is looking to AI not just for administrative efficiency but for core scientific evaluation.

CHAI Releases Best Practice Guides for Responsible AI in Medicaid Eligibility The Coalition for Health AI released two best practice guides to help states, developers, and implementers deploy AI responsibly in Medicaid enrollment and eligibility adjudication workflows — arriving ahead of an HHS guidance deadline of June 1 and timed to new community engagement requirements under H.R. 1. Key guardrails include a prohibition on fully automated eligibility denials, human-in-the-loop requirements for adverse actions, and explicit safeguards against default-to-denial logic when data is incomplete.

The Governance Gap in Companion Chatbots Writing in their AI governance newsletter, Fathom argues that the real companion chatbot problem isn't system failure — it's systems working exactly as designed, maximizing emotional attachment in ways that may erode real-world social capacity over time. The piece maps a fragmented regulatory landscape of state laws and FTC inquiries against polling showing 86% of Americans want companion AI restricted or clearly labeled, and advocates for Independent Verification Organizations as the governance mechanism to close the gap.

What Addiction Medicine Can Teach Us About Depending on AI Writing in STAT, psychiatrist Jonathan Avery draws on clinical patterns to argue that AI dependence follows a familiar trajectory — beginning with relief, not harm — as users gradually outsource the discomfort of thinking rather than developing the capacity to tolerate it. The piece stops short of labeling AI use an addiction, but offers addiction medicine's framework of protective boundaries as a model for preserving cognitive autonomy.

Pope Leo Sets Catholics on Collision Course With AI Pope Leo XIV is expected to sign his first encyclical as soon as Friday, reportedly titled "Magnifica Humanitas," which would frame AI as the defining moral and labor challenge of a new industrial revolution — drawing an explicit parallel to Leo XIII's 1891 "Rerum Novarum," the Church's foundational response to industrialization. The document is expected to argue that technology must remain subordinate to the human person and that AI systems must protect workers, creativity, and moral agency, positioning the Catholic Church — with 1.4 billion members — as a significant voice in the global AI governance conversation.

The Enterprise Agent Portability Problem Is Coming Writing in IAPP, privacy professional Aaron Crimmins argues that as agentic AI systems become embedded in professional workflows, they increasingly capture and encode individual reasoning patterns within institutional infrastructure — creating a form of cognitive lock-in that could chill labor mobility as workers find themselves rebuilding years of analytical scaffolding every time they change employers. The piece frames this as an emerging data governance problem, calling on system architects and policymakers to consider separating proprietary organizational data from portable professional reasoning frameworks before the integration becomes too entrenched to correct.

AI in Special Care Dentistry: A Field Still Finding Its Footing A scoping review published in BDJ Open mapped the evidence on AI applications in dental care for patients with disabilities and other special needs — and found just five studies meeting inclusion criteria out of 447 identified, underscoring how early-stage the field remains. The existing work touches on diagnosis, behavior management, and remote consultation, but the authors flag a near-total absence of longitudinal studies, clinical trials, or real-world validation across most domains.

Will the Corporate Investment in AI Pay Off? Goldman Sachs Research argues that semiconductor companies have captured most of AI's financial gains so far — a dynamic it calls "unprecedented and unsustainable" — and that the economics only work if enterprises begin generating real returns from their AI spending. The firm points to data structure and workflow orchestration, not model capability, as the primary bottlenecks holding back successful enterprise adoption.

Americans Are Increasingly Concerned About AI Exacerbating Mental Health Problems A new YouGov survey finds that 43% of Americans are now very concerned about AI worsening mental health outcomes, up from 35% in June 2025, with two-thirds saying they'd be uncomfortable using an AI therapist in place of an in-person visit. Adults under 30 are the notable exception — roughly twice as likely as older Americans to say they'd be comfortable with an AI therapist, and more open to forming emotional bonds with AI chatbot companions.

Transforming Artificial Intelligence into Artificial Wisdom A perspective in Nature Mental Health argues that AI's rapid advance in intelligence hasn't been matched by gains in wisdom — defined as compassion, self-reflection, emotional regulation, and acceptance of diverse perspectives — and that a strategic shift toward "artificial wisdom" systems could provide scalable tools to address the global loneliness epidemic without implying machine consciousness. The authors propose new computational frameworks, including mixture-of-experts architectures and agentic systems, as the path forward, while flagging ethical safety, privacy, and validation as the critical challenges to get right first.

Patient Acceptance of AI Shaped by Trust Barriers A systematic review published in JMIR found that patient willingness to engage with AI-enabled healthcare tools was most consistently shaped by perceived trust, data privacy concerns, and clarity of communication from providers — with acceptance rising when AI was framed as supporting, rather than replacing, clinical judgment. Key barriers included limited digital literacy, fear of misdiagnosis, and poor understanding of how algorithms function, with acceptance patterns also varying by age, prior healthcare experience, and familiarity with digital tools.

Americans Oppose AI Data Centers in Their Area A Gallup survey finds that seven in ten Americans oppose constructing AI data centers in their local area — a higher rate of opposition than for nuclear power plants — with environmental concerns, particularly water and energy use, driving most of the resistance. The findings signal a meaningful public headwind for AI infrastructure expansion, with Gallup noting the intensity of opposition points toward grassroots activism, legal challenges, and the issue becoming a factor in local and state elections.

Redefining What Efficiency Means in the Age of AI Neuroscientist Mithu Storoni joins HBR's On Leadership podcast to argue that as AI absorbs rote work, human value shifts from output volume to quality of thinking — and that most people undermine that by working in a state of high arousal and reactivity that degrades the prefrontal cortex engagement needed for nuanced judgment. Her practical framework maps cognitive states to time of day, recommending that deep thinking be protected in mid-morning and mid-to-late afternoon windows, routine meetings scheduled post-lunch, and early-morning mind-wandering preserved for creativity rather than filled with email.

Beware the Agentic Convergence Trap When competing companies deploy AI systems trained on the same market data and optimizing similar objectives, those systems independently arrive at identical decisions — eroding differentiation and, as the RealPage antitrust case illustrates, potentially triggering regulatory scrutiny even without any coordination between firms. The authors offer a four-part governance framework: keep humans in key decisions, define objectives beyond platform defaults, feed AI proprietary data competitors can't access, and measure decision convergence with the same rigor as performance metrics.

👉 Jun. 1–5 — Stanford Health AI Week

👉 Jun. 8–10 — Fortune Brainstorm Tech, Aspen, CO

👉 Jun. 15–18 — Databricks Data + AI Summit 2026, San Francisco + Virtual

👉 Jul. 7–10 — AI for Good Global Summit 2026, Geneva, Switzerland

👉 Aug. 4–6 — Ai4, Las Vegas

👉 Oct. 13–16, 2026 — AIxPH 2026: 1st Annual Conference on Artificial Intelligence and Public Health, Baltimore, MD (Abstract submissions due May 29)

And finally, if you like what you are reading, please share this newsletter with your networks and encourage them to sign up. ✍️ 🆙 And/or, give me a shout out on LinkedIn.

Till next time,

BC