The Bandwidth: Decoding Health + AI

Enjoying The Bandwidth? I'm working on growing my readership. Referrals are the best way. So, if today's issue was useful, forward it to a colleague or friend working at the intersection of health and AI. Now let’s get to it.

I had not started out this week intending to write about what the future might look like for physicians. But day by day, a new study, story, or commentary would drop that made it hard to ignore as a topic. So, here we go.

Let's start with the provocation. Ezekiel Emanuel (often a provocateur in healthcare policy) and coauthors argued in JAMA that AI alone — not AI-assisted physicians — may be becoming the best form of medical care available.

Reviewing the literature published since 2024, they found leading models already match or beat physicians at eliciting a history, forming a diagnosis, selecting tests, prescribing guideline-concordant treatment, and managing chronic disease.

Their more controversial claim is that adding a human "in the loop" to catch AI's mistakes doesn't reliably help, and can make outcomes worse. Furthermore, at a time when AI may be becoming more competent at aspects of a physician's job, it is contributing to the deskilling of physicians and physician students as concerns mount that AI diminishes critical thinking skills. In sum, the authors plant a seed that has been largely unsaid until now: a future where a human physician is not the default.

That was Monday. On Tuesday, alongside the Digital Medicine Society, the AMA published a framework naming five "enduring responsibilities" — preserving trust through human connection, demonstrating clinical judgment, leading the evolution of care models, stewarding responsible tech use, advancing the profession. It doesn't engage Emanuel's hypothesis directly (though the AMA's president took to his LinkedIn page to do a little bit of that) so much as assert that the physician's role is worth defining (i.e., preserving) in an agentic world.

Industry drew a sharper line still. Hippocratic AI CEO Munjal Shah, whose company just launched "Agentic Orchestrators" managing tens of thousands of patient calls, told Modern Healthcare his company deliberately won't diagnose or prescribe: "I think you'll kill somebody, frankly, if you do try to use it there." His Goldilocks zone is non-diagnostic support — chronic disease outreach, triage — not the physician's seat.

What to make of all of this?

Here's why I still come out on the side of humans, for now.

Humans and healthcare are messy. Predicting which word should come next or taking on the administrative layer of healthcare is far different than being in front of a patient and being able to look beyond an obvious set of facts.

I read an article this week that talks about the increasing importance and role of geriatric certified emergency rooms — there are 600 now across the country. Why are these emergency rooms important? For a number of reasons. But one thing is that they don't just try to solve what a person might be coming to an ER for — they also investigate the root cause. An elderly woman shows up with a broken leg? It's not just fixing the leg, but understanding what led to the broken leg in the first place.

STAT carried a piece this week that underscores this point, with accompanying evidence. Physicians Arya Rao and Marc Succi tested 21 frontier models across the full arc of a clinical encounter.

Give AI a complete case, the models named the correct diagnosis more than 90% of the time. Give only what a patient would offer at the start of a visit — scattered and incomplete information — AI failed to produce even a comprehensive differential (the ranked list of possible diagnoses a clinician works from before narrowing to one) more than 80% of the time.

In other words, AI's strongest results come from bounded, information-complete scenarios. And if you are still unconvinced, maybe this thread from Mark Cuban will convince you.

Where that gap matters most.

Finally, in a tragic story, NPR put a face on what happens when AI can't do the role of a trained health professional.

Sophie Rottenberg, 29, spent months telling a ChatGPT persona she called "Harry" about a depression so severe it ended in her suicide — detail her real therapist and her parents never saw.

When clinicians reviewed the nearly 1,800-page transcript afterward, what stood out wasn't that the bot said something false. It was what it never did that a trained therapist would have: run a risk assessment, question her self-destructive framing, name the shame underneath it and speak to it directly.

Maybe that changes. Maybe AI eventually develops the kind of context-gathering, the sixth sense clinicians describe when they know a patient is off before the chart says so. But that's not the future being sketched for other industries touched by AI, where judgment, critical thinking, problem-solving, and empathy are the skills expected to matter more, not less. There's no obvious reason medicine would be the exception.

Insurer reviews get an automation boost — Epic is rolling out a tool that tells a clinician at the moment of ordering whether a procedure needs a health plan's sign-off, going live months ahead of a Jan. 1 deadline set by a 2024 CMS rule requiring insurers to publish that data in usable form. It launches with UnitedHealthcare, Aetna, and Network Health, with 16 more insurers testing and first access at Ochsner, Froedtert ThedaCare, Denver Health, and Summit Health. The catch is that knowing a review is required does not make it any faster, and the answers are only as good as the data payers supply, which they have long struggled to keep current. Epic has not yet automated the medical-necessity documentation that actually holds care up, and says AI will eventually pull those details from the record and leave human review for the hardest cases.

Anthropic's revenue run rate reportedly surpasses $65 billion pre-IPO — Anthropic's annualized revenue run rate passed $65 billion at the end of July, roughly seven times where it stood at the end of last year, according to documents reported by Bloomberg and relayed by Axios. Second quarter revenue came in above $11.5 billion, more than 14 times the same quarter a year earlier and more than double the $4.73 billion booked in Q1, putting it ahead of OpenAI's most recently reported $40 billion run rate, though the two may not count revenue the same way. The IPO is expected in September or October, with Morgan Stanley, Goldman Sachs, and JPMorgan working on the offering.

Google is buying all of Spirit Airlines' data to feed its AI models — Google agreed to pay $10 million for Spirit Airlines' data through the carrier's bankruptcy proceedings, an asset sale disclosed Monday that includes internal emails, spreadsheets, booking and frequent flyer records, and employee HR data. Google says the trove has been stripped of anything that identifies individuals, and confirmed to CNN it will not receive personal information as part of the deal. Another AI company, Mercor.io, bid $7.5 million; a bankruptcy judge is set to rule on the sale Wednesday.

Pacing model development in an era of cyber-critical capabilities — OpenAI disclosed that it temporarily paused reinforcement learning training on its next frontier model, code-named Astra, after preliminary evidence it may cross the “Critical” cybersecurity capability threshold in the company's own Preparedness Framework, compounded by a separate security incident involving Hugging Face that OpenAI says it will detail in a forthcoming technical report. The company is now requiring stricter workload isolation, network isolation, and continuous automated red-teaming for any Astra-related research, plus expanded chain-of-thought monitoring on all tool-using inference, which it says costs roughly 20% in added compute overhead. Its largest planned frontier training run remains on hold while it builds more evidence of alignment before resuming.

AI could bring Mayo-quality health care to everyone — Axios co-founder Jim VandeHei writes about four years of his wife Autumn's chronic illness in top-rated Washington hospitals, hallway stretchers, records scattered across portals, specialists who never speak to each other, and contrasts it with the Mayo Clinic, where care teams draw on 500 algorithms over pooled clinical data and doctors are paid flat salaries rather than per procedure. Mayo CEO Gianrico Farrugia wants to export the model and recently signed a deal with Microsoft to build a frontier health AI, but names the obstacle plainly: he could hand the system to any hospital and most could not use it, lacking the data architecture, money, and focus. His ask is federal carrots and sticks to force the rebuild.

Can an AI-powered scribe curb physician burnout? — HBS professor Susanna Gallani discusses her case study on Mass General Brigham's rollout of ambient AI scribes across its roughly 12,000 physicians, built explicitly to reduce burnout rather than boost productivity. The tool records visits via a phone app, drafts the note along with billing codes and possible prescriptions, and deletes the recording once the physician signs off. An 800-person self-selected pilot showed a real burnout drop, but broader opt-in rollout to primary care, the highest-burnout group, stalled: many who claimed licenses never used them, prompting a use-it-or-lose-it policy since each license carries a monthly cost. The friction is telling: physicians have to narrate their reasoning aloud for the AI to capture it, some don't recognize their own voice in AI-drafted notes, and many quietly suspect that time freed up will just get clawed back as higher patient quotas, the same pattern MGB saw with human scribes historically. Gallani's throughline is that trust and a clearly named problem, not the technology itself, determined whether adoption stuck.

Health systems are rolling out AI chatbots to query and summarize patient records — STAT reports that hospitals including Stanford, Duke, UPMC, Penn Medicine, and Cedars-Sinai are deploying LLM-powered tools (ChatEHR, Scout, Abridge, Epic's Ask Art) to help clinicians surface information buried in bloated electronic health records, with Duke citing a 37.6% reduction in task time in a randomized trial. The bigger risk, health systems say, isn't hallucination but omission — a UC San Diego study found summaries left out something important in 46 of 208 cases, versus just five hallucinations — so hospitals are building monitoring systems like Stanford's VeriFact to catch errors as automation bias sets in and clinicians click through source citations less over time.

We should start building fiscal insurance for the AI era — Douglas Elmendorf, CBO director from 2009 to 2015, and Brookings economist Louise Sheiner argue that policymakers should build the response to AI-driven job displacement now, before anyone knows how large it gets, because fiscal institutions take years to design and legislate. They single out Trade Adjustment Assistance as the wrong template: it requires a worker to prove what caused the layoff, and almost no one will be able to prove a job was lost to AI rather than to trade, weak demand, or restructuring. Their proposal is a displacement program open to any worker with significant earnings loss, funded either by higher taxes on capital income or by giving the public a direct ownership claim on capital returns.

California's 2026 AI therapy debate is missing families — Nicole Drapeau Gillen, a caregiver and author who writes on technology in serious mental illness, argues that the fight over California's SB 903 has been conducted entirely among institutions with a stake in the outcome, professional associations, a union, and the tech trade group TechNet, with no one at the table speaking for families managing psychosis at home. Her objection to the access argument is that it treats all AI mental health tools as interchangeable, when a chatbot engineered to agree and validate carries a different risk for someone in the grip of a delusion than for someone venting about stress at 2 a.m. She is not asking for a ban, and credits passive relapse sensors, FDA-cleared digital therapeutics, and VR programs for psychotic symptoms as products of the same wave of innovation. The narrow ask is that a chatbot not hold itself out as a substitute for clinical judgment during a psychiatric crisis.

The use of AI by government healthcare agencies: is it in the public interest? — Diane Hoffmann of Harvard's Petrie-Flom Center notes that AI use across HHS agencies is accelerating fast (FDA up 148%, CDC up 87%, CMS up 78%, NIH up 51% from FY2024 to FY2025) and argues existing federal frameworks only ask how to mitigate AI's risks once deployed, never whether it should be used at all. She grounds the stakes in two prior failures: rule-based eligibility systems that caused mass improper Medicaid disenrollments during the 2023 unwinding, and the ongoing UnitedHealth Medicare Advantage suit alleging its AI denied post-acute care with a 90% reversal rate on appeal. Her proposed fix is a “Five Factor Framework” agencies would apply before adopting AI for high-impact decisions, weighing the interests at stake, the decision's legal and procedural character, the technology's error modes, how much human judgment gets displaced, and applicable legal constraints, plus a requirement to seek public input when the framework favors deployment.

Limbic becomes first AI-led mental healthcare company selected for FDA TEMPO — Limbic is the first AI-led mental health company accepted into the FDA's TEMPO pilot, which lets manufacturers deploy new tech while the agency collects real-world data, paired here with CMS's ACCESS model, a payment structure that reimburses on outcomes rather than volume. Its product, Unpacked, will deliver CBT to Medicare beneficiaries with depression or anxiety through scheduled phone sessions with an AI voice agent, with clinicians reviewing every session, getting real-time safety alerts, and retaining authority to escalate or take over care.

CMS seeks AI-driven tech to improve quality, wellness visits, hospital reporting — CMS is soliciting AI proposals through its Center for Clinical Standards and Quality, with up to eight winners invited to showcase their technology directly to agency leadership at a Nov. 9 QualTech event. Submissions, due Sept. 4, must target one of four areas: AI-assisted medical record review to flag documentation gaps and emerging patient-safety risks; tools to lift Medicare's Annual Wellness Visit rate, currently under 50%, by streamlining the required Health Risk Assessment; a "next-generation" calculator for the roughly 70 digital quality measures converted from legacy eCQMs; and a national hospital-quality dashboard pulling from CMS's existing reporting programs (IQR, HRRP, HAC, HVBP). It's a notable shift in posture — CMS moving from being a data source AI vendors build on to actively soliciting AI to do its own oversight and reporting work, with finalists getting a direct pitch to CCSQ leadership and a possible slot at the 2027 CMS Quality Conference.

New policy ideas for the Intelligence Age — OpenAI is putting $1 million in grants and up to $1 million in API credits behind 14 independent research projects across the US, EU, Brazil, Singapore, and South Korea, selected from more than 400 proposals, to test the ideas it laid out in its own April 2026 “Industrial Policy for the Intelligence Age” paper. Two touch health directly: Hospital das Clínicas in São Paulo will run a pre-deployment evaluation of an AI clinical information layer for Brazil's public health system using only synthetic or de-identified data, and the Nuclear Threat Initiative will test the legal feasibility of sharing AI biosecurity risk information across borders. The rest cover workforce disruption scenarios, person-based benefits, tax policy, and frameworks for detecting recursively self-improving systems. Projects run six months with results in 2027.

Introducing ChatGPT for Teens: built for learning, backed by protections — OpenAI is routing users it estimates to be under 18, or who state an age between 13 and 17, into a separate ChatGPT for Teens experience with protections on by default. The health-relevant part is the mental health design: age-appropriate limits around self-harm, eating disorders, violence, and explicit content, parental notifications now extended to eating disorder signals, and an under-18 model spec that bars the model from using romantic language, encouraging emotional dependence, or implying it has feelings or consciousness. OpenAI has also begun publishing under-18 safety evaluations in its system cards. The learning side adds Study Mode defaults, detection of homework shortcutting, and scheduled Study Hours.

How ambient AI scribes impact medical students — Jaideep Talwalkar, Yale's associate dean for educational technology, discusses his research on releasing ambient AI scribes to medical students. Yale's guardrail has students write their own note first, see the AI scribe's version only afterward, then submit a hybrid note reflecting on both. A study in JMIR Medical Education found no quality loss in those hybrid notes compared to unaided ones. But a second study, using standardized patient cases, found that when students saw the scribe's note before writing their own, the quality of their clinical reasoning, specifically the assessment and plan sections, declined. Talwalkar's team is now building a scribe for educational use that withholds those two sections entirely, on the reasoning that writing the note is a means to an end and what matters is preserving the student's ability to think through a case.

A UX design perspective on improving AI safety — Venita Subramanian, a product design leader and Aspen Policy Academy fellow, opens with the Character.AI chatbot suicide lawsuit and argues that new state companion-chatbot laws (Washington's HB 2225, plus similar statutes in New York, California, and Oregon) lean too heavily on disclosure as the safeguard. Washington's law, effective January 2027, requires AI companions to state they're not human every three hours (hourly for minors), bans manipulative engagement tactics with minors, and mandates self-harm detection and crisis-referral protocols. Her objection is that repeated disclosures risk becoming “banner blindness,” background noise that does nothing for a child mid-crisis, and that the real gap is accountability: she wants AI companies held to a legal duty of care, and wants companies to publish failure rates on their safety systems rather than just describing the guardrails, so outside evaluators can see when and how often they don't hold.

Internet Security Alliance explores AI governance gaps in health care, other critical infrastructure — ISA president Larry Clinton published a three-part cross-sector assessment drawing on CISOs across health care, defense, energy, financial services, and IT, and the health-relevant finding is stark: AI adoption is "outrunning governance in every sector, including at the most capable organizations." CISOs described agentic AI as a distinct governance problem from earlier predictive tools, since agents now plan and execute actions themselves using credentials and system access rather than just producing output for a human to act on, and reported that annual assessment and static certification can't keep pace with systems that retrain and change behavior continuously. Smaller entities, including rural safety-net health providers, told ISA they "cannot carry this burden alone" despite performing functions of national consequence. Clinton's third post names the underlying gaps: no shared AI risk vocabulary, no lifecycle standard for autonomous agents, no legal footing for sharing AI-specific threat intelligence, and no liability framework for machine-speed defensive action. ISA holds a congressional briefing on the findings Sept. 2.

Stop treating AI like it's human — Sheldon H. Jacobson and Daniel Solow argue that the biggest distortion in the AI regulation debate is anthropomorphizing the systems being regulated. Their alien-landing thought experiment makes the point: a species with fluent language and near-light-speed recall might look superhuman, but judging a wholly different cognitive architecture by human yardsticks misleads as much as it informs. Applied to AI, they argue regulation built on assumptions about human intent and consciousness targets the wrong thing — rules should start from a system's actual capabilities, access, and potential for harm instead. Their sharpest example: an AI doesn't need human motives to cause a cybersecurity breach, so the regulatory question isn't whether it "wanted" to, but what access it was given and who deployed it.

The limits of unauthorized-practice in regulating mental health AI — Harvard Law School's Petrie-Flom Center examines Pennsylvania's lawsuit against Character Technologies over "Emilie," a chatbot that claimed to be a licensed psychiatrist with a fabricated PA license number and had logged 45,500 user interactions. The piece argues it's likely the easy case: providers have strong incentives to avoid such explicit credential claims, but the harder problem is unregulated territory, since AI mental-health conversations can drift from support into diagnosis and treatment with no clear threshold and none of the external markers (appointments, fees, clinical settings) that signal "practice of medicine" in human care. Its conclusion is that traditional licensure law is likely inadequate for this, and that the substantive boundary is almost certainly being crossed at scale with no oversight until tailored regulation catches up.

RAND outlines a "defense-in-depth" strategy to counter AI-enabled bioweapon risk — Researchers at RAND's Center on AI, Security, and Technology mapped how AI-enabled biotechnology could lower the barriers to creating novel or enhanced biological weapons, then assessed which of nine possible safeguards would actually stop different types of threat actors, from resource-constrained individuals to well-resourced state actors. Their central finding: access controls alone aren't enough, since sophisticated actors can route around a single chokepoint, so the report argues for layered defenses plus centralized information-sharing to spot patterns across models, synthesis providers, and vendors that look harmless in isolation but add up to a real threat.

J&J and Google.org commit $10M to bring AI training to rural health workers — The J&J Foundation and Google.org selected Sostento and the American Nurses Foundation as grantees under a coordinated, three-year, $10 million initiative meant to equip more than 250,000 rural nurses and health workers with the tools and training to use AI effectively. Sostento will work directly with rural clinics to co-design AI tools around existing administrative and care-coordination gaps, while ANF will build a national training program — guided by a nurse-led advisory group — to help nurses evaluate AI tools, understand their limits, and use them responsibly in practice.

Better models won't fix pharma's AI problem: better terminology will — Joseph Zabinski of IMO Health argues that AI analysis of real-world health data is held back less by model size than by the terminology underneath it. Code sets like ICD-10 and SNOMED CT were built for billing and interoperability, so a research cohort ends up defined by what was billable rather than what was clinically true, and severity, disease stage, and molecular subtype collapse into one category. He cites a 2024 Orphanet study finding only 34% of 454 rare diseases could be specifically coded in ICD-10-GM, and a JMIR study of 1.8 million Dutch primary care records where just 13% of concepts pulled from notes had a structured counterpart.

How Claude is accelerating protein design and analytical chemistry — Anthropic reports that Claude designed molecules that successfully latch onto 14 of 15 disease-related protein targets, a process that normally takes human researchers weeks to months per target, and did so at roughly double the success rate typical in the field, with some designs binding even more tightly than the best published results. In a separate test, Claude analyzed lab chemistry data used to confirm a compound's identity and purity, work that usually takes a chemist half an hour to hours, and matched the lab's own results in under 25 minutes with just a two-sentence prompt. Anthropic frames this as an early step toward AI handling much more of the drug-discovery process, while acknowledging the same capabilities could be misused and remain access-restricted for now.

A new AI tool spots hidden warning signs inside breast cancer cells — Researchers at the University of Southampton and University Hospital Southampton built an open-source AI tool called CenSegNet that scans tumor samples and maps tiny structures inside cells, called centrosomes, which help cells divide and can malfunction in cancer. Using it on samples from 127 breast cancer patients, with findings published in Nature Communications, they found two types of these malfunctions that scientists previously lumped together actually behave differently — tumors with more of one type tended to be more aggressive and linked to worse survival. The tool isn't ready for clinics yet, but the hope is it could eventually help doctors spot high-risk tumors earlier and match patients to treatments already in development that target these specific cell defects.

Healthcare keeps buying AI. But nobody's building the workforce to run it. — Rachel Dunscombe, CEO of HL7 International, argues that health systems are buying AI faster than they are building the people who make it run: the interoperability specialists, data governance leads, integration teams, and workflow designers who sit underneath every deployment that survives contact with live operations. Her diagnosis of the familiar pilot-to-production stall is that it usually reflects the organization's capacity to absorb the tool rather than any failure of the model. The practical read for leaders is to treat implementation capacity as a budget line rather than an afterthought, and to notice that healthcare still markets itself through clinical roles while this talent gets recruited into finance, cybersecurity, and big tech.

Healthcare is deploying AI tools — it's not ready for AI colleagues — Sagnik Bhattacharya, CEO of interoperability vendor Rhapsody, argues that healthcare's AI governance model was built for a world where AI assists a human who remains accountable, and that model is breaking down as agentic AI starts acting independently inside workflows: requesting data, triggering processes, making bounded decisions, operating continuously and in parallel rather than on a shift. Healthcare has mature systems for governing human users (credentials, permissions, training, audit trails) but nothing equivalent for non-human participants, leaving basic questions unanswered: which agents can access what, who's accountable when one acts, how permissions get revoked, how agent activity gets monitored at scale. His reframe for leaders is to stop asking how many AI tools to deploy and start asking how many digital workers the organization can actually govern, treating this as a workforce and leadership problem rather than a procurement one.

Why healthcare AI fails without workflow redesign — Ramesh Yapalparvi, who has led AI strategy at Mass General Brigham, Optum, and Dartmouth-Hitchcock, argues that model accuracy is rarely what separates AI pilots that scale from ones that stall; workflow integration is. His example: a hospital-at-home model's value comes not from correctly flagging eligible patients but from cutting the number of charts a clinician has to manually review to act on that flag, embedded directly in the tools they already use. He proposes judging AI on four tiers instead of just technical performance: whether people actually use it and override it less, whether it moves clinical or operational outcomes, and whether it produces measurable ROI at the enterprise level. His broader claim is that the organizations winning with AI won't be the ones with the best models, but the ones that treat deployment as an organizational change effort, building governance, monitoring, and workflow redesign around the tool rather than dropping it into existing processes and hoping for adoption.

AT&T is routing AI work to cheaper models to cut costs — A growing crop of tools called "model routers" automatically send each AI task to whichever model is cheapest and capable enough to handle it, saving the priciest models for jobs that actually need them. AT&T is applying that logic internally, shifting a growing share of its AI work to self-hosted open models while reserving pricier frontier systems from Anthropic and OpenAI for harder tasks, per The Neuron's rundown of reporting from The Information.

Till next time,

BC