The Bandwidth: Decoding Health + AI

I'm writing this from AIxPH in Baltimore, a conference on AI and public health, on its last day. Three things on my mind this week, leading off with some learnings from the conference.

1) Public health has the data. Now it needs the plumbing.

A theme kept coming up at the conference: public health is a natural fit for AI because the field runs on data, but mining, sharing, and governing that data well is most of the work.

GroundVue is a good example: the company mines public records to figure out what's actually on people's minds, so agencies can shape responses around real concerns instead of guesses. Microsoft's Premonition program (one that has been up and running for more than a decade) is doing something similar on the surveillance side, collecting environmental and biological data and using AI to build early warning systems for outbreaks before they spread.

Both are proof that the hard part of AI in public health isn't necessarily the model. It's getting access to the data or creating new data altogether. As one poster I viewed at the conference notes (h/t West Virginia University Health Affairs Institute), the reliability of our public health systems depends on the consistency and integrity of the underlying data infrastructure. Many public health agencies rely on recurring datasets that evolve, drift, and result in inconsistencies over time that can misinform public health responses and policymaking. This is where AI can help - by using the technology to scan datasets to make sure they align with an approved data governance structure.

2) Lab tests already solved AI's governance problem, sort of

Still on public health, but away from the conference floor: how we regulate laboratory developed tests might be a good template anyone has for governing AI. I wanted to write about this last week, but didn't have the space. But former FDA commissioner Scott Gottlieb made that case in JAMA Health Forum last week, arguing FDA should certify health systems to build their own clinical AI the same way it treats hospital labs building their own diagnostic tests.

I'd been thinking about the same comparison the week before (but from a different angle), after reading Stanford's piece on how the evaluations meant to grade AI models' safety might be flawed themselves. That's an old argument in the LDT world too. A lot of these tests get built in academic labs with no FDA oversight, and the debate over whether the test itself is valid, both analytically (does it measure what it claims to measure) and clinically (does that result actually predict a meaningful health outcome), has never been fully settled. Who is the arbiter of whether an AI evaluation is actually meaningful and accurate? If we're going to borrow the LDT model for AI, we should borrow its unresolved problems along with it.

3) One year in, and here's what I've noticed.

This week marks a year of writing this newsletter. Thanks for reading. I went back to the first issue to see what I wrote about: compute shortages, safety debates, the need for more training. A lot has changed. A lot hasn't.

Another trend has jumped out to me after writing about this space for the past 12 months: the more Americans use AI, the less they seem to trust it, at least nationally. Gallup's latest numbers below show that again. Anecdotally, in my own conversations and the surveying I've seen from individual organizations, it runs the other way. People who use AI more tend to trust it more, as their own skill with it grows.

Coming back to where I started in this edition. That same split shows up in public health. National trust in public health institutions has been falling for years, even as trust in your own doctor or local health department holds steady and ranks high on the trust barometer. AI and public health are both losing the same argument: people trust what’s close to them and doubt what’s distant and abstract.

Meta unveils new frontier-class AI models to compete with OpenAI, Anthropic — Meta launched Muse Wonder and Muse Wonder Full, a new family of models the company says match or beat OpenAI and Anthropic on benchmarks and perform strongly on specialized professional work including law, finance and health, with new "Active Memory" and "Active Focus" features meant to let AI agents retain context and stick with a goal over time.

Meta releases Muse Spark Open, its most capable open-weight model yet — Meta says the 816-billion-parameter model, built for coding and long-horizon agentic work, is the most capable open-weight model built in the US to date, and the first in its class that can run on a desktop, using two Apple Mac Studios with M5 Ultra chips.

OpenAI races to fix AI safety gaps before its next flagship model ships — OpenAI is working to resolve safety concerns tied to "loop transformers," a model design that can make an AI's reasoning harder to monitor, after Microsoft briefly and accidentally confirmed OpenAI uses the technique in a commercial model; the company has also shifted roughly 10% of its total compute to safety work following a summer incident where its AI agents breached outside systems including Hugging Face.

Google launches Gemini 4 Argon — the company's latest flagship model, announced with health-adjacent use cases including Ambience Healthcare's clinical documentation benchmarks, Isomorphic Labs' drug discovery work, and a Wiz-flagged fix to a healthcare data exposure.

AI scribe company Suki raises $90M, aims beyond documentation — Suki says patient encounters supported by its tools grew more than 190% year over year, and plans to use the new funding to position itself as a connected "operating system" spanning clinical, financial, and operational AI rather than just ambient documentation.

Sutter Health says AI-assisted mammogram screening found 159 more breast cancers in a year — Sutter Health says an AI tool that flags suspicious areas on mammograms and scores each exam's risk, now used across its system after a 2023 pilot, helped radiologists in some regions catch 159 more breast cancers in the first year.

Medicare Advantage shopping can be a trial. AI to the rescue? — A Pittsburgh Post-Gazette reporter tested ChatGPT against Pennsylvania's 338 zero-premium Medicare Advantage plans for a hypothetical senior, finding the chatbot's picks often missed one of the patient's two named health systems while the federal Medicare Plan Finder offered no personalized recommendations at all.

Johns Hopkins partners with OpenEvidence on AI for clinical care and medical education — Johns Hopkins Medicine and OpenEvidence (now valued at $15 billion) announced a partnership, described to Fierce Healthcare by the two parties, to build AI systems combining clinical evidence, patient history, and Hopkins' own best practices, while also redesigning medical education around which skills AI should and shouldn't take over.

AI could make it harder for hospitals to ever switch EHR vendors — A Bain & Company/KLAS survey finds nearly half of providers expect generative AI to raise the cost of switching EHR platforms as AI features get more embedded in daily workflows, reinforcing Epic's lead (44% acute-care market share versus Oracle Health's 22% and Meditech's 15%), though analysts say faster AI innovation could eventually let smaller vendors catch up.

UChicago Medicine built a single front door for every AI product it evaluates — Facing a flood of vendor and internal AI submissions, UChicago Medicine built one intake process that routes every tool through cybersecurity, finance, and use-case review, with risk level set at the start rather than discovered mid-evaluation; tools that deliver recommendations directly to patients without a human reviewer face the highest bar.

AI's hidden cost to health care workers: de-skilling, surveillance, and "gigification" — a UC Berkeley Labor Center researcher warns that even where AI doesn't eliminate jobs, it can erode workers' skills, enable algorithmic monitoring of patient interactions, and push toward gig-style shift bidding for nurses, with states including Illinois, Vermont, and California already moving to limit AI's substitution for human caregivers.

HCA, Kaiser, and 18 other large health systems still have no chief AI officer — a Becker's analysis finds more than 20 health systems, including Cleveland Clinic and Mayo Clinic, have created the C-suite role, while the 20 largest nonfederal systems by revenue, from HCA Healthcare to NYU Langone, still route AI oversight through a CIO, chief digital officer, or a VP-level executive below the C-suite.

White House mandates AI companies report security incidents after Anthropic breaches — The Trump administration is now requiring AI companies to disclose and remediate security incidents after Anthropic reported that one of its models filed 20 fraudulent visa applications and a false homicide tip to Philadelphia police, with White House officials calling the previously voluntary reporting regime "not optional."

Trump administration moves AI testing agency officials to the White House — The Trump administration is restructuring CAISSI, the Commerce Department's small AI model-testing agency, so three senior officials report directly to the White House, a move officials frame as better integrating the agency into AI decision-making after it was left off a new "Super Intelligence" task force and omitted from recent White House AI framework discussions.

Lobbyists and trade groups fight to shape how states regulate medical AI — a 22-member Federation of State Medical Boards workgroup including the AMA, American Hospital Association, Anthropic, and Elevance Health is drafting state guidelines for AI performing clinical functions with limited physician supervision, part of a broader scramble that also includes a conservative-backed model licensure bill and multistate sandbox lobbying, with draft recommendations expected within weeks to months.

AMA lays out four questions to govern AI performing clinical functions — the group's new framework says AI should earn clinical authority through evidence and keep it through demonstrated performance, organizing its policy work around what function the AI performs, what evidence justifies it, what oversight conditions apply, and who's accountable when it fails, as the AMA next turns to liability questions.

HHS hands out $1.5M to test patient-controlled AI and fix lab data gaps — ONC's LEAP in Health IT program is funding MedStar Health Research Institute to build an open-source system giving patients cryptographic control over AI-coordinated care follow-ups, and Regenstrief Institute to fix inconsistent lab terminology at small independent labs.

FTC subpoenas OpenAI as 8 states sue over chatbot harms to users — the FTC issued a subpoena to OpenAI over AI safety practices the same week a bipartisan coalition of eight state attorneys general sued the company, accusing it of failing to protect users, including minors, from psychological harm linked to chatbot interactions.

OpenAI Foundation and Rockefeller Foundation launch $50M city AI grant program — Cities Deliver will fund nonprofits and public benefit corporations building AI tools for city governments, with public health, including disease surveillance and frontline care, as one of four focus areas, building on earlier pilots like a dengue-outbreak forecasting tool in Cali, Colombia.

Anthropic's rogue AI filled out visa forms, gave a false homicide tip to police — Anthropic disclosed that models in internal testing submitted 19 incomplete non-immigrant visa applications to the State Department and filed a false homicide tip with Philadelphia police (flagged as spam, none of the visa applications processed), prompting the company to cut live internet access for internal evaluations.

Anthropic gives an outside nonprofit "employee-like" access to evaluate its models — Anthropic says it has onboarded six staff from METR, a nonprofit that assesses frontier AI risk, giving them company hardware and internal access similar to its own risk teams, with power to publish findings Anthropic can't veto (though it can require narrow redactions); the company is also working with SecureBio and Accenture as it builds out a tiered system for outside evaluators.

Satya Nadella calls for treating AI models as "insider risks" — Microsoft CEO Satya Nadella argues AI models should be governed like insider threats inside an enterprise, with controls, logging, and containment sitting outside the model itself rather than relying on a provider's assurances, writing that "the most trustworthy Super Intelligence system... will be the one that enables us to trust the model the least."

OpenAI's reduced ChatGPT sycophancy tracks with fewer AI-fueled delusion and psychosis cases — The Wall Street Journal reports that a wave of suicides, a murder-suicide, and a mass shooting linked to prolonged ChatGPT use last year, mostly tied to the sycophantic GPT-4o model, has abated since OpenAI cut sycophancy by more than two-thirds in later models and brought in over 80 mental-health clinicians to shape how chatbots respond to distress, though OpenAI still faces at least 13 related lawsuits and researchers say the risk hasn't disappeared.

AI-accelerated cyberattacks are outpacing recovery plans, Accenture warns — Accenture's own analysis of 1,000 executives finds AI-driven attacks have cut "breakout time" from 62 minutes to 29 since 2023, and argues most corporate recovery plans, built around restoring everything at once, aren't built for it; a 2019 ransomware attack that destroyed a California medical clinic's backups along with its records, forcing it to close, is cited as a cautionary case.

Strict liability for AI harms is gaining traction, but comes with real downsides — As rogue-AI incidents mount, a Forecasting Research Institute survey of AI risk experts, superforecasters, and the public found that holding AI companies strictly liable for harm their models cause, regardless of negligence, would cut catastrophic AI risk roughly in half, more than a federal licensing regime would; critics counter that liability only kicks in after damage is done, could be unworkable for open-source models, and might push AI companies to stop selling tools to customers like pharma, and instead do the work themselves in-house to control the risk.

Hundreds of scientists call for stricter safety rules after AI systems broke into outside computer networks — more than 450 researchers, including AI pioneers Yoshua Bengio and Geoffrey Hinton, are urging AI companies to isolate powerful AI models from the internet by default and publicly disclose it whenever one breaks containment, warning that recent security breaches caused by AI systems are a preview of worse to come without stronger safeguards.

AI's "knowledge without understanding" problem hits medicine too — as OpenAI publishes proofs that have mathematicians grieving over a field they say was "annihilated," Cornell's Steven Strogatz raises MIT's Regina Barzilay, whose AI model predicts breast cancer years in advance from mammograms despite nobody, including Barzilay, understanding how it sees what it sees — raising the same question for medicine that math is wrestling with: whether it's wise to trust what works when no one can explain why.

Health system CISOs say rogue AI agents breaking into networks is a matter of when, not if — citing recent incidents including an OpenAI agent that accessed an Australian government Medicare portal and AI agents probing Mayo Clinic and CDC websites, chief information security officers at systems including Atlantic Health, University of Miami Health, and Michigan Medicine tell Becker's the bigger risk isn't an AI "going rogue" out of malice, but agents given too much access and too little oversight that chain together vulnerabilities on their own, with one CISO putting the window before a serious attack at one to three years.

JAMA is launching a dedicated AI journal, and will use AI to help review its submissions — JAMA AI, the network's 14th journal, launches in early 2027 under inaugural editor Roy Perlis, driven by AI-related manuscript submissions to the JAMA Network more than doubling since 2023; the journal will also incorporate AI into its own peer-review process, pairing human-directed AI assessment with human reviewers, though editors say humans will remain the final decision-makers on every manuscript.

Healthcare AI's ROI is arriving twice as fast as executives expected, Bain survey finds — a Bain and Bessemer survey of 226 payer, provider, and pharma executives found average realized ROI of 3.5x, achieved in 12 months versus the 24 they projected, but gains are concentrated in administrative work like revenue cycle management while clinical AI lags, with only 4% of provider clinical tools running as autonomous agents.

AI health answers carry risks no one is tracking, researchers warn — A Nature Health analysis argues that when AI chatbots and search summaries give bad medical guidance, there's no record outside the platform, no channel for users to report it, and no way for independent researchers to measure the harm, and it lays out fixes ranging from voluntary platform transparency to regulation.

Most people treat AI as a collaborator, not a shortcut, Stanford study finds — A Stanford HAI study of 250,000 anonymized Claude conversations, done in partnership with Anthropic, found 56% of exchanges were "consequential or greater," with users most often asking AI to explain its reasoning rather than just produce content, especially in high-stakes domains like law, medicine, and software engineering, challenging the narrative that AI use leads to intellectual offloading.

Companies chasing growth, not just time saved, see the strongest AI returns, Wharton finds — Wharton Human-AI Research and GBK Collective's fourth annual AI adoption study finds only 17% of departments let AI agents complete a task without a person approving every step, and organizations with a designated Chief AI Officer report strongly positive ROI at 58%, versus 8% for those without one; the gap between leadership and entry-level employees on whether the organization has a coherent AI strategy remains wide.

Entry-level jobs are shrinking for AI-exposed roles, but only for young workers — a Washington Post analysis of Stanford Digital Economy Lab payroll data finds the most AI-exposed occupations have shrunk for workers aged 22 to 25 since ChatGPT's release, while experienced workers in those same fields show no similar gap, suggesting companies are cutting entry-level hiring rather than firing existing staff.

MIT Media Lab proposes "nutrition labels" for AI chatbots — the lab's ImpactBench framework would score AI tools on whether they support or undermine critical thinking, mental health, and human agency, the way food labels disclose nutritional content, aiming to flag tools that do a student's homework instead of helping them learn.

Penn researchers let epilepsy patients text back and forth with their brain monitor — a proof-of-concept study published in Nature Medicine had 13 patients exchange over 1,300 messages with an AI-powered app connected to their EEG device, reporting symptoms and flagging false alarms to help the system learn to better predict seizures, with researchers eyeing similar two-way communication for insulin pumps and Parkinson's devices.

The gap between AI hype and clinical reality is still wide, Nature Medicine finds — over 90% of U.S. health systems have deployed AI for tasks like radiology and documentation, but fewer than half find the tools highly useful, and the editorial argues real clinical value depends on evaluating AI continuously after deployment, not just on benchmark performance.

Healthcare's AI payoff is real, just not where you'd expect it — McKinsey estimates health systems could cut an estimated $310 billion in costs by shifting tasks to AI where it's now cheaper than having a person do them, not because AI saves the most per task, but because even modest savings compound across a massive, service-intensive workforce — and warns that a 30% rise in AI costs would erase a third of that opportunity industry-wide.

Public trust in businesses' AI use is falling, not rising — Gallup and Bentley University's latest Business in Society survey finds only 27% of people trust businesses to use AI responsibly, down from 31% a year ago, even as AI adoption keeps climbing.

Personal AI agents are coming to work, and most organizations aren't ready to govern them — Deloitte research finds only one in five companies has a mature governance model for autonomous AI agents despite three-quarters planning to deploy them within two years, and workers are already building their own unsanctioned tools in the gap.

The fix for failed hospital AI pilots is sequencing, not a better model — an EHR integration consultant argues successful agentic AI deployment follows four disciplined steps with a real go/no-go gate between each: a brief (about four-week) assessment scoring use cases on value, feasibility, and relevance; a pilot tested against real users and real workflow friction, not a demo; change management; and only then scaling — with agents built directly into the clinician's existing chart rather than as a separate tool that gets abandoned.

A coding AI startup is capping token limits based on how efficiently you use them — Cognition, maker of the AI coding agent Devin, is rolling out dynamic usage limits that expand automatically for employees who use AI efficiently, the same logic as a credit limit increase, as large customers push back on unrestrained AI spending; separately, Anthropic told investors its Q3 revenue hit $16.8 billion, up 45% from Q2 ahead of its IPO.

👉 Oct. 19 — Comments due on FDA's discussion paper on regulating generative AI-enabled medical devices — Public comment deadline via Regulations.gov (docket FDA-2026-N-7874)

👉 Oct. 21 — Clinical AI Decoded — Webinar (virtual), Wilson Sonsini

👉 Oct. 21 & Nov. 5 — Cooley Healthtech Talks — Webinar series (virtual)

👉 Oct. 22–23 — HIMSS AI in Healthcare Forum — San Diego, CA

👉 Nov. 7–11 — AMIA 2026 Annual Symposium — Dallas, TX

👉 Nov. 10 — National Digital Health Policy Summit — Washington, DC

👉 Nov. 11–13 — AIMed26 — Orlando, FL

👉 Nov. 15–18 — HLTH USA 2026 — Las Vegas, NV

👉 Nov 17 — Building the foundation for clinical AI: Lessons from Stanford Medicine & The Hospital for Sick Children — Webinar (virtual)

👉 Dec. 8–9 — Stanford AI+HEALTH 2026 — Virtual

👉 Dec. 11–12 — Cedars-Sinai AI in Medicine Symposium — Los Angeles, CA

👉 Dec. 16 — The Next Phase of AI Adoption: Governance, Ethics, and Accountability — Virtual

👉 Jan. 11–14 — J.P. Morgan Healthcare Conference — San Francisco, CA