{"id":556332,"date":"2026-05-04T14:00:00","date_gmt":"2026-05-04T14:00:00","guid":{"rendered":"https:\/\/winklersart.com\/?p=556332"},"modified":"2026-05-04T14:00:00","modified_gmt":"2026-05-04T14:00:00","slug":"a-i-outperformed-doctors-at-diagnosing-real-world-e-r-patients-in-a-new-study-that-doesnt-mean-computers-will-replace-clinicians","status":"publish","type":"post","link":"https:\/\/winklersart.com\/?p=556332","title":{"rendered":"A.I. Outperformed Doctors at Diagnosing Real-World E.R. Patients in a New Study. That Doesn’t Mean Computers Will Replace Clinicians"},"content":{"rendered":"
\n

One of OpenAI\u2019s large language models did better than physicians in several experiments, hinting that A.I.-assisted emergency medical care could be around the corner<\/h2>\n
\n
\n
\n

\n

Rudy Molinek<\/p>\n

| Reporter<\/span><\/p>\n

\n\"ER\n <\/picture>
\n

The A.I. model outperformed two doctors when presented with data from dozens of real E.R. patients.
\n Harrison Keely via Wikimedia Commons under CC BY 4.0<\/span>
\n <\/figcaption><\/figure>\n

Since the 1950s, scientists have been comparing human doctors to computers to see if the machines\u2019 algorithms can accurately diagnose complex health conditions. In a standard test, computers attempt to puzzle out challenging case studies from the New England Journal of Medicine<\/em>.<\/p>\n

Machines have recently improved at the task, primarily because of artificial intelligence built on large language models, like the one powering OpenAI\u2019s ChatGPT. But so far, A.I. has done well only when it comes to curated case studies.<\/p>\n

Now, researchers have put an A.I. model\u2014a preview version of OpenAI\u2019s o1\u2014to the diagnostic test with real hospital records. Based on written documentation alone, the technology outperformed practicing physicians, according to a study published April 30 in the journal Science. The findings suggest that A.I. could be a powerful aid to health care workers during stressful, time-sensitive situations.<\/p>\n

\u201cThis is the big conclusion for me: It works with the messy real-world data of the emergency department,\u201d says study co-author Adam Rodman, a clinical researcher at Beth Israel Deaconess Medical Center in Boston, to NPR\u2019s Will Stone. \u201cIt works for making diagnoses in the real world.\u201d<\/p>\n

In one test, Rodman and his colleagues presented the A.I. and two doctors with E.R. health records of 76 Beth Israel patients at three stages of care: initial triage with a health worker, first interaction with a doctor and admission to the hospital. The experiment had no impact on actual patient care.<\/p>\n

These records typically include just a few sparse details, like vital signs, demographic information and a brief description written by staff, per the Guardian<\/em>\u2019s Robert Booth. Initial interactions are critical points in a patient\u2019s care, as sometimes life-or-death decisions must be made quickly in chaotic situations.<\/p>\n

Analyses revealed that the two physicians came up with exact or near-exact diagnoses in 50 percent and 55 percent of the cases, while the A.I. was close or exactly right 67 percent of the time.<\/p>\n

This test was the \u201cmost important\u201d of the study\u2019s six experiments, says co-author Thomas Buckley, a computer scientist at Harvard Medical School, to Science<\/em>\u2019s Perri Thaler. A.I. did well in the others too\u2014so well that the researchers feared people wouldn\u2019t believe the results, Rodman tells the outlet.<\/p>\n

Clinicians are increasingly incorporating A.I. into their daily work, using the programs for tasks like transcribing notes from patient interactions, reviewing health scans and detecting early signs of disease. The new findings suggest that A.I. reasoning models like o1, which can execute and explain step-by-step logical thinking, could soon regularly help doctors formulate diagnoses.<\/p>\n

\n
\n

Did you know? A.I. in breast cancer detection<\/p>\n

In a study published in January, researchers found that A.I. could help doctors find hard-to-detect signs of breast cancer. These can be missed during routine mammography screenings, which are generally recommended to take place once every one or two years.<\/p>\n<\/p><\/div>\n<\/div>\n

The study is especially notable given that o1 was first released at the end of 2024. \u201cThat\u2019s kind of like ancient history now in machine learning time,\u201d Buckley tells Science<\/em>.<\/p>\n

Still, the researchers, as well as doctors and scientists not involved in the new study, caution that these promising results don\u2019t mean physicians are about to be replaced by A.I. For one thing, purely logical reasoning excludes human aspects of a doctor\u2019s work.<\/p>\n

\u201cWhen we say clinical reasoning, it doesn\u2019t mean the same thing as moral reasoning,\u201d Arya Rao, a biomedical informaticist at Harvard Medical School who wasn\u2019t involved in the study, tells Science News<\/em>\u2019 Kathryn Hulick. \u201cThese models have been optimized to do this kind of sequential thought that we call reasoning, but it\u2019s not at all the same thing as how we teach medical students to reason.\u201d<\/p>\n

Additionally, the researchers say the A.I. might not perform as well with larger amounts of patient data, such as from someone who\u2019s admitted to the hospital for a days-long stay.<\/p>\n

Going forward, the team plans to conduct clinical trials to figure out how best to integrate A.I. into patient care, reports Science News<\/em>.<\/p>\n

This matches the view of physician Nour Khatib, of Oak Valley Health in Canada, who was not involved in the study. \u201cIt\u2019s just another tool to help us give the patient the highest quality care possible,\u201d she tells the CBC\u2019s Nick Logan.<\/p>\n