AI Symptom Checkers: How Accurate Are They?

So How Accurate Are AI Symptom Checkers?

There is no single accuracy number, because "accuracy" gets measured at least three different ways: whether the correct condition is the very first suggestion (top-1), whether it appears anywhere in the first three (top-3), and whether the urgency advice attached to it is safe. A tool can score well on one and poorly on another.

The most cited benchmark is a peer-reviewed 2020 study in BMJ Open, led by Ada Health with independent digital health experts, that tested eight symptom checkers, Ada, Babylon, Buoy, K Health, Mediktor, Symptomate, WebMD, and Your.MD, against a panel of doctors on identical cases. Ada came out on top with 99.0% condition coverage and 70.5% top-3 accuracy. Babylon scored 51.5% coverage and 32.0% accuracy on the same cases. Most of the eight apps missed the right condition in the majority of cases they were tested on.

More recent evaluations tell a similar story. A 2024 head-to-head study reported Ada at roughly 51% top-1 and 70% top-5 accuracy, with newer entrants like Ubie and Symptoma claiming higher figures, up to 71.6% top-10 and, in one company-reported case, over 95% for a narrower disease set. Independent, peer-reviewed confirmation of the highest numbers is thinner than the marketing.

How Do They Compare With an Actual Physician?

Studies compare AI-generated symptom lists against a panel of doctors to score accuracy.

This is where the numbers get more interesting than a simple pass or fail. In the 2020 BMJ Open study, general practitioners averaged 82.1% top-3 accuracy on the same vignettes, comfortably ahead of every app tested. That is not a surprising result: a trained clinician asking open-ended questions and reading subtle cues outperforms a form-based tool.

But narrow the comparison and the picture flips. A study focused specifically on inflammatory rheumatic diseases found Ada's top-1 accuracy beat physicians working from the same limited written vignette, 70% versus 54%. The lesson is not that AI beats doctors. It is that AI can match or beat a doctor working with the same restricted information a symptom checker also works with, while a doctor examining a real patient in person still has access to far more.

A separate 2025 review across 83 studies put general-purpose generative AI's overall diagnostic accuracy at 52.1%, and a 2026 Stanford and Harvard safety study found top AI models still produced severely harmful clinical recommendations in up to 22.2% of test cases. That is a meaningfully different number from the purpose-built symptom checkers above, and it is the reason experts keep separating "a chatbot answering a health question" from "a dedicated, clinically validated symptom checker." We cover that distinction in our diagnosis guide.

Why Does Accuracy Swing So Widely Between Studies?

That last row matters more than people expect. A 2026 study from Mount Sinai found ChatGPT Health undertriaged medical emergencies in roughly half of cases tested, and a separate Mass General Brigham analysis found general chatbots failed to generate an accurate diagnosis list more than 80% of the time. Purpose-built symptom checkers, trained and validated specifically on triage tasks, perform meaningfully better than an open-ended conversation with a general AI model.

How Do Popular Symptom Checkers Actually Compare?

Notice how much the numbers shift between the peer-reviewed BMJ Open study and the company-reported figures for newer apps. That gap is not necessarily dishonesty, testing methodology, case difficulty, and disease scope all vary. But it is a good reason to weight independently published, peer-reviewed studies more heavily than a company's own marketing page.

Is Triage Safety a Different Question From Diagnostic Accuracy?

Yes, and it may matter more day to day. Diagnostic accuracy asks "did it name the right disease." Triage safety asks "did it tell you the right next step," which is usually the more consequential question for an app you are using at home. The 2020 BMJ Open study found that while overall diagnostic accuracy varied widely, three apps, Ada, Babylon, and Symptomate, gave safe urgency advice in the majority of cases, performing close to the level of human GPs on that specific measure.

The failure modes run in both directions. Undertriage tells someone to stay home when they need urgent care. Overtriage sends someone to the emergency room for something that did not need it. A structured 2026 study of general-purpose chatbot triage found both errors at once: undertriage of 51.6% on clear emergency vignettes and overtriage of 64.8% on nonurgent home-care cases, an unhelpful combination that shows why calibration, not raw diagnostic accuracy, is the harder engineering problem.

How Should You Actually Use One of These Apps?

Newer conversational symptom checkers ask follow-up questions rather than a fixed form.

If you want the fuller picture of what these apps can and cannot promise beyond raw accuracy, see our guide to AI health app limits. And if a result concerns you enough to want a second read, our guide to how medical chatbots work explains what is actually happening behind the scenes.

What Does This Look Like in Practice?

Consider someone with new joint pain, fatigue, and a low-grade fever. Ada's rheumatology-focused study found the app correctly flagged an inflammatory rheumatic disease as its top suggestion 70% of the time, compared to 54% for physicians working from the same limited written information. That is a genuinely useful head start, worth bringing to a doctor.

Now consider the same symptoms entered into a general AI chatbot instead of a purpose-built checker. Research on general-purpose models found accuracy dropping to roughly 52% across a broad range of conditions, and safety studies found harmful recommendations in over a fifth of cases. Same symptoms, very different tool, very different reliability, which is the single most important lesson in this entire topic.

Should You Trust the Result?

Treat a good AI symptom checker the way you would treat a well-informed friend with some medical training: worth listening to, useful for narrowing down what to ask a real clinician, and not a substitute for one. The best-performing tools land in the 70% top-3 range on published, independent studies, meaningfully useful but well short of certain. The worst-performing ones miss the correct condition more often than they find it.

If your symptoms are severe, sudden, or match any recognized emergency warning sign, such as chest pain, difficulty breathing, sudden weakness, or confusion, seek care immediately rather than waiting on an app's output. No study of any symptom checker, dedicated or general-purpose, has found accuracy high enough to safely delay emergency care.

Frequently Asked Questions