There’s a warning out today for those who go online or to apps to figure out why they have an upset tummy or nagging cough or occasional chest pain. Symptom checkers, those tools that ask for information and suggest a diagnosis, are accurate only about half of the time.

The finding is from a Harvard Medical School study that reviewed 23 sites, such as WebMD, the Mayo Clinic and DocResponse. One third listed the correct diagnosis as the first option for patients. Half the sites had the right diagnosis among their top three results, and 58 percent listed it in their top 20 suggestions.

Dr. Ateev Mehrotra, one of the study’s authors, urges patients to be cautious when using these tools.

“These sites are not a replacement for going to the doctor and getting a full evaluation and diagnosis,” he says. “They are simply providing some information on what might be going on with you.”

About a third of U.S. adults use the sites, although not necessarily in place of going to the doctor.

Some of the diagnostic questions are also used by nurse triage phone services and,

Mehrotra says, these online tools are about as accurate as the call-in lines offered by many insurers and physician groups. “[They are] better than just a random Internet search,” he said.

Researchers entered the symptoms of 45 patients from vignettes used to train medical students. The Mayo Clinic’s first online diagnosis was right only 17 percent of the time, but had the correct diagnosis on a list of 20 in 76 percent of cases. Dr. John Wilkinson, who works on Mayo’s symptom checker, says the tool directs patients to medical research and prepares them to talk to their doctor.

“We’re always trying to improve but if most of the time the correct diagnosis is included in the list of possibilities, that’s all we’re attempting to do,” he says.

The diagnosis accuracy rate for physicians is 85 to 90 percent. But Jason Maude, who runs a high performing tool called Isabel, says he does not want a Web versus doctor showdown.

“The whole point is not to set the patient against the doctor or replace the doctor, but to make the patient much better informed and to ask the doctor much better questions, and then together they should do a much better job,” he says.

Isabel ranked well in the study, showing the correct answer more than 40 percent of the time in the first diagnosis and 84 percent in the top 20 answers. Those high results, Maude says, may be because the site lets patients type in their own description of symptoms. They might describe a “tummy ache” or “stomach cramps” rather than the more clinical choice of “abdominal pain” used by many online symptom checker tools. And Isabel asks just two or three questions before patients describe their problem, as compared to sites that ask patients to click through 20 questions — steps Maude said may discourage use.

Clarifying how and why patients use these tools is critical, say the study’s authors. They could reduce unnecessary office visits or inform patients as they talk with their doctors.  But for some, the tools may encourage people to seek unnecessary care.

Mehrotra says patients used symptom checkers more than 100 million times last year, a fact that may stun some physicians.

“While most doctors know patients are going to the Internet to search for medical advice, in terms of these symptom checkers, I’ve been surprised that few of my colleagues even knew they existed,” he says.

This story is part of a reporting partnership with NPR, WBUR and Kaiser Health News.

Kaiser Health News (KHN) is a national health policy news service. It is an editorially independent program of the Henry J. Kaiser Family Foundation.