How AI Is Changing The Way Doctors Access Medical Knowledge

Originally published in Forbes.

Imagine a 1950s doctor treating a patient with a perplexing constellation of symptoms. He first turns to his pocket handbook, the Washington Manual of Medical Therapeutics. Next, he consults Harrison's Principles of Internal Medicine, the lengthy reference text he keeps in his office. Still uncertain, he discusses the case with a colleague before searching for a relevant journal article in his hospital's medical library.

Throughout the 20th century, doctors primarily searched for medical knowledge this way. However, as the decades passed, the volume of information exploded, computers digitized it, and the internet connected the world. By the early 2000s, doctors pivoted to online information sources, first via desktop computers and later mobile devices.

Today, doctors like me have plenty of options.

We often search databases like PubMed and Google Scholar. We frequently read UpToDate topic summaries. Two-thirds use MDCalc, a point-of-care reference with various decision-support tools. We visit professional societies' websites, medical sites like WebMD, and sometimes even non-medical sites.

PROMOTED

Artificial intelligence is now again changing how we access knowledge. Let me explain how.

Staying Up To Date In An Informational Tidal Wave

PubMed indexes 36 million abstracts and adds 1 million more yearly – two per minute. Google Scholar contains roughly 400 million articles, citations and patents. Several thousand clinical practice guidelines exist. No doctor can keep up.

In the early 1990s, Dr. Bud Rose, a tennis-loving, forward-thinking Harvard nephrologist, sought to solve this problem by creating a computerized program of searchable, regularly updated clinical "topic cards" on floppy disks. He named his company UpToDate.

As the number of topics increased, distribution shifted to CD-ROM, the internet and mobile.

Now part of Wolters Kluwer Health, UpToDate's roughly 8,000 affiliated clinical experts and 60 deputy editors follow peer-reviewed, evidence-based medicine methodology to develop and maintain 12,000 clinical topic summaries across 25 medical specialties.

The product is wildly successful. Nearly 3 million clinicians at 50,000 sites use UpToDate, primarily through institutional subscriptions. Individuals may purchase subscriptions for $579 annually.

Its authoritative summaries are popular for good reasons. As Wolters Kluwer Health’s Chief Medical Officer, Dr. Peter Bonis, told me, "Everything we do is to help support clinicians in making the best decisions for their patients."

However, because it is not a querying service, doctors must review relevant topic summaries to find the specific answers they seek. For example, UpToDate cannot directly answer a user who asks, "How do I treat small intestinal bacterial overgrowth?" Instead, the user must search for SIBO, select the topic summary, and at least skim it to find that the authors recommend Rifaximin as the first-line therapy. The company reports that the median user interaction lasts 60 seconds.

Example of an UpToDate topic summary

UpToDate

Harnessing AI To Manage Medical Knowledge

UpToDate is now adding AI-powered search capabilities so users can directly access focused, relevant, word-for-word passages from existing summaries. The goal is to make the product easier and faster to use without introducing errors that potentially occur with AI-generated, synthetic content.

New, purpose-built, AI-native search engines like OpenEvidence and Consensus are taking a different approach. Instead of surfacing pre-written topic summaries, these products answer user queries directly and dynamically.

Their core challenge is ensuring their output is reliable enough for medical practice. Generative AI products may produce erratic answers; for example, Google Gemini infamously recommended people eat one rock per day because "rocks are a vital source of minerals and vitamins."

These "hallucinations" partially reflect the old "garbage in, garbage out" problem. As OpenEvidence Founder Daniel Nadler explained, "An index of websites is not an index of facts." LLMs trained on the entire Internet, including sites like Reddit and the Onion, are bound to generate false information.

Launched out of the Mayo Clinic Platform Accelerate Program, Nadler's company attempts to sidestep these issues by drawing exclusively from the peer-reviewed biomedical literature.

After users (who must be clinicians) enter a general or specific question (e.g., "How do I treat SIBO in a patient who cannot afford Rifaximin?"), OpenEvidence identifies potentially relevant sources across millions of clinical documents, including indexing meta-data from PubMed abstracts, full-text journal articles, monographs, book chapters, and more. Next, it selects the most authoritative sources based on factors such as relevance to the query, publication date, journal impact factor, and citation count. Finally, large language models synthesize a summary response with links to the cited sources.

OpenEvidence is quickly spreading across the medical community. More than 250,000 clinicians have visited the site since January, performing nearly 2 million queries in November alone. The product is supported by advertising, making it free for individual users.

Consensus is another AI search engine covering medical and non-medical scientific fields, such as biology and environmental science. Available to the public, one in five users is a clinician. After inputting a question, Consensus synthesizes an answer, including a “Consensus Meter” (for yes/no questions) that shows the strength of the recommendation.

Example of an OpenEvidence query response.

OpenEvidence

Balancing Tradeoffs

To make an analogy, searching PubMed or Google Scholar is like asking a librarian for the best sources on a given topic. UpToDate is comparable to the library's special collection. With each, users must scan a list of blue links, select and read a source, and abstract any pertinent information. In doing so, they learn about a topic and, with some time and effort, may answer specific questions.

Conversely, using AI search like OpenEvidence or Consensus is like directly asking a wise professor a question and receiving an answer with references. It is fast, convenient, and specific. The risk is that some convincing answers may be subpar or lack enough context. Clinicians must, therefore, remain "in the loop" and dig deeper when indicated.

I asked two of the brightest informatics physicians I know about AI search.

MDCalc founder and emergency physician Dr. Graham Walker warned, "There are very real concerns around automation bias — where the physician starts to implicitly trust these tools as smarter than them — and slowly erodes the physician's ability to think critically about their patients."

Conversely, cardiologist and informaticist Dr. Larry Klein finds AI search a significant net positive. He told me, "I use OpenEvidence every day, asking it questions as if I were asking one of my expert colleagues. It is truly revolutionary."

Both traditional and AI searches have roles in different situations. For example, a resident admitting a patient with acute coronary syndrome may read UpToDate to understand management principles. A distinguished nephrologist may search PubMed to to review the characteristics of patients included in glomerulonephritis treatment trials. A family nurse practitioner deciding on H. pylori treatment in the setting of a penicillin allergy may query OpenEvidence for quick guidance.

Looking Ahead

Clinicians raise at least one medical knowledge question – typically about the cause of a symptom or treatment of an ailment – for every two patients we see. Yet, we only pursue answers for half of them, mainly because we lack enough time.

AI search can help solve this problem by pushing information forward much faster. Yet this new way of working highlights the tensions between human and machine curation, nuance and brevity, automated and manual processing, and potential machine-generated and human errors. We must carefully evaluate how these tools impact our clinical workforce and patients.

Throughout history, changes to how doctors manage information have been met with concerns. For example, many physicians initially opposed learning from medical textbooks, arguing that conjuring knowledge from memory forced deeper reflection. Decades later, others "insisted that the digitization of medical information corrupted the traditions and cognitive practices enabled by tangible, paper-based tools."

We should also be hopeful. The ability to quickly tap into collective knowledge better equips us to relieve suffering and promote well-being. It may also ease our cognitive load and help sustain our energy.

With these new capabilities, we must shift our focus from recalling facts to asking the right questions. While answers will come faster than ever, there is relatively little published data for most clinical decisions. Clinical intuition and experience still matter— greatly! As pioneers of the evidence-based medicine movement explained, "Good doctors use both individual clinical expertise and the best available external evidence, and neither alone is enough."

Disclosure: after publishing this article, I started advising OpenEvidence.

Acknowledgments: I want to thank Peter Bonis, Sean Case, Hugh Harvey, Larry Klein, Daniel Nadler, Eric Olson, Graham Walker, and Patrick Wingo for discussing this topic.