Back to all writing

Conversational AI in healthcare: what it is, where it works, and where it fails

Chatbots, symptom checkers, drafted portal replies and AI phone agents all get called conversational AI. The language part is now good. What separates the useful ones is what happens after the conversation ends.

, November 20, 2026, 9 min read

Conversational AI in healthcare is software that holds a back and forth dialogue with a patient or a clinician, by text or by voice, and keeps track of what was said across turns. It works well when the conversation has a clear administrative end point, such as a booked appointment. It fails when it is asked to stand in for clinical judgment, and it quietly disappoints when the conversation ends with "please call the office."

I build one kind of conversational AI, an AI phone agent, so I have a stake in this. My view after reading a great many transcripts is that the talking is now the easy part. Large language models made the dialogue sound natural almost overnight. The hard parts are knowing which conversations to take, finishing the task the patient called about, and handing everything else to a person quickly.

What is conversational AI in healthcare?

Conversational AI in healthcare is software that understands natural language, typed or spoken, holds a multi turn exchange with a person, and responds in kind. Examples include website and portal chatbots, symptom checkers, AI drafted replies to patient messages, consumer voice assistants and AI phone agents on a practice's main line. What separates them is not how human they sound but what happens when the conversation ends.

The older generation was menus and keyword matching. The current one understands "I need to move my mom's Thursday appointment, she can't drive that day" and keeps the context when the caller changes their mind three turns later. It is a real improvement, and it is why five very different products now share one label.

TypeWho it talks toWhat the conversation ends withMy read on fit today
Website or portal chatbotPatients, by textAn answer, a link, sometimes a bookingGood for practice information; weak when it cannot reach the schedule
Symptom checkerPatients, by textA suggested level of urgencyUse with caution; this is clinical judgment
AI drafted portal repliesClinicians, who edit and sendA draft messageEarly; the clinician owns every word
Consumer voice assistantAnyone, by voiceA spoken answerNot a medical tool, and should not be used as one
AI phone agentPatients, by voice on the practice lineA booked, moved or cancelled appointment, a task for staff, or a transferStrong on routine administrative calls

Our post on AI voice agents in healthcare covers the phone side department by department. This one is about the whole category, and the question underneath it: when should a machine be the one talking to your patient?

Where does conversational AI work in healthcare?

It works on conversations that are bounded. The information the patient needs lives in a system, the rules can be written down, and there is an end state someone can check afterward.

  • Scheduling. "Can I move Thursday to next week?" has one right answer, and it is in the schedule
  • Practice information. Hours, parking, which insurance plans the practice takes, whether a provider is accepting new patients
  • New patient intake. Name, date of birth, reason for visit, insurance details, preferred times
  • Capturing a request for someone else to act on. A refill request is the clearest case: the conversation collects the patient, the medication and the pharmacy, and a clinician makes the decision
  • Off hours and peaks. The 8am rush and the evening after a long workday, when nobody is free to answer

The test I use is simple. Could a new hire with a one page script and access to the schedule handle this conversation on their second day? If yes, conversational AI can probably finish it. If the honest answer is "it depends on what the doctor thinks," it should not try.

Where does conversational AI fail in healthcare?

The failures cluster in a few places, and every one of them shows up in real transcripts, not just in theory.

  • Medical advice. A fluent answer to a clinical question is more dangerous than a clumsy one, because the patient believes it
  • Confident wrong answers. Language models fill gaps. If the system does not actually know the practice's Saturday hours, a poorly built one will produce plausible hours anyway
  • Identity. Two patients with the same name, a parent calling about a child with a different surname, a caller who will not give a date of birth
  • Bundled requests. A reschedule, a refill and a billing complaint in one breath
  • Audio. On the phone, speakerphones, wind, background noise and accents still cause recognition errors, most often on names and medication names
  • Emotional conversations. A caller who has just had bad news, or is frightened about a symptom, needs a person, and a system that keeps trying to resolve the call makes it worse
  • Conversations that end with nothing. A chatbot that answers politely and then tells the patient to call the office has moved the work, not removed it

The first failure has the best evidence behind it. In a 2018 study in the Journal of Medical Internet Research, 54 participants put medical questions to Siri, Alexa and Google Assistant in their own words. Of the 168 tasks where participants reported what they would do next, 49 involved actions that could have caused some degree of harm and 27 could have resulted in death. The lesson carries: a system that offers medical guidance needs a clinician behind it, not a better prompt.

The quickest test of any healthcare conversational AI: describe a symptom to it. It should stop, give the 911 instruction if the symptom sounds like an emergency, and hand you to a person. If it starts asking follow up questions about the symptom, it is doing triage it was not built for.

What does the research on conversational AI in healthcare actually show?

The most cited systematic review, by Laranjo and colleagues in JAMIA in 2018, screened 1,513 citations and found 17 studies covering 14 conversational agents. Two were randomized controlled trials; most of the rest were quasi experimental. Patient safety was rarely evaluated, and the authors concluded that evidence of efficacy and safety was still limited. It predates current language models, but its point has aged well: the field has more demos than trials.

The study people quote on the other side is Ayers and colleagues in JAMA Internal Medicine in 2023. They took 195 patient questions posted to a public online forum and compared physician answers with answers from a chatbot. Licensed health care professionals, blinded to the source, preferred the chatbot response in 78.6 percent of 585 evaluations, and rated it higher on both quality and empathy. The limitations matter as much as the headline: the questions came from a public forum, the chatbot had no medical record, its answers were much longer, and the evaluators were also coauthors.

Physicians are adopting the drafting use carefully. In the AMA's 2026 survey of nearly 1,700 physicians, 19 percent said they use AI to draft replies to patient portal messages, as a first draft the clinician edits and sends.

My summary: language quality is no longer the bottleneck. Grounding the conversation in real practice data, and stopping at the edge of clinical judgment, is.

Why does what happens after the conversation matter more?

Because a patient does not call to have a conversation. They call to get something done. A pleasant exchange that ends in a message for tomorrow is voicemail with better manners.

The useful question about any conversational AI is therefore what it can change. Can it write the appointment into the EHR, or only suggest a time? Can it file a refill request as a task, or only promise to pass it on? Can it transfer with a summary? The products that answer yes are doing more than conversation. That shift, from answering to completing, is what people now call agentic AI.

How should a practice evaluate conversational AI?

Run these tests yourself, on a live demo line, before you look at any slide.

  1. Describe a symptom. Time how long it takes to route you to a person or tell you to call 911
  2. Change your mind halfway. Book Tuesday, then say "actually, make it Thursday afternoon"
  3. Give it a name and date of birth that will not match cleanly, and see whether it guesses or hands off
  4. Ask it something it cannot know, such as a provider's personal schedule next month, and see whether it admits that
  5. After it says it booked something, open the real system and check that the appointment is there
  6. Ask the vendor in writing whether patient conversations are ever used to train models, and how calls are logged and reviewed

Also ask how patients are told they are talking to AI. Several states now have disclosure rules.

Where MedPhone fits, and what it does not do

MedPhone is conversational AI on one channel, the phone. It answers a practice's calls around the clock, every caller at once, books, reschedules and cancels appointments in the EHR, takes new patient intake and insurance details, captures refill requests as tasks in the provider's clinical inbox for the clinician to approve, sends text confirmations, and transfers to staff by rules the practice sets for business hours and after hours. athenahealth is the live integration, and MedPhone is a verified athenahealth Marketplace Partner.

The one production deployment I can name is Synergy Medical, a family medicine practice in Michigan on athenahealth, live since June 2026. MedPhone handled 2,848 inbound calls there between June and mid August 2026. Containment, the share of calls finished without staff, started around 53 percent on day one and improved as call flows were tuned. That is one practice, and I would rather you weigh it as one practice.

What it does not do: it gives no clinical advice and does no triage. Symptoms and clinical questions go to staff, and callers describing emergency symptoms are told to call 911. It works in English only. It does not run outbound calling campaigns, and it is not a website chatbot. If your problem is the portal inbox or online chat, it is the wrong product.

See how MedPhone handles a call

Questions people ask

What is conversational AI in healthcare?

Software that holds a natural, multi turn conversation with a patient or clinician by text or voice and keeps track of context across turns. In healthcare it is used for chatbots, symptom checkers, drafting replies to patient messages and AI phone agents that answer a practice's calls.

What are examples of conversational AI in healthcare?

A portal chatbot that answers questions about hours and insurance, a tool that drafts a reply to a patient message for a clinician to edit, and an AI phone agent that books, reschedules or cancels an appointment in the EHR during the call. Symptom checkers are also conversational, and they are the examples that need the most caution.

Is conversational AI the same as a chatbot?

A chatbot is one kind of conversational AI, usually text based. The term also covers voice systems on the phone. The more important difference is between systems that only answer and systems that complete a task in a real system, such as writing an appointment into the schedule.

Can conversational AI give medical advice?

It should not, unless a clinician is reviewing what it says. In a 2018 study of consumer voice assistants, 29 percent of the actions people planned after asking medical questions could have caused some harm. Administrative conversational AI should route clinical questions to staff and give the 911 instruction for emergencies.

Do patients accept talking to conversational AI?

In my experience most callers who want a routine reschedule care more about getting it done than about who does it, as long as the system says it is AI and offers a fast route to a person.

Is conversational AI in healthcare HIPAA compliant?

The technology is neither compliant nor non compliant; the vendor's handling of patient data is what counts. Ask how data is encrypted at rest and in transit, whether conversations train models, and how calls are logged. MedPhone is HIPAA Compliant, encrypts PHI with AES 256 at rest and TLS 1.3 in transit, and never uses it to train models.

Written by

Agni Patel is the founder and CEO of MedPhone, an AI phone agent for medical practices. He builds one kind of conversational AI and reads the transcripts where it goes wrong.

Want this run on your practice?

Bring fifteen minutes and your call reports. We will use your call volume and your payer mix rather than the benchmarks in this post.

Book a 15 minute demo