A recent article in the Pittsburgh Post-Gazette outlined the various ways that artificial intelligence (AI) is improving health care in Pennsylvania. For example, AI software can serve as a "virtual scribe," listening to the doctor-patient conversation during an office visit and drafting a note, freeing the doctor to focus on the patient for 100% of the time. AI can "draft letters to health insurers on behalf of patients who need specialty medications, medical equipment or other care that's not standard in their insurance benefits," saving time for doctors and office staff. In the future, AI could respond to patient portal messages, triage phone calls, or even suggest diagnoses.
That all sounds great, but a lot of people thought that electronic health records would make clinicians' work easier when they were implemented, too, and we know how that worked out (or didn't). So what's the evidence that AI will actually deliver on its promise in health care?
A case study published in NEJM Catalyst described Kaiser Permanente (KP) Medical Group's implementation of AI scribes using smartphone microphones to document more than 300,000 patient encounters across all medical specialties:
The response from physicians who have used the ambient AI scribe service has been favorable; they cite the technology’s capability to facilitate more personal, meaningful, and effective patient interactions and to reduce the burden of after-hours clerical work. In addition, early assessments of patient feedback have been positive, with some describing improved interaction with their physicians. Early evaluation metrics, based on an existing tool that evaluates the quality of human-generated scribe notes, find that ambient AI use produces high-quality clinical documentation for physicians’ editing. Further statistical analyses after AI scribe implementation also find that usage is linked with reduced time spent in documentation and in the EHR.
How about the electronic inbox and the increasing burden of responding to patient portal messages? One approach to streamlining this workload is making sure that requests are routed to the right person in the practice, often front office staff or nurses rather than physicians. A
research letter in
JAMA Network Open illustrated the content of nearly 5 million electronic messages from patients received by KP Northern California between April and August 2023 and classified using real-time natural language processing. In a
pilot quality improvement study in primary care and gastroenterology practices at Stanford Health Care, responses to messages were drafted by a large language model (LLM), and clinicians (physicians, advanced practice providers, nurses, and clinical pharmacists) were surveyed pre- and post-program implementation. Although the LLM drafted responses to 75% of messages, the average clinician used the draft only 20% of the time, with primary care clinical pharmacists using them the most (44%). There was no change in the amount of time clinicians spent managing their inboxes. However, task load and work exhaustion scores declined in the post-survey, and many clinicians appreciated that editing a draft required less effort than writing a response from scratch.
As a medical editor and author of hundreds of published papers, that last point makes sense - leading me finally to the use of AI outside of the clinic to draft scientific review articles. Currently, most journals either prohibit AI use or require authors to describe exactly how AI was used to develop a manuscript.
A recent study compared papers on 3 topics related to bone health (Alzheimer's disease, fracture healing regulation, and effects if COVID-19) that were written by 1) a human only; 2) ChatGPT only; and 3) a human and ChatGPT working together ("AI-assisted"). Unsurprisingly, the most accurate papers that required the least amount of time to write were AI-assisted (human-supervised?) where the AI was given not only a prompt but an outline and references. I'm still waiting to see the first
American Family Physician submission where the authors were assisted by AI. It's only a matter of time - unless, of course, it's already happened and I just didn't realize it.
