OTTAWA — During a rain-spoiled Florida vacation at the end of 2023, Dr. Braedon Hendy began hacking together an AI solution to the medical paperwork that tries daily to drown him.

As a practitioner outside a major population centre, Hendy needs a rack for all his hats. He’s the chief of the emergency department at Erie Shores HealthCare, a 58-bed community hospital an hour southeast of Windsor, Ont.; a medical officer in the Canadian Forces; and a part-time coroner. But primarily, he is a family physician in a three-doctor practice and walk-in clinic in Belle River, another small town in Windsor’s orbit.

The paperwork pileup in an ordinary medical practice is unbelievable, said Hendy. Doctors, nurse-practitioners and other health workers are expected to make meticulous records of their dealings with patients. Often called “SOAP notes”—for “subjective, objective, assessment and plan”—those documents lay out what patients report themselves, what the practitioner observes, potential diagnoses and what’s supposed to happen next.

Especially in family medicine, “you often are dealing with multiple issues at once, which brings complexity to the creation of a note, and this will lead to notes being incomplete after the visit is done,” Hendy said. While shuttling from patient to patient, doctors keep enormous amounts of information in their heads that they then type into the computerized medical files later.

“Often there are two patients waiting while you’re in the room with a patient moving through their issues, and it’s a fine balance between appropriate eye contact, conversational skills and inputting the data to help save time as you move toward the end of the appointment,” Hendy said in an interview.

If he gets his half-hour for lunch, Hendy will use it to touch up notes from five or six morning appointments, make calls and answer email.

“In the afternoon you have the same process, and now you’re bringing that work home.”

So it’s no surprise that artificially intelligent medical scribes, which make transcripts of patient appointments and use large language models (LLMs) to turn them into formal records, are taking medicine by storm.

The LLMs, which burst out with the arrival of ChatGPT on the public stage in late 2022, are key. Having been trained on numerous examples, an LLM can pluck out the words in a patient interaction that are likeliest to be important (“splitting headache,” for instance) and what words around them are significant by association (“since Tuesday,” “Tylenol didn’t help” and so on). 

That helps draft a summary note and fill in electronic forms, to take some of the load off doctors like Hendy. He still has to read and tweak the results, but doesn’t have to create the whole thing from scratch.

Hendy’s own tool, simply called “AI-Scribe,” uses open-source software components for the job. He started by hacking the pieces together during that rained-out Florida trip last year. Regular updates have addressed particular needs—like a palliative-care physician’s request to make recordings during house calls and upload them in a batch at the end of the day, or pediatricians’ need for particular categories of patient information that other doctors don’t have.

At the top end of the market are sophisticated packages backed by giants like Microsoft; they have any number of startup competitors.

Ontario is trying several tools in a pilot project in about 150 family practices. Separately, a group of Ontario hospitals is planning an experiment of its own. The federal government’s digital-technology supercluster is funding a project to promote digitalization in health care, with trials in Well Health’s clinics from B.C. to Quebec; it includes a scribe system from Toronto’s Tali.ai.

Yet the technology is very new and regulators are uneasy. Alberta’s College of Physicians and Surgeons distributed a note to its members last year that’s essentially a list of warnings about AI scribes and advised Alberta doctors to use the tools “with great caution.”

At their best, scribing tools can slash hours of charting time off a medical professional’s day, freeing up time to see more patients and to work a little less. But because they rely on large language models from the likes of OpenAI, many of them also put deeply private patient information in the cloud—in some cases on U.S. servers.

Patient permission for the use of these tools is key—consent provides exceptions to most privacy laws that would otherwise apply, whether they’re general or specific to health information. But it can be hard for patients to understand exactly what they’re consenting to, when it’s a commercial product with multiple components, Hendy said.

That’s one reason Hendy decided to develop his own tool rather than using one of the newly available cloud-based products. The data his system gathers doesn’t leave the clinic where it’s used.

“I have discussed this with patients and they feel much more comfortable that they know that everything stays within the office. It’s a safe setting,” Hendy said.

It’s a delicate problem, said Katherine Tudor, a spokesperson for the group running Ontario’s scribing experiment—a technology subsidiary of the Ontario Medical Association called OntarioMD.

“This is a key topic that needs to be explored,” she wrote in an email to The Logic. “There are likely benefits to vendors (and to the performance of the tools) if the data collected can be used to improve the large language models that power these tools. Decisions on these issues are not OntarioMD’s to make.”

Extreme paperwork is a perverse outcome of digitalizing health care, said Mohammed Loubani, co-founder of AI scribe startup Fluent. 

The designers of electronic medical records “forced physicians to structure a patient visit into a way that was readable for computers,” Loubani said. “To code it, to put it into buckets, into tiles, to use dropdowns—all of which are interfaces that do not work very well for a health care visit.”

Based in London Ont., Fluent offers a commercial tool, one of those OntarioMD is testing and the runaway winner of an informal scribe “face-off” the OMA ran last spring.

A former ER doctor, Loubani said he and his co-founders came to scribing technology from a business in electronic medical records, after finding out users of their software—doctors and other front-line practitioners—were spending a third of their time inputting data.

He remembers treating a new mother experiencing stress and depression. She needed his undivided attention—on purely medical grounds, not just humane ones, he said. “I knew I had to look at her, engage with her, observe her demeanour. That’s something that we do in a mental-status exam.”

Even if you’re typing notes as you go, sometimes a patient will say something that triggers a whole workflow of clicks and menu choices.

“The system is creating this ‘happy path’ of being a bad doctor. The system is designed to make you turn away from the patient,” Loubani said.

The burden is a major burnout factor for family physicians, Hendy said; a national survey by the Canadian Medical Association in 2021 found administrative work took up 10 hours a week of respondents’ time and the group treats it as an important advocacy problem.

“Many of my young colleagues have simply left family medicine, and statistically, once they leave, it’s very unlikely that they return,” Hendy said.

---

Conceptually, an AI-driven medical scribe is not very complicated. A microphone records the doctor-patient conversation; voice-transcription software turns the audio into text; a large language model (ideally with some heavy training on medical terms) summarizes it; the software organizes the summary into a patient-note template.

“This is like the early tests that they did where they would feed Moby-Dick [to an LLM] and say, ‘Summarize this novel,’ or, ‘Write me a short story in the same style as this novel,’” Loubani said. Fluent uses OpenAI to create its summaries and says so in its privacy policy.

Fluent’s sample consent form for patients doesn’t name OpenAI, but does say this in plain language: “The transcript of our visit will be processed by a large language model (LLM) service provider on servers located in the United States … Any of your data stored on servers in the United States would be subject to the laws of the United States and can be kept there for up to 30 days maximum.”

OpenAI does not get permission to use Fluent’s patient information to refine its models, Loubani said, an exception for which his company pays extra.

Just the fact the data crosses the border into the U.S. is a problem, according to Tali, the startup involved in the supercluster-funded trial. It competes with Fluent and is another participant in the OntarioMD trials. If data is subject to U.S. law, Tali co-founder and chief technology officer Hesam Dadafarin wrote in an email, that means the U.S. government can get it.

“We have taken a very defensive approach to protect health data,” he wrote. Tali processes Canadians’ data in Canada, and patient data isn’t used to train any models.

Even that reassurance might soon be in doubt, though. As Canada negotiates a data-exchange agreement with the United States under the terms of the U.S. Clarifying Lawful Overseas Use of Data (CLOUD) Act, the promise that Canadian data will be processed here could offer less protection than it does now. The U.S. law gives American authorities more power to seize foreign customers’ data from U.S. companies, even if it’s stored outside the United States.

Tali will not say whose model or models it uses. Its terms of use and its privacy policy say it has a “hosting contractor” and “AI service providers,” but not which ones.

Dadafarin would shed no light. “This is our IP. We use a variety of open-source and closed-source LLMs to deliver value to clinicians,” he wrote.

Another Canadian AI scribe startup, Edmonton’s ScribeBerry, pledges that it uses Microsoft Azure cloud servers only in Canada for Canadian data, and models from Anthropic for the LLM work.

Hendy’s solution keeps the data in the building, but it requires a user savvy enough to install things from GitHub.

“I’ve had a lot of physicians successfully set it up,” he said, but conceded that the technical demands are a barrier. He’s hoping to streamline the installation, and maybe offer an inexpensive computer with the necessary stuff pre-installed.

For now, “it takes a little bit of work. It takes steps. It’s not easy. I’m only one physician that did this as a passion project.”

The technology is very new, and discussion about its implications even newer. Hendy first heard about AI scribing in 2023. Fluent is barely a year old; ScribeBerry a few months older. The OntarioMD-led pilot project—the trial wrapped at the end of July, with findings to follow—is meant to inform decisions about privacy and consent, in addition to learning about the practicalities of the software.

So far, getting patients’ consent has rarely been difficult, wrote Tudor, OntarioMD’s spokesperson.

“A path forward will ideally be multi-modal, and allow patients who want more information to get transparent and easy access to the guardrails in place to protect their data,” she wrote. “This should be coordinated so that the burden is not placed on the individual clinician to explain how AI works.”