In the past few years, studies have described the many ways A.I. tools have made doctors better at their jobs: It’s aided them in spotting cancer, allowed them to make diagnoses faster and in some cases, helped them more accurately predict who’s at risk of complications.
But new research suggests that collaborating with A.I. may have a hidden cost.
A study published in the Lancet Gastroenterology and Hepatology found that after just three months of using an A.I. tool designed to help spot precancerous growths during colonoscopies, doctors were significantly worse at finding the growths on their own.
This is the first evidence that relying on A.I. tools might erode a doctor’s ability to perform fundamental skills without the technology, a phenomenon known as “deskilling.”
“This is a two-way process,” said Dr. Omer Ahmad, a gastroenterologist at University College Hospital London who published an editorial alongside the study. “We give A.I. inputs that affect its output, but it also it seems to affect our behavior as well.”
The study began like many A.I. trials in medicine. Doctors at four endoscopy centers in Poland were given access to an A.I. tool that flagged suspicious growths while they performed a colonoscopy, drawing a box around them in real time. Several other large clinical trials have shown this technology significantly improves doctors’ detection rate of precancerous growths, a widely accepted indicator of an endoscopist’s performance.
Then, unlike in past studies, the researchers measured what happened when the tool was taken away.
In the three months before the technology was introduced, the doctors spotted growths in about 28 percent of colonoscopies. Now, the detection rate had fallen to about 22 percent — well below their base line.
This was an observational study, which means it can’t answer whether the technology caused the decline in performance. There could be other explanations for the effect: For example, doctors performed about double the number of colonoscopies after the A.I. tool was introduced compared to beforehand, which might have meant they paid less attention to each scan.
But experts said the fact that there is a deskilling effect is hardly unexpected. This phenomenon is well-documented in other fields: Pilots, for instance, undergo special training to brush up on their skills in the age of autopilot.
“I think the big question is going to be: So what? Is that important?” said Dr. Robert Wachter, chair of the medicine department at the University of California, San Francisco, and author of “A Giant Leap: How AI Is Transforming Healthcare and What That Means for Our Future.”
On one hand, Dr. Wachter said, there are plenty of harmless examples of new technology making old skills obsolete. Thanks to the invention of the stethoscope, for example, many doctors would struggle to examine a patient’s heart and lungs without one, as was common in the 1700s.
But to Dr. Ahmad, A.I. is distinct in that it needs long-term oversight from humans. Algorithms are trained for a specific moment in time, and as the world changes around them, they perform differently — sometimes for the worse — and need monitoring and maintenance to make sure they still function as intended. Sometimes unexpected factors, like changes in overhead lighting, can make A.I. results “go completely wrong and haywire,” he said.
Doctors are supposed to be included in the process to protect patients against those possibilities.
“If I lose the skills, how am I going to spot the errors?” Dr. Ahmad asked.
Even if the tools were perfect, Wachter cautioned that deskilling could be dangerous for patients during the current transition period, when A.I. tools are not available in every health system and a doctor accustomed to using it might be asked by a new employer to function without it.
And while the erosion of skill is obvious to someone looking at data from thousands of procedures, Dr. Wachter said, he doubted that each individual doctor noticed a change in their own ability.
It’s still not entirely clear why a doctor’s skills might decline so quickly while using A.I. One small eye-tracking study found that while using the A.I., doctors tended to look less at the edges of the image, suggesting that some of the muscle memory involved in reviewing a scan was altered by using the tool.
Dr. Ahmad said it might also be the case that after months of relying on a helper, the cognitive stamina that’s required to carefully evaluate each scan had atrophied.
Either way, medical education experts and health care leaders are already considering how to combat the effect. Some health systems, like UC San Diego Health, have recently invested in simulation training, which may be used to help doctors practice procedures without A.I. to keep their skills sharp, said Dr. Chris Longhurst, chief clinical and innovation officer at the health system.
Dr. Adam Rodman, director of A.I. programs at Beth Israel Deaconess Medical Center in Boston, said some medical schools have also considered banning A.I. for students’ first years of training.
If just three months of using an A.I. tool could erode the skills of the experienced physicians included in the study (on average, the doctors had been practicing for about 27 years), what would happen to medical students and residents who are just starting to develop those skills?
“We’re increasingly calling it never-skilling,” Dr. Rodman said.
Teddy Rosenbluth is a Times reporter covering health news, with a special focus on medical misinformation.
The post Are A.I. Tools Making Doctors Worse at Their Jobs? appeared first on New York Times.
