Reliance of artificial intelligence (AI) has been increasing across all fields, including in the clinic. How to ensure AI is integrated practically, effectively, and ethically has been the focus of many discussions and debates, however final consensus almost always lands on the idea that however AI is integrated into clinical use, conscientious human oversight is necessary.
To test the veracity of this goal, a team of researchers in Spain put over 200 medical doctors to the test to determine if the clinicians would trust their experience and training over AI recommendations.
“It is important to investigate the errors that humans (including doctors) make when working with algorithms, in order to learn how to minimize the problems that arise from them,” said co-author of the study, Fernando Blanco, PhD, researcher at the Mind, Brain and Behavior Research Center (CIMCYC) in Granada, Spain.
“We wanted to see if professional physicians would act differently so that they would notice and correct these errors,” the authors wrote in their paper published in PLOS Digital Health.
The researchers created treatment plan options for a series of fictitious patients with a rare disease. They asked 223 physician participants whether or not to provide a treatment to a patient based on whether the patient was classified by AI as being highly or lowly sensitive to the treatment plan. The physicians were then presented with patient recovery data and asked to rate their perception on how reliable the AI classification was.
In these experiments, both groups of patients responded to treatment with similar sensitivity, resulting in ineffective AI recommendations that could be identified using the patient recovery data.
“In the first experiment, the treatment worked moderately (and equally) well for both groups. In the second, the treatment did not work at all for either group,” the authors wrote. They expected that “in both experiments, participants would administer the treatment less often to the fictitious patients classified as lowly sensitive to the treatment, consistently with the AI classification.” However, this was not the case.
“In both experiments, physicians mostly trusted the AI’s classifications and had trouble learning from the feedback,” said lead author Aranzazu Vinas, PhD, University of the Basque Country, Spain. “Furthermore, in the second experiment, professionals did not notice that the treatment was completely ineffective.”
These results present a major concern for the medical community in its use of AI in the clinic. While AI can be highly effective and useful for data collection and summary, supervision and critical thinking are still required for effective and safe patient care. This study highlights the need for physicians to take the time to critically consider all available data, regardless of recommendations by AI in their diagnostic and treatment decisions.
“People tend to say that there is always a human controlling the algorithm,” opined Helena Matute, PhD, professor, University of Deusto, Spain on the team’s findings “but our experiments show that doctors (as well as anyone else) have problems in learning from the available evidence when it contradicts the suggestions of an algorithm.”
The post Clinicians Trust Faulty AI Recommendations Over Experience appeared first on Inside Precision Medicine.