Deep learning is a computational tool that is a subset of Artificial Intelligence (AI), and these technologies are often described as having vast potential to analyze data and find conclusions that humans may not be able to identify. One area where these technologies are already being applied is in medicine, and the analysis of medical images is thought to be a particularly promising application. Radiologists are trained to assess X-rays and diagnose fractures or other issues in individuals, but AI may be able to find important patterns after analyzing vast amounts of these images.
This is still a field in its infancy, however, and caution might be advised. A new study published in Scientific Reports has shown how easy it is for AI to come to unrealistic and erroneous conclusions. The researchers showed that AI decided that X-rays can illustrate whether a person has had a beer, for example. The work noted that AI is susceptible to 'shortcut learning,' which can produce very accurate but misleading results.
In this study, the investigators used AI models to assess over 25,000 knee X-rays. The models identified ridiculous patterns that led it to use X-rays to predict whether people had eaten beans or drank beer. AI did so by finding very subtle but accurate patterns that were not meant to be revealed.
"While AI has the potential to transform medical imaging, we must be cautious," noted senior study author Dr. Peter Schilling, an assistant professor at Dartmouth College, among other appointments. "These models can see patterns humans cannot, but not all patterns they identify are meaningful or reliable. It's crucial to recognize these risks to prevent misleading conclusions and ensure scientific integrity."
Algorithms that are often used in AI were studied, and they were found to rely on strange and confounding variables, such as changes in the X-ray equipment being used, or markers of clinical sites to predict things, instead of focusing on the medically important and relevant parts of the X-ray.
Any attempt to remove these biases toward odd patterns were not very effective because the AI would then move to reveal other confounding variables and find patterns with them.
"This goes beyond bias from clues of race or gender," said study co-author Brandon Hill, a machine learning scientist at Dartmouth Hitchcock. "We found the algorithm could even learn to predict the year an X-ray was taken. It's pernicious–when you prevent it from learning one of these elements, it will instead learn another it previously ignored. This danger can lead to some really dodgy claims, and researchers need to be aware of how readily this happens when using this technique."
Because of the incredible potential for harm, the study authors emphasized that medical research using AI should be subject to rigorous evaluation.
"The burden of proof just goes way up when it comes to using models for the discovery of new patterns in medicine. Part of the problem is our own bias. It is incredibly easy to fall into the trap of presuming that the model 'sees' the same way we do. In the end, it doesn't," Hill explained.
"AI is almost like dealing with an alien intelligence. You want to say the model is cheating, but that anthropomorphizes the technology. It learned a way to solve the task given to it, but not necessarily how a person would. It doesn't have logic or reasoning as we typically understand it."
Sources: Dartmouth College, Scientific Reports
