According to a study published in Radiology, a publication of the Radiological Society of North America (RSNA), radiologists surpassed AI in accurately diagnosing the presence and absence of three prevalent lung illnesses in a survey of more than 2,000 chest X-rays.
Lead researcher Louis L. Plesner, M.D., resident radiologist and Ph.D. fellow in the Department of Radiology at Herlev and Gentofte Hospital in Copenhagen, Denmark, said: “Chest radiography is a common diagnostic tool, but significant training and experience are required to interpret exams correctly.”
While commercially accessible and FDA-approved AI technologies are available to aid radiologists, deep-learning-based AI tools for radiological diagnosis are still in their infancy, according to Dr. Plesner.
Dr. Plesner stated that although AI tools are progressively being certified for use in radiological departments, there is still a need to further test them in actual clinical circumstances. While AI techniques can help radiologists analyse chest X-rays, their actual diagnostic efficacy is still unknown.
The performance of four commercially available AI tools and a pool of 72 radiologists in reading 2,040 consecutive adult chest X-rays obtained over a two-year period at four Danish hospitals in 2020 was compared by Dr. Plesner and his research team. The patient population’s average age was 72. 669 (32.8%) of the sample chest X-rays revealed at least one target.
The chest X-rays were examined for three common findings: pneumothorax (collapsed lung), pleural effusion (a buildup of water around the lungs), and airspace disease (a chest X-ray pattern, for example, produced by pneumonia or lung edoema). AI methods achieved sensitivity rates for airspace disease, pneumothorax, and pleural effusion that ranged from 72 to 91%, 63 to 90%, and 62 to 95%, respectively.
Chest X-rays (CXR) Made Easy!
“The AI tools showed moderate to a high sensitivity comparable to radiologists for detecting airspace disease, pneumothorax, and pleural effusion on chest X-rays,” he stated. However, compared to radiologists, they produced more false-positive results (which indicate the presence of disease when none actually exists), and they performed worse for smaller targets and when numerous findings were present.
Positive predictive values for pneumothorax, or the likelihood that a patient with a positive screening test actually has the condition, varied between 56 and 86% for the AI systems and 96% for radiologists.
“AI performed worst at identifying airspace disease, with positive predictive values ranging between 40 and 50%,” Dr. Plesner stated. “In this challenging sample of older patients, the AI correctly identified airspace disease five to six times out of ten. At that pace, an AI system cannot function independently. Dr. Plesner claims that radiologists must strike a balance between the ability to detect disease and the ability to rule it out, avoiding both major diseases that are missed and overdiagnosis.
In particular when the chest X-rays are complex, he added, “AI systems seem very good at finding disease, but they aren’t as good as radiologists at identifying the absence of disease.” “Too many false-positive diagnoses would result in unnecessary imaging, radiation exposure, and increased costs.”
The majority of research, according to Dr. Plesner, typically focus on evaluating whether AI can identify the presence or absence of a single ailment, which is a lot easier challenge than in real-world situations where patients frequently arrive with many disorders.
“In many prior studies claiming AI superiority over radiologists, the radiologists reviewed only the image without access to the patient’s clinical history and previous imaging studies,” the author claimed. In real-world situations, a radiologist will combine these three pieces of information to interpret an imaging test. We believe that if the next generation of AI tools were also capable of this synthesis, they might become substantially more potent, but no such systems exist as of yet.
“Our study demonstrates that radiologists generally outperform AI in real-life scenarios where there are a wide variety of patients,” he said. The statement “AI should not be autonomous for making diagnoses” is true even though AI systems are effective at identifying normal chest X-rays. Dr. Plesner emphasised that by providing a second look at chest X-rays, these AI techniques could increase radiologists’ confidence in their findings.
