The desire to integrate AI into everyday practice is shared among almost all healthcare specialties, with radiology being no exception.

But choosing which AI solution to purchase requires careful consideration of various factors, from what problems a department faces to the performance and safety of the solution itself, say researchers at the University of California, San Francisco and MedStar Georgetown University Hospital.

“Purchasing an AI solution simply because AI is a hot topic, or because one generally thinks AI will improve the practice of radiology, is not recommended because it may consume resources without benefit to clinical practice,” said the team of scientists in a statement. “If a tangible problem is identified and AI tools targeting that problem are commercially available, then one can begin to evaluate whether purchasing an AI tool makes sense.”

The scientists in question are Dr. Marc Kohli, associate professor and associate chair of clinical informatics, and Dr. John Mongan, associate professor and associate chair for translational informatics in the UC San Francisco department of radiology and biomedical imaging. The two worked with Dr. Ross Filice, chief of imaging informatics and body imaging at MedStar Georgetown University Hospital to determine how to effectively choose the right AI solution for a radiology department.

From their research, the three compiled a list of steps that radiologists should follow when trying to choose the right tool, starting with performance evaluation. They include:

• AI implementations should address a well-defined problem in the radiology practice
• Ease of use and workflow integration quality should be assessed before and after implementation
• AI models should be monitored for patient safety, including unintended bias, and especially the potential for reinforcing healthcare disparities
• Impact on IT infrastructure and cost should be included in return-on-investment calculations.

They also encourage radiology departments to create and follow a checklist of tips and factors to consider in order to build their case for why an AI solution is needed. Their own includes:

• Written description of the problem to be solved
• How will performance of AI products be measured and compared
• Involve local IT prior to purchasing
• Will validation or fine tuning on local data be performed
• Will the system require ongoing monitoring
• Calculate a total-cost-of-ownership (including AI vendor charges, IT, validation, monitoring, etc.) and estimate benefits to measure return-on-investment
• What are the quality and safety impacts of AI, and are healthcare disparities reinforced

“Purchasing AI systems requires close coordination with many stakeholder groups, and consideration of system performance, validation, IT requirements, cost, as well as quality and safety,” they said.

The findings were published in the Journal of the American College of Radiology.