It’s important for companies introducing artificial intelligence (AI) applications in healthcare to explain to providers what the algorithms are doing and how it will help them work smarter and faster.

That was just one of many points made during the panel discussion, “How Artificial Intelligence is Moving the Needle in Medtech”, at MD&M East, a medical technology trade show in New York City.

Comprising tech companies and healthcare network executives, the panel focused their attention on the big picture of AI in the medical device industry, discussing topics such as the importance of conveying the ins and outs of how an application functions.

“When we make a prediction, we show exactly how the computer arrived at a particular conclusion," Nathan Silberman, the head of deep learning at Butterfly iQ, told the group before him at the Javits Center. "We call that provenance. Let's say we wanted to measure somebody’s infection fraction. It’s important for us to illustrate exactly how we’re coming to that conclusion, that the computer is measuring some anatomical feature of the heart to show how we arrived at the conclusion and how we go from there to a final recommendation. Showing that provenance is critical to compelling clinicians that you’re really working on something that is going to help them do their jobs better."

The panelists also addressed patient privacy concerns related to AI, particularly with the General Data Protection Regulation (GDPR) that took effect last month in Europe and regulates how companies can use people’s personal information.

“A small company doesn’t have the luxury of having different privacy regimes for different territories,” said Robert Kaul, president and chief executive officer of Cloud DX. “So we’re really leaning toward being 'best in class' by adhering to the very highest standards. GDPR includes the right to be forgotten, but on the other side, if a medical organization gets audited on the data that’s in their clinical trials, if a patient that was in your clinical trial demanded to be forgotten, what happens to your trial? There are complex issues that haven’t been worked out yet.”

He added that such concerns are especially pressing in Europe, where companies are required to declare their plans for how they will apply data. While these implications of the law on healthcare companies are not fully understood yet, penalties for any misuse are "enormous."

The group also addressed the age-old topic of AI replacing doctors, saying that while algorithms can identify patients in need for follow-ups, the constant checking and reworking of AI tools, as well as the need for clinicians to reach out to patients, signify that their presence is crucial and cannot be replaced by a machine.

“It’s a combination of factors,” Shameer Khader, director of data science and bioinformatics programs at Northwell Health, said. “It’s not just that we have AI as a magic wand.”

Kaul added that clinicians should focus on employing AI to assist in identifying higher-level issues instead of the tasks they are overqualified to perform.

“When we think about how this is all going to play out 20 years from now, when we go to the doctor there’s a combination of people and technology diagnosing and treating us,” he said. “That’s got to be the vision, because there’s 60,000 people turning 65 every single day in this country and there ain’t no money to treat those folks when they get there, because we can’t build 60,000 nursing home beds every day.”

Health IT Homepage