Artificial intelligence in radiology: Hype or hope?

There is much hype in the discussion surrounding the use of artificial intelligence (AI) in radiology. Just walking through the RSNA 2017 Machine Learning Pavilion, one couldn’t help but wonder if all the noise pointed to CAD on steroids or to technology that is so far out there it belongs in the next Star Wars movie.

But the reality is, there are some real nuggets of hope in the gold mine.

Image-analysis software could be viewed as the next-generation of CAD. FDA-cleared image-analysis applications using AI are currently being employed in the healthcare marketplace; these include applications that are: quantitating information from cardiac images (Arterys, San Francisco, CA); automating image registration (Blackford Analysis, Edinburgh, UK); segmenting and quantifying brain MRI scans (Quantib B.V, Rotterdam, The Netherlands); automating analysis of, reviewing and interchanging CT scans (RadLogics, Milpitas, CA); suppressing bone and vessels in the lung, highlighting lines and tubes, and detecting lung nodules (Riverain Technologies, Miamisburg, OH); and quantifying CT scans (Zebra Medical Vision, Kibbuitz Shefayim, Israel).

“We are already seeing the potential [of AI] to make an impact in the day-to-day workflow of the radiologist,” says Lawrence Tanenbaum, MD, FACR, Director of MRI CT and Advanced Imaging and Vice President and Medical Director Eastern Operations, RadNet, Inc. (Los Angeles, CA). “I don’t see AI as competitive but rather as an augmentation and expansion of the radiologist’s capabilities.”

Dr. Tanenbaum points out that natural language processing is already being used to leverage information in a patient’s electronic medical record to facilitate billing appropriateness as well as a more patient-focused diagnosis. CT and MRI systems are utilizing machine-based algorithms for more intelligent exams—helping reduce CT and PET dose and faster MR exam times. He says scan quality and efficiency can be enhanced with AI algorithm-based positioning, and clever image reconstructions can recognize and reduce motion.

Rasu Shrestha, MD, MBA, Chief Innovation Officer, University of Pittsburgh Medical Center (UPMC) and Executive Vice President of UPMC Enterprises, says, “In the midst of all the hype there are emerging applications, trends and really promising results that form the basis of where we move past the hype to the substance and lead radiology moving forward as we decide how best to befriend AI.”

Dr. Shrestha believes that AI could help deal with the massive amounts of data that radiologists are dealing with—and not just images and reports. Beyond the EMR, he says, this includes image exchanges, analytics, data registries and data houses.

“To embrace AI in radiology we need a mind shift change,” Dr. Shrestha says. “The first step is to look at the big picture of data elements around the patient. What we need is a holistic view of the patient, not just the reason for the study or prior reports. Much of that information resides outside radiology departments and the systems we use. So we can bring the capabilities of AI to mine that information and look at lab results, procedural reports, or waveforms—all the data elements that bring that patient’s story to life.”

AI isn’t the first technology to be hyped, says Paul Chang, MD, FSIIM, Professor and Vice-Chairman, Radiology Informatics at the University of Chicago School of Medicine. Dr. Chang’s heard it before, when departments transitioned to PACS or when speech recognition was introduced.

“Like all other hype cycles, we sorely underestimate how long it takes to appropriately consume technology,” Dr. Chang says. “Yes, it eventually changes the way we practice, but the real impact is usually only evident in retrospect.”

Dr. Chang point out that deep learning and AI are already prevalent in our lives outside of the hospital, such as using Siri and Alexa, or even when a credit card is denied when traveling due to potential fraud. Health care, he says, is 10 to 15 years behind other industries in the adoption of advanced IT technology.

“It is critically important for radiology to stay engaged and prepared for AI,” Dr. Chang says. “And one way to prepare is to improve the infrastructure we need to consume these advanced technologies, such as AI, big data and analytics. The lack of a strategic, interoperable and agile enterprise IT infrastructure perspective is a real challenge.”

Information systems, such as PACS, remain data siloes. Dr. Chang believes institutions must think beyond an EMR- or PACS-centric view of information infrastructure. “Get more strategic and prepared so radiology can consume AI, decision-support tools, Big Data, or any data-intensive application.”

Dr. Chang also believes that many data-driven support tools will likely reside in the cloud. There will also be a significant need to access highly interoperable, reliable and high-quality data, which are very rare today. That’s all the more reason why the best strategy for institutions is to invest in infrastructure today.

“While it is probably too early to pick an AI winner, improving your health enterprise is a great ‘hedge’ strategy for AI,” Dr. Chang adds.

Where will AI impact radiology?

At present, the promise of AI in radiology appears focused on image analysis and diagnosis. Yet, the real benefit of AI lies in how other industries have used it, Dr Chang says, “to improve workflow efficiency and quality and reduce errors and variability …. That’s what moves the needle.”

Artificial intelligence and machine learning will also be used to develop more clever algorithms that make CAD more intelligent. To accomplish this, companies need high-quality data in to generate high-quality data out with pathological proof. A data learning architecture can augment radiology to improve results, but the architecture should be one that can be lent to different applications across imaging, such as lung and mammography screening.

“What’s making this explosion in AI and machine learning possible is the availability of enhanced computing and gaming processes that can handle the data,” says Dr. Tanenbaum. “We now have the image processing power but need both high quality data and the ground truth for vetting of that data.” He adds that AI companies will need teams of radiologists to validate the output of machine learning algorithms. Natural language processing will be essential in mining the medical record for ground truth and outcomes.

Dr. Shrestha says an emerging bright spot for AI algorithms is to add value across the imaging chain, from helping to prevent referring physicians from placing orders for unnecessary studies to intelligent protocoling at the imaging devices. AI can even be embedded into imaging modalities so that technologists can efficiently obtain the right scans in the right manner to answer the clinical question. Or, radiologists can use AI to communicate relevant value and intelligence with referrers or specialists around the larger context.

“Then, downstream we need to quantify the value that results from all these interactions,” Dr. Shrestha says. “These are areas where AI can really improve workflow.”

He also sees promise in automated quantification and segmentation applications. Digital spreadsheets didn’t lead to accountants losing their jobs. It helped them function at a higher level. The same can be said for radiologists and AI.

“Radiologists’ roles will change, and we shouldn’t be mere diagnosticians, but physician consultants,” says Dr. Shrestha, who argues that AI enables the “intelligent presentation” of patient data across enterprises and presents an opportunity for computers to take over mundane data tasks that human radiologists cannot efficiently perform, such as data scaling or managing multiple elements of data.

“We need to take the artificial away from our embrace of the technology,” he says. “The term ‘AI’ should stand for ‘augmented intelligence.’”

The “intelligence” inside

From its history as a gaming and computing company, NVIDIA (Santa Clara, CA) launched CUDA, a parallel computing platform and programming model that developers can utilize to speed up computing applications by harnessing the power of GPUs. Over the last decade, the company has been working with medical equipment manufacturers to harness the power of CUDA within devices and software.

At RSNA 2017, NVIDIA announced two partnerships. One renews its 10-year partnership with GE Healthcare to bring AI into GE’s imaging devices. This includes introducing the NVIDIA-powered Revolution Frontier CT, reportedly two times faster in imaging processing than its predecessor, and the Vivid E95 4D ultrasound for enabling fast, accurate visualization and quantification while streamlining workflows. The second partnership with Nuance entails NVIDIA’s deep learning platform to power the training and publishing of applications to the Nuance AI Marketplace and deploying it into medical imaging workflow.

“With all this data and the computing architecture, we can use deep learning algorithms to learn important features inside of images,” explains Kimberly Powell, vice president of healthcare at NVIDIA. By embedding AI into the medical imaging device, it becomes more intelligent, similar to a self-driving car, she explains.

The partnership with Nuance is focused on seamlessly integrating AI into workflow—helping clinicians become more efficient by prioritizing studies based on normal or abnormal features, for example. Unveiled at RSNA 2017, the Nuance AI Marketplace is “the first open platform for developers, data scientists and radiologists to accelerate the development, deployment, and adoption of artificial intelligence (AI) for medical imaging,” according to a company statement.

“Companies are relying on our technology to build and deploy AI applications,” Powell adds. “We don’t build the application, we power it. And to efficiently deploy these applications, the researchers and developers need the most advanced AI computing platform.”

Powell estimates that of the companies in the RSNA 2017 Machine Learning Showcase, nearly all were utilizing the power of NVIDIA’s GPU technology.

“Radiology has been a trailblazer in technology as early adopters, and (I think) will be quick to embrace AI,” she says. One area where Powell sees potential is in ultrasound imaging, where the user (a sonographer or clinician) scans, visualizes and quantifies imaging data in real time. Yet it is also prone to variability as ultrasound is subjective to the skills and knowledge of the user. She also believes there is potential for AI to help radiologists review and read abnormal studies first—essentially stacking and prioritizing studies based on the patient’s status and condition, as well as the possible identification of abnormal tissue, organs or bones using AI. Then, there are advanced imaging modalities, such as CT and MRI,that generate enormous amounts of data. Powell believes it is possible to use AI to extract the most relevant information from the voluminous data and remove some of the added strain on radiologists to review all of that data.

As part of these efforts, NVIDIA is working with Massachusetts General Hospital (Boston, MA) as a founding partner of the MGH & BWH Clinical Data Science Center. The center is using NVIDIA’s DGX-1’s, a supercomputer designed for AI applications that will be used to process the hospital’s vast archive of phenotype, genetic and imaging data to develop algorithms that may aid in a radiologist’s interpretations.

Emerging AI technologies

Image analysis

CureMetrix (La Jolla, CA) is intent on developing a computer aided detection (CAD) that works. Kevin Harris, CEO and Board Director, says, “CAD solutions on the market today haven’t lived up to their promise. A key challenge that remains is the number of false positives, which leads to an increase in recall rates.”

To address this need, Harris says the company has focused its development on improving sensitivity and specificity, using AI to help radiologists detect anomalies and understand what is detected. First, the company is tackling screening mammography because of the opportunities to improve clinical outcomes and clinical efficiency.

Over 500,000 images from leading institutions across the country are being used to train and validate the company’s algorithm so it can recognize and quantify regions of interest. So far, Harris says the investigational software has maintained mid-90s sensitivity with extremely low false-positive rates. In 2018, he anticipates the company will collect another 20 million images from its clinical partners.

“The real challenge in developing AI algorithms is the data curation—how well it is collected and managed,” Harris says.

cmAssist, the company’s investigational CAD solution, is designed to assist radiologists by combining AI with “real” intelligence. “There are opportunities to relegate mundane tasks to AI to optimize workflow and help doctors become more efficient,” Harris says. He uses the analogy of in-home pregnancy testing: the test didn’t replace the doctor, it simply changed the paradigm.

“I think AI has the potential to do something similar… an opportunity to change the paradigm in breast cancer screening. Statistics indicate that 5 in 1,000 patients undergoing screening mammography have cancer. Reimbursement is not going up but the volume is. To that extent, if we can help doctors be more efficient and increase their confidence that creates a tremendous benefit for both patients and doctors,” he adds.

According to Harris, CureMetrix chose screening mammography as a first target because it’s a global challenge that needs an innovative solution. Radiology has been waiting for and needs a CAD that works and Harris believes that they can provide that to physicians.

A core challenge, he adds, is findings ways to help radiologists trust artificial intelligence. Through internal research CureMetrix has demonstrated that the more time a radiologist spends with AI-based solutions, the more they feel comfortable relying on the results delivered by the software.

Deep learning

Enlitic (San Francisco, CA) is developing a deep learning tool for radiologists that augments their reading and interpretation. Last May, the company won the top prize of €1 million at the first Cube Tech Fair in Berlin, Germany.

“We are developing AI with the goal to cover 95% of diagnostic radiology by 2020,” says Kevin Lyman, COO at Enlitic. The company’s focus, he explains, is to enhance radiologists’ efficiency, proficiency, and more importantly, accuracy.

There are three different ways that Enlitic sees the potential for deep learning AI in radiology. One is to perform quality assessments, or a second read, after initial interpretation on the images and the report. Two, to triage incoming studies in order to prioritize and appropriately route through the organization. Three, to deliver real-time diagnostic support alongside a radiologist.

Working with Capitol Health Ltd. in Melbourne, Australia, Enlitic deployed a wrist fracture detection system using in-house deep learning models to circle fractures in X-rays, displaying these annotations in the PACS viewer for reading by radiologists. The study measured the accuracy and efficiency of three specialist radiologists, each reading a total of 400 studies, with and without assistance from Enlitic models. The study found that radiologists augmented by Enlitic were 21% faster, 11% more sensitive and 9% more specific in their reading.

The company has also trained a deep learning model to detect visual patterns in a chest X-ray for a differential diagnosis. In a blind test deployed in several countries, the model was used as a triage tool to separate normal from abnormal exams and demonstrated a 16% increase in AUC (areas under the ROC curve) compared to human (radiologist) interpretation.

“Interestingly, we learned that in different parts of the world, there are different ways to use these models,” Lyman explains. “I often talk about how we aren’t seeking to replace radiologists. One of our goals is to help bring radiology to regions in the world that don’t have enough radiologists,” he adds.

Take China, for example. Lyman says that although the country has five times the population of the US, it has one-third fewer radiologists. He shares that a single network of clinics in China can perform up to 5 million chest X-rays a year, and would perform even more if they had enough radiologists.

In another ongoing clinical study that Lyman hopes will be accepted for publication in late 2018, Enlitic’s AI model is capable of finding malignancies in low-dose chest CT lung cancer screening studies up to two years earlier than radiologists. In Lyman’s opinion, this level of accuracy and foresight is one of the most interesting applications of clinical AI.

The development of these algorithms at Enlitic is a mix of deep learning and other AI tools, Lyman explains. “If we take a complex process, it needs to be a series of algorithms that together better replicate the system as a whole. First, we focus on the quality of the data for building the algorithms, then determine if it actually answers the clinical question. However, we must also consider the user experience. These are tools designed by radiologists for radiologists, not to just use them but to trust them. That idea of trust needs to be packed into our product development.”