A team led by researchers at Google AI in Mountain View, CA, has developed a deep-learning model that can predict lung malignancies. The model, a neural network trained with lung cancer CT scans, performs as well as, or better than, trained radiologists.
“This extensive deep neural net assessment represents a step forward for CT lung cancer screening in smokers, which has been plagued by very high rates of false positives and negatives,” notes Eric Topol, MD, executive vice president of Scripps Research, and founder and director, Scripps Research Translational Institute. He tells GEN that “the algorithms require prospective clinical validation, but are certainly promising.”
The collaborative work is published today in Nature Medicine in a paper titled, “End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography.”
Lung cancer is the most common cause of cancer death in the United States, resulting in an estimated 160,000 deaths in 2018. Since 2013, low-dose computed tomography (LDCT) lung cancer screening in high-risk populations has been the recommendation. This guideline is based on trials showing that LDCT can reduce mortality by 20–43% in high-risk patients. The guidelines set for image interpretation by radiologists are based on a variety of image findings, but primarily “nodule size, density, and growth,” note the authors. But, Lily Peng, MD, PhD, product manager at Google, notes that “very early stage cancer is miniscule and can be hard to see” adding that “over 80% of lung cancer cases are not caught early.”
“Several challenges remain” in the detection of lung cancer, says Shravya Shetty, senior staff software engineer at Google and senior author on the paper, “including high frequency of false positives and false negatives, inter-grader variability, and the operational hurdles to implementing widespread imaging screening programs.”
Google has been working on applying AI to medical images for several years and has announced promising results in detecting diabetic eye diseases and breast cancer metastasis from pathology slides. This particular research on lung cancer screening, Shetty told GEN, has been going on for over a year.
To develop their approach, the model was trained on 42,290 CT scan images, using both current and prior scans from patients. The researchers first developed a three-dimensional (3D) convolutional neural networks (CNN) model that performed end-to-end analysis of whole-CT volumes, using LDCT volumes with pathology-confirmed cancer as training data. Then, they focused on the region-of-interest (ROI.) The CNN cancer risk prediction model was developed from both the full-volume and the ROI models, taken together. The model predicted nodules with an accuracy of 94% in 6,716 test cases.
They then conducted two studies to compare the model’s performance to that of radiologists. Six U.S. board-certified radiologists with an average of eight years clinical experience (range 4–20 years) participated. When prior computed tomography imaging was not available, the model outperformed all six radiologists with absolute reductions of 11% in false positives and 5% in false negatives. Where prior computed tomography imaging was available, the model performance was comparable to the radiologists.
“We saw our AI models progressively improve over time so the final results were not a complete surprise,” notes Shetty. “But, it was great to see that the results were strong on our held-out test set and the reader study with six radiologists and that the results seemed to generalize to an independent dataset from Northwestern Medicine that the AI had not been exposed to before.”
Another exciting component of this technology is how early Google’s AI is able to detect the cancer in early scans. In one case, it detected a patient’s cancer on a scan taken one year before the patient was diagnosed. Peng said, “For patients like this, early detection could translate to an increased survival rate of 40%.”
This is an inherently difficult task, notes Shetty. “We are trying to directly predict the biopsy confirmed cancer outcome for a patient within a year and these CT scans are large 3D images and the cancerous region can sometimes be small and hard to detect.” And although they used a relatively large dataset of CT scans for training, there were only about 500 cancer positive cases.
The research was a collaboration between the researchers at Google AI, Stanford Health Care and Palo Alto Veterans Affairs, Northwestern Medicine, and New York University-Langone Medical Center, Center for Biological Imaging.
The authors note that this is promising early work which is still in the research phase and “that these findings need to be clinically validated in large patient populations.” The next steps, notes Shetty, include running studies to ensure the model continues to generalize to new care settings and populations.