Deep Learning for Screening Mammography

Current State of the Art

Deep learning has been applied to screening mammography, with a rapidly growing body of scientific literature that suggests the field is on the cusp of prospective, clinical use. These applications can broadly be categorized into the following overlapping groups:

■ Triage systems

■ Cancer and abnormality detection

■ Density and cancer risk assessment.

Triage

Around 99% of women who undergo screening mammography do not have malignant breast cancer. The vast majority of mammograms do not warrant any further investigation or change to the routine screening frequency

(2 - yearly in most countries) (Lehman et al., 2017). The following studies used deep learning to classify mammograms as effectively normal and cancer-free, and address the large total number and associated human workload of reviewing these mammograms. Importantly, any such triage system will have subsequent effects on the distribution of normal to abnormal images in the remaining mammograms and potentially on radiologists’ cancer detection rate, which warrant further study. Apart from reducing workload and burden on cancer screening services, a synthetically increased incidence of cancer in radiologist-reviewed datasets may increase the cancer detection rate (Evans et al., 2013).

Rodriguez-Ruiz et al. (2019b) used a proprietary deep learning model to assign a score from 1 to 10 representing risk of malignancy, iteratively testing each score as a cut-off point for exclusion from human interpretation. They acknowledge that such a triage system requires further validation and balances reducing workload with risk of increasing false negatives relative to the chosen threshold for exclusion. Such a threshold could be chosen based on local practices and incidence of breast cancer, and could be integrated into a system where mammograms pre-screened as normal are single-read and others are double-read. The most feasible result was a cut-off score of two which resulted in a 17% workload reduction and exclusion of 1% of exams containing cancer (Rodriguez-Ruiz et al., 2019b).

Yala et al. (2019a) simulated the use of a deep learning model to triage screening mammograms as cancer-free, also with a preset probability threshold, and subsequent diversion from radiologist assessment. They demonstrated a human workload reduction of 19%, improved specificity, and non-inferior sensitivity. The authors recognized the limitations of their work, which included the use of data from a single institution and scanner manufacturer and the need for prospective external validation (Yala et al., 2019a).

Cancer Detection

A large number of studies have been published on the ability of deep learning models to classify mammograms into cancer’ or cancer-free’, although for some women, this is potentially an oversimplification of tumor biology (see the ‘Differentiating Ductal Carcinoma In Situ and Invasive Breast Cancer’ section). Regardless, diagnosis of cancer with a view to definitive treatment when suitable is obviously important. Broadly, deep learning approaches to cancer detection can be grouped into image-level classification (or classifying entire images) and object detection (or classifying abnormal image features within an image) or a combination of the two (Wu et al., 2019; Bionetworks, 2019). Some papers exclusively address 'mass’ detection. Masses represent a subset of the mammograms of women with cancer, others having more subtle signs such as architectural distortion or clustered microcalcifications or no mammographically visible findings at all. Caution should be taken when comparing results across different datasets and given the variable attention to locating particular findings (like masses or architectural distortion). Also, stand-alone deep learning models are typically assessed with Area Under the Receiver Operating Characteristic Curve (AUROC), and human performance with sensitivity and specificity, which can be compared only by extrapolating one or more human metrics to a Receiver Operating Characteristic Curve (ROC) or selecting a specific operating point for a deep learning model.

Wu et al. (2019), using a database of over 1 million mammography images, found that deep learning improves radiologists’ performance at breast cancer screening, when simulated retrospectively. They initially trained a fine-grained, auxiliary 256 X 256 patch-level classifier on pixellevel annotations with ImageNet pre-trained weights and a DenseNet-121 architecture. This model was then inferred on full images to create separate heatmaps for benign and malignant lesions, then forming input channels on full image and patient-level classifiers. Image-level classifiers were based on a ResNet architecture variously combined into breast-level and patient-level models (see Wu et al. (2019) for further details). Tested on over 14,000 screening episodes with an unadjusted incidence of abnormalities, their model was approximately as accurate as an experienced radiologist in detecting malignancy, with an AUC of 0.895. Importantly, they showed that the average of cancer probability of a human radiologist and of their deep learning model was superior to either method alone, the first research to demonstrate this. They extrapolate that radiologists and their deep learning model had learnt somewhat different aspects of mammographic cancer detection and that such a model could be used as a complementary secondary reader (Wu et al., 2019). Their model is available publicly (Nyukat, 2019).

Rodriguez-Ruiz et al. collected digital mammograms from nine datasets and seven countries across Europe and the USA, totaling 2,652 screening exams. Of these, 653 exams contained malignancies. The cumulative dataset consisted of mammography images, the assessment of a total of 101 radiologists, and the presence or absence of malignancy, as determined by either biopsy or at least 1 year of follow-up. For five of the nine datasets, radiologists’ assessments were a probability of malignancy (from 1 to 100), with the remainder as a Breast Imaging Reporting and Data System (BI-RADS) score (American College of Radiology, 2013) (see Table 4.4). These were converted to a ROC by averaging each reader’s nonparametric curves along lines perpendicular to the chance line. For each dataset, sensitivity and specificity of radiologists assessment for malignant cancer were calculated with a recall threshold of BI-RADS score 3 (probably benign (American College of Radiology, 2013)) or higher. Radiologists and Al system sensitivities were compared with two-modality analysis of variance (ANOVA) at the same specificity level. With this methodology, demonstrating the challenges in comparing Al with human performance, the human AUC was 0.814, and proprietary Al system was 0.840 (Rodriguez-Ruiz et al., 2019).

Table 4.4 Breast Imaging Reporting and Data System (BI-RADS), American College of Radiology (Ribli et al., 2018)

Kim et al. (2018) aimed to develop an algorithm that could discriminate cancer from normal cases based on raw images without annotation. They analyzed a total of 29,107 digital screening episodes, each consisting of four standard views. Of these, 4,399 cases or 14.9% had pathology-confirmed cancers. There were 3,101 cancer cases versus 23,530 normal cases in the training set (13-2% cancer). Equal numbers of cancer and non-cancer cases were used for cross-validation and test sets. Women with mammograms assigned a BI-RADS score of 2 through five (including probably benign’, ‘suspicious’, and ‘highly suspicious for malignancy’), where pathology was non-cancerous, were excluded. The training, validation, and test datasets were evenly distributed in age, breast density, manufacturer, cancer type (invasive, non-invasive), feature, and mass size (> or<20 mm) to control for selection bias. Cancerous features were mass (54%) and microcalcifications (45%), focal asymmetry (10.7% of cancers), and distortion (2.3% of cancers). Results were a test-set AUC for cancer/non-cancer of 0.906 and Fl score of 0.81. Sensitivity was not affected by density (with statistical significance -p = 0.3); however, specificity and accuracy decreased with increasing density (p<0.06 and 0.02). Importantly, they demonstrate statistically differing model sensitivity and specificity per manufacturer (see Table 4.5), despite random perturbation of pixel intensity contrast and brightness (±10%) during training, indicating that the distribution of manufacturers in any training set may affect diagnostic performance.

As part of a DREAM challenge, Nikulin first trained a 224x224 patch-level model using five labels on Digital Database for Screening Mammography (DDSM (Lee et al., 2017)) data:

■ 0 - healthy tissue, the patch is randomly selected from a healthy breast image.

■ 1 - calcification benign.

■ 2 - mass benign.

Table 4.5 Kim et al. (2018) Deep Learning Differentiation of Normal versus Cancerous Mammograms; Test Set Results by Vendor

■ 3 - calcification malignant.

■ 4 - mass malignant.

This model was then incorporated into an image-level classifier and finetuned (additional training of parameters). With ground truth based on a tissue biopsy within 1 year of the given screening exam, their result in classifying exams into cancer or cancer-free in the challenge was an AUC of 0.874. Nikulin concludes that depth (or the number of layers of a neural network) is better than width (or the nodes per layer) and identifies that, compared to problems generating much of the theory and practice of computer vision like ImageNet, objects for detection in screening mammography are much smaller and have less components (Bionetworks, 2019).

Ribli et al. published an independent model trained on both INBreast and DDSM, and achieved the second place in the DREAM mammography challenge. Their model was pre-trained on ImageNet and re-trained on DDSM and their own Semmelweis University dataset of 11,300 images. Results were an AUC on INBreast of 0.95 and an AUC of 0.85 on the dream dataset (Ribli et al., 2018). A demonstration version of this model is available at GitHub (https://github.com/riblidezso/frcnn_cad).

Kyono et al. (2019) present a model primarily for cancer detection, that is also capable of and tested as a triage system. They utilize multi-task learning, whereby additional labels (target variables or) were used in addition to the binary outcome of core biopsy or surgical specimen-proven cancer to assist network regularization and overall performance. There were five auxiliary labels: ‘sign’, ‘suspicion’, conspicuity’, breast density (per radiologist-assessed percent density), and age (where ‘sign’ was the abnormal finding, e.g., ‘spiculated’, 'microcalcification’; suspicion was one of ‘normal’, ‘benign’, 'probably benign’, ‘suspicious’, and 'malignant’; and ‘conspicuity’ was one of 'not visible’, 'barely visible’, 'visible, not clear’, and ‘clearly visible’). Using four views from 1,000 randomly held out test set, their model achieved an AUROC of 0.791 for cancer detection. They demonstrated that multi-task labels, testtime augmentation, and multiview inputs all improved performance. As a triage system, their model was able to reduce radiologist examinations by 42.8% while improving the overall diagnostic accuracy (Kyono et al., 2019).

All methodological issues and difficulties comparing human and machine performance considered, deep learning appears to currently achieve retrospective results at least approximating human performance. For further overview of the use of artificial intelligence including deep learning for mammography, the reader is referred to reviews by Hamidinekoo et al.

(2018), Houssami et al. (2019), Abdelhafiz et al. (2019), Gao et al. (2019), Harvey et al. (2019), and Geras et al. (2019).

Density and Cancer Risk

Breast density is associated with higher risk of breast cancer, although the degree of relative risk increase varies across the literature. Assessment of density is variable across computational methods, human radiologist assessment, and its correlation with breast cancer (Destounis et al., 2017). Screening programs may or may not notify women with high density, with 38 states in the USA recently legislating mandatory notification (Keating & Pace, 2019)- Density is thought to confer increased risk via two mechanisms:

• 1. Dense breast tissue can have the effect of masking cancerous lesions.
• 2. Women with dense breasts have relatively more tissue from which cancer can generate (Kerlikowske & Vachon, 2016).

There is currently limited evidence about the most appropriate screening interval or course of action given a Gaussian distribution of breast density (Keating & Pace, 2019).

Yala et al. (2019b) compared image-only and an image plus traditional risk factors (hybrid) deep learning models with logistic regression of traditional risk factors and the Tyrer-Cuzick (TC) model using 39,558 women. These risk factors included breast density, age, weight, height, age at first menstrual period, menopausal status, hormone replacement therapy, BRCA gene status, past history of ovarian cancer and breast biopsy, family history, and Ashkenazi inheritance. Data included 1,821 patients with a diagnosis of invasive breast cancer or DCIS within 5 years from baseline imaging. They found superior performance of the hybrid model with an AUC of 0.70 (95% CI 0.66-0.75). An image-only model resulted in an AUC of 0.68 (95% CI 0.64-0.73). Traditional risk factor logistic regression and the Tyrer-Cuzick model were performed at AUCs of 0.67 (95% CI 0.62-0.72) and 0.62 (95% CI 0.57-0.66), respectively.