A New Hierarchical Deep Learning Model Accurately Predicts Response to Neoadjuvant Chemotherapy in Breast Cancer

by Christos Evangelou, MSc, PhD – Medical Writer and Editor

In a recent study, researchers at York University and Sunnybrook Health Sciences Center in Toronto, Canada, developed an automated hierarchical deep learning model that can analyze digital pathology images of tumor biopsies taken before chemotherapy and accurately predict whether the tumor will have a complete pathological response.

The framework combines convolutional and self-attention modules to extract informative features from high-resolution histopathological images and uses multi-head attention modules to capture global relations between tumor areas.

In a proof-of-concept validation of the deep learning model in pre‑treatment tumor biopsies from patients with breast cancer, the proposed methodology demonstrated promising results in predicting response to neoadjuvant chemotherapy and could pave the way for a response-guided therapy paradigm for individual breast cancer patients.1

“Early prediction of chemotherapy response can facilitate therapeutic modifications for individual patients, possibly improving overall treatment outcomes and patient survival,” said lead researcher Dr. Ali Sadeghi-Naini, professor at York University.

He added that this study demonstrates the promise of deep learning and digital pathology for bringing precision medicine closer to reality in breast cancer treatment to potentially guide future therapies.

The report was published in the journal Medical Physics.

Study Rationale: Improving Prediction of Response to Neoadjuvant Chemotherapy

Chemotherapy remains a crucial component of treatment for many patients with breast cancer. Pathological complete response after neoadjuvant chemotherapy is strongly associated with improved survival. However, only approximately 30% of patients with breast cancer show a complete pathological response after neoadjuvant chemotherapy.

Biopsy specimens are acquired as part of the standard of care for patients with suspected breast tumors to diagnose malignancy and develop a treatment plan. The ability to predict response in individual patients before starting chemotherapy using pre-treatment biopsy samples would be invaluable, as doctors could modify treatment early on to improve outcomes and survival for predicted low- or non-responders. However, existing approaches for predicting response to chemotherapy have shown limited accuracy.

Approach: A Hierarchical Deep Learning Model for Predicting Response to Chemotherapy

The researchers used digitized hematoxylin and eosin (H&E)-stained pre-treatment biopsy slides from 207 patients with breast cancer who received neoadjuvant chemotherapy before surgery.1 A pathologist annotated the tumor areas in each digital slide.

The researchers subsequently developed a hierarchical self-attention-guided deep learning model consisting of patch-, tumor-, and patient-level modules. This hierarchical architecture allows the model to capture both local and global features of histopathological images and generate patient-level response predictions.

“The novel research presented in this paper features the first clinical investigation of an innovative automatic system developed for hierarchical attention-guided deep learning of digital histopathology images of pre-treatment biopsy specimens to predict pathological response to neoadjuvant chemotherapy in breast cancer patients a priori,” said Dr. Sadeghi-Naini.

At the patch level, the model uses convolutional layers and transformer self-attention blocks to analyze small tumor regions and learns to extract key features predictive of response that are used to generate optimized feature maps.

The tumor-level module aggregates information from different patches to characterize each tumor bed. Finally, the patient-level module examines all tumor beds to make an overall chemotherapy response prediction for the patient.

The proposed framework combines state-of-the-art deep learning techniques, including convolutional neural networks (CNNs) and transformer self-attention blocks, to extract informative features from high-resolution histopathological images and capture global relations between tumor areas. The hierarchical framework enables the integration of local and global features to generate patient-level response predictions.

“A key innovation of our model is that it examines both local features within the tumor and global relationships between different tumor regions,” explained Dr. Sadeghi-Naini.

“This hierarchical approach allows very high-resolution analysis of whole tumor slices and overcomes the difficulty of deriving the global relations between different tumor areas in high-resolution whole-slide digital histopathology images,” he added.

Model Validation

The proposed hierarchical deep learning model was used to analyze a set of digitized images of core needle biopsies from 207 patients who were treated with neoadjuvant chemotherapy followed by surgery. The proposed framework provided an area under the curve (AUC) of 0.89 and an F1-score of 90% for predicting pathological complete response after treatment with neoadjuvant chemotherapy.

Validation studies in an independent set of 3,574 annotated tumor beds and 173,637 patches of digital slides from 63 patients showed that the model provided a sensitivity of 87%, specificity of 83%, and accuracy of 86% for predicting complete pathological response.

Notably, the proposed framework outperformed several state-of-the-art deep learning models and traditional machine learning methods that did not incorporate hierarchical tumor- and patient-level processing, indicating that the hierarchical structure of the proposed framework is essential for its high performance in predicting response to neoadjuvant chemotherapy.

The study also showed that the proposed methodology can capture both local and global features of histopathological images and generate patient-level response predictions. These results provide strong evidence that the model can meaningfully analyze tumor characteristics in biopsy slides and identify predictive signals, even before chemotherapy begins.

According to Dr. Sadeghi-Naini, the findings of this study suggest that the proposed framework could pave the way for a response-guided therapy paradigm for individual breast cancer patients, enabling early identification of non-responders and tailoring treatment strategies accordingly.

The researchers plan to validate the model on larger multicenter datasets. If successful, it could eventually assist oncologists in selecting the best chemotherapy regimen for patients at the time of diagnosis.

Looking Ahead

This study exemplifies the potential of artificial intelligence and digital pathology to unlock new personalized approaches for breast cancer treatment. However, validation of the proposed framework in larger cohorts is required.

“Future multi-institutional studies on larger patient cohorts are required for further evaluation of the developed framework for prediction of response to neoadjuvant chemotherapy,” Dr. Sadeghi-Naini noted.

In addition, future studies are warranted to assess the generalizability of the proposed methodology to different populations and settings.


  1. Saednia K, Tran WT, Sadeghi-Naini A. A hierarchical self-attention-guided deep learning framework to predict breast cancer response to chemotherapy using pre-treatment tumor biopsies. Med Phys. July 2023. doi:10.1002/mp.16574

Share This Post

Leave a Reply