Modeling of Textures to Predict Immune Cell Status and Survival of Brain Tumour Patients

Gliomas, the most common brain-initiating tumours, are classified by histopathological process into four grades (I, II, III or IV), depending on their aggressiveness, as noted by the World Health Organization (WHO) [1]. Specifically, grade I is a non-invasive tumours and considered as generally benign, grade II and III are known as lower grade glioma (LGG) while grade IV represents the high grade glioma that is the most malignant tumours such as glioblastoma multiforme (GBM). GBM is the most deadly brain tumor with a median survival of 15 months [2] while LGG patients is longer survival with an average approximately 7 years. At the time of diagnosis, most patients initially receive a surgical resection / biopsy and then radiation therapy (XRT) and/or the single chemotherapy agent temozolamide (TMZ) [3]. Many of these treated patients are leading to recurrence and death faster than others. Immmnotherapy is a remarkable cancer treatment approach that uses the patient’s own immune system to kill cancer cells [4]. This strategy is indicated that the intratumoral immune response is related to the tumor progression and prognosis in gliomas [5]. Notwithstanding the development in treatment improvements, foreseeing reaction to immunotherapy preceding treatment stays a difficult errand for clinicians.

With this favorable position of treatment, the identification of the response to immune therapeutics requires an invasive technique by either biopsy or medical procedure strategy, considering the natural dangers, restrictions of testing, difficulties and expenses related with biopsies. Therefore, a non-invasive biomarker is required to avoid these limitations. Radiomics is a non-invasive approach that is able to convert the images into quantitative data [6]. It examined the relationships between tumor imaging features, genomics, proteomics, transcriptomic, and clinical outcome [7, 8, 9, 10]. Moreover, radiomics signatures has been used for assessing CD8 cell count and its corresponding relation to clinical outcomes of patients treated with immunotherapy [11]. For example, the tumour volume ratio in T2-FLAIR scans relative to the contrast enhancement volume was shown to be related to GBM mesenchymal subtype [12] that has a stronger immunological response compared to other GBM subtypes [13]. Recent work has examined the relationships between tumor imaging features (e.g., shape, texture, histogram), immune cell markers and the corresponding survival outcome of patients with glioma [14, 15].

To date, limited studies have been considered the deep CNNs in the context of radiomics. For example, CNN is used to generate the deep features located in fully connected layers to predict the survival outcome [16]. While the CNN hidden layers with their feature-maps have been quantified and used for classification tasks as a panel of informative descriptors[17, 10]. Despite the potential of CNN-based radiomics in classification, no study yet has explored the deep radiomic analysis in predicting the immune cell status and/or impact on survival outcome for patients with brain tumours (i.e., LGG and GBM). We propose in this study to extract the radiomic features from the activation maps of a pre-trained 3D CNN to solve this problem. This model extends the work of [17], where the CNN activations applied to CT scans were modelled by Gaussian mixture model (GMM). We hypothesise that this approach, applied to four MRI sequences, can encoded important characteristics of tumor heterogeneity that are associated with immune cell markers.

Radiomics has been shown to be successful in the computer-aided diagnosis of different cancer diseases [6]. Specifically in the context of personalised medicine, the deep CNN algorithms have indicated an incredible capacity to identify complex high-dimensional data associations and have been adequately utilized for disease diagnosis and treatment planning [18]. These models additionally showed the potential in survival analysis [19]. Regardless of their successfully applications using the natural image, CNNs have had a more limited achievement in clinical imaging, mainly because of the limited data-sets that in CNN models lead to serious overfitting. A simple solution is to quantify the pre-trained CNN models for prediction tasks using information theory [10], standard radiomic functions [6] and GMM parameters [17]. In [10], the quantifier functions are unable to provide full information as suggested in [17] that the CNN has been separately trained on MRI scans and used to generate GMM descriptors. Inspired from [17], we assume that the samples observed are generated from a mixture of Gaussian distributions whose parameters (average, covariance matrix and weights) are learned using the Expectation Maximization (EM) [20]. In particular, to predict the status of immune cells and survival groups of patients, we applied the GMM directly to CNN feature maps of brain tumour images.

Figure 1 shows the pipeline of deep radiomic model. MRI is the first way to obtain T1-weighted (T1-WI), T1-weighted post-contrast/contrast enhancement (T1-CE), T2-weighted (T2-WI) and fluid-attenuated inversion recovery (FLAIR) imaging data from patients with brain tumor (i.e., LGG and GBM). In axial slice, the tumour region of interest (ROI) is semi-automatically delineated. For the generation of multi-scale texture from CNN features of each brain tumour, a pre-trained 3D CNN is then used. These CNN feature distributions is converted into elements vector of tumor heterogeneity using the GMM. Combined imaging features in a vector is corresponding to immune cell markers and clinical information (age, gender and overall survival). Last, we applied the random forests for predicting immune response status (low versus high, below or above median value of each of the three immune markers) and survival patient groups (short vs. long survival, below or above the median survival of patients). In addition, we performed the log-rank test and Kaplan-Meier (KM) estimator to evaluate the predicted survival groups from RF models.

For classifying immune cell status, Figure 3 shows the AUC value (accuracy) of 78.67(74.8), 83.93(76.8) and 75.67(73.5)% for Macrophage M1, Neutrophils and T Cells Follicular Helper, respectively. We found that the highest performance metrics (i.e., confusion matrix and AUC) were obtained to predict lower or higher values for the Neutrophil marker using deep features with RF models.

Table I reports the performance metrics of survival analysis. For classifying short-term from long-term survival, RF model shows that the combination of deep radiomic features (R), immune cell markers (I) and clinical variables (C) is achieved a highest AUC of 74.12% compared to 73.01% for R (73.51% for R+C), 65.54% for I (73.91% for R+I) and 62.51% for C (65.84% for I+C), respectively. The predicted survival groups by RF models are then used to measure the significance by applying the Log-rank test and KM estimator. Figure 4 shows KM survival curves, except for clinical features for which the curves of shorter-term and longer-term survival overlap on   20 months, all features give clearly-separated curves after a period of   5 months. These findings are consistent with AUC values. Specifically, the combination of R+I+C leads to best significance p of 4.31$\times$10^-7 compared to the RF models using separately the R (p = 1.45 $\times$10^-5), I (p=0.003) and C (p= 0.07), respectively. We found that the immune cell markers and deep radiomic features are statistically significant for predicting the survival of patients with brain tumor. Considering k$\bm{\geq}$ 2, the predicted survival groups are insignificant with p$\bm{>}$0.05.

This research examined deep radiomic descriptors that modelled 3D CNN features using Gaussian mixture model to predict the immune cell makers and the survival in brain tumour patients. Our results suggest interactions between the deep radiomic signature, the markers of immune cells and overall survival. Inspired by these findings, we plan to extend the suggested deep radiomic pipeline through different types of cancer.

This research study was conducted retrospectively using human subject data made available in open access by TCGA and TCIA. Ethical approval was not required as confirmed by the license attached with the open access data.

This work was supported by Foreign Young Talents Program (No. QN20200233001). The funding agency has no role in the conceptualization of the study, data collection and analysis, or the decision to publish these results.