Figure 3. Example of patient detection using synthetic MRI (syn-MRI) versus CT in the testing set. Brain baseline CT (left) fails to show any definite hypoattenuating lesions, although the gray-white matter junction of the right insula is suspected. Synthetic MRI (middle) shows distinct hyperintensity in the territory of middle cerebral artery on the right, which is corresponding to the finding on the follow-up MRI (right).

Figure 1. Training and Testing of the generative adversarial network model.

Figure 1. Image preprocessing and analysis workflow. A) Images were first automatically segmented and then manually adjusted if necessary. B) Images were bias corrected, smoothed, and normalized. C) Radiomic features were extracted from original and filtered images. D) Patients were divided into training (80%) and validation (20%) sets. Feature selection was performed in only the training set. Subtypes were identified using selected features, and survival analysis was performed on the two subtypes identified.

Figure 3. Survival rates of the two subtypes identified. A) & B), progression free survival (PFS). C) & D), overall survival (OS). A) & C) Training set. B) & D) Validation set.

Figure 1. Magnitude resolution degradation (MD) vs. Complex resolution degradation (CD) in MR single image super-resolution (SISR).

Figure 3. An example testing slice with LR image, bicubic interpolation image, HR ground truth and output images from different networks. The regions marked by green boxes are zoomed in.

Figure 2: Illustration of the fused CNN and tabular data architecture

Figure 3: Model interpretation based on Gradient-weighted Class Activation Mapping (Grad-CAM).

Figure 1. Workflow of the images, pre-processing, data input, network training, and evaluation of the HighRes3DNet deep learning model. Black blood images from 2 vendor scanners were used for manual segmentation in ITK-SNAP, which served as supervision. Images were cropped to the same volume with subcortical coverage regardless of resolution, underwent non-local means filtering, and split into hemispheres to increase the sample size. The HighRes3DNet was trained with 10-fold cross-validation on the training set that included 3T and 7T images, and evaluated on a separate test set.

Figure 3. 3D renderings by ITK-SNAP of LSA segmentation results using ten-fold validated HighRes3DNet from the test group never seen by the model during training. The outputs generally agree well with the identification of vessels by manual labeling, although there is still some disagreement regarding the distal portions of the vessels. Note the manual segmentation is subject to human interpretation.

Figure 1: Pipeline of the proposed Ensemble and Interpretability models for EDSS prediction and visualization of the brain networks responsible for the prediction.

Figure 2: Comparison of measured (red) and estimated (blue) EDSS score in MS patients.

Figure 3: Schematic diagram of an optimized Faster R-CNN for automated detection and classification between HT and PTMC.

Figure 2: A papillary thyroid microcarcinoma (arrow-heads) in the right lobe (zoom in twice) on DW-MRIs and ADC maps with b=0,800,2000 sec/mm².

Fig. 1: Flowchart for using GAMER-MRI to select the most discriminating subject-wise normalized diffusion measures and correlating the combinations of selected diffusion measures with the Expanded Disability Status Scale and the serum level of neurofilament light chain.

Fig. 2: GAMER-MRI. (A) The neural network. Conv is a convolutional block consisting of a 3x3x3 convolutional layer, exponential leaky units and batch normalization. FC is a fully connected layer. Attention weights are obtained from the softmax function after the attention blocks. Each diffusion measure is encoded parallelly before the softmax function. The hidden features of diffusion measures are linearly combined with the attention weights and input to the classifier. (B) Attention block. ⊙ represents an element-wise multiplication.

Fig 1 ResUNetModel training scheme. Two networks: whole head ResUNet and bone enhanced networks were trained. The patches were randomly selected from the whole head. For the bone enhanced network the placement of the patch is determined by the patches center voxels located within the bone.

Fig 3 ResUNet Model testing scheme. The final pCT output was created by combining the pCT from the two networks. The whole head network pCT output was multiplied by brain mask and the bone enhanced network pCT output was multiplied by one minus the brain mask. The results of both multiplications were added for the final pCT output.

Figure 2 Heatmaps showing the 5 different MRI phenotypes after cluster analysis. Subjects are shown on the x-axis and the JHU WM tracts on the y-axis. HC are shown on the left and are not included in the clustering as well as SLE patients without WMH. (Top) l2-normalized WMH pattern on which the clustering was performed. Non-normalized WMH load sorted by cohorts and clinical labels (Middle) summed lesion burden (Bottom).The colour bars at the top indicate cohorts (Leiden = brown), FLAIR information (3D = pink) and clinical labels (Healthy controls (HC) = green, nonNPSLE = blue, NPSLE = red).

Figure 3 Lesion frequency map for HC and each cluster in MNI-space. WMH in cluster 1 can be mainly assigned to Forceps Major, cluster 2 to right Anterior Thalamic Radiation, cluster 3 to Forceps Minor and 4 to the left Anterior Thalamic Radiation. Cluster 5 cannot be assigned to any specific WMH tract due to high WMH burden. The main WMH which corresponds to the WM tracts (copper colour) are emphasised with red arrows.

Figure 1. Connectome analysis procedure. Subject-level structural network is derived from DWI MRI data using probabilistic fiber tracking between the segmented regions of interest. Subject-level functional network is derived from resting state fMRI data based on the pairwise correlations between the regions of interest. Graph theoretical nodal centrality features are computed for both structural and functional networks and their coupling using multilayer analysis.

Figure 3. Classification comparison. The multi-modal multilayer based SVM classification model resulted in higher predictive values compared to classification models based on single layer of structural or functional connectomes. These results indicate the ability of the multilayer approach to provide improved classification between EP and HC. *p<0.001

Figure 2. Three samples of the nine image rulers in use. From top to bottom: for F/S elbow, hip, and F/S brain scans.

Figure 3. Plots shown to technicians on scanner. The red threshold line is chosen by radiologists, and the two class scores around it are defined as moderate for the pie chart.

Fig. 5. Mean ROC curves of 4 settings of feature permutation. Mean AUCs and their 95% CIs are shown in the legend.

Fig. 4. PIMPs with their 95% CIs and F-values of the selected 30 features.

Fig. 2. Visualization of the segmentation. A slice from the test participant is classified using single-voxel information only (right) and compared to the gold-standard segmentation in the image domain (left).

Fig. 1. Visualization of the classification approach.

Figure 3. Segmentation results of the 3D lumbosacral plexus. 3D model of manual segmentation (a,b) and U-Net algorithm segmentation (c,d) of a 53 years old woman with degeneration of lumbar vertebrae and disc herniation. The DCE, PPV and SEN between the U-net and rater is 0.867, 0.884, and 0.850, respectively.

Figure 2. Segmentation results of the 2D lumbosacral plexus. Masks of manual segmentation (red line) and U-Net algorithm segmentation (yellow line) of a 63 years old woman with degeneration of lumbar vertebrae. The DCE, PPV and SEN between the U-net and rater is 0.873, 0.907, and 0.841, respectively.

Figure-2: Representative feature value comparisons between SMag_vasculature and SMag_hemorrhage for the top-ranked four features

Figure-1: Flowchart of methodology

Figure 1. Architecture of the CNN model used in the experiments. The model processes the ON and OFF transients separately and subtracts the results to obtain the final J-difference edited spectrum.

Figure 4. Sample spectra reconstructed by the different CNN models for the different samples in the test set.

Figure 3: One case from testing on T1w images. The model performs well regarding face removal without destroying irrelevant tissues, such as the skull base and bottom of the frontal brain lobe. The raw and deface-refaced images were unidentical, showing the deface was irreversible. Difference maps were from the subtraction of the raw and deface-refaced images. (red arrows).

Figure 4: Face removal results done on unseen MPRAGE images, targeting different MRI contrasts. The face was mostly removed, while other tissues were retained.

Figure 1

Figure 4

Figure 1: Substantia Nigra Pars Compacta (SNc) regions of interest (ROI) using ConvNet and manual segmentations for a representative Parkinson's disease patient and a healthy volunteer.

Table 1: Demographic and clinical characteristics of early Parkinson's Disease patients and healthy volunteers.

Fig.1 The network architecture of our proposed deep learning-based MSH-EPI reconstruction method.

Fig. 3: Representative diffusion weighted images from a patient with old cerebral lacunar infarction with SSH-EPI, conventional and deep learning-based MSH-EPI reconstructions, T1-weighted and T2-weighted images.

Figure 1. Mean Z-scored FA and MD distribution across the discriminative voxels selected via RFE-SVM.

Figure 2. Selected voxels for outcome prediction. Color-coding according to the image modalities from which each voxel was selected. Radiological orientation.

Figure 2: Training results on the dataset from PD cohort. Most brain structures were delineated accurately by S2Q, compared with real QSM calculated from STAR-QSM. Furthermore, some residual artifacts near the boundary (red arrows) were disappeared in S2Q.

Figure 3: Testing results on another dataset from PD cohort. The residual artifacts near the nasal cavity were all removed. The testing set's model performance was comparable with that on the training set, suggesting minimal generalization error.

Figure 2. Flowchart for the radiomics experiments. The last inset on the right contains the tuning feature number in the model with 5-fold cross-validation. A total of 28 features were kept in the final model when the 1-SE rule was used (top). The ROC curves for the final model over training and testing datasets (bottom).

Table 3. Six most important features and their corresponding coefficients in the final model.

Figure 1. Deep learning-based automatic segmentation. A pre-trained DCNN⁵ takes in pre- and post-contrast T1-weighted, T2-weighted and T2-FLAIR images, and generates segmentation results including three subregions: peritumoral edema, enhancing tumor, and necrotic & non-enhancing tumor core.

Figure 3. Receiver operating characteristic curve analyses for OS classification models.

(Left) input T2-W images, (middle) manual segmentation, (right) automatic segmentation of a case from the testing dataset. Green and red regions correspond to tissue identified as clot and edema, respectively, and demonstrate good agreement between the manual and automated segmentations.

After autonomous-generation of a set of clot and edema segmentations in the axial view, coronal and sagittal views can be extrapolated from the known dimensions of the original dataset. This allows for better visualization and localization of clot components, and their relevant volumes. The small blue region is uncertain- the model identifies it as either clot or edema, but cannot distinguish between the two categories.

Fig. 1. Flow diagram for first approach: Multi-view 2D CNN

Fig. 2. Flow diagram for second approach: Multi-view 3D anisotropic CNN

Segmentations made by our model, SCT, and three radiology residents in comparison to the consensus ground truth on an MR image of the spine with lesions. The DSC for this case is highlighted for each rater.

Comparison of the U-Net++, SCT, and three radiology residents on the 20 testing images of the cervical spinal cord. 15 images had lesions present and 5 had no lesions. Some control cases had other imaging artifacts to represent difficult or uncertain cases. The best performance is highlighted in bold.

Figure 2 – Comparison of Dice coefficients between the tumor segmentation algorithm (HD-GLIOMA) and expert masks: Histograms showing the distribution of the values (100% indicates a perfect overlap and 0 a complete lack of overlap).

Figure 1 – Comparison of Dice coefficients between different brain extraction tools HD-BET and expert masks: Histograms showing the distribution of the values where values of 100% indicate e perfect overlap and 0 a complete lack of overlap.

Deep-learning-based noise reduction incorporating spatial distribution of noise. Parallel imaging creates a low sensitivity region in the center of the reconstructed image (indicated by the dashed line in the g-factor map), so the thresholds segmenting the high g-factor region in the g-factor map include the central low sensitivity region. The two g-factor regions were segmented by using thresholds of 1.4 and 2.0 for R of 3 and 4, respectively.

T2 weighed brain image: (a) full sampling image, (b) input image with acceleration rate of 3; input image denoised by (c) conventional method and (d) MA-CNNR; (e) high g-factor region registered in full sampling image. Due to the higher g-factor, the noise in the central region of the input image (b) was higher than that in the peripheral region.

1. All images were resampled to 1mm isotropic resolution and co-registered to the MNI registered T₁W image. 2. Ischaemic VOIs were created using previously described ADC and T₂ limits to reduce CSF contribution.^2,10 3. Non-ischaemic VOIs were created by reflecting the ischaemic VOI across the vertical axis and applying the ADC and T₂ limits. 4. Image intensity ratios were computed by dividing the mean values of ischaemic VOIs by mean non-ischaemic VOIs. SI = signal intensity.

Accuracy and confusion matrices for cumulative ordinal regression models. Darker shades indicate the higher number of correct predictions. The standardised T₂ relaxation time ratio was the most accurate at identifying patients within each treatment window. All models identified patients within the middle IA treatment window. In this figure, a + indicates a linear combination of input features, and * indicates the inclusion of an interaction term.

Top row: group differences in the distribution indices of M1 QSM positive values. Asterisks indicate that all differences were statistically significant. Bottom row: diagnostic accuracy of each feature.

Maximum diagnostic accuracy of SVM classifiers as a function of the number of QSM distibution indices jointly considered. For example, the last black bar on the right indicates the maximum diagnostic accuracy (A = 0.90) obtained with the joint use of all four distribution indices of QSM positive values in M1.

Fig. 3 Some examples of successful/failed (differences ≥ 10ml/cycle) hemodynamic measurements in multi-site training. Segmentations of aorta (red) and pulmonary arteries (PA, blue) from both single-site and multi-site CNN are presented with Dice scores. The letters correspond to those in Fig. 2. ToF, tetralogy of Fallot; TR, tricuspid regurgitation; HLHS, hypoplastic left heart syndrome.

Fig. 2 Bland-Altman plots for net flow in the ascending aorta (Qs, upper row) and main pulmonary trunk (Qp, lower row) quantified by site1 CNN, site2 CNN, and multi-site CNN. Institution1 data are plotted by open circles while institution2 data are shown by solid circles. Limits of agreement and mean differences are presented as green and red lines. The letter labels indicate successful and failed (differences ≥ 10ml/cycle) examples for flow quantification. The labels correspond to the segmentations shown in Fig. 3.

Pipeline for radiomics analysis and feature extraction

Classification results based on T1 radiomics

Figure 1 Representative original images and undersampled images with different undersampled rate.

Figure 2 Representative images of the segmentation results of different three group experiments.

Estimation of the region and the VF for two patients. The first column shows where the manual region was drawn. The coordinate of the center of mass (in voxels) is placed below each image. The second column shows the predicted region. It is possible to observe that different regions over the transverse sinus can yield similar vascular functions, as illustrated in the plots of the third column. Plots: Red = manual VF and blue = predicted VF.

A region of interest is selected manually over the transverse sinus (left) to estimate the VF (right). To compute the VF, the region was propagated across the dynamic T1-weighted images and the average of the intensities of the pixels was computed for each time point. The mean (black line) and standard deviation (red shaded region) of the 155 VF curves is presented on the right image.

Figure 1. Flowchart of feature extraction. For all four DTI derived indices in each subject, the feature extraction procedure can be divided into the primary feature from each anatomical region in the brain (panel A), and the secondary feature set (panel B), which was derived from the disease affected pattern. The secondary features were calculated from the product of the value difference to the distance between two primary features. Only features were involved in the disease affected pattern and passed the neighborhood selection were selected.

Figure 3. The selected secondary feature set in four DTI derived indices. The selected secondary feature set was showed in mean diffusivity (21 links), fractional anisotropy (37 links), axial diffusivity (31 links), and radial diffusivity (13 links), respectively. Blue nodes indicate the anatomical regions in the AAL template. Links between nodes indelicate the secondary features and the thicker link means the more significance among classes.

Figure 1: Patient B has a CEL volume of 28.4 cm³, progressed at 84 weeks, and died at 146 weeks. Although Patient A has smaller CEL & T2L volumes (CEL=10.9 cm³), they progressed much sooner at 10 weeks, and died at only 47 weeks

Table 1: Performance of RandomForest model to predict whether or not OS<45 weeks for each mask using just pre-RT images, and both pre-RT and mid-RT images

Figure 3: Averaged saliency maps obtained after training and optimizing the CNN to predict WM subconstruct scores.

Figure 2: Network outputs for both CNN and KRR vs ground truth WM score. Blue dots are CNN outputs and black triangles are KRR outputs.

Heatmaps from MRI of a SPMS patient. Shown are FLAIR image slices 57-59 (left to right, top row) out of 135, and corresponding Grad-CAM heatmaps from VGG16 (left) and VGG19 with GAP (right). The CNN layers highlighted in VGG16 with GAP are: A) last convolutional layer – ‘block5_conv3’, B) second last convolutional layer – ‘block5_conv2’, and C) first convolutional layer of the last convolutional block – ‘block5_conv1’; and in VGG19 with GAP: A) last max pooling layer – ‘block5_pool’, B) last convolutional layer – ‘block5_conv4’, and C) second last convolutional layer – ‘block5_conv3’.

Further heatmaps generated from the brain MRI of the same SPMS patient using VGG19 with GAP. Shown are also MRI slices at 57-59 (left to right, top row) out of 135. The CNN layers highlighted are: A) third last convolutional layer – ‘block5_conv2’, B) first convolutional layer of last convolutional block – ‘block5_conv1’, and C) last convolutional layer of second last convolutional block – ‘block4_conv4’.

Figure 1: Graphical Model for the brain region volumetric matrix factorization. C = {C_ik} is brain region adjacency matrix, B and Z are latent brain region and factor feature matrices with B_i and Z_k representing brain-region specific and factor-specific latent feature vectors. P_j represents patient latent vector for patient j. R_ij represents the volumetric observation (value) of brain region i for patient j.

Figure 2: AUROC and Accuracy plots for predicting patient long-term outcome (PCS labels) using different feature sets.

Figure 1.: Lesion segmentation and iron classification results of the CNNs from the same slice of one representative MS patient: A segmentation comparison for two MS lesions, between manual expert labeling (blue) and CNN labeling (red) is shown in the top image. An example for a non-iron (left) and iron lesion (right) classification from the same area as above is shown on the bottom. A prominent hypointense and hyperintense iron-rim is visible in the SWI and QSM image, respectively.

Figure 2.: Receiver operating characteristic curves for all network combinations: The Graph shows ROC curves with true-positive rate plotted against false-positive rate for lesion-wise prediction of iron.

Figure. 1 A: Flow chart of the image-analysis pipeline. B Main procedures of predictive model construction and evaluation.

Figure. 2 A: Bland-Altman analysis of measured and predicted UPDRS improvement, the paired t-test significance p=0.94 and mean prediction error of 9.0% UPDRS improvement. B: Correlation analysis of measured and predicted UPDRS improvement, the correlation r=0.87, p<0.001.

Figure 1: Flowchart detailing the methods used to build the predictive model.

Figure 2: Results of multiple articulator segmentation on the test data. Three sample postures are shown in the figure. Reference segmentation from User1 is compared against segmentation from User2 and the proposed stacked transfer learned based U-NET. The DICE similarities for the tongue (T), airway (A), and the velum (V) are embedded. These segmentations demonstrate good quality from the proposed U-NET scheme with variability in the range of differences between user1-user2 segmentations.

Figure1: Stacked U-net architecture with hybrid learning. Each of the red boxes represents the U-NET model. The U-NET models for the tongue and velum segmentation are pre-trained with an open-source brain MRI dataset [6], and the UNET model for the airway is trained with an In-house airway MRI dataset. Later the velum and tongue U-nets are trained with an in-house MRI dataset which has few manually labeled (~60 images) articulator segmentations. The final output airway, tongue, velum, segmentation is the concatenated segmentations from the individual U-NET outputs.

Figure 1. Flowchart of the modeling process for BraTS 2019 dataset. Pipeline for CAI dataset is similar but for the number of selected features from each category.

Table 1. Comparison of performance of proposed approach with classic feature selectors over BraTS2019 dataset.

Figure 3: Three different profiles (left column) with the native FLAIR images and the segmentation masks overlaid on FLAIR images; reference mask in red, automated mask in green and common area in yellow. Corresponding histograms of the normalized T1 and FLAIR signal intensities with manual mask (red), false positive values in the automated mask (green), and false negative values in the automated mask (purple). a : Best automated segmentation; b : Poorest automated segmentation; c : Moderate automated segmentation.

Table 2: Performance of the automated tumor segmentation in the test patients.

On the left, two CMBs are shown on a SWI image. On the right, voxel intensities of original SWI (green), predicted SWI (orange) and magnitude image (blue) along the yellow line are shown.

(a) Original SWI, (b) magnitude image, (c) difference image of original SWI and magnitude image, (d) predicted SWI, (e) difference image of original SWI and predicted SWI. Red circles show CMBs in difference images. Blue arrows show phase artifacts in difference images.

Figure1. Best-performed multi-task learning model predicting multiple genotype of gliomas preoperatively based on MR images. The figure beside the convolution block means the number of convolution kernel.

AUC, Area Under Receiver Operating Characteristic Curve

Sharing blocks means the number of blocks different branches owned jointly

Remaining blocks means the number of blocks different branches owned respectively