IEEE ISBI 2020 Virtual Conference April 2020

Signal models based on sparsity, low-rank and other properties have been exploited for image reconstruction from limited and corrupted data in medical imaging and other computational imaging applications. In particular, sparsifying transform models have shown promise in various applications, and offer numerous advantages such as efficiencies in sparse coding and learning. This work investigates pre-learning a two-layer extension of the transform model for image reconstruction, wherein the transform domain or filtering residuals of the image are further sparsified in the second layer. The proposed block coordinate descent optimization algorithms involve highly efficient updates. Preliminary numerical experiments demonstrate the usefulness of a two-layer model over the previous related schemes for CT image reconstruction from low-dose measurements.

Adenocarcinoma is the most common cancer, the pathological diagnosis for it is of great significance. Specifically, the degree of gland differentiation is vital for defining the grade of adenocarcinoma. Following this domain knowledge, we encode glandular regions as prior information in convolutional neural network (CNN), guiding the network's preference for glands when inferring. In this work, we propose a prior-aware CNN framework with multi-task learning for pathological colon images analysis, which contains gland segmentation and grading classification branches simultaneously. The segmentation's probability map also acts as the spatial attention for grading, emphasizing the glandular tissue and removing noise of irrelevant parts. Experiments reveal that the proposed framework achieves accuracy of 97.04% and AUC of 0.9971 on grading. Meanwhile, our model can predict gland regions with mIoU of 0.8134. Importantly, it is based on the clinical-pathological diagnostic criteria of adenocarcinoma, which makes our model more interpretable.

Noise and aliasing artifacts arise in various accelerated cardiac magnetic resonance (CMR) imaging applications. In accelerated myocardial T1-mapping, the traditional three-parameter based nonlinear regression may not provide accurate estimates due to sensitivity to noise. A deep neural network-based framework is proposed to address this issue. The DeepT1 framework consists of recurrent and U-net convolution networks to produce a single output map from the noisy and incomplete measurements. The results show that DeepT1 provides noise-robust estimates compared to the traditional pixel-wise three parameter fitting.

Medical image segmentation is one of the most important pre-processing steps in computer-aided diagnosis, but it is a challenging task because of the complex background and fuzzy boundary. To tackle these issues, we propose a double U-shape-based architecture named WNet, which is capable of capturing exact positions as well as sharpening their boundary. We first build an atlas-guided segmentation network (AGSN) to obtain a position-aware segmentation map by incorporating prior knowledge on human anatomy. We further devise a boundary-enhanced refinement network (BERN) to yield a clear boundary by hybridizing a Multi-scale Structure Similarity (MS-SSIM) loss function and making full use of refinement at training and inference in an end-to-end way. Experimental results show that the proposed WNet can accurately capture an organ with sharpened details and hence improves the performance on two datasets compared to the previous state-of-the-arts. Index Terms?Probabilistic atlas, Atlas-guided segmentation network, Boundary-enhanced refinement network.

Conventional single-shell diffusion MRI experiments acquire sampled values of the diffusion signal from the surface of a sphere in q-space. However, to reduce data acquisition time, there has been recent interest in using regularization to enable q-space undersampling. Although different regularization strategies have been proposed for this purpose (i.e., sparsity-promoting of the spherical ridgelet representation and Laplace-Beltrami Tikhonov regularization), there has not been a systematic evaluation of the strengths, weaknesses, and potential synergies of the different regularizers. In this work, we use real diffusion MRI data to systematically evaluate the performance characteristics of these different approaches and determine whether one approach is fundamentally more powerful than the other. Results from retrospective subsampling experiments suggest that both regularization strategies offer largely similar reconstruction performance (though with different levels of computational complexity) with some degree of synergy (albeit, relatively minor).

Although diffusion tensor imaging (DTI) can identify white matter (WM) changes due to mild traumatic brain injury (mTBI), the task of within-subject longitudinal matching of DTI streamlines remains challenging in this condition. Here we combine (A) automatic, atlas-informed labeling of WM streamline clusters with (B) streamline prototyping and (C) Riemannian matching of elastic curves to quantify within-subject changes in WM structure properties, focusing on the arcuate fasciculus. The approach is demonstrated in a group of geriatric mTBI patients imaged acutely and ~6 months post-injury. Results highlight the utility of differen-tial geometry approaches when quantifying brain connectivity alterations due to mTBI.

Multimodal CT scans, including non-contrast CT, CT perfusion, and CT angiography, are widely used in acute stroke diagnosis and therapeutic planning. While each imaging modality has its advantage in brain cross-sectional feature visualizations, the varying image resolution of different modalities hinders the ability of the radiologist to discern consistent but subtle suspicious findings. Besides, higher image quality requires a high radiation dose, leading to increases in health risks such as cataract formation and cancer induction. In this work, we propose a deep learning-based method Transfer-GAN that utilizes generative adversarial networks and transfer learning to improve multimodal CT image resolution and to lower the necessary radiation exposure. Through extensive experiments, we demonstrate that transfer learning from multimodal CT provides substantial visualization and quantity enhancement compare to the training without learning the prior knowledge.

Magnetic Resonance Imaging (MRI) is a noninvasive imaging technique that provides exquisite soft-tissue contrast without using ionizing radiation. The clinical application of MRI may be limited by long data acquisition times; therefore, MR image reconstruction from highly undersampled k-space data has been an active area of research. Many works exploit rank deficiency in a Hankel data matrix to recover unobserved k-space samples; the resulting problem is non-convex, so the choice of numerical algorithm can significantly affect performance, computation, and memory. We present a simple, scalable approach called Convolutional Framework (CF). We demonstrate the feasibility and versatility of CF using measured data from 2D, 3D, and dynamic applications.

This work explores a hybrid approach to segmentation as an alternative to a purely data driven approach. We introduce an end-to-end U-Net based network called DU-Net, which uses additional frequency preserving features, namely the Scattering Coefficients (SC) for medical image segmentation. SC are translation invariant and Lipschitz continuous to deformations which help DU-Net outperform other conventional CNN counterparts on four datasets and two segmentation tasks: Optic Disc and Optic Cup in color fundus images and fetal Head in ultrasound images. The proposed method shows remarkable improvement over the basic U-Net with performance competitive to state-of-the-art methods. The results indicate that it is possible to use a lighter network trained with fewer images (without any augmentation) to attain good segmentation results.

EEG recorded during MRI acquisition suffers from severe artifacts due to the imaging gradients. Here, we explore the possibility of using denoising autoencoders for correcting for these artifacts. After hyperparameter optimization, the performance of the algorithm was compared against PCA on two different synthesized datasets. The first dataset was created by adding a template artifact to clean EEG data and randomly shifting it in time to simulate aliasing. While the second dataset was formed by filtering out the EEG frequencies and adding a known ground-truth clean EEG signal. The performance of each method was assessed by the RMSE relative to the clean EEG signal. In addition, the correlation coefficient compared to the artifact signal was used to measure the residual artifact level. On the second synthesized dataset, denoising autoencoders outperformed PCA by 4.3% in terms of RMSE, meaning they were able to better preserve the original signal while at the same time the correlation with the underlying artifact was reduced by 40%. These preliminary results merit further investigation on a larger dataset.

We derive a method-independent universal precision bound for estimating the orientations of fluorescent molecules in the presence of Poisson shot noise using quantum detection and estimation theory. Combined with classical Fisher information and numerical analysis, we obtain a fundamental limit on the accuracy of measuring the ""wobble"", or rotational diffusion, of fluorescent molecules.

We investigated learned denoiser-based approximate message passing (LDAMP) with undersampled real MR measurements. In our preliminary results, LDAMP yielded favorable performance over BM3D-based AMP even though ground truth images are noisy and deep denoisers were trained only for Gaussian noise, not for undersampling artifacts. We further investigated the feasibility of using Stein?s unbiased risk estimator (SURE) to fine-tune deep denoisers with given undersampled MR measurement to reconstruct. Even though slight performance improvements (0.04dB) were observed for an example case, no visual improvement was observed.

We study the fully convolutional neural networks in the context of malignancy detection for breast cancer screening. We solve a supervised segmentation task looking for an acceptable compromise between the precision of the network and the computational complexity.

An approach based on the combination of femtosecond laser tissue removal, Optical Coherent Tomography (OCT) imaging and Mass Spectroscopic (MS) analysis is applied to cardiac tissue removal surgery. The spatial accuracy of the femtosecond laser tissue removal and the tissue specific MS fragmentation is demonstrated. The tissue specific regions of MS spectra are identified. The tissue classification utilizing Support Vector Machine was performed.

The image-based simulations in biomedicine play an important role as the real image data are difficult to be fully and precisely annotated. An increasing capability of contemporary computers allows to model and simulate reasonably complicated structures and in the last years also the dynamic processes. In this paper, we introduce a complex 3D model that describes the structure and dynamics of the population of endothelial cells. The model is based on standard cellular Potts model. It describes the formation process of a complex tubular network of endothelial cells fully in 3D together with the simulation of the cell death called apoptosis. The generated network imitates the structure and behavior that can be observed in real phase-contrast microscopy. The generated image data may serve as a benchmark dataset for newly designed detection or tracking algorithms.

The potential pattern changes in brain micro-structure can be used for the brain development assessment in children and adolescents by MRI scans. In this paper, we propose a highly accurate and efficient end-to-end brain age estimation network (BAENET) on T1-weighted MRI images. On the network, 3D skipping and outlier constraint loss are designed to accommodate deeper network and increase the robustness. Besides, we incorporate the neuroimage domain knowledge into stratified sampling for better generalization ability for datasets with different age distributions, and gender learning for more gender-specific features during modeling. We verify the effectiveness of the proposed method on the public ABIDE2 and ADHD200 benchmark, consisting of 382 and 378 normal children scans respectively. Our BAENET achieves MAE of 1.11 and 1.16, significantly outperforming the best reported methods by 5.1% and 9.4%.

Computer-aided polyp detection during colonoscopy is beneficial to reduce the risk of colorectal cancers. Deep learning techniques have made significant process in natural object detection. However, when applying those fully supervised methods to polyp detection, the performance is greatly depressed by the deficiency of labeled data. In this paper, we propose a novel bootstrapping method for polyp detection in colonoscopy videos by augmenting training data with temporal consistency. For a detection network that is trained on a small set of annotated polyp images, we fine-tune it with new samples selected from the test video itself, in order to more effectively represent the polyp morphology of current video. A strategy of selecting new samples is proposed by considering temporal consistency in the test video. Evaluated on 11954 endoscopic frames of the CVC-ClinicVideoDB dataset, our method yields great improvement on polyp detection for several detection networks, and achieves state-of-the-art performance on the benchmark dataset.

Deep learning systems are now being widely used to analyze lung cancer. However, recent work has shown a deep learning system can be easily fooled by intentionally adding some noise in the image. This is called as Adversarial attack. This paper presents an adversarial attack for malignancy prediction of lung nodules. We found that the adversarial attack can cause significant changes in lung nodule malignancy prediction accuracy. An ensemble-based defense strategy was developed to reduce the effect of an adversarial attack. A multi-initialization based CNN ensemble was utilized. We also explored adding adversarial images in the training set, which eventually reduced the rate of mis-classification and made the CNN models more robust to an adversarial attack. A subset of cases from the National Lung Screening Trial (NLST) dataset were used in our study. Initially, 75.1%, 75.5% and 76% classification accuracy were obtained from the three CNNs on original images (without an adversarial attack). Fast Gradient Sign Method (FGSM) and one-pixel attacks were analyzed. After the FGSM attack, 46.4%, 39.24%, and 39.71% accuracy was obtained from the 3 CNNs. Whereas, after a one pixel attack 72.15%, 73%, and 73% classification accuracy was achieved. FGSM caused much more damaged to CNN prediction. With a multi-initialization based ensemble and including adversarial images in the training set, 82.27% and 81.43% classification accuracy were attained after FGSM and one-pixel attacks respectively.

Learning-based approaches, especially those based on deep networks, have enabled high-quality estimation of tissue microstructure from low-quality diffusion magnetic resonance imaging (dMRI) scans, which are acquired with a limited number of diffusion gradients and a relatively poor spatial resolution. These learning-based approaches to tissue microstructure estimation require acquisitions of training dMRI scans with high-quality diffusion signals, which are densely sampled in the q-space and have a high spatial resolution. However, the acquisition of training scans may not be available for all datasets. Therefore, we explore knowledge transfer between different dMRI datasets so that learning-based tissue microstructure estimation can be applied for datasets where training scans are not acquired. Specifically, for a target dataset of interest, where only low-quality diffusion signals are acquired without training scans, we exploit the information in a source dMRI dataset acquired with high-quality diffusion signals. We interpolate the diffusion signals in the source dataset in the q-space using a dictionary-based signal representation, so that the interpolated signals match the acquisition scheme of the target dataset. Then, the interpolated signals are used together with the high-quality tissue microstructure computed from the source dataset to train deep networks that perform tissue microstructure estimation for the target dataset. Experiments were performed on brain dMRI scans with low-quality diffusion signals, where the benefit of the proposed strategy is demonstrated.

Deep learning methods are emerging as powerful alternatives for compressed sensing MRI to recover images from highly undersampled data. Unlike compressed sensing, the image redundancies that are captured by these models are not well understood. The lack of theoretical understanding also makes it challenging to choose the sampling pattern that would yield the best possible recovery. To overcome these challenges, we propose to optimize the sampling patterns and the parameters of the reconstruction block in a model-based deep learning framework. We show that the joint optimization by the model-based strategy results in improved performance than direct inversion CNN schemes due to better decoupling of the effect of sampling and image properties. The quantitative and qualitative results confirm the benefits of joint optimization by the model-based scheme over the direct inversion strategy.

Acute aortic syndrome (AAS) is a group of life threatening conditions of the aorta. We have developed an end-to-end automatic approach to detect AAS in computed tomography (CT) images. Our approach consists of two steps. At first, we extract $N$ cross sections along the segmented aorta centerline for each CT scan. These cross sections are stacked together to form a new volume which is then classified using two different classifiers, a 3D convolutional neural network (3D CNN) and a multiple instance learning (MIL). We trained, validated, and compared two models on 2291 contrast CT volumes. We tested on a set aside cohort of 230 normal and 50 positive CT volumes. Our models detected AAS with an Area under Receiver Operating Characteristic curve (AUC) of 0.965 and 0.985 using 3DCNN and MIL, respectively.

Although thyroid ultrasonography (US) has been widely applied, it is still difficult to distinguish benign and malignant nodules. Currently, convolutional neural network (CNN) based methods have been proposed and shown promising performance for benign and malignant nodules classification. It is known that the US images are usually captured in multi-angles, and the same thyroid in different US images have inconsistent content. However, most of the existing CNN based methods extract features using fixed convolution kernels, which could be a big issue for processing US images. Moreover, fully-connected (FC) layers are usually adopted in CNN, which could cause the loss of inter-pixel relations. In this paper, we propose a new CNN which is integrated with squeeze-and-excitation (SE) module and maximum retention of inter-pixel relations module (CNN-SE-MPR). It can adaptively select features from different US images and preserve the inter-pixel relations. Moreover, we introduce transfer learning to avoid problems such as local optimum and data insufficiency. The proposed network is tested on 407 thyroid US images collected from cooperated hospitals. Confirmed by ablation experiments and the comparison experiments under the state-of-the-art methods, it is shown that our method improves the accuracy of the diagnosis results.

2D-US to 3D-CT/MR registration is a crucial module during minimally invasive ultrasound-guided liver tumor ablations. Many modern registration methods still require manual or semi-automatic slice pose initialization due to insufficient robustness of automatic methods. The state-of-the-art regression networks do not work well for liver 2D US to 3DCT/MR registration because of the tremendous inter-patientvariability of the liver anatomy. To address this unsolved problem, we propose a deep learning network pipeline which? instead of a regression ? starts with a classification network to recognize the coarse ultrasound transducer pose followed by a segmentation network to detect the target plane of the US image in the CT/MR volume. The rigid registration result is derived using plane regression. In contrast to the state-of-the-art regression networks, we do not estimate registration parameters from multi-modal images directly, but rather focus on segmenting the target slice plane in the volume. The experiments reveal that this novel registration strategy can identify the initial slice phase in a 3D volume more reliably than the standard regression-based techniques. The proposed method was evaluated with 1035 US images from 52 patients. We achieved angle and distance errors of 12.7?6.2? and 4.9?3.1 mm, clearly outperforming state-of-the-art re-gression strategy which results in 37.0?15.6? angle error and 19.0?11.6 mm distance error.

A 3D deep learning-based convolution neural network (CNN)is developed for accurate segmentation of pathological bladder(both wall border and pathology) using T2-weighted magnetic resonance imaging (T2W-MRI). Our system starts with a preprocessing step for data normalization to a unique space and extraction of a region-of-interest (ROI). The major stage utilizes a 3D CNN for pathological bladder segmentation, which contains a network, called CNN1, aims to segment the bladder wall (BW) with pathology. However, due to the similar visual appearance of BW and pathology, the CNN1 can not separate them. Thus, we developed another network (CNN2) with an additional pathway to extract BW only. The second pathway in CNN2 is fed with a 3Dlearnable adaptive shape prior model. To remove noisy and scattered predictions, the networks? soft outputs are refined using a fully connected conditional random field. Our framework achieved accurate segmentation results for the BW and tumor as documented by the Dice similarity coefficient and Hausdorff distance. Moreover, comparative results against the other segmentation approach documented the superiority of our framework to provide accurate results for pathological BW segmentation.

Speed of sound (SOS) is a biomarker that aides clinicians in tracking the onset and progression of diseases such as breast cancer and fatty liver disease. In this paper, we propose a framework to generate accurate, 2D SOS maps with a commercial ultrasound probe. We simulate freehand ultrasound probe motion and use a multi-look framework for reflection travel time tomography. In these simulations, the ``measured'' travel times are computed using a bent-ray Eikonal solver and direct inversion for compressional speed of sound is performed. We have shown that the assumption of straight rays breaks down for large velocity perturbations (greater than 1 percent). The error increases 70 fold for a velocity perturbation increase of 1.5 percent. Moreover, the use of multiple looks greatly aides the inversion process. Simulated RMSE drops by roughly 15 dB when the maximum scanning angle is increased from 0 to 45 degrees.

Several studies have explored the relationship between intelligence and neuroimaging features. However, little is known about whether the temporal variations of functional connectivity of the brain regions at rest are relevant to the differences in intelligence. In this study, we have used the fMRI data and intelligence scores of 50 healthy adult subjects from the Human Connectome Project (HCP) database. We have investigated the correlation between individual intelligence scores and the total power of the high frequency components of the Fast Fourier transform (FFT) of the dynamic functional connectivity time series of the brain regions. We have found temporal variations of specific functional connections highly correlated with the individual intelligence scores. In other words, functional connections of individuals with high levels of the intelligence have smoother temporal variation or higher temporal stability than those of the individuals with low intelligence levels.

Although deep learning systems trained on medical images have shown state-of-the-art performance in many clinical pre- diction tasks, recent studies demonstrate that these systems can be fooled by carefully crafted adversarial images. It has raised concerns on the practical deployment of deep learning based medical image classification systems. To tackle this problem, we propose an unsupervised learning approach to detect adversarial attacks on medical images. Our approach is capable of detecting a wide range of adversarial attacks without knowing the attackers nor sacrificing the classification performance. More importantly, our approach can be easily embedded into any deep learning-based medical imaging system as a module to improve the system?s robustness. Experiments on a public chest X-ray dataset demonstrate the strong performance of our approach in defending adversarial attacks under both white-box and black-box settings.

In the neuroimaging and brain mapping communities, researchers have proposed a variety of computational methods and tools to learn functional brain networks (FBNs). Recently, it has already been proven that deep learning can be applied on fMRI data with superb representation power over traditional machine learning methods. Limited by the high-dimension of fMRI volumes, deep learning suffers from the lack of data and overfitting. Generative models are known to have intrinsic ability of modeling small dataset and a deep variational autoencoder (DVAE) was proposed in this work to tackle the challenge of insufficient data and incomplete supervision. The FBNs learned from fMRI were examined to be interpretable and meaningful and it was proven that DVAE has better performance on neuroimaging dataset over traditional models. With an evaluation on ADHD200 dataset, DVAE performed excellent on classification accuracies on 4 sites.

We propose an algorithm based on marker-controlled watershed and the monogenic signal phase asymmetry for the segmentation of bone and micro-vessels in mouse bone. The images are acquired using synchrotron radiation micro-computed tomography (SR-?CT). The marker image is generated with hysteresis thresholding and morphological filters. The control surface is generated using the phase asymmetry of the monogenic signal in order to detect edge-like structures only, as well as improving detection in low contrast areas, such as bone-vessel interfaces. The quality of segmentation is evaluated by comparing to manually segmented images using the Dice coefficient. The proposed method shows substantial improvement compared to a previously proposed method based on hysteresis thresholding, as well as compared to watershed using the gradient image as control surface. The algorithm was applied to images of healthy and metastatic bone, permitting quantification of both bone and vessel structures.

Cerebral blood volume (CBV) is a hemodynamic correlate of oxygen metabolism and reflects brain activity and function. High-resolution CBV maps can be generated using the steady-state gadolinium-enhanced MRI technique. Such technique requires an intravenous injection of exogenous gadolinium based contrast agent (GBCA) and recent studies suggest that the GBCA can accumulate in the brain after frequent use. We hypothesize that endogenous sources of contrast might exist within the most conventional and commonly acquired structural MRI, potentially obviating the need for exogenous contrast. Here, we test this hypothesis by developing and optimizing a deep learning algorithm, which we call DeepContrast, in mice. We find that DeepContrast performs equally well as exogenous GBCA in mapping CBV of the normal brain tissue and enhancing glioblastoma. Together, these studies validate our hypothesis that a deep learning approach can potentially replace the need for GBCAs in brain MRI.

In this paper, we present a deep learning based image processing technique for extraction of bone structures in chest radiographs using a U-Net FCNN. The U-Net was trained to accomplish the task in a fully supervised setting. To create the training image pairs, we employed simulated X-Ray or Digitally Reconstructed Radiographs (DRR), derived from 664 CT scans belonging to the LIDC-IDRI dataset. Using HU based segmentation of bone structures in the CT domain,a synthetic 2D ?Bone x-ray? DRR is produced and used for training the network. For the reconstruction loss, we utilize two loss functions- L1 Loss and perceptual loss. Once the bone structures are extracted, the original image can be enhanced by fusing the original input x-ray and the synthesized ?Bone X-ray?. We show that our enhancement technique is applicable to real x-ray data, and display our results on the NIH Chest X-Ray-14 dataset.

Optical coherence tomography (OCT) plays an important role in retinal disease screening. Traditional classification-based screening methods require complicated annotation works. Due to the difficulty of collecting abnormal samples, some anomaly detection methods have been applied to screen retinal lesions only based on normal samples. However, most existing anomaly detection methods are time consuming and easily misjudging abnormal OCT images with implicit lesions like small drusen. To solve these problems, we propose a memory-augmented anomaly generative adversarial network (MA-GAN) for retinal OCT screening. Within the generator, we establish a memory module to enhance the detail expressing abilities of typical OCT normal patterns. Meanwhile, the discriminator of MA-GAN is decomposed orthogonally so that it has the encoding ability simultaneously. As a result, the abnormal image can be screened by the greater difference in the distribution of pixels and features between the original image and its reconstructed image. The model trained with 13000 normal OCT images reaches 0.875 AUC on the test set of 2000 normal images and 1000 anomalous images. And the inference time only takes 35 milliseconds for each image. Compared to other anomaly detection methods, our MA-GAN has the advantages in model accuracy and computation time for retinal OCT screening.

Lung biomechanical properties can be used to detect disease, assess abnormal lung function, and track disease progression.In this work, we used computed tomography (CT) imaging to measure three biomechanical properties in the lungs of subjects with varying degrees of chronic obstructive pulmonary disease (COPD): the Jacobian determinant (J), a measure of volumetric expansion or contraction; the anisotropic deformation index (ADI), a measure of the magnitude of anisotropic deformation; and the the slab-rod index (SRI), a measure of the nature of anisotropy (i.e., whether the volume is deformed to a rod-like or slab-like shape). We analyzed CT data from247 subjects collected as part of the Subpopulations and Inter-mediate Outcome Measures in COPD Study (SPIROMICS). The results show that the mean J and mean ADI decrease as disease severity increases, indicating less volumetric expansion and more isotroic expansion with increased disease. No differences in average SRI index were observed across the different levels of disease. The methods and analysis described in this study may provide new insights into our understanding of the biomechanical behavior of the lung and the changesthat occur with COPD.

We develop a fully automated airway segmentation method to segment the vocal tract airway from surrounding soft tissue in speech MRI. We train a U-net architecture to learn the end to end mapping between a mid-sagittal image (at the input), and the manually segmented airway (at the output). We base our training on the open source University of Southern California?s (USC) speech morphology MRI database consisting of speakers producing a variety of sustained vowel and consonant sounds. Once trained, our model performs fast airway segmentations on unseen images at the order of 210 ms/slice on a modern CPU with 12 cores. Using manual segmentation as a reference, we evaluate the performances of the proposed U-net airway segmentation, against existing seed-growing segmentation, and manual segmentation from a different user. We demonstrate improved DICE similarity with U-net compared to seed-growing, and minor differences in DICE similarity of U-net compared to manual segmentation from the second user.

Histology-based grade classification is clinically important for many cancer types in stratifying patients into distinct treatment groups. In prostate cancer, the Gleason score is a grading system used to measure the aggressiveness of prostate cancer from the spatial organization of cells and the distribution of glands. However, the subjective interpretation of Gleason score often suffers from large interobserver and intraobserver variability. Previous work in deep learning-based objective Gleason grading requires manual pixel-level annotation. In this work, we propose a weakly-supervised approach for grade classification in tissue micro-arrays (TMA) using graph convolutional networks (GCNs), in which we model the spatial organization of cells as a graph to better capture the proliferation and community structure of tumor cells. We learn the morphometry of each cell using a contrastive predictive coding (CPC)-based self-supervised approach. Using five-fold cross-validation we demonstrate that our method can achieve a 0.9637?0.0131 AUC using only TMA-level labels. Our method also demonstrates a 36.36% improvement in AUC over standard GCNs with texture features and a 15.48% improvement over GCNs with VGG19 features. Our proposed pipeline can be used to objectively stratify low and high-risk cases, reducing inter- and intra-observer variability and pathologist workload.

Custom double-tuned birdcage coils were constructed to enable concurrent evaluation of a number of NMR indices in the golden retriever muscular dystrophy (GRMD) model of Duchenne muscular dystrophy (DMD). Seven rectus femoris muscle samples from dogs with ages ranging from 3 to 30 months were studied. 1H T1-weighted (T1w) and T2-weighted (T2w) images, 23Na images, and 31P spectra were acquired for each sample. 1H T1w and T2w images showed a decrease in T2w/T1w signal ratio for the four older (=12 months) samples when compared to younger samples. Other NMR indices unexpectedly showed no significant correlation with age. The collection time of samples and varying levels of disease severity may have attributed to these results. Regardless, the associated custom coils and positioner developed to enable multi-nuclear studies will enable future work to investigate NMR-based biomarkers in the numerous GRMD samples available to our group.

Functional magnetic resonance imaging (fMRI) is commonly used to better understand brain function. Data becomes contaminated with motion artifact when a subject moves during an fMRI acquisition. Numerous methods have been suggested to target motion artifacts in fMRI. One of these methods, ?scrubbing?, removes motion-corrupted volumes but must be performed after temporal filtering since it creates temporal discontinuities. Thus, it does not prevent the spread of corrupted time samples from high motion volumes to their neighbors during temporal filtering. To mitigate this spread, which we refer to as ``leakage'', we propose a novel method, Dynamic Missing-data Completion (DMC), that replaces motion-corrupted volumes with synthetic data before temporal filtering. We analyzed the effect of DMC on an exemplary timeseries from a resting state fMRI (rsfMRI) and compared functional connectivity results of six rsfMRI scans from a single subject with different levels of subject motion. Our results suggest that DMC provides added benefit in further reduction of motion contamination that remains after scrubbing. DMC reduced the standard deviation of signal near scrubbed volumes by about 10% compared to scrubbing only, yielding this average closer to that of uncorrupted no motion volumes.

Recent advancements in biomedical imaging focus on fluorescent imaging using laser scanning confocal microscopy. However, high-resolution imaging of cellular activity remains considerably expensive for both in vitro and in vivo model. In this context, integration of mathematical modeling and imaging data analysis to predict the cellular activity may aid understanding of cell signaling. Here we performed dynamic imaging using confocal microscopy and propose a model considering cell to cell connectivity that can predict the effect of drug on Ca2+ oscillations. The proposed model consists of large number of ordinary differential (ODE) equations and uses the concept of adjoint matrix containing coupling factors to capture the activity of cells with random arrangement. The results show that the cell-to-cell connection plays a crucial role in controlling the calcium oscillations through a diffusion-based mechanism. The present simulation tool can be used as generalized framework to generate and visualize the time-lapse videos required for in vitro drug testing for various drug doses.

In ophthalmology, color fundus photography is an economic and effective tool for early-stage ocular disease screening. Since the left and right eyes are highly correlated, we utilize paired color fundus photographs for our task of automated multi-label ocular disease detection. We propose a Dense Correlation Network (DCNet) to exploit the dense spatial correlations between the paired CFPs. Specifically, DCNet is composed of a backbone Convolutional Neural Network (CNN), a Spatial Correlation Module (SCM), and a classifier. The SCM can capture the dense correlations between the features extracted from the paired CFPs in a pixel-wise manner, and fuse the relevant feature representations. Experiments on a public dataset show that our proposed DCNet can achieve better performance compared to the respective baselines regardless of the backbone CNN architectures.

We have developed a three-dimensional (3D) convolutional autoencoder (CAE) in conjunction with the exploratory factor analysis to predict pulmonary functions and discover possible tissue patterns which are associated with alterations of pulmonary functions.

OCT of retina indicates presence of macular edema, variation in thickness of different layers of retina and choroid. The size of the edema and the thickness of the choroid layers can be ascertained by proper segmentation of the OCT images. In this work, we propose a model using Convolutional Neural Network (CNN) similar to the SegNet architecture for segmenting choroid layers and edema in OCT images. Our CNN model is an encoder - decoder architecture designed specifically for pixel wise classification of images where boundary delineation is vital. To this end, first we train the CNN to obtain the pixel wise label of the choroid and edema. Once these labels are obtained, the pixel labels are converted into binary segmentation using morphological operations followed by edge detection. The results show good consistency when compared with the ophthalmologist's labeling. The idea can be extended to develop a tool to detect retinal disorders automatically.

Machine learning and especially deep learning hold great promise to improve patient care. In several domains, algorithms perform as good as or better than fellowship trained radiologists for identification of abnormalities in clinically acquired images. However, there are much broader applications beyond image analysis such as patient selection and examination scheduling, image acquisition and reconstruction, using image data for prognostic purposes, and combing image data with information from electronic health records, laboratory and genetic data. Furthermore, in order for algorithms to be broadly accepted, there are many scenarios where it is important for the clinician that results are explainable. In addition, clinical deployment and workflow should be taken into consideration when designing the algorithm and bringing it to clinical practice. In my lecture I will focus on these aspects from a cardiovascular imaging perspective. [1] Hampe N, Wolterink JM, van Velzen SGM, Leiner T, I?gum I. Machine Learning for Assessment of Coronary Artery Disease in Cardiac CT: A Survey. Front Cardiovasc Med, 26 November 2019 https://doi.org/10.3389/fcvm.2019.00172 [2] Leiner T, Rueckert D, Suinesiaputra A, Bae?ler B, Nezafat R, I?gum I, Young AA. Machine learning in cardiovascular magnetic resonance: basic concepts and applications. J Cardiovasc Magn Reson 21(1):61, 2019.

The segmentation of hippocampus subfields from MR images is challenging due to the small size of the organ, multiple interconnected regions, and low contrast. Motivated by the success of active shape models in segmentation, we propose a shape constrained CNN algorithm. We use a two-step strategy, where we first use an auto-encoder CNN to learn the anatomical shape priors from labels. After shape training, the decoder compactly represents the shape manifold; it generates the labels from a compact code. We then train an encoder CNN to predict the code from MR images. The decoder (shape generator) is fixed during the segmentation training.

Tissue microarray (TMA) assessment of histomorphological biomarkers contributes to more accurate prediction of outcome of patients with colorectal cancer (CRC), a common disease. Unfortunately, a typical problem with the use of TMAs is that the material contained in each TMA spot changes as the TMA block is cut repeatedly. A re-classification of the content within each spot would be necessary for accurate biomarker evaluation. The major challenges to this end however lie in the high heterogeneity of TMA quality and of tissue characterization in structure, size, appearance and tissue type. In this work, we propose an end-to-end framework using deep learning for TMA spot classification into three classes: tumor, normal epithelium and other tissue types. It includes a detection of TMA spots in an image, an extraction of overlapping tiles from each TMA spot image and a classification integrated into two effective deep learning architectures: convolutional neural network (CNN) and Capsule Network with the prior information of intended tissue type. A set of digitized H&E stained images from 410 CRC patients with clinicopathological information is used for the validation of the proposed method. We show experimentally that our approach brings state-of-the-art performances for several relevant CRC H&E tissue classification and that these results are promising for use in clinical practice.

Conventional methods designed for Mild Cognitive Impairment (MCI) classification usually assume that the MCI subjects are homogeneous. However, recent discoveries indicate that MCI has heterogeneous neuropathological origins which may contribute to the sub-optimal performance of conventional methods. To compensate for the limitations of existing methods, we propose Maximum Margin Heterogeneous Feature Selection (MMHFS) by explicitly considering the heterogeneous distribution of MCI data. More specifically, the proposed method simultaneously performs unsupervised clustering discovery on MCI data and conducts discriminant feature selection to help classify MCI from Normal Control (NC). It is worth noting that these two processes can benefit from each other, thus enabling the proposed method to achieve better performance.

Epilepsy is a debilitating neurological disorder that directly impacts millions of people and exerts a tremendous economic burden on society at large. While traumatic brain injury (TBI) is a common cause, there remain many open questions regarding its pathological mechanism. The goal of the Epilepsy Bioinformatics Study for Antiepileptogenic Therapy (EpiBioS4Rx) is to identify epileptogenic biomarkers through a comprehensive project spanning multiple species, modalities, and research institutions; in particular, diffusion magnetic resonance imaging (MRI) is a critical component, as it probes tissue microstructure and structural connectivity. The project includes in vivo imaging of a rodent fluid-percussion model of TBI, and we developed a computational diffusion MRI framework for EpiBioS4Rx which employs advanced techniques for preprocessing, modeling, spatial normalization, region analysis, and tractography to derive imaging metrics at group and individual levels. We describe the system's design, present characteristic results from a longitudinal cohort, and discuss its role in biomarker discovery and further studies.

Optical coherence tomography (OCT) is a micrometer-resolution, cross-sectional imaging modality for biological tissue. It has been widely applied for retinal imaging in ophthalmology. However, the large speckle noise affects the analysis of OCT retinal images and their diagnostic utility. In this article, we present a new speckle reduction algorithm for 3D OCT images. The OCT speckle noise is approximated as Poisson distribution, which is dif?cult to be removed for its signal-dependent characteristic. Thus our algorithm is consisted by two steps: ?rst, a variance-stabilizing trans-formation, named Anscombe transformation, is applied to redistribute the multiplicative speckle noise into an additive Gaussian noise; then the transformed data is decomposed and ?ltered in 3D Shearlet domain, which provides better representation of the edge information of the retinal layers than wavelet and curvelet. The proposed method is evaluated through the three parameters using high-de?nition B-scans as the ground truth. Quantitative experimental results show that our method gives out the best evaluation parameters, and highest edge contrast, compared with state-of-the-art OCT denoising algorithms.

Semi-supervised approaches have been developed to improve brain lesion segmentation based on convolutional neural networks (CNNs) when annotated data is scarce. Existing methods have exploited unannotated images with lesions to improve the training of CNNs. In this work, we explore semi-supervised brain lesion segmentation by further incorporating images without lesions. Specifically, using information learned from annotated and unannotated scans with lesions, we propose a framework to generate synthesized lesions and their annotations simultaneously. Then, we attach them to normal-appearing scans using a statistical model to produce synthesized training samples, which are used together with true annotations to train CNNs for segmentation. Experimental results show that our method outperforms competing semi-supervised brain lesion segmentation approaches.

Herein we propose a deep learning-based approach for the prediction of lung lesion response based on radiomic features extracted from clinical CT scans of patients in non-small cell lung cancer trials. The approach starts with the classification of lung lesions from the set of primary and metastatic lesions at various anatomic locations. Focusing on the lung lesions, we perform automatic segmentation to extract their 3D volumes. Radiomic features are then extracted from the lesion on the pre-treatment scan and the first follow-up scan to predict which lesions will shrink at least 30% in diameter during treatment (either pembrolizumab or combinations of chemotherapy and pembrolizumab), which is defined as a partial response by the Response Evaluation Criteria In Solid Tumors (RECIST) guidelines. A 5-fold cross validation on the training set led to an AUC of 0.84 ? 0.03, and the prediction on the testing dataset reached AUC of 0.73 ? 0.02 for the outcome of 30% diameter shrinkage.

It is vital to automatically detect the Extramural Vascular Invasion (EMVI) in rectal cancer before surgery, which facilitates to guide the patient's treatment planning. Nevertheless, there are few studies about EMVI detection through magnetic resonance imaging (MRI). Moreover, since EMVI has three main characteristics: highly-variable appearances, relatively-small sizes and similar shapes with surrounding tissues, current deep learning based methods can not be directly used. In this paper, we propose a novel and efficient EMVI detection framework, which gives rise to three main contributions. Firstly, we introduce a self-attention module to capture dependencies ranging from local to global. Secondly, we design a parallel atrous convolution (PAC) block and a global pyramid pooling (GPP) module to encode richer context information at multiple scales. Thirdly, we fuse the whole-scene and local-region information together to improve the feature representation ability. Experimental results show that our framework can significantly improve the detection accuracy and outperform other state-of-the-art methods.