IEEE ISBI 2020 Virtual Conference April 2020

Restoring the tissue reflectivity function (TRF) from ultrasound (US) images is an extensively explored research field. It is well-known that human tissues and contrast agents have a non-linear behavior when interacting with US waves. In this work, we investigate this non-linearity and the interest of including harmonic US images in the TRF restoration process. Therefore, we introduce a new US image restoration method taking advantage of the fundamental and harmonic components of the observed radiofrequency (RF) image. The depth information contained in the fundamental component and the good resolution of the harmonic image are combined to create an image with better properties than the fundamental and harmonic images considered separately. Under the hypothesis of weak scattering, the RF image is modeled as the 2D convolution between the TRF and the system point spread function (PSF). An inverse problem is formulated based on this model able to jointly estimate the TRF and the PSF. The interest of the proposed blind deconvolution algorithm is shown through an in vivo result and compared to a conventional US restoration method.

The detection of signet ring cells in histology images is of great value in clinical practice. However, several reasons such as appearance variations and lack of well-labelled data make it a challenging task. Considering the intrinsic characteristics of signet ring cell images, a dedicated similarity learning network is designed in this paper to help the discovery of distinctive feature representations for ring cells. Specifically, we adapt the region proposal network and add an embedding layer to enable similarity learning for training the model. Experimental results show that similarity learning can strengthen the performance of the state-of-the-art and makes our approach competent for the task of signet ring cell detection.

We propose a novel deep neural network architecture for normalcy detection in chest x-ray images. This architecture treats the problem as fine-grained binary classification in which the normal cases are well-defined as a class while leaving all other cases in the broad class of abnormal. It employs several components that allow generalization and prevent overfitting across demographics. The model is trained and validated on a large public dataset of frontal chest X-ray images. It is then tested independently on images from a clinical institution of differing patient demographics using a three radiologist consensus for ground truth labeling. The model provides an area under ROC curve of 0.96 when tested on 1271 images. Using this model, we can automatically remove nearly a third of disease-free chest X-ray screening images from the workflow, without introducing any false negatives thus raising the potential of expediting radiology workflows in hospitals in future.

Scanning transmission X-ray microscopy (STXM) produces images in which each pixel value is related to the measured attenuation of an X-ray beam. In practice, the location of the illuminated region does not exactly match the desired uniform pixel grid. This error can be measured using an interferometer. In this paper, we propose a spline-based reconstruction method for STXM which takes these position errors into account. We achieve this by formulating the reconstruction problem as a continuous-domain inverse problem in a spline basis, and by using Hessian nuclear-norm regularization. We solve this problem using the standard ADMM algorithm, and we demonstrate the pertinence of our approach on both simulated and real STXM data.

In this paper, we propose a novel approach for manifold learning that combines the Earthmover's distance (EMD) with the diffusion maps method for dimensionality reduction. We demonstrate the potential benefits of this approach for learning shape spaces of proteins and other flexible macromolecules using a simulated dataset of 3-D density maps that mimic the non-uniform rotary motion of ATP synthase. Our results show that EMD-based diffusion maps require far fewer samples to recover the intrinsic geometry than the standard diffusion maps algorithm that is based on the Euclidean distance. To reduce the computational burden of calculating the EMD for all volume pairs, we employ a wavelet-based approximation to the EMD which reduces the computation of the pairwise EMDs to a computation of pairwise weighted-l1 distances between wavelet coefficient vectors.

Extracting multi-scale information is key to semantic segmentation. However, the classic convolutional neural networks (CNNs) encounter difficulties in achieving multi-scale information extraction: expanding convolutional kernel incurs the high computational cost and using maximum pooling sacrifices image information. The recently developed dilated convolution solves these problems, but with the limitation that the dilation rates are fixed and therefore the receptive field cannot fit for all objects with different sizes in the image. We propose an adaptive-scale convolutional neural network (ASCNet), which introduces a 3-layer convolution structure in the end-to-end training, to adaptively learn an appropriate dilation rate for each pixel in the image. Such pixel-level dilation rates produce optimal receptive fields so that the information of objects with different sizes can be extracted at the corresponding scale. We compare the segmentation results using the classic CNN, the dilated CNN and the proposed ASCNet on two types of medical images (The Herlev dataset and SCD RBC dataset). The experimental results show that ASCNet achieves the highest accuracy. Moreover, the automatically generated dilation rates are positively correlated to the sizes of the objects, confirming the effectiveness of the proposed method.

Deep neural networks are being increasingly used for disease diagnosis and lesion localization in biomedical images. However, training deep neural networks not only requires large sets of expensive ground truth (image labels or pixel annotations), they are also susceptible to adversarial attacks. Transfer learning alleviates the former problem to some extent by pre-training the lower layers of a neural network on a large labeled dataset from a different domain (e.g., ImageNet). In transfer learning, the final few layers are trained on the target domain (e.g., chest X-rays), while the pre-trained layers are only fine-tuned or even kept frozen. An alternative to transfer learning is self-supervised learning in which a supervised task is created by transforming the unlabeled images from the target domain itself. The lower layers are pre-trained to invert the transformation in some sense. In this work, we show that self-supervised learning combined with adversarial training offers additional advantages over transfer learning as well as vanilla self-supervised learning. In particular, the process of adversarial training leads to both a reduction in the amount of supervised data required for comparable accuracy, as well as a natural robustness to adversarial attacks. We support our claims using experiments on the two modalities and tasks -- classification of chest X-rays, and segmentation of MRI images, as well as two types of adversarial attacks -- PGD and FGSM.

Data augmentation is a popular technique with which new dataset samples are artificially synthesized to the end of aiding training of learning-based algorithms and avoiding overfitting. Methods based on Generative adversarial networks (GANs) have recently rekindled interest in research on new techinques for data augmentation. With the current paper we propose a new GAN-based model for data augmentation, comprising a suitable Markov Random Field-based spatial constraint that encourages synthesis of spatially smooth outputs. Oriented towards use with medical imaging sets where a localization/segmentation annotation is available, our model can simultaneously also produce artificial annotations. We gauge performance numerically by measuring performance of U-Net trained to detect cells on microscopy images, by taking into account the produced augmented dataset. Numerical trials, as well as qualitative results validate the usefulness of our model.

Reliable attenuation and scatter compensation (ASC) is a pre-requisite for quantification tasks and beneficial for visual interpretation tasks in single-photon emission computed tomography (SPECT) imaging. For this purpose, we develop a SPECT reconstruction method that uses the entire SPECT emission data, i.e. data in both the photopeak and scattered windows, and acquired in list-mode format, to perform ASC. Further, the method uses the energy attribute of the detected photons while performing the reconstruction. We implemented a GPU-version of this method using an ordered subsets expectation maximization (OSEM) algorithm for faster convergence and quicker computation. The performance of the method was objectively evaluated using realistic simulation studies on the task of estimating activity uptake in the caudate, putamen, and globus pallidus regions of the brain in a dopamine transporter (DaT)-SPECT study. The method yielded improved performance in terms ob bias, variance, and mean square error compared to existing ASC techniques in quantifying activity in all three regions. Overall, the results provide promising evidence of the potential of the proposed technique for ASC in SPECT imaging.

The study of protein transport in living cell requires automated techniques to capture and quantify dynamics of the protein packaged into secretory vesicles. The movement of the vesicles is not consistent along the trajectory, therefore the quantitative study of their dynamics requires trajectories segmentation. This paper explores quantification of such vesicle dynamics and introduces a novel 1D U-Net based trajectory segmentation. Unlike existing mean squared displacement based methods, our proposed framework is not restricted under the requirement of long trajectories for effective segmentation. Moreover, as our approach provides segmentation within each sliding window, it enables effectively capture even short segments. The approach is quantified by the data acquired from spinning disk microscopy imaging of protein trafficking in Drosophila epithelial cells. The extracted trajectories have lengths ranging from 5 (short tracks) to 135 (long tracks) points. The proposed approach achieves 77.7% accuracy for the trajectory segmentation.

We consider adaptive weighted minimax-concave (WMC) penalty as a generalization of the minimax-concave penalty (MCP) and vector MCP (VMCP). We develop a computationally efficient algorithm for sparse recovery considering the WMC penalty. Our algorithm in turn employs the fast iterative soft-thresholding algorithm (FISTA) but with the key difference that the threshold is adapted from one iteration to the next. The new sparse recovery algorithm when used for dictionary learning has a better representation capability as demonstrated by an application to magnetic resonance image denoising. The denoising performance turns out to be superior to the state-of-the-art techniques considering the standard performance metrics namely peak signal-to-noise ratio (PSNR) and structural similarity index metric (SSIM).

Predicting individual chronological age based on neuroimaging data is very promising and important for understanding the trajectory of normal brain development. In this work, we proposed a new model to predict brain age ranging from 12 to 30 years old, based on structural magnetic resonance imaging and a deep learning approach with reduced model complexity and computational cost. We found that this model can predict brain age accurately not only in the training set (N = 1721, mean absolute error is 1.89 in 10-fold cross validation) but in an independent validation set (N = 226, mean absolute error is 1.96), substantially outperforming the previous published models. Given the considerable accuracy and generalizability, it is promising to further deploy our model in the clinic and help to investigate the pathophysiology of neurodevelopmental disorders.

In this work, a fusion framework is proposed for automatic cancer detection and segmentation in whole-slide histopathology images. The framework includes two parts of fusion, multi-scale fusion, and sub-datasets fusion. For a particular type of cancer, histopathological images often demonstrate large morphological variances, the performance of an individual trained network is usually limited. We develop a fusion model that integrates two types of U-net structures: Shallow U-net (SU-net) and Deep U-net (DU-net), trained with a variety of multiple re-scaled images and different subsets of images, and finally ensemble a unified output. Smoothing and noise elimination are conducted using convolutional Conditional Random Fields (CRFs)cite{CRF}. The proposed model is validated on Automatic Cancer Detection and Classification in Whole-slide Lung Histopathology (ACDC@LungHP) challenge in ISBI2019 and Digestive-System Pathological Detection and Segmentation Challenge 2019(DigestPath 2019) in MICCAI2019. Our method achieves a dice coefficient of 0.7968 in ACDC@LungHP and 0.773 in DigestPath 2019, and the result of ACDC@LungHP challenge is ranked in third place on the board.

Head motion during functional Magnetic Resonance Imaging acquisition can significantly contaminate the neural signal and introduce spurious, distance-dependent changes in signal correlations. This can heavily confound studies of development, aging, and disease. Previous approaches to suppress head motion artifacts have involved sequential regression of nuisance covariates, but this has been shown to reintroduce artifacts. We propose a new motion correction pipeline using an omnibus regression model that avoids this problem by simultaneously capturing multiple artifact sources using the best performing algorithms for each artifact. We quantitatively evaluate its motion artifact suppression performance against sequential regression pipelines using a large heterogeneous dataset (n=151) which includes high-motion subjects and multiple disease phenotypes. The proposed concatenated regression pipeline significantly reduces the association between head motion and functional connectivity while significantly outperforming the traditional sequential regression pipelines in eliminating distance-dependent head motion artifacts.

Chimeric antigen receptor is an immunotherapy whereby T lymphocytes are engineered to selectively attack cancer cells. Image-based screens of CAR-T cells, combining phase contrast and fluorescence microscopy, suffer from the gradual quenching of the fluorescent signal, making the reliable monitoring of cell populations across time-lapse imagery difficult. We propose to leverage the available fluorescent markers as an experimentally-generated ground truth, without recourse to manual annotation. With some simple image processing, we are able to segment and assign cell type classes automatically. This ground truth is sufficient to train a neural object detection system from the phase contrast signal alone, potentially eliminating the need for the cumbersome fluorescent markers. This approach will underpin the development of cheap and robust microscope-based protocols to quantify CAR-T activity against tumor cells in vitro.

Alzheimer's disease (AD) is a multi-factor neurodegenerative disease that selectively affects certain regions of the brain while other areas remain unaffected. The underlying mechanisms of this selectivity, however, are still largely elusive. To address this challenge, we propose a novel longitudinal network analysis method employing sparse logistic regression to identify frequency-specific oscillation patterns which contribute to the selective network vulnerability for patients at risk of advancing to the more severe stage of dementia. We fit and apply our statistical method to more than 100 longitudinal brain networks, and validate it on synthetic data. A set of critical connectome pathways are identified that exhibit strong association to the progression of AD.

We propose a new loss formulation to further advance the multiclass segmentation of cluttered cells under weakly supervised conditions. When adding a Youden?s J statistic regularization term to the cross entropy loss we improve the separation of touching and immediate cells, obtaining sharp segmentation boundaries with high adequacy. This regularization intrinsically supports class imbalance thus eliminating the necessity of explicitly using weights to balance training. Simulations demonstrate this capability and show how the regularization leads to correct results by helping advancing the optimization when cross entropy stagnates. We build upon our previous work on multiclass segmentation by adding yet another training class representing gaps between adjacent cells. This addition helps the classifier identify narrow gaps as background and no longer as touching regions. We present results of our methods for 2D and 3D images, from bright field images to confocal stacks containing different types of cells, and we show that they accurately segment individual cells after training with a limited number of images, some of which are poorly annotated.

Image feature extraction based on deep disentangled representation learning of PET images is proposed for the prediction of lymphoma treatment response. Our method encodes PET images as spatial representations and modality representations by performing supervised tumor segmentation and image reconstruction. In this way, the whole image features (global features) as well as tumor region features (local features) can be extracted without the labor-intensive tumor segmentation and feature calculation procedure. The learned global and local image features are then joined with several prognostic factors evaluated by physicians based on clinical information, and used as input of a SVM classifier for predicting outcome results of lymphoma patients. In this study, 186 lymphoma patient data were included for training and testing the proposed model. The proposed method was compared with the traditional straightforward feature extraction method. The better prediction results of the proposed method also show its efficiency for prognostic prediction related feature extraction in PET images.

Cryo-electron microscopy (cryo-EM) has proven to be a promising tool for recovering the 3D structure of biological macromolecules. The cryo-EM map which is reconstructed from a large set of projection images, is then used for recovering the atomic model of the molecule. The accuracy of the fitted atomic model depends on the quality of the cryo-EM map. Due to current limitations during imaging or reconstruction process, the reconstructed map usually lacks interpretability and require further quality enhancement post-processing. In this work, we present a data-driven solution to improve the quality of low-resolution cryo-EM maps. For this purpose, we generate and make publicly available a dataset generated from deposited protein structures in protein data bank (PDB). This dataset includes low and high-resolution map pairs in multiple resolutions. We also provide our implementation for generating the dataset, to further accommodate the generation of variations of the dataset benefiting a wide range of problems in cryo-EM. Our results justify the potential of our method in successfully recovering details for simulated and experimental density maps. Furthermore, compared to state-of-the-art cryo-EM map sharpening methods, our approach not only provides good results but is also computationally efficient.

Model-based image reconstruction (MBIR) approaches provide a principled way to combine sensor and image prior models to solve imaging inverse problems. Plug-and-play MBIR (PnP-MBIR) approaches use variable splitting and allow the use of rich image priors. This work extends this idea to data-domain modeling and presents a new MBIR framework that enables integration of data-domain and image-domain prior models and allows high-quality reconstructions even for imperfect data imaging problems such as limited-angle CT, accelerated MRI, or diverging-wave Ultrasound imaging. In this work, we use this newly developed framework to explore the potential of learned data and image models for biomedical applications of tomographic imaging problems.

The Automap method of MRI reconstruction provides an abstract way for direct mapping from undersampled k-space to image domain in two steps: domain transformation (DT) and image refinement (IR). The two steps have very different functions: The first step aims to learn the bases of a transform, and the second learns image recovery from the transformed sampled sensor data, subject to data consistency. We begin with the premise that representational sparsity and multiresolution nonlinearity are useful to permit sharp local discontinuities while globally optimizing IR, and consider the U-net architecture for IR. Additionally we consider the dAutomap method of reducing the transform computational complexity to O(N^3), a model of reduced representational power compared to the O(N^4) fully connected part of Automap, but which offers tractability to realistic k-space sizes and exact initializability.

Parkinson?s disease (PD) is a neuro-degenerative disorder whose symptoms include slowness of movement, tremors, muscle stiffness, changes in speech and writing, depression, anxiety, sleep and emotional problems. According to the Parkinson?s foundation, almost one million Americans live with PD and approximately 60,000 Americans are diagnosed with the disease every year. Deep learning approaches have been shown promising for medical image analysis (Litjens et al., 2017). Deep learning is a branch of machine learning that aims at learning various features of the provided data or images in an automated fashion without the need for extracting specific handcrafted features. The most popular deep learning approaches are: Convolutional Neural Networks (CNN), Deep Belief Networks (DBF) and Deep Boltzmann Machines (DBM). Few works have investigated the use of CNN for diagnosing and classifying PD based on speech patterns, handwriting exams and handwriting dynamics captured by a smart pen (Frid et al., 2016; Pereira et al., 2016; Eskofier et al., 2016; Pereira et al., 2017; Um et al., 2017). In the current work, two novel CNN models of two convolutional, two maximum pooling and fully connected layers are introduced. The proposed models were trained on spiral and wave handwriting datasets of 102 images each to distinguish between controls and subjects with PD.A validation accuracy of 83% and 87% was achieved when the proposed models were trained on 70% of both datasets for 300 epochs respectively with 30% of the datasets reserved for testing. The proposed models provide a promising solution for early diagnosis and screening of PD patients and may possibly support the clinical assessments for medical specialists that may be subjective and insensitive. In addition to using handwritten datasets for PD diagnosis, frequency analysis of the Electroencephalography (EEG) data may contain useful information about motor and non-motor dysfunction of PD by exploiting theta [4-8 Hz], alpha [8-12 Hz], beta [12-35 Hz] and gamma [35-45 Hz] waves. The future research will aim to analyze a public EEG dataset for controls and PD patients and extract unique features from both the beta and gamma frequency bands with an objective to detect the cognitive decline of PD patients and hence predict the existence of the disease.

Prostate MRI segmentation has become an important tool for quantitative estimation of the gland volume during diagnostic imaging. It is also a critical step in the fusion between MRI and transrectal ultrasound (TRUS) for fusion guided biopsy or therapy. 3-D neural networks have demonstrated strong potential for this task, but require substantial computational resources due to their large number of parameters. In this work, we focus on the efficiency of the segmentation network in terms of speed and memory requirements. Specifically, we aim at reaching state-of-the-art results with ?smaller? networks, involving significantly fewer parameters, thus making the network easier to train and operate. A novel 3-D network architecture, called V-net Light (VnL) is proposed, based on an efficient 3-D Module called 3-D Light, that minimizes the number of network parameters while maintaining state-of-the-art segmentation results. The proposed method is validated on the PROMISE12 challenge data[1]. The proposed VnL has only 9.1% of V-net's parameters, 3.2% of its floating point operations (FLOPs) and uses only 9.1% of hard-disk storage compared to V-net, yet V-net and VnL has comparable accuracy.

Microsatellite instability is an important clinical marker for various types of cancers and is related to patients' prognosis and response to immunotherapy. Currently, identifying microsatellite status relies on genetic tests, which are not widely accessible for every patient. We propose a novel pipeline to predict MSI directly from histology slides which represent the gold standard for cancer diagnosis and are ubiquitously available for cancer patients. Our method outperformed existing method on the uterine corpus endometrial carcinoma cohort in The Cancer Genome Atlas (AUC 0.73 vs. 0.56).

We develop and evaluate a novel hybrid tractography algorithm for improved segmentation of complex fiber bundles from diffusion magnetic resonance imaging datasets. We propose an approach inspired by reinforcement learning that combines the strengths of both probabilistic and deterministic tractography to better resolve pathways dominated by crossing fibers. Given a fiber bundle query, our approach first explores an array of possible pathways probabilistically, and then exploits this information with streamline tractography using globally optimal fiber compartment assignment in a conditional random field. We quantitatively evaluated our approach in comparison with deterministic and probabilistic approaches using a realistic phantom with Tractometer and 88 test-retest scans from the Human Connectome Project. We found that the proposed hybrid method offers improved accuracy with phantom data and more biologically plausible topographic organization and higher reliability with in vivo data. This demonstrates the benefits of combining tractography approaches and indicates opportunities for integrating reinforcement learning strategies into tractograpy algorithms.

We study the effect of the selection of diffeomorphic normalization in the performance of Spasov's deep-learning system for the problem of progressive MCI vs stable MCI discrimination. We considered different degrees of normalization (no, affine and non-rigid normalization) and two diffeomorphic registration methods (ANTS and BL PDE-LDDMM) with different image similarity metrics (SSD, NCC, and lNCC) yielding qualitatively different deformation models and quantitatively different degrees of registration accuracy. BL PDE-LDDMM NCC achieved the best performing accuracy with median values of 89%. Surprisingly, the accuracy of no and affine normalization was also among the highest, indicating that the deep-learning system is powerful enough to learn accurate models for pMCI vs sMCI discrimination without the need for normalization. However, the best sensitivity values were obtained by BL PDE-LDDMM SSD and NCC with median values of 97% and 94% while the sensitivity of the remaining methods stayed under 88%.

As one of the major types of glaucoma, closed-angle glaucoma is the leading cause of irreversible blindness in the world. The ability of Anterior Segment Optical Coherence Tomography (AS-OCT) to obtain high-resolution cross-sectional images of the entire anterior chamber in a single image makes it an important tool for glaucoma diagnosis. In this paper, we propose a practical and efficient system based on deep learning to accurately classify anterior chamber angle (ACA) closure by using the location of scleral spur (SS) points. First, the localization problem is reformulated as a pixel-wise regression task. A fully convolutional deep neural network is optimized to predict the probability that each pixel belongs to the SS points, and the numerical coordinates are obtained by the maximum likelihood estimation theory. Second, the ACA region centered on the detected SS is cropped as the input of the classification model. The single model applied for classification is SE-ResNet18 and optimized with focal loss. In the AGE Challenge 2019[1], our proposed method obtained superior performance for angle-closure glaucoma screening.

We propose a novel procedure to improve liver and lesion segmentation from CT scans for U-Net based models. Our method extends standard segmentation pipelines to focus on higher target recall or reduction of noisy false-positive predictions, boosting overall segmentation performance. To achieve this, we include segmentation errors into a new learning process appended to the main training setup, allowing the model to find features which explain away previous errors. We evaluate this on semantically distinct architectures: cascaded two- and three-dimensional as well as combined learning setups for multitask segmentation. Liver and lesion segmentation data are provided by the Liver Tumor Segmentation challenge (LiTS), with an increase in dice score of up to 2 points.

Deep learning based image segmentation has achieved the state-of-the-art performance in many medical applications such as lesion quantification, organ detection, etc. However, most of the methods rely on supervised learning, which require a large set of high-quality labeled data. Data annotation is generally an extremely time-consuming process. To address this problem, we propose a generic semi-supervised learning framework for image segmentation based on a deep convolutional neural network (DCNN). An encoder-decoder based DCNN is initially trained using a few annotated training samples. This initially trained model is then copied into sub-models and improved iteratively using random subsets of unlabeled data with pseudo labels generated from models trained in the previous iteration. The number of sub-models is gradually decreased to one in the final iteration. We evaluate the proposed method on a public grand-challenge dataset for skin lesion segmentation. Our method is able to significantly improve beyond fully supervised model learning by incorporating unlabeled data. The code is available on Github.

In this paper, we introduce a fully unsupervised approach for the synthesis of CT images of the lumbar spine, used for image-guided surgical procedures, from a T2-weighted MRI acquired for diagnostic purposes. Our approach makes use of a trainable pre-processing pipeline using a low-capacity fully convolutional network, to normalize the input MRI data, in cascade with FC-ResNets, to segment the vertebral bodies and pedicles. A pseudo-3D Cycle GAN architecture is proposed to include neighboring slices in the synthesis process, along with a cyclic loss function ensuring consistency between MRI and CT synthesis. Clinical experiments were performed on the SpineWeb dataset, totalling 18 patients with both MRI and CT. Quantitative comparison to expert CT segmentations yields an average Dice score of 83 +/- 1.6 on synthetic CTs, while a comparison to CT annotations yielded a landmark localization error of 2.2 +/- 1.4mm. Intensity distributions and mean absolute errors in Hounsfield units also show promising results, illustrating the strong potential and versatility of the pipeline by achieving clinically viable CT scans which can be used for surgical guidance.

In this paper we introduce the Ladder Algorithm; a novel recurrent algorithm to detect repetitive structures in natural images with high accuracy using little training data. We then demonstrate the algorithm on the task of extracting vertebrae from whole spine magnetic resonance scans with only lumbar MR scans for training data. It is shown to achieve high performance with 99.8% precision and recall, exceeding current state of the art approaches for lumbar vertebrae detection in T1 and T2 weighted scans. It also generalises to whole spine images with minimal drop in accuracy, achieving a detection rate of 99.4%.

Liver vessel segmentation from contrast CT images is critical in liver surgical planning and navigation. Due to the complex vessel systems, manual segmentation of liver vessel is laborious and error-prone. In this work we propose liver vessel segmentation deep neural networks which are able to perform the segmentation automatically. Dense image labelling incurred by normal deep learning based segmentation makes it impractical for vessel segmentation. On the contrary, our method only requires a few of initial voxel-level labels, and thus tremendously reduces the workload of manual annotating. A small number of liver vessels are firstly annotated and used to train a sparse dictionary and logistic regressor to get the preliminary vessel predictions. Then the proposed deep neural network is trained based on bootstrapping technique where the training target is a convex combination of the model predictions and those preliminary vessel predictions. Followed by the post-processing step consisting of region connection and noisy removal, the liver vessel tree is finally built from model predictions. Experimental results of3DIRCADb dataset show that the proposed method can effectively extract liver vessel trees from contrast CT images with minimum annotations.The state-of-art results have been achieved on this small dataset with the average dice and sensitivity 0.687?0.041 and 0.786?0.105 respectively.

Semantic segmentation is an essential step for electron microscopy (EM) image analysis. Although supervised models have achieved significant progress, the need for labor intensive pixel-wise annotation is a major limitation. To complicate matters further, supervised learning models may not generalize well on a novel dataset due to domain shift. In this study, we introduce an adversarial-prediction guided multi-task network to learn the adaption of a well-trained model on a source domain for use on an unlabeled target domain. Since no label is available on target domain, we learn an encoding representation not only for the supervised segmentation on source domain but also for unsupervised reconstruction of the target data. To improve the discriminative ability of the supervised learned model on unlabeled target domain, we further guide the representation learning by multi-level adversarial learning in semantic prediction space. Comparisons and ablation study on public benchmark demonstrated state-of-the-art performance and effectiveness of our approach.

Image segmentation is one of the most important building blocks in many medical imaging applications. Often, it is the first step in any artificial intelligence (AI) assisted diagnosis system. Most convolutional neural network image-to-image segmentation algorithms compute binary mask segmentation and extract the object boundary as the edge of the binary mask always leading to a closed boundary. In this paper, we present a novel image-to-image segmentation algorithm that learns open boundaries. The object delineation is directly learnt by training a U-Net like network on distance map representation of the boundary without any constraints on its shape or topology. We validate the proposed approach on the segmentation of the left atrium in intra-cardiac echocardiography images. For this application, it is important to produce segmentation only where a strong evidence of the anatomy exists. To our knowledge, this is the first work to train a U-Net on distance map ground truth representation for open boundary segmentation.

Magnetic resonance imaging (MRI) acquisition is an inherently slow process whose acceleration has been the subject of much investigation. In recent years, the explosive advance of deep learning techniques for computer vision and image reconstruction has led to the investigation of deep neural networks for the reconstruction of MRI with under-sampled k-space. In this work, we propose a new image domain architecture that directly produces a sum-of-squares image from under-sampled multi-coil MRI acquisition. This model, called BarbellNet, is a fully convolutional neural network architecture that utilizes the channel attention mechanism using the residual channel attention block (RCAB). Through extensive experiments with the fastMRI data set, we confirm the efficacy of BarbellNet.

Medical image classification using deep learning techniques rely on highly curated datasets, which are difficult and expensive to obtain in real world due significant expertise required to annotate the dataset. We propose a novel framework called Class Specific Convolutional Coders (CSCC) to tackle the problem of learning highly discriminative, compact and non-redundant feature space from a relatively small amount of labelled images. We design separate attention-driven convolution network based feature extractors for the categories. These feature learning modules are further intuitively combined so as to make the whole image recognition system end-to-end trainable. Results on different medical image classification tasks show the advantages of our contributions, where our proposed methods outperforms the benchmark supervised deep convolutional networks (CNNs) trained from scratch.

Early diagnosis of liver cancer is particularly important in reducing the high mortality of liver cancer because some symptoms will not appear until the cancer cannot be reversed. Dynamic contrast-enhanced MRI (DCE-MRI) plays an important role in distinguishing benign and malignant tumors by dynamically monitoring the uptake and washout of contrast agent (gadolinium based) in different tissues. However, patient motion during the few minutes of data acquisition may result in the voxel-based mis-correspondence in adjacent time frames. Therefore, DCE-MRI registration is a necessary pre-processing step for motion correction. The challenge of dynamic contrast-enhanced MRI (DCE-MRI) registration is the huge intensity changes caused by contrast agent injections. These changes may lead to the unrealistic deformation of contrast enhancement regions by traditional intensity-based registration algorithms. Our method used to overcome the challenge is based on the idea of remove the intensity enhancement. In this paper, we apply sparse representation to reconstruct the time intensity curves of contrast agent, the dictionary used to sparse represent only includes the information of intensity changes over different time points caused by contrast agent, and then remove the contrast enhancement by simple subtracting operation. The de-enhanced images reshaped from ?non-contrast? curves can be easily aligned using traditional registration schemes. The main contribution of our work for the clinical application is to raise the accuracy of the prediction of characteristic parameter which can distinguish the benign and malignant tumor.

Microwave hyperthermia makes use of microwaves to deliver heat to biological tissues. Real time temperature monitoring during treatment is important for efficacy and effectiveness of the treatment. Non-invasive methods such as CT, MR and ultrasound (US) have been actively researched for use in hyperthermia monitoring. US has inherent advantages of real-time imaging, portability and non-ionizing nature. It is also known from the literature that acoustic properties of tissue are sensitive to temperature and this has been harnessed to track the evolution of hotspot and temperature in high temperature zones encountered in ablation treatments. However, their usage in low temperature zones typically observed in hyperthermia appears to be less explored. This study introduces an improved method of regularized Kurtosis imaging (RKI) and compares its performance against regularized log spectral difference (RLSD) and regularized Nakagami imaging methods. The performance of these methods is compared against the ground truth estimated from IR thermal images in an experimental study on tissue-mimicking PAG-agar based phantoms. The error in the area estimated by RKI was 10.6%. The error in the lateral and axial co-ordinate of the centroid was 5.92 % and 0.47%, respectively. The structural similarity index was 0.82 for RKI when compared with RLSD and RNI having score of 0.76 and 0.72, respectively. The results are promising and offer an alternative way to track the hotspot during microwave hyperthermia treatment.

Early diagnosis of ocular diseases is key to prevent severe vision damage and other healthcare-related issues. Color fundus photography is a commonly utilized screening tool. However, due to the small symptoms present for early-stage ocular diseases, it is difficult to accurately diagnose the fundus photographs. To this end, we propose an attention-based unilateral and bilateral feature weighting and fusion network (AUBNet) to automatically classify patients into the corresponding disease categories. Specifically, AUBNet is composed of a feature extraction module (FEM), a feature fusion module (FFM), and a classification module (CFM). The FEM extracts two feature vectors from the bilateral fundus photographs of a patient independently. With the FFM, two levels of feature weighting and fusion are proceeded to prepare the feature representations of bilateral eyes. Finally, multi-label classifications are conducted by the CFM. Our model achieves competitive results on a real-life large-scale dataset.

Registration of lungs in thoracic computed tomography (CT) images produces a dense correspondence which can be analyzed to estimate local tissue expansion. However, the validity of this local expansion estimate is dependent on the accuracy of the image registration. In this work, a convolutional neural network (CNN) model is used to directly estimate the local tissue expansion between lungs imaged at two lung volumes, without requiring image registration. The network was trained with 5705 subjects from COPDGene with varying degrees of disease severity. The CNN-based model was evaluated with 3046 subjects from COPDGene. At the global scale, the mean lung expansion estimated from the CNN-based and registration-based models were highly correlated (rs = 0.945). At the local scale, the proposed method achieved a voxelwise Spearman correlation of 0.871 ? 0.080. At the regional scale, Dice coefficient for high and low func- tioning regions was 0.806 ? 0.065 and 0.805 ? 0.066, respectively. The results indicate the CNN-based model was able to reproduce image registration derived tissue expansion images without explicitly estimating the correspondence.

Echo-planar imaging (EPI), which is the main workhorse of functional MRI, suffers from field inhomogeneity-induced geometric distortions. The amount of distortion is proportional to the readout duration, which restricts the maximum achievable spatial resolution. The spatially varying nature of the T2* decay makes it challenging for EPI schemes with a single echo time to obtain good sensitivity to functional activations in different brain regions. Despite the use of parallel MRI and multislice acceleration, the number of different echo times that can be acquired in a reasonable TR is limited. The main focus of this work is to introduce a rosette-based acquisition scheme and a structured low-rank reconstruction algorithm to overcome the above challenges. The proposed scheme exploits the exponential structure of the time series to recover distortion-free images from several echoes simultaneously.

Earliest neuropathological changes in Alzheimer?s Disease (AD) emerge in the medial temporal lobe (MTL). In order for MRI biomarkers to detect changes linked specifically to AD pathology (as opposed to aging or other pathological factors) macroscopic patterns of structural change in the MTL must be linked to the underlying neuropathology. To provide such a linkage, we are conducting an autopsy imaging study combining ex vivo MRI and serial histopathology. Information from multiple subjects can be studied by creating a ?population average? atlas of the MTL. We present a groupwise registration approach for constructing the atlas that is able to successfully capture the complex structure of the MTL, and anatomical variability across subjects. This atlas allows us to generate maps of cortical thickness measurements and identify regions in the MTL where structural changes correlate most strongly with AD progression. We show that using this atlas, we are able to find a significant correlation between atrophy and AD pathology in the MTL sub-regions associated with the earliest stages of AD pathology as described by Braak and Braak.

Early detection is essential to reduce cancer mortality. Oral cancer could be subject to screening programs (similar as for cervical cancer) by collecting Pap smear samples at any dentist visit. However, manual analysis of the resulting massive amount of data is prohibitively costly. Convolutional neural networks (CNNs) have shown promising results in discriminating between cancerous and non-cancerous cells, which allows efficient automated processing of cancer screening data. We investigate different CNN architectures which explicitly aim to utilize texture information in cytological cancer classification, motivated by studies showing that chromatin texture is among the most important discriminative features for that purpose. Results show that CNN classifiers guided by Local Binary Patterns (LBPs) achieve better performance than general purpose CNNs, even when different levels of general data augmentation, as well as pretraining, are considered.

The one-to-one mapping is necessary for MRI image synthesis as MRI images are unique to the patient. State-of-the-art approaches for image synthesis from domain X to domain Y learn a convolutional neural network that meticulously maps between the domains. A different network is typically implemented to map along the opposite direction, from Y to X. In this paper, we explore the possibility of only wielding one network for bi-directional image synthesis. In other words, such an autonomous learning network implements a self-inverse function. A self-inverse network shares several distinct advantages: only one network instead of two, better generalization and more restricted parameter space. Most importantly, a self-inverse function guarantees a one-to-one mapping, a property that cannot be guaranteed by earlier approaches that are not self-inverse. The experiments on MRI T1 and T2 images show that, compared with the baseline approaches that use two separate models for the image synthesis along with two directions, our self-inverse network achieves better synthesis results in terms of standard metrics. Finally, our sensitivity analysis confirms the feasibility of learning a one-to-one mapping function for MRI image synthesis.

Autism spectrum disorder (ASD) is a complex neurodevelopmental disability, which is lack of biologic diagnostic markers. Therefore, exploring the ASD Identification directly from brain imaging data has been an important topic. In this work, we propose the Siamese verification model to identify ASD using 6 and 12 months cortical features. Rather than directly classifying a testing subject is ASD or not, we determine whether it has the same or different label with the reference subject who has been successfully diagnosed. Then, based on the comparison to all the reference subjects, we can predict the label of the testing subject. The advantage of modeling the classification problem as a verification framework is that it can greatly enlarge the training data size and enable us to train a more accurate and reliable model in an end-to-end manner. In addition, to further improve the classification performance, we introduce the path signature (PS) features, which can capture the dynamic longitudinal information of the brain development for the ASD Identification. Experiments showed that our proposed method reaches the best result, i.e., 87% accuracy, 83% sensitivity and 90% specificity comparing to the state-of-the-art methods.

The use of quantitative radiomic features of MRI to predict the aggressiveness of prostate cancer has attracted increasing amounts of attention due to its potential as a non-invasive biomarker for prostate cancer. In this study, to predict prostate cancer aggressiveness, we investigate the usefulness of multi-modal radiomic features according to combination method such as concatenation or averaging and compare multi-modal radiomic features to uni-modal radiomic features. To define the prostate cancer region of T2wMR based on ground truth pathology, a radiologist manually segmented prostate cancer referring to a fusion result of registration of histopathology image and T2wMR. The prostate cancer region of the ADC is then defined as the same region as the T2wMR through registration of the ADC on the T2wMR. To extract radiomic features to predict prostate cancer aggressiveness, total 68 features are calculated for each region of T2wMR and ADC. To predict the aggressiveness of prostate cancer, a random forest classifier is trained for each region in T2wMR and ADC. The prostate cancer regions were categorized as Low GS Group (GS = 4+3). As results, the sensitivity of combined features was the highest at 82.0% which is higher 2.4%, 2.7% and 4.8% than ADC, T2wMR, and average combined features. Experiment results showed that the possibility of determining the aggressiveness of prostate cancer through the multi-modal radiomic features of T2wMR and ADC.

Organs-at-risk (OAR) delineation in computed tomography (CT) is an important step in Radiation Therapy (RT) planning. Recently, deep learning based methods for OAR delineation have been proposed and applied in clinical practice for separate regions of the human body (head and neck, thorax, and abdomen). However, there are few researches regarding the end-to-end whole-body OARs delineation because the existing datasets are mostly partially or incompletely annotated for such task. In this paper, our proposed end-to-end convolutional neural network model, called AttentionAnatomy, can be jointly trained with three partially annotated datasets, segmenting OARs from whole body. Our main contributions are: 1) an attention module implicitly guided by body region label to modulate the segmentation branch output; 2) a prediction re-calibration operation, exploiting prior information of the input images, to handle partial-annotation(HPA) problem; 3) a new hybrid loss function combining batch Dice loss and spatially balanced focal loss to alleviate the organ size imbalance problem. Experimental results of our proposed framework presented significant improvements in both S?rensen-Dice coefficient (DSC) and 95% Hausdorff distance compared to the baseline model.

Obtaining paired image data is a major obstacle for applying deep learning in CT denoising, where imaging the same target at two different radiation doses would expose patients to unnecessary radiation. In this session, we propose a method to overcome this limitation and train a model that can denoise low dose CT images with only unpaired high dose data using the CycleGAN model. Additionally, we propose the use of Strided Patch Cropping Self-Ensemble (SPACE), a method for stable reconstruction of arbitrarily sized images that does not lose fine details important for accurate clinical analysis.

It is shown that classic expressive language areas are involved in passive sentence-level speech processing. However, this interaction between expressive and receptive language pathways during passive language perception might only exist because of high degree of cognitive demand which modulates large integrated brain networks such as memory. To answer whether such interaction exists only due to the task-evoked increase of brain networks integration, or the nature of language processing itself, we studied modulations of power and functional connectivity (FC) in Broca?s area during a passive word recognition task (WRT) using electrocorticographic (ECoG) recordings. Our results revealed no power modulation over Broca?s area during the WRT, but we found FC modulation in this area with frequency-specific spatiotemporal profiles. We conclude that Broca?s area is involved in passive single word processing, implying the integration of expressive and receptive language pathways even at the word-level language processing.

Multiple magnetic resonance imaging (MRI) modalities are currently used for the diagnosis and characterization of temporal lobe epilepsy (TLE). The objective of this study is to assess the feasibility of combination of multimodal MRI datasets to provide an accurate classification of TLE by employing a multi-channel deep neural network.