IEEE ISBI 2020 Virtual Conference April 2020

Vulnerable plaque is a major factor leading to the onset of acute coronary syndrome (ACS), and accordingly, the detection of vulnerable plaques (VPs) could guide cardiologists to provide appropriate surgical treatments before the occurrence of an event. In general, hundreds of images are acquired for each patient during surgery. Hence a fast and accurate automatic detection algorithm is needed. However, VPs? detection requires extensive annotation of lesion?s boundary by an expert practitioner, unlike diagnoses. Therefore in this paper, a multiple instances learning-based method is proposed to locate VPs with the image-level labels only. In the proposed method, the clip proposal module, the feature extraction module, and the detection module are integrated to recognize VPs and detect the lesion area. Finally, experiments are performed on the 2017 IVOCT dataset to examine the task of weakly supervised detection of VPs. Although the bounding box of VPs is not used, the proposed method yields comparable performance with supervised learning methods.

Image classification using deep convolutional neural networks (DCNN) has a competitive performance as compared to other state-of-the-art methods. Here, attention can be visualized as a heatmap to improve the explainability of DCNN. We generated the initial heatmaps by using gradient-based classification activation map (Grad-CAM). We first assume that these Grad-CAM heatmaps can reveal the lesion regions well, then apply the attention mining on these heatmaps. Another, we assume that these Grad-CAM heatmaps can't reveal the lesion regions well then apply the dissimilarity loss on these Grad-CAM heatmaps. In this study, we asked the ophthalmologists to select 30% of the heatmaps. Furthermore, we design knowledge preservation (KP) loss to minimize the discrepancy between heatmaps generated from the updated network and the selected heatmaps. Experiments revealed that our method improved accuracy from 90.1% to 96.2%. We also found that the attention regions are closer to the GT lesion regions.

In this work we address the problem of domain adaptation for segmentation tasks with deep convolutional neural networks. We focus on managing the domain shift from MRI to CT volumes on the example of 3D liver segmentation. Domain adaptation between modalities is particularly of practical importance, as different hospital departments usually tend to use different imaging modalities and protocols in their clinical routine. Thus, training a model with source data from one department may not be sufficient for application in another institution. Most adaptation strategies make use of target domain samples and often additionally incorporate the corresponding ground truths from the target domain during the training process. In contrast to these approaches, we investigate the possibility of training our model solely on source domain data sets, i.e. we apply zero-shot domain adaptation. To compensate the missing target domain data, we use prior knowledge about both modalities to steer the model towards more general features during the training process. We particularly make use of fixed Sobel kernels to enhance contour information and apply anatomical priors, learned separately by a convolutional autoencoder. Although we completely discard including the target domain in the training process, our proposed approach improves a vanilla U-Net implementation drastically and yields promising segmentation results.

Visualization and classification of cell cycle stages in live cells requires the introduction of transient or stably expressing fluorescent markers. This is not feasible for all cell types, and can be time consuming to implement. Labelling of living cells also has the potential to perturb normal cellular function. Here we describe a computational strategy to estimate core cell cycle stages without markers by taking advantage of features extracted from information-rich ptychographic time-lapse movies. We show that a deep-learning approach can estimate the cell cycle trajectories of individual human melanoma cells from short 3-frame (~23 minute) snapshots, and can identify cell cycle arrest induced by chemotherapeutic agents targeting melanoma driver mutations.

Anatomical landmarks are a crucial prerequisite for many medical imaging tasks. Usually, the set of landmarks for a given task is predefined by experts. The landmark locations for a given image are then annotated manually or via machine learning methods trained on manual annotations. In this paper, in contrast, we present a method to automatically discover and localize anatomical landmarks in medical images. Specifically, we consider landmarks that attract the visual attention of humans, which we term visually salient landmarks. We illustrate the method for fetal neurosonographic images. First, full-length clinical fetal ultrasound scans are recorded with live sonographer gaze-tracking. Next, a convolutional neural network (CNN) is trained to predict the gaze point distribution (saliency map) of the sonographers on scan video frames. The CNN is then used to predict saliency maps of unseen fetal neurosonographic images, and the landmarks are extracted as the local maxima of these saliency maps. Finally, the landmarks are matched across images by clustering the landmark CNN features. We show that the discovered landmarks can be used within affine image registration, with average landmark alignment errors between 4.1% and 10.9% of the fetal head long axis length.

This paper proposes an automatic method for scapula bone segmentation from Magnetic Resonance (MR) images using deep learning. The purpose of this work is to incorporate anatomical priors into a conditional adversarial framework, given a limited amount of heterogeneous annotated images. Our approach encourages the segmentation model to follow the global anatomical properties of the underlying anatomy through a learnt non-linear shape representation while the adversarial contribution refines the model by promoting realistic delineations. These contributions are evaluated on a dataset of 15 pediatric shoulder examinations, and compared to state-of-the-art architectures including UNet and recent derivatives. The significant improvements achieved bring new perspectives for the pre-operative management of musculo-skeletal diseases.

For circular trajectory conebeam CT, Feldkamp, Davis, and Kress (FDK) algorithm is widely used for its reconstruction. However, the existence of cone-angle artifacts is fatal for the quality when using this algorithm. There are several model-based iterative reconstruction methods for the cone-angle artifacts removal, but these algorithms usually require repeated applications of computational expensive forward and backward.In this paper, we propose a novel deep learning approach for cone-angle artifact removal on differentiated backprojection domain, which performs a data-driven inversion of an ill-posed deconvolution problem related to the Hilbert transform. The reconstruction results along the coronal and sagittal directions are then combined by a spectral blending technique to minimize the spectral leakage. Experimental results show that our method provides superior performance to the existing methods.

Residual network (ResNet) and densely connected network (DenseNet) have significantly improved the training efficiency and performance of deep convolutional neural networks (DCNNs) mainly for object classification tasks. In this paper, we propose an efficient network architecture by considering advantages of both networks. The proposed method is integrated into an encoder-decoder DCNN model for medical image segmentation. Our method adds additional skip connections compared to ResNet but uses significantly fewer model parameters than DenseNet. We evaluate the proposed method on a public dataset (ISIC 2018 grand-challenge) for skin lesion segmentation and a local brain MRI dataset. In comparison with ResNet-based, DenseNet-based and attention network (AttnNet) based methods within the same encoder- decoder network structure, our method achieves significantly higher segmentation accuracy with fewer number of model parameters than DenseNet and AttnNet. The code is available on GitHub (GitHub link: https://github.com/MinaJf/DRU-net).

Fully automatic detection of skin lesions in dermatoscopic images can facilitate early diagnosis and repression of malignant melanoma and non-melanoma skin cancer. Although convolutional neural networks are a powerful solution, they are limited by the illumination spectrum of annotated dermatoscopic screening images, where color is an important discriminative feature. In this paper, we propose an adaptive color augmentation technique to amplify data expression and model performance, while regulating color difference and saturation to minimize the risks of using synthetic data. Through deep visualization, we qualitatively identify and verify the semantic structural features learned by the network for discriminating skin lesions against normal skin tissue. The overall system achieves a Dice Ratio of 0.891 with 0.943 sensitivity and 0.932 specificity on the ISIC 2018 Testing Set for segmentation.

Clinical applicability of automated decision support systems depends on a robust, well-understood classification interpretation. Artificial neural networks while achieving class-leading scores fall short in this regard. Therefore, numerous approaches have been proposed that map a salient region of an image to a diagnostic classification. Utilizing heuristic methodology, like blurring and noise, they tend to produce diffuse, sometimes misleading results, hindering their general adoption. In this work we overcome these issues by presenting a model agnostic saliency mapping framework tailored to medical imaging. We replace heuristic techniques with a strong neighborhood conditioned inpainting approach, which avoids anatomically implausible artefacts. We formulate saliency attribution as a map-quality optimization task, enforcing constrained and focused attributions. Experiments on public mammography data show quantitatively and qualitatively more precise localization and clearer conveying results than existing state-of-the-art methods.

Traction Force Microscopy (TFM) is a technique used to determine the tensions that a biological cell conveys to the underlying surface. Typically, TFM requires culturing cells on gels with fluorescent beads, followed by bead displacement calculations. We present a new method allowing to predict those forces from a regular fluorescent image of the cell. Using Deep Learning, we trained a Bayesian Neural Network adapted for pixel regression of the forces and show that it generalises on different cells of the same strain. The predicted forces are computed along with an approximated uncertainty, which shows whether the prediction is trustworthy or not. Using the proposed method could help estimating forces when calculating non-trivial bead displacements and can also free one of the fluorescent channels of the microscope. Code is available at https://github.com/wahlby-lab/InSilicoTFM.

In this paper, we propose a deep learning approach for the segmentation of body parts in computer tomography (CT) localizer images. Such images pose difficulties in the automatic image analysis on account of variable field-of-view, diverse patient positioning and image acquisition at low dose, but are of importance pertaining to their most prominent applications in scan planning and dose modulation. Following the success of deep learning technology in image segmentation applications, we investigate the use of a fully convolutional neural network architecture to achieve the segmentation of four anatomies: abdomen, chest, pelvis and brain. The method is further extended to generate plan boxes for individual as well as multiple combined anatomies, and compared against the existing techniques. The performance of the method is evaluated on 771 multi-site localizer images.

Mammographic mass detection and segmentation are usually performed as serial and separate tasks, with segmentation often only performed on manually confirmed true positive detections in previous studies. We propose a fully-integrated computer-aided detection (CAD) system for simultaneous mammographic mass detection and segmentation without user intervention. The proposed CAD only consists of a pseudo-color image generation and a mass detection-segmentation stage based on Mask R-CNN. Grayscale mammograms are transformed into pseudo-color images based on multi-scale morphological sifting where mass-like patterns are enhanced to improve the performance of Mask R-CNN. Transfer learning with the Mask R-CNN is then adopted to simultaneously detect and segment masses on the pseudo-color images. Evaluated on the public dataset INbreast, the method outperforms the state-of-the-art methods by achieving an average true positive rate of 0.90 at 0.9 false positive per image and an average Dice similarity index of 0.88 for mass segmentation.

Plant parasitic nematodes cause damage to crop plants on a global scale. Robust detetection on image data is a prerequisite for monitoring such nematodes, as well as for many biological studies involving the nematode C. elegans, a common model organism. Here, we propose a framework for detecting worm-shaped objects in microscopic images that is based on convolutional neural networks (CNNs). We annotate nematodes with curved lines along the body, which is more suitable for worm-shaped objects than bounding boxes. The trained model predicts worm skeletons and body endpoints. The endpoints serve to untangle the skeletons from which segmentation masks are reconstructed by estimating the body width at each location along the skeleton. With light-weight backbone networks, we achieve 75.85% precision, 73.02% recall on a potato cyst nematode data set and 84.20% precision, 85.63% recall on a public C. elegans data set.

Optical diffraction tomography (ODT) allows one to quantitatively measure the distribution of the refractive index of the sample. It relies on the resolution of an inverse scattering problem. Due to the limited range of views as well as optical aberrations and speckle noise, the quality of ODT reconstructions is usually better in lateral planes than in the axial direction. In this work, we propose an adaptive regularization to mitigate this issue. We first learn a dictionary from the lateral planes of an initial reconstruction that is obtained with a total-variation regularization. This dictionary is then used to enhance both the lateral and axial planes within a final reconstruction step. The proposed pipeline is validated on real data using an accurate nonlinear forward model. Comparisons with standard reconstructions are provided to show the benefit of the proposed framework.

Intracranial hemorrhage (ICH) is a critical disease that requires immediate diagnosis and treatment. Accurate detection, subtype classification and volume quantification of ICH are critical aspects in ICH diagnosis. Previous studies have applied deep learning techniques for ICH analysis but usually tackle the aforementioned tasks in a separate manner without taking advantage of information sharing between tasks. In this paper, we propose a multi-task fully convolutional network, ICHNet, for simultaneous detection, classification and segmentation of ICH. The proposed framework utilizes the inter-slice contextual information and has the flexibility in handling various label settings and task combinations. We evaluate the performance of our proposed architecture using a total of 1176 head CT scans and show that it improves the performance of both classification and segmentation tasks compared with single-task and baseline models.

The automatic reconstruction of single neuron cells from microscopic images is essential to enabling large-scale data-driven investigations in neuron morphology research. However, the performances of single neuron reconstruction algorithms are constrained by both the quantity and the quality of the annotated 3D microscopic images since the annotating single neuron models is highly labour intensive. We propose a framework for synthesizing microscopy-realistic 3D neuron images from simulated single neuron skeletons using conditional Generative Adversarial Networks (cGAN). We build the generator network with multi-resolution sub-modules to improve the output fidelity. We evaluate our framework on Janelia-Fly dataset from the BigNeuron project. With both qualitative and quantitative analysis, we show that the proposed framework outperforms the other state-of-the-art methods regarding the quality of the synthetic neuron images. We also show that combining the real neuron images and the synthetic images generated from our framework can improve the performance of neuron segmentation.

In this work, we present a new strategy for the multi-modal registration of atypical structures with boundaries that are difficult to define in medical imaging (e.g. lymph nodes). Instead of using a standard Mutual Information (MI) similarity metric, we propose to use the combination of MI with the Modality Independent Neighbourhood Descriptors (MIND) that can help enhancing the organs of interest from their adjacent structures. Our key contribution is then to learn the MIND parameters which optimally represent specific registered structures. As we register atypical organs, Neural-Network approaches requiring large databases of annotated training data cannot be used. We rather strongly constrain our learning problem using the MIND formalism, so that the optimal representation of images depends on a limited amount of parameters. In our results, pure MI-based registration is compared with MI-MIND registration on 3D synthetic images and CT/MR images, leading to improved structure overlaps by using MI-MIND. To our knowledge, this is the first time that MIND-MI is evaluated and appears as relevant for multi-modal registration.

The pulmonary fissures divide the lungs into lobes and canvary widely in shape, appearance, and completeness. Fis-sure completeness, or integrity, has been studied to assessrelationships with airway function measurements, chronicobstructive pulmonary disease (COPD) progression, and col-lateral ventilation between lobes. Fissure integrity measuredfrom computed tomography (CT) images is already usedas a non-invasive method to screen emphysema patients forendobronchial valve treatment, as the procedure is not ef-fective when collateral ventilation is present. We describea method for automatically computing fissure integrity fromlung CT images. Our method is tested using 60 subjectsfrom a COPD study. We examine the repeatability of fis-sure integrity measurements across inspiration and expirationimages, assess changes in fissure integrity over time using alongitudinal dataset, and explore fissure integrity?s relation-ship with COPD severity.

Decoding auditory stimulus from neural activity can enable neuroprosthetics and direct communication with the brain. Some recent studies have shown successful speech decoding from intracranial recording using deep learning models. However, scarcity of training data leads to low quality speech reconstruction which prevents a complete brain-computer-interface (BCI) application. In this work, we propose a transfer learning approach with a pre-trained GAN to disentangle representation and generation layers for decoding. We first pre-train a generator to produce spectrograms from a representation space using a large corpus of natural speech data. With a small amount of paired data containing the stimulus speech and corresponding ECoG signals, we then transfer it to a bigger network with an encoder attached before, which maps the neural signal to the representation space. To further improve the network generalization ability, we introduce a Gaussian prior distribution regularizer on the latent representation during the transfer phase. With at most 150 training samples for each tested subject, we achieve a state-of-the-art decoding performance. By visualizing the attention mask embedded in the encoder, we observe brain dynamics that are consistent with findings from previous studies investigating dynamics in the superior temporal gyrus (STG), pre-central gyrus (motor) and inferior frontal gyrus (IFG). Our findings demonstrate a high reconstruction accuracy using deep learning networks together with the potential to elucidate interactions across different brain regions during a cognitive task.

Myocardial infarction (MI) is a scientific term that refers to heart attack. In this study, we combine induction of highly specific second harmonic generation (SHG) signals from non-centrosymmetric macromolecules such as fibrillar collagens together with two-photon excited cellular autofluorescence in infarcted mouse heart to quantitatively probe fibrosis, especially targeted at an early stage after MI. We present robust one-shot machine learning algorithms that enable determination of spatially resolved 2D structural organization of collagen as well as structural morphologies in heart tissues post-MI with spectral specificity and sensitivity. Detection, evaluation, and precise quantification of fibrosis extent at early stage would guide one to develop treatment therapies that may prevent further progression and determine heart transplant needs for patient survival.

Removal of noise from fluorescence microscopy images is an important first step in many biological analysis pipelines. Current state-of-the-art supervised methods employ convolutional neural networks that are trained with clean (ground-truth) images. Recently, it was shown that self-supervised image denoising with blind spot networks achieves excellent performance even when ground-truth images are not available, as is common in fluorescence microscopy. However, these approaches, e.g. Noise2Void (N2V), generally assume pixel-wise independent noise, thus limiting their applicability in situations where spatially correlated (structured) noise is present. To overcome this limitation, we present Structured Noise2Void (StructN2V), a generalization of blind spot networks that enables removal of structured noise without requiring an explicit noise model or ground truth data. Specifically, we propose to use an extended blind mask (rather than a single pixel/blind spot), whose shape is adapted to the structure of the noise. We evaluate our approach on two real datasets and show that StructN2V considerably improves the removal of structured noise compared to existing standard and blind-spot based techniques.

Identifying changes in functional connectivity in Attention Deficit Hyperactivity Disorder (ADHD) using functional magnetic resonance imaging (fMRI) can help us understand the neural substrates of this brain disorder. Many studies of ADHD using resting state fMRI (rs-fMRI) data have been conducted in the past decade with either manually crafted features that do not yield satisfactory performance, or automatically learned features that often lack interpretability. In this work, we present a tensor-based approach to identify brain networks and extract features from rs-fMRI data. Results show the identified networks are interpretable and consistent with our current understanding of ADHD conditions. The extracted features are not only predictive of ADHD score but also discriminative for classification of ADHD subjects from typically developed children.

Compressed sensing (CS) reconstruction methods employing sparsity regularization and prior constraints are successfully applied in sparse-view computed tomography (CT) reconstruction and yield high-quality images compared with other low-dose imaging methods. In this paper, we proposed an adaptive prior patch size (APPS) strategy in sparse-view CT reconstruction. The method adopts sparse representation (SR) using adaptive patch size instead of a constant one to synthesize prior image of higher quality, because the optimal patch size should vary from each distribution range of local feature. The simulation experiments show that the proposed strategy has the better performance than the method with fixed patch size in terms of artifact reduction and edge preservation.

Histopathological images can reveal the reason and severity of the disease, so it plays an important role in clinical diagnosis. Due to the lack of clear correspondence between possible lesion area and the image feature of histopathology, and the lack of detailed annotated histopathological image sets, there are few researches on computer-aided diagnosis of skin cancer based on histopathological images. In addition, for medical imaging, results that are interpretable and consistent with medical knowledge are even more important. Based on the above analysis, we propose an improved and trainable C-ALGL model (CNN-AttendLSTM-GenerateLSTM Net), which can both generate meaningful visual image results and reasonable diagnostic descriptions from histopathological images. The model includes an image processing module, a diagnostic text processing module, and a visual image-diagnostic result text generating module, etc., as shown in the Figure 1. We propose an improved language model structure, which consists of three parts: Attention-LSTM, Attention, and Generate LSTM. The language model reduces the burden of learning both attention and generating diagnostic texts when there is only one layer of LSTM by adding an attention module to the LSTM mezzanine and changing the LSTM parameter transmission path. This module finally generates the corresponding visualized image of the lesion area and diagnoses the text. Our experiments were trained, validated, and tested on a skin histopathological image dataset collected from a major dermatology hospital located in Northeast China. The dataset contains 1.2K histopathological images of 11 skin diseases with corresponding disease category labels and diagnostic text, which was annotated and maintained by numbers of pathologists during the past ten years. We have selected several classical convolutional neural network frameworks for feature extraction of pathological images. Based on the feature of pathological images, we chose two popular image caption methods as the baseline to compare its performance with ours. The results are shown in Table 1, which indicate that our method has higher quality text generation ability for diagnosis results. Furthermore, the C-ALGL NET can be generated to other types of medical images with the purpose of obtaining meaningful diagnostic results.

X-ray PCDs are highly desirable in applications of material decomposition, high-resolution and low-dose imaging due to their low-noise nature and energy discrimination capability. However, practical use of PCDs is limited by several technical issues, especially the spectral distortions caused by charge splitting (CS) (charge sharing, K-shell fluorescence escape or re-absorption) and pulse pileup effects. Here, we propose a deep neural network-based data correction method for the long-standing distortion problems. In this pilot study, we demonstrate the feasibility of the approach with realistic synthetic PCD data generated with our PCD detection model incorporating both CS and PU effects. The testing results suggest the trained network's ability of recovering ideal spectrum from distorted data within 6% error. Major fidelity improvements in both projections and reconstructions are also demonstrated with qualitative and quantitative comparisons.

Recent advances in deep learning algorithms have triggered increased interest in applying machine learning techniques in all areas of signal processing. While classical machine learning approaches applying prior knowledge learned from training or navigator data are an integral part of many MR reconstruction techniques, following the success of deep learning approaches in the field of image processing, neural networks are now also increasingly used in MR image reconstruction. This talk will consider several different applications of machine learning in MRI reconstruction.

The brain's topological differences in resting state functional connectivity (rsfc) via resting state fMRI (rsfMRI) may provide important insight into brain function and dysfunction if heritable. Topological data analysis (TDA) is one such tool, robust to noise, for analyzing how the data's topological properties vary, and typically operates via filtration of the data and construction of a persistence diagram (PD). Therefore, the purpose of this study was to compute PDs to determine TDA-based heritability of static brain network topological features.

Pathological myopia, potentially causing loss of vision, has been considered as one of the common issues that threatens visual health of human throughout the world. Identification of retinal lesions including atrophy and detachment is meaningful because it provides ophthalmologists with quantified reference for accurate diagnosis and treatment. However, segmentation of lesions on fundus photography is still challenging because of widespread data and complexity of lesion shape. Fundus images may vary from each other distinctly as they are taken by different devices with different surroundings. False positive predictions are also inevitable on negative samples. In this paper, we propose originally invented lesion-aware segmentation network to segment atrophy and detachment on fundus images. Based on the existing architecture of paired encoder and decoder, we introduce three innovations. Firstly, our proposed network is aware of existence of lesion by including an extra classification branch. Secondly, feature fusion module is integrated to the decoder that makes the output node sufficiently absorbs the features in various scales. Last, the network is trained with an elegant objective function, called edge overlap rate, that eventually boosts the model?s sensitivity of lesion edges. The proposed network wins PALM challenge in ISBI 2019 with a large margin, that could be seen as evidence of effectiveness. Our team, PingAn Smart Health, leads the leaderboards in all metrics in scope of lesion segmentation. Permission of using the dataset outside PALM challenge was issued by the sponsor.

Brain magnetic resonance (MR) segmentation for hydrocephalus patients is considered as a challenging work. Encoding the variation of the brain anatomical structures from different individuals cannot be easily achieved. The task becomes even more difficult especially when the image data from hydrocephalus patients are considered, which often have large deformations and differ significantly from the normal subjects. Here, we propose a novel strategy with hard and soft attention modules to solve the segmentation problems for hydrocephalus MR images. Our main contributions are three-fold: 1) the hard-attention module generates coarse segmentation map using multi-atlas-based method and the VoxelMorph tool, which guides subsequent segmentation process and improves its robustness; 2) the soft-attention module incorporates position attention to capture precise context information, which further improves the segmentation accuracy; 3) we validate our method by segmenting insula, thalamus and many other regions-of-interests (ROIs) that are critical to quantify brain MR images of hydrocephalus patients in real clinical scenario. The proposed method achieves much improved robustness and accuracy when segmenting all 17 consciousness-related ROIs with high variations for different subjects. To the best of our knowledge, this is the first work to employ deep learning for solving the brain segmentation problems of hydrocephalus patients.

We introduce a novel method to extract the vessels in eye fundus images which is adaptive to lighting variations. In the Logarithmic Image Processing framework, a 3-segment probe detects the vessels by probing the topographic surface of an image from below. A map of contrasts between the probe and the image allows to detect the vessels by a threshold. In a lowly contrasted image, results show that our method better extract the vessels than another state-of the-art method. In a highly contrasted image database (DRIVE) with a reference, ours has an accuracy of 0.9454 which is similar or better than three state-of-the-art methods and below three others. The three best methods have a higher accuracy than a manual segmentation by another expert. Importantly, our method automatically adapts to the lighting conditions of the image acquisition.

Automatic change detection in longitudinal brain MRI classically consists in a sequential pipeline where registration is estimated as a pre-processing step before detecting pathological changes such as lesion appearance. Deformable registration can advantageously be considered over rigid or affine transform in the presence of geometrical distortions or brain atrophy to reduce false positive detections. However, this may be at the cost of underestimating the changes of interest due to the over-compensation of the differences between baseline and follow-up studies. In this article, we propose to overcome this limitation with a framework where deformable registration and lesion change are estimated jointly. We compare this joint framework with its sequential counterpart based on either affine or deformable registration. We demonstrate the benefits for detecting multiple sclerosis lesion evolutions on both synthetic and real data.

Bronchoscopy and three-dimensional (3D) computed tomography (CT) are important complementary tools for managing lung diseases. Endobronchial video captured during bronchoscopy gives live views of the airway-tree interior with vivid detail of the airway mucosal surfaces, while the 3D CT images give considerable anatomical detail. Unfortunately, little effort has been made to link these rich data sources. This paper describes a rapid interactive multi-frame method for registering the video frames constituting a complete bronchoscopic video sequence onto their respective locations within the CT-based 3D airway tree. Registration results using both phantom and human cases show our method?s efficacy compared to ground-truth data, with a maximum position error = 8.5mm, orientation error of 17 degrees, and a minimum trajectory accuracy = 94.1%. We also apply our method to multimodal 3D airway-wall analysis within a comprehensive bronchoscopic video analysis system.

Although deep encoder-decoder networks have achieved astonishing performance for mitochondria segmentation from electron microscopy (EM) images, they still produce coarse segmentations with discontinuities and false positives. Besides, the need for labor intensive pixel-wise annotations of large 3D volumes and huge memory overhead by 3D models are also major limitations. To address these problems, we introduce a multi-task network named EM-Net, which includes an auxiliary centerline detection task to account for shape in- formation of mitochondria represented by centerline. There- fore, the centerline detection sub-network is able to enhance the accuracy and robustness of segmentation task, especially when only a small set of annotated data are available. To achieve a light-weight 3D network, we introduce a novel hierarchical view-ensemble convolution module to reduce number of parameters, and facilitate multi-view information ag- gregation. Validations on public benchmark showed state-of- the-art performance by EM-Net. Even with significantly re- duced training data, our method still showed quite promising results.

Accurate choroid segmentation in optical coherence tomography (OCT) image is vital because the choroid thickness is a major quantitative biomarker of many ocular diseases. Deep learning has shown its superiority in the segmentation of the choroid region but subjects to the performance degeneration caused by the domain discrepancies (e.g., noise level and distribution) among datasets obtained from the OCT devices of different manufacturers. In this paper, we present an unsupervised perceptual-assisted adversarial adaptation (PAAA) framework for efficiently segmenting the choroid area by narrowing the domain discrepancies between different domains. The adversarial adaptation module in the proposed framework encourages the prediction structure information of the target domain to be similar to that of the source domain. Besides, a perceptual loss is employed for matching their shape information (the curvatures of Bruch?s membrane and choroid-sclera interface) which can result in a fine boundary prediction. The results of quantitative experiments show that the proposed PAAA segmentation framework outperforms other state-of-the-art methods.

Tooth instance segmentation provides important assistance for computer-aided orthodontic treatment. Many previous studies on this issue have limited performance on distinguishing adjacent teeth and obtaining accurate tooth boundaries. To address the challenging task, in this paper, we present a novel method achieving tooth instance segmentation and classification from cone beam CT (CBCT) images. The core of our method is a two-level hierarchical deep neural network. We first embed center-sensitive mechanism with global stage heatmap, so as to ensure accurate tooth centers and guide the localization of tooth instances. Then in the local stage, DenseASPP-UNet is proposed for fine segmentation and classification of individual tooth. Further, in order to improve the accuracy of tooth segmentation boundary and refine the boundaries of overlapped teeth, a boundary-aware dice loss and a novel label optimization are also applied in our method. Comparative experiments show that the proposed framework exhibits high segmentation performance and outperforms the state-of-the-art methods.

The Pancreatic beta cell is an important target in diabetes research. For scalable modeling of beta cell ultastructure, we investigate automatic segmentation of whole cell imaging data acquired through soft X-ray tomography. During the course of the study, both complete and partial ultrastrucutre annotations were produced manually for different subsets of the data. To more effectively use existing annotations, we propose a method that enables the application of partially labeled data for full label segmentation. For experimental validation, we apply our method to train a convolutional neural network with a set of 12 fully annotated data and 12 partially annotated data and show promising improvement over standard training that uses fully annotated data alone.

Three-dimensional (3D) H-scan ultrasound (US) is a new high-resolution imaging technology for voxel-level tissue classification. For the purpose of validation, a simulated H-scan US imaging system was developed to comprehensively study the sensitivity to scatterer size in volume space. A programmable research US system (Vantage 256, Verasonics Inc, Kirkland, WA) equipped with a custom volumetric imaging transducer (4DL7, Vermon, Tours, France) was used for US data acquisition and comparison to simulated findings. Preliminary studies were conducted using homogeneous phantoms embedded with acoustic scatterers of varying sizes (15, 30, 40 or 250 ?m). Both simulation and experimental results indicate that the H-scan US imaging method is more sensitive than B-mode US in differentiating US scatterers of varying size. Overall, this study proved useful for evaluating H-scan US imaging of tissue scatterer patterns and will inform future technology research and development.

Functional connectivity (FC) has been widely investigated to understand the cognition and behavior that emerge from human brain. Recently, there is overwhelming evidence showing that quantifying temporal changes in FC may provide greater insight into fundamental properties of brain network. However, scant attentions has been given to characterize the functional dynamics of network organization. To address this challenge, we propose a novel spatio-temporal hub identification method for functional brain networks by simultaneously identifying hub nodes in each static sliding window and maintaining the reasonable dynamics across the sliding windows, which allows us to further characterize the full-spectrum evolution of hub nodes along with the subject-specific functional dynamics. We have evaluated our spatio-temporal hub identification method on resting-state functional resonance imaging (fMRI) data from an obsessive-compulsive disease (OCD) study, where our new functional hub detection method outperforms current methods (without considering functional dynamics) in terms of accuracy and consistency. Index Terms? Dynamic functional network, brain network, graph spectrum, hub node

We propose a deep self-learning algorithm to learn the manifold structure of free-breathing and ungated cardiac data and to recover the cardiac CINE MRI from highly undersampled measurements. Our method learns the manifold structure in the dynamic data from navigators using autoencoder network. The trained autoencoder is then used as a prior in the image reconstruction framework. We have tested the proposed method on free-breathing and ungated cardiac CINE data, which is acquired using a navigated golden-angle gradient-echo radial sequence. Results show the ability of our method to better capture the manifold structure, thus providing us reduced spatial and temporal blurring as compared to the SToRM reconstruction.

In this paper, we propose a framework for accelerated reconstruction of 2D phase contrast magnetic resonance images from undersampled k-space domain by using deep learning methods. Undersampling in k-space violates Nyquist Sampling and creates artifacts in the image domain. In the proposed method, we consider the reconstruction problem as a de-aliasing problem in complex spatial domain. To test the proposed method, from fully sampled k-space data undersampling in k-space was performed in the phase-encode direction based on a probability density function which ensures maximum rate of sampling in low frequency regions. For the deep convolutional neural network (CNN) we chose the U-net architecture. The proposed CNN was trained and tested on 4D flow MRI data in 14 subjects with aortic stenosis. The reconstructed complex two channel image showed that the U-net is able to unaliase the undersampled flow images with resulting magnitude and phase difference images showing good agreement with the fully sampled magnitude and phase images. We show that the proposed method outperforms 2D compressed sensing approach of spatial total variation regularization method. Flow waveforms derived from reconstructed images closely follow flow waveforms derived from the original data. Moreover, the method is computationally fast. Each 2D magnitude and phase image is reconstructed within a second using a single GPU.

Skin melanoma represents a major health issue. Today, diagnosis and follow-up maybe performed thanks to computer-aided diagnosis tool, to help dermatologists segment and quantitatively describe the image content. In particular, deep convolutional neural networks (CNN) have lately been become the state-of-the-art in medical image segmentation. The loss function plays an important role in CNN in the process of backpropagation. In this work, we propose a metric-inspired loss function, based on the Kappa index. Unlike the Dice loss, the Kappa loss takes into account all the pixels in the image, including the true negative, which we believe can improve the accuracy of the evaluation process between prediction and ground truth. We demonstrate the differentiability of the Kappa loss and present some results on six public datasets of skin lesion. Experiments have shown promising results in skin lesion segmentation.

CT image denoising can be treated as an image-to-image translation task where the goal is to learn the transform between a source domain X (noisy images) and a target domain Y (clean images). Recently, cycle-consistent adversarial denoising network (CCADN) has achieved state-of-the-art results by enforcing cycle-consistent loss without the need of paired training data. Our detailed analysis of CCADN raises a number of interesting questions. For example, if the noise is large leading to significant difference between domain X and domain Y, can we bridge X and Y with a intermediate domain Z such that both the denoising process between X and Z and that between Z and Y are easier to learn? As such intermediate domains lead to multiple cycles, how do we best enforce cycle-consistency? Driven by these questions, we propose a multi-cycle-consistent adversarial network (MCCAN) that builds intermediate domains and enforces both local and global cycle-consistency. The global cycle-consistency couples all generators together to model the whole denoising process, while the local cycle-consistency imposes effective supervision on the process between adjacent domains. Experiments show that both local and global cycle-consistency are important for the success of MCCAN, which outperforms the state-of-the-art.

Magnetic Resonance Imaging (MRI) suffers from several artifacts, the most common of which are motion artifacts. These artifacts often yield images that are of non-diagnostic quality. To detect such artifacts, images are prospectively evaluated by experts for their diagnostic quality, which necessitates patient-revisits and rescans whenever non-diagnostic quality scans are encountered. This motivates the need to develop an automated framework capable of accessing medical image quality and detecting diagnostic and non-diagnostic images. In this paper, we explore several convolutional neural network-based frameworks for medical image quality assessment and investigate several challenges therein.

Prostate cancer (PCa) is the most prevalent and one of the leading causes of cancer death among men. Multi-parametric MRI (mp-MRI) is a prominent diagnostic scan, which could help in avoiding unnecessary biopsies for men screened for PCa. Artificial intelligence (AI) systems could help radiologists to be more accurate and consistent in diagnosing clinically significant cancer from mp-MRI scans. Lack of specificity has been identified recently as one of weak points of such assistance systems. In this paper, we propose a novel false positive reduction network to be added to the overall detection system to further analyze lesion candidates. The new network utilizes multiscale 2D image stacks of these candidates to discriminate between true and false positive detections. We trained and validated our network on a dataset with 2090 cases from seven different institutions and tested it on a separate independent dataset with 243 cases. With the proposed model, we achieved area under curve (AUC) of 0.876 on discriminating between true and false positive detected lesions and improved the AUC from 0.825 to 0.867 on overall identification of clinically significant cases.

Vessels are regions of prominent interest in retinal fundus images. Classification of vessels into arteries and veins can be used to assess the oxygen saturation level, which is one of the indicators for the risk of stroke, condition of diabetic retinopathy, and hypertension. In practice, dual-wavelength images are obtained to emphasize arteries and veins separately. In this paper, we propose an automated technique for the classification of arteries and veins from single-wavelength fundus images using convolutional neural networks employing the ResNet-50 backbone and squeeze-excite blocks. We formulate the artery-vein identification problem as a three-class classification problem where each pixel is labeled as belonging to an artery, vein, or the background. The proposed method is trained on publicly available fundus image datasets, namely RITE, LES-AV, IOSTAR, and cross-validated on the HRF dataset. The standard performance metrics, such as average sensitivity, specificity, accuracy, and area under the curve for the datasets mentioned above, are 92.8%, 93.4%, 93.4%, and 97.5%, respectively, which are superior to the state-of-the-art methods.

Currently, lung cancer is the most lethal in the world. In order to make screening and follow-up a little more systematic, guidelines have been proposed. Therefore, this study aimed to create a diagnostic support approach by providing a patient label based on the LUNG-RADS^TM guidelines. The only input required by the system is the nodule centroid to take the region of interest for the input of the classification system. With this in mind, two deep learning networks were evaluated: a Wide Residual Network and a DenseNet. Taking into account the annotation uncertainty we proposed to use sample weights that are introduced in the loss function, allowing nodules with a high agreement in the annotation process to take a greater impact on the training error than its counterpart. The best result was achieved with the Wide Residual Network with sample weights achieving a nodule-wise LUNG-RADS^TM labelling accuracy of 0.735?0.003.

Image normalization is a critical step in medical imaging. This step is often done on a per-dataset basis, preventing current segmentation algorithms from the full potential of exploiting jointly normalized information across multiple datasets. To solve this problem, we propose an adversarial normalization approach for image segmentation which learns common normalizing functions across multiple datasets while retaining image realism. The adversarial training provides an optimal normalizer that improves both the segmentation accuracy and the discrimination of unrealistic normalizing functions. Our contribution therefore leverages common imaging information from multiple domains. The optimality of our common normalizer is evaluated by combining brain images from both infants and adults. Results on the challenging iSEG and MRBrainS datasets reveal the potential of our adversarial normalization approach for segmentation, with Dice improvements of up to 59.6% over the baseline.

Wireless capsule endoscopy (WCE) is a minimally invasive procedure performed with a tiny swallowable optical endoscope that allows exploration of the human digestive tract. The medical device transmits tens of thousands of colour images, which are manually reviewed by a medical expert. This paper highlights the significance of using inputs from multiple colour spaces to train a classical U-Net model for automated semantic segmentation of eight WCE abnormalities. We also present a novel approach of grouping similar abnormalities during the training phase. Experimental results on the KID datasets demonstrate that a U-Net with 4-channel inputs outperforms the single-channel U-Net providing state-of-the-art semantic segmentation of WCE abnormalities.

Cardiac resynchronization therapy (CRT) can improve cardiac function in some patients with heart failure (HF) and dyssynchrony. However, as many as half of patients selected for CRT by conventional criteria (HF and ECG QRS broadening greater than 150 ms, preferably with left bundle branch block (LBBB) pattern) do not benefit from it. LBBB leads to characteristic motion changes seen with echocardiography and magnetic resonance imaging (MRI). Attempts to use echocardiography to quantitatively characterize dyssynchrony have failed to improve prediction of response to CRT. We introduce a novel hybrid model-based and machine learning approach to characterize regional 3D cardiac motion in dyssynchrony from MRI, using deformable models and deep learning. First, 3D left ventricle (LV) models of the moving heart are constructed from multiple planes of cine MRI. Using the conventional 17-segment model (AHA), we capture the regional 3D motion of each segment of the LV wall. Then, a neural network is used to detect and classify abnormalities of cardiovascular regional motions. Using over 100 patient data, we show that different types of dyssynchrony can be accurately demonstrated in 3D+t space and their correlation to CRT response.