REPRESENTATION LEARNING IN MULTIMODAL SPATIOTEMPORAL IMAGE-GUIDED MEDICAL PROCEDURES

Abstract

Medical image computing and computer-assisted analytics are playing a vital role in healthcare by helping early and accurate diagnosis as well as guidance in the intervention. Advances in imaging technology and robotics increase the demand for developing image-guided computational models to analyze the data and assist in clinical decision making. In this thesis, we propose several deep convolutional neural networks (DCNNs) to automate and enhance the image-guided medical procedures by addressing challenges of (1) insufficient and imbalanced dataset, (2) synthesizing missing modality, (3) processing spatiotemporal, multimodal and high-resolution data for online application, and (4) designing and optimizing multitask learning (MTL) model to enable the system with concurrent tasks at a time. The thesis focuses on medical applications such as detection and outcome prediction of brain tumor, ischemic stroke, intracerebral hemorrhage, and tracking and scanpath prediction in the image-guided intervention using imaging sources of MRI, CT, and endoscope.