Work

Multimodal Data Fusion and Feature Visualization in Convolutional Neural Networks

Public

Convolutional neural networks have become a staple in computer vision and image processing tasks. The capacity for these networks to perform visual pattern recognition in a data-driven fashion has prompted explosive growth in a myriad of applications. That said, despite their popularity, there are still facets of these networks that merit further investigation. This dissertation will describe two such directions. The first is related to multimodal data fusion, in which a network takes in multiple sources of data. The specific application in this instance is medical imaging, and the investigation gives insight into the relative efficacies of each data type in a traditional classification setting as well as a regression and longitudinal prediction scenario. The second part of the dissertation concerns deep visualization. This concept attempts to develop understanding of neural networks through the generation of descriptive images. Here, these techniques are applied to a variety of networks using standard computer vision datasets. Fundamentally, the multimodal fusion study highlights the potential power of the convolutional neural network, while the deep visualization study develops intuition and interpretability of these often obfuscated algorithms.

Creator
DOI
Subject
Language
Alternate Identifier
Date created
Resource type
Rights statement

Relationships

Items