Multi View Learning Through Different Levels of Abstraction Extracted by Deep Neural Networks

The present work focuses on combining two novel approaches that are becoming more and more important nowa- days, namely neural networks (NN) and multi-view learning. Thanks to the contribution of Geoffrey Hinton, neural networks, in particular deep learning approaches, became increasingly important to the point of being a key part in various systems for different disciplines. For instance, the convolutional neural network (CNN) is the best performing architecture for visual recognition tasks. On the other hand, nowadays, we have different sources to gather data and information for a tasks of our interest. For example, on a website we can have both images and texts that describe the same event, or various newspapers can refer to the same story so we will have different points of view. The goal of multi view learning (MVL) is to increase the performance and improve the results by exploiting the agreement and complementary information between different sources. However, even if we have large datasets, sometimes it is too expensive to collect data from different sources or simply, we do not have them. It is possible that there is not enough information or there is access only to one view. To avoid this problem, different techniques were developed in order to extract multiple views from one single view. This research combines NN with MVL techniques based on the following intuition. Usually, in a deep neural network (DNN) we use only the ”features” of the last layer to make a prediction ignoring all the features that we can extract from the previous hidden layers. The underlying idea of this thesis is to treat each layer as a different view of the original object and combine them to improve the performance of the classic DNN. For the best of my knowledge, this is the first attempt in the literature and different possibilities of such approach are explored in this work.