Towards Model Size-Agnostic Memory-Based Inference of Deep Neural Networks

The rapid advancement of deep neural networks has led to significant improvements in various tasks such as image and speech recognition. However, as the complexity of these models increases, so does the computational cost and the number of parameters, making it difficult to deploy them on resource-constrained devices [1]. In this thesis, to address this issue, I will propose a novel approach called Memory-Based Inference (MBI). MBI’s objective is to eliminate the heavy and time-consuming computation required in- ference. As networks grow deeper and larger, the power and time required for inference also increase, making it more difficult to deploy them in constrained applications. MBI addresses this issue by solely relying on a pre-trained table made by inputs and outputs of the model. Without computing matrix or vector multiplications, MBI extracts outputs directly from the trained table, comparing the new sample with the inputs present in the table. As a result, inference becomes size-agnostic and is no longer dependent on the depth or size of the network. Furthermore, this approach is highly scalable, even when modifications are made to the network’s structure.