Novel Neural Networks Solutions for Dental X-ray Analysis Using Input-Normalized Gradient Descent

In this dissertation, we introduce novel deep learning models for the precise identification and classification of medical images within the domain of dentistry. Additionally, we present an innovative optimization algorithm meticulously designed to elevate the efficacy of our proposed deep learning architectures. We begin by automating the detection and classification of Cervical Vertebrae Maturation (CVM) stages. Cervical Vertebra Maturation (CVM) staging in lateral cephalometric radiographs offers an orthodontic alternative for assessing skeletal maturation without the need for additional radiographs, as these are routinely used for diagnosis and treatment planning in orthodontics. Given the lateral cephalograms, we build a machine learning based pipeline that is constructed by a deep learning model and directional filters to segment and classify the X-ray image based on its CVM stage. The images need to be segmented and filtered to extract and highlight the relevant information, which is then used to train a convolutional neural network for the classification of the CVM stages. We observe that the proposed pipeline can automate the process and eliminate the limitation of traditional CVM method which requires access to experienced practitioners and is not user-friendly. Moreover, we show that the classification accuracy can be improved with the use of convolutional neural networks connected in a parallel configuration. Multi-channel convolutional networks exhibit improved performance in image classification by concurrently processing the same input through different sub-networks and merging their outputs before reaching the final layers. To enhance our design, we employ window patching, inspired by vision transformers, which have demonstrated superior performance in benchmark image classification tasks. While the maturation staging in the CVM method is discrete, that is, the number of stages are finite, we also explore the option of creating a continuous scale system to assess the CVM stages and the skeletal maturity. As human growth and development is inherently not discrete in nature, a continuous scale system is better suited for the maturation staging. The model to classify the lateral cephalogram is used as the baseline and an additional regression model is incorporated to generate a Continuous-Valued Cervical Vertebrae Maturity (CVCVM) parameter. In addition to investigating Cervical Vertebrae Maturation staging, we also examined the problem of third molar development staging. We explored several deep learning models for classifying third molar development stages. Similar to CVM staging, estimating the development stages of third molars offers valuable information for determining dental age, which, in turn, can be utilized in dental diagnosis, treatment planning, and forensic applications. We introduced a novel optimization algorithm for training machine learning models called Input Normalized Stochastic Gradient Descent (INSGD), inspired by the Normalized Least Mean Squared (NLMS) algorithm used in adaptive learning. The algorithm also exhibits improvement in training deep learning algorithms in large benchmark data sets. When training complex models on large datasets, the robustness to variability is crucial to avoid divergence. Our algorithm updates the network weights using stochastic gradient descent with $\ell_1$ and $\ell_2$-based normalization applied to the update term, similar to NLMS. The proposed optimization algorithm also shows better performance in the classification of CVM staging. By incorporating input normalization into backpropagation, INSGD has the potential to replace normalization layers, reduce overfitting, and find use in online learning applications.