Intraoperative Artificial Intelligence and Surgical Data Science in Ophthalmology

In recent years, the use of artificial intelligence (AI)-based solutions in the operating room enabled significant advancements in surgical procedures. The integration of AI technology has revolutionized various aspects of surgical care, enhancing precision, efficiency, and patient outcomes. The importance of these solutions can be translated into data-driven decision-making, where AI algorithms analyze vast amounts of patient data, enabling surgeons to make more informed choices regarding treatment plans and surgical approaches. Additionally, AI-powered systems can assist in real-time monitoring during procedures, providing valuable feedback and enhancing surgical precision. Furthermore, these solutions have the potential to revolutionize surgical training and understanding by providing virtual simulations and personalized learning experiences for aspiring ophthalmic surgeons. Despite these advancements, the implementation of such AI tools in ophthalmic procedures is still limited data retrieving and understanding of intraoperative data in ophthalmic microsurgery procedures is restricted, and surgical behavior of surgeons with heterogeneous data modalities has not yet been analyzed in the field. In this thesis, we begin our investigation using deep learning neural networks for the semantic understanding of instruments and tissues in vitreoretinal procedures, from the process of dataset acquisition and annotation, up to the deployment of a model that can be generalized throughout a set of heterogeneous surgical tasks. We then develop upon this initial work a platform for the extraction of data on surgical maneuvers and instrument-tissue interactions. We further explore the data acquired from the semantic understanding of vitreoretinal surgical procedures to identify objective performance metrics. By tracking the eye gaze of subjects, we explore the divergences in the visual behavior of ophthalmologists when observing surgical procedures according to their experience level. Concluding our research, we investigate in a surgical training setting how residents, fellows, and attending surgeons’ surgical behavior can be assessed by employing deep learning neural networks in order to automate the classification of their surgical experience.