Breaking the Energy Cage of Insect-Scale Drones: The Interplay of Probabilistic Hardware and Algorithms

Spatial intelligence, where vision and other sensors allow autonomous navigation of an embodied device, has critical use cases in surveillance, assistance and hazard detection. Also, there is tremendous focus towards extreme miniaturization of autonomous bots commonly termed as insect-scale drones or ant-robotics, due to their non-intrusive nature. However, insect-scale drones are extremely area/power constrained being so tiny. In addition to energy efficiency, robustness (or awareness) to flying domain variations and sensor inaccuracies, i.e. accounting prediction uncertainty, by intelligent systems is crucial for robust decision making. Bayesian probabilistic methods of inference are promising candidates to account for prediction uncertainty. However, they come with humongous computational complexity, posing a great challenge for edge deployment. To address these issues, we propose novel hardware frameworks for prediction confidence-aware, robust and ultra-low-power, classical reasoning-based and modern experience-based visual localization targeting insect-drone platforms. Localization is a critical subroutine in robotics estimating position and orientation (pose) of a mounted camera on a moving object. We exploit floating-gate CMOS transistor properties for a novel representation of map and perform highly efficient pose-likelihood evaluation for classical Bayesian particle filtering-based robust and energy-efficient localization. For experience-based localization, we propose a novel compute-in-SRAM framework with efficient probabilistic inference hardware primitives for Monte-Carlo Dropout approximation-based pose estimation with prediction uncertainty. We further propose a novel framework integrating deep neural network-based depth prediction and particle filtering for extremely lightweight visual localization, adaptive to depth inaccuracies and domain variations, resulting into highly accurate and robust pose prediction at low computation cost.