Imitation Learning for Autonomous Highway Merging with Safety Guarantees

Thanks to the innovations in Artificial Intelligence, the autonomous driving field experienced an enormous growth in recent years, opening the road to a new safe and efficient way of conceiving transportation. One of the most challenging aspects is designing a driverless car able to safely navigate the highway. A particularly critical maneuver is merging in the highway traffic coming from an on-ramp, which will be the focus of this thesis. The described task is an highly interactive process, that requires an advanced level of cooperation between drivers. In similar cases, machine learning techniques have demonstrated to be more efficient than manually designed rule-based approaches. In particular, in this work we consider the Imitation Learning (IL) approach, whose objective is to learn how to perform a task by imitating the demonstrations of an expert. The result is a policy that the agent can follow to perform as similar as possible to the expert. Behavioral Cloning (BC) and Inverse Reinforcement Learning (IRL) are the two main approaches in Imitation Learning. The former uses supervised learning to find a policy that imitates the expert. This method is simple and efficient but it suffers from cascading error. The latter, instead, resorts to methods of solving an MDP when the reward function is unknown. This approach allows to learn a reward function, that constitutes a transferable representation of the desired behavior, but it is computationally more expensive and requires the knowledge of the model dynamics. The imitation technique proposed in this thesis builds on the Behavioral Cloning approach and augments it with a safety filter that covers the cascading error issue. This approach preserves the benefits of using a BC-based technique, while overcoming its limitations with an additional controller to ensure safety. In this work, we explore the state-of-the-art of Imitation Learning, present our method and analyze the results of applying it to the highway merging scenario.