PLUTO: a POMDP-based Serverless Autoscaler for FPGA-as-a-Service Accelerated Functions

Field Programmable Gate Arrayss (FPGAs) in serverless platforms allow to share accelerators and reduce overprovisioning of resources. Scaling serverless functions depending on user needs is a challenging task that is currently addressed with underperforming threshold-based approaches, especially in the case of FPGAs. In this thesis, we will present PLUTO, an autoscaler for FPGA-as-a-Service accelerated functions based on a Partially Observable Markov Decision Process (POMDP). First, we will introduce the work by dening the background and the addressed problem. We will analyze the current State of the Art (SoA), represented by the threshold-based autoscalers used by the main providers and other Reinforcement Learning (RL) solutions proposed by recent research. Then, we will present the implementation of PLUTO, a decision agent that works as a decentralized per-application autoscaler, building a model of the application without expensive data-driven computations. The thesis will explain both the POMDP modeling and resolution, and the integration of the abstract model with the real system. We will then show the results of the tests on the real system, analyzing and comparing them with two of the most popular autoscalers currently available - Kubernetes's Horizontal Pod Autoscaler (HPA) and OpenFaas's autoscaler. We will demonstrate that the proposed solution has an overall better performance w.r.t. the two aforementioned approaches. In fact, PLUTO proved to be able to maintain a good Quality of Service (QoS) in terms of service latency with negligible violations, while avoiding unnecessary scaling operations that would lead to a wastage of resources. Finally, we will describe some interesting future works that can be carried out to enhance the performance and flexibility of the proposed solution.