Integrated Energy Scheduling and Routing for a Network of Mobile Prosumers

Due to the spatio-temporal complexity in energy consumption profiles of multiple consumers at different regions, it is expected that more cost savings and higher resiliency to power disruptions can be achieved using a network of mobile prosumers, defined as entities that can both provide and consume energy. In this dissertation, the prosumer is assumed to be an autonomous vehicle equipped with different distributed energy resources (e.g., solar panel, battery). In addition, the demand on energy is uncertain and subject to change with time due to several factors including the emergence of new technology, entertainment, divergence of people’s consumption habits, changing weather conditions, etc. Moreover, increases in energy demand are growing every day due to increases in world population and growth of the global economy, which substantially increase the chances of disruptions in power supply. This makes the security of power supply a more challenging task especially during peak usage seasons (e.g., summer and winter). Furthermore, the majority of the applications of smart grids in the existing literature include a large number of agents (e.g., electric vehicles), and each agent interacts with other agents and affects their decisions (i.e., mobility and scheduling). This causes the operational complexity of smart grids to increase exponentially. To date, most of the existing research in energy management in smart grids aims to find efficient controlling schemes that seek renewable energy resources and dispatch energy to shift energy peaks between different daily periods. However, there are limitations and gaps in the current state-of-art in the field of energy management in smart grid including: (i) most of the literature focuses on energy transactions between EV's and power grids while the benefits of mobility in EV to mitigate the impact of spatio-temporal complexity in energy load and production are not maximally exploited; (ii) the existing solution approaches (e.g., mixed integer programming, dynamic programming, etc.) are not computationally efficient to handle system dynamics with the uncertainty inherent in energy problems; (iii) traditional/exact optimization tools are computationally expensive and are not efficient in solving high dimensional energy problems. Thus, the aim of this dissertation is to thus advance the state-of-the-art in energy management of smart grids to contribute to reducing energy costs, dependency of the main power grid, computational time, and carbon emissions. This is done in several stages. First, an integrated vehicle routing and energy scheduling decision model is proposed to adaptively dispatch vehicles to balance the temporally and spatially distributed energy requests subject to vehicle mobility constraints, and thus to maximally exploit the potential of a mobile prosumer network for cost savings and carbon emission reductions. Second, a reinforcement learning model is developed to address the uncertainties in power supply and demand by dispatching a set of electric vehicles to supply energy to different consumers at different locations. Finally, a multiagent reinforcement learning (MARL) algorithm is introduced to address the scalability issues of large-scale smart grid systems. This algorithm uses centralized training and distributed execution, where all agents are trained using an actor network for each agent and sharing the same critic network, and then executed to make actions independent of other agents to reduce computation time.