Resource Allocation for Vehicular Fog Computing Using Reinforcement Learning Combined with Heuristic Information

Internet of Vehicles (IoV) has emerged as a key component of smart cities. Connected vehicles are increasingly processing real-time data to respond immediately to user requests. However, the data must be sent to a remote cloud for processing. To resolve this issue, vehicular fog computing (VFC) has emerged as a promising paradigm that improves the quality of computation experiences for vehicles by offloading computation tasks from the cloud to network edges. Nevertheless, due to the resource restrictions of fog computing, only a limited number of vehicles are able to use it while it is still challenging to provide real-time responses for vehicular applications, such as traffic and accident warnings in the highly dynamic IoV environment. Therefore, in this article, we formulate the problem of allocating the limited fog resources to vehicular applications such that the service latency is minimized, by utilizing parked vehicles. We then propose a heuristic algorithm to efficiently find the solutions of the problem formulation. In addition, the proposed algorithm is combined with reinforcement learning to make more efficient resource allocation decisions, leveraging the vehicles' movement and parking status collected from the smart environment of the city. Our simulation results show that our VFC resource allocation algorithm can achieve higher service satisfaction compared to conventional resource allocation algorithms.