Distributed DRL-Based Downlink Power Allocation for Hybrid RF/VLC Networks

Hybrid radio frequency (RF) and visible light communication (VLC) networks can provide high throughput and energy efficiency with VLC access points (APs) while ensuring ubiquitous coverage with RF APs. Due to dynamic channel conditions and limited resources, the hybrid RF/VLC networks' resource allocation problem is complex and challenging. Conventional resource allocation techniques fail to overcome these challenges. Heuristic methods can solve high complexity problems; however, they are not robust against changes such as dynamic channel conditions or alternating user requirements. Heuristic methods require centralized control for stability which adds communication overhead between APs. Deep Reinforcement Learning (DRL) based solutions can solve high complexity, dynamic channel conditions, and alternating user requirements while not requiring centralized control. In this paper, we formulate a distributed downlink power allocation problem to optimize the transmit power for users to reach target data rates in hybrid RF/VLC networks. Then, we propose a distributed DRL-based algorithm Deep Deterministic Policy Gradient (DDPG), to solve the formulated computationally-intensive problem. We implement a simulation environment to benchmark the proposed distributed DRL-based method against other methods such as Q-Learning (QL) and Deep Q-Networks (DQN), and centralized heuristic power allocation algorithms. Our simulation results show that the distributed DDPG-based algorithm learns to adapt against changes in the channel or user requirements, while centralized Genetic Algorithm and Particle Swarm Optimization-based algorithms fail to endure against these changes even with coordination between APs. Additionally, we quantify the performance of the DDPG-based algorithm to prevail amid DRL-based algorithms at the expense of higher implementation complexity.