DRL-Empowered Secure UAV-RIS Communication: A Multihead DSAC Optimization Framework

This work investigates a secure uncrewed aerial vehicle (UAV)-assisted communication system enhanced by a ground-deployed reconfigurable intelligent surface (RIS), aiming to ensure secure communications for multiple ground users under the threat of a malicious eavesdropper. The core challenge lies in the joint optimization of UAV trajectory, beamforming vectors, and RIS phase shifts to maximize the average secrecy rate, which is a highly nonconvex problem involving coupled continuous and discrete variables. To address this, we propose a multihead distributional soft actor–critic (DSAC) framework that formulates the problem as a Markov decision process (MDP) and integrates continuous flight control variables, including UAV position and beamforming and discrete communication variables, which are RIS phase shifts into a unified reinforcement learning model. The proposed framework employs a multihead actor network to handle mixed actions and distributional critic networks to enhance value estimation robustness. Simulation results demonstrate that the proposed multihead DSAC outperforms existing reinforcement learning algorithms in terms of average secrecy rate and convergence stability, validating its effectiveness in dynamic and hostile wireless environments.