Distributed Deep Reinforcement Learning-Based Resource Management for Underwater Acoustic Communication Networks

This paper investigates the problem of distributed resource management in underwater acoustic communication networks (UACNs) involving multiple transmitters and receivers. In this setting, each transmitter autonomously selects a power allocation strategy based solely on local observations, without reliance on a central controller. Given that the optimization problem incorporating fairness and quality of service (QoS) constraints is non-convex and NP-hard, it is reformulated as a Markov Decision Process (MDP). To address the high complexity of underwater networks and the large state and action spaces, we propose a distributed learning framework based on a multi-agent dueling deep Q-network (MAD3QN). The proposed scheme enables each transmitter to dynamically adjust its transmission power based on local observations by integrating the Jain fairness index, QoS interruption penalty, and energy consumption constraints. Furthermore, by incorporating a dueling network architecture and a neighborhood cooperation mechanism, the learning efficiency is significantly enhanced, leading to a stable and effective resource optimization policy. Simulation results demonstrate that the proposed distributed learning algorithm outperforms existing approaches in terms of convergence speed, network fairness, and communication rate.