Overlapping Coalition Formation-Enabled Noncooperative Game-Combined Multi-Agent DRL for UAV-Assisted Resource Allocation

Coalition Formation (CF) game emerges as a pioneering framework for resource allocation in uncrewed aerial vehicles (UAVs) equipped with various types of complementary resources. However, both the overlapping-enabled collaborative CF and inter-coalition competitive behaviors significantly impact the system performance in complex multi-UAV scenarios. In this paper, we propose a Multiple Overlapping Coalitions (MOC) noncooperative game. Specifically, we first establish an optimization model encompassing coupled resource constraints. Subsequently, a task-priority-based incentive mechanism is designed to better motivate participation. To achieve the Nash equilibrium, a two-step solution technique incorporating relaxation and fine-tuning of resource granularity is designed. We propose a MOC noncooperative game-combined Multi-agent Proximal Policy Optimization (MAOPPPO). The simulation results substantiate that our approach outperforms the other five state-of-the-art learning countermeasures in terms of average reward with a gain of up to 4.59% after 800 training episodes. In terms of throughput, the proposed MOC noncooperative game increases by 66.67%, 93.68%, and 11.76% compared with that of CF noncooperative game, non-CF noncooperative game, and consensus-based algorithm, respectively. For total resource contribution, the improvements are 62.99%, 94.59%, and 23.16%, respectively. The energy efficiency enhances by 6.82%, 23.68%, and 4.78% compared to the other three baselines, respectively.