BD-RIS-Aided RSMA-ISAC Systems: Joint Rate Splitting and Beamforming Design

An integrated sensing and communication (ISAC) system with a fully connected beyond-diagonal reconfigurable intelligent surface (BD-RIS) and rate splitting multiple access (RSMA) is investigated. To maximize the sum rate of multiple users while meeting sensing quality requirements, we jointly optimize the active beamforming at the base station and the reflective beamforming at the BD-RIS. The nonconvex optimization problem is addressed using an alternating algorithm based on weighted minimum mean square error (WMMSE), penalty-dual-decomposition (PDD), and successive convex approximation (SCA). Simulation results validate the benefits of the proposed scheme and the performance gains from deploying BD-RIS in RSMA-ISAC systems.