Multi-Agent Continual Deep Reinforcement Learning with Artificial Potential Fields and Switching Control for a Cooperative Robotic Task

Autonomy and intelligence are the key directions of modern manufacturing, with Deep Reinforcement Learning (DRL) providing important technical support for robots to acquire skills autonomously. Multi-agent reinforcement learning (MARL) offers a principled framework for coordinated decision-making in collaborative robotics, yet it often suffers from poor sample efficiency and brittleness when demonstrations are limited. To address these issues, we propose a unified framework that couples Multi-Agent Artificial Potential Fields (MAPF) with Behavior Cloning (BC), continual adaptation, and a safety-oriented switching controller. Specifically, MAPF generates structured expert rollouts for joint-space bimanual control under explicit joint constraints, yielding an effectively unlimited demonstration stream. We then train a policy via a critic-gated BC loss and integrate an online continual refreshing mechanism to mitigate BC overfitting during long-horizon training. Finally, we deploy a switching controller that uses the learned policy as the default solver and falls back to MAPF when the time budget is exceeded, improving reliability while retaining low runtime. We evaluate the approach on two RoboSuite bimanual tasks to test generalization under goal variations. Under identical observations and rewards, we additionally include a strong SAC-based CTDE/off-policy backbone within the same MABCRL pipeline (MA(SAC)BCRL) for comprehensive comparison. Across both tasks, the proposed MABCRL training improves learning stability and final performance compared with plain MARL and naive BC. On ENV1, MA(DDPG)BCRL achieves $$\varvec{0.87\pm 0.02}$$ success, and switching further boosts success to $$\varvec{0.99\pm 0.004}$$ while maintaining fast execution (about $$\varvec{9}$$ – $$\varvec{11}$$ s/episode versus $$\varvec{\sim 34}$$ s/episode for MAPF). On ENV2, switching similarly reaches $$\varvec{0.99\pm 0.005}$$ success. We further show that performance remains essentially unchanged under object-state occlusion with a hold scheme, suggesting that the overall framework is robust to transient object-state missingness under the observation degradation applied to both the learned policy and MAPF. These results suggest that combining MAPF demonstrations, continual adaptation, and switching control is an effective route to reliable and efficient cooperative robotic manipulation.