MxGPU: Efficient and safe communication between GPGPU applications in an OS-controlled GPGPU multiplexing environment

With the growing demand for artificial intelligence and other data-intensive applications, the demand for graphics processing units (GPUs) has also increased. Even though there are many approaches on multiplexing GPUs, none of the approaches known to us enable the operating system to coherently integrate GPU resources alongside CPU resources into a holistic resource management. Due to the history of GPUs, GPU drivers are still a large, isolated part within the driver stack of operating systems. This paper aims to conduct a case study on how a multiplexing solution for GPGPUs could look like, where the OS is able to define scheduling policies for GPGPU tasks and manage GPU memory. OS-controlled GPU memory management can especially be helpful for efficient and safe communication between GPGPU applications. We will discuss and evaluate the architecture of MxGPU , which offers software-based multiplexing of integrated Intel GPUs. MxGPU has a tiny code base, which is a precondition for formal verification approaches and usage in safety-critical environments. Experiments with our prototype show that MxGPU can grant the operating system control over GPU resources while allowing more GPU sessions. Furthermore, MxGPU allows for execution of GPGPU tasks with less latency compared to Linux and enables efficient and safe communication between GPU applications. • MxGPU has a source code complexity much lower compared to existing solutions. • MxGPU supports any number of GPU sessions as long as enough memory is available. • MxGPU allows the operating system manage GPU resources by defining scheduling policies and managing GPU memory. • MxGPU has less overhead than Linux.