doc-exports/docs/cce/umn/cce_10_0645.html

Preparing Virtualized GPU Resources

CCE uses xGPU virtualization technologies to dynamically divide the GPU memory and computing power. A single GPU can be virtualized into a maximum of 20 virtual GPU devices. This section describes how to implement GPU scheduling and isolation capabilities on GPU nodes.

Prerequisites

Item	Supported Version
Cluster version	v1.23.8-r0, v1.25.3-r0, or later
OS	HCE OS 2.0 with the kernel version of 5.10 or later
GPU type	Tesla T4 and Tesla V100
Driver version	570.86.15, 535.216.03, 535.54.03, 510.47.03, and 470.57.02
CUDA version	CUDA 12.2.0 to 12.8.0
Runtime	containerd
Add-on	The following add-ons must be installed in the cluster: Volcano Scheduler: 1.10.5 or later CCE AI Suite (NVIDIA GPU): 2.0.5 or later

Step 1: Enable GPU Virtualization

Both CCE AI Suite (NVIDIA GPU) and Volcano Scheduler must be installed in the cluster.

1. Log in to the CCE console and click the cluster name to access the cluster console. In the navigation pane, choose Settings.
2. Switch to the Heterogeneous Resources tab and enable GPU Virtualization.
   • Node pool-level GPU virtualization: If CCE AI Suite (NVIDIA GPU) of version 2.7.2 or later is installed, GPU virtualization can be configured by node pool.
     1. In Node Pool Configurations under GPU Settings, click Add.
     2. In the Node Pools list, select the node pool where you want to enable GPU virtualization and choose a driver that supports GPU virtualization from Driver. After you customize a GPU driver for a node pool, nodes in that pool will preferentially use the custom driver. Nodes for which no driver is specified will use the cluster's default driver.
        

        • The specified driver version will be installed for new nodes added to the pool. This configuration does not affect existing nodes in the pool.
        • An updated driver version applies only to new nodes added to the node pool. Existing nodes must be restarted to apply the changes.
     3. Click under GPU Virtualization to enable GPU virtualization for the node pool. To configure GPU virtualization for multiple node pools, click Add.
     4. In the lower right corner of the page, click Confirm Settings.

3. After configuring GPU virtualization, verify the settings.

   In the navigation pane, choose Cluster > Nodes. In the right pane, click the Nodes tab and find the node where GPU virtualization has been configured. In the Operation column of the target node, choose More > View YAML. If the node-status.volcano.sh/nvidia value in the YAML file is {"enableXGPU":true}, GPU virtualization has been configured on the node.

Step 2: Create a GPU Node

Create nodes that support GPU virtualization in the cluster to use the GPU virtualization function. For details, see Creating a Node or Creating a Node Pool. If there are GPU nodes in your cluster that meet the prerequisites requirements, skip this step.

Parent topic: GPU Virtualization