yexinru/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
Files
main
doc-exports/docs/cce/umn/cce_10_0652.html
qiujiandong1 ab1e53a279 CCE UMN 20251031 version 
Reviewed-by: Gergo-Bence Lorincz <a200452876@noreply.gitea.eco.tsi-dev.otc-service.com>
Co-authored-by: qiujiandong1 <qiujiandong1@huawei.com>
Co-committed-by: qiujiandong1 <qiujiandong1@huawei.com>
2026-01-15 10:25:22 +00:00

90 KiB
Raw Permalink Blame History

Managing Node Pool Configurations

If the default node configurations in a cluster do not meet service requirements, you can fine-tune parameters for core components, such as kubelet, kube-proxy, and the container engine, within the node pool. Fine-tuning these parameters ensures efficient resource utilization and meet workload demands.

The following parameters can be configured for a node pool:

Notes and Constraints

  • The configuration parameters of the default node pool (specified by DefaultPool) cannot be modified.
  • This function is supported only in clusters of v1.15 or later. It is not displayed for versions earlier than v1.15.

Using the Console

  1. Log in to the CCE console and click the cluster name to access the cluster console.
  2. In the navigation pane, choose Nodes. Then click the Node Pools tab.
  3. Locate the target node pool and choose More > Manage.

  4. In the Manage Configurations window, modify the node pool parameter values based on service requirements. For details about the parameters, see kube-apiserver configurations.

Using an API

You can modify the CCE cluster parameter configurations using an API.

PUT /api/v3/projects/{project_id}/clusters/{cluster_id}/nodepools/{nodepool_id}/configuration

Parameter

Description

project_id

Project ID.

cluster_id

Cluster ID.

nodepool_id

Node pool ID.

The request body is as follows:

{
  "kind" : "Configuration",
  "apiVersion" : "v3",
  "metadata" : {
    "name" : "configuration"
  },
  "spec" : {
    "packages" : [ {
      "name" : "kubelet",
      "configurations" : [ {
        "name" : "system-reserved-mem",
        "value" : 600
      }, {
        "name" : "kube-reserved-mem",
        "value" : 800
      } ]
    } ]
  }
}

The following parameters are modified:

  • system-reserved-mem: Set it to 600 MiB.
  • kube-reserved-mem: Set it to 800 MiB.

kubelet

Item

Parameter

Description

Value

Modification

Configuration Method

CPU Management Policy

cpu-manager-policy

CPU management policy configuration. For details, see CPU Scheduling.

  • none: disables pods from exclusively occupying CPUs. Select this value if you want a large pool of shareable CPU cores.
  • static: enables pods to exclusively occupy CPUs. Select this value if your workload is sensitive to latency in CPU cache and scheduling.
  • enhanced-static: allows burstable pods to preferentially use CPU cores. Select this value if your workload has huge peak-trough difference and is in the trough state most of the time.

Default: none

The CPU management policy does not apply to ECS (PM) node pools in CCE Turbo clusters.

Console/API

QPS for Requests to kube-apiserver

kube-api-qps

Number of queries per second for communication with the API server.

Default: 100

None

Console/API

Burst for Requests to kube-apiserver

kube-api-burst

Maximum number of burst requests sent to the API server per second.

Default: 100

None

Console/API

Maximum Pods Managed by kubelet

max-pods

Maximum number of pods that can run on a node.

None

Console/API

Maximum Processes in Pod

pod-pids-limit

Maximum number of PIDs that can be used in each pod.

Default: -1, which indicates that the number of PIDs is not limited

None

Console/API

Use a Local IP Address as a Node's ClusterDNS

with-local-dns

The node's kubelet configuration will automatically use the default network interface's IP address as the preferred DNS address.

Default: false

None

Console/API

Event QPS

event-qps

Number of events that can be generated per second.

Default: 5

None

Console/API

Upper Limit for Burst Events

event-burst

Upper limit for burst event creation. The number of burst events can be temporarily increased to the specified value.

Default: 10

None

Console/API

Allowed Unsafe sysctls

allowed-unsafe-sysctls

Insecure system configuration allowed.

Starting from v1.17.17, CCE enables pod security policies for kube-apiserver. Add corresponding configurations to allowedUnsafeSysctls of a pod security policy to make the policy take effect. (This configuration is not required for clusters earlier than v1.17.17.) For details, see Example of Enabling Unsafe sysctls in a PSP.

Default: []

None

Console/API

Topology Manager Policy

topology-manager-policy

Set the topology management policy.

Valid values are as follows:

  • restricted: kubelet accepts only pods that achieve optimal NUMA alignment on the requested resources.
  • best-effort: kubelet preferentially selects pods that implement NUMA alignment on CPU and device resources.
  • none (default): The topology management policy is disabled.
  • single-numa-node: kubelet allows only pods that are aligned to the same NUMA node in terms of CPU and device resources.

Default: none
NOTICE:

Modifying topology-manager-policy and topology-manager-scope will restart kubelet, and the resource allocation of pods will be recalculated based on the modified policy. In this case, running pods may restart or even fail to receive any resources.

Console/API

Topology Manager Scope

topology-manager-scope

Configure the resource alignment granularity of the topology management policy. Valid values are as follows:

  • container (default)
  • pod

Default: container

Console/API

Specified DNS Configuration File

resolv-conf

DNS resolution configuration file specified by the container

Default: null

None

Console/API

Timeout for All Runtime Requests Except Long-running Requests

runtime-request-timeout

Timeout interval of all runtime requests except long-running requests (pull, logs, exec, and attach).

Default: 2m0s

This parameter is available only in clusters of v1.21.10-r0, v1.23.8-r0, v1.25.3-r0, or later versions.

Console/API

Allow kubelet to Pull Only One Image at a Time

serialize-image-pulls

Pull an image in serial mode.

  • false: recommended configuration so that an image can be pulled in parallel mode to improve pod startup.
  • true: allows images to be pulled in serial mode.
  • Enabled by default if the cluster version is earlier than v1.21.12-r0, v1.23.11-r0, v1.27.3-r0, v1.28.1-r0, or v1.25.6-r0
  • Disabled by default if the cluster version is v1.21.12-r0, v1.23.11-r0, v1.25.6-r0, v1.27.3-r0, v1.28.1-r0, or later

This parameter is available only in clusters of v1.21.10-r0, v1.23.8-r0, v1.25.3-r0, or later versions.

Console/API

Image Repository Pull Limit per Second

registry-pull-qps

QPS upper limit of an image repository.

Default: 5

Value range: 1 to 50

This parameter is available only in clusters of v1.21.10-r0, v1.23.8-r0, v1.25.3-r0, or later versions.

Console/API

Maximum Bursty Image Pulls

registry-burst

Maximum number of burst image pulls.

Default: 10

The value ranges from 1 to 100 and must be greater than or equal to the value of registry-pull-qps.

This parameter is available only in clusters of v1.21.10-r0, v1.23.8-r0, v1.25.3-r0, or later versions.

Console/API

Maximum Number of Container Log Files

container-log-max-files

Maximum number of container log files. When the number of existing log files exceeds this value, the earliest log file will be deleted to release space for new log files.

Default: 10

Value range: 2 to 100

This parameter is available only in clusters of v1.23.14-r0, v1.25.9-r0, v1.27.6-r0, v1.28.4-r0, or later versions.

Console/API

Maximum Container Log File Size

container-log-max-size

Maximum size of a single container log file. When the size of a log file reaches this value, the current log file will be closed and a new log file will be created to continue logging.

Default: 50 (in MiB)

Value range: 1 to 4096 (in MiB)

This parameter is available only in clusters of v1.23.14-r0, v1.25.9-r0, v1.27.6-r0, v1.28.4-r0, or later versions.

Console/API

Upper Limit for Image Garbage Collection

image-gc-high-threshold

When the kubelet disk usage reaches this value, kubelet starts to collect image garbage.

Default: 80

Value range: 1 to 100

To disable image garbage collection, set this parameter to 100.

This parameter is available only in clusters of v1.23.14-r0, v1.25.9-r0, v1.27.6-r0, v1.28.4-r0, or later versions.

Console/API

Lower Limit for Image Garbage Collection

image-gc-low-threshold

Image garbage collection stops when the disk usage drops below the specified threshold.

Default: 70

Value range: 1 to 100

The value of this parameter cannot be greater than the upper limit for image garbage collection.

This parameter is available only in clusters of v1.23.14-r0, v1.25.9-r0, v1.27.6-r0, v1.28.4-r0, or later versions.

Console/API

Node Memory Reservation

system-reserved-mem

Reserved system memory for OS system daemons like sshd and udev.

Default: automatically calculated, which varies depending on node flavors. For details, see Node Resource Reservation Rules.

The kube-reserved-mem and system-reserved-mem combined cannot exceed 50% of the minimum node's memory flavor in the node pool.

Console/API

kube-reserved-mem

Reserved Kubernetes memory for Kubernetes daemons like kubelet and container runtime.

Console/API

Hard Eviction

memory.available

Available memory on a node.

The value is fixed at 100 MiB.

For details, see Node-pressure Eviction.

NOTICE:

Exercise caution when modifying an eviction configuration item. Improper configuration may cause pods to be frequently evicted or fail to be evicted when the node is overloaded.

kubelet can identify the following specific file system identifiers:

  • nodefs: main file system of a node. It is used for emptyDir volumes that are not supported by memory, local disk volumes, and log storage. For example, nodefs contains /var/lib/kubelet/.
  • imagefs: file system partition used by a container engine.

Console/API

nodefs.available

Percentage of the available capacity in the filesystem used by kubelet.

Default: 10%

Value range: 1% to 99%

Console/API

nodefs.inodesFree

Percentage of available inodes in the filesystem used by kubelet.

Default: 5%

Value range: 1% to 99%

Console/API

imagefs.available

Percentage of the available capacity in the filesystem used by container runtimes to store resources such as images.

Default: 10%

Value range: 1% to 99%

Console/API

imagefs.inodesFree

Percentage of available inodes in the filesystem used by container runtimes to store resources such as images.

This parameter is left blank by default.

Value range: 1% to 99%

Console/API

pid.available

Percentage of allocatable PIDs reserved for pods.

Default: 10%

Value range: 1% to 99%

Console/API

Soft Eviction

memory.available

Available memory on a node.

The value must be greater than the hard eviction value of the same parameter, and the eviction grace period (evictionSoftGracePeriod) must be configured accordingly.

This parameter is left blank by default.

Value range: 100 to 1000000 (in MiB)

Console/API

nodefs.available

Percentage of the available capacity in the filesystem used by kubelet.

The value must be greater than the hard eviction value of the same parameter, and the eviction grace period (evictionSoftGracePeriod) must be configured accordingly.

This parameter is left blank by default.

Value range: 1% to 99%

Console/API

nodefs.inodesFree

Percentage of available inodes in the filesystem used by kubelet.

The value must be greater than the hard eviction value of the same parameter, and the eviction grace period (evictionSoftGracePeriod) must be configured accordingly.

This parameter is left blank by default.

Value range: 1% to 99%

Console/API

imagefs.available

Percentage of the available capacity in the filesystem used by container runtimes to store resources such as images.

The value must be greater than the hard eviction value of the same parameter, and the eviction grace period (evictionSoftGracePeriod) must be configured accordingly.

This parameter is left blank by default.

Value range: 1% to 99%

Console/API

imagefs.inodesFree

Percentage of available inodes in the filesystem used by container runtimes to store resources such as images.

The value must be greater than the hard eviction value of the same parameter, and the eviction grace period (evictionSoftGracePeriod) must be configured accordingly.

This parameter is left blank by default.

Value range: 1% to 99%

Console/API

pid.available

Percentage of allocatable PIDs reserved for pods.

The value must be greater than the hard eviction value of the same parameter, and the eviction grace period (evictionSoftGracePeriod) must be configured accordingly.

This parameter is left blank by default.

Value range: 1% to 99%

Console/API

kube-proxy

Item

Parameter

Description

Value

Modification

Configuration Method

Minimum Connection Tracking Entries

conntrack-min

The minimum number of connection tracking entries that can be reserved in a Linux conntrack table. It guarantees that the system maintains a fixed number of connection tracking entries at all times, even when the actual number of connections is low. This prevents performance overhead associated with dynamically allocating or releasing resources.

To obtain the value, run the following command:

sysctl net.nf_conntrack_max

Default: 131072

None

Console/API

Wait Time for a Closed TCP Connection

conntrack-tcp-timeout-close-wait

Wait time of a closed TCP connection

To obtain the value, run the following command:

sysctl net.netfilter.nf_conntrack_tcp_timeout_close_wait

Default: 1h0m0s

None

Console/API

Docker (Available Only for Docker Node Pools)

Item

Parameter

Description

Value

Modification

Configuration Method

Native Umask

native-umask

The default value normal indicates that the umask value of the started container is 0022.

Default: normal

The parameter value cannot be changed.

N/A

Base Size

docker-base-size

Maximum data space that can be used by each container.

Default: 0

The parameter value cannot be changed.

N/A

Insecure Image Source Address

insecure-registry

Whether an insecure image source address can be used.

false

The parameter value cannot be changed.

N/A

Maximum Container Corefile Size

limitcore

Maximum size of a core file in a container. The unit is byte.

If not specified, the value is infinity.

Default: 5368709120

None

Console/API

Maximum Handles in a Container

default-ulimit-nofile

Maximum number of handles that can be used in a container.

Default: {soft}:{hard}

The value cannot exceed the value of the kernel parameter nr_open and cannot be a negative number.

You can run the following command to obtain the kernel parameter nr_open:

sysctl -a | grep nr_open

Console/API

Image Pull Timeout

image-pull-progress-timeout

If the image fails to be pulled before the timeout, the image pull will be canceled.

Default: 1m0s

This parameter is supported in v1.25.3-r0 and later.

Console/API

containerd (Available Only for containerd Node Pools)

Item

Parameter

Description

Value

Modification

Configuration Method

Maximum Container Corefile Size

limitcore

Maximum size of a core file in a container. The unit is byte.

If not specified, the value is infinity.

Default: 5368709120

None

Console/API

Maximum Handles in a Container

default-ulimit-nofile

Maximum number of handles that can be used in a container.

Default: 1048576

The value cannot exceed the value of the kernel parameter nr_open and cannot be a negative number.

You can run the following command to obtain the kernel parameter nr_open:

sysctl -a | grep nr_open

Console/API

Image Pull Timeout

image-pull-progress-timeout

If the image fails to be pulled before the timeout, the image pull will be canceled.

Default: 1m0s

This parameter is supported in v1.25.3-r0 and later.

Console/API

Verification on Insure Skips

Verification on Insure Skips

Whether to skip repository certificate verification.

Default: false

The parameter value cannot be changed.

N/A

Maximum Number of Concurrent Requests for Downloading an Image at a Time

max-concurrent-downloads

This parameter specifies the maximum number of concurrent requests for downloading an image at a time.

Default: 3

Value range: 1 to 20

If this parameter is set to a large value, the network performance of other services on the node may be affected or the disk I/O and CPU usage may increase.

This parameter is available only in clusters of v1.23.14-r0, v1.25.9-r0, v1.27.6-r0, v1.28.4-r0, or later versions.

Console/API

Maximum Container Log Line Size

max-container-log-line-size

Maximum log line size of a container, in the unit of bytes. The log lines exceeding the limit will be split into multiple lines.

Default: 16384

Value range: 1 to 2097152

A larger value will lead to more containerd memory consumption.

This parameter is available only in clusters of v1.23.14-r0, v1.25.9-r0, v1.27.6-r0, v1.28.4-r0, or later versions.

Console/API

Base Size

container-base-size

Maximum data space that can be used by each container.

Default: 0

The parameter value cannot be changed.

N/A

Substitute Image Repository

registry-mirrors

You can configure one or multiple substitute image repositories to be selected when obtaining images from the container runtime.

If you do not specify this parameter, the docker.io image repository will be used by default, and the SWR image repository will be used as a substitute.

An image repository must be an IP address or domain name. A substitute image repository must be an IP address or domain name starting with http:// or https://.
  • Add local image repositories for faster image pulling.
  • Configure multiple image repositories for higher fault tolerance and availability.

This parameter is available only in clusters of v1.23.17-r0, v1.25.12-r0, v1.27.9-r0, v1.28.7-r0, v1.29.3-r0, or later versions.

NOTICE:

If the image repository or its substitute is configured incorrectly, containers may not be able to pull the necessary image.

Console/API

Certificate Authentication Skipped Image Repository

insecure-registries

When you specify image repositories, you can bypass security certificate-based authentication. This is typically done to connect to an insecure or self-signed image repository.

This parameter is left blank by default.

Enter an IP address or domain name.
  • Use this function only in development or test environments, not in production environments.
  • Enable this option only when using a self-signed certificate or when attempting to access a private image repository that cannot obtain an authorized certificate.

This parameter is available only in clusters of v1.23.17-r0, v1.25.12-r0, v1.27.9-r0, v1.28.7-r0, v1.29.3-r0, or later versions.

Console/API

Networking Components (Available Only for CCE Turbo Clusters)

Item

Parameter

Description

Value

Modification

Configuration Method

Node Pool Container ENI Pre-binding

enable-node-nic-configuration

Whether to enable network interface pre-binding in a node pool.

Default: false

After network component configuration is disabled in a node pool, the dynamic container network interface pre-binding parameter settings of the node pool are the same as those of cluster-level parameter settings.

Console/API

Network interface threshold

Network interface threshold

Low threshold of the number of bound network interfaces: High threshold of the number of bound network interfaces

Default: 0:0
NOTE:

This parameter is being discarded. Use the dynamic pre-binding parameters of the other four network interfaces.

Console/API

Minimum number of network interfaces bound to a node in a node pool

nic-minimum-target

Minimum number of container network interfaces bound to a node.

The parameter value must be a positive integer. The value 10 indicates that at least 10 container network interfaces must be bound to a node. If the number you specified exceeds the container network interface quota of the node, the network interface quota will be used.

Default: 10

Configure these parameters based on the number of pods typically running on most nodes.

Console/API

Maximum number of network interfaces pre-bound to a node in a node pool

nic-maximum-target

After the number of network interfaces bound to a node exceeds the nic-maximum-target value, CCE will not proactively pre-bind network interfaces.

Checking the upper limit of pre-bound container network interfaces is enabled only when the value of this parameter is greater than or equal to the minimum number of container network interfaces (nic-minimum-target) bound to a node.

The parameter value must be a positive integer. The value 0 indicates that checking the upper limit of pre-bound container network interfaces is disabled. If the number you specified exceeds the container network interface quota of the node, the network interface quota will be used.

Default: 0

Configure these parameters based on the maximum number of pods running on most nodes.

Console/API

Number of network interfaces dynamically pre-bound to a node in a node pool

nic-warm-target

Extra network interfaces will be pre-bound after the nic-minimum-target is used up in a pod. The value can only be a number.

When the sum of the nic-warm-target value and the number of network interfaces already bound to the node exceeds the nic-maximum-target value, CCE will pre-bind the number of network interfaces specified by the difference between the nic-maximum-target value and the current number of network interfaces bound to the node.

Default: 2

Set the parameter value to the number of pods that can be scaled out instantaneously within 10 seconds on most nodes.

Console/API

Threshold for reclaiming the network interfaces pre-bound to a node in a node pool

nic-max-above-warm-target

Only when the difference between the number of idle network interfaces on a node and the nic-warm-target value is greater than the threshold, the pre-bound network interfaces will be unbound and reclaimed. The value can only be a number.

  • A large value will speed up pod startup but slow down the unbinding of idle container network interfaces and decrease the IP address usage. Exercise caution when performing this operation.
  • A small value will speed up the unbinding of idle container network interfaces and increase the IP address usage but will slow down pod startup, especially when a large number of pods increase instantaneously.

Default: 2

Set the parameter value to the difference between the number of pods that are frequently scaled on most nodes within minutes and the number of pods that are instantly scaled out on most nodes within 10 seconds.

Console/API

Pod Security Groups (Available Only for CCE Turbo Clusters)

Item

Parameter

Description

Value

Modification

Configuration Method

Default Pod Security Group

security_groups_for_nodepool

You can enter the security group ID. If this parameter is not configured, the default security group of the cluster container network will be used, and a maximum of five security group IDs that are separated by semicolons (;) can be specified at a time.

The priority of the security group is lower than that of the security group configured for SecurityGroups.

None

None

Console/API

Node Pools

Item

Parameter

Description

Value

Modification

Configuration Method

Container-isolated Resource View

enable-container-isolation

Enable this function to give each container a separate resource view. Containers only access and manage their own assigned CPUs, memory, and other resources.

Privileged containers do not support resource isolation.

Default: false

Clusters of v1.27.16-r60, v1.28.15-r50, v1.29.15-r10, v1.30.14-r10, v1.31.10-r10, v1.32.6-r10, v1.33.5-r0, or later support this configuration, and only some HCE OS 2.0 kernels are allowed.

  • x86: 5.10.0-182.0.0.95.r2825_235.hce2.x86_64 or later
  • Arm: 5.10.0-182.0.0.95.r2825_235.hce2.aarch64 or later

If the cluster version or OS kernel version does not meet the requirements, upgrade the cluster or upgrade the OS (Upgrading an OS).

Console/API
Parent topic: Managing Node Pools