doc-exports/docs/cce/umn/cce_10_0213.html

Modifying Cluster Configurations

Cluster configuration parameters are underlying rules that define node behavior, resource allocation, communication rules, and scaling policies in a distributed system. They affect the cluster's performance, stability, scalability, and fault tolerance. You can customize the core components of a CCE cluster by adjusting these parameter settings. The following table lists cluster configuration parameters that you can adjust as needed.

Table 1 Cluster configuration parameters

Category	Description
kube-apiserver configurations	Control resource scheduling, manage requests, ensure security, and toggle feature statuses to maintain efficient cluster operation and proper resource allocation. Modifying kube-apiserver parameters will restart the cluster and terminate existing persistent connections. Exercise caution when performing this operation.
Scheduler configurations	Manage and optimize resource scheduling, request control, and GPU resource allocation in clusters. You can dynamically adjust scheduling policies based on cluster loads and resource requirements to ensure efficient cluster running and maximize resource utilization.
kube-controller-manager configurations	Control the behavior and synchronization frequency of different controllers in a cluster to optimize cluster resource management and task scheduling.
Network component configurations (supported only by CCE Turbo clusters)	Control and optimize the management of network resources in clusters, especially in heavy-load and large-scale clusters, to ensure efficient network running and proper resource allocation.
Network component configurations (supported only by CCE clusters using VPC networks)	Specify the IP address range that does not require SNAT. This avoids unnecessary SNAT and optimizes network performance.
External controller manager configurations (supported only by clusters v1.21 or later)	Restrict the resource usage in the namespace to ensure fair and reasonable resource allocation.

Using the Console

1. Log in to the CCE console. In the navigation pane, choose Clusters.
2. Locate the target cluster, click ... to view more operations on the cluster, and choose Manage. This function allows you to modify parameter settings of Kubernetes native components and proprietary components.

3. In the Manage Component window, modify the Kubernetes parameter values based on service requirements. For details about the parameters, see kube-apiserver configurations.

Using an API

You can modify the CCE cluster configuration parameters using an API.

PUT /api/v3/projects/{project_id}/clusters/{cluster_id}/nodepools/master/configuration

Parameter	Description
project_id	Project ID. For details about how to obtain the project ID, see .
cluster_id	Cluster ID. For details about how to obtain the cluster ID, see .

The request body is as follows:

{
  "kind" : "Configuration",
  "apiVersion" : "v3",
  "metadata" : {
    "name" : "configuration"
  },
  "spec" : {
    "packages" : [ {
      "name" : "kube-apiserver",
      "configurations" : [ {
        "name" : "default-not-ready-toleration-seconds",
        "value" : 300
      }, {
        "name" : "default-unreachable-toleration-seconds",
        "value" : 300
      } ]
    } ]
  }
}

In this example, the configuration parameters of the cluster server (kube-apiserver) are modified as follows:

• Toleration time for nodes in the NotReady state during container migration (default-not-ready-toleration-seconds): 300
• Toleration time for nodes in the unreachable state during container migration (default-unreachable-toleration-seconds): 300

kube-apiserver configurations

Item	Parameter	Description	Value	Configuration Method
Toleration time for nodes in NotReady state	default-not-ready-toleration-seconds	Tolerance time during which containers can continue running before being automatically evicted if their node becomes unavailable. This setting applies to all containers by default. In a CCE cluster, you can configure separate tolerance policies for different pods for refined management. For details, see Configuring Tolerance Policies. Configuration suggestion: Unless otherwise specified, retain the default settings. Potential risks: If the specified tolerance time is too short, pods may be frequently migrated due to transient issues like network jitter. If the specified tolerance time is too long, services may remain interrupted for an extended period after a node failure.	Default: 300s	Console/API
Toleration time for nodes in unreachable state	default-unreachable-toleration-seconds	Tolerance time during which containers can continue running before being automatically evicted if their node cannot be accessed. This setting applies to all containers by default. In a CCE cluster, you can configure separate tolerance policies for different pods for refined management. For details, see Configuring Tolerance Policies. Configuration suggestion: Unless otherwise specified, retain the default settings. Potential risks: If the specified tolerance time is too short, pods may be frequently migrated due to transient issues like network jitter. If the specified tolerance time is too long, services may remain interrupted for an extended period after a node failure.	Default: 300s	Console/API
Maximum number of concurrent modification API calls	max-mutating-requests-inflight	Maximum number of concurrent mutating requests. Any requests exceeding the specified value will be rejected. Value 0 indicates that there is no limit on the maximum number of concurrent mutating requests. Configuration suggestion: Retain the default setting. Potential risks: Increasing the value of this parameter may cause overload.	Manual configuration is no longer supported since cluster v1.21. The value is automatically specified based on the cluster scale. 200 for clusters with 50 or 200 nodes 500 for clusters with 1,000 nodes 1000 for clusters with 2,000 nodes	Console/API
Maximum number of concurrent non-modification API calls	max-requests-inflight	Maximum number of concurrent non-mutating requests. Any requests exceeding the specified value will be rejected. Value 0 indicates that there is no limit on the maximum number of concurrent non-mutating requests. Configuration suggestion: Retain the default setting. Potential risks: Increasing the value of this parameter may cause overload.	Manual configuration is no longer supported since cluster v1.21. The value is automatically specified based on the cluster scale. 400 for clusters with 50 or 200 nodes 1000 for clusters with 1,000 nodes 2000 for clusters with 2,000 nodes	Console/API
NodePort port range	service-node-port-range	Port range for a NodePort Service. After changing the value, go to the security group page and change the TCP/UDP port range of node security groups 30000 to 32767. Otherwise, ports other than the default port cannot be accessed externally. Configuration suggestion: Retain the default setting. Potential risks: If the port number is smaller than 20106, a conflict may occur between the port and the CCE health check port, which may further lead to unavailable cluster. If the port number is greater than 32767, a conflict may occur between the port and the ports in net.ipv4.ip_local_port_range, which may further affect the network performance.	Default: 30000 to 32767 Value range: Min > 20105 Max < 32768	Console/API
Request timeout	request-timeout	Request timeout of the kube-apiserver component. Configuration suggestion: Retain the default setting to prevent frequent API timeouts and other exceptions. Applicable cluster version: This parameter is available only in clusters v1.19.16-r30, v1.21.10-r10, v1.23.8-r10, v1.25.3-r10, or later.	Default: 1m0s Value range: Min ≥ 1s Max ≤ 1 hour	Console/API
Overload control	support-overload	Cluster overload control. After this function is enabled, CCE will dynamically adjust concurrent requests based on the resource demands received by master nodes to ensure the stability and reliability of the master nodes and the cluster. Configuration suggestion: Enable this function. In scenarios like short-term request bursts, a cluster may still become overloaded even with overload control enabled. In such cases, you are advised to manage and control access to the cluster promptly. Applicable cluster version: This parameter is available only in clusters v1.23 or later.	false: Overload control is disabled. true: Overload control is enabled.	Console/API
Node restriction add-on	enable-admission-plugin-node-restriction	Restrict kubelet to modify only the pods on its own node. This prevents unauthorized operations and enhances isolation in high-security or multi-tenant scenarios. Applicable cluster version: This parameter is available only in clusters v1.23.14-r0, v1.25.9-r0, v1.27.6-r0, v1.28.4-r0, or later.	Default: true	Console/API
Pod node selector add-on	enable-admission-plugin-pod-node-selector	Allow cluster administrators to configure default node selectors through namespace annotations. In this way, pods run only on specific nodes and configurations are simplified. Applicable cluster version: This parameter is available only in clusters v1.23.14-r0, v1.25.9-r0, v1.27.6-r0, v1.28.4-r0, or later.	Default: true	Console/API
Pod toleration limit add-on	enable-admission-plugin-pod-toleration-restriction	Allow cluster administrators to configure default pod toleration values and limits through namespaces. This enables fine-grained control over pod scheduling and protects key resources. Applicable cluster version: This parameter is available only in clusters v1.23.14-r0, v1.25.9-r0, v1.27.6-r0, v1.28.4-r0, or later.	Default: false	Console/API
API audiences	api-audiences	Specify the audiences associated with a ServiceAccount token in service account token volume projection. For details, see the official document. Configuration suggestion: Retain the default setting. To ensure the proper running of the original service account authentication, add audiences instead of deleting existing ones when configuring this parameter. Potential risks: Deleting the original configuration that is still in use or setting it to an incorrect URL may result in a service account authentication failure. Applicable cluster version: This parameter is available only in clusters v1.23.16-r0, v1.25.11-r0, v1.27.8-r0, v1.28.6-r0, v1.29.2-r0, or later.	Default value: "https://kubernetes.default.svc.cluster.local" Multiple values can be configured, which are separated by commas (,).	Console/API
Service Account Issuer	service-account-issuer	Entity identifier for issuing a service account token, which is the value identified by the iss field in the payload of the service account token. Configuration suggestion: Ensure the configured issuer URL can be accessed in the cluster and trusted by the authentication system in the cluster. Potential risks: If your specified issuer URL is untrusted or inaccessible, the authentication process based on the service account may fail. Applicable cluster version: This parameter is available only in clusters v1.23.16-r0, v1.25.11-r0, v1.27.8-r0, v1.28.6-r0, v1.29.2-r0, or later.	Default value: "https://kubernetes.default.svc.cluster.local" Multiple values can be configured, which are separated by commas (,).	Console/API
OIDC	oidc-enable	CCE supports OpenID Connect (OIDC) authentication. You can configure the necessary identity authentication information to obtain an access token (known as an ID token) and add it to kubectl. For details, see Using Dex for OIDC Authentication on CCE. For more details, see OIDC parameters.	Default: false	Console/API

Scheduler configurations

Item	Parameter	Description	Value	Configuration Method
QPS for communicating with kube-apiserver	kube-api-qps	QPS for communicating with kube-apiserver.	If the number of nodes in a cluster is less than 1,000, the default value is 100. If the number of nodes in a cluster is 1,000 or more, the default value is 200.	Console/API
Burst for communicating with kube-apiserver	kube-api-burst	Burst QPS for communicating with kube-apiserver.	If the number of nodes in a cluster is less than 1,000, the default value is 100. If the number of nodes in a cluster is 1,000 or more, the default value is 200.	Console/API
GPU Sharing	enable-gpu-share	Determine whether to enable GPU sharing as needed. When disabled, ensure that pods in the cluster cannot use shared GPUs (no cce.io/gpu-decision annotation in pods) and that GPU virtualization is disabled. When enabled, ensure that there is a cce.io/gpu-decision annotation on all pods that use GPU resources in the cluster. Applicable cluster version: This parameter is available only in clusters v1.23.7-r10, v1.25.3-r0, or later.	Default: true	Console/API

kube-controller-manager configurations

Item	Parameter	Description	Value	Configuration Method
Deployment	concurrent-deployment-syncs	Number of deployment objects that are allowed to sync concurrently	Default: 5	Console/API
Endpoint	concurrent-endpoint-syncs	Number of service endpoint syncing operations that will be done concurrently	Default: 5	Console/API
Garbage Collection	concurrent-gc-syncs	Number of garbage collector workers that can be synchronized concurrently	Default: 20	Console/API
Job	concurrent-job-syncs	Number of job objects that can be synchronized concurrently	Default: 5	Console/API
CronJob	concurrent-cron-job-syncs	Number of scheduled jobs that can be synchronized concurrently	Default: 5	Console/API
Namespace	concurrent-namespace-syncs	Number of namespace objects that can be synchronized concurrently	Default: 10	Console/API
ReplicaSet	concurrent-replicaset-syncs	Number of replica sets that can be synchronized concurrently	Default: 5	Console/API
Resource Quota	concurrent-resource-quota-syncs	Number of resource quotas that can be synchronized concurrently	Default: 5	Console/API
Service	concurrent-service-syncs	Number of services that can be synchronized concurrently	Default: 10	Console/API
Service Account Tokens	concurrent-serviceaccount-token-syncs	Number of service account token objects that can be synchronized concurrently	Default: 5	Console/API
TTLAfterFinished	concurrent-ttl-after-finished-syncs	Number of ttl-after-finished workers that can be synchronized concurrently	Default: 5	Console/API
RC	concurrent_rc_syncs (used in clusters v1.19 or earlier) concurrent-rc-syncs (used in clusters v1.21 through v1.25.3-r0)	Number of replication controllers that can be synchronized concurrently This parameter is deprecated in clusters v1.25.3-r0 and later versions.	Default: 5	Console/API
Number of HPA concurrent requests	concurrent-horizontal-pod-autoscaler-syncs	Number of HPA auto scaling requests that can be concurrently processed	Default 1 for clusters earlier than v1.27 and 5 for clusters v1.27 or later Value range: 1 to 50	Console/API
HPA synchronization period	horizontal-pod-autoscaler-sync-period	The period of horizontal pod autoscaler for syncing the number of pods. A smaller value will result in a faster auto scaling response and higher CPU load. Configuration suggestion: Retain the default setting. Potential risks: If the period is too long, HPA will respond slowly. If it exceeds the upper limit (153,722,867 minutes), system components may malfunction. If the period is too short, the cluster management plane will be overloaded.	Default: 15s	Console/API
Horizontal Pod Scaling Tolerance	horizontal-pod-autoscaler-tolerance	The configuration determines how quickly HPA will act to auto scaling policies. If the parameter is set to 0, auto scaling will be triggered immediately when the related metrics are met. Configuration suggestion: Configure this parameter based on service resource usage. If the service resource usage increases sharply over time, configure a tolerance to prevent unexpected auto scaling in short-term high-resource usage scenarios.	Default: 0.1	Console/API
HPA CPU initialization period	horizontal-pod-autoscaler-cpu-initialization-period	During this period, HPA can only utilize the metrics of a pod if the pod is ready and its latest metric collection is finished. You can use this parameter to filter out unstable CPU usage data during the early stage of pod startup. This helps prevent incorrect scaling decisions based on momentary peak values. Configuration suggestion: If HPA makes an incorrect scaling decision due to fluctuating CPU usage during pod startup, increase the value of this parameter. Potential risks: A small parameter value may trigger unnecessary scaling based on peak CPU usage, while a large value may cause delayed scaling. Applicable cluster version: This parameter is available only in clusters v1.23.16-r0, v1.25.11-r0, v1.27.8-r0, v1.28.6-r0, v1.29.2-r0, or later.	Default: 5 minutes	Console/API
HPA initial readiness delay	horizontal-pod-autoscaler-initial-readiness-delay	Following HPA CPU initialization, this period enables HPA to get CPU metrics with a less strict criterion. During this period, HPA will gather data on the CPU usage of the pods for scaling, regardless of any changes in the pods readiness status. This parameter ensures continuous tracking of CPU usage, even when the pod statuses change frequently. Configuration suggestion: To prevent HPA misjudgment caused by pod readiness fluctuations after startup, increase the value of this parameter. Potential risks: If this parameter is set to a small value, an unnecessary scale-out may occur due to CPU data fluctuations when the pod is just ready. If it is set to a large value, the HPA may not respond quickly enough in situations requiring rapid scaling. Applicable cluster version: This parameter is available only in clusters v1.23.16-r0, v1.25.11-r0, v1.27.8-r0, v1.28.6-r0, v1.29.2-r0, or later.	Default: 30s	Console/API
QPS for communicating with kube-apiserver	kube-api-qps	QPS for communicating with kube-apiserver	If the number of nodes in a cluster is less than 1,000, the default value is 100. If the number of nodes in a cluster is 1,000 or more, the default value is 200.	Console/API
Burst for communicating with kube-apiserver	kube-api-burst	Burst QPS for communicating with kube-apiserver.	If the number of nodes in a cluster is less than 1,000, the default value is 100. If the number of nodes in a cluster is 1,000 or more, the default value is 200.	Console/API
Terminated Pods to Be Reclaimed	terminated-pod-gc-threshold	Number of terminated pods that can exist in a cluster. When the number of terminated pods exceeds the expected threshold, the excess terminated pods will be automatically deleted. If this parameter is set to 0, all terminated pods will be retained.	Default: 1000 Value range: 10 to 12500 If the cluster version is v1.21.11-r40, v1.23.8-r0, v1.25.6-r0, v1.27.3-r0, or later, the value range is changed to 0 to 100000.	Console/API
Unhealthy AZ threshold	unhealthy-zone-threshold	If the number of not-ready nodes exceeds the specified threshold in a given AZ, that AZ will be marked as unhealthy. In such unhealthy AZs, the frequency of service migration for faulty nodes will be reduced to prevent further negative impacts caused by large-scale migrations during major fault scenarios. Configuration suggestion: Retain the default setting. Potential risks: If the parameter is set to a large value, pods in unhealthy AZs will be migrated in a large scale, which can lead to risks such as overloading the cluster. Applicable cluster version: This parameter is available only in clusters v1.23.14-r0, v1.25.9-r0, v1.27.6-r0, v1.28.4-r0, or later.	Default: 0.55 Value range: 0 to 1	Console/API
Node eviction rate	node-eviction-rate	The maximum number of pods that can be evicted per second when a node is faulty in a healthy AZ. The default value is 0.1, indicating that pods from at most one node can be evicted every 10 seconds. Configuration suggestion: Ensure that the number of pods migrated in each batch does not exceed 300. If the parameter is set to a large value, the cluster may be overloaded. Additionally, if too many pods are evicted, they cannot be rescheduled, which will slow down fault recovery. Applicable cluster version: This parameter is available only in clusters v1.23.14-r0, v1.25.9-r0, v1.27.6-r0, v1.28.4-r0, or later.	Default: 0.1	Console/API
Secondary node eviction rate	secondary-node-eviction-rate	The maximum number of pods that can be evicted per second when a node is faulty in an unhealthy AZ. The default value is 0.01, indicating that pods from at most one node can be evicted every 100 seconds. Configuration suggestion: Configure this parameter to be one-tenth of node-eviction-rate. Potential risks: For nodes in an unhealthy AZ, there is no need to set this parameter to a large value. Doing so may result in overloaded clusters. Applicable cluster version: This parameter is available only in clusters v1.23.14-r0, v1.25.9-r0, v1.27.6-r0, v1.28.4-r0, or later.	Default: 0.01	Console/API
Large cluster threshold	large-cluster-size-threshold	The criterion for determining whether a cluster is a large-scale cluster. If the number of nodes in a cluster exceeds the value of this parameter, the cluster is considered a large-scale cluster. Configuration suggestion: For the clusters with a large number of nodes, configure a relatively larger value than the default one for higher performance and faster responses of controllers. Retain the default value for small clusters. Before adjusting the value of this parameter in a production environment, check the impact of the change on cluster performance in a test environment. Potential risks: In a large-scale cluster, kube-controller-manager adjusts specific configurations to optimize the performance of the cluster. Setting an excessively small threshold for small clusters will deteriorate the cluster performance. Applicable cluster version: This parameter is available only in clusters v1.23.14-r0, v1.25.9-r0, v1.27.6-r0, v1.28.4-r0, or later.	Default: 50	Console/API

Network component configurations (supported only by CCE Turbo clusters)

Item	Parameter	Description	Value	Configuration Method
The minimum number of network interfaces bound to a node in a cluster	nic-minimum-target	The minimum number of network interfaces bound to a node The parameter value must be a positive integer. The value 10 indicates that at least 10 container network interfaces must be bound to a node. If the number you specified exceeds the container network interface quota of the node, the network interface quota will be used.	Default: 10	Console/API
Maximum number of network interfaces pre-bound to a node in a cluster	nic-maximum-target	After the number of network interfaces bound to a node exceeds the nic-maximum-target value, CCE will not proactively pre-bind network interfaces. Checking the upper limit of pre-bound container network interfaces is enabled only when the value of this parameter is greater than or equal to the minimum number of container network interfaces (nic-minimum-target) bound to a node. The parameter value must be a positive integer. The value 0 indicates that checking the upper limit of pre-bound container network interfaces is disabled. If the number you specified exceeds the container network interface quota of the node, the network interface quota will be used.	Default: 0	Console/API
The number of network interfaces dynamically pre-bound to a node in a cluster	nic-warm-target	The target number of network interfaces to be pre-bound to a node before it starts. When the sum of the nic-warm-target value and the number of network interfaces already bound to the node exceeds the nic-maximum-target value, CCE will pre-bind the number of network interfaces specified by the difference between the nic-maximum-target value and the current number of network interfaces bound to the node.	Default: 2	Console/API
The threshold for reclaiming the network interfaces pre-bound to a node in a cluster	nic-max-above-warm-target	Pre-bound network interfaces on a node will only be unbound and reclaimed if the difference between the number of idle network interfaces and the nic-warm-target value exceeds the threshold. The value can only be a number. A large value will speed up pod startup but slow down the unbinding of idle container network interfaces and decrease the IP address usage. Exercise caution when performing this operation. A small value will speed up the unbinding of idle container network interfaces and increase the IP address usage but will slow down pod startup, especially when a large number of pods increase instantaneously.	Default: 2	Console/API
Pod access to metadata	allow-metadata-network-access	After this function is enabled, pods in a cluster can access the node's metadata. If a pod is created after this function is enabled, its ability to access metadata depends on the function's status. If a pod is created after this function is disabled, or in a cluster of an earlier version without this function, it cannot access metadata. To enable a pod to access metadata, it must be rebuilt while the function is enabled.	Default: false	Console/API

Network component configurations (supported only by CCE clusters using VPC networks)

Item	Parameter	Description	Value	Configuration Method
Retaining the non-masqueraded CIDR block of the original pod IP address	nonMasqueradeCIDRs	In a CCE cluster using the VPC network model, if a container in the cluster needs to access externally, the source pod IP address must be masqueraded as the IP address of the node where the pod resides through SNAT. After the configuration, the node will not perform SNAT on IP addresses within the specified CIDR block by default. By default, nodes in a cluster do not perform SNAT on packets destined for the private CIDR blocks 10.0.0.0/8, 172.16.0.0/12, or 192.168.0.0/16. Instead, these packets are directly transferred using the upper-layer VPC. These three CIDR blocks are considered internal networks within the cluster and are reachable at Layer 3 by default. Applicable cluster version: This parameter is available only in clusters v1.23.14-r0, v1.25.9-r0, v1.27.6-r0, v1.28.4-r0, or later.	Default: 10.0.0.0/8, 172.16.0.0/12, or 192.168.0.0/16 NOTE: To enable cross-node pod access, the CIDR block of the node where the target pod runs must be added. Similarly, to enable cross-ECS pod access in a VPC, the CIDR block of the ECS where the target pod runs must be added.	Console/API

External controller manager configurations (supported only by clusters v1.21 or later)

Item	Parameter	Description	Value	Configuration Method
Resource Quota Management (enable-resource-quota)	enable-resource-quota	Determine whether to automatically create a ResourceQuota when creating a namespace. With quota management, you can control the number of workloads of each type and the upper limits of resources in a namespace or related dimensions. false: Auto creation is disabled. true: Auto creation is enabled. For details about the resource quota defaults, see Configuring Resource Quotas. Configuration suggestion: In high-concurrency scenarios (for example, batch creation of pods), resource quota management may cause some requests to fail due to conflicts. This function should not be enabled unless necessary. If you enable it, ensure that the request client has a retry mechanism.	Default: false	Console/API

References

• kube-apiserver
• kube-controller-manager
• kube-scheduler

Parent topic: Managing Clusters