Run the following command for Elasticsearch earlier than 7.x:
PUT /my_store
+With CSS, you can use open-source Elasticsearch APIs on Kibana or an ECS server to import data to an Elasticsearch cluster. JSON files are supported.
+
+
Using an Open-Source Elasticsearch API to Import Data on Kibana
On Kibana, you can run POST commands to import single pieces of data using an open-source Elasticsearch API.
- Log in to Kibana.
- Log in to the CSS management console.
- In the navigation pane on the left, choose Clusters > Elasticsearch.
- In the cluster list, find the target cluster, and click Kibana in the Operation column to log in to the Kibana console.
- In the left navigation pane, choose Dev Tools.
The left part of the console is the command input box, and the triangle icon in its upper-right corner is the execution button. The right part shows the execution result.
+
+ - Run the following command on the console to check whether the cluster has indexes:
GET _cat/indices?v
+- If indexes are available in the cluster to which you want to import data, you do not need to create an index. Go to 7.
- If no indexes are available in the cluster, go to the next step to create an index.
+ - Run the following command to create an index for storing imported data and create a custom mapping to define the data type.
For example, run the following command to create index my_store:
+Sample code for Elasticsearch 7.x or later:
+PUT /my_store
+{
+ "settings": {
+ "number_of_shards": 1
+ },
+ "mappings": {
+ "properties": {
+ "productName": {
+ "type": "text"
+ },
+ "size": {
+ "type": "keyword"
+ }
+ }
+ }
+}
+Sample code for Elasticsearch earlier than 7.x:
PUT /my_store
{
"settings": {
"number_of_shards": 1
@@ -25,78 +44,43 @@
}
}
Run the following command for Elasticsearch 7.x or later:
-PUT /my_store
-{
- "settings": {
- "number_of_shards": 1
- },
- "mappings": {
- "properties": {
- "productName": {
- "type": "text"
- },
- "size": {
- "type": "keyword"
- }
- }
- }
-}
- - Run commands on Kibana to import data. In the example below, only a single piece of data is imported:
Run the following command for Elasticsearch earlier than 7.x:
POST /my_store/products/_bulk
+
- Run commands on Kibana to import data. In the example below, only a single piece of data is imported:
Sample code for Elasticsearch 7.x or later:
+POST /my_store/_bulk
+{"index":{}}
+{"productName":"Latest art shirts for women in 2017 autumn","size":"L"}
+Sample code for Elasticsearch earlier than 7.x:
POST /my_store/products/_bulk
{"index":{}}
{"productName":"Latest art shirts for women in 2017 autumn","size":"L"}
Run the following command for Elasticsearch 7.x or later:
-POST /my_store/_bulk
-{"index":{}}
-{"productName":"Latest art shirts for women in 2017 autumn","size":"L"}
-The command output is similar to that shown in Figure 1. If the value of the errors field in the result is false, the data is successfully imported.
-Figure 1 Response message
+The command output is similar to that shown in Figure 1. If the value of the errors field in the result is false, the data is successfully imported.
+Figure 1 Response message
-
Using an Open-Source Elasticsearch API to Import Data on an ECS Server
On an ECS server, you can run cURL commands to use an open-source Elasticsearch API to import JSON files.
-
In the example below, a cluster in non-security mode is used to describe how to import data using cURL commands. For the commands for a security cluster, see Accessing an Elasticsearch Cluster Using cURL Commands.
-
- Log in to the CSS management console.
- In the navigation pane on the left, choose Clusters > Elasticsearch to switch to the Clusters page.
- In the cluster list, obtain the IP address of the target cluster from the Private Network Address column. Generally, the IP address format is <host>:<port> or <host>:<port>,<host>:<port>.
If the cluster has only one node, the IP address and port number of this one node are displayed, for example, 10.62.179.32:9200. If the cluster has multiple nodes, the IP addresses and port numbers of all nodes are displayed, for example, 10.62.179.32:9200,10.62.179.33:9200.
- - Log in to the ECS that you are going to use to import data.
- Upload a JSON file to the ECS.
For example, save the following data as a JSON file and upload the file to the ECS:
-For Elasticsearch earlier than 7.x:
{"index": {"_index":"my_store","_type":"products"}}
+Using an Open-Source Elasticsearch API to Import Data on an ECS Server
On an ECS server, you can run cURL commands to use an open-source Elasticsearch API to import JSON files.
+
In the example below, a cluster in non-security mode is used to describe how to import data using cURL commands. For the commands for a security cluster, see Accessing an Elasticsearch Cluster Using cURL Commands.
+
- Obtain the private network address of the cluster.
- Log in to the CSS management console.
- In the navigation pane on the left, choose Clusters > Elasticsearch.
- In the cluster list, obtain the target cluster's private IP address from the Private IP Address column. Generally, the IP address format is <host>:<port> or <host>:<port>,<host>:<port>.
If the cluster has only one node, the IP address and port number of this single node are displayed, for example, 10.62.179.32:9200. If the cluster has multiple nodes and all of them are data nodes, the IP addresses and port numbers of all these nodes are displayed; if some of them are client nodes, only the IP addresses and port numbers of these client nodes are displayed; for example, 10.62.179.32:9200,10.62.179.33:9200.
+
+ - Log in to the ECS that you are going to use to import data.
- Upload a JSON file to the ECS.
For example, save the following data as a JSON file and upload the file to the ECS:
+For Elasticsearch 7.x or later:
{"index": {"_index":"my_store"}}
+{"productName":"Autumn new woman blouses 2019","size":"M"}
+{"index": {"_index":"my_store"}}
+{"productName":"Autumn new woman blouses 2019","size":"L"}
+
For Elasticsearch earlier than 7.x:
{"index": {"_index":"my_store","_type":"products"}}
{"productName":"Autumn new woman blouses 2019","size":"M"}
{"index": {"_index":"my_store","_type":"products"}}
{"productName":"Autumn new woman blouses 2019","size":"L"}
For Elasticsearch 7.x or later:
{"index": {"_index":"my_store"}}
-{"productName":"Autumn new woman blouse 2019","size":"M"}
-{"index": {"_index":"my_store"}}
-{"productName":"Autumn new woman blouses 2019","size":"L"}
-
- Run the following command in the path where the JSON file is stored in the ECS to import the JSON file to an Elasticsearch cluster.
curl -X PUT "http://{Private network address and port number of the node} /_bulk" -H 'Content-Type: application/json' --data-binary @test.json
-If communication encryption is enabled for the cluster where you are importing data, you need to send HTTPS requests by add -k to the cURL commands.
-curl -X PUT -k "https://{Private network address and port number of the node} /_bulk" -H 'Content-Type: application/json' --data-binary @test.json
-Replace {Private network address and port number of the node} with the private network address and port number of a node in the cluster. In the case the failure of a cluster node, if the cluster contains multiple nodes, you can replace {Private network address and port number of the node} with the private network address and port number of any available node in the cluster; if the cluster contains only one node, restore this node and execute the command again. test.json indicates the JSON file to be imported.
-
The value of the -X parameter is a command and that of the -H parameter is a message header. In the preceding command, PUT is the value of the -X parameter and 'Content-Type: application/json' --data-binary @test.json is the value of the -H parameter. Do not add -k between a parameter and its value.
+
- Run the following command in the path where the JSON file is stored in the ECS to import the JSON file to an Elasticsearch cluster.
curl -X PUT "http://{Private network address and port number of the node}/_bulk" -H 'Content-Type: application/json' --data-binary @test.json
+If communication encryption is enabled for the cluster where you are importing data, you need to send HTTPS requests by add -k to the cURL commands.
+curl -X PUT -k "https://{Private network address and port number of the node}/_bulk" -H 'Content-Type: application/json' --data-binary @test.json
+Replace {Private network address and port number of the node} with the private network address and port number of a node in the cluster. The value of the -X parameter is a command and that of the -H parameter is a message header. In the preceding command, PUT is the value of the -X parameter and 'Content-Type: application/json' --data-binary @test.json is the value of the -H parameter. Do not add -k between a parameter and its value. test.json indicates the JSON file to be ingested.
+ In the case of the failure of a cluster node, if the cluster contains multiple nodes, you can replace {Private network address and port number of the node} with the private network address and port number of any available node in the cluster; if the cluster contains only one node, restore this node and execute the command again.
-Example 1: In this example, assume that you need to import data in the test.json file to an Elasticsearch cluster, where communication encryption is disabled and the private network address and port number of one node are 192.168.0.90 and 9200 respectively.
-- Run the following command to create an index named my_store:
Run the following command for Elasticsearch earlier than 7.x:
curl -X PUT http://192.168.0.90:9200/my_store -H 'Content-Type: application/json' -d '
- {
- "settings": {
- "number_of_shards": 1
- },
- "mappings": {
- "products": {
- "properties": {
- "productName": {
- "type": "text"
- },
- "size": {
- "type": "keyword"
- }
- }
- }
- }
- }'
-
Run the following command for Elasticsearch 7.x or later:
-curl -X PUT http://192.168.0.90:9200/my_store -H 'Content-Type: application/json' -d '
+Example 1: In this example, assume that you need to import data in the test.json file to an Elasticsearch cluster, where communication encryption is disabled and the private network address and port number of one node are 192.168.0.90 and 9200 respectively.
+- Run the following command to create an index named my_store:
Sample code for Elasticsearch 7.x or later:
+curl -X PUT http://192.168.0.90:9200/my_store -H 'Content-Type: application/json' -d '
{
"settings": {
"number_of_shards": 1
@@ -112,11 +96,7 @@
}
}
}'
- - Run the following command to import the test.json file:
curl -X PUT "http://192.168.0.90:9200/_bulk" -H 'Content-Type: application/json' --data-binary @test.json
-In this case, if the following information is displayed, the data is successfully imported:
-{"took":204,"errors":false,"items":[{"index":{"_index":"my_store","_type":"_doc","_id":"DJQkBIwBbJvUd2769Wi-","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1,"status":201}},{"index":{"_index":"my_store","_type":"_doc","_id":"DZQkBIwBbJvUd2769Wi_","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":1,"_primary_term":1,"status":201}}]}
-
-Example 2: In this example, assume that you need to import data in the
test.json file to an Elasticsearch cluster, where communication encryption has been enabled The private network address and port of one node are
192.168.0.90 and
9200, respectively.
- Run the following command to create an index named my_store:
curl -X PUT -k https://192.168.0.90:9200/my_store -H 'Content-Type: application/json' -d '
+Sample code for Elasticsearch earlier than 7.x:
curl -X PUT http://192.168.0.90:9200/my_store -H 'Content-Type: application/json' -d '
{
"settings": {
"number_of_shards": 1
@@ -134,9 +114,32 @@
}
}
}'
-
- Run the following command to import the test.json file:
curl -X PUT -k "https://192.168.0.90:9200/_bulk" -H 'Content-Type: application/json' --data-binary @test.json
-In this case, if the following information is displayed, the data is successfully imported:
-{"took":204,"errors":false,"items":[{"index":{"_index":"my_store","_type":"_doc","_id":"DJQkBIwBbJvUd2769Wi-","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1,"status":201}},{"index":{"_index":"my_store","_type":"_doc","_id":"DZQkBIwBbJvUd2769Wi_","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":1,"_primary_term":1,"status":201}}]}
+
- Run the following command to import the test.json file:
curl -X PUT "http://192.168.0.90:9200/_bulk" -H 'Content-Type: application/json' --data-binary @test.json
+In this case, if the following information is displayed, the data is successfully imported:
+{"took":204,"errors":false,"items":[{"index":{"_index":"my_store","_type":"_doc","_id":"DJQkBIwBbJvUd2769Wi-","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1,"status":201}},{"index":{"_index":"my_store","_type":"_doc","_id":"DZQkBIwBbJvUd2769Wi_","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":1,"_primary_term":1,"status":201}}]}
+
+
Example 2: In this example, assume that you need to import data in the
test.json file to an Elasticsearch cluster, where communication encryption has been enabled The private network address and port of one node are
192.168.0.90 and
9200, respectively.
- Run the following command to create an index named my_store:
curl -X PUT -k https://192.168.0.90:9200/my_store -H 'Content-Type: application/json' -d '
+ {
+ "settings": {
+ "number_of_shards": 1
+ },
+ "mappings": {
+ "products": {
+ "properties": {
+ "productName": {
+ "type": "text"
+ },
+ "size": {
+ "type": "keyword"
+ }
+ }
+ }
+ }
+ }'
+ - Run the following command to import the test.json file:
curl -X PUT -k "https://192.168.0.90:9200/_bulk" -H 'Content-Type: application/json' --data-binary @test.json
+In this case, if the following information is displayed, the data is successfully imported:
+{"took":204,"errors":false,"items":[{"index":{"_index":"my_store","_type":"_doc","_id":"DJQkBIwBbJvUd2769Wi-","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1,"status":201}},{"index":{"_index":"my_store","_type":"_doc","_id":"DZQkBIwBbJvUd2769Wi_","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":1,"_primary_term":1,"status":201}}]}
@@ -144,7 +147,7 @@
diff --git a/docs/css/umn/css_01_0423.html b/docs/css/umn/css_01_0025.html
similarity index 52%
rename from docs/css/umn/css_01_0423.html
rename to docs/css/umn/css_01_0025.html
index 05a382977..6b2c3a824 100644
--- a/docs/css/umn/css_01_0423.html
+++ b/docs/css/umn/css_01_0025.html
@@ -1,21 +1,21 @@
-
+
Elasticsearch Cluster Monitoring Metrics
-
+
diff --git a/docs/css/umn/css_01_0426.html b/docs/css/umn/css_01_0026.html
similarity index 59%
rename from docs/css/umn/css_01_0426.html
rename to docs/css/umn/css_01_0026.html
index a40d9a126..3369ee19a 100644
--- a/docs/css/umn/css_01_0426.html
+++ b/docs/css/umn/css_01_0026.html
@@ -1,20 +1,20 @@
-
+
Configuring Elasticsearch Cluster Monitoring
-
+
diff --git a/docs/css/umn/css_01_0027.html b/docs/css/umn/css_01_0027.html
new file mode 100644
index 000000000..83f641f0e
--- /dev/null
+++ b/docs/css/umn/css_01_0027.html
@@ -0,0 +1,128 @@
+
+
+
Procedure for Using OpenSearch
+
+
Table 1 Procedure for using an OpenSearch clusterCategory
+ |
+Operation
+ |
+Details
+ |
+
|---|
+
+Use
+ |
+Planning a cluster
+ |
+Before creating an OpenSearch cluster, develop a plan for it, such as whether to deploy the cluster across multiple AZs to improve availability; the node quantity and specifications; the cluster version and security mode; and index sharding, in order to ensure the desired performance and reliability.
+ |
+OpenSearch Cluster Planning Suggestions
+ |
+
+Creating a cluster
+ |
+Create an OpenSearch cluster based on the plan.
+ |
+Creating an OpenSearch Cluster
+ |
+
+Accessing a cluster
+ |
+There are many ways to access an OpenSearch cluster, such as OpenSearch Dashboards, Cerebro, open-source APIs, Java, Python, and Go clients, as well as multiple network configurations over an internal network or the public network. You can select the most appropriate access method based on the programming language you prefer as well as your network environment.
+
+ |
+OpenSearch Cluster Access Methods
+ |
+
+Importing data
+ |
+There are many ways to import data to an OpenSearch cluster, including Logstash, open-source OpenSearch APIs, Cloud Data Migration (CDM), and Data Replication Service (DRS), with support for different data sources and formats, as well as real-time synchronization for relational databases. You can select the best way for yourself based on your use case and the characteristics of your data.
+
+ |
+Different Ways to Ingest Data into an OpenSearch Cluster
+ |
+
+Searching for data
+ |
+With CSS, you are advised to use DSL for data search in OpenSearch clusters. You may also use SQL.
+ |
+Using DSL to Search for Data in OpenSearch
+Using SQL to Search for Data in OpenSearch
+ |
+
+Enhancing the cluster's search capability
+ |
+On top of the open-source version, CSS's OpenSearch clusters offer a range of enhanced features, including vector search, storage-compute decoupling, and switchover between hot and cold data storage classes. These features help you meet performance and cost optimization requirements for different use cases, while enhancing the service's cluster stability and search capability.
+
+ |
+Search Enhancement Features for OpenSearch Clusters
+ |
+
+O&M
+ |
+Backup and restoration
+ |
+Snapshots can be created to back up the data of an OpenSearch cluster, so that data can be quickly restored in the case of accidental data loss or in case historical data is needed, improving cluster data availability.
+ |
+Creating Snapshots to Back Up the Data of an OpenSearch Cluster
+Restoring a Snapshot of an OpenSearch Cluster
+ |
+
+Scaling a cluster
+ |
+CSS provides flexible scale-out and scale-in options, using which you can add or reduce nodes (either randomly or with specified nodes), add node types, and increase or reduce node specifications. This allows you to dynamically adjust cluster resources to meet changing demand and optimize costs.
+ |
+Scaling Out/Up an OpenSearch Cluster
+Scaling In/Down an OpenSearch Cluster
+ |
+
+Upgrade
+ |
+OpenSearch clusters support both same-version upgrade and cross-version upgrade. Same-version upgrade means to upgrade the kernel patches to fix problems or optimize performance. Cross-version upgrade means to upgrade the cluster version to enhance functionality or incorporate versions.
+ |
+Upgrading the Version of an OpenSearch Cluster
+ |
+
+Managing clusters
+ |
+CSS provides comprehensive cluster management functions. Users can check cluster information, authorize cluster access, change the cluster's security mode, replace nodes, bind clusters with enterprise projects, switches AZs, and configure custom word dictionaries for OpenSearch clusters. They help users efficiently manage OpenSearch clusters and ensure cluster security, high availability, and optimized performance.
+ |
+Managing OpenSearch Clusters
+ |
+
+Managing cluster index policies
+ |
+The Index State Management (ISM) plugin of OpenSearch can be used to create and manage index lifecycle policies. These policies help automate index rollovers and deletions, helping optimize cluster performance and cut storage costs.
+ |
+Creating and Managing OpenSearch Index Policies
+ |
+
+Monitoring and log management
+ |
+CSS provides comprehensive monitoring and log management functions. Users can configure and check monitoring metrics for clusters and nodes, configure alarm rules, and back up and view logs. Intelligent O&M tools help users efficiently monitor, analyze, and maintain OpenSearch clusters and ensure cluster stability and performance.
+ |
+Monitoring Metrics for OpenSearch Clusters in Cloud Eye
+Using Cloud Eye to Monitor OpenSearch Clusters
+Configuring Index Monitoring for an OpenSearch Cluster
+Setting Alarm Alerting via SMN for an OpenSearch Cluster
+Synchronizing OpenSearch Alerts to Prometheus
+Managing OpenSearch Cluster Logs
+ |
+
+Audit logs
+ |
+Cloud Trace Service (CTS) can be used to log mission-critical operations related to OpenSearch clusters. Used for auditing and accountability purposes, these log records are retained for seven days on the management console.
+ |
+Viewing OpenSearch Cluster Audit Logs
+ |
+
+
+
+
+
+
+
diff --git a/docs/css/umn/css_01_0028.html b/docs/css/umn/css_01_0028.html
new file mode 100644
index 000000000..0e2edab2c
--- /dev/null
+++ b/docs/css/umn/css_01_0028.html
@@ -0,0 +1,19 @@
+
+
+
OpenSearch Cluster Planning Suggestions
+
+
+
diff --git a/docs/css/umn/css_01_0029.html b/docs/css/umn/css_01_0029.html
new file mode 100644
index 000000000..2f2654533
--- /dev/null
+++ b/docs/css/umn/css_01_0029.html
@@ -0,0 +1,21 @@
+
+
+
Logging In to an OpenSearch Cluster Through Cerebro
+
Cerebro is an open-source Elasticsearch web admin tool, designed for real-time monitoring and efficient cluster operations. Cerebro works for OpenSearch just as well. In CSS, Cerebro is pre-built for each OpenSearch cluster. You can start Cerebro with one click, without installing anything. Cerebro allows you to quickly check a cluster's health status, node distribution, and index details. For routine O&M tasks, including shard adjustment, index management, and performance monitoring, we recommend using Cerebro. Its intuitive cluster topology views and comprehensive management features streamline O&M workflows, reducing complexity while improving efficiency. Cerebro is suitable for different roles, including developers, O&M teams, and data analysts.
+
Accessing a Cluster Through Cerebro on the CSS Console
The OpenSearch clusters created in CSS provide a built-in Cerebro component, using which you can quickly access these clusters.
+
- Log in to the CSS management console.
- In the navigation pane on the left, choose Clusters > OpenSearch.
- Find the target cluster, and choose More > Cerebro in the Operation column. The Cerebro login page is displayed.
- Non-security cluster: Click the cluster name on the Cerebro login page to go to the Cerebro console.
- Security cluster: Click the cluster name on the Cerebro login page, enter the username and password, and click Authenticate to go to the Cerebro console. The default username is admin and the password is the one specified during cluster creation.
+ - After the login is successful, you can access clusters through Cerebro.
+
+
Accessing a Cluster Using an In-House Built Cerebro
When using in-house built Cerebro to access a cluster, make sure the network between them is connected.
+
+
- Obtain the cluster access address. For details about how to obtain the cluster access address for different network configurations, see Network Configuration.
- Start Cerebro, and enter the cluster access address.
+
+
+
+
+
diff --git a/docs/css/umn/css_01_0030.html b/docs/css/umn/css_01_0030.html
new file mode 100644
index 000000000..35a829428
--- /dev/null
+++ b/docs/css/umn/css_01_0030.html
@@ -0,0 +1,79 @@
+
+
+
Accessing an OpenSearch Cluster Using cURL Commands
+
Open-source OpenSearch provides a series of RESTful APIs. You can run cURL commands to access these APIs using tools such as OpenSearch Dashboards and Postman. This topic describes how to run cURL commands to access an OpenSearch cluster.
+
Prerequisites
- The target CSS OpenSearch cluster is available.
- An ECS has been created, and it is in the same VPC and security group as the CSS cluster.
+
+
+
Accessing a Cluster
- Obtain the private network address of the cluster. It is used to access the cluster.
This topic uses a private IP address as an example to describe how to access a cluster. The cluster access address varies depending on the network configurations used. For details, see Network Configuration.
+- Log in to the CSS management console.
- In the navigation pane on the left, choose Clusters > OpenSearch.
- In the cluster list, obtain the target cluster's private IP address from the Private IP Address column. Generally, the IP address format is <host>:<port> or <host>:<port>,<host>:<port>.
If the cluster has only one node, the IP address and port number of this single node are displayed, for example, 10.62.179.32:9200. If the cluster has multiple nodes and all of them are data nodes, the IP addresses and port numbers of all these nodes are displayed; if some of them are client nodes, only the IP addresses and port numbers of these client nodes are displayed; for example, 10.62.179.32:9200,10.62.179.33:9200.
+
+ - Run one of the following commands on the ECS to access the cluster. The access command varies depending on the security mode of the cluster.
- For a cluster with the security mode disabled:
curl "http://<host>:<port>"
+ - For a security-mode cluster that uses HTTP:
curl -u <user>:<password> "http://<host>:<port>"
+ - For a security-mode cluster that uses HTTPS:
curl -u <user>:<password> -k "https://<host>:<port>"
+
+
+Table 1 VariablesVariable
+ |
+Description
+ |
+
|---|
+
+<host>
+ |
+IP address of each node in the cluster. If the cluster contains multiple nodes, there will be multiple IP addresses. You can use any of them.
+ |
+
+<port>
+ |
+Port number for accessing a cluster node. Generally, the port number is 9200.
+ |
+
+<user>
+ |
+Username for accessing the cluster.
+ |
+
+<password>
+ |
+Password of the user.
+If the password contains special characters, enclose the username and password in single quotation marks, for example, curl -u "user:password!" "http://<host>:<port>".
+ |
+
+
+
+
Example command:
+curl "http://10.62.176.32:9200"
+Example response:
+HTTP/1.1 200 OK
+content-type: application/json; charset=UTF-8
+content-length: 513
+
+{
+ "name" : "css-xxx-ess-esn-3-1",
+ "cluster_name" : "css-xxx",
+ "cluster_uuid" : "xxx_uuid",
+ "version" : {
+ "number" : "7.10.2",
+ "build_type" : "tar",
+ "build_hash" : "unknown",
+ "build_date" : "unknown",
+ "build_snapshot" : true,
+ "lucene_version" : "9.12.1",
+ "minimum_wire_compatibility_version" : "7.10.0",
+ "minimum_index_compatibility_version" : "7.0.0"
+ },
+ "tagline" : "The OpenSearch Project: https://opensearch.org/"
+}
+
+
+
+
+
+
diff --git a/docs/css/umn/css_01_0031.html b/docs/css/umn/css_01_0031.html
new file mode 100644
index 000000000..6ff01da0c
--- /dev/null
+++ b/docs/css/umn/css_01_0031.html
@@ -0,0 +1,18 @@
+
+
+
Accessing an OpenSearch Cluster Using Java
+
+
+
diff --git a/docs/css/umn/css_01_0032.html b/docs/css/umn/css_01_0032.html
new file mode 100644
index 000000000..ed0ebe866
--- /dev/null
+++ b/docs/css/umn/css_01_0032.html
@@ -0,0 +1,1080 @@
+
+
+
Accessing an OpenSearch Cluster Using the High Level REST Client
+
You can query and manage data in a CSS OpenSearch cluster using the High Level REST Client. The High Level REST Client encapsulates OpenSearch APIs. You only need to construct the required structures to access an OpenSearch cluster. This simplifies the process of working with an OpenSearch cluster. For details about how to use the REST Client, see Java high-level REST client.
+
Prerequisites
- The target OpenSearch cluster is available.
- The server that runs the Java code can communicate with the OpenSearch cluster.
- Depending on the network configuration method used, obtain the cluster access address. For details, see Network Configuration.
- Java has been installed on the server and the JDK version is 11 or later. Download JDK 11 from Java Archive Downloads.
+
+
Introducing Dependencies
Introduce the required Java dependencies on the server where you run Java code.
+
CSS allows you to use the Elasticsearch 7.10 Java client to connect to OpenSearch clusters. To ensure better compatibility, however, you are advised to use an OpenSearch Java client that has the same version as the target OpenSearch cluster.
+
If your High Level REST Client version is later than the OpenSearch cluster version and there are incompatibility issues with some requests, you can use RestHighLevelClient.getLowLevelClient() to obtain the Low Level REST Client and customize OpenSearch requests. For details, see Accessing an OpenSearch Cluster Using the Low Level REST Client.
+
Select a reference example based on the Java client you use.
+
Scenario 1 (recommended): Use the OpenSearch Java client to access an OpenSearch cluster and introduce dependencies.
+
- Maven:
Replace 1.3.6 with the actual Java client version.
+<dependency>
+ <groupId>org.opensearch.client</groupId>
+ <artifactId>opensearch-rest-high-level-client</artifactId>
+ <version>1.3.6</version>
+</dependency>
+ - Gradle:
Replace 1.3.6 with the actual Java client version.
+compile group: 'org.opensearch.client', name: 'opensearch-rest-high-level-client', version: '1.3.6'
+
+
Scenario 2: Use the Elasticsearch 7.10.2 Java client to access an OpenSearch cluster and introduce dependencies.
+
- Maven:
<dependency>
+ <groupId>org.elasticsearch.client</groupId>
+ <artifactId>elasticsearch-rest-high-level-client</artifactId>
+ <version>7.10.2</version>
+</dependency>
+<dependency>
+ <groupId>org.elasticsearch</groupId>
+ <artifactId>elasticsearch</artifactId>
+ <version>7.10.2</version>
+</dependency>
+ - Gradle:
compile group: 'org.elasticsearch.client', name: 'elasticsearch-rest-high-level-client', version: '7.10.2'
+
+
+
Accessing a Cluster
The sample code varies depending on the Java client and the security mode settings of the target OpenSearch cluster. Select the right reference document based on your service scenario.
+
+
+
+
Connecting to a Non-Security Mode Cluster Using the OpenSearch High Level REST Client
Use the OpenSearch High Level REST Client to connect to an OpenSearch cluster for which the security mode is disabled, and query whether the test index exists. The sample code is as follows:
+
import org.apache.http.HttpHost;
+import org.opensearch.client.RequestOptions;
+import org.opensearch.client.RestClient;
+import org.opensearch.client.RestClientBuilder;
+import org.opensearch.client.RestHighLevelClient;
+import org.opensearch.client.indices.GetIndexRequest;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.List;
+public class RestHighLevelClientExample {
+ public static void main(String[] args) throws IOException {
+ List<String> host = Arrays.asList("{Cluster access address}");
+ RestClientBuilder builder = RestClient.builder(constructHttpHosts(host, 9200, "http"));
+ final RestHighLevelClient client = new RestHighLevelClient(builder);
+ GetIndexRequest indexRequest = new GetIndexRequest("test");
+ boolean exists = client.indices().exists(indexRequest, RequestOptions.DEFAULT);
+ System.out.println(exists);
+ client.close();
+ }
+ /**
+ * Use the constructHttpHosts function to convert the node IP address list of the host cluster.
+ */
+ public static HttpHost[] constructHttpHosts(List<String> host, int port, String protocol) {
+ return host.stream().map(p -> new HttpHost(p, port, protocol)).toArray(HttpHost[]::new);
+ }
+}
+
This piece of code checks whether the test index exists in the cluster. If true (the index exists) or false (the index does not exist) is returned, it indicates that the cluster is connected.
+
+
Connecting to a Security-Mode Cluster Using the OpenSearch High Level REST Client (Without a Certificate)
Use the OpenSearch High Level REST Client to connect to a security-mode OpenSearch cluster (HTTP or HTTPS) without loading a security certificate, and query whether the test index exists. The sample code is as follows:
+
import org.apache.http.HttpHost;
+import org.apache.http.auth.AuthScope;
+import org.apache.http.auth.UsernamePasswordCredentials;
+import org.apache.http.client.CredentialsProvider;
+import org.apache.http.impl.client.BasicCredentialsProvider;
+import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
+import org.apache.http.nio.conn.ssl.SSLIOSessionStrategy;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.opensearch.action.admin.cluster.health.ClusterHealthRequest;
+import org.opensearch.action.admin.cluster.health.ClusterHealthResponse;
+import org.opensearch.client.RequestOptions;
+import org.opensearch.client.RestClient;
+import org.opensearch.client.RestClientBuilder;
+import org.opensearch.client.RestHighLevelClient;
+import org.opensearch.client.indices.GetIndexRequest;
+import org.opensearch.common.Nullable;
+import java.io.IOException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.security.SecureRandom;
+import java.security.cert.CertificateException;
+import java.security.cert.X509Certificate;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Objects;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+/**
+ *Connect to a security-model cluster using the High Level REST Client (without a certificate).
+ */
+public class RestHighLevelClientExample {
+ /**
+ * Create a class for the client. Define the create function.
+ */
+ public static RestHighLevelClient create(List<String> host, int port, String protocol, int connectTimeout, int connectionRequestTimeout, int socketTimeout, String username, String password) throws IOException{
+ final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
+ credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, password));
+ SSLContext sc = null;
+ try {
+ sc = SSLContext.getInstance("SSL");
+ sc.init(null, trustAllCerts, new SecureRandom());
+ } catch (KeyManagementException | NoSuchAlgorithmException e) {
+ e.printStackTrace();
+ }
+ SSLIOSessionStrategy sessionStrategy = new SSLIOSessionStrategy(sc, new NullHostNameVerifier());
+ SecuredHttpClientConfigCallback httpClientConfigCallback = new SecuredHttpClientConfigCallback(sessionStrategy,
+ credentialsProvider);
+ RestClientBuilder builder = RestClient.builder(constructHttpHosts(host, port, protocol))
+ .setRequestConfigCallback(requestConfig -> requestConfig.setConnectTimeout(connectTimeout)
+ .setConnectionRequestTimeout(connectionRequestTimeout)
+ .setSocketTimeout(socketTimeout))
+ .setHttpClientConfigCallback(httpClientConfigCallback);
+ final RestHighLevelClient client = new RestHighLevelClient(builder);
+ logger.info("opensearch rest client build success {} ", client);
+
+ ClusterHealthRequest request = new ClusterHealthRequest();
+ ClusterHealthResponse response = client.cluster().health(request, RequestOptions.DEFAULT);
+ logger.info("opensearch rest client health response {} ", response);
+ return client;
+ }
+
+ /**
+ * Use the constructHttpHosts function to convert the node IP address list of the host cluster.
+ */
+ public static HttpHost[] constructHttpHosts(List<String> host, int port, String protocol) {
+ return host.stream().map(p -> new HttpHost(p, port, protocol)).toArray(HttpHost[]::new);
+ }
+
+ /**
+ * Configure trustAllCerts to ignore the certificate configuration.
+ */
+ public static TrustManager[] trustAllCerts = new TrustManager[] {
+ new X509TrustManager() {
+ @Override
+ public void checkClientTrusted(X509Certificate[] chain, String authType) throws CertificateException {
+ }
+
+ @Override
+ public void checkServerTrusted(X509Certificate[] chain, String authType) throws CertificateException {
+ }
+
+ @Override
+ public X509Certificate[] getAcceptedIssuers() {
+ return null;
+ }
+ }
+ };
+
+ private static final Logger logger = LogManager.getLogger(RestHighLevelClientExampleHttpWithSecurityNoCert.class);
+
+ static class SecuredHttpClientConfigCallback implements RestClientBuilder.HttpClientConfigCallback {
+ @Nullable
+ private final CredentialsProvider credentialsProvider;
+ /**
+ * The {@link SSLIOSessionStrategy} for all requests to enable SSL / TLS encryption.
+ */
+ private final SSLIOSessionStrategy sslStrategy;
+ /**
+ * Create a new {@link SecuredHttpClientConfigCallback}.
+ *
+ * @param credentialsProvider The credential provider, if a username/password have been supplied
+ * @param sslStrategy The SSL strategy, if SSL / TLS have been supplied
+ * @throws NullPointerException if {@code sslStrategy} is {@code null}
+ */
+ SecuredHttpClientConfigCallback(final SSLIOSessionStrategy sslStrategy,
+ @Nullable final CredentialsProvider credentialsProvider) {
+ this.sslStrategy = Objects.requireNonNull(sslStrategy);
+ this.credentialsProvider = credentialsProvider;
+ }
+ /**
+ * Get the {@link CredentialsProvider} that will be added to the HTTP client.
+ *
+ * @return Can be {@code null}.
+ */
+ @Nullable
+ CredentialsProvider getCredentialsProvider() {
+ return credentialsProvider;
+ }
+ /**
+ * Get the {@link SSLIOSessionStrategy} that will be added to the HTTP client.
+ *
+ * @return Never {@code null}.
+ */
+ SSLIOSessionStrategy getSSLStrategy() {
+ return sslStrategy;
+ }
+ /**
+ * Sets the {@linkplain HttpAsyncClientBuilder#setDefaultCredentialsProvider(CredentialsProvider) credential provider},
+ *
+ * @param httpClientBuilder The client to configure.
+ * @return Always {@code httpClientBuilder}.
+ */
+ @Override
+ public HttpAsyncClientBuilder customizeHttpClient(final HttpAsyncClientBuilder httpClientBuilder) {
+ // enable SSL / TLS
+ httpClientBuilder.setSSLStrategy(sslStrategy);
+ // enable user authentication
+ if (credentialsProvider != null) {
+ httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
+ }
+ return httpClientBuilder;
+ }
+ }
+
+ public static class NullHostNameVerifier implements HostnameVerifier {
+ @Override
+ public boolean verify(String arg0, SSLSession arg1) {
+ return true;
+ }
+ }
+
+ /**
+ * The following is an example of the main function. Call the create function to create a client and check whether the test index exists.
+ */
+ public static void main(String[] args) throws IOException {
+ RestHighLevelClient client = create(Arrays.asList("{host}"), 9200, "https", 30000, 30000, 30000, "username", "password");
+ GetIndexRequest indexRequest = new GetIndexRequest("test");
+ boolean exists = client.indices().exists(indexRequest, RequestOptions.DEFAULT);
+ System.out.println(exists);
+ client.close();
+ }
+}
+
+
Table 2 VariablesParameter
+ |
+Description
+ |
+
|---|
+
+host
+ |
+IP address for accessing the cluster. If there are multiple IP addresses, separate them with commas (,).
+ |
+
+port
+ |
+Access port of the cluster. The default value is 9200.
+ |
+
+protocol
+ |
+Connection protocol, which can be http or https.
+ |
+
+connectTimeout
+ |
+Socket connection timeout (in ms).
+ |
+
+connectionRequestTimeout
+ |
+Socket connection request timeout (in ms).
+ |
+
+socketTimeout
+ |
+Socket request timeout (in ms).
+ |
+
+username
+ |
+Username for accessing the cluster.
+ |
+
+password
+ |
+Password of the user.
+ |
+
+
+
+
+
This piece of code checks whether the test index exists in the cluster. If true (the index exists) or false (the index does not exist) is returned, it indicates that the cluster is connected.
+
+
Connecting to a Security-Mode Cluster Using the OpenSearch High Level REST Client (With a Certificate)
Use the OpenSearch High Level REST Client to connect to a security-mode OpenSearch cluster that uses HTTPS with a security certificate loaded, and query whether the test index exists. The sample code is as follows:
+
+
import org.apache.http.HttpHost;
+import org.apache.http.auth.AuthScope;
+import org.apache.http.auth.UsernamePasswordCredentials;
+import org.apache.http.client.CredentialsProvider;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.impl.client.BasicCredentialsProvider;
+import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
+import org.apache.http.nio.conn.ssl.SSLIOSessionStrategy;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.opensearch.action.admin.cluster.health.ClusterHealthRequest;
+import org.opensearch.action.admin.cluster.health.ClusterHealthResponse;
+import org.opensearch.client.RequestOptions;
+import org.opensearch.client.RestClient;
+import org.opensearch.client.RestClientBuilder;
+import org.opensearch.client.RestHighLevelClient;
+import org.opensearch.client.indices.GetIndexRequest;
+import org.opensearch.common.Nullable;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.security.KeyStore;
+import java.security.SecureRandom;
+import java.security.cert.CertificateException;
+import java.security.cert.X509Certificate;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Objects;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.TrustManagerFactory;
+import javax.net.ssl.X509TrustManager;
+/**
+ * Use Hive Level REST Client to connect to a security-mode cluster (with an HTTPS certificate).
+ */
+public class RestHighLevelClientExample {
+ public static RestHighLevelClient create(List<String> host, int port, String protocol, int connectTimeout, int connectionRequestTimeout, int socketTimeout, String username, String password, String certFilePath,
+ String certPassword) throws IOException {
+
+ final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
+ credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, password));
+ SSLContext sc = null;
+ try {
+ TrustManager[] tm = {new MyX509TrustManager(certFilePath, certPassword)};
+ sc = SSLContext.getInstance("SSL", "SunJSSE");
+ //You can also use SSLContext sslContext = SSLContext.getInstance("TLSv1.2");
+ sc.init(null, tm, new SecureRandom());
+ } catch (Exception e) {
+ e.printStackTrace();
+ }
+
+ SSLIOSessionStrategy sessionStrategy = new SSLIOSessionStrategy(sc, new NoopHostnameVerifier());
+ SecuredHttpClientConfigCallback httpClientConfigCallback = new SecuredHttpClientConfigCallback(sessionStrategy,
+ credentialsProvider);
+
+ RestClientBuilder builder = RestClient.builder(constructHttpHosts(host, port, protocol))
+ .setRequestConfigCallback(requestConfig -> requestConfig.setConnectTimeout(connectTimeout)
+ .setConnectionRequestTimeout(connectionRequestTimeout)
+ .setSocketTimeout(socketTimeout))
+ .setHttpClientConfigCallback(httpClientConfigCallback);
+ final RestHighLevelClient client = new RestHighLevelClient(builder);
+ logger.info("opensearch rest client build success {} ", client);
+
+ ClusterHealthRequest request = new ClusterHealthRequest();
+ ClusterHealthResponse response = client.cluster().health(request, RequestOptions.DEFAULT);
+ logger.info("opensearch rest client health response {} ", response);
+ return client;
+ }
+
+ /**
+ * Use the constructHttpHosts function to convert the node IP address list of the host cluster.
+ */
+ public static HttpHost[] constructHttpHosts(List<String> host, int port, String protocol) {
+ return host.stream().map(p -> new HttpHost(p, port, protocol)).toArray(HttpHost[]::new);
+ }
+
+ /**
+ * SecuredHttpClientConfigCallback class definition
+ */
+ static class SecuredHttpClientConfigCallback implements RestClientBuilder.HttpClientConfigCallback {
+ @Nullable
+ private final CredentialsProvider credentialsProvider;
+
+ private final SSLIOSessionStrategy sslStrategy;
+
+ SecuredHttpClientConfigCallback(final SSLIOSessionStrategy sslStrategy,
+ @Nullable final CredentialsProvider credentialsProvider) {
+ this.sslStrategy = Objects.requireNonNull(sslStrategy);
+ this.credentialsProvider = credentialsProvider;
+ }
+
+ @Nullable
+ CredentialsProvider getCredentialsProvider() {
+ return credentialsProvider;
+ }
+
+ SSLIOSessionStrategy getSSLStrategy() {
+ return sslStrategy;
+ }
+
+ @Override
+ public HttpAsyncClientBuilder customizeHttpClient(final HttpAsyncClientBuilder httpClientBuilder) {
+ httpClientBuilder.setSSLStrategy(sslStrategy);
+ if (credentialsProvider != null) {
+ httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
+ }
+ return httpClientBuilder;
+ }
+ }
+
+ private static final Logger logger = LogManager.getLogger(RestHighLevelClient.class);
+
+ public static class MyX509TrustManager implements X509TrustManager {
+ X509TrustManager sunJSSEX509TrustManager;
+
+ MyX509TrustManager(String certFilePath, String certPassword) throws Exception {
+ File file = new File(certFilePath);
+ if (!file.isFile()) {
+ throw new Exception("Wrong Certification Path");
+ }
+ System.out.println("Loading KeyStore " + file + "...");
+ InputStream in = new FileInputStream(file);
+ KeyStore ks = KeyStore.getInstance("JKS");
+ ks.load(in, certPassword.toCharArray());
+ TrustManagerFactory tmf = TrustManagerFactory.getInstance("SunX509", "SunJSSE");
+ tmf.init(ks);
+ TrustManager[] tms = tmf.getTrustManagers();
+ for (TrustManager tm : tms) {
+ if (tm instanceof X509TrustManager) {
+ sunJSSEX509TrustManager = (X509TrustManager) tm;
+ return;
+ }
+ }
+ throw new Exception("Couldn't initialize");
+ }
+
+ @Override
+ public void checkClientTrusted(X509Certificate[] chain, String authType) throws CertificateException {
+
+ }
+
+ @Override
+ public void checkServerTrusted(X509Certificate[] chain, String authType) throws CertificateException {
+
+ }
+
+ @Override
+ public X509Certificate[] getAcceptedIssuers() {
+ return new X509Certificate[0];
+ }
+ }
+
+ /**
+ * The following is an example of the main function. Call the create function to create a client and check whether the test index exists.
+ */
+ public static void main(String[] args) throws IOException {
+ RestHighLevelClient client = create(Arrays.asList("{host}"), 9200, "https", 30000, 30000, 30000, "username", "password", "certFilePath", "certPassword");
+ GetIndexRequest indexRequest = new GetIndexRequest("test");
+ boolean exists = client.indices().exists(indexRequest, RequestOptions.DEFAULT);
+ System.out.println(exists);
+ client.close();
+ }
+}
+
+
Table 3 Function parametersParameter
+ |
+Description
+ |
+
|---|
+
+host
+ |
+IP address for accessing the cluster. If there are multiple IP addresses, separate them with commas (,).
+ |
+
+port
+ |
+Access port of the cluster. The default value is 9200.
+ |
+
+protocol
+ |
+Connection protocol. Set this parameter to https.
+ |
+
+connectTimeout
+ |
+Socket connection timeout (in ms).
+ |
+
+connectionRequestTimeout
+ |
+Socket connection request timeout (in ms).
+ |
+
+socketTimeout
+ |
+Socket request timeout (in ms).
+ |
+
+username
+ |
+Username for accessing the cluster.
+ |
+
+password
+ |
+Password of the user.
+ |
+
+certFilePath
+ |
+Path for storing the security certificate.
+ |
+
+certPassword
+ |
+Password of the security certificate.
+ |
+
+
+
+
+
This piece of code checks whether the test index exists in the cluster. If true (the index exists) or false (the index does not exist) is returned, it indicates that the cluster is connected.
+
+
Connecting to a Non-Security Mode Cluster Using the Elasticsearch High Level REST Client
Use the Elasticsearch 7.10.2 High Level REST Client to connect to an OpenSearch cluster for which the security mode is disabled, and query whether the test index exists. The sample code is as follows:
+
import org.apache.http.HttpHost;
+import org.elasticsearch.client.RequestOptions;
+import org.elasticsearch.client.RestClient;
+import org.elasticsearch.client.RestClientBuilder;
+import org.elasticsearch.client.RestHighLevelClient;
+import org.elasticsearch.client.indices.GetIndexRequest;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.List;
+public class RestHighLevelClientExample {
+ public static void main(String[] args) throws IOException {
+ List<String> host = Arrays.asList("{Cluster access address}");
+ RestClientBuilder builder = RestClient.builder(constructHttpHosts(host, 9200, "http"));
+ final RestHighLevelClient client = new RestHighLevelClient(builder);
+ GetIndexRequest indexRequest = new GetIndexRequest("test");
+ boolean exists = client.indices().exists(indexRequest, RequestOptions.DEFAULT);
+ System.out.println(exists);
+ client.close();
+ }
+ /**
+ * Use the constructHttpHosts function to convert the node IP address list of the host cluster.
+ */
+ public static HttpHost[] constructHttpHosts(List<String> host, int port, String protocol) {
+ return host.stream().map(p -> new HttpHost(p, port, protocol)).toArray(HttpHost[]::new);
+ }
+}
+
This piece of code checks whether the test index exists in the cluster. If true (the index exists) or false (the index does not exist) is returned, it indicates that the cluster is connected.
+
+
Connecting to a Security-Mode Cluster Using the Elasticsearch High Level REST Client (Without a Certificate)
Use the Elasticsearch 7.10.2 High Level REST Client to connect to a security-mode OpenSearch cluster (HTTP or HTTPS) without loading a security certificate, and query whether the test index exists. The sample code is as follows:
+
import org.apache.http.HttpHost;
+import org.apache.http.auth.AuthScope;
+import org.apache.http.auth.UsernamePasswordCredentials;
+import org.apache.http.client.CredentialsProvider;
+import org.apache.http.impl.client.BasicCredentialsProvider;
+import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
+import org.apache.http.nio.conn.ssl.SSLIOSessionStrategy;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.elasticsearch.action.admin.cluster.health.ClusterHealthRequest;
+import org.elasticsearch.action.admin.cluster.health.ClusterHealthResponse;
+import org.elasticsearch.client.RequestOptions;
+import org.elasticsearch.client.RestClient;
+import org.elasticsearch.client.RestClientBuilder;
+import org.elasticsearch.client.RestHighLevelClient;
+import org.elasticsearch.client.indices.GetIndexRequest;
+import org.elasticsearch.common.Nullable;
+
+import java.io.IOException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.security.SecureRandom;
+import java.security.cert.CertificateException;
+import java.security.cert.X509Certificate;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Objects;
+
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.X509TrustManager;
+/**
+ *Connect to a security-model cluster using the High Level REST Client (without a certificate).
+ */
+public class RestHighLevelClientExample {
+ /**
+ * Create a class for the client. Define the create function.
+ */
+ public static RestHighLevelClient create(List<String> host, int port, String protocol, int connectTimeout, int connectionRequestTimeout, int socketTimeout, String username, String password) throws IOException{
+ final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
+ credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, password));
+ SSLContext sc = null;
+ try {
+ sc = SSLContext.getInstance("SSL");
+ sc.init(null, trustAllCerts, new SecureRandom());
+ } catch (KeyManagementException | NoSuchAlgorithmException e) {
+ e.printStackTrace();
+ }
+ SSLIOSessionStrategy sessionStrategy = new SSLIOSessionStrategy(sc, new NullHostNameVerifier());
+ SecuredHttpClientConfigCallback httpClientConfigCallback = new SecuredHttpClientConfigCallback(sessionStrategy,
+ credentialsProvider);
+ RestClientBuilder builder = RestClient.builder(constructHttpHosts(host, port, protocol))
+ .setRequestConfigCallback(requestConfig -> requestConfig.setConnectTimeout(connectTimeout)
+ .setConnectionRequestTimeout(connectionRequestTimeout)
+ .setSocketTimeout(socketTimeout))
+ .setHttpClientConfigCallback(httpClientConfigCallback);
+ final RestHighLevelClient client = new RestHighLevelClient(builder);
+ logger.info("es rest client build success {} ", client);
+
+ ClusterHealthRequest request = new ClusterHealthRequest();
+ ClusterHealthResponse response = client.cluster().health(request, RequestOptions.DEFAULT);
+ logger.info("es rest client health response {} ", response);
+ return client;
+ }
+
+ /**
+ * Use the constructHttpHosts function to convert the node IP address list of the host cluster.
+ */
+ public static HttpHost[] constructHttpHosts(List<String> host, int port, String protocol) {
+ return host.stream().map(p -> new HttpHost(p, port, protocol)).toArray(HttpHost[]::new);
+ }
+
+ /**
+ * Configure trustAllCerts to ignore the certificate configuration.
+ */
+ public static TrustManager[] trustAllCerts = new TrustManager[] {
+ new X509TrustManager() {
+ @Override
+ public void checkClientTrusted(X509Certificate[] chain, String authType) throws CertificateException {
+ }
+
+ @Override
+ public void checkServerTrusted(X509Certificate[] chain, String authType) throws CertificateException {
+ }
+
+ @Override
+ public X509Certificate[] getAcceptedIssuers() {
+ return null;
+ }
+ }
+ };
+
+ private static final Logger logger = LogManager.getLogger(RestHighLevelClientExampleHttpWithSecurityNoCert.class);
+
+ static class SecuredHttpClientConfigCallback implements RestClientBuilder.HttpClientConfigCallback {
+ @Nullable
+ private final CredentialsProvider credentialsProvider;
+ /**
+ * The {@link SSLIOSessionStrategy} for all requests to enable SSL / TLS encryption.
+ */
+ private final SSLIOSessionStrategy sslStrategy;
+ /**
+ * Create a new {@link SecuredHttpClientConfigCallback}.
+ *
+ * @param credentialsProvider The credential provider, if a username/password have been supplied
+ * @param sslStrategy The SSL strategy, if SSL / TLS have been supplied
+ * @throws NullPointerException if {@code sslStrategy} is {@code null}
+ */
+ SecuredHttpClientConfigCallback(final SSLIOSessionStrategy sslStrategy,
+ @Nullable final CredentialsProvider credentialsProvider) {
+ this.sslStrategy = Objects.requireNonNull(sslStrategy);
+ this.credentialsProvider = credentialsProvider;
+ }
+ /**
+ * Get the {@link CredentialsProvider} that will be added to the HTTP client.
+ *
+ * @return Can be {@code null}.
+ */
+ @Nullable
+ CredentialsProvider getCredentialsProvider() {
+ return credentialsProvider;
+ }
+ /**
+ * Get the {@link SSLIOSessionStrategy} that will be added to the HTTP client.
+ *
+ * @return Never {@code null}.
+ */
+ SSLIOSessionStrategy getSSLStrategy() {
+ return sslStrategy;
+ }
+ /**
+ * Sets the {@linkplain HttpAsyncClientBuilder#setDefaultCredentialsProvider(CredentialsProvider) credential provider},
+ *
+ * @param httpClientBuilder The client to configure.
+ * @return Always {@code httpClientBuilder}.
+ */
+ @Override
+ public HttpAsyncClientBuilder customizeHttpClient(final HttpAsyncClientBuilder httpClientBuilder) {
+ // enable SSL / TLS
+ httpClientBuilder.setSSLStrategy(sslStrategy);
+ // enable user authentication
+ if (credentialsProvider != null) {
+ httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
+ }
+ return httpClientBuilder;
+ }
+ }
+
+ public static class NullHostNameVerifier implements HostnameVerifier {
+ @Override
+ public boolean verify(String arg0, SSLSession arg1) {
+ return true;
+ }
+ }
+
+ /**
+ * The following is an example of the main function. Call the create function to create a client and check whether the test index exists.
+ */
+ public static void main(String[] args) throws IOException {
+ RestHighLevelClient client = create(Arrays.asList("{host}"), 9200, "https", 30000, 30000, 30000, "username", "password");
+ GetIndexRequest indexRequest = new GetIndexRequest("test");
+ boolean exists = client.indices().exists(indexRequest, RequestOptions.DEFAULT);
+ System.out.println(exists);
+ client.close();
+ }
+}
+
+
Table 4 VariablesParameter
+ |
+Description
+ |
+
|---|
+
+host
+ |
+IP address for accessing the cluster. If there are multiple IP addresses, separate them with commas (,).
+ |
+
+port
+ |
+Access port of the cluster. The default value is 9200.
+ |
+
+protocol
+ |
+Connection protocol, which can be http or https.
+ |
+
+connectTimeout
+ |
+Socket connection timeout (in ms).
+ |
+
+connectionRequestTimeout
+ |
+Socket connection request timeout (in ms).
+ |
+
+socketTimeout
+ |
+Socket request timeout (in ms).
+ |
+
+username
+ |
+Username for accessing the cluster.
+ |
+
+password
+ |
+Password of the user.
+ |
+
+
+
+
+
This piece of code checks whether the test index exists in the cluster. If true (the index exists) or false (the index does not exist) is returned, it indicates that the cluster is connected.
+
+
Connecting to a Security-Mode Cluster Using the Elasticsearch High Level REST Client (With a Certificate)
Use the Elasticsearch 7.10.2 High Level REST Client to connect to a security-mode OpenSearch cluster that uses HTTPS with a security certificate loaded, and query whether the
test index exists. The sample code is as follows:
+
import org.apache.http.HttpHost;
+import org.apache.http.auth.AuthScope;
+import org.apache.http.auth.UsernamePasswordCredentials;
+import org.apache.http.client.CredentialsProvider;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.impl.client.BasicCredentialsProvider;
+import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
+import org.apache.http.nio.conn.ssl.SSLIOSessionStrategy;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.elasticsearch.action.admin.cluster.health.ClusterHealthRequest;
+import org.elasticsearch.action.admin.cluster.health.ClusterHealthResponse;
+import org.elasticsearch.client.RequestOptions;
+import org.elasticsearch.client.RestClient;
+import org.elasticsearch.client.RestClientBuilder;
+import org.elasticsearch.client.RestHighLevelClient;
+import org.elasticsearch.client.indices.GetIndexRequest;
+import org.elasticsearch.common.Nullable;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.security.KeyStore;
+import java.security.SecureRandom;
+import java.security.cert.CertificateException;
+import java.security.cert.X509Certificate;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Objects;
+
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.TrustManagerFactory;
+import javax.net.ssl.X509TrustManager;
+/**
+ * Use Hive Level REST Client to connect to a security-mode cluster (with an HTTPS certificate).
+ */
+public class RestHighLevelClientExample {
+ public static RestHighLevelClient create(List<String> host, int port, String protocol, int connectTimeout, int connectionRequestTimeout, int socketTimeout, String username, String password, String certFilePath,
+ String certPassword) throws IOException {
+
+ final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
+ credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, password));
+ SSLContext sc = null;
+ try {
+ TrustManager[] tm = {new MyX509TrustManager(certFilePath, certPassword)};
+ sc = SSLContext.getInstance("SSL", "SunJSSE");
+ //You can also use SSLContext sslContext = SSLContext.getInstance("TLSv1.2");
+ sc.init(null, tm, new SecureRandom());
+ } catch (Exception e) {
+ e.printStackTrace();
+ }
+
+ SSLIOSessionStrategy sessionStrategy = new SSLIOSessionStrategy(sc, new NoopHostnameVerifier());
+ SecuredHttpClientConfigCallback httpClientConfigCallback = new SecuredHttpClientConfigCallback(sessionStrategy,
+ credentialsProvider);
+
+ RestClientBuilder builder = RestClient.builder(constructHttpHosts(host, port, protocol))
+ .setRequestConfigCallback(requestConfig -> requestConfig.setConnectTimeout(connectTimeout)
+ .setConnectionRequestTimeout(connectionRequestTimeout)
+ .setSocketTimeout(socketTimeout))
+ .setHttpClientConfigCallback(httpClientConfigCallback);
+ final RestHighLevelClient client = new RestHighLevelClient(builder);
+ logger.info("es rest client build success {} ", client);
+
+ ClusterHealthRequest request = new ClusterHealthRequest();
+ ClusterHealthResponse response = client.cluster().health(request, RequestOptions.DEFAULT);
+ logger.info("es rest client health response {} ", response);
+ return client;
+ }
+
+ /**
+ * Use the constructHttpHosts function to convert the node IP address list of the host cluster.
+ */
+ public static HttpHost[] constructHttpHosts(List<String> host, int port, String protocol) {
+ return host.stream().map(p -> new HttpHost(p, port, protocol)).toArray(HttpHost[]::new);
+ }
+
+ /**
+ * SecuredHttpClientConfigCallback class definition
+ */
+ static class SecuredHttpClientConfigCallback implements RestClientBuilder.HttpClientConfigCallback {
+ @Nullable
+ private final CredentialsProvider credentialsProvider;
+
+ private final SSLIOSessionStrategy sslStrategy;
+
+ SecuredHttpClientConfigCallback(final SSLIOSessionStrategy sslStrategy,
+ @Nullable final CredentialsProvider credentialsProvider) {
+ this.sslStrategy = Objects.requireNonNull(sslStrategy);
+ this.credentialsProvider = credentialsProvider;
+ }
+
+ @Nullable
+ CredentialsProvider getCredentialsProvider() {
+ return credentialsProvider;
+ }
+
+ SSLIOSessionStrategy getSSLStrategy() {
+ return sslStrategy;
+ }
+
+ @Override
+ public HttpAsyncClientBuilder customizeHttpClient(final HttpAsyncClientBuilder httpClientBuilder) {
+ httpClientBuilder.setSSLStrategy(sslStrategy);
+ if (credentialsProvider != null) {
+ httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
+ }
+ return httpClientBuilder;
+ }
+ }
+
+ private static final Logger logger = LogManager.getLogger(RestHighLevelClient.class);
+
+ public static class MyX509TrustManager implements X509TrustManager {
+ X509TrustManager sunJSSEX509TrustManager;
+
+ MyX509TrustManager(String certFilePath, String certPassword) throws Exception {
+ File file = new File(certFilePath);
+ if (!file.isFile()) {
+ throw new Exception("Wrong Certification Path");
+ }
+ System.out.println("Loading KeyStore " + file + "...");
+ InputStream in = new FileInputStream(file);
+ KeyStore ks = KeyStore.getInstance("JKS");
+ ks.load(in, certPassword.toCharArray());
+ TrustManagerFactory tmf = TrustManagerFactory.getInstance("SunX509", "SunJSSE");
+ tmf.init(ks);
+ TrustManager[] tms = tmf.getTrustManagers();
+ for (TrustManager tm : tms) {
+ if (tm instanceof X509TrustManager) {
+ sunJSSEX509TrustManager = (X509TrustManager) tm;
+ return;
+ }
+ }
+ throw new Exception("Couldn't initialize");
+ }
+
+ @Override
+ public void checkClientTrusted(X509Certificate[] chain, String authType) throws CertificateException {
+
+ }
+
+ @Override
+ public void checkServerTrusted(X509Certificate[] chain, String authType) throws CertificateException {
+
+ }
+
+ @Override
+ public X509Certificate[] getAcceptedIssuers() {
+ return new X509Certificate[0];
+ }
+ }
+
+ /**
+ * The following is an example of the main function. Call the create function to create a client and check whether the test index exists.
+ */
+ public static void main(String[] args) throws IOException {
+ RestHighLevelClient client = create(Arrays.asList("{host}"), 9200, "https", 30000, 30000, 30000, "username", "password", "certFilePath", "certPassword");
+ GetIndexRequest indexRequest = new GetIndexRequest("test");
+ boolean exists = client.indices().exists(indexRequest, RequestOptions.DEFAULT);
+ System.out.println(exists);
+ client.close();
+ }
+}
+
+
Table 5 Function parametersParameter
+ |
+Description
+ |
+
|---|
+
+host
+ |
+IP address for accessing the cluster. If there are multiple IP addresses, separate them with commas (,).
+ |
+
+port
+ |
+Access port of the cluster. The default value is 9200.
+ |
+
+protocol
+ |
+Connection protocol. Set this parameter to https.
+ |
+
+connectTimeout
+ |
+Socket connection timeout (in ms).
+ |
+
+connectionRequestTimeout
+ |
+Socket connection request timeout (in ms).
+ |
+
+socketTimeout
+ |
+Socket request timeout (in ms).
+ |
+
+username
+ |
+Username for accessing the cluster.
+ |
+
+password
+ |
+Password of the user.
+ |
+
+certFilePath
+ |
+Path for storing the security certificate.
+ |
+
+certPassword
+ |
+Password of the security certificate.
+ |
+
+
+
+
+
+
This piece of code checks whether the test index exists in the cluster. If true (the index exists) or false (the index does not exist) is returned, it indicates that the cluster is connected.
+
+
Obtaining and Uploading a Security Certificate
To access a security-mode OpenSearch cluster that uses HTTPS, a security certificate must be loaded. Perform the following steps to obtain the security certificate and upload it to the client:
+
- Obtain the security certificate CloudSearchService.cer.
- Log in to the CSS management console.
- In the navigation pane on the left, choose Clusters > OpenSearch.
- In the cluster list, click the name of the target cluster. The cluster information page is displayed.
- Click the Overview tab. In the Configuration area, click Download Certificate next to HTTPS Access.
+ - Convert the security certificate CloudSearchService.cer. Upload the downloaded security certificate to the client and use keytool to convert the .cer certificate into a .jks certificate that can be read by Java.
+
In the preceding command, newname indicates the user-defined certificate name.
+After this command is executed, you will be prompted to set the certificate password and confirm the password. Securely store the password. It will be used for accessing the cluster.
+
+
+
+
+
diff --git a/docs/css/umn/css_01_0033.html b/docs/css/umn/css_01_0033.html
new file mode 100644
index 000000000..3e51fe554
--- /dev/null
+++ b/docs/css/umn/css_01_0033.html
@@ -0,0 +1,19 @@
+
+
+
Backing up and Restoring the Data of an Elasticsearch Cluster
+
+
+
diff --git a/docs/css/umn/css_01_0037.html b/docs/css/umn/css_01_0037.html
new file mode 100644
index 000000000..562b17e26
--- /dev/null
+++ b/docs/css/umn/css_01_0037.html
@@ -0,0 +1,3584 @@
+
+
+
Accessing an OpenSearch Cluster Using the Low Level REST Client
+
You can query and manage data in a CSS OpenSearch cluster using the Low Level REST Client. The Low Level REST Client encapsulates OpenSearch APIs. You only need to construct the required structures to access an OpenSearch cluster. This simplifies the process of working with an OpenSearch cluster. The Low Level REST Client allows you to customize the request structure, which is more flexible and supports all the request formats of OpenSearch, such as GET, POST, DELETE, and HEAD.
+
You can access an OpenSearch cluster with the Low Level REST Client using either of the following ways: directly creating the Low Level REST Client; or creating the High Level REST Client first and then calling RestHighLevelClient.getLowLevelClient() to obtain the Low Level REST Client.
+
Prerequisites
- The target OpenSearch cluster is available.
- The server that runs the Java code can communicate with the OpenSearch cluster.
- Depending on the network configuration method used, obtain the cluster access address. For details, see Network Configuration.
- Java has been installed on the server and the JDK version is 11 or later. Download JDK 11 from Java Archive Downloads.
+
+
Introducing Dependencies
Introduce the required Java dependencies on the server where you run Java code.
+
CSS allows you to use the Elasticsearch 7.10 Java client to connect to OpenSearch clusters. To ensure better compatibility, however, you are advised to use an OpenSearch Java client that has the same version as the target OpenSearch cluster.
+
Select a reference example based on the Java client you use.
+
- Scenario 1 (recommended): Use the OpenSearch Java client to access the OpenSearch cluster and introduce the Apache version using Maven.
Replace 1.3.6 with the actual Java client version.
+<dependency>
+ <groupId>org.opensearch.client</groupId>
+ <artifactId>opensearch-rest-high-level-client</artifactId>
+ <version>1.3.6</version>
+</dependency>
+ - Scenario 2: Use the Elasticsearch 7.10.2 Java client to access the OpenSearch cluster and introduce the Apache version using Maven.
<dependency>
+ <groupId>org.elasticsearch.client</groupId>
+ <artifactId>elasticsearch-rest-client</artifactId>
+ <version>7.10.2</version>
+</dependency>
+<dependency>
+ <groupId>org.elasticsearch</groupId>
+ <artifactId>elasticsearch</artifactId>
+ <version>7.10.2</version>
+</dependency>
+
+
+
Accessing a Cluster
The sample code varies depending on the Java client and the security mode settings of the target OpenSearch cluster. Select the right reference document based on your service scenario.
+
+
+
+
Connecting to a Non-Security Mode Cluster Using the OpenSearch Low Level REST Client
Use the OpenSearch Low Level REST Client to connect to an OpenSearch cluster for which the security mode is disabled, and query whether the test index exists. The sample code is as follows:
+
import org.apache.http.HttpHost;
+import org.opensearch.client.Request;
+import org.opensearch.client.Response;
+import org.opensearch.client.RestClient;
+import org.opensearch.client.RestClientBuilder;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.List;
+public class RestLowLevelClientExample {
+ public static void main(String[] args) throws IOException {
+ List<String> host = Arrays.asList("{Cluster access address}");
+ RestClientBuilder builder = RestClient.builder(constructHttpHosts(host, 9200, "http"));
+ /**
+ *Create the Low Level Rest Client.
+ */
+ RestClient lowLevelClient = builder.build();
+ /**
+ * Check whether the test index exists. If the index exists, 200 is returned. If the index does not exist, 404 is returned.
+ */
+ Request request = new Request("HEAD", "/test");
+ Response response = lowLevelClient.performRequest(request);
+ System.out.println(response.getStatusLine().getStatusCode());
+ lowLevelClient.close();
+ }
+
+ /**
+ * Use the constructHttpHosts function to convert the node IP address list of the host cluster.
+ */
+ public static HttpHost[] constructHttpHosts(List<String> host, int port, String protocol) {
+ return host.stream().map(p -> new HttpHost(p, port, protocol)).toArray(HttpHost[]::new);
+ }
+}
+
This piece of code checks whether the test index exists in the cluster. If 200 (the index exists) or 404 (the index does not exist) is returned, it indicates that the cluster is connected.
+
+
Connecting to a Security-Mode Cluster Using the OpenSearch Low Level REST Client (Without a Certificate)
Use the OpenSearch Low Level REST Client to connect to a security-mode OpenSearch cluster (HTTP or HTTPS) without loading a security certificate, and query whether the test index exists. The sample code is as follows:
+
1
+ 2
+ 3
+ 4
+ 5
+ 6
+ 7
+ 8
+ 9
+ 10
+ 11
+ 12
+ 13
+ 14
+ 15
+ 16
+ 17
+ 18
+ 19
+ 20
+ 21
+ 22
+ 23
+ 24
+ 25
+ 26
+ 27
+ 28
+ 29
+ 30
+ 31
+ 32
+ 33
+ 34
+ 35
+ 36
+ 37
+ 38
+ 39
+ 40
+ 41
+ 42
+ 43
+ 44
+ 45
+ 46
+ 47
+ 48
+ 49
+ 50
+ 51
+ 52
+ 53
+ 54
+ 55
+ 56
+ 57
+ 58
+ 59
+ 60
+ 61
+ 62
+ 63
+ 64
+ 65
+ 66
+ 67
+ 68
+ 69
+ 70
+ 71
+ 72
+ 73
+ 74
+ 75
+ 76
+ 77
+ 78
+ 79
+ 80
+ 81
+ 82
+ 83
+ 84
+ 85
+ 86
+ 87
+ 88
+ 89
+ 90
+ 91
+ 92
+ 93
+ 94
+ 95
+ 96
+ 97
+ 98
+ 99
+100
+101
+102
+103
+104
+105
+106
+107
+108
+109
+110
+111
+112
+113
+114
+115
+116
+117
+118
+119
+120
+121
+122
+123
+124
+125
+126
+127
+128
+129
+130
+131
+132
+133
+134
+135
+136
+137
+138
+139
+140
+141
+142
+143
+144
+145
+146
+147
+148
+149
+150
+151
+152
+153
+154
+155
+156
+157
+158
+159
+160
+161
+162
+163
+164
+165
+166
+167
+168
+169
+170
+171
+172
+173
+174
+175
+176
+177
+178
+179
+180
+181
+182
+183
+184
+185
+186
+187
+188
+189 | import org.apache.http.HttpHost;
+import org.apache.http.HttpResponse;
+import org.apache.http.auth.AuthScope;
+import org.apache.http.auth.UsernamePasswordCredentials;
+import org.apache.http.client.CredentialsProvider;
+import org.apache.http.impl.client.BasicCredentialsProvider;
+import org.apache.http.impl.client.DefaultConnectionKeepAliveStrategy;
+import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
+import org.apache.http.nio.conn.ssl.SSLIOSessionStrategy;
+import org.apache.http.protocol.HttpContext;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.opensearch.client.Request;
+import org.opensearch.client.Response;
+import org.opensearch.client.RestClient;
+import org.opensearch.client.RestClientBuilder;
+import org.opensearch.common.Nullable;
+import java.io.IOException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.security.SecureRandom;
+import java.security.cert.CertificateException;
+import java.security.cert.X509Certificate;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Objects;
+import java.util.concurrent.TimeUnit;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.TrustManager;import javax.net.ssl.X509TrustManager;
+
+public class RestLowLevelClientExample {
+ /**
+ * Create a class for the client. Define the create function.
+ */
+ public static RestClient create(List<String> host, int port, String protocol, int connectTimeout, int connectionRequestTimeout, int socketTimeout, String username, String password) throws IOException {
+ final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
+ credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, password));
+ SSLContext sc = null;
+ try {
+ sc = SSLContext.getInstance("SSL");
+ sc.init(null, trustAllCerts, new SecureRandom());
+ } catch (KeyManagementException | NoSuchAlgorithmException e) {
+ e.printStackTrace();
+ }
+ SSLIOSessionStrategy sessionStrategy = new SSLIOSessionStrategy(sc, new NullHostNameVerifier());
+ SecuredHttpClientConfigCallback httpClientConfigCallback = new SecuredHttpClientConfigCallback(sessionStrategy,
+ credentialsProvider);
+
+ RestClientBuilder builder = RestClient.builder(constructHttpHosts(host, port, protocol))
+ .setRequestConfigCallback(requestConfig -> requestConfig.setConnectTimeout(connectTimeout)
+ .setConnectionRequestTimeout(connectionRequestTimeout)
+ .setSocketTimeout(socketTimeout))
+ .setHttpClientConfigCallback(httpClientConfigCallback);
+ final RestClient client = builder.build();
+ logger.info("opensearch rest client build success {} ", client);
+ return client;
+ }
+
+ /**
+ * Use the constructHttpHosts function to convert the node IP address list of the host cluster.
+ */
+ public static HttpHost[] constructHttpHosts(List<String> host, int port, String protocol) {
+ return host.stream().map(p -> new HttpHost(p, port, protocol)).toArray(HttpHost[]::new);
+ }
+
+ /**
+ * Configure trustAllCerts to ignore the certificate configuration.
+ */
+ public static TrustManager[] trustAllCerts = new TrustManager[] {
+ new X509TrustManager() {
+ @Override
+ public void checkClientTrusted(X509Certificate[] chain, String authType) throws CertificateException {
+ }
+
+ @Override
+ public void checkServerTrusted(X509Certificate[] chain, String authType) throws CertificateException {
+ }
+
+ @Override
+ public X509Certificate[] getAcceptedIssuers() {
+ return null;
+ }
+ }
+ };
+
+ /**
+* The CustomConnectionKeepAliveStrategy function is used to set the connection keepalive during when there are a large number of short connections or when there are not many data requests.
+ */
+ public static class CustomConnectionKeepAliveStrategy extends DefaultConnectionKeepAliveStrategy {
+ public static final CustomConnectionKeepAliveStrategy INSTANCE = new CustomConnectionKeepAliveStrategy();
+
+ private CustomConnectionKeepAliveStrategy() {
+ super();
+ }
+
+ /**
+ * Maximum keepalive time (in minutes)
+ * The default value is 10 minutes. You can set it based on the number of TCP connections in TIME_WAIT state. If there are too many TCP connections, you can increase this value.
+ */
+ private final long MAX_KEEP_ALIVE_MINUTES = 10;
+
+ @Override
+ public long getKeepAliveDuration(HttpResponse response, HttpContext context) {
+ long keepAliveDuration = super.getKeepAliveDuration(response, context);
+ // <0 indicates that the keepalive period is unlimited.
+ // Change the period from unlimited to a default period.
+ if (keepAliveDuration < 0) {
+ return TimeUnit.MINUTES.toMillis(MAX_KEEP_ALIVE_MINUTES);
+ }
+ return keepAliveDuration;
+ }
+ }
+
+ private static final Logger logger = LogManager.getLogger(RestLowLevelClientExample.class);
+
+ static class SecuredHttpClientConfigCallback implements RestClientBuilder.HttpClientConfigCallback {
+ @Nullable
+ private final CredentialsProvider credentialsProvider;
+ /**
+ * The {@link SSLIOSessionStrategy} for all requests to enable SSL / TLS encryption.
+ */
+ private final SSLIOSessionStrategy sslStrategy;
+ /**
+ * Create a new {@link SecuredHttpClientConfigCallback}.
+ *
+ * @param credentialsProvider The credential provider, if a username/password have been supplied
+ * @param sslStrategy The SSL strategy, if SSL / TLS have been supplied
+ * @throws NullPointerException if {@code sslStrategy} is {@code null}
+ */
+ SecuredHttpClientConfigCallback(final SSLIOSessionStrategy sslStrategy,
+ @Nullable final CredentialsProvider credentialsProvider) {
+ this.sslStrategy = Objects.requireNonNull(sslStrategy);
+ this.credentialsProvider = credentialsProvider;
+ }
+ /**
+ * Get the {@link CredentialsProvider} that will be added to the HTTP client.
+ *
+ * @return Can be {@code null}.
+ */
+ @Nullable
+ CredentialsProvider getCredentialsProvider() {
+ return credentialsProvider;
+ }
+ /**
+ * Get the {@link SSLIOSessionStrategy} that will be added to the HTTP client.
+ *
+ * @return Never {@code null}.
+ */
+ SSLIOSessionStrategy getSSLStrategy() {
+ return sslStrategy;
+ }
+ /**
+ * Sets the {@linkplain HttpAsyncClientBuilder#setDefaultCredentialsProvider(CredentialsProvider) credential provider},
+ *
+ * @param httpClientBuilder The client to configure.
+ * @return Always {@code httpClientBuilder}.
+ */
+ @Override
+ public HttpAsyncClientBuilder customizeHttpClient(final HttpAsyncClientBuilder httpClientBuilder) {
+ // enable SSL / TLS
+ httpClientBuilder.setSSLStrategy(sslStrategy);
+ // enable user authentication
+ if (credentialsProvider != null) {
+ httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
+ }
+ return httpClientBuilder;
+ }
+ }
+
+ public static class NullHostNameVerifier implements HostnameVerifier {
+ @Override
+ public boolean verify(String arg0, SSLSession arg1) {
+ return true;
+ }
+ }
+
+ /**
+ * The following is an example of the main function. Call the create function to create a Low Level REST Client and check whether the test index exists.
+ */
+ public static void main(String[] args) throws IOException {
+ RestClient lowLevelClient = create(Arrays.asList("xxx.xxx.xxx.xxx", "xxx.xxx.xxx.xxx"), 9200, "http", 30000, 30000, 30000, "username", "password");
+ Request request = new Request("HEAD", "/test");
+ Response response = lowLevelClient.performRequest(request);
+ System.out.println(response.getStatusLine().getStatusCode());
+ lowLevelClient.close();
+ }
+}
+ |
+
+
+
Table 2 VariablesParameter
+ |
+Description
+ |
+
|---|
+
+host
+ |
+IP address for accessing the cluster. If there are multiple IP addresses, separate them with commas (,).
+ |
+
+port
+ |
+Access port of the cluster. The default value is 9200.
+ |
+
+protocol
+ |
+Connection protocol, which can be http or https.
+ |
+
+connectTimeout
+ |
+Socket connection timeout (in ms).
+ |
+
+connectionRequestTimeout
+ |
+Socket connection request timeout (in ms).
+ |
+
+socketTimeout
+ |
+Socket request timeout (in ms).
+ |
+
+username
+ |
+Username for accessing the cluster.
+ |
+
+password
+ |
+Password of the user.
+ |
+
+
+
+
+
This piece of code checks whether the test index exists in the cluster. If 200 (the index exists) or 404 (the index does not exist) is returned, it indicates that the cluster is connected.
+
+
Connecting to a Security-Mode Cluster Using the OpenSearch Low Level REST Client (With a Certificate)
Use the OpenSearch Low Level REST Client to connect to a security-mode OpenSearch cluster that uses HTTPS with a security certificate loaded, and query whether the test index exists. The sample code is as follows:
+
+
1
+ 2
+ 3
+ 4
+ 5
+ 6
+ 7
+ 8
+ 9
+ 10
+ 11
+ 12
+ 13
+ 14
+ 15
+ 16
+ 17
+ 18
+ 19
+ 20
+ 21
+ 22
+ 23
+ 24
+ 25
+ 26
+ 27
+ 28
+ 29
+ 30
+ 31
+ 32
+ 33
+ 34
+ 35
+ 36
+ 37
+ 38
+ 39
+ 40
+ 41
+ 42
+ 43
+ 44
+ 45
+ 46
+ 47
+ 48
+ 49
+ 50
+ 51
+ 52
+ 53
+ 54
+ 55
+ 56
+ 57
+ 58
+ 59
+ 60
+ 61
+ 62
+ 63
+ 64
+ 65
+ 66
+ 67
+ 68
+ 69
+ 70
+ 71
+ 72
+ 73
+ 74
+ 75
+ 76
+ 77
+ 78
+ 79
+ 80
+ 81
+ 82
+ 83
+ 84
+ 85
+ 86
+ 87
+ 88
+ 89
+ 90
+ 91
+ 92
+ 93
+ 94
+ 95
+ 96
+ 97
+ 98
+ 99
+100
+101
+102
+103
+104
+105
+106
+107
+108
+109
+110
+111
+112
+113
+114
+115
+116
+117
+118
+119
+120
+121
+122
+123
+124
+125
+126
+127
+128
+129
+130
+131
+132
+133
+134
+135
+136
+137
+138
+139
+140
+141
+142
+143
+144
+145
+146
+147
+148
+149
+150
+151
+152
+153 | import org.apache.http.HttpHost;
+import org.apache.http.auth.AuthScope;
+import org.apache.http.auth.UsernamePasswordCredentials;
+import org.apache.http.client.CredentialsProvider;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.impl.client.BasicCredentialsProvider;
+import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
+import org.apache.http.nio.conn.ssl.SSLIOSessionStrategy;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.opensearch.client.Request;
+import org.opensearch.client.Response;
+import org.opensearch.client.RestClient;
+import org.opensearch.client.RestClientBuilder;
+import org.opensearch.common.Nullable;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.security.KeyStore;
+import java.security.SecureRandom;
+import java.security.cert.CertificateException;
+import java.security.cert.X509Certificate;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Objects;
+
+import javax.net.ssl.SSLContext;import javax.net.ssl.TrustManager;
+import javax.net.ssl.TrustManagerFactory;
+import javax.net.ssl.X509TrustManager;
+
+public class RestLowLevelClientExample {
+
+ private static final Logger logger = LogManager.getLogger(RestLowLevelClientExample.class);
+
+ /**
+ * Create a class for the client. Define the create function.
+ */
+ public static RestClient create(List<String> host, int port, String protocol, int connectTimeout, int connectionRequestTimeout, int socketTimeout, String username, String password, String certFilePath, String certPassword) throws IOException {
+ final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
+ credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, password));
+ SSLContext sc = null;
+ try {
+ TrustManager[] tm = {new MyX509TrustManager(certFilePath, certPassword)};
+ sc = SSLContext.getInstance("SSL", "SunJSSE");
+ //You can also use SSLContext sslContext = SSLContext.getInstance("TLSv1.2");
+ sc.init(null, tm, new SecureRandom());
+ } catch (Exception e) {
+ e.printStackTrace();
+ }
+
+ SSLIOSessionStrategy sessionStrategy = new SSLIOSessionStrategy(sc, new NoopHostnameVerifier());
+ SecuredHttpClientConfigCallback httpClientConfigCallback = new SecuredHttpClientConfigCallback(sessionStrategy,
+ credentialsProvider);
+
+ RestClientBuilder builder = RestClient.builder(constructHttpHosts(host, port, protocol))
+ .setRequestConfigCallback(requestConfig -> requestConfig.setConnectTimeout(connectTimeout)
+ .setConnectionRequestTimeout(connectionRequestTimeout)
+ .setSocketTimeout(socketTimeout))
+ .setHttpClientConfigCallback(httpClientConfigCallback);
+ final RestClient client = builder.build();
+ logger.info("opensearch rest client build success {} ", client);
+ return client;
+ }
+
+ /**
+ * Use the constructHttpHosts function to convert the node IP address list of the host cluster.
+ */
+ public static HttpHost[] constructHttpHosts(List<String> host, int port, String protocol) {
+ return host.stream().map(p -> new HttpHost(p, port, protocol)).toArray(HttpHost[]::new);}
+
+ static class SecuredHttpClientConfigCallback implements RestClientBuilder.HttpClientConfigCallback {
+ @Nullable
+ private final CredentialsProvider credentialsProvider;
+
+ private final SSLIOSessionStrategy sslStrategy;
+
+ SecuredHttpClientConfigCallback(final SSLIOSessionStrategy sslStrategy,
+ @Nullable final CredentialsProvider credentialsProvider) {
+ this.sslStrategy = Objects.requireNonNull(sslStrategy);
+ this.credentialsProvider = credentialsProvider;
+ }
+
+ @Nullable
+ CredentialsProvider getCredentialsProvider() {
+ return credentialsProvider;
+ }
+
+ SSLIOSessionStrategy getSSLStrategy() {
+ return sslStrategy;
+ }
+
+ @Override
+ public HttpAsyncClientBuilder customizeHttpClient(final HttpAsyncClientBuilder httpClientBuilder) {
+ httpClientBuilder.setSSLStrategy(sslStrategy);
+ if (credentialsProvider != null) {
+ httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
+ }
+ return httpClientBuilder;
+ }}
+
+ public static class MyX509TrustManager implements X509TrustManager {
+ X509TrustManager sunJSSEX509TrustManager;
+
+ MyX509TrustManager(String certFilePath, String certPassword) throws Exception {
+ File file = new File(certFilePath);
+ if (!file.isFile()) {
+ throw new Exception("Wrong Certification Path");
+ }
+ System.out.println("Loading KeyStore " + file + "...");
+ InputStream in = new FileInputStream(file);
+ KeyStore ks = KeyStore.getInstance("JKS");
+ ks.load(in, certPassword.toCharArray());
+ TrustManagerFactory tmf = TrustManagerFactory.getInstance("SunX509", "SunJSSE");
+ tmf.init(ks);
+ TrustManager[] tms = tmf.getTrustManagers();
+ for (TrustManager tm : tms) {
+ if (tm instanceof X509TrustManager) {
+ sunJSSEX509TrustManager = (X509TrustManager) tm;
+ return;
+ }
+ }
+ throw new Exception("Couldn't initialize");
+ }
+
+ @Override
+ public void checkClientTrusted(X509Certificate[] chain, String authType) throws CertificateException {
+
+ }
+
+ @Override
+ public void checkServerTrusted(X509Certificate[] chain, String authType) throws CertificateException {
+
+ }
+
+ @Override
+ public X509Certificate[] getAcceptedIssuers() {
+ return new X509Certificate[0];
+ }
+ }
+
+ /**
+ * The following is an example of the main function. Call the create function to create a Low Level REST Client and check whether the test index exists.
+ */
+ public static void main(String[] args) throws IOException {
+ RestClient lowLevelClient = create(Arrays.asList("xxx.xxx.xxx.xxx", "xxx.xxx.xxx.xxx"), 9200, "https", 30000, 30000, 30000, "username", "password", "certFilePath", "certPassword");
+ Request request = new Request("HEAD", "test");
+ Response response = lowLevelClient.performRequest(request);
+ System.out.println(response.getStatusLine().getStatusCode());
+ lowLevelClient.close();
+ }
+}
+ |
+
+
+
Table 3 Function parametersParameter
+ |
+Description
+ |
+
|---|
+
+host
+ |
+IP address for accessing the cluster. If there are multiple IP addresses, separate them with commas (,).
+ |
+
+port
+ |
+Access port of the cluster. The default value is 9200.
+ |
+
+protocol
+ |
+Connection protocol. Set this parameter to https.
+ |
+
+connectTimeout
+ |
+Socket connection timeout (in ms).
+ |
+
+connectionRequestTimeout
+ |
+Socket connection request timeout (in ms).
+ |
+
+socketTimeout
+ |
+Socket request timeout (in ms).
+ |
+
+username
+ |
+Username for accessing the cluster.
+ |
+
+password
+ |
+Password of the user.
+ |
+
+certFilePath
+ |
+Path for storing the security certificate.
+ |
+
+certPassword
+ |
+Password of the security certificate.
+ |
+
+
+
+
+
This piece of code checks whether the test index exists in the cluster. If 200 (the index exists) or 404 (the index does not exist) is returned, it indicates that the cluster is connected.
+
+
Connecting to a Non-Security Mode Cluster Using the OpenSearch High Level REST Client
Use the OpenSearch High Level REST Client to obtain the Low Level REST Client by calling getLowLevelClient(), use the low-level client to connect to an OpenSearch cluster for which the security mode is disabled, and query whether the test index exists. The sample code is as follows:
+
import org.apache.http.HttpHost;
+import org.opensearch.client.Request;
+import org.opensearch.client.Response;
+import org.opensearch.client.RestClient;
+import org.opensearch.client.RestClientBuilder;
+import org.opensearch.client.RestHighLevelClient;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.List;
+public class RestLowLevelClientExample {
+
+ public static void main(String[] args) throws IOException {
+ List<String> host = Arrays.asList("{Cluster access address}");
+ RestClientBuilder builder = RestClient.builder(constructHttpHosts(host, 9200, "http"));
+ final RestHighLevelClient restHighLevelClient = new RestHighLevelClient(builder);
+ /**
+ * Create a high-level client and then call getLowLevelClient() to obtain a low-level client. The code differs from the client creation code only in the following line:
+ */
+ final RestClient lowLevelClient = restHighLevelClient.getLowLevelClient();
+ /**
+ * Check whether the test index exists. If the index exists, 200 is returned. If the index does not exist, 404 is returned.
+ */
+ Request request = new Request("HEAD", "/test");
+ Response response = lowLevelClient.performRequest(request);
+ System.out.println(response.getStatusLine().getStatusCode());
+ lowLevelClient.close();
+ }
+
+ /**
+ * Use the constructHttpHosts function to convert the node IP address list of the host cluster.
+ */
+ public static HttpHost[] constructHttpHosts(List<String> host, int port, String protocol) {
+ return host.stream().map(p -> new HttpHost(p, port, protocol)).toArray(HttpHost[]::new);
+ }
+}
+
This piece of code checks whether the test index exists in the cluster. If 200 (the index exists) or 404 (the index does not exist) is returned, it indicates that the cluster is connected.
+
+
Connecting to a Security-Mode Cluster Using the OpenSearch High Level REST Client (Without a Certificate)
Use the OpenSearch High Level REST Client to obtain the Low Level REST Client by calling getLowLevelClient(), use the low-level client to connect to a security-mode OpenSearch cluster that uses HTTP or HTTPS without loading a security certificate, and query whether the test index exists. The sample code is as follows:
+
1
+ 2
+ 3
+ 4
+ 5
+ 6
+ 7
+ 8
+ 9
+ 10
+ 11
+ 12
+ 13
+ 14
+ 15
+ 16
+ 17
+ 18
+ 19
+ 20
+ 21
+ 22
+ 23
+ 24
+ 25
+ 26
+ 27
+ 28
+ 29
+ 30
+ 31
+ 32
+ 33
+ 34
+ 35
+ 36
+ 37
+ 38
+ 39
+ 40
+ 41
+ 42
+ 43
+ 44
+ 45
+ 46
+ 47
+ 48
+ 49
+ 50
+ 51
+ 52
+ 53
+ 54
+ 55
+ 56
+ 57
+ 58
+ 59
+ 60
+ 61
+ 62
+ 63
+ 64
+ 65
+ 66
+ 67
+ 68
+ 69
+ 70
+ 71
+ 72
+ 73
+ 74
+ 75
+ 76
+ 77
+ 78
+ 79
+ 80
+ 81
+ 82
+ 83
+ 84
+ 85
+ 86
+ 87
+ 88
+ 89
+ 90
+ 91
+ 92
+ 93
+ 94
+ 95
+ 96
+ 97
+ 98
+ 99
+100
+101
+102
+103
+104
+105
+106
+107
+108
+109
+110
+111
+112
+113
+114
+115
+116
+117
+118
+119
+120
+121
+122
+123
+124
+125
+126
+127
+128
+129
+130
+131
+132
+133
+134
+135
+136
+137
+138
+139
+140
+141
+142
+143
+144
+145
+146
+147
+148
+149
+150
+151
+152
+153
+154
+155
+156
+157
+158
+159
+160
+161
+162
+163
+164
+165
+166
+167
+168
+169
+170
+171
+172
+173
+174
+175
+176
+177
+178
+179
+180
+181
+182
+183
+184
+185
+186
+187
+188
+189
+190
+191
+192
+193
+194
+195 | import org.apache.http.HttpHost;
+import org.apache.http.HttpResponse;
+import org.apache.http.auth.AuthScope;
+import org.apache.http.auth.UsernamePasswordCredentials;
+import org.apache.http.client.CredentialsProvider;
+import org.apache.http.impl.client.BasicCredentialsProvider;
+import org.apache.http.impl.client.DefaultConnectionKeepAliveStrategy;
+import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
+import org.apache.http.nio.conn.ssl.SSLIOSessionStrategy;
+import org.apache.http.protocol.HttpContext;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.opensearch.client.Request;
+import org.opensearch.client.Response;
+import org.opensearch.client.RestClient;
+import org.opensearch.client.RestClientBuilder;
+import org.opensearch.client.RestHighLevelClient;
+import org.opensearch.common.Nullable;
+
+import java.io.IOException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.security.SecureRandom;
+import java.security.cert.CertificateException;
+import java.security.cert.X509Certificate;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Objects;
+import java.util.concurrent.TimeUnit;
+
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.TrustManager;import javax.net.ssl.X509TrustManager;
+
+public class RestLowLevelClientExample {
+
+ /**
+ * Create a class for the client. Define the create function.
+ */
+ public static RestHighLevelClient create(List<String> host, int port, String protocol, int connectTimeout, int connectionRequestTimeout, int socketTimeout, String username, String password) throws IOException {
+
+ final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
+ credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, password));
+ SSLContext sc = null;
+ try {
+ sc = SSLContext.getInstance("SSL");
+ sc.init(null, trustAllCerts, new SecureRandom());
+ } catch (KeyManagementException | NoSuchAlgorithmException e) {
+ e.printStackTrace();
+ }
+ SSLIOSessionStrategy sessionStrategy = new SSLIOSessionStrategy(sc, new NullHostNameVerifier());
+ SecuredHttpClientConfigCallback httpClientConfigCallback = new SecuredHttpClientConfigCallback(sessionStrategy,
+ credentialsProvider);
+
+ RestClientBuilder builder = RestClient.builder(constructHttpHosts(host, port, protocol))
+ .setRequestConfigCallback(requestConfig -> requestConfig.setConnectTimeout(connectTimeout)
+ .setConnectionRequestTimeout(connectionRequestTimeout)
+ .setSocketTimeout(socketTimeout))
+ .setHttpClientConfigCallback(httpClientConfigCallback);
+ final RestHighLevelClient client = new RestHighLevelClient(builder);
+ logger.info("opensearch rest client build success {} ", client);
+ return client;
+ }
+
+ /**
+ * Use the constructHttpHosts function to convert the node IP address list of the host cluster.
+ */
+ public static HttpHost[] constructHttpHosts(List<String> host, int port, String protocol) {
+ return host.stream().map(p -> new HttpHost(p, port, protocol)).toArray(HttpHost[]::new);
+ }
+
+ /**
+ * Configure trustAllCerts to ignore the certificate configuration.
+ */
+ public static TrustManager[] trustAllCerts = new TrustManager[] {
+ new X509TrustManager() {
+ @Override
+ public void checkClientTrusted(X509Certificate[] chain, String authType) throws CertificateException {
+ }
+
+ @Override
+ public void checkServerTrusted(X509Certificate[] chain, String authType) throws CertificateException {
+ }
+
+ @Override
+ public X509Certificate[] getAcceptedIssuers() {
+ return null;
+ }
+ }
+ };
+
+ /**
+* The CustomConnectionKeepAliveStrategy function is used to set the connection keepalive during when there are a large number of short connections or when there are not many data requests.
+ */
+ public static class CustomConnectionKeepAliveStrategy extends DefaultConnectionKeepAliveStrategy {
+ public static final CustomConnectionKeepAliveStrategy INSTANCE = new CustomConnectionKeepAliveStrategy();
+
+ private CustomConnectionKeepAliveStrategy() {
+ super();
+ }
+
+ /**
+ * Maximum keepalive time (in minutes)
+ * The default value is 10 minutes. You can set it based on the number of TCP connections in TIME_WAIT state. If there are too many TCP connections, you can increase this value.
+ */
+ private final long MAX_KEEP_ALIVE_MINUTES = 10;
+
+ @Override
+ public long getKeepAliveDuration(HttpResponse response, HttpContext context) {
+ long keepAliveDuration = super.getKeepAliveDuration(response, context);
+ // <0 indicates an unlimited keepalive period.
+ // Change the period from unlimited to a default period.
+ if (keepAliveDuration < 0) {
+ return TimeUnit.MINUTES.toMillis(MAX_KEEP_ALIVE_MINUTES);
+ }
+ return keepAliveDuration;
+ }
+ }
+
+ private static final Logger logger = LogManager.getLogger(RestLowLevelClientExample.class);
+
+ static class SecuredHttpClientConfigCallback implements RestClientBuilder.HttpClientConfigCallback {
+ @Nullable
+ private final CredentialsProvider credentialsProvider;
+ /**
+ * The {@link SSLIOSessionStrategy} for all requests to enable SSL / TLS encryption.
+ */
+ private final SSLIOSessionStrategy sslStrategy;
+ /**
+ * Create a new {@link SecuredHttpClientConfigCallback}.
+ *
+ * @param credentialsProvider The credential provider, if a username/password have been supplied
+ * @param sslStrategy The SSL strategy, if SSL / TLS have been supplied
+ * @throws NullPointerException if {@code sslStrategy} is {@code null}
+ */
+ SecuredHttpClientConfigCallback(final SSLIOSessionStrategy sslStrategy,
+ @Nullable final CredentialsProvider credentialsProvider) {
+ this.sslStrategy = Objects.requireNonNull(sslStrategy);
+ this.credentialsProvider = credentialsProvider;
+ }
+ /**
+ * Get the {@link CredentialsProvider} that will be added to the HTTP client.
+ *
+ * @return Can be {@code null}.
+ */
+ @Nullable
+ CredentialsProvider getCredentialsProvider() {
+ return credentialsProvider;
+ }
+ /**
+ * Get the {@link SSLIOSessionStrategy} that will be added to the HTTP client.
+ *
+ * @return Never {@code null}.
+ */
+ SSLIOSessionStrategy getSSLStrategy() {
+ return sslStrategy;
+ }
+ /**
+ * Sets the {@linkplain HttpAsyncClientBuilder#setDefaultCredentialsProvider(CredentialsProvider) credential provider},
+ *
+ * @param httpClientBuilder The client to configure.
+ * @return Always {@code httpClientBuilder}.
+ */
+ @Override
+ public HttpAsyncClientBuilder customizeHttpClient(final HttpAsyncClientBuilder httpClientBuilder) {
+ // enable SSL / TLS
+ httpClientBuilder.setSSLStrategy(sslStrategy);
+ // enable user authentication
+ if (credentialsProvider != null) {
+ httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
+ }
+ return httpClientBuilder;
+ }
+ }
+
+ public static class NullHostNameVerifier implements HostnameVerifier {
+ @Override
+ public boolean verify(String arg0, SSLSession arg1) {
+ return true;
+ }
+ }
+
+ /**
+* The following is an example of the main function. Call the create function to create the high-level client, then call the getLowLevelClient() function to obtain the low-level client, and query whether the test index exists.
+ */
+ public static void main(String[] args) throws IOException {
+ RestHighLevelClient client = create(Arrays.asList("xxx.xxx.xxx.xxx", "xxx.xxx.xxx.xxx"), 9200, "http", 30000, 30000, 30000, "username", "password");
+ RestClient lowLevelClient = client.getLowLevelClient();
+ Request request = new Request("HEAD", "test");
+ Response response = lowLevelClient.performRequest(request);
+ System.out.println(response.getStatusLine().getStatusCode());
+ lowLevelClient.close();
+ }
+}
+ |
+
+
+
Table 4 VariablesParameter
+ |
+Description
+ |
+
|---|
+
+host
+ |
+IP address for accessing the cluster. If there are multiple IP addresses, separate them with commas (,).
+ |
+
+port
+ |
+Access port of the cluster. The default value is 9200.
+ |
+
+protocol
+ |
+Connection protocol, which can be http or https.
+ |
+
+connectTimeout
+ |
+Socket connection timeout (in ms).
+ |
+
+connectionRequestTimeout
+ |
+Socket connection request timeout (in ms).
+ |
+
+socketTimeout
+ |
+Socket request timeout (in ms).
+ |
+
+username
+ |
+Username for accessing the cluster.
+ |
+
+password
+ |
+Password of the user.
+ |
+
+
+
+
+
This piece of code checks whether the test index exists in the cluster. If 200 (the index exists) or 404 (the index does not exist) is returned, it indicates that the cluster is connected.
+
+
Connecting to a Security-Mode Cluster Using the OpenSearch High Level REST Client (With a Certificate)
Use the OpenSearch High Level REST Client to connect to a security-mode OpenSearch cluster that uses HTTPS with a security certificate loaded, and query whether the test index exists. The sample code is as follows:
+
+
1
+ 2
+ 3
+ 4
+ 5
+ 6
+ 7
+ 8
+ 9
+ 10
+ 11
+ 12
+ 13
+ 14
+ 15
+ 16
+ 17
+ 18
+ 19
+ 20
+ 21
+ 22
+ 23
+ 24
+ 25
+ 26
+ 27
+ 28
+ 29
+ 30
+ 31
+ 32
+ 33
+ 34
+ 35
+ 36
+ 37
+ 38
+ 39
+ 40
+ 41
+ 42
+ 43
+ 44
+ 45
+ 46
+ 47
+ 48
+ 49
+ 50
+ 51
+ 52
+ 53
+ 54
+ 55
+ 56
+ 57
+ 58
+ 59
+ 60
+ 61
+ 62
+ 63
+ 64
+ 65
+ 66
+ 67
+ 68
+ 69
+ 70
+ 71
+ 72
+ 73
+ 74
+ 75
+ 76
+ 77
+ 78
+ 79
+ 80
+ 81
+ 82
+ 83
+ 84
+ 85
+ 86
+ 87
+ 88
+ 89
+ 90
+ 91
+ 92
+ 93
+ 94
+ 95
+ 96
+ 97
+ 98
+ 99
+100
+101
+102
+103
+104
+105
+106
+107
+108
+109
+110
+111
+112
+113
+114
+115
+116
+117
+118
+119
+120
+121
+122
+123
+124
+125
+126
+127
+128
+129
+130
+131
+132
+133
+134
+135
+136
+137
+138
+139
+140
+141
+142
+143
+144
+145
+146
+147
+148
+149
+150
+151
+152
+153
+154
+155
+156
+157
+158
+159
+160 | import org.apache.http.HttpHost;
+import org.apache.http.auth.AuthScope;
+import org.apache.http.auth.UsernamePasswordCredentials;
+import org.apache.http.client.CredentialsProvider;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.impl.client.BasicCredentialsProvider;
+import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
+import org.apache.http.nio.conn.ssl.SSLIOSessionStrategy;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.opensearch.action.admin.cluster.health.ClusterHealthRequest;
+import org.opensearch.action.admin.cluster.health.ClusterHealthResponse;
+import org.opensearch.client.Request;
+import org.opensearch.client.RequestOptions;
+import org.opensearch.client.Response;
+import org.opensearch.client.RestClient;
+import org.opensearch.client.RestClientBuilder;
+import org.opensearch.client.RestHighLevelClient;
+import org.opensearch.common.Nullable;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.security.KeyStore;
+import java.security.SecureRandom;
+import java.security.cert.CertificateException;
+import java.security.cert.X509Certificate;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Objects;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.TrustManagerFactory;
+import javax.net.ssl.X509TrustManager;
+
+public class RestLowLevelClientExample {
+ private static final Logger logger = LogManager.getLogger(RestLowLevelClientExample.class);
+
+ /**
+ * Create a class for the client. Define the create function.
+ */
+ public static RestHighLevelClient create(List<String> host, int port, String protocol, int connectTimeout, int connectionRequestTimeout, int socketTimeout, String username, String password, String certFilePath, String certPassword) throws IOException {
+ final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
+ credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, password));
+ SSLContext sc = null;
+ try {
+ TrustManager[] tm = {new MyX509TrustManager(certFilePath, certPassword)};
+ sc = SSLContext.getInstance("SSL", "SunJSSE");
+ //You can also use SSLContext sslContext = SSLContext.getInstance("TLSv1.2");
+ sc.init(null, tm, new SecureRandom());
+ } catch (Exception e) {
+ e.printStackTrace();
+ }
+
+ SSLIOSessionStrategy sessionStrategy = new SSLIOSessionStrategy(sc, new NoopHostnameVerifier());
+ SecuredHttpClientConfigCallback httpClientConfigCallback = new SecuredHttpClientConfigCallback(sessionStrategy,
+ credentialsProvider);
+
+ RestClientBuilder builder = RestClient.builder(constructHttpHosts(host, port, protocol))
+ .setRequestConfigCallback(requestConfig -> requestConfig.setConnectTimeout(connectTimeout)
+ .setConnectionRequestTimeout(connectionRequestTimeout)
+ .setSocketTimeout(socketTimeout))
+ .setHttpClientConfigCallback(httpClientConfigCallback);
+ final RestHighLevelClient client = new RestHighLevelClient(builder);
+ logger.info("opensearch rest client build success {} ", client);
+
+ ClusterHealthRequest request = new ClusterHealthRequest();
+ ClusterHealthResponse response = client.cluster().health(request, RequestOptions.DEFAULT);
+ logger.info("opensearch rest client health response {} ", response);
+ return client;
+ }
+
+ /**
+ * Use the constructHttpHosts function to convert the node IP address list of the host cluster.
+ */
+ public static HttpHost[] constructHttpHosts(List<String> host, int port, String protocol) {
+ return host.stream().map(p -> new HttpHost(p, port, protocol)).toArray(HttpHost[]::new);}
+
+ static class SecuredHttpClientConfigCallback implements RestClientBuilder.HttpClientConfigCallback {
+ @Nullable
+ private final CredentialsProvider credentialsProvider;
+
+ private final SSLIOSessionStrategy sslStrategy;
+
+ SecuredHttpClientConfigCallback(final SSLIOSessionStrategy sslStrategy,
+ @Nullable final CredentialsProvider credentialsProvider) {
+ this.sslStrategy = Objects.requireNonNull(sslStrategy);
+ this.credentialsProvider = credentialsProvider;
+ }
+
+ @Nullable
+ CredentialsProvider getCredentialsProvider() {
+ return credentialsProvider;
+ }
+
+ SSLIOSessionStrategy getSSLStrategy() {
+ return sslStrategy;
+ }
+
+ @Override
+ public HttpAsyncClientBuilder customizeHttpClient(final HttpAsyncClientBuilder httpClientBuilder) {
+ httpClientBuilder.setSSLStrategy(sslStrategy);
+ if (credentialsProvider != null) {
+ httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
+ }
+ return httpClientBuilder;
+ }}
+
+ public static class MyX509TrustManager implements X509TrustManager {
+ X509TrustManager sunJSSEX509TrustManager;
+
+ MyX509TrustManager(String certFilePath, String certPassword) throws Exception {
+ File file = new File(certFilePath);
+ if (!file.isFile()) {
+ throw new Exception("Wrong Certification Path");
+ }
+ System.out.println("Loading KeyStore " + file + "...");
+ InputStream in = new FileInputStream(file);
+ KeyStore ks = KeyStore.getInstance("JKS");
+ ks.load(in, certPassword.toCharArray());
+ TrustManagerFactory tmf = TrustManagerFactory.getInstance("SunX509", "SunJSSE");
+ tmf.init(ks);
+ TrustManager[] tms = tmf.getTrustManagers();
+ for (TrustManager tm : tms) {
+ if (tm instanceof X509TrustManager) {
+ sunJSSEX509TrustManager = (X509TrustManager) tm;
+ return;
+ }
+ }
+ throw new Exception("Couldn't initialize");
+ }
+
+ @Override
+ public void checkClientTrusted(X509Certificate[] chain, String authType) throws CertificateException {
+
+ }
+
+ @Override
+ public void checkServerTrusted(X509Certificate[] chain, String authType) throws CertificateException {
+
+ }
+
+ @Override
+ public X509Certificate[] getAcceptedIssuers() {
+ return new X509Certificate[0];
+ }
+ }
+
+ /**
+* The following is an example of the main function. Call the create function to create the high-level client, then call the getLowLevelClient() function to obtain the low-level client, and query whether the test index exists.
+ */
+ public static void main(String[] args) throws IOException {
+ RestHighLevelClient client = create(Arrays.asList("xxx.xxx.xxx.xxx", "xxx.xxx.xxx.xxx"), 9200, "https", 30000, 30000, 30000, "username", "password", "certFilePath", "certPassword");
+ RestClient lowLevelClient = client.getLowLevelClient();
+ Request request = new Request("HEAD", "test");
+ Response response = lowLevelClient.performRequest(request);
+ System.out.println(response.getStatusLine().getStatusCode());
+ lowLevelClient.close();
+ }
+}
+ |
+
+
+
Table 5 Function parametersParameter
+ |
+Description
+ |
+
|---|
+
+host
+ |
+IP address for accessing the cluster. If there are multiple IP addresses, separate them with commas (,).
+ |
+
+port
+ |
+Access port of the cluster. The default value is 9200.
+ |
+
+protocol
+ |
+Connection protocol. Set this parameter to https.
+ |
+
+connectTimeout
+ |
+Socket connection timeout (in ms).
+ |
+
+connectionRequestTimeout
+ |
+Socket connection request timeout (in ms).
+ |
+
+socketTimeout
+ |
+Socket request timeout (in ms).
+ |
+
+username
+ |
+Username for accessing the cluster.
+ |
+
+password
+ |
+Password of the user.
+ |
+
+certFilePath
+ |
+Path for storing the security certificate.
+ |
+
+certPassword
+ |
+Password of the security certificate.
+ |
+
+
+
+
+
This piece of code checks whether the test index exists in the cluster. If 200 (the index exists) or 404 (the index does not exist) is returned, it indicates that the cluster is connected.
+
+
Connecting to a Non-Security Mode Cluster Using the Elasticsearch Low Level REST Client
Use the Elasticsearch 7.10.2 Low Level REST Client to connect to an OpenSearch cluster for which the security mode is disabled, and query whether the test index exists. The sample code is as follows:
+
import org.apache.http.HttpHost;
+import org.elasticsearch.client.Request;
+import org.elasticsearch.client.Response;
+import org.elasticsearch.client.RestClient;
+import org.elasticsearch.client.RestClientBuilder;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.List;
+public class RestLowLevelClientExample {
+ public static void main(String[] args) throws IOException {
+ List<String> host = Arrays.asList("{Cluster access address}");
+ RestClientBuilder builder = RestClient.builder(constructHttpHosts(host, 9200, "http"));
+ /**
+ *Create the Low Level Rest Client.
+ */
+ RestClient lowLevelClient = builder.build();
+ /**
+ * Check whether the test index exists. If the index exists, 200 is returned. If the index does not exist, 404 is returned.
+ */
+ Request request = new Request("HEAD", "/test");
+ Response response = lowLevelClient.performRequest(request);
+ System.out.println(response.getStatusLine().getStatusCode());
+ lowLevelClient.close();
+ }
+
+ /**
+ * Use the constructHttpHosts function to convert the node IP address list of the host cluster.
+ */
+ public static HttpHost[] constructHttpHosts(List<String> host, int port, String protocol) {
+ return host.stream().map(p -> new HttpHost(p, port, protocol)).toArray(HttpHost[]::new);
+ }
+}
+
This piece of code checks whether the test index exists in the cluster. If 200 (the index exists) or 404 (the index does not exist) is returned, it indicates that the cluster is connected.
+
+
Connecting to a Security-Mode Cluster Using the Elasticsearch Low Level REST Client (Without a Certificate)
Use the Elasticsearch 7.10.2 Low Level REST Client to connect to a security-mode OpenSearch cluster (HTTP or HTTPS) without loading a security certificate, and query whether the test index exists. The sample code is as follows:
+
1
+ 2
+ 3
+ 4
+ 5
+ 6
+ 7
+ 8
+ 9
+ 10
+ 11
+ 12
+ 13
+ 14
+ 15
+ 16
+ 17
+ 18
+ 19
+ 20
+ 21
+ 22
+ 23
+ 24
+ 25
+ 26
+ 27
+ 28
+ 29
+ 30
+ 31
+ 32
+ 33
+ 34
+ 35
+ 36
+ 37
+ 38
+ 39
+ 40
+ 41
+ 42
+ 43
+ 44
+ 45
+ 46
+ 47
+ 48
+ 49
+ 50
+ 51
+ 52
+ 53
+ 54
+ 55
+ 56
+ 57
+ 58
+ 59
+ 60
+ 61
+ 62
+ 63
+ 64
+ 65
+ 66
+ 67
+ 68
+ 69
+ 70
+ 71
+ 72
+ 73
+ 74
+ 75
+ 76
+ 77
+ 78
+ 79
+ 80
+ 81
+ 82
+ 83
+ 84
+ 85
+ 86
+ 87
+ 88
+ 89
+ 90
+ 91
+ 92
+ 93
+ 94
+ 95
+ 96
+ 97
+ 98
+ 99
+100
+101
+102
+103
+104
+105
+106
+107
+108
+109
+110
+111
+112
+113
+114
+115
+116
+117
+118
+119
+120
+121
+122
+123
+124
+125
+126
+127
+128
+129
+130
+131
+132
+133
+134
+135
+136
+137
+138
+139
+140
+141
+142
+143
+144
+145
+146
+147
+148
+149
+150
+151
+152
+153
+154
+155
+156
+157
+158
+159
+160
+161
+162
+163
+164
+165
+166
+167
+168
+169
+170
+171
+172
+173
+174
+175
+176
+177
+178
+179
+180
+181
+182
+183
+184
+185
+186
+187
+188
+189
+190 | import org.apache.http.HttpHost;
+import org.apache.http.HttpResponse;
+import org.apache.http.auth.AuthScope;
+import org.apache.http.auth.UsernamePasswordCredentials;
+import org.apache.http.client.CredentialsProvider;
+import org.apache.http.impl.client.BasicCredentialsProvider;
+import org.apache.http.impl.client.DefaultConnectionKeepAliveStrategy;
+import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
+import org.apache.http.nio.conn.ssl.SSLIOSessionStrategy;
+import org.apache.http.protocol.HttpContext;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.elasticsearch.client.Request;
+import org.elasticsearch.client.Response;
+import org.elasticsearch.client.RestClient;
+import org.elasticsearch.client.RestClientBuilder;
+import org.elasticsearch.common.Nullable;
+
+import java.io.IOException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.security.SecureRandom;
+import java.security.cert.CertificateException;
+import java.security.cert.X509Certificate;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Objects;
+import java.util.concurrent.TimeUnit;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.TrustManager;import javax.net.ssl.X509TrustManager;
+
+public class RestLowLevelClientExample {
+ /**
+ * Create a class for the client. Define the create function.
+ */
+ public static RestClient create(List<String> host, int port, String protocol, int connectTimeout, int connectionRequestTimeout, int socketTimeout, String username, String password) throws IOException {
+ final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
+ credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, password));
+ SSLContext sc = null;
+ try {
+ sc = SSLContext.getInstance("SSL");
+ sc.init(null, trustAllCerts, new SecureRandom());
+ } catch (KeyManagementException | NoSuchAlgorithmException e) {
+ e.printStackTrace();
+ }
+ SSLIOSessionStrategy sessionStrategy = new SSLIOSessionStrategy(sc, new NullHostNameVerifier());
+ SecuredHttpClientConfigCallback httpClientConfigCallback = new SecuredHttpClientConfigCallback(sessionStrategy,
+ credentialsProvider);
+
+ RestClientBuilder builder = RestClient.builder(constructHttpHosts(host, port, protocol))
+ .setRequestConfigCallback(requestConfig -> requestConfig.setConnectTimeout(connectTimeout)
+ .setConnectionRequestTimeout(connectionRequestTimeout)
+ .setSocketTimeout(socketTimeout))
+ .setHttpClientConfigCallback(httpClientConfigCallback);
+ final RestClient client = builder.build();
+ logger.info("es rest client build success {} ", client);
+ return client;
+ }
+
+ /**
+ * Use the constructHttpHosts function to convert the node IP address list of the host cluster.
+ */
+ public static HttpHost[] constructHttpHosts(List<String> host, int port, String protocol) {
+ return host.stream().map(p -> new HttpHost(p, port, protocol)).toArray(HttpHost[]::new);
+ }
+
+ /**
+ * Configure trustAllCerts to ignore the certificate configuration.
+ */
+ public static TrustManager[] trustAllCerts = new TrustManager[] {
+ new X509TrustManager() {
+ @Override
+ public void checkClientTrusted(X509Certificate[] chain, String authType) throws CertificateException {
+ }
+
+ @Override
+ public void checkServerTrusted(X509Certificate[] chain, String authType) throws CertificateException {
+ }
+
+ @Override
+ public X509Certificate[] getAcceptedIssuers() {
+ return null;
+ }
+ }
+ };
+
+ /**
+* The CustomConnectionKeepAliveStrategy function is used to set the connection keepalive during when there are a large number of short connections or when there are not many data requests.
+ */
+ public static class CustomConnectionKeepAliveStrategy extends DefaultConnectionKeepAliveStrategy {
+ public static final CustomConnectionKeepAliveStrategy INSTANCE = new CustomConnectionKeepAliveStrategy();
+
+ private CustomConnectionKeepAliveStrategy() {
+ super();
+ }
+
+ /**
+ * Maximum keepalive time (in minutes)
+ * The default value is 10 minutes. You can set it based on the number of TCP connections in TIME_WAIT state. If there are too many TCP connections, you can increase this value.
+ */
+ private final long MAX_KEEP_ALIVE_MINUTES = 10;
+
+ @Override
+ public long getKeepAliveDuration(HttpResponse response, HttpContext context) {
+ long keepAliveDuration = super.getKeepAliveDuration(response, context);
+ // <0 indicates an unlimited keepalive period.
+ // Change the period from unlimited to a default period.
+ if (keepAliveDuration < 0) {
+ return TimeUnit.MINUTES.toMillis(MAX_KEEP_ALIVE_MINUTES);
+ }
+ return keepAliveDuration;
+ }
+ }
+
+ private static final Logger logger = LogManager.getLogger(RestLowLevelClientExample.class);
+
+ static class SecuredHttpClientConfigCallback implements RestClientBuilder.HttpClientConfigCallback {
+ @Nullable
+ private final CredentialsProvider credentialsProvider;
+ /**
+ * The {@link SSLIOSessionStrategy} for all requests to enable SSL / TLS encryption.
+ */
+ private final SSLIOSessionStrategy sslStrategy;
+ /**
+ * Create a new {@link SecuredHttpClientConfigCallback}.
+ *
+ * @param credentialsProvider The credential provider, if a username/password have been supplied
+ * @param sslStrategy The SSL strategy, if SSL / TLS have been supplied
+ * @throws NullPointerException if {@code sslStrategy} is {@code null}
+ */
+ SecuredHttpClientConfigCallback(final SSLIOSessionStrategy sslStrategy,
+ @Nullable final CredentialsProvider credentialsProvider) {
+ this.sslStrategy = Objects.requireNonNull(sslStrategy);
+ this.credentialsProvider = credentialsProvider;
+ }
+ /**
+ * Get the {@link CredentialsProvider} that will be added to the HTTP client.
+ *
+ * @return Can be {@code null}.
+ */
+ @Nullable
+ CredentialsProvider getCredentialsProvider() {
+ return credentialsProvider;
+ }
+ /**
+ * Get the {@link SSLIOSessionStrategy} that will be added to the HTTP client.
+ *
+ * @return Never {@code null}.
+ */
+ SSLIOSessionStrategy getSSLStrategy() {
+ return sslStrategy;
+ }
+ /**
+ * Sets the {@linkplain HttpAsyncClientBuilder#setDefaultCredentialsProvider(CredentialsProvider) credential provider},
+ *
+ * @param httpClientBuilder The client to configure.
+ * @return Always {@code httpClientBuilder}.
+ */
+ @Override
+ public HttpAsyncClientBuilder customizeHttpClient(final HttpAsyncClientBuilder httpClientBuilder) {
+ // enable SSL / TLS
+ httpClientBuilder.setSSLStrategy(sslStrategy);
+ // enable user authentication
+ if (credentialsProvider != null) {
+ httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
+ }
+ return httpClientBuilder;
+ }
+ }
+
+ public static class NullHostNameVerifier implements HostnameVerifier {
+ @Override
+ public boolean verify(String arg0, SSLSession arg1) {
+ return true;
+ }
+ }
+
+ /**
+ * The following is an example of the main function. Call the create function to create a Low Level REST Client and check whether the test index exists.
+ */
+ public static void main(String[] args) throws IOException {
+ RestClient lowLevelClient = create(Arrays.asList("xxx.xxx.xxx.xxx", "xxx.xxx.xxx.xxx"), 9200, "http", 30000, 30000, 30000, "username", "password");
+ Request request = new Request("HEAD", "/test");
+ Response response = lowLevelClient.performRequest(request);
+ System.out.println(response.getStatusLine().getStatusCode());
+ lowLevelClient.close();
+ }
+}
+ |
+
+
+
Table 6 VariablesParameter
+ |
+Description
+ |
+
|---|
+
+host
+ |
+IP address for accessing the cluster. If there are multiple IP addresses, separate them with commas (,).
+ |
+
+port
+ |
+Access port of the cluster. The default value is 9200.
+ |
+
+protocol
+ |
+Connection protocol, which can be http or https.
+ |
+
+connectTimeout
+ |
+Socket connection timeout (in ms).
+ |
+
+connectionRequestTimeout
+ |
+Socket connection request timeout (in ms).
+ |
+
+socketTimeout
+ |
+Socket request timeout (in ms).
+ |
+
+username
+ |
+Username for accessing the cluster.
+ |
+
+password
+ |
+Password of the user.
+ |
+
+
+
+
+
This piece of code checks whether the test index exists in the cluster. If 200 (the index exists) or 404 (the index does not exist) is returned, it indicates that the cluster is connected.
+
+
Connecting to a Security-Mode Cluster Using the Elasticsearch Low Level REST Client (With a Certificate)
Use the Elasticsearch 7.10.2 Low Level REST Client to connect to a security-mode OpenSearch cluster that uses HTTPS with a security certificate loaded, and query whether the test index exists. The sample code is as follows:
+
+
1
+ 2
+ 3
+ 4
+ 5
+ 6
+ 7
+ 8
+ 9
+ 10
+ 11
+ 12
+ 13
+ 14
+ 15
+ 16
+ 17
+ 18
+ 19
+ 20
+ 21
+ 22
+ 23
+ 24
+ 25
+ 26
+ 27
+ 28
+ 29
+ 30
+ 31
+ 32
+ 33
+ 34
+ 35
+ 36
+ 37
+ 38
+ 39
+ 40
+ 41
+ 42
+ 43
+ 44
+ 45
+ 46
+ 47
+ 48
+ 49
+ 50
+ 51
+ 52
+ 53
+ 54
+ 55
+ 56
+ 57
+ 58
+ 59
+ 60
+ 61
+ 62
+ 63
+ 64
+ 65
+ 66
+ 67
+ 68
+ 69
+ 70
+ 71
+ 72
+ 73
+ 74
+ 75
+ 76
+ 77
+ 78
+ 79
+ 80
+ 81
+ 82
+ 83
+ 84
+ 85
+ 86
+ 87
+ 88
+ 89
+ 90
+ 91
+ 92
+ 93
+ 94
+ 95
+ 96
+ 97
+ 98
+ 99
+100
+101
+102
+103
+104
+105
+106
+107
+108
+109
+110
+111
+112
+113
+114
+115
+116
+117
+118
+119
+120
+121
+122
+123
+124
+125
+126
+127
+128
+129
+130
+131
+132
+133
+134
+135
+136
+137
+138
+139
+140
+141
+142
+143
+144
+145
+146
+147
+148
+149
+150
+151
+152
+153 | import org.apache.http.HttpHost;
+import org.apache.http.auth.AuthScope;
+import org.apache.http.auth.UsernamePasswordCredentials;
+import org.apache.http.client.CredentialsProvider;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.impl.client.BasicCredentialsProvider;
+import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
+import org.apache.http.nio.conn.ssl.SSLIOSessionStrategy;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.elasticsearch.client.Request;
+import org.elasticsearch.client.Response;
+import org.elasticsearch.client.RestClient;
+import org.elasticsearch.client.RestClientBuilder;
+import org.elasticsearch.common.Nullable;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.security.KeyStore;
+import java.security.SecureRandom;
+import java.security.cert.CertificateException;
+import java.security.cert.X509Certificate;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Objects;
+
+import javax.net.ssl.SSLContext;import javax.net.ssl.TrustManager;
+import javax.net.ssl.TrustManagerFactory;
+import javax.net.ssl.X509TrustManager;
+
+public class RestLowLevelClientExample {
+
+ private static final Logger logger = LogManager.getLogger(RestLowLevelClientExample.class);
+
+ /**
+ * Create a class for the client. Define the create function.
+ */
+ public static RestClient create(List<String> host, int port, String protocol, int connectTimeout, int connectionRequestTimeout, int socketTimeout, String username, String password, String certFilePath, String certPassword) throws IOException {
+ final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
+ credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, password));
+ SSLContext sc = null;
+ try {
+ TrustManager[] tm = {new MyX509TrustManager(certFilePath, certPassword)};
+ sc = SSLContext.getInstance("SSL", "SunJSSE");
+ //You can also use SSLContext sslContext = SSLContext.getInstance("TLSv1.2");
+ sc.init(null, tm, new SecureRandom());
+ } catch (Exception e) {
+ e.printStackTrace();
+ }
+
+ SSLIOSessionStrategy sessionStrategy = new SSLIOSessionStrategy(sc, new NoopHostnameVerifier());
+ SecuredHttpClientConfigCallback httpClientConfigCallback = new SecuredHttpClientConfigCallback(sessionStrategy,
+ credentialsProvider);
+
+ RestClientBuilder builder = RestClient.builder(constructHttpHosts(host, port, protocol))
+ .setRequestConfigCallback(requestConfig -> requestConfig.setConnectTimeout(connectTimeout)
+ .setConnectionRequestTimeout(connectionRequestTimeout)
+ .setSocketTimeout(socketTimeout))
+ .setHttpClientConfigCallback(httpClientConfigCallback);
+ final RestClient client = builder.build();
+ logger.info("es rest client build success {} ", client);
+ return client;
+ }
+
+ /**
+ * Use the constructHttpHosts function to convert the node IP address list of the host cluster.
+ */
+ public static HttpHost[] constructHttpHosts(List<String> host, int port, String protocol) {
+ return host.stream().map(p -> new HttpHost(p, port, protocol)).toArray(HttpHost[]::new);}
+
+ static class SecuredHttpClientConfigCallback implements RestClientBuilder.HttpClientConfigCallback {
+ @Nullable
+ private final CredentialsProvider credentialsProvider;
+
+ private final SSLIOSessionStrategy sslStrategy;
+
+ SecuredHttpClientConfigCallback(final SSLIOSessionStrategy sslStrategy,
+ @Nullable final CredentialsProvider credentialsProvider) {
+ this.sslStrategy = Objects.requireNonNull(sslStrategy);
+ this.credentialsProvider = credentialsProvider;
+ }
+
+ @Nullable
+ CredentialsProvider getCredentialsProvider() {
+ return credentialsProvider;
+ }
+
+ SSLIOSessionStrategy getSSLStrategy() {
+ return sslStrategy;
+ }
+
+ @Override
+ public HttpAsyncClientBuilder customizeHttpClient(final HttpAsyncClientBuilder httpClientBuilder) {
+ httpClientBuilder.setSSLStrategy(sslStrategy);
+ if (credentialsProvider != null) {
+ httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
+ }
+ return httpClientBuilder;
+ }}
+
+ public static class MyX509TrustManager implements X509TrustManager {
+ X509TrustManager sunJSSEX509TrustManager;
+
+ MyX509TrustManager(String certFilePath, String certPassword) throws Exception {
+ File file = new File(certFilePath);
+ if (!file.isFile()) {
+ throw new Exception("Wrong Certification Path");
+ }
+ System.out.println("Loading KeyStore " + file + "...");
+ InputStream in = new FileInputStream(file);
+ KeyStore ks = KeyStore.getInstance("JKS");
+ ks.load(in, certPassword.toCharArray());
+ TrustManagerFactory tmf = TrustManagerFactory.getInstance("SunX509", "SunJSSE");
+ tmf.init(ks);
+ TrustManager[] tms = tmf.getTrustManagers();
+ for (TrustManager tm : tms) {
+ if (tm instanceof X509TrustManager) {
+ sunJSSEX509TrustManager = (X509TrustManager) tm;
+ return;
+ }
+ }
+ throw new Exception("Couldn't initialize");
+ }
+
+ @Override
+ public void checkClientTrusted(X509Certificate[] chain, String authType) throws CertificateException {
+
+ }
+
+ @Override
+ public void checkServerTrusted(X509Certificate[] chain, String authType) throws CertificateException {
+
+ }
+
+ @Override
+ public X509Certificate[] getAcceptedIssuers() {
+ return new X509Certificate[0];
+ }
+ }
+
+ /**
+ * The following is an example of the main function. Call the create function to create a Low Level REST Client and check whether the test index exists.
+ */
+ public static void main(String[] args) throws IOException {
+ RestClient lowLevelClient = create(Arrays.asList("xxx.xxx.xxx.xxx", "xxx.xxx.xxx.xxx"), 9200, "https", 30000, 30000, 30000, "username", "password", "certFilePath", "certPassword");
+ Request request = new Request("HEAD", "test");
+ Response response = lowLevelClient.performRequest(request);
+ System.out.println(response.getStatusLine().getStatusCode());
+ lowLevelClient.close();
+ }
+}
+ |
+
+
+
Table 7 Function parametersParameter
+ |
+Description
+ |
+
|---|
+
+host
+ |
+IP address for accessing the cluster. If there are multiple IP addresses, separate them with commas (,).
+ |
+
+port
+ |
+Access port of the cluster. The default value is 9200.
+ |
+
+protocol
+ |
+Connection protocol. Set this parameter to https.
+ |
+
+connectTimeout
+ |
+Socket connection timeout (in ms).
+ |
+
+connectionRequestTimeout
+ |
+Socket connection request timeout (in ms).
+ |
+
+socketTimeout
+ |
+Socket request timeout (in ms).
+ |
+
+username
+ |
+Username for accessing the cluster.
+ |
+
+password
+ |
+Password of the user.
+ |
+
+certFilePath
+ |
+Path for storing the security certificate.
+ |
+
+certPassword
+ |
+Password of the security certificate.
+ |
+
+
+
+
+
This piece of code checks whether the test index exists in the cluster. If 200 (the index exists) or 404 (the index does not exist) is returned, it indicates that the cluster is connected.
+
+
Connecting to a Non-Security Mode Cluster Using the Elasticsearch High Level REST Client
Use the Elasticsearch 7.10.2 High Level REST Client obtain the Low Level REST Client by calling getLowLevelClient(), use the low-level client to connect to an OpenSearch cluster for which the security mode is disabled, and query whether the test index exists. The sample code is as follows:
+
import org.apache.http.HttpHost;
+import org.elasticsearch.client.Request;
+import org.elasticsearch.client.Response;
+import org.elasticsearch.client.RestClient;
+import org.elasticsearch.client.RestClientBuilder;
+import org.elasticsearch.client.RestHighLevelClient;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.List;
+public class RestLowLevelClientExample {
+
+ public static void main(String[] args) throws IOException {
+ List<String> host = Arrays.asList("{Cluster access address}");
+ RestClientBuilder builder = RestClient.builder(constructHttpHosts(host, 9200, "http"));
+ final RestHighLevelClient restHighLevelClient = new RestHighLevelClient(builder);
+ /**
+ * Create a high-level client and then call getLowLevelClient() to obtain a low-level client. The code differs from the client creation code only in the following line:
+ */
+ final RestClient lowLevelClient = restHighLevelClient.getLowLevelClient();
+ /**
+ * Check whether the test index exists. If the index exists, 200 is returned. If the index does not exist, 404 is returned.
+ */
+ Request request = new Request("HEAD", "/test");
+ Response response = lowLevelClient.performRequest(request);
+ System.out.println(response.getStatusLine().getStatusCode());
+ lowLevelClient.close();
+ }
+
+ /**
+ * Use the constructHttpHosts function to convert the node IP address list of the host cluster.
+ */
+ public static HttpHost[] constructHttpHosts(List<String> host, int port, String protocol) {
+ return host.stream().map(p -> new HttpHost(p, port, protocol)).toArray(HttpHost[]::new);
+ }
+}
+
This piece of code checks whether the test index exists in the cluster. If 200 (the index exists) or 404 (the index does not exist) is returned, it indicates that the cluster is connected.
+
+
Connecting to a Security-Mode Cluster Using the Elasticsearch High Level REST Client (Without a Certificate)
Use the Elasticsearch 7.10.2 High Level REST Client to obtain the Low Level REST Client by calling getLowLevelClient(), use the low-level client to connect to a security-mode OpenSearch cluster that uses HTTP or HTTPS without loading a security certificate, and query whether the test index exists. The sample code is as follows:
+
1
+ 2
+ 3
+ 4
+ 5
+ 6
+ 7
+ 8
+ 9
+ 10
+ 11
+ 12
+ 13
+ 14
+ 15
+ 16
+ 17
+ 18
+ 19
+ 20
+ 21
+ 22
+ 23
+ 24
+ 25
+ 26
+ 27
+ 28
+ 29
+ 30
+ 31
+ 32
+ 33
+ 34
+ 35
+ 36
+ 37
+ 38
+ 39
+ 40
+ 41
+ 42
+ 43
+ 44
+ 45
+ 46
+ 47
+ 48
+ 49
+ 50
+ 51
+ 52
+ 53
+ 54
+ 55
+ 56
+ 57
+ 58
+ 59
+ 60
+ 61
+ 62
+ 63
+ 64
+ 65
+ 66
+ 67
+ 68
+ 69
+ 70
+ 71
+ 72
+ 73
+ 74
+ 75
+ 76
+ 77
+ 78
+ 79
+ 80
+ 81
+ 82
+ 83
+ 84
+ 85
+ 86
+ 87
+ 88
+ 89
+ 90
+ 91
+ 92
+ 93
+ 94
+ 95
+ 96
+ 97
+ 98
+ 99
+100
+101
+102
+103
+104
+105
+106
+107
+108
+109
+110
+111
+112
+113
+114
+115
+116
+117
+118
+119
+120
+121
+122
+123
+124
+125
+126
+127
+128
+129
+130
+131
+132
+133
+134
+135
+136
+137
+138
+139
+140
+141
+142
+143
+144
+145
+146
+147
+148
+149
+150
+151
+152
+153
+154
+155
+156
+157
+158
+159
+160
+161
+162
+163
+164
+165
+166
+167
+168
+169
+170
+171
+172
+173
+174
+175
+176
+177
+178
+179
+180
+181
+182
+183
+184
+185
+186
+187
+188
+189
+190
+191
+192
+193
+194
+195 | import org.apache.http.HttpHost;
+import org.apache.http.HttpResponse;
+import org.apache.http.auth.AuthScope;
+import org.apache.http.auth.UsernamePasswordCredentials;
+import org.apache.http.client.CredentialsProvider;
+import org.apache.http.impl.client.BasicCredentialsProvider;
+import org.apache.http.impl.client.DefaultConnectionKeepAliveStrategy;
+import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
+import org.apache.http.nio.conn.ssl.SSLIOSessionStrategy;
+import org.apache.http.protocol.HttpContext;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.elasticsearch.client.Request;
+import org.elasticsearch.client.Response;
+import org.elasticsearch.client.RestClient;
+import org.elasticsearch.client.RestClientBuilder;
+import org.elasticsearch.client.RestHighLevelClient;
+import org.elasticsearch.common.Nullable;
+
+import java.io.IOException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.security.SecureRandom;
+import java.security.cert.CertificateException;
+import java.security.cert.X509Certificate;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Objects;
+import java.util.concurrent.TimeUnit;
+
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLSession;
+import javax.net.ssl.TrustManager;import javax.net.ssl.X509TrustManager;
+
+public class RestLowLevelClientExample {
+
+ /**
+ * Create a class for the client. Define the create function.
+ */
+ public static RestHighLevelClient create(List<String> host, int port, String protocol, int connectTimeout, int connectionRequestTimeout, int socketTimeout, String username, String password) throws IOException {
+
+ final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
+ credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, password));
+ SSLContext sc = null;
+ try {
+ sc = SSLContext.getInstance("SSL");
+ sc.init(null, trustAllCerts, new SecureRandom());
+ } catch (KeyManagementException | NoSuchAlgorithmException e) {
+ e.printStackTrace();
+ }
+ SSLIOSessionStrategy sessionStrategy = new SSLIOSessionStrategy(sc, new NullHostNameVerifier());
+ SecuredHttpClientConfigCallback httpClientConfigCallback = new SecuredHttpClientConfigCallback(sessionStrategy,
+ credentialsProvider);
+
+ RestClientBuilder builder = RestClient.builder(constructHttpHosts(host, port, protocol))
+ .setRequestConfigCallback(requestConfig -> requestConfig.setConnectTimeout(connectTimeout)
+ .setConnectionRequestTimeout(connectionRequestTimeout)
+ .setSocketTimeout(socketTimeout))
+ .setHttpClientConfigCallback(httpClientConfigCallback);
+ final RestHighLevelClient client = new RestHighLevelClient(builder);
+ logger.info("es rest client build success {} ", client);
+ return client;
+ }
+
+ /**
+ * Use the constructHttpHosts function to convert the node IP address list of the host cluster.
+ */
+ public static HttpHost[] constructHttpHosts(List<String> host, int port, String protocol) {
+ return host.stream().map(p -> new HttpHost(p, port, protocol)).toArray(HttpHost[]::new);
+ }
+
+ /**
+ * Configure trustAllCerts to ignore the certificate configuration.
+ */
+ public static TrustManager[] trustAllCerts = new TrustManager[] {
+ new X509TrustManager() {
+ @Override
+ public void checkClientTrusted(X509Certificate[] chain, String authType) throws CertificateException {
+ }
+
+ @Override
+ public void checkServerTrusted(X509Certificate[] chain, String authType) throws CertificateException {
+ }
+
+ @Override
+ public X509Certificate[] getAcceptedIssuers() {
+ return null;
+ }
+ }
+ };
+
+ /**
+* The CustomConnectionKeepAliveStrategy function is used to set the connection keepalive during when there are a large number of short connections or when there are not many data requests.
+ */
+ public static class CustomConnectionKeepAliveStrategy extends DefaultConnectionKeepAliveStrategy {
+ public static final CustomConnectionKeepAliveStrategy INSTANCE = new CustomConnectionKeepAliveStrategy();
+
+ private CustomConnectionKeepAliveStrategy() {
+ super();
+ }
+
+ /**
+ * Maximum keepalive time (in minutes)
+ * The default value is 10 minutes. You can set it based on the number of TCP connections in TIME_WAIT state. If there are too many TCP connections, you can increase this value.
+ */
+ private final long MAX_KEEP_ALIVE_MINUTES = 10;
+
+ @Override
+ public long getKeepAliveDuration(HttpResponse response, HttpContext context) {
+ long keepAliveDuration = super.getKeepAliveDuration(response, context);
+ // <0 indicates an unlimited keepalive period.
+ // Change the period from unlimited to a default period.
+ if (keepAliveDuration < 0) {
+ return TimeUnit.MINUTES.toMillis(MAX_KEEP_ALIVE_MINUTES);
+ }
+ return keepAliveDuration;
+ }
+ }
+
+ private static final Logger logger = LogManager.getLogger(RestLowLevelClientExample.class);
+
+ static class SecuredHttpClientConfigCallback implements RestClientBuilder.HttpClientConfigCallback {
+ @Nullable
+ private final CredentialsProvider credentialsProvider;
+ /**
+ * The {@link SSLIOSessionStrategy} for all requests to enable SSL / TLS encryption.
+ */
+ private final SSLIOSessionStrategy sslStrategy;
+ /**
+ * Create a new {@link SecuredHttpClientConfigCallback}.
+ *
+ * @param credentialsProvider The credential provider, if a username/password have been supplied
+ * @param sslStrategy The SSL strategy, if SSL / TLS have been supplied
+ * @throws NullPointerException if {@code sslStrategy} is {@code null}
+ */
+ SecuredHttpClientConfigCallback(final SSLIOSessionStrategy sslStrategy,
+ @Nullable final CredentialsProvider credentialsProvider) {
+ this.sslStrategy = Objects.requireNonNull(sslStrategy);
+ this.credentialsProvider = credentialsProvider;
+ }
+ /**
+ * Get the {@link CredentialsProvider} that will be added to the HTTP client.
+ *
+ * @return Can be {@code null}.
+ */
+ @Nullable
+ CredentialsProvider getCredentialsProvider() {
+ return credentialsProvider;
+ }
+ /**
+ * Get the {@link SSLIOSessionStrategy} that will be added to the HTTP client.
+ *
+ * @return Never {@code null}.
+ */
+ SSLIOSessionStrategy getSSLStrategy() {
+ return sslStrategy;
+ }
+ /**
+ * Sets the {@linkplain HttpAsyncClientBuilder#setDefaultCredentialsProvider(CredentialsProvider) credential provider},
+ *
+ * @param httpClientBuilder The client to configure.
+ * @return Always {@code httpClientBuilder}.
+ */
+ @Override
+ public HttpAsyncClientBuilder customizeHttpClient(final HttpAsyncClientBuilder httpClientBuilder) {
+ // enable SSL / TLS
+ httpClientBuilder.setSSLStrategy(sslStrategy);
+ // enable user authentication
+ if (credentialsProvider != null) {
+ httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
+ }
+ return httpClientBuilder;
+ }
+ }
+
+ public static class NullHostNameVerifier implements HostnameVerifier {
+ @Override
+ public boolean verify(String arg0, SSLSession arg1) {
+ return true;
+ }
+ }
+
+ /**
+* The following is an example of the main function. Call the create function to create the high-level client, then call the getLowLevelClient() function to obtain the low-level client, and query whether the test index exists.
+ */
+ public static void main(String[] args) throws IOException {
+ RestHighLevelClient client = create(Arrays.asList("xxx.xxx.xxx.xxx", "xxx.xxx.xxx.xxx"), 9200, "http", 30000, 30000, 30000, "username", "password");
+ RestClient lowLevelClient = client.getLowLevelClient();
+ Request request = new Request("HEAD", "test");
+ Response response = lowLevelClient.performRequest(request);
+ System.out.println(response.getStatusLine().getStatusCode());
+ lowLevelClient.close();
+ }
+}
+ |
+
+
+
Table 8 VariablesParameter
+ |
+Description
+ |
+
|---|
+
+host
+ |
+IP address for accessing the cluster. If there are multiple IP addresses, separate them with commas (,).
+ |
+
+port
+ |
+Access port of the cluster. The default value is 9200.
+ |
+
+protocol
+ |
+Connection protocol, which can be http or https.
+ |
+
+connectTimeout
+ |
+Socket connection timeout (in ms).
+ |
+
+connectionRequestTimeout
+ |
+Socket connection request timeout (in ms).
+ |
+
+socketTimeout
+ |
+Socket request timeout (in ms).
+ |
+
+username
+ |
+Username for accessing the cluster.
+ |
+
+password
+ |
+Password of the user.
+ |
+
+
+
+
+
This piece of code checks whether the test index exists in the cluster. If 200 (the index exists) or 404 (the index does not exist) is returned, it indicates that the cluster is connected.
+
+
Connecting to a Security-Mode Cluster Using the Elasticsearch High Level REST Client (With a Certificate)
Use the Elasticsearch 7.10.2 High Level REST Client to obtain the Low Level REST Client by calling getLowLevelClient(), use the low-level client to connect to a security-mode OpenSearch cluster that uses HTTPS with a security certificate loaded, and query whether the test index exists. The sample code is as follows:
+
+
1
+ 2
+ 3
+ 4
+ 5
+ 6
+ 7
+ 8
+ 9
+ 10
+ 11
+ 12
+ 13
+ 14
+ 15
+ 16
+ 17
+ 18
+ 19
+ 20
+ 21
+ 22
+ 23
+ 24
+ 25
+ 26
+ 27
+ 28
+ 29
+ 30
+ 31
+ 32
+ 33
+ 34
+ 35
+ 36
+ 37
+ 38
+ 39
+ 40
+ 41
+ 42
+ 43
+ 44
+ 45
+ 46
+ 47
+ 48
+ 49
+ 50
+ 51
+ 52
+ 53
+ 54
+ 55
+ 56
+ 57
+ 58
+ 59
+ 60
+ 61
+ 62
+ 63
+ 64
+ 65
+ 66
+ 67
+ 68
+ 69
+ 70
+ 71
+ 72
+ 73
+ 74
+ 75
+ 76
+ 77
+ 78
+ 79
+ 80
+ 81
+ 82
+ 83
+ 84
+ 85
+ 86
+ 87
+ 88
+ 89
+ 90
+ 91
+ 92
+ 93
+ 94
+ 95
+ 96
+ 97
+ 98
+ 99
+100
+101
+102
+103
+104
+105
+106
+107
+108
+109
+110
+111
+112
+113
+114
+115
+116
+117
+118
+119
+120
+121
+122
+123
+124
+125
+126
+127
+128
+129
+130
+131
+132
+133
+134
+135
+136
+137
+138
+139
+140
+141
+142
+143
+144
+145
+146
+147
+148
+149
+150
+151
+152
+153
+154
+155
+156
+157
+158
+159
+160
+161 | import org.apache.http.HttpHost;
+import org.apache.http.auth.AuthScope;
+import org.apache.http.auth.UsernamePasswordCredentials;
+import org.apache.http.client.CredentialsProvider;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.impl.client.BasicCredentialsProvider;
+import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
+import org.apache.http.nio.conn.ssl.SSLIOSessionStrategy;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.elasticsearch.action.admin.cluster.health.ClusterHealthRequest;
+import org.elasticsearch.action.admin.cluster.health.ClusterHealthResponse;
+import org.elasticsearch.client.Request;
+import org.elasticsearch.client.RequestOptions;
+import org.elasticsearch.client.Response;
+import org.elasticsearch.client.RestClient;
+import org.elasticsearch.client.RestClientBuilder;
+import org.elasticsearch.client.RestHighLevelClient;
+import org.elasticsearch.common.Nullable;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.security.KeyStore;
+import java.security.SecureRandom;
+import java.security.cert.CertificateException;
+import java.security.cert.X509Certificate;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Objects;
+
+import javax.net.ssl.SSLContext;import javax.net.ssl.TrustManager;
+import javax.net.ssl.TrustManagerFactory;
+import javax.net.ssl.X509TrustManager;
+
+public class RestLowLevelClientExample {
+ private static final Logger logger = LogManager.getLogger(RestLowLevelClientExample.class);
+
+ /**
+ * Create a class for the client. Define the create function.
+ */
+ public static RestHighLevelClient create(List<String> host, int port, String protocol, int connectTimeout, int connectionRequestTimeout, int socketTimeout, String username, String password, String certFilePath, String certPassword) throws IOException {
+ final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
+ credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, password));
+ SSLContext sc = null;
+ try {
+ TrustManager[] tm = {new MyX509TrustManager(certFilePath, certPassword)};
+ sc = SSLContext.getInstance("SSL", "SunJSSE");
+ //You can also use SSLContext sslContext = SSLContext.getInstance("TLSv1.2");
+ sc.init(null, tm, new SecureRandom());
+ } catch (Exception e) {
+ e.printStackTrace();
+ }
+
+ SSLIOSessionStrategy sessionStrategy = new SSLIOSessionStrategy(sc, new NoopHostnameVerifier());
+ SecuredHttpClientConfigCallback httpClientConfigCallback = new SecuredHttpClientConfigCallback(sessionStrategy,
+ credentialsProvider);
+
+ RestClientBuilder builder = RestClient.builder(constructHttpHosts(host, port, protocol))
+ .setRequestConfigCallback(requestConfig -> requestConfig.setConnectTimeout(connectTimeout)
+ .setConnectionRequestTimeout(connectionRequestTimeout)
+ .setSocketTimeout(socketTimeout))
+ .setHttpClientConfigCallback(httpClientConfigCallback);
+ final RestHighLevelClient client = new RestHighLevelClient(builder);
+ logger.info("es rest client build success {} ", client);
+
+ ClusterHealthRequest request = new ClusterHealthRequest();
+ ClusterHealthResponse response = client.cluster().health(request, RequestOptions.DEFAULT);
+ logger.info("es rest client health response {} ", response);
+ return client;
+ }
+
+ /**
+ * Use the constructHttpHosts function to convert the node IP address list of the host cluster.
+ */
+ public static HttpHost[] constructHttpHosts(List<String> host, int port, String protocol) {
+ return host.stream().map(p -> new HttpHost(p, port, protocol)).toArray(HttpHost[]::new);}
+
+ static class SecuredHttpClientConfigCallback implements RestClientBuilder.HttpClientConfigCallback {
+ @Nullable
+ private final CredentialsProvider credentialsProvider;
+
+ private final SSLIOSessionStrategy sslStrategy;
+
+ SecuredHttpClientConfigCallback(final SSLIOSessionStrategy sslStrategy,
+ @Nullable final CredentialsProvider credentialsProvider) {
+ this.sslStrategy = Objects.requireNonNull(sslStrategy);
+ this.credentialsProvider = credentialsProvider;
+ }
+
+ @Nullable
+ CredentialsProvider getCredentialsProvider() {
+ return credentialsProvider;
+ }
+
+ SSLIOSessionStrategy getSSLStrategy() {
+ return sslStrategy;
+ }
+
+ @Override
+ public HttpAsyncClientBuilder customizeHttpClient(final HttpAsyncClientBuilder httpClientBuilder) {
+ httpClientBuilder.setSSLStrategy(sslStrategy);
+ if (credentialsProvider != null) {
+ httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
+ }
+ return httpClientBuilder;
+ }}
+
+ public static class MyX509TrustManager implements X509TrustManager {
+ X509TrustManager sunJSSEX509TrustManager;
+
+ MyX509TrustManager(String certFilePath, String certPassword) throws Exception {
+ File file = new File(certFilePath);
+ if (!file.isFile()) {
+ throw new Exception("Wrong Certification Path");
+ }
+ System.out.println("Loading KeyStore " + file + "...");
+ InputStream in = new FileInputStream(file);
+ KeyStore ks = KeyStore.getInstance("JKS");
+ ks.load(in, certPassword.toCharArray());
+ TrustManagerFactory tmf = TrustManagerFactory.getInstance("SunX509", "SunJSSE");
+ tmf.init(ks);
+ TrustManager[] tms = tmf.getTrustManagers();
+ for (TrustManager tm : tms) {
+ if (tm instanceof X509TrustManager) {
+ sunJSSEX509TrustManager = (X509TrustManager) tm;
+ return;
+ }
+ }
+ throw new Exception("Couldn't initialize");
+ }
+
+ @Override
+ public void checkClientTrusted(X509Certificate[] chain, String authType) throws CertificateException {
+
+ }
+
+ @Override
+ public void checkServerTrusted(X509Certificate[] chain, String authType) throws CertificateException {
+
+ }
+
+ @Override
+ public X509Certificate[] getAcceptedIssuers() {
+ return new X509Certificate[0];
+ }
+ }
+
+ /**
+* The following is an example of the main function. Call the create function to create the high-level client, then call the getLowLevelClient() function to obtain the low-level client, and query whether the test index exists.
+ */
+ public static void main(String[] args) throws IOException {
+ RestHighLevelClient client = create(Arrays.asList("xxx.xxx.xxx.xxx", "xxx.xxx.xxx.xxx"), 9200, "https", 30000, 30000, 30000, "username", "password", "certFilePath", "certPassword");
+ RestClient lowLevelClient = client.getLowLevelClient();
+ Request request = new Request("HEAD", "test");
+ Response response = lowLevelClient.performRequest(request);
+ System.out.println(response.getStatusLine().getStatusCode());
+ lowLevelClient.close();
+ }
+}
+ |
+
+
+
Table 9 Function parametersParameter
+ |
+Description
+ |
+
|---|
+
+host
+ |
+IP address for accessing the cluster. If there are multiple IP addresses, separate them with commas (,).
+ |
+
+port
+ |
+Access port of the cluster. The default value is 9200.
+ |
+
+protocol
+ |
+Connection protocol. Set this parameter to https.
+ |
+
+connectTimeout
+ |
+Socket connection timeout (in ms).
+ |
+
+connectionRequestTimeout
+ |
+Socket connection request timeout (in ms).
+ |
+
+socketTimeout
+ |
+Socket request timeout (in ms).
+ |
+
+username
+ |
+Username for accessing the cluster.
+ |
+
+password
+ |
+Password of the user.
+ |
+
+certFilePath
+ |
+Path for storing the security certificate.
+ |
+
+certPassword
+ |
+Password of the security certificate.
+ |
+
+
+
+
+
This piece of code checks whether the test index exists in the cluster. If 200 (the index exists) or 404 (the index does not exist) is returned, it indicates that the cluster is connected.
+
+
Obtaining and Uploading a Security Certificate
To access a security-mode OpenSearch cluster that uses HTTPS, a security certificate must be loaded. Perform the following steps to obtain the security certificate and upload it to the client:
+
- Obtain the security certificate CloudSearchService.cer.
- Log in to the CSS management console.
- In the navigation pane on the left, choose Clusters > OpenSearch.
- In the cluster list, click the name of the target cluster. The cluster information page is displayed.
- Click the Overview tab. In the Configuration area, click Download Certificate next to HTTPS Access.
+ - Convert the security certificate CloudSearchService.cer. Upload the downloaded security certificate to the client and use keytool to convert the .cer certificate into a .jks certificate that can be read by Java.
+
In the preceding command, newname indicates the user-defined certificate name.
+After this command is executed, you will be prompted to set the certificate password and confirm the password. Securely store the password. It will be used for accessing the cluster.
+
+
+
+
+
diff --git a/docs/css/umn/css_01_0042.html b/docs/css/umn/css_01_0042.html
new file mode 100644
index 000000000..7d8a1d329
--- /dev/null
+++ b/docs/css/umn/css_01_0042.html
@@ -0,0 +1,3968 @@
+
+
+
Monitoring Metrics for Elasticsearch Clusters in Cloud Eye
+
You can use Cloud Eye to monitor CSS cluster metrics in real time and quickly handle exceptions. For details about Cloud Eye, see Cloud Eye User Guide.
+
Overview
This section describes CSS metrics that can be monitored by Cloud Eye as well as their namespaces and dimensions. You can query metrics and alerts generated by CSS on the Cloud Eye console or using an API.
+
Cloud Eye can monitor dimensions (or objects) nested to a maximum depth of four levels (levels 0 to 3). 3 is the deepest level. For example, if the monitored dimension of a metric is cluster_id,instance_id, cluster_id indicates level 0 and instance_id indicates level 1.
+
+
+
+
Cluster Monitoring Metrics
Accumulated value: The value is accumulated from when a node is started. After the node is restarted, the value is reset to zero and starts accumulating again.
+
+
+
Table 1 Elasticsearch cluster monitoring metricsMetric ID
+ |
+Metric
+ |
+Description
+ |
+Value Range
+ |
+Unit
+ |
+Number System
+ |
+Dimension
+ |
+Monitoring Interval (Raw)
+ |
+
|---|
+
+status
+ |
+Cluster Health Status
+ |
+Health status of the monitored object
+- 0: The cluster is 100% available.
- 1: All primary shards are allocated, but some replicas are missing. No data is missing, so search results will still be complete. However, high availability is compromised to some degree. If more shards disappear, you might lose data. Think of this status as a warning that should prompt investigation.
- 2: Data is missing and the cluster cannot work properly.
- 3: Failed to obtain the cluster status.
+ |
+0, 1, 2, 3
+ |
+N/A
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+disk_util
+ |
+Max. Disk Usage
+ |
+Maximum disk usage of nodes in a CSS cluster
+ |
+0~100
+ |
+%
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_jvm_heap_usage
+ |
+Max. JVM Heap Usage
+ |
+Maximum JVM heap usage of nodes in a CSS cluster.
+ |
+0~100
+ |
+%
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_jvm_young_gc_time
+ |
+Max. JVM Young GC Duration
+ |
+Maximum accumulated JVM Young GC duration of nodes in a CSS cluster.
+ |
+≥ 0
+ |
+ms
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_jvm_young_gc_count
+ |
+Max. JVM Young GC Count
+ |
+Maximum accumulated JVM Young GC count of nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_jvm_old_gc_time
+ |
+Max. JVM Old GC Duration
+ |
+Maximum accumulated JVM Old GC duration of nodes in a CSS cluster.
+ |
+≥ 0
+ |
+ms
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_jvm_old_gc_count
+ |
+Max. JVM Old GC Count
+ |
+Maximum accumulated JVM Old GC count of nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+total_fs_size
+ |
+Total Size of File Systems
+ |
+Total size of file systems in a CSS cluster.
+ |
+≥ 0
+ |
+byte
+ |
+1024
+ |
+cluster_id
+ |
+1 minute
+ |
+
+free_fs_size
+ |
+Available Size of File Systems
+ |
+Available size of file systems in a CSS cluster.
+ |
+≥ 0
+ |
+byte
+ |
+1024
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_cpu_usage
+ |
+Max. CPU Usage
+ |
+Maximum node CPU usage in a CSS cluster.
+ |
+0~100
+ |
+%
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_cpu_time_of_jvm_process
+ |
+Max. CPU Time of JVM Process
+ |
+Maximum accumulated CPU usage duration of node JVM processes in a CSS cluster.
+ |
+≥ 0
+ |
+ms
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_virtual_memory_size_of_jvm_process
+ |
+Max. Virtual Memory Size of JVM Process
+ |
+Maximum virtual memory size of node JVM processes in a CSS cluster.
+ |
+≥ 0
+ |
+byte
+ |
+1024
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_current_opened_http_count
+ |
+Current Max. Opened HTTP Connections
+ |
+Maximum number of HTTP connections that are currently open for nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_total_opened_http_count
+ |
+Total Max. Opened HTTP Connections
+ |
+Maximum number of HTTP connections that were open for nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+indices_count
+ |
+Indexes
+ |
+Number of indexes in a CSS cluster
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+total_shards_count
+ |
+Shards
+ |
+Number of shards in a CSS cluster
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+primary_shards_count
+ |
+Primary Shards
+ |
+Number of primary shards in a CSS cluster
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+docs_count
+ |
+Documents
+ |
+Number of documents in a CSS cluster
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+docs_deleted_count
+ |
+Deleted Documents
+ |
+Number of documents deleted in a CSS cluster
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+nodes_count
+ |
+Nodes
+ |
+Number of nodes in a CSS cluster
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+data_nodes_count
+ |
+Data Nodes
+ |
+Number of data nodes in a CSS cluster
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+coordinating_nodes_count
+ |
+Coordinator Nodes
+ |
+Number of coordinator nodes in a CSS cluster
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+master_nodes_count
+ |
+Master Nodes
+ |
+Number of master nodes in a CSS cluster
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+ingest_nodes_count
+ |
+Client Nodes
+ |
+Number of client nodes in a CSS cluster
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_load_average
+ |
+Max. Node Load
+ |
+Maximum number of average queued tasks per minute on nodes in a cluster
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_cpu_usage
+ |
+Avg. CPU Usage
+ |
+Average node CPU usage in a CSS cluster.
+ |
+0~100
+ |
+%
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_load_average
+ |
+Avg. Node Load
+ |
+Average number of queued tasks per minute on nodes in a CSS cluster
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_jvm_heap_usage
+ |
+Avg. JVM Heap Usage
+ |
+Average node JVM heap usage in a CSS cluster.
+ |
+0~100
+ |
+%
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_open_file_descriptors
+ |
+Max. Open File Descriptors
+ |
+Maximum number of node file descriptors that are currently open in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_open_file_descriptors
+ |
+Avg. Open File Descriptors
+ |
+Average number of node file descriptors that are currently open in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_max_file_descriptors
+ |
+Max. Allowed File Descriptors
+ |
+Maximum number of allowed node file descriptors in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_open_file_descriptors
+ |
+Open File Descriptors
+ |
+Number of node file descriptors that are currently open in a cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_write_queue
+ |
+Tasks in Write Queue
+ |
+Number of tasks in queue for the write thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_search_queue
+ |
+Tasks in Search Queue
+ |
+Total number of tasks in queue for the search thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_force_merge_queue
+ |
+Tasks in ForceMerge Queue
+ |
+Total number of tasks in queue for the force merge thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_write_rejected
+ |
+Rejected Tasks in Write Queue
+ |
+Total number of tasks rejected by the write thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_search_rejected
+ |
+Rejected Tasks in Search Queue
+ |
+Total number of tasks rejected by the search thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_force_merge_rejected
+ |
+Rejected Tasks in ForceMerge Queue
+ |
+Total number of tasks rejected by the force merge thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_thread_pool_search_queue
+ |
+Max. Tasks in Search Queue
+ |
+Maximum number of tasks in queue for the search thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_thread_pool_force_merge_queue
+ |
+Max. Tasks in ForceMerge Queue
+ |
+Maximum number of tasks in queue for the force merge thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_write_threads
+ |
+Size of Write Thread Pool
+ |
+Total size of the write thread pools on nodes in the CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_search_threads
+ |
+Size of Search Thread Pool
+ |
+Total size of the search thread pools on nodes in the CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_force_merge_threads
+ |
+Size of ForceMerge Thread Pool
+ |
+Total size of the force merge thread pools on nodes in the CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_thread_pool_write_queue
+ |
+Avg. Tasks in Write Queue
+ |
+Average number of tasks in queue for the write thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_thread_pool_search_queue
+ |
+Avg. Tasks in Search Queue
+ |
+Average number of tasks in queue for the search thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_thread_pool_force_merge_queue
+ |
+Avg. Tasks in ForceMerge Queue
+ |
+Average number of tasks in queue for the force merge thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_thread_pool_search_threads
+ |
+Avg. Size of Search Thread Pool
+ |
+Average size of the search thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_thread_pool_write_threads
+ |
+Avg. Size of Write Thread Pool
+ |
+Average size of the write thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_thread_pool_force_merge_threads
+ |
+Avg. Size of ForceMerge Thread Pool
+ |
+Average size of the force merge thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_thread_pool_write_rejected
+ |
+Avg. Rejected Tasks in Write Queue
+ |
+Average number of tasks rejected by the write thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_flush_queue
+ |
+Tasks in Flush Queue
+ |
+Total number of tasks in queue for the flush thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_flush_rejected
+ |
+Rejected Tasks in Flush Queue
+ |
+Total number of tasks rejected by the flush thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_thread_pool_flush_queue
+ |
+Max. Tasks in Flush Queue
+ |
+Maximum number of tasks in queue for the flush thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_flush_threads
+ |
+Size of Flush Thread Pool
+ |
+Total size of the flush thread pools on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_thread_pool_flush_queue
+ |
+Avg. Tasks in Flush Queue
+ |
+Average number of tasks in queue for the flush thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_thread_pool_flush_threads
+ |
+Avg. Size of Flush Thread Pool
+ |
+Average number of tasks in queue for the flush thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_generic_queue
+ |
+Tasks in Generic Queue
+ |
+Total number of tasks in queue for the generic thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_generic_rejected
+ |
+Rejected Tasks in Generic Queue
+ |
+Total number of tasks rejected by the generic thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_thread_pool_generic_queue
+ |
+Max. Tasks in Generic Queue
+ |
+Maximum number of tasks in queue for the generic thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_generic_threads
+ |
+Size of Generic Thread Pool
+ |
+Total size of the generic thread pools on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_thread_pool_generic_queue
+ |
+Avg. Tasks in Generic Queue
+ |
+Average number of tasks in queue for the generic thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_thread_pool_generic_threads
+ |
+Avg. Size of Generic Thread Pool
+ |
+Average number of tasks in queue for the generic thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_management_queue
+ |
+Tasks in Management Queue
+ |
+Total number of tasks in queue for the management thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_management_rejected
+ |
+Rejected Tasks in Management Queue
+ |
+Total number of tasks rejected by the management thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_thread_pool_management_queue
+ |
+Max. Tasks in Management Queue
+ |
+Maximum number of tasks in queue for the management thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_management_threads
+ |
+Size of Management Thread Pool
+ |
+Total size of the management thread pools on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_thread_pool_management_queue
+ |
+Avg. Tasks in Management Queue
+ |
+Average number of tasks in queue for the management thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_thread_pool_management_threads
+ |
+Avg. Size of Management Thread Pool
+ |
+Average number of tasks in queue for the management thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_refresh_queue
+ |
+Tasks in Refresh Queue
+ |
+Total number of tasks in queue for the refresh thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_refresh_rejected
+ |
+Rejected Tasks in Refresh Queue
+ |
+Total number of tasks rejected by the refresh thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_thread_pool_refresh_queue
+ |
+Max. Tasks in Refresh Queue
+ |
+Maximum number of tasks in queue for the refresh thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_refresh_threads
+ |
+Size of Refresh Thread Pool
+ |
+Total size of the refresh thread pools on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_thread_pool_refresh_queue
+ |
+Avg. Tasks in Refresh Queue
+ |
+Average number of tasks in queue for the refresh thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_thread_pool_refresh_threads
+ |
+Avg. Size of Refresh Thread Pool
+ |
+Average number of tasks in queue for the refresh thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_obs_searcher_queue
+ |
+Tasks in OBS Searcher Queue
+ |
+Total number of tasks in queue for the OBS Searcher thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_obs_searcher_rejected
+ |
+Rejected Tasks in OBS Searcher Queue
+ |
+Total number of tasks rejected by the OBS Searcher thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_thread_pool_obs_searcher_queue
+ |
+Max. Tasks in OBS Searcher Queue
+ |
+Maximum number of tasks in queue for the OBS Searcher thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_obs_searcher_threads
+ |
+Size of OBS Searcher Thread Pool
+ |
+Total size of the OBS Searcher thread pools on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_thread_pool_obs_searcher_queue
+ |
+Avg. Tasks in OBS Searcher Queue
+ |
+Average number of tasks in queue for the OBS Searcher thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_thread_pool_obs_searcher_threads
+ |
+Avg. Size of OBS Searcher Thread Pool
+ |
+Average number of tasks in queue for the OBS Searcher thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_obs_queue
+ |
+Tasks in OBS Queue
+ |
+Total number of tasks in queue for the OBS thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_obs_rejected
+ |
+Rejected Tasks in OBS Queue
+ |
+Total number of tasks rejected by the OBS thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_thread_pool_obs_queue
+ |
+Max. Tasks in OBS Queue
+ |
+Maximum number of tasks in queue for the OBS thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_obs_threads
+ |
+Size of OBS Thread Pool
+ |
+Total size of the OBS thread pools on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_thread_pool_obs_queue
+ |
+Avg. Tasks in OBS Queue
+ |
+Average number of tasks in queue for the OBS thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_thread_pool_obs_threads
+ |
+Avg. Size of OBS Thread Pool
+ |
+Average number of tasks in queue for the OBS thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_obs_upload_queue
+ |
+Tasks in OBS Upload Queue
+ |
+Total number of tasks in queue for the OBS Upload thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_obs_upload_rejected
+ |
+Rejected Tasks in OBS Upload Queue
+ |
+Total number of tasks rejected by the OBS Upload thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_thread_pool_obs_upload_queue
+ |
+Max. Tasks in OBS Upload Queue
+ |
+Maximum number of tasks in queue for the OBS Upload thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_obs_upload_threads
+ |
+Size of OBS Upload Thread Pool
+ |
+Total size of the OBS Upload thread pools on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_thread_pool_obs_upload_queue
+ |
+Avg. Tasks in OBS Upload Queue
+ |
+Average number of tasks in queue for the OBS Upload thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_thread_pool_obs_upload_threads
+ |
+Avg. Size of OBS Upload Thread Pool
+ |
+Average number of tasks in queue for the OBS Upload thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_obs_download_queue
+ |
+Tasks in OBS Download Queue
+ |
+Total number of tasks in queue for the OBS Download thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_obs_download_rejected
+ |
+Rejected Tasks in OBS Download Queue
+ |
+Total number of tasks rejected by the OBS Download thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_thread_pool_obs_download_queue
+ |
+Max. Tasks in OBS Download Queue
+ |
+Maximum number of tasks in queue for the OBS Download thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_thread_pool_obs_download_threads
+ |
+Size of OBS Download Thread Pool
+ |
+Total size of the OBS Download thread pools on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_thread_pool_obs_download_queue
+ |
+Avg. Tasks in OBS Download Queue
+ |
+Average number of tasks in queue for the OBS Download thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_thread_pool_obs_download_threads
+ |
+Avg. Size of OBS Download Thread Pool
+ |
+Average number of tasks in queue for the OBS Download thread pool on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+min_free_fs_size
+ |
+Min. Available Storage Space
+ |
+Minimum available storage space of nodes in a CSS cluster.
+ |
+≥ 0
+ |
+byte
+ |
+1024
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_jvm_old_gc_count
+ |
+Avg. GCs of Old-Generation JVM
+ |
+Average number of old-generation garbage collections of nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_jvm_old_gc_time
+ |
+Avg. GC Duration of Old-Generation JVM
+ |
+Average old-generation garbage collection duration of nodes in a CSS cluster.
+ |
+≥ 0
+ |
+ms
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_jvm_young_gc_count
+ |
+Avg. GCs of Young-Generation JVM
+ |
+Average number of young-generation garbage collections of nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_jvm_young_gc_time
+ |
+Avg. GC Duration of Young-Generation JVM
+ |
+Average young-generation garbage collection duration of nodes in a CSS cluster.
+ |
+≥ 0
+ |
+ms
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_max_file_descriptors
+ |
+Avg. Maximum Allowed File Descriptors
+ |
+Average value of the maximum number of allowed file descriptors on each node in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_mem_free_in_bytes
+ |
+Avg. Available Memory
+ |
+Average unused memory capacity of nodes in a CSS cluster.
+ |
+≥ 0
+ |
+byte
+ |
+1024
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_mem_free_percent
+ |
+Avg. Available Memory Percentage
+ |
+Average percentage of unused memory of nodes in a CSS cluster.
+ |
+0~100
+ |
+%
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_mem_used_in_bytes
+ |
+Avg. Used Memory
+ |
+Average used memory of nodes in a CSS cluster.
+ |
+≥ 0
+ |
+byte
+ |
+1024
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_mem_used_percent
+ |
+Avg. Used Memory Percentage
+ |
+Average percentage of used memory of nodes in a CSS cluster.
+ |
+0~100
+ |
+%
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_mem_free_in_bytes
+ |
+Max. Available Memory
+ |
+Maximum unused memory of nodes in a CSS cluster.
+ |
+≥ 0
+ |
+byte
+ |
+1024
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_mem_free_percent
+ |
+Max. Available Memory Percentage
+ |
+Maximum percentage of unused memory of nodes in a CSS cluster.
+ |
+0~100
+ |
+%
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_mem_used_in_bytes
+ |
+Max. Used Memory
+ |
+Maximum used memory of nodes in a CSS cluster.
+ |
+≥ 0
+ |
+byte
+ |
+1024
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_mem_used_percent
+ |
+Max. Used Memory Percentage
+ |
+Maximum percentage of used memory of nodes in a CSS cluster.
+ |
+0~100
+ |
+%
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_jvm_old_gc_count
+ |
+Total GCs of Old-Generation JVM
+ |
+Number of old-generation garbage collections of nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_jvm_old_gc_time
+ |
+Total GC Duration of Old-Generation JVM
+ |
+Total old-generation garbage collection duration of nodes in the CSS cluster.
+ |
+≥ 0
+ |
+ms
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_jvm_young_gc_count
+ |
+Total GCs of Young-Generation JVM
+ |
+Number of young-generation garbage collections of nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_jvm_young_gc_time
+ |
+Total GC Duration of Young-Generation JVM
+ |
+Total young-generation garbage collection duration of nodes in the CSS cluster.
+ |
+≥ 0
+ |
+ms
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_current_opened_http_count
+ |
+Currently Open HTTP Connections
+ |
+Number of HTTP connections that are open on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_total_opened_http_count
+ |
+Historical Open HTTP Connections
+ |
+Number of HTTP connections that were open on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+IndexingLatency
+ |
+Average Index Latency
+ |
+Average time required for a shard to complete an indexing operation.
+ |
+≥ 0
+ |
+ms
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+IndexingRate
+ |
+Average Index Rate
+ |
+Average number of index operations per second in a cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+SearchLatency
+ |
+Average Search Latency
+ |
+Average time required for a shard to complete a search operation.
+ |
+≥ 0
+ |
+ms
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+SearchRate
+ |
+Average QPS
+ |
+Average queries per second (QPS) in a cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+task_max_running_time
+ |
+Max. Task Runtime
+ |
+Duration of the most time-consuming search or write task in the cluster.
+ |
+≥ 0
+ |
+ms
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+number_of_pending_tasks
+ |
+Number of Pending Tasks in Queue
+ |
+Number of pending tasks to be processed by the master node in the CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_disk_read_requests_rate
+ |
+Total Disk Read IOPS
+ |
+Number of disk read requests to all nodes in the CSS cluster per second.
+ |
+≥ 0
+ |
+request/s
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_disk_write_requests_rate
+ |
+Total Disk Write IOPS
+ |
+Number of disk write requests to all nodes in the CSS cluster per second.
+ |
+≥ 0
+ |
+request/s
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_disk_read_bytes_rate
+ |
+Total Disk Read Rate
+ |
+Number of bytes of data read from the disks of all nodes in the CSS cluster per second.
+ |
+≥ 0
+ |
+Byte/s
+ |
+1024
+ |
+cluster_id
+ |
+1 minute
+ |
+
+sum_disk_write_bytes_rate
+ |
+Total Disk Write Rate
+ |
+Number of bytes of data written into the disks of all nodes in the CSS cluster per second.
+ |
+≥ 0
+ |
+Byte/s
+ |
+1024
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_vector_index_off_heap_used_in_bytes
+ |
+Average Off-Heap Memory Usage of Vector Indexes
+ |
+Average off-heap memory (bytes) used for querying vector indexes on each node in a CSS cluster.
+ |
+≥ 0
+ |
+byte
+ |
+1024
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_vector_index_off_heap_usage
+ |
+Average Off-Heap Memory Utilization (%) of Vector Indexes
+ |
+Average off-heap memory (in %) used for querying vector indexes on each node in a CSS cluster.
+This metric is calculated only when vector index queries occur.
+ |
+0~100
+ |
+%
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_vector_index_off_heap_used_in_bytes
+ |
+Maximum Off-Heap Memory Usage of Vector Indexes
+ |
+Maximum off-heap memory (bytes) used for querying vector indexes on each node in a CSS cluster.
+ |
+≥ 0
+ |
+byte
+ |
+1024
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_vector_index_off_heap_usage
+ |
+Maximum Off-Heap Memory Utilization (%) of Vector Indexes
+ |
+Maximum off-heap memory (in %) used for querying vector indexes on each node in a CSS cluster.
+This metric is calculated only when vector index queries occur.
+ |
+0~100
+ |
+%
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+vector_index_circuit_breaker_status
+ |
+Vector Index Circuit Breaker Status
+ |
+Circuit breaker status for vector indexes.
+- 0: Not enabled.
- 1: Enabled. Check whether the cluster's off-heap memory usage exceeds the threshold.
+ |
+0, 1
+ |
+N/A
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+request_count
+ |
+Total Requests
+ |
+Total number of requests sent to all nodes in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+successfully_request_count
+ |
+Successful requests
+ |
+Number of requests successfully handled by all nodes in a CSS cluster.
+Status code: 200.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+failed_request_count
+ |
+Failed requests
+ |
+Number of failed requests for all nodes in a CSS cluster.
+Status code: non-200.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+limited_request_count
+ |
+Throttled requests
+ |
+Number of throttled requests for all nodes in a CSS cluster.
+Status code: 429.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+cold_data_storage
+ |
+The size of cold data stored
+ |
+The total size of cold data stored in a CSS cluster, which is the total size of the data of frozen indexes stored in an OBS bucket. This metric is measured hourly. Users cannot check this OBS bucket on the OBS console.
+ |
+≥ 0
+ |
+byte
+ |
+1024
+ |
+cluster_id
+ |
+1 minute
+ |
+
+number_of_index_creation_failures
+ |
+Number of index creation failures
+ |
+Number of index creation failures in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+shard_doc_exceed_threshold_count
+ |
+Number of shards where the number of documents exceeds the threshold
+ |
+Number of shards where the number of documents exceeds the threshold in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+write_fail_alias_error
+ |
+Number of Write Failures Due to Alias Exceptions
+ |
+Number of write failures caused by abnormal index aliases in a CSS cluster.
+Only Elasticsearch 7.6.2 and 7.10.2 clusters support this metric.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+write_fail_documents_count_exceed
+ |
+Number of Write Failures Due to Excessive Documents
+ |
+Number of write failures caused by too many index documents in a CSS cluster.
+Only Elasticsearch 7.6.2 and 7.10.2 clusters support this metric.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+write_fail_fields_count_exceed
+ |
+Number of Write Failures Due to Excessive Fields
+ |
+Number of write failures caused by too many index fields in a CSS cluster.
+Only Elasticsearch 7.6.2 and 7.10.2 clusters support this metric.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+write_fail_shards_count_exceed
+ |
+Number of Write Failures Due to Excessive Shards
+ |
+Number of write failures caused by too many index shards in a CSS cluster.
+Only Elasticsearch 7.6.2 and 7.10.2 clusters support this metric.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+write_fail_index_blocks_write
+ |
+Number of Write Failures Due to Index Write Prohibition
+ |
+Number of write failures caused by the setting of write prohibition in index configuration in a CSS cluster.
+Only Elasticsearch 7.6.2 and 7.10.2 clusters support this metric.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+write_fail_mapping_malformed
+ |
+Number of Write Failures Due to Incorrect Field Types
+ |
+Number of write failures caused by incorrect index field types in a CSS cluster.
+Only Elasticsearch 7.6.2 and 7.10.2 clusters support this metric.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+remote_sync_task_count
+ |
+Number of Primary/Secondary Synchronization Jobs
+ |
+Number of primary/secondary synchronization jobs for a CSS cluster.
+Only Elasticsearch 7.6.2 and 7.10.2 clusters support this metric.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+remote_sync_failed_task_count
+ |
+Number of Failed Primary/Secondary Synchronization Jobs
+ |
+Number of failed primary/secondary synchronization jobs for a CSS cluster.
+Only Elasticsearch 7.6.2 and 7.10.2 clusters support this metric.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+remote_sync_diff_bytes_of_primary_shards
+ |
+Size of Data to Be Synchronized via Primary/Secondary Synchronization Jobs for Primary Shards
+ |
+Total size of data to be synchronized via primary/secondary synchronization jobs for all primary shards in a CSS cluster.
+Only Elasticsearch 7.6.2 and 7.10.2 clusters support this metric.
+ |
+≥ 0
+ |
+byte
+ |
+1024
+ |
+cluster_id
+ |
+1 minute
+ |
+
+remote_sync_finished_bytes_of_primary_shards
+ |
+Size of Data Synchronized via Primary/Secondary Synchronization Jobs for Primary Shards
+ |
+Total size of data already synchronized via primary/secondary synchronization jobs for all primary shards in a CSS cluster.
+Only Elasticsearch 7.6.2 and 7.10.2 clusters support this metric.
+ |
+≥ 0
+ |
+byte
+ |
+1024
+ |
+cluster_id
+ |
+1 minute
+ |
+
+remote_sync_diff_docs_of_primary_shards
+ |
+Number of Documents to Be Synchronized via Primary/Secondary Synchronization Jobs for Primary Shards
+ |
+Total number of documents to be synchronized via primary/secondary synchronization jobs for all primary shards in a CSS cluster.
+Only Elasticsearch 7.6.2 and 7.10.2 clusters support this metric.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+remote_sync_diff_bytes
+ |
+Size of Data to Be Synchronized via Primary/Secondary Synchronization Jobs
+ |
+Total size of data to be synchronized via primary/secondary synchronization jobs for all shards in a CSS cluster.
+Only Elasticsearch 7.6.2 and 7.10.2 clusters support this metric.
+ |
+≥ 0
+ |
+byte
+ |
+1024
+ |
+cluster_id
+ |
+1 minute
+ |
+
+remote_sync_finished_bytes
+ |
+Size of Data Synchronized via Primary/Secondary Synchronization Jobs
+ |
+Total size of data already synchronized via primary/secondary synchronization jobs for all shards in a CSS cluster.
+Only Elasticsearch 7.6.2 and 7.10.2 clusters support this metric.
+ |
+≥ 0
+ |
+byte
+ |
+1024
+ |
+cluster_id
+ |
+1 minute
+ |
+
+remote_sync_diff_docs
+ |
+Number of Documents to Be Synchronized via Primary/Secondary Synchronization Jobs
+ |
+Total number of documents to be synchronized via primary/secondary synchronization jobs for all shards in a CSS cluster.
+Only Elasticsearch 7.6.2 and 7.10.2 clusters support this metric.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+index_storage
+ |
+Index Data Volume
+ |
+Total data size of all indexes in a CSS cluster.
+ |
+≥ 0
+ |
+byte
+ |
+1024
+ |
+cluster_id
+ |
+1 minute
+ |
+
+backup_failed_task_count
+ |
+Number of Failed Backup Tasks
+ |
+Number of failed snapshot backup tasks in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+ism_fail_count
+ |
+Number of Failed Index Lifecycle Tasks
+ |
+Number of failed index lifecycle tasks in a CSS cluster.
+Only Elasticsearch 7.10.2 clusters support this metric.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+disk_total
+ |
+Total Disk Space
+ |
+Total data disk capacity of all nodes in a CSS cluster.
+ |
+≥ 0
+ |
+byte
+ |
+1024
+ |
+cluster_id
+ |
+1 minute
+ |
+
+number_of_unassigned_shards
+ |
+Number of Unallocated Shards
+ |
+Total number of unallocated shards in a CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+disk_iops_qos_num
+ |
+Disk IOPS Overload Times
+ |
+Total number of times the disk IOPS exceeds the predefined threshold.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+disk_iobw_qos_num
+ |
+Disk I/O Bandwidth Overload Times
+ |
+Total number of times the disk I/O bandwidth usage exceeds the predefined threshold.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+max_disk_io_util
+ |
+Maximum Disk I/O Usage
+ |
+Maximum disk I/O usage among all disks.
+ |
+0~100
+ |
+%
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+avg_disk_io_util
+ |
+Average Disk I/O Usage
+ |
+Average disk I/O usage among all disks.
+ |
+0~100
+ |
+%
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+shard_quota_usage
+ |
+Shard Quota Usage
+ |
+Ratio of allocated shards to the upper limit of the shard quantity.
+ |
+0~100
+ |
+%
+ |
+N/A
+ |
+cluster_id
+ |
+1 minute
+ |
+
+
+
+
+
+
Node Monitoring Metrics
+
Table 2 Elasticsearch cluster node monitoring metricsMetric ID
+ |
+Metric
+ |
+Description
+ |
+Value Range
+ |
+Unit
+ |
+Number System
+ |
+Dimension
+ |
+Monitoring Interval (Raw)
+ |
+
|---|
+
+jvm_heap_usage
+ |
+JVM Heap Usage
+ |
+JVM heap memory usage of a node.
+ |
+0~100
+ |
+%
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+cpu_usage
+ |
+CPU Usage
+ |
+CPU usage.
+ |
+0~100
+ |
+%
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+disk_util
+ |
+Max. Disk Usage
+ |
+Disk usage of a node
+ |
+0~100
+ |
+%
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+load_average
+ |
+Average Load
+ |
+Average number of queued tasks per minute on a node
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+open_file_descriptors
+ |
+Open File Descriptors
+ |
+Number of opened file descriptors on a node
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+max_file_descriptors
+ |
+Max. Allowed File Descriptors
+ |
+Maximum number of allowed file descriptors
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_write_queue
+ |
+Tasks in Write Queue
+ |
+Number of tasks in queue for the write thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_search_queue
+ |
+Tasks in Search Queue
+ |
+Number of tasks in queue for the search thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_force_merge_queue
+ |
+Tasks in ForceMerge Queue
+ |
+Number of tasks in queue for the force merge thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_write_rejected
+ |
+Rejected Tasks in Write Queue
+ |
+Number of tasks rejected by the write thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_search_rejected
+ |
+Rejected Tasks in Search Queue
+ |
+Number of tasks rejected by the search thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_force_merge_rejected
+ |
+Rejected Tasks in ForceMerge Queue
+ |
+Number of tasks rejected by the force merge thread pool
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_write_threads
+ |
+Size of Write Thread Pool
+ |
+Size of a write thread pool
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_search_threads
+ |
+Size of Search Thread Pool
+ |
+Size of a search thread pool
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_force_merge_threads
+ |
+Size of ForceMerge Thread Pool
+ |
+Size of a force merge thread pool
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_flush_queue
+ |
+Tasks in Flush Queue
+ |
+Number of tasks in queue for the flush thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_flush_rejected
+ |
+Rejected Tasks in Flush Queue
+ |
+Number of tasks rejected by the flush thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_flush_threads
+ |
+Size of Flush Thread Pool
+ |
+Size of the flush thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_generic_queue
+ |
+Tasks in Generic Queue
+ |
+Number of tasks in queue for the generic thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_generic_rejected
+ |
+Rejected Tasks in Generic Queue
+ |
+Number of tasks rejected by the generic thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_generic_threads
+ |
+Size of Generic Thread Pool
+ |
+Size of the generic thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_management_queue
+ |
+Tasks in Management Queue
+ |
+Number of tasks in queue for the management thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_management_rejected
+ |
+Rejected Tasks in Management Queue
+ |
+Number of tasks rejected by the management thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_management_threads
+ |
+Size of Management Thread Pool
+ |
+Size of the management thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_refresh_queue
+ |
+Tasks in Refresh Queue
+ |
+Number of tasks in queue for the refresh thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_refresh_rejected
+ |
+Rejected Tasks in Refresh Queue
+ |
+Number of tasks rejected by the refresh thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_refresh_threads
+ |
+Size of Refresh Thread Pool
+ |
+Size of the refresh thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_obs_searcher_queue
+ |
+Tasks in OBS Searcher Queue
+ |
+Number of tasks in queue for the OBS Searcher thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_obs_searcher_rejected
+ |
+Rejected Tasks in OBS Searcher Queue
+ |
+Number of tasks rejected by the OBS Searcher thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_obs_searcher_threads
+ |
+Size of OBS Searcher Thread Pool
+ |
+Size of the OBS Searcher thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_obs_queue
+ |
+Tasks in OBS Queue
+ |
+Number of tasks in queue for the OBS thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_obs_rejected
+ |
+Rejected Tasks in OBS Queue
+ |
+Number of tasks rejected by the OBS thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_obs_threads
+ |
+Size of OBS Thread Pool
+ |
+Size of the OBS thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_obs_upload_queue
+ |
+Tasks in OBS Upload Queue
+ |
+Number of tasks in queue for the OBS Upload thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_obs_upload_rejected
+ |
+Rejected Tasks in OBS Upload Queue
+ |
+Number of tasks rejected by the OBS Upload thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_obs_upload_threads
+ |
+Size of OBS Upload Thread Pool
+ |
+Size of the OBS Upload thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_obs_download_queue
+ |
+Tasks in OBS Download Queue
+ |
+Number of tasks in queue for the OBS Download thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_obs_download_rejected
+ |
+Rejected Tasks in OBS Download Queue
+ |
+Number of tasks rejected by the OBS Upload thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+thread_pool_obs_download_threads
+ |
+Size of OBS Download Thread Pool
+ |
+Size of the OBS Download thread pool.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+free_fs_size
+ |
+Available Size of File Systems
+ |
+Available size of file systems in a CSS cluster.
+ |
+≥ 0
+ |
+byte
+ |
+1024
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+total_fs_size
+ |
+Total Size of File Systems
+ |
+Total size of file systems in a CSS cluster.
+ |
+≥ 0
+ |
+byte
+ |
+1024
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+jvm_old_gc_count
+ |
+Total GCs of Old-Generation JVM
+ |
+Number of old-generation garbage collection times
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+jvm_old_gc_time
+ |
+Total GC Duration of Old-Generation JVM
+ |
+Time spent on old-generation garbage collection
+ |
+≥ 0
+ |
+ms
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+jvm_young_gc_count
+ |
+Total GCs of Young-Generation JVM
+ |
+Number of young-generation garbage collection times
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+jvm_young_gc_time
+ |
+GC Duration of Young-Generation JVM
+ |
+Time spent on young-generation garbage collection
+ |
+≥ 0
+ |
+ms
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+mem_free_in_bytes
+ |
+Available Memory
+ |
+Unused memory space of a node.
+ |
+≥ 0
+ |
+byte
+ |
+1024
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+mem_free_percent
+ |
+Available Memory Percentage
+ |
+Percentage of unused memory space on a node.
+ |
+0~100
+ |
+%
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+mem_used_in_bytes
+ |
+Used Memory
+ |
+Used memory space of a node.
+ |
+≥ 0
+ |
+byte
+ |
+1024
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+current_opened_http_count
+ |
+Currently Open HTTP Connections
+ |
+Number of HTTP connections currently open on a node
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+total_opened_http_count
+ |
+Total Open HTTP Connections
+ |
+Total number of HTTP connections on a node
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+disk_read_requests_rate
+ |
+Disk Read IOPS
+ |
+Number of disk read requests to a node in the CSS cluster per second.
+ |
+≥ 0
+ |
+request/s
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+disk_write_requests_rate
+ |
+Disk Write IOPS
+ |
+Number of disk write requests to a node in the CSS cluster per second.
+ |
+≥ 0
+ |
+request/s
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+disk_read_bytes_rate
+ |
+Disk Read Rate
+ |
+Number of bytes of data read from the disks of a node in the CSS cluster per second.
+ |
+≥ 0
+ |
+Byte/s
+ |
+1024
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+disk_write_bytes_rate
+ |
+Disk Write Rate
+ |
+Number of bytes of data written into the disks of a node in the CSS cluster per second.
+ |
+≥ 0
+ |
+Byte/s
+ |
+1024
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+shards_count
+ |
+Shards
+ |
+Number of shards per node in the CSS cluster.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+vector_index_off_heap_used_in_bytes
+ |
+Off-Heap Memory Usage of Vector Indexes
+ |
+Off-heap memory (bytes) used for querying vector indexes on nodes in a CSS cluster.
+ |
+≥ 0
+ |
+byte
+ |
+1024
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+vector_index_off_heap_usage
+ |
+Off-Heap Memory Utilization (%) of Vector Indexes
+ |
+Off-heap memory (in %) used for querying vector indexes on nodes in a CSS cluster.
+ |
+0~100
+ |
+%
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+disk_iops_qos_num
+ |
+Disk IOPS Overload Times
+ |
+Total number of times the disk IOPS exceeds the predefined threshold.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+disk_iobw_qos_num
+ |
+Disk I/O Bandwidth Overload Times
+ |
+Total number of times the disk I/O bandwidth usage exceeds the predefined threshold.
+ |
+≥ 0
+ |
+Count
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+max_disk_io_util
+ |
+Maximum Disk I/O Usage
+ |
+Maximum disk I/O usage among all disks.
+ |
+0~100
+ |
+%
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+avg_disk_io_util
+ |
+Average Disk I/O Usage
+ |
+Average disk I/O usage among all disks.
+ |
+0~100
+ |
+%
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+shard_quota_usage
+ |
+Shard Quota Usage
+ |
+Ratio of allocated shards to the upper limit of the shard quantity.
+ |
+0~100
+ |
+%
+ |
+N/A
+ |
+cluster_id,instance_id
+ |
+1 minute
+ |
+
+
+
+
+
If an object is in a hierarchical system, specify the monitored dimension in hierarchical form when you use an API to query the metrics of this object.
+
For example, to query the CPU usage (cpu_usage) of an Elasticsearch cluster node in CSS, the dimension information of this metric is cluster_id,instance_id, where cluster_id indicates level 0 and instance_id level 1.
+
- To query a single metric by calling an API, the instance_id dimension is used as follows:
dim.0=cluster_id,3d65c1ac-9a9f-4c5f-a054-35184a087bb2&dim.1=instance_id,6666cd76f96956469e7be39d750cc7d9
+where, 3d65c1ac-9a9f-4c5f-a054-35184a087bb2 and 6666cd76f96956469e7be39d750cc7d9 are the values of cluster_id and instance_id, respectively. For how to obtain these values, see Dimensions.
+ - To query multiple metrics by calling an API, the instance_id dimension is used as follows:
"dimensions": [
+ {
+ "name": "cluster_id",
+ "value": "3d65c1ac-9a9f-4c5f-a054-35184a087bb2"
+ },
+ {
+ "name": "instance_id",
+ "value": "6666cd76f96956469e7be39d750cc7d9"
+ }
+]
+where, 3d65c1ac-9a9f-4c5f-a054-35184a087bb2 and 6666cd76f96956469e7be39d750cc7d9 are the values of cluster_id and instance_id, respectively. For how to obtain the values, see Dimensions.
+
+
+
Dimensions
+
Table 3 Dimension descriptionKey
+ |
+Value
+ |
+
|---|
+
+cluster_id
+ |
+Cluster ID.
+You can obtain this value from the clusters[].id field in the response body of the Querying the Cluster List API.
+ |
+
+cluster_id,instance_id
+ |
+Cluster node ID.
+You can obtain this value from the clusters[].instances[].id field in the response body of the Querying the Cluster List API.
+ |
+
+
+
+
+
+
+
+
diff --git a/docs/css/umn/css_01_0043.html b/docs/css/umn/css_01_0043.html
new file mode 100644
index 000000000..c8dd8c83b
--- /dev/null
+++ b/docs/css/umn/css_01_0043.html
@@ -0,0 +1,435 @@
+
+
+
Accessing an OpenSearch Cluster with the Python Client
+
You can access and manage data in a CSS OpenSearch cluster using the official OpenSearch Python client (opensearch-py) or Elasticsearch Python client (elasticsearch-py). Both client libraries are compatible with OpenSearch REST APIs, though version compatibility needs to be verified for elasticsearch-py. Developers can use Python APIs to manage indexes; create, read, update, and delete (CRUD) documents; and search data.
+
You are advised to use the OpenSearch Python client (opensearch-py) to access complete OpenSearch features. For details, see OpenSearch Python Client Documentation or Elasticsearch Python Client Documentation.
+
Prerequisites
- The target OpenSearch cluster is available.
- The server that runs the Python code can communicate with the OpenSearch cluster.
- Depending on the network configuration method used, obtain the cluster access address. For details, see Network Configuration.
- Python 3 has been installed on the server.
+
+
Installing the Python Client Library
Install the Python client library on the server that runs the Python code.
+
CSS allows you to use the Elasticsearch 7.10 Python client to connect to OpenSearch clusters. To ensure better compatibility, however, you are advised to use a Python client library that has the same version as the target OpenSearch cluster.
+
Select a reference example based on the Python client you use.
+
+
+
Cluster Access Scenarios
The sample code varies depending on the Python client and the security mode settings of the target OpenSearch cluster. Select the right reference document based on your service scenario.
+
+
+
+
Connecting to a Non-Security Mode Cluster with the OpenSearch Python Client
Use the OpenSearch Python client to connect to an OpenSearch cluster for which the security mode is disabled, and query whether the test index exists.
+
The following is an example of the code for creating a client instance using the cluster connection information:
+
1
+ 2
+ 3
+ 4
+ 5
+ 6
+ 7
+ 8
+ 9
+10
+11
+12
+13
+14
+15
+16
+17
+18
+19
+20
+21
+22
+23 | from opensearchpy import OpenSearch
+
+class OpenSearchFactory(object):
+
+ def __init__(self, host: list, port: str, username: str, password: str):
+ self.port = port
+ self.host = host
+ self.username = username
+ self.password = password
+
+ def create(self) -> OpenSearch:
+ addrs = []
+ for host in self.host:
+ addr = {'host': host, 'port': self.port}
+ addrs.append(addr)
+
+ if self.username and self.password:
+ opensearch = OpenSearch(addrs, http_auth=(self.username, self.password))
+ else:
+ opensearch = OpenSearch(addrs)
+ return opensearch
+os = OpenSearchFactory(["{cluster address}"], "9200", None, None).create()
+print(os.indices.exists(index='test'))
+ |
+
+
This piece of code checks whether the test index exists in the cluster. If true (the index exists) or false (the index does not exist) is returned, it indicates that the cluster is connected.
+
+
Connecting to a Security-Mode+HTTP Cluster with the OpenSearch Python Client
Use the OpenSearch Python client to connect to a security-mode OpenSearch cluster that uses HTTP, and query whether the test index exists.
+
The following is an example of the code for creating a client instance using the cluster connection information:
+
1
+ 2
+ 3
+ 4
+ 5
+ 6
+ 7
+ 8
+ 9
+10
+11
+12
+13
+14
+15
+16
+17
+18
+19
+20
+21
+22
+23 | from opensearchpy import OpenSearch
+
+class OpenSearchFactory(object):
+
+ def __init__(self, host: list, port: str, username: str, password: str):
+ self.port = port
+ self.host = host
+ self.username = username
+ self.password = password
+
+ def create(self) -> OpenSearch:
+ addrs = []
+ for host in self.host:
+ addr = {'host': host, 'port': self.port}
+ addrs.append(addr)
+
+ if self.username and self.password:
+ opensearch = OpenSearch(addrs, http_auth=(self.username, self.password))
+ else:
+ opensearch = OpenSearch(addrs)
+ return opensearch
+os = OpenSearchFactory(["xxx.xxx.xxx.xxx"], "9200", "username", "password").create()
+print(os.indices.exists(index='test'))
+ |
+
+
+
Table 2 VariablesParameter
+ |
+Description
+ |
+
|---|
+
+host
+ |
+IP address for accessing the cluster. If there are multiple IP addresses, separate them with commas (,).
+ |
+
+port
+ |
+Access port of the cluster. Enter 9200.
+ |
+
+username
+ |
+Username for accessing the cluster.
+ |
+
+password
+ |
+Password of the user.
+ |
+
+
+
+
+
This piece of code checks whether the test index exists in the cluster. If true (the index exists) or false (the index does not exist) is returned, it indicates that the cluster is connected.
+
+
Connecting to a Security-Mode+HTTPS Cluster with the OpenSearch Python Client
Use the OpenSearch Python client to connect to a security-mode OpenSearch cluster that uses HTTPS, and query whether the test index exists.
+
The following is an example of the code for creating a client instance using the cluster connection information:
+
1
+ 2
+ 3
+ 4
+ 5
+ 6
+ 7
+ 8
+ 9
+10
+11
+12
+13
+14
+15
+16
+17
+18
+19
+20
+21
+22
+23
+24
+25
+26 | from opensearchpy import OpenSearch
+import ssl
+
+class OpenSearchFactory(object):
+
+ def __init__(self, host: list, port: str, username: str, password: str):
+ self.port = port
+ self.host = host
+ self.username = username
+ self.password = password
+
+ def create(self) -> OpenSearch:
+ context = ssl._create_unverified_context()
+
+ addrs = []
+ for host in self.host:
+ addr = {'host': host, 'port': self.port}
+ addrs.append(addr)
+
+ if self.username and self.password:
+ opensearch = OpenSearch(addrs, http_auth=(self.username, self.password), scheme="https", ssl_context=context)
+ else:
+ opensearch = OpenSearch(addrs)
+ return opensearch
+os = OpenSearchFactory(["xxx.xxx.xxx.xxx"], "9200", "username", "password").create()
+print(os.indices.exists(index='test'))
+ |
+
+
+
Table 3 VariablesParameter
+ |
+Description
+ |
+
|---|
+
+host
+ |
+IP address for accessing the cluster. If there are multiple IP addresses, separate them with commas (,).
+ |
+
+port
+ |
+Access port of the cluster. Enter 9200.
+ |
+
+username
+ |
+Username for accessing the cluster.
+ |
+
+password
+ |
+Password of the user.
+ |
+
+
+
+
+
This piece of code checks whether the test index exists in the cluster. If true (the index exists) or false (the index does not exist) is returned, it indicates that the cluster is connected.
+
+
Connecting to a Non-Security Mode Cluster with the Elasticsearch Python Client
Use the Elasticsearch 7.10.2 Python client to connect to an OpenSearch cluster for which the security mode is disabled, and query whether the test index exists.
+
The following is an example of the code for creating a client instance using the cluster connection information:
+
from elasticsearch import Elasticsearch
+
+class ElasticFactory(object):
+
+ def __init__(self, host: list, port: str, username: str, password: str):
+ self.port = port
+ self.host = host
+ self.username = username
+ self.password = password
+
+ def create(self) -> Elasticsearch:
+ addrs = []
+ for host in self.host:
+ addr = {'host': host, 'port': self.port}
+ addrs.append(addr)
+
+ if self.username and self.password:
+ elasticsearch = Elasticsearch(addrs, http_auth=(self.username, self.password))
+ else:
+ elasticsearch = Elasticsearch(addrs)
+ return elasticsearch
+
+es = ElasticFactory(["{cluster address}"], "9200", None, None).create()
+print(es.indices.exists(index='test'))
+
This piece of code checks whether the test index exists in the cluster. If true (the index exists) or false (the index does not exist) is returned, it indicates that the cluster is connected.
+
+
Connecting to a Security-Mode+HTTP Cluster with the Elasticsearch Python Client
Use the Elasticsearch 7.10.2 Python client to connect to a security-mode OpenSearch cluster that uses HTTP, and query whether the test index exists.
+
The following is an example of the code for creating a client instance using the cluster connection information:
+
from elasticsearch import Elasticsearch
+
+class ElasticFactory(object):
+
+ def __init__(self, host: list, port: str, username: str, password: str):
+ self.port = port
+ self.host = host
+ self.username = username
+ self.password = password
+
+ def create(self) -> Elasticsearch:
+ addrs = []
+ for host in self.host:
+ addr = {'host': host, 'port': self.port}
+ addrs.append(addr)
+
+ if self.username and self.password:
+ elasticsearch = Elasticsearch(addrs, http_auth=(self.username, self.password))
+ else:
+ elasticsearch = Elasticsearch(addrs)
+ return elasticsearch
+
+es = ElasticFactory(["xxx.xxx.xxx.xxx"], "9200", "username", "password").create()
+print(es.indices.exists(index='test'))
+
+
Table 4 VariablesParameter
+ |
+Description
+ |
+
|---|
+
+host
+ |
+IP address for accessing the cluster. If there are multiple IP addresses, separate them with commas (,).
+ |
+
+port
+ |
+Access port of the cluster. Enter 9200.
+ |
+
+username
+ |
+Username for accessing the cluster.
+ |
+
+password
+ |
+Password of the user.
+ |
+
+
+
+
+
This piece of code checks whether the test index exists in the cluster. If true (the index exists) or false (the index does not exist) is returned, it indicates that the cluster is connected.
+
+
Connecting to a Security-Mode+HTTPS Cluster with the Elasticsearch Python Client
Use the Elasticsearch 7.10.2 Python client to connect to a security-mode OpenSearch cluster that uses HTTPS, and query whether the test index exists.
+
The following is an example of the code for creating a client instance using the cluster connection information:
+
from elasticsearch import Elasticsearch
+import ssl
+
+class ElasticFactory(object):
+
+ def __init__(self, host: list, port: str, username: str, password: str):
+ self.port = port
+ self.host = host
+ self.username = username
+ self.password = password
+
+ def create(self) -> Elasticsearch:
+ context = ssl._create_unverified_context()
+
+ addrs = []
+ for host in self.host:
+ addr = {'host': host, 'port': self.port}
+ addrs.append(addr)
+
+ if self.username and self.password:
+ elasticsearch = Elasticsearch(addrs, http_auth=(self.username, self.password), scheme="https", ssl_context=context)
+ else:
+ elasticsearch = Elasticsearch(addrs)
+ return elasticsearch
+
+es = ElasticFactory(["xxx.xxx.xxx.xxx"], "9200", "username", "password").create()
+print(es.indices.exists(index='test'))
+
+
Table 5 VariablesParameter
+ |
+Description
+ |
+
|---|
+
+host
+ |
+IP address for accessing the cluster. If there are multiple IP addresses, separate them with commas (,).
+ |
+
+port
+ |
+Access port of the cluster. Enter 9200.
+ |
+
+username
+ |
+Username for accessing the cluster.
+ |
+
+password
+ |
+Password of the user.
+ |
+
+
+
+
+
This piece of code checks whether the test index exists in the cluster. If true (the index exists) or false (the index does not exist) is returned, it indicates that the cluster is connected.
+
+
+
+
diff --git a/docs/css/umn/css_01_0045.html b/docs/css/umn/css_01_0045.html
index 39db4b093..6ec3f9bd7 100644
--- a/docs/css/umn/css_01_0045.html
+++ b/docs/css/umn/css_01_0045.html
@@ -1,23 +1,337 @@
-
+
-
Importing Data to an Elasticsearch Cluster
-
+
Configuring Index Monitoring for an OpenSearch Cluster
+
Index monitoring helps you monitor the index status, statistics, and trends of a cluster, allowing you to promptly handle potential risks to ensure cluster reliability. By collecting index statistics, you can gain insights into various aspects of index performance and resource utilization, and make informed O&M decisions.
+
How the Feature Works
Index monitoring consists of two steps:
+
- Data ingestion
- Index stats (such as the number of documents, data storage, and shard status) are ingested periodically.
- Data is written into the monitoring-eye-css-* index, where a data aging mechanism is applied.
- The default data retention duration is seven days. Data is automatically deleted when this duration expires.
+ - Monitoring chart generation
- Preset dashboards and custom visualizations on OpenSearch Dashboards enable intuitive index monitoring.
- Time series analysis, trend analysis and comparison, and anomaly detection are supported.
+
+
+
Constraints
- Only OpenSearch 2.19.0 supports index monitoring.
+
- Indexes starting with monitoring-eye-css-* are identified as monitoring indexes and will not be monitored. Do not use this prefix for regular indexes.
- The index pattern of monitoring-eye-css-* must not be deleted while index monitoring is enabled. Otherwise, monitoring charts will become abnormal.
+
+
Logging In to OpenSearch Dashboards
Log in to OpenSearch Dashboards and go to the command execution page. OpenSearch clusters support multiple access methods. This topic uses OpenSearch Dashboards as an example to describe the operation procedures.
+
- Log in to the CSS management console.
- In the navigation pane on the left, choose Clusters > OpenSearch.
- In the cluster list, find the target cluster, and click Dashboards in the Operation column to log in to OpenSearch Dashboards.
- In the left navigation pane, choose Dev Tools.
The left part of the console is the command input box, and the triangle icon in its upper-right corner is the execution button. The right part shows the execution result.
+
+
+
Enabling Index Monitoring
- Run the following command to enable index monitoring:
PUT _cluster/settings
+{
+ "persistent": {
+ "css.monitoring.index.enabled": "true"
+ }
+}
+ - To monitor a specified index, run the following command to set the monitoring period, target index name, and monitoring index retention duration.
PUT _cluster/settings
+{
+ "persistent": {
+ "css.monitoring.index.enabled": "true",
+ "css.monitoring.index.interval": "30s",
+ "css.monitoring.index.indices": ["index_name"],
+ "css.monitoring.history.duration": "3d"
+ }
+}
+
+Table 1 Parameters for index monitoringParameter
+ |
+Type
+ |
+Description
+ |
+
|---|
+
+css.monitoring.index.enabled
+ |
+Boolean
+ |
+Whether to enable index monitoring. Setting this parameter to true enables index monitoring.
+Default value: false
+ |
+
+css.monitoring.index.interval
+ |
+Time
+ |
+Index monitoring period.
+Minimum value: 1s
+Default value: 10s
+ |
+
+css.monitoring.index.indices
+ |
+String
+ |
+Name of an index to be monitored. By default, all indexes are monitored. You can configure specific indexes or a specific type of indexes to monitor.
+Example:
+- "css.monitoring.index.indices": ["index_name"] indicates only index_name is monitored.
- "css.monitoring.index.indices": ["log_*"] indicates that only indexes starting with log_ are monitored.
- "css.monitoring.index.indices": ["index1", "index2"] indicates that index1 and index2 are monitored.
+Default value: * (indicating that all indexes are monitored)
+ |
+
+css.monitoring.history.duration
+ |
+Time
+ |
+Retention period of the monitoring index monitoring-eye-css-*. The default period is one week.
+Minimum value: 1d
+Default value: 7d
+ |
+
+
+
+
+
+
Checking the Read and Write Traffic of Indexes
After index monitoring is enabled for a cluster, you can check the read/write traffic of this cluster's indexes during specific periods of time.
+
- Run the following command to check the read and write traffic of all indexes:
GET /_cat/monitoring
+ - Run the following command to check the read and write traffic of a specified index:
GET /_cat/monitoring/{indexname}
+{indexName} indicates the name of the index whose read and write traffic you want to check.
+ - Run the following command to check the read and write traffic of an index during specified periods of time:
GET _cat/monitoring?begin=1650099461000
+GET _cat/monitoring?begin=2022-04-16T08:57:41
+GET _cat/monitoring?begin=2022-04-16T08:57:41&end=2022-04-17T08:57:41
+
+
+
Table 2 Parameters for checking read and write trafficParameter
+ |
+Type
+ |
+Description
+ |
+
|---|
+
+indexname
+ |
+String
+ |
+Index name.
+You cannot check the traffic of system indexes, whose names start with a period (.).
+ |
+
+begin
+ |
+Date
+ |
+Start time (UTC time) of the monitoring data you want to view.
+Time format: strict_date_optional_time|epoch_millis
+The default start time is 5 minutes before the current time.
+ |
+
+end
+ |
+Date
+ |
+End time (UTC time) of the monitoring data you want to view.
+Time format: strict_date_optional_time|epoch_millis
+The default end time is the current time.
+ |
+
+
+
+
+
Information similar to the following is displayed:
+
index begin end status pri rep init unassign docs.count docs.deleted store.size pri.store.size delete.rate indexing.rate search.rate
+test 2022-03-25T09:46:53.765Z 2022-03-25T09:51:43.767Z yellow 1 1 0 1 9 0 5.9kb 5.9kb 0/s 0/s 0/s
+
+
Table 3 Parameters in the returned informationParameter
+ |
+Description
+ |
+
|---|
+
+index
+ |
+Index name
+ |
+
+begin
+ |
+Start time of the monitoring data you queried.
+ |
+
+end
+ |
+End time of the monitoring data you queried.
+ |
+
+status
+ |
+Index status within the queried monitoring interval.
+ |
+
+pri
+ |
+The number of index shards within the queried monitoring interval.
+ |
+
+rep
+ |
+The number of index replicas within the queried monitoring interval.
+ |
+
+init
+ |
+The number of initialized indexes within the queried monitoring interval.
+ |
+
+unassign
+ |
+The number of unallocated indexes within the queried monitoring interval.
+ |
+
+docs.count
+ |
+The number of documents within the queried monitoring interval.
+ |
+
+docs.deleted
+ |
+The number of deleted documents within the queried monitoring interval.
+ |
+
+store.size
+ |
+Index storage size within the queried monitoring interval.
+ |
+
+pri.store.size
+ |
+Size of the primary index shard within the queried monitoring interval.
+ |
+
+delete.rate
+ |
+Number of indexes deleted per second within the queried monitoring interval.
+ |
+
+indexing.rate
+ |
+Number of indexes wrote per second within the queried monitoring interval.
+ |
+
+search.rate
+ |
+Number of indexes queried per second within the queried monitoring interval.
+ |
+
+
+
+
+
+
Checking Index Monitoring Data in OpenSearch Dashboards Charts
You can check preconfigured index monitoring charts on the Dashboard and Visualizations pages of OpenSearch Dashboards. You can also customize tables and charts.
+
- Check index monitoring results in preconfigured dashboards.
- On the OpenSearch Dashboards console, expand the menu in the upper-left corner, and choose Dashboards.
- Click [Monitoring] Index monitoring Dashboard to check preconfigured dashboards.
Figure 1 Preconfigured dashboards
+The preconfigured dashboard displays the number of read and write operations per second in the cluster and the top 10 indexes that have the most read and write operations per second.
+
+Table 4 Preconfigured chartsChart Name
+ |
+Description
+ |
+
|---|
+
+[monitoring] markdown
+ |
+Markdown chart, which briefly describes the dashboard content.
+ |
+
+[monitoring] Indexing Rate (/s)
+ |
+Number of documents written to a cluster per second.
+ |
+
+[monitoring] Search Rate (/s)
+ |
+Average number of queries per second in a cluster.
+ |
+
+[monitoring] indexing rate of index for top10
+ |
+Top 10 indexes with the most documents written per second.
+ |
+
+[monitoring] search rate of index for top10
+ |
+Top 10 indexes with the most queries per second.
+ |
+
+[monitoring] total docs count
+ |
+Total number of documents in a cluster.
+ |
+
+[monitoring] total docs delete
+ |
+Total number of deleted documents in a cluster.
+ |
+
+[monitoring] total store size in bytes
+ |
+Total storage space occupied by documents in a cluster.
+ |
+
+[monitoring] indices store_size for top10
+ |
+Top 10 indexes that occupy the largest storage space.
+ |
+
+[monitoring] indices docs_count for top10
+ |
+Top 10 indexes that store the largest number of documents.
+ |
+
+[monitoring] indexing time in millis of index for top10(ms)
+ |
+Top 10 indexes with the longest document write latency in a unit time (ms).
+ |
+
+[monitoring] search query time in millis of index for top10(ms)
+ |
+Top 10 indexes with the longest index query time in a unit time (ms).
+ |
+
+[monitoring] segment count of index for top10
+ |
+Top 10 indexes with the largest number of index segments.
+ |
+
+[monitoring] segment memory in bytes of index for top10
+ |
+Top 10 indexes with the largest heap memory usage of index segments.
+ |
+
+
+
+
+ - Check index monitoring results in custom visualizations.
The index monitoring module periodically stores the index/stats information in the monitoring-eys-css index. You can create custom charts on OpenSearch Dashboards to visualize the information.
+The following procedure describes how to check the trend of document quantities in a chart as an example.
+- Click Visualize in the navigation tree on the left of the OpenSearch Dashboards console.
- Click Create visualization and select TSVB.
- Set chart parameters and view the visualizations.
On the Data tab page, set the parameters as needed.
+- Select Max for Aggregation, and select index_stats.primaries.docs.count in Field, indicating the number of documents in a primary shard.
- Select Derivative from Aggregation to indicate differences between aggregation buckets. Set Units to 1s to visualize network rates as "per second".
- Set Aggregation to Positive Only to prevent negative numbers after resetting.
- To show statistics by index, set Group by to Terms and By to index_stats.index. Statistics will be grouped by index name.
- To view data in different time segments, set the aggregation interval, or the displayed data will be incomplete. On the Panel options tab page, set Interval to 1m or 30m to adjust the interval of timestamp.
+Figure 2 TSVB page
+Figure 3 Setting the interval
+
+
+
If the index monitoring charts are not displayed, re-load them on OpenSearch Dashboards.
If the preset dashboards and visualizations cannot be found for a security-mode OpenSearch cluster, try switching to the Private or Global space. If the issue persists, import the charts and graphs again.
+
+
- Create the monitoring-dashboards.ndjson file by referring to dashboards-monitor Configuration File.
- On OpenSearch Dashboards, choose Management > Dashboards Management > Saved objects.
- Click Import and upload the monitoring-dashboards.ndjson file created in 1.
- After the file is uploaded, click done. The index monitoring charts are imported successfully.
Figure 4 Index monitoring charts imported successfully
+
+
+
+
dashboards-monitor Configuration File
The content of dashboards-monitor configuration file is as follows. You are advised to save the file as monitoring-dashboards.ndjson.
+
{"attributes":{"description":"","kibanaSavedObjectMeta":{"searchSourceJSON":"{}"},"title":"[monitoring] segment memory in bytes of index for top10","uiStateJSON":"{}","version":1,"visState":"{\"title\":\"[monitoring] segment memory in bytes of index for top10\",\"type\":\"metrics\",\"aggs\":[],\"params\":{\"id\":\"61ca57f0-469d-11e7-af02-69e470af7417\",\"type\":\"timeseries\",\"series\":[{\"id\":\"61ca57f1-469d-11e7-af02-69e470af7417\",\"color\":\"#68BC00\",\"split_mode\":\"terms\",\"split_color_mode\":\"kibana\",\"metrics\":[{\"id\":\"61ca57f2-469d-11e7-af02-69e470af7417\",\"type\":\"max\",\"field\":\"index_stats.total.segments.memory_in_bytes\"}],\"separate_axis\":0,\"axis_position\":\"right\",\"formatter\":\"bytes\",\"chart_type\":\"line\",\"line_width\":1,\"point_size\":1,\"fill\":0.5,\"stacked\":\"none\",\"label\":\"segments memory in bytes \",\"type\":\"timeseries\",\"terms_field\":\"index_stats.index\",\"terms_order_by\":\"61ca57f2-469d-11e7-af02-69e470af7417\"}],\"time_field\":\"timestamp\",\"index_pattern\":\"monitoring-eye-css-*\",\"interval\":\"\",\"axis_position\":\"left\",\"axis_formatter\":\"number\",\"axis_scale\":\"normal\",\"show_legend\":1,\"show_grid\":1,\"tooltip_mode\":\"show_all\",\"default_index_pattern\":\"monitoring-eye-css-*\",\"default_timefield\":\"timestamp\",\"isModelInvalid\":false}}"},"id":"3ae5d820-6628-11ed-8cd7-973626cf6f70","references":[],"type":"visualization","updated_at":"2022-12-01T12:41:01.165Z","version":"WzIwNiwyXQ=="}
+{"attributes":{"description":"","kibanaSavedObjectMeta":{"searchSourceJSON":"{}"},"title":"[monitoring] segment count of index for top10","uiStateJSON":"{}","version":1,"visState":"{\"aggs\":[],\"params\":{\"axis_formatter\":\"number\",\"axis_position\":\"left\",\"axis_scale\":\"normal\",\"default_index_pattern\":\"monitoring-eye-css-*\",\"default_timefield\":\"timestamp\",\"filter\":{\"language\":\"kuery\",\"query\":\"\"},\"id\":\"61ca57f0-469d-11e7-af02-69e470af7417\",\"index_pattern\":\"monitoring-eye-css-*\",\"interval\":\"\",\"isModelInvalid\":false,\"series\":[{\"axis_position\":\"right\",\"chart_type\":\"line\",\"color\":\"rgba(231,102,76,1)\",\"fill\":0.5,\"formatter\":\"number\",\"id\":\"61ca57f1-469d-11e7-af02-69e470af7417\",\"label\":\"segment count of index for top10\",\"line_width\":1,\"metrics\":[{\"field\":\"index_stats.total.segments.count\",\"id\":\"61ca57f2-469d-11e7-af02-69e470af7417\",\"type\":\"max\"}],\"point_size\":1,\"separate_axis\":0,\"split_color_mode\":\"kibana\",\"split_mode\":\"terms\",\"stacked\":\"none\",\"terms_field\":\"index_stats.index\",\"terms_order_by\":\"61ca57f2-469d-11e7-af02-69e470af7417\",\"type\":\"timeseries\"}],\"show_grid\":1,\"show_legend\":1,\"time_field\":\"timestamp\",\"tooltip_mode\":\"show_all\",\"type\":\"timeseries\"},\"title\":\"[monitoring] segment count of index for top10\",\"type\":\"metrics\"}"},"id":"45d571c0-6626-11ed-8cd7-973626cf6f70","references":[],"type":"visualization","updated_at":"2022-12-01T12:41:01.165Z","version":"WzIwNywyXQ=="}
+{"attributes":{"description":"","kibanaSavedObjectMeta":{"searchSourceJSON":"{}"},"title":"[monitoring] markdown","uiStateJSON":"{}","version":1,"visState":"{\"title\":\"[monitoring] markdown\",\"type\":\"markdown\",\"params\":{\"fontSize\":12,\"openLinksInNewTab\":false,\"markdown\":\"### Index Monitoring \\nThis dashboard contains default table for you to play with. You can view it, search it, and interact with the visualizations.\"},\"aggs\":[]}"},"id":"b2811c70-a5f1-11ec-9a68-ada9d754c566","references":[],"type":"visualization","updated_at":"2022-12-01T12:41:01.165Z","version":"WzIwOCwyXQ=="}
+{"attributes":{"description":"number of document being indexing for primary and replica shards","kibanaSavedObjectMeta":{"searchSourceJSON":"{}"},"title":"[monitoring] Indexing Rate (/s)","uiStateJSON":"{}","version":1,"visState":"{\"title\":\"[monitoring] Indexing Rate (/s)\",\"type\":\"metrics\",\"params\":{\"id\":\"61ca57f0-469d-11e7-af02-69e470af7417\",\"type\":\"timeseries\",\"series\":[{\"id\":\"61ca57f1-469d-11e7-af02-69e470af7417\",\"color\":\"rgba(0,32,188,1)\",\"split_mode\":\"everything\",\"metrics\":[{\"id\":\"61ca57f2-469d-11e7-af02-69e470af7417\",\"type\":\"max\",\"field\":\"indices_stats._all.total.indexing.index_total\"},{\"unit\":\"1s\",\"id\":\"fed72db0-a5f8-11ec-aa10-992297d21a2e\",\"type\":\"derivative\",\"field\":\"61ca57f2-469d-11e7-af02-69e470af7417\"},{\"unit\":\"\",\"id\":\"14b66420-a5f9-11ec-aa10-992297d21a2e\",\"type\":\"positive_only\",\"field\":\"fed72db0-a5f8-11ec-aa10-992297d21a2e\"}],\"separate_axis\":0,\"axis_position\":\"right\",\"formatter\":\"number\",\"chart_type\":\"line\",\"line_width\":1,\"point_size\":1,\"fill\":0.5,\"stacked\":\"none\",\"label\":\"Indexing Rate (/s)\",\"type\":\"timeseries\",\"split_color_mode\":\"rainbow\",\"hidden\":false}],\"time_field\":\"timestamp\",\"index_pattern\":\"monitoring-eye-css-*\",\"interval\":\"\",\"axis_position\":\"left\",\"axis_formatter\":\"number\",\"axis_scale\":\"normal\",\"show_legend\":1,\"show_grid\":1,\"default_index_pattern\":\"monitoring-eye-css-*\",\"default_timefield\":\"timestamp\",\"isModelInvalid\":false,\"legend_position\":\"bottom\"},\"aggs\":[]}"},"id":"de4f8ab0-a5f8-11ec-9a68-ada9d754c566","references":[],"type":"visualization","updated_at":"2022-12-01T12:41:01.165Z","version":"WzIwOSwyXQ=="}
+{"attributes":{"description":"number of search request being executed in primary and replica shards","kibanaSavedObjectMeta":{"searchSourceJSON":"{}"},"title":"[monitoring] Search Rate (/s)","uiStateJSON":"{}","version":1,"visState":"{\"title\":\"[monitoring] Search Rate (/s)\",\"type\":\"metrics\",\"params\":{\"id\":\"61ca57f0-469d-11e7-af02-69e470af7417\",\"type\":\"timeseries\",\"series\":[{\"id\":\"61ca57f1-469d-11e7-af02-69e470af7417\",\"color\":\"rgba(0,33,224,1)\",\"split_mode\":\"everything\",\"metrics\":[{\"id\":\"61ca57f2-469d-11e7-af02-69e470af7417\",\"type\":\"max\",\"field\":\"indices_stats._all.total.search.query_total\"},{\"unit\":\"1s\",\"id\":\"b1093ac0-a5f7-11ec-aa10-992297d21a2e\",\"type\":\"derivative\",\"field\":\"61ca57f2-469d-11e7-af02-69e470af7417\"},{\"unit\":\"\",\"id\":\"c17db930-a5f7-11ec-aa10-992297d21a2e\",\"type\":\"positive_only\",\"field\":\"b1093ac0-a5f7-11ec-aa10-992297d21a2e\"}],\"separate_axis\":0,\"axis_position\":\"right\",\"formatter\":\"number\",\"chart_type\":\"line\",\"line_width\":1,\"point_size\":1,\"fill\":0.5,\"stacked\":\"none\",\"split_color_mode\":\"rainbow\",\"label\":\"Search Rate (/s)\",\"type\":\"timeseries\",\"filter\":{\"query\":\"\",\"language\":\"kuery\"}}],\"time_field\":\"timestamp\",\"index_pattern\":\"monitoring-eye-css-*\",\"interval\":\"\",\"axis_position\":\"left\",\"axis_formatter\":\"number\",\"axis_scale\":\"normal\",\"show_legend\":1,\"show_grid\":1,\"default_index_pattern\":\"monitoring-eye-css-*\",\"default_timefield\":\"timestamp\",\"isModelInvalid\":false,\"legend_position\":\"bottom\"},\"aggs\":[]}"},"id":"811df7a0-a5f8-11ec-9a68-ada9d754c566","references":[],"type":"visualization","updated_at":"2022-12-01T12:41:01.165Z","version":"WzIxMCwyXQ=="}
+{"attributes":{"description":"","kibanaSavedObjectMeta":{"searchSourceJSON":"{}"},"title":"[monitoring] total docs count","uiStateJSON":"{}","version":1,"visState":"{\"title\":\"[monitoring] total docs count\",\"type\":\"metrics\",\"aggs\":[],\"params\":{\"id\":\"61ca57f0-469d-11e7-af02-69e470af7417\",\"type\":\"timeseries\",\"series\":[{\"id\":\"61ca57f1-469d-11e7-af02-69e470af7417\",\"color\":\"rgba(218,139,69,1)\",\"split_mode\":\"everything\",\"split_color_mode\":\"kibana\",\"metrics\":[{\"unit\":\"\",\"id\":\"61ca57f2-469d-11e7-af02-69e470af7417\",\"type\":\"max\",\"field\":\"indices_stats._all.total.docs.count\"}],\"separate_axis\":0,\"axis_position\":\"right\",\"formatter\":\"number\",\"chart_type\":\"line\",\"line_width\":1,\"point_size\":1,\"fill\":0.5,\"stacked\":\"none\",\"label\":\"total_docs_count\",\"type\":\"timeseries\"}],\"time_field\":\"timestamp\",\"index_pattern\":\"monitoring-eye-css-*\",\"interval\":\"\",\"axis_position\":\"left\",\"axis_formatter\":\"number\",\"axis_scale\":\"normal\",\"show_legend\":1,\"show_grid\":1,\"tooltip_mode\":\"show_all\",\"default_index_pattern\":\"monitoring-eye-css-*\",\"default_timefield\":\"timestamp\",\"isModelInvalid\":false,\"legend_position\":\"bottom\"}}"},"id":"eea89780-664b-11ed-8cd7-973626cf6f70","references":[],"type":"visualization","updated_at":"2022-12-01T12:41:01.165Z","version":"WzIxMSwyXQ=="}
+{"attributes":{"description":"","kibanaSavedObjectMeta":{"searchSourceJSON":"{}"},"title":"[monitoring] total docs delete","uiStateJSON":"{}","version":1,"visState":"{\"title\":\"[monitoring] total docs delete\",\"type\":\"metrics\",\"aggs\":[],\"params\":{\"id\":\"61ca57f0-469d-11e7-af02-69e470af7417\",\"type\":\"timeseries\",\"series\":[{\"id\":\"61ca57f1-469d-11e7-af02-69e470af7417\",\"color\":\"rgba(214,191,87,1)\",\"split_mode\":\"everything\",\"split_color_mode\":\"kibana\",\"metrics\":[{\"id\":\"61ca57f2-469d-11e7-af02-69e470af7417\",\"type\":\"max\",\"field\":\"indices_stats._all.total.docs.deleted\"}],\"separate_axis\":0,\"axis_position\":\"right\",\"formatter\":\"number\",\"chart_type\":\"line\",\"line_width\":1,\"point_size\":1,\"fill\":0.5,\"stacked\":\"none\",\"label\":\"totol_docs_delete\",\"type\":\"timeseries\",\"hidden\":false}],\"time_field\":\"timestamp\",\"index_pattern\":\"monitoring-eye-css-*\",\"interval\":\"\",\"axis_position\":\"left\",\"axis_formatter\":\"number\",\"axis_scale\":\"normal\",\"show_legend\":1,\"show_grid\":1,\"tooltip_mode\":\"show_all\",\"default_index_pattern\":\"monitoring-eye-css-*\",\"default_timefield\":\"timestamp\",\"isModelInvalid\":false,\"drop_last_bucket\":1,\"legend_position\":\"bottom\"}}"},"id":"cfbb4e20-664c-11ed-8cd7-973626cf6f70","references":[],"type":"visualization","updated_at":"2022-12-01T12:41:01.165Z","version":"WzIxMiwyXQ=="}
+{"attributes":{"description":"","kibanaSavedObjectMeta":{"searchSourceJSON":"{}"},"title":"[monitoring] total store size in bytes","uiStateJSON":"{}","version":1,"visState":"{\"title\":\"[monitoring] total store size in bytes\",\"type\":\"metrics\",\"aggs\":[],\"params\":{\"id\":\"61ca57f0-469d-11e7-af02-69e470af7417\",\"type\":\"timeseries\",\"series\":[{\"id\":\"61ca57f1-469d-11e7-af02-69e470af7417\",\"color\":\"#68BC00\",\"split_mode\":\"everything\",\"split_color_mode\":\"kibana\",\"metrics\":[{\"id\":\"61ca57f2-469d-11e7-af02-69e470af7417\",\"type\":\"max\",\"field\":\"indices_stats._all.total.store.size_in_bytes\"}],\"separate_axis\":0,\"axis_position\":\"right\",\"formatter\":\"bytes\",\"chart_type\":\"line\",\"line_width\":1,\"point_size\":1,\"fill\":0.5,\"stacked\":\"none\",\"label\":\"total store size in bytes\",\"type\":\"timeseries\"}],\"time_field\":\"timestamp\",\"index_pattern\":\"monitoring-eye-css-*\",\"interval\":\"\",\"axis_position\":\"left\",\"axis_formatter\":\"number\",\"axis_scale\":\"normal\",\"show_legend\":1,\"show_grid\":1,\"tooltip_mode\":\"show_all\",\"default_index_pattern\":\"monitoring-eye-css-*\",\"default_timefield\":\"timestamp\",\"isModelInvalid\":false,\"legend_position\":\"bottom\",\"background_color_rules\":[{\"id\":\"7712e550-664f-11ed-8b5d-8db37e5b4cc4\"}],\"bar_color_rules\":[{\"id\":\"77680a30-664f-11ed-8b5d-8db37e5b4cc4\"}]}}"},"id":"c7f72ae0-664e-11ed-8cd7-973626cf6f70","references":[],"type":"visualization","updated_at":"2022-12-01T12:41:01.165Z","version":"WzIxMywyXQ=="}
+{"attributes":{"description":"","kibanaSavedObjectMeta":{"searchSourceJSON":"{}"},"title":"[monitoring] indexing rate of index for top10(/s)","uiStateJSON":"{}","version":1,"visState":"{\"title\":\"[monitoring] indexing rate of index for top10(/s)\",\"type\":\"metrics\",\"aggs\":[],\"params\":{\"id\":\"61ca57f0-469d-11e7-af02-69e470af7417\",\"type\":\"timeseries\",\"series\":[{\"id\":\"61ca57f1-469d-11e7-af02-69e470af7417\",\"color\":\"#68BC00\",\"split_mode\":\"terms\",\"metrics\":[{\"id\":\"61ca57f2-469d-11e7-af02-69e470af7417\",\"type\":\"max\",\"field\":\"index_stats.total.indexing.index_total\"},{\"unit\":\"1s\",\"id\":\"541ed8f0-a5ee-11ec-aa10-992297d21a2e\",\"type\":\"derivative\",\"field\":\"61ca57f2-469d-11e7-af02-69e470af7417\"},{\"unit\":\"\",\"id\":\"67ec1f50-a5ee-11ec-aa10-992297d21a2e\",\"type\":\"positive_only\",\"field\":\"541ed8f0-a5ee-11ec-aa10-992297d21a2e\"}],\"separate_axis\":0,\"axis_position\":\"right\",\"formatter\":\"number\",\"chart_type\":\"line\",\"line_width\":1,\"point_size\":1,\"fill\":0.5,\"stacked\":\"none\",\"label\":\"indexing_rate\",\"type\":\"timeseries\",\"split_filters\":[{\"color\":\"#68BC00\",\"id\":\"81004200-a5ee-11ec-aa10-992297d21a2e\",\"filter\":{\"query\":\"\",\"language\":\"kuery\"}}],\"filter\":{\"query\":\"\",\"language\":\"kuery\"},\"terms_field\":\"index_stats.index\",\"terms_order_by\":\"61ca57f2-469d-11e7-af02-69e470af7417\",\"terms_size\":\"10\",\"terms_direction\":\"desc\",\"split_color_mode\":\"rainbow\"}],\"time_field\":\"timestamp\",\"index_pattern\":\"monitoring-eye-css-*\",\"interval\":\"\",\"axis_position\":\"left\",\"axis_formatter\":\"number\",\"axis_scale\":\"normal\",\"show_legend\":1,\"show_grid\":1,\"default_index_pattern\":\"monitoring-eye-css-*\",\"default_timefield\":\"timestamp\",\"isModelInvalid\":false,\"tooltip_mode\":\"show_all\"}}"},"id":"943b3e00-a5ef-11ec-9a68-ada9d754c566","references":[],"type":"visualization","updated_at":"2022-12-01T12:41:01.165Z","version":"WzIxNCwyXQ=="}
+{"attributes":{"description":"","kibanaSavedObjectMeta":{"searchSourceJSON":"{}"},"title":"[monitoring] search rate of index for top10(/s)","uiStateJSON":"{}","version":1,"visState":"{\"title\":\"[monitoring] search rate of index for top10(/s)\",\"type\":\"metrics\",\"aggs\":[],\"params\":{\"id\":\"61ca57f0-469d-11e7-af02-69e470af7417\",\"type\":\"timeseries\",\"series\":[{\"id\":\"61ca57f1-469d-11e7-af02-69e470af7417\",\"color\":\"rgba(99,157,12,1)\",\"split_mode\":\"terms\",\"metrics\":[{\"id\":\"61ca57f2-469d-11e7-af02-69e470af7417\",\"type\":\"max\",\"field\":\"index_stats.total.search.query_total\"},{\"unit\":\"1s\",\"id\":\"fdfdfad0-a5ef-11ec-aa10-992297d21a2e\",\"type\":\"derivative\",\"field\":\"61ca57f2-469d-11e7-af02-69e470af7417\"},{\"unit\":\"\",\"id\":\"0aaa26a0-a5f0-11ec-aa10-992297d21a2e\",\"type\":\"positive_only\",\"field\":\"fdfdfad0-a5ef-11ec-aa10-992297d21a2e\"}],\"separate_axis\":0,\"axis_position\":\"right\",\"formatter\":\"number\",\"chart_type\":\"line\",\"line_width\":1,\"point_size\":1,\"fill\":0.5,\"stacked\":\"none\",\"label\":\"search rate\",\"type\":\"timeseries\",\"terms_field\":\"index_stats.index\",\"terms_order_by\":\"61ca57f2-469d-11e7-af02-69e470af7417\",\"split_color_mode\":\"rainbow\"}],\"time_field\":\"timestamp\",\"index_pattern\":\"monitoring-eye-css-*\",\"interval\":\"\",\"axis_position\":\"left\",\"axis_formatter\":\"number\",\"axis_scale\":\"normal\",\"show_legend\":1,\"show_grid\":1,\"default_index_pattern\":\"monitoring-eye-css-*\",\"default_timefield\":\"timestamp\",\"isModelInvalid\":false,\"tooltip_mode\":\"show_all\"}}"},"id":"ab503550-a5ef-11ec-9a68-ada9d754c566","references":[],"type":"visualization","updated_at":"2022-12-01T12:41:01.165Z","version":"WzIxNSwyXQ=="}
+{"attributes":{"description":"","kibanaSavedObjectMeta":{"searchSourceJSON":"{}"},"title":"[monitoring] indices store_size for top10","uiStateJSON":"{}","version":1,"visState":"{\"title\":\"[monitoring] indices store_size for top10\",\"type\":\"metrics\",\"aggs\":[],\"params\":{\"id\":\"61ca57f0-469d-11e7-af02-69e470af7417\",\"type\":\"timeseries\",\"series\":[{\"id\":\"38474c50-a5f5-11ec-aa10-992297d21a2e\",\"color\":\"#68BC00\",\"split_mode\":\"terms\",\"metrics\":[{\"id\":\"38474c51-a5f5-11ec-aa10-992297d21a2e\",\"type\":\"max\",\"field\":\"index_stats.total.store.size_in_bytes\"}],\"separate_axis\":0,\"axis_position\":\"right\",\"formatter\":\"bytes\",\"chart_type\":\"line\",\"line_width\":1,\"point_size\":1,\"fill\":0.5,\"stacked\":\"none\",\"label\":\"store_size for index\",\"type\":\"timeseries\",\"terms_field\":\"index_stats.index\",\"terms_order_by\":\"38474c51-a5f5-11ec-aa10-992297d21a2e\",\"filter\":{\"query\":\"\",\"language\":\"kuery\"},\"split_color_mode\":\"rainbow\"}],\"time_field\":\"timestamp\",\"index_pattern\":\"monitoring-eye-css-*\",\"interval\":\"\",\"axis_position\":\"left\",\"axis_formatter\":\"number\",\"axis_scale\":\"normal\",\"show_legend\":1,\"show_grid\":1,\"default_index_pattern\":\"monitoring-eye-css-*\",\"default_timefield\":\"timestamp\",\"isModelInvalid\":false,\"filter\":{\"query\":\"\",\"language\":\"kuery\"},\"bar_color_rules\":[{\"id\":\"7d9d3cb0-a5f5-11ec-aa10-992297d21a2e\"}],\"tooltip_mode\":\"show_all\"}}"},"id":"c78119a0-a5f5-11ec-9a68-ada9d754c566","references":[],"type":"visualization","updated_at":"2022-12-01T12:41:01.165Z","version":"WzIxNiwyXQ=="}
+{"attributes":{"description":"","kibanaSavedObjectMeta":{"searchSourceJSON":"{}"},"title":"[monitoring] search query time in millis of index for top10(ms)","uiStateJSON":"{}","version":1,"visState":"{\"title\":\"[monitoring] search query time in millis of index for top10(ms)\",\"type\":\"metrics\",\"aggs\":[],\"params\":{\"axis_formatter\":\"number\",\"axis_max\":\"\",\"axis_min\":\"\",\"axis_position\":\"left\",\"axis_scale\":\"normal\",\"default_index_pattern\":\"monitoring-eye-css-*\",\"default_timefield\":\"timestamp\",\"id\":\"61ca57f0-469d-11e7-af02-69e470af7417\",\"index_pattern\":\"monitoring-eye-css-*\",\"interval\":\"\",\"isModelInvalid\":false,\"series\":[{\"axis_position\":\"right\",\"chart_type\":\"line\",\"color\":\"#68BC00\",\"fill\":0.5,\"formatter\":\"number\",\"id\":\"61ca57f1-469d-11e7-af02-69e470af7417\",\"label\":\"index_query_time_in_millis\",\"line_width\":1,\"metrics\":[{\"field\":\"index_stats.total.search.query_time_in_millis\",\"id\":\"61ca57f2-469d-11e7-af02-69e470af7417\",\"type\":\"max\"},{\"unit\":\"1s\",\"id\":\"42c92b10-6645-11ed-925a-6de90846447d\",\"type\":\"derivative\",\"field\":\"61ca57f2-469d-11e7-af02-69e470af7417\"}],\"point_size\":1,\"separate_axis\":0,\"split_color_mode\":\"kibana\",\"split_mode\":\"terms\",\"stacked\":\"none\",\"terms_field\":\"index_stats.index\",\"terms_order_by\":\"61ca57f2-469d-11e7-af02-69e470af7417\",\"type\":\"timeseries\"}],\"show_grid\":1,\"show_legend\":1,\"time_field\":\"timestamp\",\"tooltip_mode\":\"show_all\",\"type\":\"timeseries\",\"background_color\":null,\"filter\":{\"query\":\"\",\"language\":\"kuery\"},\"legend_position\":\"right\"}}"},"id":"c8109100-6627-11ed-8cd7-973626cf6f70","references":[],"type":"visualization","updated_at":"2022-12-01T12:41:01.165Z","version":"WzIxNywyXQ=="}
+{"attributes":{"description":"","hits":0,"kibanaSavedObjectMeta":{"searchSourceJSON":"{\"query\":{\"language\":\"kuery\",\"query\":\"\"},\"filter\":[]}"},"optionsJSON":"{\"hidePanelTitles\":false,\"useMargins\":true}","panelsJSON":"[{\"gridData\":{\"x\":0,\"y\":0,\"w\":48,\"h\":5,\"i\":\"971ed6c6-81b9-491b-9f08-e3ae9c382abd\"},\"panelIndex\":\"971ed6c6-81b9-491b-9f08-e3ae9c382abd\",\"embeddableConfig\":{},\"panelRefName\":\"panel_0\"},{\"gridData\":{\"x\":0,\"y\":5,\"w\":24,\"h\":15,\"i\":\"5a6982e7-0c6c-4733-8a2d-e4c57cdf7397\"},\"panelIndex\":\"5a6982e7-0c6c-4733-8a2d-e4c57cdf7397\",\"embeddableConfig\":{},\"panelRefName\":\"panel_1\"},{\"gridData\":{\"x\":24,\"y\":5,\"w\":24,\"h\":15,\"i\":\"662476f4-739c-4a05-858c-2ee8230cf410\"},\"panelIndex\":\"662476f4-739c-4a05-858c-2ee8230cf410\",\"embeddableConfig\":{},\"panelRefName\":\"panel_2\"},{\"gridData\":{\"x\":0,\"y\":20,\"w\":16,\"h\":15,\"i\":\"d89c38e2-33f3-4592-b503-20460a6a7a57\"},\"panelIndex\":\"d89c38e2-33f3-4592-b503-20460a6a7a57\",\"embeddableConfig\":{},\"panelRefName\":\"panel_3\"},{\"gridData\":{\"x\":16,\"y\":20,\"w\":16,\"h\":15,\"i\":\"1f693b49-79fa-4807-94e8-0c12f51e54f8\"},\"panelIndex\":\"1f693b49-79fa-4807-94e8-0c12f51e54f8\",\"embeddableConfig\":{},\"panelRefName\":\"panel_4\"},{\"gridData\":{\"x\":32,\"y\":20,\"w\":16,\"h\":15,\"i\":\"616b143d-74e9-4dac-98ba-5849536f0fba\"},\"panelIndex\":\"616b143d-74e9-4dac-98ba-5849536f0fba\",\"embeddableConfig\":{},\"panelRefName\":\"panel_5\"},{\"gridData\":{\"x\":0,\"y\":35,\"w\":24,\"h\":11,\"i\":\"cfa82f27-1b8d-49ba-a7b9-d8809d3b258c\"},\"panelIndex\":\"cfa82f27-1b8d-49ba-a7b9-d8809d3b258c\",\"embeddableConfig\":{},\"panelRefName\":\"panel_6\"},{\"gridData\":{\"x\":24,\"y\":35,\"w\":24,\"h\":11,\"i\":\"135d13eb-aab6-43ca-9029-7d26e91d90e3\"},\"panelIndex\":\"135d13eb-aab6-43ca-9029-7d26e91d90e3\",\"embeddableConfig\":{},\"panelRefName\":\"panel_7\"},{\"gridData\":{\"x\":0,\"y\":46,\"w\":24,\"h\":11,\"i\":\"28a77de1-9110-49e8-b273-724f880b1653\"},\"panelIndex\":\"28a77de1-9110-49e8-b273-724f880b1653\",\"embeddableConfig\":{},\"panelRefName\":\"panel_8\"},{\"gridData\":{\"x\":24,\"y\":46,\"w\":24,\"h\":11,\"i\":\"80ece867-cf23-4935-bfbc-430afa51bcca\"},\"panelIndex\":\"80ece867-cf23-4935-bfbc-430afa51bcca\",\"embeddableConfig\":{},\"panelRefName\":\"panel_9\"},{\"gridData\":{\"x\":0,\"y\":57,\"w\":24,\"h\":11,\"i\":\"2ba970aa-c9c4-491b-bdd3-c1b1ee9bc8d3\"},\"panelIndex\":\"2ba970aa-c9c4-491b-bdd3-c1b1ee9bc8d3\",\"embeddableConfig\":{},\"panelRefName\":\"panel_10\"},{\"gridData\":{\"x\":24,\"y\":57,\"w\":24,\"h\":11,\"i\":\"f2e1b6ab-ddf7-492e-aaca-9460f11aa4aa\"},\"panelIndex\":\"f2e1b6ab-ddf7-492e-aaca-9460f11aa4aa\",\"embeddableConfig\":{},\"panelRefName\":\"panel_11\"},{\"gridData\":{\"x\":0,\"y\":68,\"w\":24,\"h\":11,\"i\":\"dd14182d-d8b9-47f2-bf36-6cba3b09586c\"},\"panelIndex\":\"dd14182d-d8b9-47f2-bf36-6cba3b09586c\",\"embeddableConfig\":{},\"panelRefName\":\"panel_12\"},{\"gridData\":{\"x\":24,\"y\":68,\"w\":24,\"h\":11,\"i\":\"a47f9333-52b7-49b7-8cac-f470cf405131\"},\"panelIndex\":\"a47f9333-52b7-49b7-8cac-f470cf405131\",\"embeddableConfig\":{},\"panelRefName\":\"panel_13\"}]","timeRestore":false,"title":"[Monitoring] Index monitoring Dashboard","version":1},"id":"524eb000-a5f2-11ec-9a68-ada9d754c566","references":[{"id":"b2811c70-a5f1-11ec-9a68-ada9d754c566","name":"panel_0","type":"visualization"},{"id":"de4f8ab0-a5f8-11ec-9a68-ada9d754c566","name":"panel_1","type":"visualization"},{"id":"811df7a0-a5f8-11ec-9a68-ada9d754c566","name":"panel_2","type":"visualization"},{"id":"eea89780-664b-11ed-8cd7-973626cf6f70","name":"panel_3","type":"visualization"},{"id":"cfbb4e20-664c-11ed-8cd7-973626cf6f70","name":"panel_4","type":"visualization"},{"id":"c7f72ae0-664e-11ed-8cd7-973626cf6f70","name":"panel_5","type":"visualization"},{"id":"943b3e00-a5ef-11ec-9a68-ada9d754c566","name":"panel_6","type":"visualization"},{"id":"ab503550-a5ef-11ec-9a68-ada9d754c566","name":"panel_7","type":"visualization"},{"id":"c78119a0-a5f5-11ec-9a68-ada9d754c566","name":"panel_8","type":"visualization"},{"id":"225f6020-a5f1-11ec-9a68-ada9d754c566","name":"panel_9","type":"visualization"},{"id":"17d49220-662a-11ed-8cd7-973626cf6f70","name":"panel_10","type":"visualization"},{"id":"c8109100-6627-11ed-8cd7-973626cf6f70","name":"panel_11","type":"visualization"},{"id":"45d571c0-6626-11ed-8cd7-973626cf6f70","name":"panel_12","type":"visualization"},{"id":"3ae5d820-6628-11ed-8cd7-973626cf6f70","name":"panel_13","type":"visualization"}],"type":"dashboard","updated_at":"2022-12-01T12:41:01.165Z","version":"WzIxOCwyXQ=="}
+{"exportedCount":16,"missingRefCount":0,"missingReferences":[]}
+
+
diff --git a/docs/css/umn/css_01_0046.html b/docs/css/umn/css_01_0046.html
new file mode 100644
index 000000000..5d805f513
--- /dev/null
+++ b/docs/css/umn/css_01_0046.html
@@ -0,0 +1,141 @@
+
+
+
Using CDM to Import Data to Elasticsearch
+
With CSS, you can use the web user interface of the Cloud Data Migration (CDM) service to import data from an Oracle database or OBS to an Elasticsearch cluster. Only JSON files are supported.
+
+
Table 1 Using CDM to import data to CSSScenario
+ |
+Source Data
+ |
+Target Cluster
+ |
+
|---|
+
+Importing data from an Oracle database to CSS
+ |
+A local or third-party Oracle database
+ |
+Elasticsearch 5.5, 6.2, 6.5, 7.1, 7.6, 7.9, or 7.10
+ |
+
+Importing data from OBS to CSS
+ |
+JSON files in OBS buckets
+ |
+Elasticsearch 5.5, 6.2, 6.5, 7.1, 7.6, 7.9, or 7.10
+ |
+
+
+
+
+
Preparations
- Confirm the source data.
For example, the source data can be the following JSON files:
+For Elasticsearch 7.x or later:
{"index": {"_index":"my_store"}}
+{"productName":"Autumn new woman blouses 2019","size":"M"}
+{"index": {"_index":"my_store"}}
+{"productName":"Autumn new woman blouses 2019","size":"L"}
+
For Elasticsearch earlier than 7.x:
{"index": {"_index":"my_store","_type":"products"}}
+{"productName":"Autumn new woman blouses 2019","size":"M"}
+{"index": {"_index":"my_store","_type":"products"}}
+{"productName":"Autumn new woman blouses 2019","size":"L"}
+
- Obtain data source information.
- If the data source is an Oracle database, you need to obtain the IP address, database name, username, and password of the database.
- If the data source is JSON files in an OBS bucket, you need to obtain its domain name, port, AK, and SK.
+ - If the data source is an Oracle database, make sure the Oracle database can be accessed using a public IP address, or a VPN or Direct Connect connection has been established between the customer's data center and the cloud service.
+
+
Importing Data
- Log in to the CSS management console.
- In the navigation pane on the left, choose Clusters > Elasticsearch.
- In the cluster list, obtain the target cluster's private IP address from the Private IP Address column. Generally, the IP address format is <host>:<port> or <host>:<port>,<host>:<port>.
If the cluster has only one node, the IP address and port number of this one node are displayed, for example, 10.62.179.32:9200. If the cluster has multiple nodes, the IP addresses and port numbers of all nodes are displayed, for example, 10.62.179.32:9200,10.62.179.33:9200.
+ - In the cluster list, locate the destination cluster, and click Access Kibana in the Operation column to log in to the Kibana console.
- In the Kibana navigation pane on the left, choose Dev Tools.
- Run the following command on the console to check whether the cluster has indexes:
GET _cat/indices?v
+- If indexes are available in the cluster to which you want to import data, you do not need to create an index. Go to 8.
- If no indexes are available in the cluster, go to the next step to create an index.
+ - Run the following command to create an index for storing imported data and create a custom mapping to define the data type.
For example, run the following command to create index demo:
+For Elasticsearch 7.x or later:
+PUT /demo
+{
+ "settings": {
+ "number_of_shards": 1
+ },
+ "mappings": {
+ "properties": {
+ "productName": {
+ "type": "text",
+ "analyzer": "ik_smart"
+ },
+ "size": {
+ "type": "keyword"
+ }
+ }
+ }
+ }
+For Elasticsearch earlier than 7.x:
PUT /demo
+{
+ "settings": {
+ "number_of_shards": 1
+ },
+ "mappings": {
+ "products": {
+ "properties": {
+ "productName": {
+ "type": "text",
+ "analyzer": "ik_smart"
+ },
+ "size": {
+ "type": "keyword"
+ }
+ }
+ }
+ }
+}
+
The command is successfully executed if the following information is displayed.
+{
+ "acknowledged" : true,
+ "shards_acknowledged" : true,
+ "index" : "demo"
+}
+ - Use CDM to import data from Oracle or OBS to the Elasticsearch cluster.
- After the data migration is complete, go to the Kibana console of the Elasticsearch cluster again, and search for the imported data.
Run the following command to search for data. If the imported data is consistent with the source data, data importing is successful.
+GET demo/_search
+demo is the name of the created index. Replace it if another index is used.
+The command is successfully executed if the following information is displayed.
+{
+ "took": 18,
+ "timed_out": false,
+ "_shards": {
+ "total": 1,
+ "successful": 1,
+ "skipped": 0,
+ "failed": 0
+ },
+ "hits": {
+ "total": 2,
+ "max_score": 1,
+ "hits": [
+ {
+ "_index": "demo",
+ "_type": "products",
+ "_id": "g6UepnEBuvdFwWkRmn4V",
+ "_score": 1,
+ "_source": {
+ "size": """"size":"L"}""",
+ "productName": """{"productName":"Latest art shirts for women in autumn 2019""""
+ }
+ },
+ {
+ "_index": "demo",
+ "_type": "products",
+ "_id": "hKUepnEBuvdFwWkRmn4V",
+ "_score": 1,
+ "_source": {
+ "size": """"size":"M"}""",
+ "productName": """{"productName":"Latest art shirts for women in autumn 2019""""
+ }
+ }
+ ]
+ }
+}
+
+
+
+
+
diff --git a/docs/css/umn/css_01_0047.html b/docs/css/umn/css_01_0047.html
new file mode 100644
index 000000000..41ed29bcf
--- /dev/null
+++ b/docs/css/umn/css_01_0047.html
@@ -0,0 +1,150 @@
+
+
+
Configuring an Index Recycle Bin for an OpenSearch Cluster
+
By default, when indexes are deleted in an OpenSearch cluster, the index data is deleted permanently. To prevent data loss caused by accidental deletion, CSS provides an index recycle bin. When enabled, deleted indexes are temporarily stored in the recycle bin, allowing for recovery before they are permanently removed. This feature improves data reliability and operational security.
+
How the Feature Works
The index recycle bin works as follows:
+
- Data recycling mechanism: When an index is deleted, it is first moved to the recycle bin. From there, it will be automatically cleared when a predefined retention duration expires. Indexes in the recycle bin are still part of the cluster metadata and can be restored using an API.
- Status management
- When the DELETE API is used to delete an index, the index is closed, and the cluster status temporarily changes to Red.
- When the restore API is used to restore an index, the index is reopened and shards are initialized, which may also cause the cluster status to temporarily change to Red.
+
+
+
Constraints
- Only OpenSearch 2.19.0 supports the index recycle bin.
+
- Indexes in the recycle bin are still part of the cluster metadata. Before they are removed from the recycle bin, same-name indexes cannot be created for the cluster.
+
+
Logging In to OpenSearch Dashboards
Log in to OpenSearch Dashboards and go to the command execution page. OpenSearch clusters support multiple access methods. This topic uses OpenSearch Dashboards as an example to describe the operation procedures.
+
- Log in to the CSS management console.
- In the navigation pane on the left, choose Clusters > OpenSearch.
- In the cluster list, find the target cluster, and click Dashboards in the Operation column to log in to OpenSearch Dashboards.
- In the left navigation pane, choose Dev Tools.
The left part of the console is the command input box, and the triangle icon in its upper-right corner is the execution button. The right part shows the execution result.
+
+
+
Enabling the Index Recycle Bin
Run the following command to enable the index recycle bin:
+
PUT _cluster/settings
+{
+ "persistent": {
+ "index.trash.enabled": true
+ }
+}
+
+
Table 1 Parameters for enabling the index recycle binParameter
+ |
+Type
+ |
+Description
+ |
+
|---|
+
+index.trash.enabled
+ |
+Boolean
+ |
+Whether to enable the index recycle bin. When the index recycle bin is enabled, deleting indexes will first move them to the recycle bin. The indexes will be permanently deleted only when they are cleared from the recycle bin.
+- true: Enable the index recycle bin.
- false (default): Disable the index recycle bin.
+ |
+
+indices.trash.keep.time
+ |
+String
+ |
+Index retention duration in the recycle bin. Indexes will be automatically removed from the recycle bin when this duration expires.
+Examples of supported time formats: 1d (one day), 7d (seven days), 1w (one week), 1h (one hour). The minimum duration is 1d.
+Default value: 1d.
+ |
+
+
+
+
+
The following command deletes an index:
DELETE {INDEX_NAME}
+
+
INDEX_NAME indicates the name of the index to be deleted. Wildcards can be used to specify indexes.
+
+
+
Viewing Indexes in the Recycle Bin
Run the following command to view indexes in the index recycle bin:
+
GET _cat/trash?v=true&s=index
+
+
Table 2 Parameter descriptionParameter
+ |
+Description
+ |
+
|---|
+
+v
+ |
+Whether to display the table header when the return format is a table.
+- true: display the table header.
- false: not display the table header.
+The default value is false.
+ |
+
+s
+ |
+Index sorting fields. Options include index, uuid, pri, rep, trash.ts, and delete.time.
+ |
+
+format
+ |
+Return format of the command. The default format is table. Other options include json, yaml, cobr, and smile. cobr and smile are binary formats.
+ |
+
+h
+ |
+Set the column names to be displayed. By default, all column names are displayed. To specify multiple column names, separate them using commas (,), for example, h=index,uuid,delete.time.
+ |
+
+
+
+
+
The following is an example of the output.
+
index uuid pri rep trash.ts delete.time
+index1 CMD3FCLzTOyTg4RUekWNNA 1 1 1714465116615 23.6h
+index1 6ATijuu6SfqamVI-WMyOKg 1 1 1714466233898 23.9h
+
+
Table 3 Parameters in the outputColumn
+ |
+Description
+ |
+
|---|
+
+index
+ |
+Index name
+ |
+
+uuid
+ |
+Index UUID
+ |
+
+pri
+ |
+Number of shards of an index
+ |
+
+rep
+ |
+Number of replicas of an index
+ |
+
+trash.ts
+ |
+Time when an index was moved to the recycle bin
+ |
+
+delete.time
+ |
+Remaining retention duration of an index in the recycle bin. When the value changes to 0, the index is permanently deleted.
+ |
+
+
+
+
+
+
Restoring an Index from the Recycle Bin
Run the following command to restore an index from the recycle bin:
+
POST /trash/recover/{INDEX_NAME}
+
INDEX_NAME indicates the name of the index to be restored. Wildcards can be used to specify indexes.
+
+
Emptying the Recycle Bin
Run the following command to empty the recycle bin:
+
POST trash/empty
+
Emptying the recycle bin will permanently delete all data in it. Please exercise caution.
+
+
+
+
diff --git a/docs/css/umn/css_01_0048.html b/docs/css/umn/css_01_0048.html
index 64f511dc4..35f119cd4 100644
--- a/docs/css/umn/css_01_0048.html
+++ b/docs/css/umn/css_01_0048.html
@@ -1,26 +1,21 @@
-
+
Using In-house Built Logstash to Import Data to Elasticsearch
-
You can use in-house built Logstash to migrate data to Elasticsearch or OpenSearch in CSS. This helps you effectively ingest and manage data through CSS. Data files can be in the JSON or CSV format.
-
Logstash is an open-source, server-side data processing pipeline that can ingest data from multiple sources simultaneously, transform data, and then send data to Elasticsearch or OpenSearch. For details about Logstash, visit the following website: https://www.elastic.co/guide/en/logstash/current/getting-started-with-logstash.html
-
The following two scenarios are involved depending on the Logstash deployment:
-
-
Prerequisites
+
With CSS, you can use in-house developed Logstash to ingest data into Elasticsearch for efficient search and exploration. Data files can be in the JSON or CSV format.
+
Logstash is an open-source, server-side data processing pipeline that ingests data from multiple sources simultaneously, processes and transforms the data, and then sends it to Elasticsearch. For details about Logstash, visit the following website: https://www.elastic.co/guide/en/logstash/current/getting-started-with-logstash.html
+
The following two scenarios are involved depending on the Logstash deployment:
+
+
Prerequisites
- To facilitate operations, you are advised to deploy Logstash on a host that runs the Linux operating system (OS).
- Logstash must use an OSS version that is consistent with that of the CSS cluster. To download Logstash, visit the following website: https://www.elastic.co/downloads/logstash-oss
- The JDK must be installed before Logstash is installed. In Linux OS, you can run the yum -y install java-1.8.0 command to install JDK 1.8.0. In Windows OS, you can download the required JDK version from the official website of JDK, and install it by following the installation guide.
- After installing Logstash, perform the following steps to import data. For details about how to install Logstash, visit the following website: https://www.elastic.co/guide/en/logstash/current/installing-logstash.html
- In the Importing Data When Logstash Is Deployed on an ECS scenario, ensure that the ECS and the Elasticsearch cluster to which data is imported reside in the same VPC.
-
Importing Data When Logstash Is Deployed on the External Network
Figure 1 illustrates how data is imported when Logstash is deployed on an external network.
-
Figure 1 Importing data when Logstash is deployed on an external network
-
- Create a jump host and configure it as follows:
- The jump host is an ECS running the Linux OS and has been bound with an EIP.
- The jump host resides in the same VPC as the CSS cluster.
- SSH local port forwarding is configured for the jump host to forward requests from a chosen local port to port 9200 on one node of the CSS cluster.
- Refer to SSH documentation for the local port forwarding configuration.
- - Use PuTTY to log in to the created jump host with the EIP.
- Run the following command to perform port mapping and transfer the request sent to the port on the jump host to the target cluster:
ssh -g -L <Local port of the jump host:Private network address and port number of a node> -N -f root@<Private IP address of the jump host>
- - In the preceding command, <Local port of the jump host> refers to the port obtained in 1.
- In the preceding command, <Private network address and port number of a node> refers to the private network address and port number of a node in the cluster. If the node is faulty, the command execution will fail. If the cluster contains multiple nodes, you can replace the value of <private network address and port number of a node> with the private network address and port number of any available node in the cluster. If the cluster contains only one node, restore the node and execute the command again.
- Replace <Private IP address of the jump host> in the preceding command with the IP address (with Private IP) of the created jump host in the IP Address column in the ECS list on the ECS management console.
-
-For example, port 9200 on the jump host is assigned external network access permissions, the private network address and port number of the node are 192.168.0.81 and 9200, respectively, and the private IP address of the jump host is 192.168.0.227. You need to run the following command to perform port mapping:
-ssh -g -L 9200:192.168.0.81:9200 -N -f root@192.168.0.227
- - Log in to the server where Logstash is deployed and store the data files to be imported on the server.
For example, data file access_20181029_log needs to be imported, the file storage path is /tmp/access_log/, and the data file includes the following data:
- Create the access_log folder if it does not exist.
-
-| All | Heap used for segments | | 18.6403 | MB |
+Importing Data When Logstash Is Deployed on the External Network
Figure 1 illustrates how data is imported when Logstash is deployed on an external network.
+
Figure 1 Importing data when Logstash is deployed on an external network
+
- Create a jump host and configure it as follows:
- The jump host is an ECS running the Linux OS and has been bound with an EIP.
- The jump host resides in the same VPC as the CSS cluster.
- SSH local port forwarding is configured for the jump host to forward requests from a chosen local port to port 9200 on one node of the CSS cluster.
- Refer to SSH documentation for the local port forwarding configuration.
+ - Use PuTTY to log in to the created jump host with the EIP.
- Run the following command to configure port mapping to forward requests sent to the opened port on the jump host to the destination cluster:
ssh -g -L <Local port of the jump host:Private network address and port number of a node> -N -f root@<Private IP address of the jump host>
+- In the preceding command, <Local port of the jump host> refers to the port obtained in 1.
- In the preceding command, <Private network address and port number of a node> refers to the private network address and port number of a node in the cluster. If the node is faulty, the command execution will fail. If the cluster contains multiple nodes, you can replace the value of <private network address and port number of a node> with the private network address and port number of any available node in the cluster. If the cluster contains only one node, restore the node and execute the command again.
- Replace <Private IP address of the jump host> in the preceding command with the IP address (with Private IP) of the created jump host in the IP Address column in the ECS list on the ECS management console.
+For example, port 9200 on the jump host is assigned external network access permissions, the private network address and port number of the node are 192.168.0.81 and 9200, respectively, and the private IP address of the jump host is 192.168.0.227. You need to run the following command to perform port mapping:
+ssh -g -L 9200:192.168.0.81:9200 -N -f root@192.168.0.227
+ - Log in to the server where Logstash is deployed and store the data files to be imported on the server.
For example, data file access_20181029_log needs to be imported, the file storage path is /tmp/access_log/ (create the access_log folder if it does not already exist), and the data file includes the following data:
+| All | Heap used for segments | | 18.6403 | MB |
| All | Heap used for doc values | | 0.119289 | MB |
| All | Heap used for terms | | 17.4095 | MB |
| All | Heap used for norms | | 0.0767822 | MB |
@@ -31,25 +26,25 @@
| All | Median Throughput | index-append | 66735.3 | docs/s |
| All | Max Throughput | index-append | 67745.6 | docs/s |
| All | 50th percentile latency | index-append | 510.261 | ms |
- - In the server where Logstash is deployed, run the following command to create configuration file logstash-simple.conf in the Logstash installation directory:
cd /<Logstash installation directory>/
+
- On the server where Logstash is deployed, run the following command to create configuration file logstash-simple.conf in the Logstash installation directory:
cd /<Logstash installation directory>/
vi logstash-simple.conf
- - Input the following content in logstash-simple.conf:
input {
-Location of data
+ - Enter the following content in logstash-simple.conf:
input {
+Location of data
}
filter {
-Related data processing
+Related data processing
}
output {
elasticsearch {
- hosts => "<EIP of the jump host>:<Number of the port assigned external network access permissions on the jump host>"
- (Optional) If communication encryption has been enabled on the cluster, you need to add the following configuration:
+ hosts => "<EIP of the jump host>:<Number of the port assigned external network access permissions on the jump host>"
+ (Optional) If communication encryption has been enabled for the cluster, you need to add the following configuration:
ssl => true
ssl_certificate_verification => false
}
}
-- The input parameter indicates the data source. Set this parameter based on the actual conditions. For details about the input parameter and parameter usage, visit the following website: https://www.elastic.co/guide/en/logstash/current/input-plugins.html
- The filter parameter specifies the mode in which data is processed. For example, extract and process logs to convert unstructured information into structured information. For details about the filter parameter and parameter usage, visit the following website: https://www.elastic.co/guide/en/logstash/current/filter-plugins.html
- The output parameter indicates the destination address of the data. For details about the output parameter and parameter usage, visit https://www.elastic.co/guide/en/logstash/current/output-plugins.html. Replace <EIP address of the jump host> with the IP address (with EIP) of the created jump host in the IP Address column in the ECS list on the ECS management console. <Number of the port assigned external network access permissions on the jump host> is the number of the port obtained in 1, for example, 9200.
-Consider the data files in the /tmp/access_log/ path mentioned in 4 as an example. Assume that data import starts from data in the first row of the data file, the filtering condition is left unspecified (indicating no data processing operations are performed), the public IP address and port number of the jump host are 192.168.0.227 and 9200, respectively, and the name of the target index is myindex. Edit the configuration file as follows, and enter :wq to save the configuration file and exit.
-input {
+- The input parameter indicates the data source. Set this parameter based on the actual conditions. For details about the input parameter and parameter usage, visit the following website: https://www.elastic.co/guide/en/logstash/current/input-plugins.html
- The filter parameter specifies the mode in which data is processed. For example, extract and process logs to convert unstructured information into structured information. For details about the filter parameter and parameter usage, visit the following website: https://www.elastic.co/guide/en/logstash/current/filter-plugins.html
- The output parameter indicates the destination address of the data. For details about the output parameter and parameter usage, visit https://www.elastic.co/guide/en/logstash/current/output-plugins.html. Replace <EIP address of the jump host> with the IP address (with EIP) of the created jump host in the IP Address column in the ECS list on the ECS management console. <Number of the port assigned external network access permissions on the jump host> is the number of the port obtained in 1, for example, 9200.
+Consider the data files in the /tmp/access_log/ path mentioned in 4 as an example. Assume that data import starts from data in the first row of the data file, the filtering condition is left unspecified (indicating no data processing operations are performed), the public IP address and port number of the jump host are 192.168.0.227 and 9200, respectively, and the name of the target index is myindex. Edit the configuration file as follows, and enter :wq to save the configuration file and exit.
+input {
file{
path => "/tmp/access_log/*"
start_position => "beginning"
@@ -64,11 +59,10 @@ output {
}
}
- If a license error is reported, set ilm_enabled to false.
+
If a license error is reported, set ilm_enabled to false to try and rectify the error.
-
If the cluster has the security mode enabled, you need to download a certificate first.
-
- Log in to the CSS management console.
- On the Clusters page, click the name of the cluster for which you want to download a certificate. The Basic Information page is displayed.
- Download a certificate on the Basic Information page of the cluster.
Figure 2 Downloading a certificate
- - Store the certificate to the server where Logstash is deployed.
- Modify the logstash-simple.conf configuration file.
Consider the data files in the
/tmp/access_log/ path mentioned in
4 as an example. Assume that data import starts from data in the first row of the data file, the filtering condition is left unspecified (indicating no data processing operations are performed), and the public IP address and port number of the jump host are
192.168.0.227 and
9200, respectively. The name of the index for importing data is
myindex, and the certificate is stored in
/logstash/logstash6.8/config/CloudSearchService.cer. Edit the configuration file as follows, and enter
:wq to save the configuration file and exit.
input{
+If the cluster has the security mode enabled, you need to download a certificate first.
+- Obtaining the Security Certificate.
- Store the downloaded certificate to the server where Logstash is deployed.
- Modify the logstash-simple.conf configuration file.
Consider the data files in the
/tmp/access_log/ path mentioned in
4 as an example. Assume that data import starts from data in the first row of the data file, the filtering condition is left unspecified (indicating no data processing operations are performed), and the public IP address and port number of the jump host are
192.168.0.227 and
9200, respectively. The name of the index for importing data is
myindex, and the certificate is stored in
/logstash/config/CloudSearchService.cer. Edit the configuration file as follows, and enter
:wq to save the configuration file and exit.
input{
file {
path => "/tmp/access_log/*"
start_position => "beginning"
@@ -80,31 +74,27 @@ output{
elasticsearch{
hosts => ["https://192.168.0.227:9200"]
index => "myindex"
- user => "admin"
- password => "******"
- cacert => "/logstash/logstash6.8/config/CloudSearchService.cer"
+ user => "admin" # Username for accessing the security-mode cluster
+ password => "******" # Password for accessing the security-mode cluster
+ cacert => "/logstash/config/CloudSearchService.cer"
manager_template => false
ilm_enabled => false
ssl => true
ssl_certificate_verification => false
}
}
-
password: password for logging in to the cluster
-
-- Run the following command to import the data collected by Logstash to the cluster:
./bin/logstash -f logstash-simple.conf
- This command must be executed in the directory where the logstash-simple.conf file is stored. For example, if the logstash-simple.conf file is stored in /root/logstash-7.1.1/, go to the directory before running the command.
-
- - Log in to the CSS management console.
- In the navigation pane on the left, expand Clusters and select a cluster type. A cluster list is displayed.
- From the cluster list, locate the row that contains the cluster to which you want to import data and click Access Kibana in the Operation column.
- In the Kibana navigation pane on the left, choose Dev Tools.
- On the Console page of Kibana, search for the imported data.
On the Console page of Kibana, run the following command to search for data. View the search results. If the searched data is consistent with the imported data, the data has been imported successfully.
-GET myindex/_search
+ - Run the following command to import the data collected by Logstash to the cluster:
./bin/logstash -f logstash-simple.conf
+This command must be executed in the directory where the logstash-simple.conf file is located. For example, if the logstash-simple.conf file is stored in /root/logstash-7.1.1/, navigate to this directory before executing the command.
+ - Log in to the CSS management console.
- In the navigation pane on the left, choose Clusters > Elasticsearch.
- In the cluster list, find the target cluster, and click Kibana in the Operation column to log in to the Kibana console.
- In the left navigation pane, choose Dev Tools.
- On the Kibana console, search for the ingested data.
Run the following command to search for data. View the search results. If the searched data is consistent with the imported data, the data has been imported successfully.
+GET myindex/_search
Importing Data When Logstash Is Deployed on an ECS
Figure 3 illustrates how data is imported when Logstash is deployed on an ECS that resides in the same VPC as the cluster to which data is to be imported.
-
Figure 3 Importing data when Logstash is deployed on an ECS
-
- Ensure that the ECS where Logstash is deployed and the cluster to which data is to be imported reside in the same VPC, port 9200 of the ECS security group has been assigned external network access permissions, and an EIP has been bound to the ECS.
- If there are multiple servers in a VPC, you do not need to associate EIPs to other servers as long as one server is associated with an EIP. Switch to the node where Logstash is deployed from the node with which the EIP is associated.
- If a private line or VPN is available, you do not need to associate an EIP.
-
- - Use PuTTY to log in to the ECS.
For example, data file
access_20181029_log is stored in the
/tmp/access_log/ path of the ECS, and the data file includes the following data:
| All | Heap used for segments | | 18.6403 | MB |
+Importing Data When Logstash Is Deployed on an ECS
Figure 2 illustrates how data is imported when Logstash is deployed on an ECS that resides in the same VPC as the cluster to which data is to be imported.
+
Figure 2 Importing data when Logstash is deployed on an ECS
+
- Ensure that the ECS where Logstash is deployed and the cluster to which data is to be imported reside in the same VPC, port 9200 of the ECS security group has been assigned external network access permissions, and an EIP has been bound to the ECS.
- If there are multiple servers in a VPC, you only need to associate an EIP with one of these servers. Switch to the node where Logstash is deployed from the node with which the EIP is associated.
- If a private line or VPN is available, there is no need for an EIP.
+ - Use PuTTY to log in to the ECS.
For example, data file
access_20181029_log is stored in the
/tmp/access_log/ path of the ECS, and the data file includes the following data:
| All | Heap used for segments | | 18.6403 | MB |
| All | Heap used for doc values | | 0.119289 | MB |
| All | Heap used for terms | | 17.4095 | MB |
| All | Heap used for norms | | 0.0767822 | MB |
@@ -116,29 +106,29 @@ output{
| All | Max Throughput | index-append | 67745.6 | docs/s |
| All | 50th percentile latency | index-append | 510.261 | ms |
- Run the following command to create configuration file logstash-simple.conf in the Logstash installation directory:
cd /<Logstash installation directory>/
+
- Run the following command to create configuration file logstash-simple.conf in the Logstash installation directory:
cd /<Logstash installation directory>/
vi logstash-simple.conf
-Input the following content in
logstash-simple.conf:
input {
-Location of data
+Enter the following content in
logstash-simple.conf:
input {
+Location of data
}
filter {
-Related data processing
+Related data processing
}
output {
elasticsearch{
- hosts => "<Private network address and port number of the node>"}
- (Optional) If communication encryption has been enabled on the cluster, you need to add the following configuration:
+ hosts => "<Private network address and port number of the node>"}
+ (Optional) If communication encryption has been enabled for the cluster, you need to add the following configuration:
ssl => true
ssl_certificate_verification => false
}
-
- The input parameter indicates the data source. Set this parameter based on the actual conditions. For details about the input parameter and parameter usage, visit the following website: https://www.elastic.co/guide/en/logstash/current/input-plugins.html
- The filter parameter specifies the mode in which data is processed. For example, extract and process logs to convert unstructured information into structured information. For details about the filter parameter and parameter usage, visit the following website: https://www.elastic.co/guide/en/logstash/current/filter-plugins.html
- The output parameter indicates the destination address of the data. For details about the output parameter and parameter usage, visit https://www.elastic.co/guide/en/logstash/current/output-plugins.html. <private network address and port number of a node> refers to the private network address and port number of a node in the cluster.
If the cluster contains multiple nodes, you are advised to replace the value of <Private network address and port number of a node> with the private network addresses and port numbers of all nodes in the cluster to prevent node faults. Use commas (,) to separate the nodes' private network addresses and port numbers. The following is an example:
-hosts => ["192.168.0.81:9200","192.168.0.24:9200"]
-If the cluster contains only one node, the format is as follows:
-hosts => "192.168.0.81:9200"
+- The input parameter indicates the data source. Set this parameter based on the actual conditions. For details about the input parameter and parameter usage, visit the following website: https://www.elastic.co/guide/en/logstash/current/input-plugins.html
- The filter parameter specifies the mode in which data is processed. For example, extract and process logs to convert unstructured information into structured information. For details about the filter parameter and parameter usage, visit the following website: https://www.elastic.co/guide/en/logstash/current/filter-plugins.html
- The output parameter indicates the destination address of the data. For details about the output parameter and parameter usage, visit https://www.elastic.co/guide/en/logstash/current/output-plugins.html. <private network address and port number of a node> refers to the private network address and port number of a node in the cluster.
If the cluster contains multiple nodes, you are advised to replace the value of <Private network address and port number of a node> with the private network addresses and port numbers of all nodes in the cluster to prevent node faults. Use commas (,) to separate the nodes' private network addresses and port numbers. The following is an example:
+hosts => ["192.168.0.81:9200","192.168.0.24:9200"]
+If the cluster contains only one node, the format is as follows:
+hosts => "192.168.0.81:9200"
Consider the data files in the /tmp/access_log/ path mentioned in 2 as an example. Assume that data import starts from data in the first row of the data file, the filtering condition is left unspecified (indicating no data processing operations are performed), the private network address and port number of the node in the cluster where data is to be imported are 192.168.0.81 and 9200, respectively, and the name of the target index is myindex. Edit the configuration file as follows, and enter :wq to save the configuration file and exit.
-input {
+Consider the data files in the /tmp/access_log/ path mentioned in 2 as an example. Assume that data import starts from data in the first row of the data file, the filtering condition is left unspecified (indicating no data processing operations are performed), the private network address and port number of the node in the cluster where data is to be imported are 192.168.0.81 and 9200, respectively, and the name of the target index is myindex. Edit the configuration file as follows, and enter :wq to save the configuration file and exit.
+input {
file{
path => "/tmp/access_log/*"
start_position => "beginning"
@@ -153,9 +143,8 @@ output {
}
}
-If the cluster has the security mode enabled, you need to download a certificate first.
-- Download a certificate on the Basic Information page of the cluster.
Figure 4 Downloading a certificate
- - Store the certificate to the server where Logstash is deployed.
- Modify the logstash-simple.conf configuration file.
Consider the data files in the
/tmp/access_log/ path mentioned in step
2 as an example. Assume that data import starts from data in the first row of the data file, the filtering condition is left unspecified (indicating no data processing operations are performed), the public IP address and port number of the jump host are
192.168.0.227 and
9200, respectively. The name of the index for importing data is
myindex, and the certificate is stored in
/logstash/logstash6.8/config/CloudSearchService.cer. Edit the configuration file as follows, and enter
:wq to save the configuration file and exit.
input{
+If the cluster has the security mode enabled, you need to download a certificate first.
- Obtaining the Security Certificate.
- Store the downloaded certificate to the server where Logstash is deployed.
- Modify the logstash-simple.conf configuration file.
Consider the data files in the /tmp/access_log/ path mentioned in step 2 as an example. Assume that data import starts from data in the first row of the data file, the filtering condition is left unspecified (indicating no data processing operations are performed), the public IP address and port number of the jump host are 192.168.0.227 and 9200, respectively. The name of the index for importing data is myindex, and the certificate is stored in /logstash/config/CloudSearchService.cer. Edit the configuration file as follows, and enter :wq to save the configuration file and exit.
+input{
file {
path => "/tmp/access_log/*"
start_position => "beginning"
@@ -167,28 +156,30 @@ output{
elasticsearch{
hosts => ["https://192.168.0.227:9200"]
index => "myindex"
- user => "admin"
- password => "******"
- cacert => "/logstash/logstash6.8/config/CloudSearchService.cer"
+ user => "admin" # Username for accessing the security-mode cluster
+ password => "******" # Password for accessing the security-mode cluster
+ cacert => "/logstash/config/CloudSearchService.cer"
manager_template => false
ilm_enabled => false
ssl => true
ssl_certificate_verification => false
}
}
- password: password for logging in to the cluster
-
-
- Run the following command to import the ECS data collected by Logstash to the cluster:
./bin/logstash -f logstash-simple.conf
- - Log in to the CSS management console.
- In the navigation pane on the left, expand Clusters and select a cluster type. A cluster list is displayed.
- From the cluster list, locate the row that contains the cluster to which you want to import data and click Access Kibana in the Operation column.
- In the Kibana navigation pane on the left, choose Dev Tools.
- On the Console page of Kibana, search for the imported data.
On the Console page of Kibana, run the following command to search for data. View the search results. If the searched data is consistent with the imported data, the data has been imported successfully.
-GET myindex/_search
+ - Run the following command to import the ECS data collected by Logstash to the cluster:
./bin/logstash -f logstash-simple.conf
+ - Log in to the CSS management console.
- In the navigation pane on the left, choose Clusters > Elasticsearch.
- In the cluster list, find the target cluster, and click Kibana in the Operation column to log in to the Kibana console.
- In the left navigation pane, choose Dev Tools.
- On the Kibana console, search for the ingested data.
Run the following command to search for data. View the search results. If the searched data is consistent with the imported data, the data has been imported successfully.
+GET myindex/_search
+
+Obtaining the Security Certificate
To access a security-mode Elasticsearch cluster that uses HTTPS, a security certificate must be loaded. To obtain this security certificate (CloudSearchService.cer), follow these steps:
+
- Log in to the CSS management console.
- In the navigation pane on the left, choose Clusters > Elasticsearch.
- In the cluster list, click the name of the target cluster. The cluster information page is displayed.
- Click the Overview tab. In the Configuration area, click Download Certificate next to HTTPS Access.
Figure 3 Downloading a security certificate
Configuring Read/Write Splitting Between Two OpenSearch Clusters
+As business expands, and data volumes and access requests grow exponentially, a monolithic architecture where a single cluster handles both write and query requests faces typical challenges like resource contention and overload. To address these challenges, CSS introduces read/write splitting between OpenSearch clusters.
+
OpenSearch read/write splitting works by having leader and follower clusters collaborate together. The feature delivers the following benefits:
+
- Decoupled read and write loads: The leader cluster ensures data ingestion performance, while the follower clusters deliver scalable, high-concurrency query performance. There is no more resource contention, and peak loads are reduced.
- Flexible scalability: The write and query clusters can be scaled horizontally and independently. Cross-region cluster deployment is supported.
- Data consistency guarantee: Data can be synchronized in real time, with low latency. Incremental synchronization is also supported.
+
How the Feature Works
Figure 1 Read/write splitting
+
How read/write splitting between OpenSearch clusters works:
+
- Data writes: Users send write requests, and the leader cluster handles these requests.
- Data synchronization.
The leader synchronizes data changes to the follower through a REST API. Two synchronization methods are supported:
- Exact match: synchronization of a specified index
- Match by wildcard: batch synchronization of indexes matching a wildcard expression
+
- Query processing: Users send query requests. The follower cluster handles these requests and returns the results.
+
Figure 2 illustrates how read/write splitting works.
+
Figure 2 How read/write splitting works
+
- Read and write split when both clusters are available (left): The leader handles writes, and the follower handles queries.
- Leader-to-follower switchover when the leader fails (right): If the leader becomes unavailable, the follower automatically upgrades to leader to ensure service continuity. For details, see Switching the Roles of the Leader and Follower Clusters.
+
+
Constraints
- Only OpenSearch 2.19.0 clusters support read/write splitting.
- The leader and follower clusters must use the same software version.
+
+
Prerequisites
Two clusters of the same version have been created. One functions as the leader cluster, and the other the follower cluster. The follower cluster must be able to access the REST API (default port: 9200) of the leader cluster.
+
+
Logging In to OpenSearch Dashboards
Log in to OpenSearch Dashboards and go to the command execution page. OpenSearch clusters support multiple access methods. This topic uses OpenSearch Dashboards as an example to describe the operation procedures.
+
- Log in to the CSS management console.
- In the navigation pane on the left, choose Clusters > OpenSearch.
- In the cluster list, find the target cluster, and click Dashboards in the Operation column to log in to OpenSearch Dashboards.
- In the left navigation pane, choose Dev Tools.
The left part of the console is the command input box, and the triangle icon in its upper-right corner is the execution button. The right part shows the execution result.
+
+
+
Connecting the Leader and Follower Clusters
- Run the following command to configure information about the leader cluster in the follower cluster:
PUT /_cluster/settings
+{
+ "persistent" : {
+ "cluster" : {
+ "remote.rest" : {
+ "leader1" : {
+ "seeds" : [
+ "http://10.0.0.1:9200",
+ "http://10.0.0.2:9200",
+ "http://10.0.0.3:9200"
+ ] ,
+ "username": "elastic",
+ "password": "*****"
+ }
+ }
+ }
+ }
+}
+
+Table 1 Request body parametersParameter
+ |
+Description
+ |
+
|---|
+
+leader1
+ |
+Name of the leader cluster configuration task, which is user-defined and will be used for configuring read/write splitting later.
+ |
+
+seeds
+ |
+Address for accessing the leader cluster. When HTTPS is enabled for the cluster, the URL schema must use HTTPS.
+ |
+
+username
+ |
+Username of the leader cluster. This parameter is required only when security mode is enabled for the leader cluster.
+ |
+
+password
+ |
+Password of the leader cluster. This parameter is required only when security mode is enabled for the leader cluster.
+ |
+
+
+
+
Example response:
+{
+ "acknowledged" : true, //Whether the operation is successful
+ "persistent" : {
+ "cluster" : {
+ "remote" : {
+ "rest" : {
+ "leader1" : {
+ "seeds" : [
+ "http://10.0.0.1:9200",
+ "http://10.0.0.2:9200",
+ "http://10.0.0.3:9200"
+ ] ,
+ "username": "elastic",
+ "password": "*****"
+ }
+ }
+ }
+ }
+ },
+ "transient" : { }
+}
+ - After the configuration is complete, run the following command in the follower cluster to check the connection between the follower and leader clusters:
GET _remote/rest/info
+Example response:
+{
+ "leader1" : {
+ "connected" : true //The two clusters are connected.
+ }
+}
+
+
+
Index Synchronization
There are two ways to synchronize indexes: synchronization of a specified index and synchronization of indexes matching a wildcard expression.
+
During synchronization, indexes in the follower cluster become read-only. The synchronization is performed periodically. The default synchronization interval is 30 seconds. For how to change it, see Changing the Synchronization Interval.
+
The following index configuration items cannot be modified: number_of_shards, version.created, uuid, creation_date, and soft_deletes.enabled.
+
+
Method 1: synchronizing a specified index
+
- Run the following command in the follower cluster to synchronize a single index from the leader cluster to the follower cluster without modifying index settings:
PUT start_remote_sync
+{
+ "remote_cluster": "leader1",
+ "remote_index": "data1_leader",
+ "local_index": "data1_follower"
+}
+ - Run the following command in the follower cluster to synchronize a single index from the leader cluster to the follower cluster while modifying some of the index settings—enabling synchronization of index settings:
PUT start_remote_sync
+{
+ "remote_cluster": "leader1",
+ "remote_index": "data1_leader",
+ "local_index": "data1_follower",
+ "settings": {
+ "number_of_replicas": 4
+ },
+ "settings_sync_enable": true,
+ "settings_sync_patterns": ["*"],
+ "settings_sync_exclude_patterns": ["index.routing.allocation.*"],
+ "alias_sync_enable": true,
+ "state_sync_enable": true
+}
+
+Table 2 Request body parametersParameter
+ |
+Description
+ |
+
|---|
+
+remote_cluster
+ |
+Name of the leader cluster configuration task, which was set in Connecting the Leader and Follower Clusters. leader1 was set in our example.
+ |
+
+remote_index
+ |
+Name of the index to be synchronized in the leader cluster
+ |
+
+local_index
+ |
+Index name in the follower cluster
+ |
+
+settings
+ |
+Index settings to be synchronized
+ |
+
+settings_sync_enable
+ |
+Whether to enable synchronization of index settings in the leader cluster. The default value is false.
+ |
+
+settings_sync_patterns
+ |
+Prefix of leader cluster index settings to be synchronized. The default value is *. This parameter takes effect when settings_sync_enable is set to true. The index settings configured in settings will not be synchronized.
+ |
+
+settings_sync_exclude_patterns
+ |
+Prefix of leader cluster index settings not to be synchronized. The default value is empty. This parameter is valid only when settings_sync_enable is set to true.
+ |
+
+alias_sync_enable
+ |
+Whether to enable index alias synchronization in the leader cluster. The default value is false.
+ |
+
+state_sync_enable
+ |
+Whether to enable index status synchronization in the leader cluster. The default value is false.
+ |
+
+
+
+
+
Method 2: synchronizing all indexes matching a wildcard expression
+
- Run the following command in the follower cluster to create a pattern-matching index synchronization policy, which synchronizes matched indexes from the leader cluster to the follower cluster:
PUT auto_sync/pattern/${PATTERN}
+{
+ "remote_cluster": "leader1",
+ "remote_index_patterns": "log*",
+ "local_index_pattern": "{{remote_index}}-sync",
+ "apply_exist_index": true
+}
+ - Run the following command in the follower cluster to create a pattern-matching index synchronization policy, which synchronizes matched indexes from the leader cluster to the follower cluster, with some of the index settings modified—enabling synchronization of index settings:
PUT auto_sync/pattern/${PATTERN}
+{
+ "remote_cluster": "leader1",
+ "remote_index_patterns": "log*",
+ "local_index_pattern": "{{remote_index}}-sync",
+ "apply_exist_index": true,
+ "settings": {
+ "number_of_replicas": 4
+ },
+ "settings_sync_enable": true,
+ "settings_sync_patterns": ["*"],
+ "settings_sync_exclude_patterns": ["index.routing.allocation.*"],
+ "alias_sync_enable": true,
+ "state_sync_enable": true
+}
+
+Table 3 Request body parametersParameter
+ |
+Description
+ |
+
|---|
+
+PATTERN
+ |
+Name of the pattern for index matching.
+ |
+
+remote_cluster
+ |
+Name of the leader cluster configuration task, which was set in Connecting the Leader and Follower Clusters. In our example, leader1 is used.
+ |
+
+remote_index_patterns
+ |
+Pattern for matching indexes to be synchronized in the leader cluster. The wildcard (*) is supported.
+ |
+
+local_index_pattern
+ |
+Index pattern in the follower cluster. The index template can be replaced. For example, if this parameter is set to {{remote_index}}-sync, the index log1 changes to log1-sync after synchronization.
+If leader/follower switchover is likely to happen, set this parameter to {{remote_index}} to ensure that the leader and follower clusters use identical index names.
+ |
+
+apply_exist_index
+ |
+Whether to synchronize existing indexes in the leader cluster. The default value is true.
+ |
+
+settings
+ |
+Index settings to be synchronized
+ |
+
+settings_sync_enable
+ |
+Whether to enable synchronization of index settings in the leader cluster. The default value is false.
+ |
+
+settings_sync_patterns
+ |
+Prefix of leader cluster index settings to be synchronized. The default value is *. This parameter takes effect when settings_sync_enable is set to true. The index settings configured in settings will not be synchronized.
+ |
+
+settings_sync_exclude_patterns
+ |
+Prefix of leader cluster index settings not to be synchronized. The default value is empty. This parameter is valid only when settings_sync_enable is set to true.
+ |
+
+alias_sync_enable
+ |
+Whether to enable index alias synchronization in the leader cluster. The default value is false.
+ |
+
+state_sync_enable
+ |
+Whether to enable index status synchronization in the leader cluster. The default value is false.
+ |
+
+
+
+
+
+
Stopping Index Synchronization
Run the following command in the follower cluster to stop synchronization tasks for specified indexes. Subsequent changes to the indexes in the leader cluster will not be synchronized to the follower cluster. The read-only state of the indexes in the follower cluster will be cancelled, so that new data can be written into them.
+
PUT log*/stop_remote_sync
+
In this command, log* indicates the index name. You can specify multiple index names (separated by commas) or use a wildcard. In this example, synchronization tasks for all indexes that start with log are stopped.
+
+
Querying and Deleting Created Patterns
- Run the following command in the follower cluster to query created patterns:
+
The following is an example of the response:
+{
+ "patterns" : [
+ {
+ "name" : "pattern1",
+ "pattern" : {
+ "remote_cluster" : "leader",
+ "remote_index_patterns" : [
+ "log*"
+ ],
+ "local_index_pattern" : "{{remote_index}}-sync",
+ "settings" : { }
+ }
+ }
+ ]
+}
+ - Run the following command in the follower cluster to delete a specified pattern by name:
DELETE auto_sync/pattern/{PATTERN}
+
+
+
Enabling Forcible Synchronization
By default, the plug-in determines whether to synchronize metadata based on whether the number of documents in the index of the leader cluster changes. If the leader cluster only updates documents and the number of documents remains unchanged, the plug-in does not synchronize the updates to the follower cluster. The configuration can be modified. After forcible synchronization is enabled, the index metadata of the leader cluster is forcibly synchronized to the follower cluster in each synchronization cycle.
+
The following is an example of enabling forcible synchronization:
+
PUT _cluster/settings
+{
+ "persistent": {
+ "remote_sync.force_synchronize": true
+ }
+}
+
+
Changing the Synchronization Interval
The default synchronization interval between the leader and follower clusters is 30 seconds and can be changed.
+
The example request below changes the synchronization interval to 2 seconds:
+
PUT {index_name}/_settings
+{
+ "index.remote_sync.sync_interval": "2s"
+}
+
+
Changing the Synchronization Speed
You can change the data synchronization speed between the leader and follower clusters by configuring cluster-level settings.
+
The following is an example request: set the block size to 2 MB, the number of blocks to 20, and the maximum synchronization traffic to 100 MB per second.
+
PUT _cluster/settings
+{
+ "persistent": {
+ "remote_sync.chunk_size": "2MB",
+ "remote_sync.max_concurrent_file_chunks": 20,
+ "remote_sync.max_bytes_per_sec": "100MB"
+ }
+}
+
+
Table 4 Request body parametersParameter
+ |
+Description
+ |
+
|---|
+
+remote_sync.chunk_size
+ |
+Block size for index synchronization.
+Default value: 1 MB
+Format: a string of characters
+ |
+
+remote_sync.max_concurrent_file_chunks
+ |
+Number of blocks for concurrent index synchronization.
+Default value: 10
+Format: a number
+ |
+
+remote_sync.max_bytes_per_sec
+ |
+Maximum data synchronization traffic per second.
+Default value: 40 MB
+Format: a string of characters
+ |
+
+
+
+
+
+
Querying Index Synchronization Status
Obtain the auto synchronization status of a specified index.
+
An example request is as follows:
+
GET {index_name}/sync_stats
+
The following is an example of the response:
+
{
+ "indices" : {
+ "data1_follower" : {
+ "shards" : {
+ "0" : [
+ {
+ "primary" : false, // Whether it is a primary shard
+ "total_synced_times" : 27, // Total synchronization times
+ "total_empty_times" : 25, // Total number of times when no data is synchronized between the leader and follower clusters because they have identical shards and data
+ "total_synced_files" : 4, // Number of synchronized files
+ "total_synced_bytes" : 3580, // Total size of synchronized files
+ "total_paused_nanos" : 0, //Duration of synchronization pauses due to traffic throttling
+ "total_paused_times" : 0, //Number of synchronization pauses due to traffic throttling
+ "current" : {
+ "files_count" : 0, //Number of files that are being synchronized
+ "finished_files_count" : 0, //Number of files that have been synchronized
+ "bytes" : 0, //Size of files that are being synchronized
+ "finished_bytes" : 0 //Size of files that have been synchronized
+ }
+ },
+ {
+ "primary" : true, // Whether it is a primary shard
+ "total_synced_times" : 28, // Total synchronization times
+ "total_empty_times" : 26, // Total number of times when no data is synchronized between the leader and follower clusters because they have identical shards and data
+ "total_synced_files": 20, // Number of synchronized files
+ "total_synced_bytes": 17547, // Total size of synchronized files
+ "total_paused_nanos" : 0, //Duration of synchronization pauses due to traffic throttling
+ "total_paused_times" : 0, //Number of synchronization pauses due to traffic throttling
+ "current" : {
+ "files_count" : 0, //Number of files that are being synchronized
+ "finished_files_count" : 0, //Number of files that have been synchronized
+ "bytes" : 0, //Size of files that are being synchronized
+ "finished_bytes" : 0 //Size of files that have been synchronized
+ }
+ }
+ ]
+ }
+ }
+ }
+}
+
+
Switching the Roles of the Leader and Follower Clusters
When the leader cluster becomes faulty, perform a leader/follower switchover to have the follower cluster take over services. The steps are as follows:
+
Leader/follower switchover won't work unless the leader and follower clusters use identical index names.
+
+
- Determine the index synchronization method between the leader and follower clusters. Check whether pattern-matching index synchronization policies have been configured in the follower cluster. For the command to use, see Querying and Deleting Created Patterns.
- If there are no such policies, synchronization is performed for specified indexes between the leader and follower clusters. In this case, go to 3.
- If there are such policies, index synchronization between the leader and follower clusters is based on index patterns. In this case, go to 2.
+ - Delete pattern-matching index synchronization policies in the follower cluster. For the command to use, see Querying and Deleting Created Patterns.
- Perform Stopping Index Synchronization in the follower cluster. Then redirect read and write traffic to it. If the leader and follower clusters synchronize indexes based on index patterns, use a wildcard to match indexes when running the command that stops index synchronization.
- After the leader cluster recovers, configure information about the follower cluster in the leader cluster, and connect the leader and follower clusters again. For details, see Connecting the Leader and Follower Clusters.
- Under the leader cluster, perform Index Synchronization to synchronize data from the follower cluster to the leader cluster, and then perform a leader/follower switchover to switch back.
+
+
Viewing Elasticsearch Cluster Audit Logs
-Cloud Trace Service (CTS) is available on the public cloud platform. With CTS, you can record operations associated with CSS for later query, audit, and backtrack operations.
-
Prerequisites
CTS has been enabled. For details, see Enabling CTS.
+
Cloud Trace Service (CTS) is available on the public cloud platform. With CTS, you can record operations associated with CSS for later query, audit, and backtrack operations.
+
Prerequisites
CTS has been enabled. For details, see Enabling CTS.
-
Key Operations Recorded by CTS
-
Table 1 Key operations recorded by CTSOperation
+Key Operations Recorded by CTS
+ Table 1 Key operations recorded by CTSOperation
|
-Resource Type
+ | Resource Type
|
-Event Name
+ | Event Name
|
|---|
-Creating a cluster
+ | Creating a cluster
|
-cluster
+ | cluster
|
-createCluster
+ | createCluster
|
-Querying cluster details
+ | Querying cluster details
|
-cluster
+ | cluster
|
-showClusterDetail
+ | showClusterDetail
|
-Deleting a cluster
+ | Deleting a cluster
|
-cluster
+ | cluster
|
-deleteCluster
+ | deleteCluster
|
-Changing a cluster name
+ | Changing a cluster name
|
-cluster
+ | cluster
|
-updateClusterName
+ | updateClusterName
|
-Querying the cluster list
+ | Querying the cluster list
|
-cluster
+ | cluster
|
-listClusters
+ | listClusters
|
-Changing the password of a cluster
+ | Changing the password of a cluster
|
-cluster
+ | cluster
|
-resetPassword
+ | resetPassword
|
-Restarting a cluster
+ | Restarting a cluster
|
-cluster
+ | cluster
|
-restartCluster
+ | restartCluster
|
-Expanding cluster capacity
+ | Expanding cluster capacity
|
-cluster
+ | cluster
|
-updateExtendCluster
+ | updateExtendCluster
|
-Adding instances and expanding storage
+ | Adding instances and expanding storage
|
-cluster
+ | cluster
|
-updateExtendInstanceStorage
+ | updateExtendInstanceStorage
|
-Changing specifications
+ | Changing specifications
|
-cluster
+ | cluster
|
-updateFlavor
+ | updateFlavor
|
-Obtaining the instance specifications list
+ | Obtaining the instance specifications list
|
-cluster
+ | cluster
|
-listFlavors
+ | listFlavors
|
-Removing specified nodes
+ | Removing specified nodes
|
-cluster
+ | cluster
|
-updateShrinkNodes
+ | updateShrinkNodes
|
-Changing the specifications of a specified node type
+ | Changing the specifications of a specified node type
|
-cluster
+ | cluster
|
-updateFlavorByType
+ | updateFlavorByType
|
-Scaling in nodes of a specific type
+ | Scaling in nodes of a specific type
|
-cluster
+ | cluster
|
-updateShrinkCluster
+ | updateShrinkCluster
|
-Downloading a security certificate
+ | Downloading a security certificate
|
-cluster
+ | cluster
|
-downloadCert
+ | downloadCert
|
-Replacing a node
+ | Replacing a node
|
-cluster
+ | cluster
|
-updateInstance
+ | updateInstance
|
-Changing the security mode
+ | Changing the security mode
|
-cluster
+ | cluster
|
-changeMode
+ | changeMode
|
-Adding a dedicated Master or Client node
+ | Adding a dedicated Master or Client node
|
-cluster
+ | cluster
|
-addIndependentNode
+ | addIndependentNode
|
-Upgrading the cluster kernel
+ | Upgrading the cluster kernel
|
-cluster
+ | cluster
|
-upgradeCore
+ | upgradeCore
|
-Obtaining a target image ID
+ | Obtaining a target image ID
|
-cluster
+ | cluster
|
-listImages
+ | listImages
|
-Obtaining upgrade details
+ | Obtaining upgrade details
|
-cluster
+ | cluster
|
-upgradeDetail
+ | upgradeDetail
|
-Retrying a failed upgrade task
+ | Retrying a failed upgrade task
|
-cluster
+ | cluster
|
-retryUpgradeTask
+ | retryUpgradeTask
|
-Changing the security group
+ | Changing the security group
|
-cluster
+ | cluster
|
-changeSecurityGroup
+ | changeSecurityGroup
|
-Creating a V2 cluster
+ | Creating a V2 cluster
|
-cluster
+ | cluster
|
-createClusterV2
+ | createClusterV2
|
-Restarting a V2 cluster
+ | Restarting a V2 cluster
|
-cluster
+ | cluster
|
-restartCluster
+ | restartCluster
|
-Rolling restart
+ | Rolling restart
|
-cluster
+ | cluster
|
-rollingRestart
+ | rollingRestart
|
-Enabling Kibana public access
+ | Enabling Kibana public access
|
-cluster
+ | cluster
|
-startKibanaPublic
+ | startKibanaPublic
|
-Disabling Kibana public access
+ | Disabling Kibana public access
|
-cluster
+ | cluster
|
-updateCloseKibana
+ | updateCloseKibana
|
-Modifying the Kibana public network bandwidth
+ | Modifying the Kibana public network bandwidth
|
-cluster
+ | cluster
|
-updateAlterKibana
+ | updateAlterKibana
|
-Modifying Kibana public network access control
+ | Modifying Kibana public network access control
|
-cluster
+ | cluster
|
-updatePublicKibanaWhitelist
+ | updatePublicKibanaWhitelist
|
-Disabling Kibana public network access control
+ | Disabling Kibana public network access control
|
-cluster
+ | cluster
|
-stopPublicKibanaWhitelist
+ | stopPublicKibanaWhitelist
|
-Enabling logging
+ | Enabling logging
|
-cluster
+ | cluster
|
-startLogs
+ | startLogs
|
-Disabling logging
+ | Disabling logging
|
-cluster
+ | cluster
|
-stopLogs
+ | stopLogs
|
-Querying the job list
+ | Querying the job list
|
-cluster
+ | cluster
|
-listLogsJob
+ | listLogsJob
|
-Querying basic log configurations
+ | Querying basic log configurations
|
-cluster
+ | cluster
|
-showGetLogSetting
+ | showGetLogSetting
|
-Modifying basic log configurations
+ | Modifying basic log configurations
|
-cluster
+ | cluster
|
-updateLogSetting
+ | updateLogSetting
|
-Enabling automatic log backup
+ | Enabling automatic log backup
|
-cluster
+ | cluster
|
-startLogAutoBackupPolicy
+ | startLogAutoBackupPolicy
|
-Disabling automatic log backup
+ | Disabling automatic log backup
|
-cluster
+ | cluster
|
-stopLogAutoBackupPolicy
+ | stopLogAutoBackupPolicy
|
-Backing up logs
+ | Backing up logs
|
-cluster
+ | cluster
|
-createLogBackup
+ | createLogBackup
|
-Querying logs
+ | Querying logs
|
-cluster
+ | cluster
|
-showLogBackup
+ | showLogBackup
|
-Enabling public network access
+ | Enabling public network access
|
-cluster
+ | cluster
|
-createBindPublic
+ | createBindPublic
|
-Disabling public access
+ | Disabling public access
|
-cluster
+ | cluster
|
-updateUnbindPublic
+ | updateUnbindPublic
|
-Modifying public network access bandwidth
+ | Modifying public network access bandwidth
|
-cluster
+ | cluster
|
-updatePublicBandWidth
+ | updatePublicBandWidth
|
-Enabling the public network access whitelist
+ | Enabling the public network access whitelist
|
-cluster
+ | cluster
|
-startPublicWhitelist
+ | startPublicWhitelist
|
-Disabling the public network access whitelist
+ | Disabling the public network access whitelist
|
-cluster
+ | cluster
|
-stopPublicWhitelist
+ | stopPublicWhitelist
|
-Automatically setting basic configurations of a cluster snapshot
+ | Automatically setting basic configurations of a cluster snapshot
|
-cluster
+ | cluster
|
-startAutoSetting
+ | startAutoSetting
|
-Modifying basic configurations of a cluster snapshot
+ | Modifying basic configurations of a cluster snapshot
|
-cluster
+ | cluster
|
-updateSnapshotSetting
+ | updateSnapshotSetting
|
-Manually creating a snapshot
+ | Manually creating a snapshot
|
-snapshot
+ | snapshot
|
-createSnapshot
+ | createSnapshot
|
-Restoring a snapshot
+ | Restoring a snapshot
|
-snapshot
+ | snapshot
|
-restoreSnapshot
+ | restoreSnapshot
|
-Deleting a specified snapshot
+ | Deleting a specified snapshot
|
-snapshot
+ | snapshot
|
-deleteSnapshot
+ | deleteSnapshot
|
-Setting an automatic snapshot creation policy
+ | Setting an automatic snapshot creation policy
|
-cluster
+ | cluster
|
-createAutoCreatePolicy
+ | createAutoCreatePolicy
|
-Querying automatic snapshot creation policies
+ | Querying automatic snapshot creation policies
|
-cluster
+ | cluster
|
-showAutoCreatePolicy
+ | showAutoCreatePolicy
|
-Querying the snapshot list
+ | Querying the snapshot list
|
-cluster
+ | cluster
|
-listSnapshots
+ | listSnapshots
|
-Disabling snapshot function
+ | Disabling snapshot function
|
-cluster
+ | cluster
|
-stopSnapshot
+ | stopSnapshot
|
-Enabling automatic snapshot creation
+ | Enabling automatic snapshot creation
|
-cluster
+ | cluster
|
-startAutoCreateSnapshots
+ | startAutoCreateSnapshots
|
-Disabling automatic snapshot creation
+ | Disabling automatic snapshot creation
|
-cluster
+ | cluster
|
-stopAutoCreateSnapshots
+ | stopAutoCreateSnapshots
|
-Enabling VPC Endpoint Service
+ | Enabling VPC Endpoint Service
|
-cluster
+ | cluster
|
-startVpecp
+ | startVpecp
|
-Disabling VPC Endpoint Service
+ | Disabling VPC Endpoint Service
|
-cluster
+ | cluster
|
-stopVpecp
+ | stopVpecp
|
-Obtaining a VPCEP connection
+ | Obtaining a VPCEP connection
|
-cluster
+ | cluster
|
-showVpcepConnection
+ | showVpcepConnection
|
-Updating a VPCEP connection
+ | Updating a VPCEP connection
|
-cluster
+ | cluster
|
-updateVpcepConnection
+ | updateVpcepConnection
|
-Modifying the VPCEP whitelist
+ | Modifying the VPCEP whitelist
|
-cluster
+ | cluster
|
-updateVpcepWhitelist
+ | updateVpcepWhitelist
|
-Modifying parameter settings
+ | Modifying parameter settings
|
-cluster
+ | cluster
|
-listYmls
+ | listYmls
|
-Obtaining the task list of parameter settings
+ | Obtaining the task list of parameter settings
|
-cluster
+ | cluster
|
-listYmlsJob
+ | listYmlsJob
|
-Obtaining the parameter settings list
+ | Obtaining the parameter settings list
|
-cluster
+ | cluster
|
-updateYmls
+ | updateYmls
|
-Querying ELB V3 load balancers supported by a cluster
+ | Querying ELB V3 load balancers supported by a cluster
|
-cluster
+ | cluster
|
-listElbs
+ | listElbs
|
-Enabling or disabling the Elasticsearch Load Balancer
+ | Enabling or disabling the Elasticsearch Load Balancer
|
-cluster
+ | cluster
|
-enableOrDisableElb
+ | enableOrDisableElb
|
-Configuring an Elasticsearch listener
+ | Configuring an Elasticsearch listener
|
-cluster
+ | cluster
|
-createElbListener
+ | createElbListener
|
-Obtaining information about an Elasticsearch ELB
+ | Obtaining information about an Elasticsearch ELB
|
-cluster
+ | cluster
|
-showElbDetail
+ | showElbDetail
|
-Updating an Elasticsearch listener
+ | Updating an Elasticsearch listener
|
-cluster
+ | cluster
|
-updateEsListener
+ | updateEsListener
|
-Querying the certificate list
+ | Querying the certificate list
|
-cluster
+ | cluster
|
-listElbCerts
+ | listElbCerts
|
-Obtaining the intelligent O&M task list and details
+ | Obtaining the intelligent O&M task list and details
|
-cluster
+ | cluster
|
-listAiOps
+ | listAiOps
|
-Creating a cluster detection task
+ | Creating a cluster detection task
|
-cluster
+ | cluster
|
-createAiOps
+ | createAiOps
|
-Deleting a detection task record
+ | Deleting a detection task record
|
-cluster
+ | cluster
|
-deleteAiOps
+ | deleteAiOps
|
-Obtaining SMN topics available for intelligent O&M alarms
+ | Obtaining SMN topics available for intelligent O&M alarms
|
-cluster
+ | cluster
|
-listSmnTopics
+ | listSmnTopics
|
Querying Real-Time TracesAfter a management tracker is created on the CTS console, the system starts recording operations performed on cloud service resources. After a data tracker is created, the system starts recording operations performed on data in OBS buckets. CTS retains operation records generated in the latest seven days.
-
+ Querying Real-Time TracesAfter a management tracker is created on the CTS console, the system starts recording operations performed on cloud service resources. After a data tracker is created, the system starts recording operations performed on data in OBS buckets. CTS retains operation records generated in the latest seven days.
diff --git a/docs/css/umn/css_01_0051.html b/docs/css/umn/css_01_0051.html
new file mode 100644
index 000000000..5df7931c3
--- /dev/null
+++ b/docs/css/umn/css_01_0051.html
@@ -0,0 +1,546 @@
+
+
+ Viewing OpenSearch Cluster Audit Logs
+ Cloud Trace Service (CTS) is available on the public cloud platform. CTS can record operations associated with CSS for subsequent query, auditing, and backtracking purposes.
+ PrerequisitesCTS has been enabled. For details, see Enabling CTS.
+
+ Key Operations Recorded by CTS
+ Table 1 Key operations recorded by CTSOperation
+ |
+Resource Type
+ |
+Event Name
+ |
+
|---|
+
+Creating a cluster
+ |
+cluster
+ |
+createCluster
+ |
+
+Querying cluster details
+ |
+cluster
+ |
+showClusterDetail
+ |
+
+Deleting a cluster
+ |
+cluster
+ |
+deleteCluster
+ |
+
+Renaming a cluster
+ |
+cluster
+ |
+updateClusterName
+ |
+
+Querying the cluster list
+ |
+cluster
+ |
+listClusters
+ |
+
+Changing the password of a cluster
+ |
+cluster
+ |
+resetPassword
+ |
+
+Restarting a cluster
+ |
+cluster
+ |
+restartCluster
+ |
+
+Expanding cluster capacity
+ |
+cluster
+ |
+updateExtendCluster
+ |
+
+Adding nodes and expanding storage
+ |
+cluster
+ |
+updateExtendInstanceStorage
+ |
+
+Changing the node specifications
+ |
+cluster
+ |
+updateFlavor
+ |
+
+Obtaining the node specifications list
+ |
+cluster
+ |
+listFlavors
+ |
+
+Removing specified nodes
+ |
+cluster
+ |
+updateShrinkNodes
+ |
+
+Changing the flavor of a specified node type
+ |
+cluster
+ |
+updateFlavorByType
+ |
+
+Removing nodes of a specific type
+ |
+cluster
+ |
+updateShrinkCluster
+ |
+
+Downloading a security certificate
+ |
+cluster
+ |
+downloadCert
+ |
+
+Replacing nodes
+ |
+cluster
+ |
+updateInstance
+ |
+
+Changing the security mode
+ |
+cluster
+ |
+changeMode
+ |
+
+Adding dedicated Master or Client nodes
+ |
+cluster
+ |
+addIndependentNode
+ |
+
+Upgrading the cluster kernel
+ |
+cluster
+ |
+upgradeCore
+ |
+
+Obtaining a target image ID
+ |
+cluster
+ |
+listImages
+ |
+
+Obtaining upgrade details
+ |
+cluster
+ |
+upgradeDetail
+ |
+
+Retrying a failed upgrade task
+ |
+cluster
+ |
+retryUpgradeTask
+ |
+
+Changing the security group
+ |
+cluster
+ |
+changeSecurityGroup
+ |
+
+Creating a cluster (V2)
+ |
+cluster
+ |
+createClusterV2
+ |
+
+Restarting a cluster (V2)
+ |
+cluster
+ |
+restartCluster
+ |
+
+Rolling restart
+ |
+cluster
+ |
+rollingRestart
+ |
+
+Enabling OpenSearch Dashboards public network access
+ |
+cluster
+ |
+startKibanaPublic
+ |
+
+Disabling OpenSearch Dashboards public network access
+ |
+cluster
+ |
+updateCloseKibana
+ |
+
+Modifying the OpenSearch Dashboards public network bandwidth
+ |
+cluster
+ |
+updateAlterKibana
+ |
+
+Modifying OpenSearch Dashboards public network access control
+ |
+cluster
+ |
+updatePublicKibanaWhitelist
+ |
+
+Disabling OpenSearch Dashboards public network access control
+ |
+cluster
+ |
+stopPublicKibanaWhitelist
+ |
+
+Enabling logging
+ |
+cluster
+ |
+startLogs
+ |
+
+Disabling logging
+ |
+cluster
+ |
+stopLogs
+ |
+
+Querying the job list
+ |
+cluster
+ |
+listLogsJob
+ |
+
+Querying basic log configurations
+ |
+cluster
+ |
+showGetLogSetting
+ |
+
+Modifying basic log configurations
+ |
+cluster
+ |
+updateLogSetting
+ |
+
+Enabling automatic log backup
+ |
+cluster
+ |
+startLogAutoBackupPolicy
+ |
+
+Disabling automatic log backup
+ |
+cluster
+ |
+stopLogAutoBackupPolicy
+ |
+
+Backing up logs
+ |
+cluster
+ |
+createLogBackup
+ |
+
+Querying logs
+ |
+cluster
+ |
+showLogBackup
+ |
+
+Enabling public network access
+ |
+cluster
+ |
+createBindPublic
+ |
+
+Disabling public network access
+ |
+cluster
+ |
+updateUnbindPublic
+ |
+
+Modifying public network bandwidth
+ |
+cluster
+ |
+updatePublicBandWidth
+ |
+
+Enabling the public network access whitelist
+ |
+cluster
+ |
+startPublicWhitelist
+ |
+
+Disabling the public network access whitelist
+ |
+cluster
+ |
+stopPublicWhitelist
+ |
+
+Automatically configuring the basic settings of a cluster snapshot
+ |
+cluster
+ |
+startAutoSetting
+ |
+
+Modifying the basic settings of a cluster snapshot
+ |
+cluster
+ |
+updateSnapshotSetting
+ |
+
+Manually creating a snapshot
+ |
+snapshot
+ |
+createSnapshot
+ |
+
+Restoring a snapshot
+ |
+snapshot
+ |
+restoreSnapshot
+ |
+
+Deleting a specified snapshot
+ |
+snapshot
+ |
+deleteSnapshot
+ |
+
+Setting an automatic snapshot creation policy
+ |
+cluster
+ |
+createAutoCreatePolicy
+ |
+
+Querying automatic snapshot creation policies
+ |
+cluster
+ |
+showAutoCreatePolicy
+ |
+
+Querying the snapshot list
+ |
+cluster
+ |
+listSnapshots
+ |
+
+Disabling snapshots
+ |
+cluster
+ |
+stopSnapshot
+ |
+
+Enabling automatic snapshot creation
+ |
+cluster
+ |
+startAutoCreateSnapshots
+ |
+
+Disabling automatic snapshot creation
+ |
+cluster
+ |
+stopAutoCreateSnapshots
+ |
+
+Enabling VPC Endpoint
+ |
+cluster
+ |
+startVpecp
+ |
+
+Disabling VPC Endpoint
+ |
+cluster
+ |
+stopVpecp
+ |
+
+Obtaining a VPCEP connection
+ |
+cluster
+ |
+showVpcepConnection
+ |
+
+Updating a VPCEP connection
+ |
+cluster
+ |
+updateVpcepConnection
+ |
+
+Modifying the VPCEP whitelist
+ |
+cluster
+ |
+updateVpcepWhitelist
+ |
+
+Modifying parameter settings
+ |
+cluster
+ |
+listYmls
+ |
+
+Obtaining the task list of parameter settings
+ |
+cluster
+ |
+listYmlsJob
+ |
+
+Obtaining the parameter settings list
+ |
+cluster
+ |
+updateYmls
+ |
+
+Querying ELB V3 load balancers supported by a cluster
+ |
+cluster
+ |
+listElbs
+ |
+
+Enabling or disabling the Elasticsearch Load Balancer
+ |
+cluster
+ |
+enableOrDisableElb
+ |
+
+Configuring an Elasticsearch listener
+ |
+cluster
+ |
+createElbListener
+ |
+
+Obtaining information about an Elasticsearch ELB
+ |
+cluster
+ |
+showElbDetail
+ |
+
+Updating an Elasticsearch listener
+ |
+cluster
+ |
+updateEsListener
+ |
+
+Querying the certificate list
+ |
+cluster
+ |
+listElbCerts
+ |
+
+Obtaining the intelligent O&M task list and details
+ |
+cluster
+ |
+listAiOps
+ |
+
+Creating a cluster detection task
+ |
+cluster
+ |
+createAiOps
+ |
+
+Deleting a detection task record
+ |
+cluster
+ |
+deleteAiOps
+ |
+
+Obtaining SMN topics available for intelligent O&M alarms
+ |
+cluster
+ |
+listSmnTopics
+ |
+
+
+
+
+
+ Querying Real-Time TracesAfter a management tracker is created on the CTS console, the system starts recording operations performed on cloud service resources. After a data tracker is created, the system starts recording operations performed on data in OBS buckets. CTS retains operation records generated in the latest seven days.
+
+
+
+
diff --git a/docs/css/umn/css_01_0053.html b/docs/css/umn/css_01_0053.html
new file mode 100644
index 000000000..71aee5918
--- /dev/null
+++ b/docs/css/umn/css_01_0053.html
@@ -0,0 +1,270 @@
+
+
+ Configuring Large Query Isolation for an OpenSearch Cluster
+ Large query isolation can be configured to manage queries that have high memory usage or take too long to complete. This helps improve the stability of OpenSearch clusters and prevent out-of-memory (OOM) exceptions.
+ As business grows, your OpenSearch clusters may face mounting query pressure. Some complex queries may occupy excessive node memory, triggering frequent garbage collection or even OOM exceptions, which may compromise cluster performance and stability. Large query isolation enables effective management of memory-intensive, time-consuming query requests, ensuring cluster stability. Large query isolation includes the following: - Isolating large queries: Manages memory-intensive/time-consuming queries separately to avoid impacting other queries.
- Query cancelation based on a heap memory usage threshold: Cancels a large query in the isolation pool when the node heap memory usage reaches a predefined threshold.
- Global query timeout: Automatically cancels tasks when they last longer than a predefined timeout. This timeout applies globally.
+
+ How the Feature Works- Isolation pool management: Large queries are placed in an isolation pool, where they are managed separately from other normal queries. Tasks in the isolation pool may be canceled based on preset memory or duration thresholds.
- Query cancelation policies:
- fair: Selects which query to cancel by considering both memory usage and query duration.
- mem-first: Cancels the query that has the highest memory usage.
- time-first: Cancels the query that has lasted the longest.
+ - Native cancel API: OpenSearch's native cancel API can be used to cancel tasks, ensuring compatibility.
+
+ Constraints- Only OpenSearch 2.19.0 supports large query isolation.
- Large query isolation is enabled by default, whereas global query timeout is disabled by default. If you enable them, the configuration will take effect immediately.
+
+ Logging In to OpenSearch DashboardsLog in to OpenSearch Dashboards and go to the command execution page. OpenSearch clusters support multiple access methods. This topic uses OpenSearch Dashboards as an example to describe the operation procedures.
+ - Log in to the CSS management console.
- In the navigation pane on the left, choose Clusters > OpenSearch.
- In the cluster list, find the target cluster, and click Dashboards in the Operation column to log in to OpenSearch Dashboards.
- In the left navigation pane, choose Dev Tools.
The left part of the console is the command input box, and the triangle icon in its upper-right corner is the execution button. The right part shows the execution result.
+
+
+ Configuring Large Query IsolationLarge query isolation places large queries in an isolation pool, where they may be canceled based on preset memory or duration thresholds. Large query isolation is enabled by default. Any change takes effect immediately.
+ - Run the following command to enable large query isolation:
PUT _cluster/settings
+{
+ "persistent": {
+ "search.isolator.enabled": true
+ }
+}
+
+Table 1 Enabling large query isolationParameter
+ |
+Type
+ |
+Description
+ |
+
|---|
+
+search.isolator.enabled
+ |
+Boolean
+ |
+Whether to enable large query isolation. When enabled, large queries are managed separately from other normal queries.
+- true (default): Enable large query isolation.
- false: Disable large query isolation.
+ |
+
+
+
+ - Run the following commands to configure thresholds for defining a large query task:
PUT _cluster/settings
+{
+ "persistent": {
+ "search.isolator.memory.task.limit": "50MB",
+ "search.isolator.time.management": "10s"
+ }
+}
+
+Table 2 Parameters for configuring large query isolation thresholdsParameter
+ |
+Type
+ |
+Description
+ |
+
|---|
+
+search.isolator.memory.task.limit
+ |
+String
+ |
+Large query memory threshold: When a query requests more memory than specified by this threshold, it is placed into an isolation pool.
+Value range: 0b to the maximum heap memory of a node.
+Unit: MB, GB, or other units supported by OpenSearch.
+The default value is 50 MB.
+ |
+
+search.isolator.time.management
+ |
+String
+ |
+Large query duration threshold: When a query has lasted longer than specified by this threshold, it is placed into an isolation pool.
+Value range: ≥ 0ms
+Unit: s (second), ms (millisecond), m (minute), or h (hour).
+The default value is 10s.
+ |
+
+
+
+ - Configure the resource usage thresholds for triggering query cancelation.
PUT _cluster/settings
+{
+ "persistent": {
+ "search.isolator.memory.pool.limit": "50%",
+ "search.isolator.memory.heap.limit": "90%",
+ "search.isolator.count.limit": 1000
+ }
+}
+
+Table 3 Parameters for configuring query cancelation thresholdsParameter
+ |
+Type
+ |
+Description
+ |
+
|---|
+
+search.isolator.memory.pool.limit
+ |
+String
+ |
+Isolation pool memory usage threshold, which indicates the maximum node heap memory usage allowed. When the total memory requested by all large queries in the isolation pool exceeds this threshold, one of the large queries is automatically canceled based on a predefined policy.
+Value range: 0.0–100.0%
+The default value is 50.
+ |
+
+search.isolator.memory.heap.limit
+ |
+String
+ |
+Heap memory usage threshold, which indicates the actual node heap memory usage, contributed by both writes and queries. When the node heap memory usage exceeds this threshold, one of the large queries is automatically canceled based on a predefined policy.
+Value range: 0.0–100.0%
+The default value is 90%.
+ |
+
+search.isolator.count.limit
+ |
+Integer
+ |
+The maximum number of queries in the isolation pool. When this threshold is reached, query cancelation is triggered, and no new queries will be accepted.
+Value range: 10–50000
+The default value is 1000.
+ |
+
+
+
+ In addition to search.isolator.memory.pool.limit and search.isolator.count.limit, you can configure search.isolator.memory.task.limit and search.isolator.time.management to control the number of query tasks that enter the isolation pool.
+
+ - Configure a policy for selecting queries to cancel in the isolation pool.
PUT _cluster/settings
+{
+ "persistent": {
+ "search.isolator.strategy": "fair",
+ "search.isolator.strategy.ratio": "0.5%"
+ }
+}
+
+Table 4 Parameters for configuring a query cancelation policyParameter
+ |
+Type
+ |
+Description
+ |
+
|---|
+
+search.isolator.strategy
+ |
+String
+ |
+Policy for selecting which query to cancel when query isolation is triggered.
+- fair (default): Selects which query to cancel by considering both memory usage and query duration. If the difference between the memory used by two queries ≤ maximum node heap memory x fair policy threshold, the query that has lasted the longest is canceled; on the contrary, if it is greater than that, the most memory-intensive query is canceled instead.
- mem-first: Cancels the query that has the highest memory usage.
- time-first: Cancels the query that has lasted the longest.
+The large query isolation pool is checked every second until the heap memory is within the safe range.
+ |
+
+search.isolator.strategy.ratio
+ |
+String
+ |
+Fair policy threshold, which is the ratio of the memory difference between two queries in the isolation pool to the maximum node heap memory. If this threshold is not reached, the query that has lasted the longest will be canceled. Otherwise, the query that occupies the most memory will be canceled.
+This parameter is valid only when search.isolator.strategy is set to fair.
+Value range: 0.0–100.0%
+The default value is 1.
+ |
+
+
+
+
+
+ Configuring Global Query TimeoutA global query timeout, as the name indicates, applies to all queries. Global query timeout is disabled by default. Any change takes effect immediately.
+ Run the following command to enable and configure a global query timeout: PUT _cluster/settings
+{
+ "persistent": {
+ "search.isolator.time.enabled": true,
+ "search.isolator.time.limit": "110s"
+ }
+}
+
+ Table 5 Parameter descriptionParameter
+ |
+Type
+ |
+Description
+ |
+
|---|
+
+search.isolator.time.enabled
+ |
+Boolean
+ |
+Whether to enable a global query timeout. When enabled, queries are automatically canceled when they last longer than a predefined timeout.
+- true: Enable global query timeout.
- false (default): Disable global query timeout.
+ |
+
+search.isolator.time.limit
+ |
+String
+ |
+The value of the global query timeout.
+Value range: ≥ 0 ms
+The default value is 120s.
+ |
+
+
+
+
+
+
+ Configuring the Maximum Number of Log Records Retained for Query CancelationRun the following command to set the maximum number of log records retained for canceled queries: PUT _cluster/settings
+{
+ "persistent": {
+ "search.isolator.log.count": "100"
+ }
+}
+
+ Parameter
+ |
+Data Type
+ |
+Description
+ |
+
|---|
+
+search.isolator.log.count
+ |
+Integer
+ |
+Maximum number of log records retained for canceled queries. Canceled query requests are recorded in the memory for the purpose of analyzing and optimizing large queries. Excess records will be discarded.
+This parameter is valid only when search.isolator.enabled is set to true.
+Value range: 0–5000
+The default value is 100.
+You can use the following API to check query cancelation logs:
+- Query all nodes:
GET /_isolator_metrics
+ - Query a single node:
GET /_isolator_metrics/{nodeId}
+ - Query details about canceled queries on all nodes:
GET /_isolator_metrics?detailed
+ - Query details about canceled queries on a single node:
GET /_isolator_metrics/{nodeId}?detailed
+
+In the commands above, nodeId indicates the node ID. Example response:
+{
+ "_nodes": {
+ "total": 1,
+ "successful": 1,
+ "failed": 0
+ },
+ "cluster_name": "test",
+ "nodes": {
+ "CTqrZFXWTzmLonSZyNMKkQ": {
+ "name": "test-ess-esn-1-1",
+ "host": "172.16.101.116",
+ "total_cancel": 0, //Total number of canceled queries
+ "isolator_cancel": 0, //Number of queries canceled because isolation pool thresholds were exceeded
+ "out_of_time_cancel": 0 //Number of queries canceled due to timeout
+ }
+ }
+}
+ |
+
+
+
+
+
+
+
+
+
diff --git a/docs/css/umn/en-us_topic_0000001972160305.html b/docs/css/umn/css_01_0055.html
similarity index 80%
rename from docs/css/umn/en-us_topic_0000001972160305.html
rename to docs/css/umn/css_01_0055.html
index 82448197b..a47cdf378 100644
--- a/docs/css/umn/en-us_topic_0000001972160305.html
+++ b/docs/css/umn/css_01_0055.html
@@ -8,18 +8,24 @@
|
|---|
-2025-04-11
+ |
2025-11-15
+ |
+Added:
+- OpenSearch 2.19 is supported.
+ |
+
+2025-04-11
|
Modified:
-
+
|
2024-11-04
|
Added:
-
+
Modified:
-
+
|
2024-05-17
@@ -45,14 +51,14 @@
|
2023-09-25
|
Added:
-
+
|
2023-03-28
|
- Optimized the content structure of the following sections:
- What Is Cloud Search Service?
- Related Services
- Clusters in Security Mode
- Updated the procedure description in the following sections:
- Getting Started with Elasticsearch
- Changing Specifications
- - Added:
+
- Added:
- Deleted the following sections:
- What Is Kibana?
- What is Cerebro?
- Suggestions on Using Elasticsearch
- Viewing Monitoring Metrics
- Creating Alarm Rules
|
diff --git a/docs/css/umn/css_01_0056.html b/docs/css/umn/css_01_0056.html
new file mode 100644
index 000000000..d0c73a263
--- /dev/null
+++ b/docs/css/umn/css_01_0056.html
@@ -0,0 +1,615 @@
+
+
+Configuring Flow Control for an OpenSearch Cluster
+Configure flow control policies for your OpenSearch cluster in both the inbound and outbound directions, ensuring cluster stability by safeguarding against abnormal traffic.
+
An OpenSearch cluster can become overloaded due to traffic surges, malicious requests, and internal resource competition, which can even lead to node failures. Through policies like client request throttling, write backpressure, and traffic pattern analysis, flow control ensures proper resource allocation, thereby protecting clusters from overload. It covers the following scenarios:
- High-concurrency write handling: mitigates the risk of out-of-memory (OOM) exceptions under heavy write loads.
- Security defense: controls access by IP address using both blacklists and whitelists.
- Emergency response: blocks malicious or abnormal traffic in one click.
- Performance optimization: optimizes flow control thresholds and policies based on collected statistics.
+
+
How the Feature Works
+
Table 1 Flow control policiesPolicy
+ |
+What It Does
+ |
+Details
+ |
+
|---|
+
+HTTP/HTTPS flow control
+ |
+Controls client access traffic using blacklists and whitelists, an upper limit on concurrent connections, and a rate limit on new connection attempts.
+- Blacklists and whitelists: A whitelist takes precedence over a blacklist. If an IP address is on both lists, it is allowed. Requests sent through blacklisted connections will be ignored.
- Concurrent connection limit: Limits the total number of HTTP connections per second to prevent overload.
- New connection limit: Limits the number of new connections that can be set up per second. The warmup_period parameter protects against connection floods, allowing traffic to grow gradually and steadily.
+ |
+Configuring HTTP/HTTPS Flow Control
+ |
+
+Memory flow control
+ |
+When the heap memory usage exceeds a pre-defined threshold (for example, 80%), the system stops receiving large requests, and garbage collection (GC) is triggered to reclaim memory.
+Write traffic is throttled by setting the backpressure factor (in_flight_factor) and the maximum delay for request handling (max).
+ |
+Configuring Memory-based Flow Control
+ |
+
+One-click traffic blocking
+ |
+When enabled, the system immediately disconnects all client connections, but not those used for OpenSearch Dashboards access or O&M and monitoring APIs, in order to restore the cluster.
+ |
+Configuring One-Click Traffic Blocking
+ |
+
+Request statistics sampling and analysis
+ |
+Records request metrics (such as bulk writes and queries) by client IP address, and exposes them via a statistics API to evaluate the cluster load and proactively identify abnormal traffic patterns.
+ |
+Configuring Request Statistics Sampling and Analysis
+ |
+
+Access logging
+ |
+Records the URLs and bodies of HTTP/HTTPS requests for cluster load and client request analysis.
+Access logs can also be saved to files (that is, persisted to disk) to facilitate troubleshooting and performance analysis.
+ |
+
+ |
+
+
+
+
+
+
Constraints
- Only OpenSearch 2.19.0 supports flow control.
- Enabling flow control may compromise cluster and node performance. For example, memory-based flow control may cause query failures on OpenSearch Dashboards.
- Configuring too many paths for memory-based flow control has a significant impact on performance.
- Enabling access logging will hurt cluster performance.
- After flow control policies are enabled, excess or non-compliant requests are rejected directly.
+
+
Logging In to OpenSearch Dashboards
Log in to OpenSearch Dashboards and go to the command execution page. OpenSearch clusters support multiple access methods. This topic uses OpenSearch Dashboards as an example to describe the operation procedures.
+
- Log in to the CSS management console.
- In the navigation pane on the left, choose Clusters > OpenSearch.
- In the cluster list, find the target cluster, and click Dashboards in the Operation column to log in to OpenSearch Dashboards.
- In the left navigation pane, choose Dev Tools.
The left part of the console is the command input box, and the triangle icon in its upper-right corner is the execution button. The right part shows the execution result.
+
+
+
Configuring HTTP/HTTPS Flow Control
Control client access traffic using blacklists and whitelists, an upper limit on concurrent connections, and a rate limit on new connection attempts.
+
- Enable HTTP/HTTPS flow control.
PUT /_cluster/settings
+{
+ "persistent": {
+ "flowcontrol.http.enabled": true,
+ "flowcontrol.http.allow": ["192.168.0.1/24", "192.168.2.1/24"],
+ "flowcontrol.http.deny": "192.168.1.1/24",
+ "flowcontrol.http.concurrent": 1000,
+ "flowcontrol.http.newconnect": 1000,
+ "flowcontrol.http.warmup_period": 0
+ }
+}
+
+Table 2 Parameters for configuring HTTP/HTTPS flow controlParameter
+ |
+Type
+ |
+Description
+ |
+
|---|
+
+flowcontrol.http.enabled
+ |
+Boolean
+ |
+Whether to enable HTTP/HTTPS flow control. Enabling it may impact node performance.
+- true: Enable HTTP/HTTPS flow control.
- false (default): Disable HTTP/HTTPS flow control.
+ |
+
+flowcontrol.http.allow
+ |
+List<String>
+ |
+A whitelist of client IP addresses or CIDR blocks that are allowed to access the cluster, supporting:
+- Individual IP addresses, for example, 192.18.0.1.
- CIDR blocks, for example, 192.168.0.0/24.
- Multiple IP addresses or CIDR blocks separated by commas (,), for example, 192.168.0.1/24, 192.168.2.1/24.
+The default is an empty list, indicating no whitelist.
+Setting this parameter to null restores the default value.
+ |
+
+flowcontrol.http.deny
+ |
+List<String>
+ |
+A blacklist of client IP addresses or CIDR blocks that are not allowed to access the cluster. A whitelist takes precedence over a blacklist. A blacklist supports the following:
+- Individual IP addresses, for example, 192.18.0.1.
- CIDR blocks, for example, 192.168.0.0/24.
- Multiple IP addresses or CIDR blocks separated by commas (,), for example, 192.168.0.1/24, 192.168.2.1/24.
+The default is an empty list, indicating no blacklist.
+Setting this parameter to null restores the default value.
+ |
+
+flowcontrol.http.concurrent
+ |
+Integer
+ |
+Maximum number of concurrent HTTP/HTTPS connections that can be handled by each node per second. The default value is the number of available node vCPUs multiplied by 600.
+ |
+
+flowcontrol.http.newconnect
+ |
+Integer
+ |
+Maximum number of new HTTP/HTTPS connections that can be created per second per node.
+The default value is the number of available node vCPUs multiplied by 200.
+Setting this parameter to null restores the default value.
+ |
+
+flowcontrol.http.warmup_period
+ |
+Integer
+ |
+A grace period during which a system gradually ramp up from accepting zero HTTP/HTTPS requests to its full, maximum capacity.
+Value range: 0–10000
+Unit: ms
+The default value is 0, indicating that there is no such warm-up period, and the maximum rate can be reached instantly.
+Setting this parameter to null restores the default value.
+For example, if flowcontrol.http.newconnect is set to 100 and flowcontrol.http.warmup_period is set to 5000ms, it takes 5 seconds for the system to reach 100 new connections per second.
+ |
+
+
+
+
- Disable HTTP/HTTPS flow control.
PUT /_cluster/settings
+{
+ "persistent": {
+ "flowcontrol.http.enabled": false
+ }
+}
+
+
+
Configuring Memory-based Flow Control
Enable write throttling to mitigate the risk of OOM exceptions when the heap memory usage of a node exceeds a predefined threshold.
+
- Enable memory-based flow control.
PUT /_cluster/settings
+{
+ "persistent": {
+ "flowcontrol.memory.enabled": true,
+ "flowcontrol.memory.heap_limit": "80%"
+ }
+}
+
+Table 3 Memory-based flow control parametersParameter
+ |
+Type
+ |
+Description
+ |
+
|---|
+
+flowcontrol.memory.enabled
+ |
+Boolean
+ |
+Whether to enable memory-based flow control. When enabled, node heap memory usage is monitored and a threshold is set, and writes are throttled when this threshold is reached. - true: Enable memory-based flow control.
- false (default): Disable memory-based flow control.
+ |
+
+flowcontrol.memory.heap_limit
+ |
+String
+ |
+Node heap memory usage threshold. When this threshold is exceeded, a write backpressure mechanism is triggered.
+Value range: 10%–100%
+The default value 90% of flowcontrol.memory.heap_limit is a conservative threshold. When the heap memory usage exceeds 90%, the system stops accepting client requests that exceed 64 KB, until heap memory usage decreases. Once the heap memory usage decreases to 85%, client data equivalent to 5% x maximum heap memory capacity can be read. If the heap memory usage stays above 90% for a long time, client requests cannot be processed. In this case, garbage collection is triggered until the heap memory usage drops below this threshold.
+Generally, you can set this threshold to 80% or less to ensure that cluster nodes have reserved some heap memory for operations besides data writing, such as segment merges.
+Setting this parameter to null restores the default value.
+ |
+
+flowcontrol.holding.in_flight_factor
+ |
+Float
+ |
+Backpressure factor, which controls the sensitivity of memory-based backpressure. A larger value indicates more powerful write throttling.
+Value range: ≥ 0.5
+The default value is 1.0.
+Setting this parameter to null restores the default value.
+ |
+
+flowcontrol.holding.max
+ |
+TimeValue
+ |
+Maximum request handling delay allowed before requests are handled according to the policy defined by flowcontrol.holding.max_strategy.
+Value range: ≥ 15s
+Unit: second
+Default value: 60s
+Setting this parameter to null restores the default value.
+ |
+
+flowcontrol.holding.max_strategy
+ |
+String
+ |
+Handling policy for requests delayed longer than flowcontrol.holding.max.
+- keep (default): Keep the backpressure status while waiting for the memory usage to drop. The server determines when to execute the requests based on real-time memory usage.
- soft: Forcibly execute the requests, but the inFlight circuit breaker gets to decide whether to reject them.
- hard: Reject the requests immediately and disconnect client connections.
+Setting this parameter to null restores the default value.
+ |
+
+flowcontrol.memory.once_free_max
+ |
+String
+ |
+Maximum memory that can be made available at a time for a re-enabled request queue. This parameter can be configured to prevent cluster overload caused by a flood of incoming requests.
+Value range: 1%–50%
+The default value is 5%.
+Setting this parameter to null restores the default value.
+ |
+
+flowcontrol.memory.nudges_gc
+ |
+Boolean
+ |
+Whether to trigger garbage collection (GC) to reclaim memory when the write pressure is too high. (The backpressure connection pool is checked every second. The write pressure is considered high if all existing connections are blocked and new write requests cannot be accepted.)
+- true (default): Trigger GC.
- false: Not to trigger GC.
+Setting this parameter to null restores the default value.
+ |
+
+
+
+
- Disable memory-based flow control.
PUT /_cluster/settings
+{
+ "persistent": {
+ "flowcontrol.memory.enabled": false
+ }
+}
+
+
+
Configuring One-Click Traffic Blocking
In case of emergencies, the system immediately disconnects all client connections (excluding those used for OpenSearch Dashboards access or O&M and monitoring APIs) to restore clusters.
+
- Enable one-click traffic blocking.
PUT /_cluster/settings
+{
+ "persistent": {
+ "flowcontrol.break.enabled": true
+ }
+}
+
+Table 4 Parameters for configuring one-click traffic blockingParameter
+ |
+Type
+ |
+Description
+ |
+
|---|
+
+flowcontrol.break.enabled
+ |
+Boolean
+ |
+Whether to enable one-click traffic blocking (similar to a circuit breaker). When enabled, the system immediately disconnects all client connections, but not those used for OpenSearch Dashboards access or O&M and monitoring APIs.
+- true: Enable one-click disconnection.
- false (default): Disable one-click disconnection.
+ |
+
+
+
+
- Disable one-click disconnection:
PUT /_cluster/settings
+{
+ "persistent": {
+ "flowcontrol.break.enabled": false
+ }
+}
+
+
+
Configuring Request Statistics Sampling and Analysis
Collect request metrics by client IP address to help identify abnormal traffic patterns.
+
- Enable request statistics sampling.
PUT _cluster/settings
+{
+ "transient": {
+ "flowcontrol.log.access.enabled": true
+ }
+}
+
+Table 5 Parameters for request statistics samplingParameter
+ |
+Type
+ |
+Description
+ |
+
|---|
+
+flowcontrol.log.access.enabled
+ |
+Boolean
+ |
+Whether to enable request statistics sampling, that is, whether to collect request metrics (such as bulk writes and search/msearch requests) by client IP address. - true: Enable request statistics sampling.
- false (default): Disable request statistics sampling.
+ |
+
+flowcontrol.log.access.count
+ |
+Integer
+ |
+Maximum number of client IP addresses sampled.
+Value range: 0–100
+Default value: 10
+Setting this parameter to null restores the default value.
+ |
+
+
+
+
- Check the sampled statistics.
- Check the flow control status of all nodes.
GET /_nodes/stats/filter/v2
+ - Check the flow control details of all nodes.
GET /_nodes/stats/filter/v2?detail
+ - Check the flow control status of a specified node.
GET /_nodes/{nodeId}/stats/filter/v2
+{nodeId} indicates the node ID.
+
+Example response:
{
+ "_nodes" : {
+ "total" : 1,
+ "successful" : 1,
+ "failed" : 0
+ },
+ "cluster_name" : "css-xxxx",
+ "nodes" : {
+ "d3qnVIpPTtSoadkV0LQEkA" : {
+ "name" : "css-xxxx-ess-esn-1-1",
+ "host" : "192.168.x.x",
+ "timestamp" : 1672236425112,
+ "flow_control" : {
+ "http" : {
+ "current_connect" : 52,
+ "rejected_concurrent" : 0,
+ "rejected_rate" : 0,
+ "rejected_black" : 0,
+ "rejected_breaker" : 0
+ },
+ "access_items" : [
+ {
+ "remote_address" : "10.0.0.x",
+ "search_count" : 0,
+ "bulk_count" : 0,
+ "other_count" : 4
+ }
+ ],
+ "holding_requests" : 0
+ }
+ }
+ }
+}
+
+
Table 6 Response parametersParameter
+ |
+Description
+ |
+
|---|
+
+current_connect
+ |
+Number of HTTP connections to a node, which is recorded regardless of whether flow control is enabled. This value is equivalent to the current_open value of GET /_nodes/stats/http API. It shows the current client connections of each node.
+ |
+
+rejected_concurrent
+ |
+Number of concurrent connections rejected during flow control.
+This metric is available only when flowcontrol.http.enabled is set to true. The count will not be cleared when flow control is disabled.
+ |
+
+rejected_rate
+ |
+Number of new connections rejected during flow control.
+This metric is available only when flowcontrol.http.enabled is set to true. The count will not be cleared when flow control is disabled.
+ |
+
+rejected_black
+ |
+Number of new connections rejected by a preconfigured blacklist during flow control.
+This metric is available only when flowcontrol.http.enabled is set to true. The count will not be cleared when flow control is disabled.
+ |
+
+rejected_breaker
+ |
+Number of new connections rejected during one-click traffic blocking.
+This metric is available only when flowcontrol.break.enabled is set to true. The count will not be cleared when one-click traffic blocking is disabled.
+ |
+
+access_items
+ |
+IP addresses of clients that recently accessed the cluster.
+The number of client IP addresses sampled is determined by flowcontrol.log.access.count.
+ |
+
+remote_address
+ |
+IP addresses and the number of requests.
+ |
+
+search_count
+ |
+Number of times a client accessed a database using _search and _msearch.
+ |
+
+bulk_count
+ |
+Number of times a client accessed a database using _bulk.
+ |
+
+other_count
+ |
+Number of times a client accessed a database using other request methods.
+ |
+
+holding_requests
+ |
+Number of connections to the current node where writes are halted due to flow control.
+ |
+
+
+
+
+
- Disable request statistics sampling.
PUT /_cluster/settings
+{
+ "persistent": {
+ "flowcontrol.log.access.enabled": false
+ }
+}
+
+
+
Configuring Access Logging
When access logging is enabled, the system records the URLs and bodies of HTTP/HTTPS requests for cluster load and request analysis.
+
- Enable access logging.
- Enable access logging for all nodes in a cluster.
PUT /_access_log?duration_limit=30s&capacity_limit=1mb
+ - Enable access logging for a specified node in a cluster.
PUT /_access_log/{nodeId}?duration_limit=30s&capacity_limit=1mb
+{nodeId} indicates the node ID.
+
+
+Table 7 Parameters for enabling access loggingParameter
+ |
+Type
+ |
+Description
+ |
+
|---|
+
+duration_limit
+ |
+String
+ |
+Maximum duration of access log records. When this duration is reached, access logging stops.
+Value range: 10 to 120
+Unit: s
+The default value is 30.
+Setting this parameter to null restores the default value.
+Access logging stops when either duration_limit or capacity_limit is reached.
+ |
+
+capacity_limit
+ |
+String
+ |
+Maximum memory capacity for recording access logs. When the size of an access log reaches this limit, access logging stops.
+Value range: 1 to 5
+Unit: MB
+The default value is 1.
+Setting this parameter to null restores the default value.
+Access logging stops when either duration_limit or capacity_limit is reached.
+ |
+
+
+
+
- Check access logs.
+
Example response:
{
+ "_nodes" : {
+ "total" : 1,
+ "successful" : 1,
+ "failed" : 0
+ },
+ "cluster_name" : "css-flowcontroller",
+ "nodes" : {
+ "8x-ZHu-wTemBQwpcGivFKg" : {
+ "name" : "css-flowcontroller-ess-esn-1-1",
+ "host" : "10.0.0.98",
+ "count" : 2,
+ "access" : [
+ {
+ "time" : "2021-02-23 02:09:50",
+ "remote_address" : "/10.0.0.98:28191",
+ "url" : "/_access/security/log?pretty",
+ "method" : "GET",
+ "content" : ""
+ },
+ {
+ "time" : "2021-02-23 02:09:52",
+ "remote_address" : "/10.0.0.98:28193",
+ "url" : "/_access/security/log?pretty",
+ "method" : "GET",
+ "content" : ""
+ }
+ ]
+ }
+ }
+}
+
+
Table 8 Response parametersParameter
+ |
+Description
+ |
+
|---|
+
+name
+ |
+Node name
+ |
+
+host
+ |
+Node IP address
+ |
+
+count
+ |
+Number of node access requests in a statistical period
+ |
+
+access
+ |
+Details about node access requests in a statistical period
+ |
+
+time
+ |
+Request time
+ |
+
+remote_address
+ |
+Source IP address and port number in the request
+ |
+
+url
+ |
+Original URL of the request
+ |
+
+method
+ |
+Request method
+ |
+
+content
+ |
+Request content. If the value is an empty string (""), there is no request body.
+ |
+
+
+
+
+
- Delete access logs. Logs are recorded in the memory. After reading the logs, you are advised to delete them to reclaim resources.
Delete access logs for all nodes:
DELETE /_access_log
+
+
+
Configuring Access Logging in Files
Access logs can be persisted to disk for troubleshooting and analysis. Use this function sparingly, as it can impact cluster performance. Remember to disable it immediately after resolving the issue.
+
- Enable access logging in files.
PUT /_cluster/settings
+{
+ "persistent": {
+ "flowcontrol.log.file.enabled": true
+ }
+}
+
+Table 9 Enable access logging in filesParameter
+ |
+Type
+ |
+Description
+ |
+
|---|
+
+flowcontrol.log.file.enabled
+ |
+Boolean
+ |
+Whether to record access logs in files. When enabled, the log of each access request is recorded in files.
+The log file name is Cluster name_access_log.log. You can check this file only through the log backup function.
+- true: Record access logs in files.
- false (default): Not to record access logs in files.
+ |
+
+
+
+
- Disable access logging in files.
PUT /_cluster/settings
+{
+ "persistent": {
+ "flowcontrol.log.file.enabled": false
+ }
+}
+
+
+
Binding an Elasticsearch Cluster to an Enterprise Project
-You can create enterprise projects based on your organizational structure. Then you can manage resources across different regions by enterprise project, add users and user groups to enterprise projects, and grant different permissions to the users and user groups. This section describes how to bind a CSS cluster to an enterprise project and how to modify an enterprise project.
-
Prerequisites
Before binding an enterprise project, you have created an enterprise project.
+
Associating an Elasticsearch Cluster with an Enterprise Project
+
In CSS, you can associate your Elasticsearch cluster with an enterprise project. You can create enterprise projects based on your organizational structure. Then you can manage resources across different regions by enterprise project, add users and user groups to enterprise projects, and grant different permissions to the users and user groups.
+
You can associate your cluster with an enterprise project either when creating the cluster or afterward.
+
Prerequisites
You have created an enterprise project.
-
Binding an Enterprise Project
When creating a cluster, you can bind an existing enterprise project to the cluster, or click View Enterprise Project to go to the enterprise project management console and create a new project or view existing projects.
-
-
Modifying an Enterprise Project
For a cluster that has been created, you can modify its enterprise project based on the site requirements.
-
- Log in to the CSS management console.
- In the navigation pane on the left, select a cluster type. The cluster management page is displayed.
- In the cluster list on the displayed page, click the target cluster name to switch to the Cluster Information page.
- On the Cluster Information page, click the enterprise project name on the right of Enterprise Project. The project management page is displayed.
- On the Resources tab page, select the region of the current cluster, and select CSS for Service. In this case, the corresponding CSS cluster is displayed in the resource list.
- Select the cluster whose enterprise project you want to modify and click Remove.
- On the Remove Resource page, specify Mode and select Destination Enterprise Project, and click OK.
- After the resource is removed, you can view the modified enterprise project information on the Clusters page.
+
Managing Enterprise Projects
- Log in to the CSS management console.
- In the navigation pane on the left, choose Clusters > Elasticsearch.
- In the cluster list, click the name of the target cluster. The cluster information page is displayed.
- On the Overview tab, find Enterprise Project in the Configuration area, and click the enterprise project name next to it. The Project Management page is displayed.
- On the Resources tab, select the region of the current cluster, and select CloudSearchService as Service. Then click Search. The CSS clusters are displayed in the resource list below.
- Select the cluster whose enterprise project you want to modify and click Remove.
- On the Remove Resource page, specify Mode and select Destination Enterprise Project, and click OK.
- After the change is complete, return to the Elasticsearch cluster list on the CSS console and check the cluster's new enterprise project.
diff --git a/docs/css/umn/css_01_0059.html b/docs/css/umn/css_01_0059.html
new file mode 100644
index 000000000..0083f0097
--- /dev/null
+++ b/docs/css/umn/css_01_0059.html
@@ -0,0 +1,26 @@
+
+
+
+
Importing Data to an OpenSearch Cluster
+
+
+
+
+
diff --git a/docs/css/umn/css_01_0060.html b/docs/css/umn/css_01_0060.html
index 8ac6c0a7d..e61ba12cb 100644
--- a/docs/css/umn/css_01_0060.html
+++ b/docs/css/umn/css_01_0060.html
@@ -1,78 +1,78 @@
-
+
Managing Failed Tasks for Elasticsearch Clusters
-
In the Failed Tasks dialog box, you can view the failed tasks related to a cluster, such as failing to create, restart, scale out, back up, or restore a cluster. In addition, you can view the failure cause of each task and choose to delete one or all failed tasks.
-
Viewing Failed Tasks
- Log in to the CSS management console.
- Click Clusters to switch to the Clusters page. Click the digit next to Failed Tasks to switch to the Failed Tasks dialog box.
Figure 1 Clicking the digit next to Failed Tasks
- - In the Failed Tasks dialog box, view all failed tasks of the current account. The following information about the failed tasks is displayed: Name/ID, Task Status, and Failure Time.
- Click the question mark in the Task Status column to view the failure cause of a task. You are advised to troubleshoot faults based on failure causes. For details about failure causes, see Error Code.
Figure 2 Viewing the failure cause of a task
+In the Failed Tasks dialog box, you can view the failed tasks related to a cluster, such as failing to create, restart, scale out, back up, or restore a cluster. In addition, you can view the failure cause of each task and choose to delete one or all failed tasks.
+
Viewing Failed Tasks
- Log in to the CSS management console.
- Click Clusters to switch to the Clusters page. Click the digit next to Failed Tasks to switch to the Failed Tasks dialog box.
Figure 1 Clicking the digit next to Failed Tasks
+ - In the Failed Tasks dialog box, view all failed tasks of the current account. The following information about the failed tasks is displayed: Name/ID, Task Status, and Failure Time.
- Click the question mark in the Task Status column to view the failure cause of a task. You are advised to troubleshoot faults based on failure causes. For details about failure causes, see Error Code.
Figure 2 Viewing the failure cause of a task
-
Deleting a Failed Task
You can delete one or all failed tasks at a time.
-
- To delete a failed task, perform the following operations: Locate the row that contains the target task and click Delete in the Operation column. In the displayed dialog box, confirm the task you want to delete and click Yes.
- To delete all failed tasks, perform the following operations: In the Failed Tasks dialog box, click Delete All. In the displayed dialog box, confirm the information about all failed tasks and click Yes.
-
Figure 3 Deleting a failed task
+
Deleting a Failed Task
You can delete one or all failed tasks at a time.
+
- To delete a failed task, perform the following operations: Locate the row that contains the target task and click Delete in the Operation column. In the displayed dialog box, confirm the task you want to delete and click Yes.
- To delete all failed tasks, perform the following operations: In the Failed Tasks dialog box, click Delete All. In the displayed dialog box, confirm the information about all failed tasks and click Yes.
+
Figure 3 Deleting a failed task
-
Error Code
-
Table 1 Failure causesError Code
+Error Code
+ Table 1 Failure causesError Code
|
-Failure Cause
+ | Failure Cause
|
-Solution
+ | Solution
|
|---|
-CSS.6000
+ | CSS.6000
|
-Failed to create the cluster because of an internal error. Please try again later. If the problem persists, contact customer service.
+ | Failed to create the cluster because of an internal error. Please try again later. If the problem persists, contact customer service.
|
-Please try again later or contact customer service.
+ | Please try again later or contact customer service.
|
-CSS.6001
+ | CSS.6001
|
-Failed to scale out the cluster because of an internal error. Please try again later. If the problem persists, contact customer service.
+ | Failed to scale out the cluster because of an internal error. Please try again later. If the problem persists, contact customer service.
|
-CSS.6002
+ | CSS.6002
|
-Failed to restart the cluster because of an internal error. Please try again later. If the problem persists, contact customer service.
+ | Failed to restart the cluster because of an internal error. Please try again later. If the problem persists, contact customer service.
|
-CSS.6003
+ | CSS.6003
|
-Failed to restore the cluster because of an internal error. Please try again later. If the problem persists, contact customer service.
+ | Failed to restore the cluster because of an internal error. Please try again later. If the problem persists, contact customer service.
|
-CSS.6004
+ | CSS.6004
|
-Failed to create the node because of ECS exceptions (<ECS error code>). Please try again later. If the problem persists, contact customer service.
- NOTE: <ECS error code> indicates the error information reported by ECS. For details about the cause and solution, see ECS Error Code Description.
+ Failed to create the node because of ECS exceptions (<ECS error code>). Please try again later. If the problem persists, contact customer service.
+ NOTE: <ECS error code> indicates the error information reported by ECS. For details about the cause and solution, see ECS Error Code Description.
|
|
-CSS.6005
+ | CSS.6005
|
-Failed to initialize the service because of an internal error. Please try again later. If the problem persists, contact customer service.
+ | Failed to initialize the service because of an internal error. Please try again later. If the problem persists, contact customer service.
|
-CSS.6007
+ | CSS.6007
|
-Failed to create the snapshot because of an internal error. Please try again later. If the problem persists, contact customer service.
+ | Failed to create the snapshot because of an internal error. Please try again later. If the problem persists, contact customer service.
|
-CSS.6008
+ | CSS.6008
|
-Failed to create the snapshot because the OBS bucket you select does not exist or has been deleted.
+ | Failed to create the snapshot because the OBS bucket you select does not exist or has been deleted.
|
-Modify the OBS bucket.
+ | Modify the OBS bucket.
|
-CSS.6009
+ | CSS.6009
|
-Failed to restore the snapshot because the OBS bucket you select does not exist or has been deleted.
+ | Failed to restore the snapshot because the OBS bucket you select does not exist or has been deleted.
|
-CSS.6010
+ | CSS.6010
|
-Failed to restore the snapshot because the OBS object does not exist or has been deleted.
+ | Failed to restore the snapshot because the OBS object does not exist or has been deleted.
|
diff --git a/docs/css/umn/css_01_0061.html b/docs/css/umn/css_01_0061.html
index 9b37de087..45560f3de 100644
--- a/docs/css/umn/css_01_0061.html
+++ b/docs/css/umn/css_01_0061.html
@@ -1,10 +1,10 @@
-
+
Using SQL to Search for Data in Elasticsearch
-For Elasticsearch 6.5.4 and later versions, Open Distro for Elasticsearch SQL lets you write queries in SQL rather than in the Elasticsearch query domain-specific language (DSL).
- Users who are familiar with SQL can use SQL to search for data in Elasticsearch clusters.
- |
|---|
