yexinru/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
Files
de9f8a7cc5b4e61816cbd4e52cc108c5f9fc3b4c
doc-exports/docs/css/umn/css_01_0405.html
zhengxiu 2125539080 css umn 25.1.0 version 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: zhengxiu <zhengxiu@huawei.com>
Co-committed-by: zhengxiu <zhengxiu@huawei.com>
2025-07-04 09:10:17 +00:00

1034 lines
85 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<a name="css_01_0405"></a><a name="css_01_0405"></a>
<h1 class="topictitle1">Configuring Storage-Compute Decoupling for an Elasticsearch Cluster</h1>
<div id="body0000001080601921"><p id="css_01_0405__p179481391618">CSS stores new data as hot data on SSDs to ensure optimal query performance, and historical data as cold data in OBS to cut storage costs.</p>
<div class="section" id="css_01_0405__section660612435475"><h4 class="sectiontitle">Scenario</h4><p id="css_01_0405__p15317202606">With storage-compute decoupling, you freeze an index and then transfer historical data to OBS for cold data storage. This allows you to reduce storage costs.</p>
<p id="css_01_0405__en-us_topic_0000001268314513_p920355781919">In terms of how frequent data is accessed, data can be divided into cold and hot data. Data newly written in is usually hot data, which is usually stored on SSDs for fast retrieval. As time goes by, the data is no longer updated and the query QPS decreases, the data has turned into cold data. By this time, you transfer the data to OBS for less expensive storage. In the meantime, you can still keep hot data on SSDs.</p>
<div class="fignone" id="css_01_0405__fig0485740132317"><span class="figcap"><b>Figure 1 </b>Decoupled storage and compute</span><br><span><img id="css_01_0405__image13485164072311" src="en-us_image_0000002060244816.png"></span></div>
<p id="css_01_0405__p12898123102410">Storage-compute decoupling enables index lifecycle management.</p>
<ul id="css_01_0405__ul47761588396"><li id="css_01_0405__li5776165873920">Hot indexes are writable, and their data can be retrieved within milliseconds.</li><li id="css_01_0405__li1077613582393">Frozen indexes store historical data that is rarely accessed for a long time and is dumped to an OBS bucket of CSS for less expensive storage. This also frees up SSDs for hot data. (Users are unaware of the existence of this OBS bucket and cannot check it on the OBS service console. However, the OBS usage does incur additional charges.) It may take seconds or even minutes to retrieve data stored in a frozen index.</li><li id="css_01_0405__li1277645813916">If historical data in OBS becomes obsolete, you can close the corresponding indexes to make these indexes unsearchable.</li><li id="css_01_0405__li677645803912">For closed indexes whose data is stored in OBS, you can change their storage class in OBS from Standard to Archive to further cut the storage cost.<p id="css_01_0405__p646805113106"><a name="css_01_0405__li677645803912"></a><a name="li677645803912"></a>By default, an archived index is unsearchable, but you can restore it to make it searchable while within the restoration validity period.</p>
</li><li id="css_01_0405__li109916916183">Index data stored in OBS Archive can be automatically deleted based on a predefined policy to reclaim storage resources.</li></ul>
<p id="css_01_0405__p14823181318418">This section describes how to use storage-compute decoupling after accessing a cluster using Kibana.</p>
</div>
<div class="section" id="css_01_0405__section1721134964711"><h4 class="sectiontitle">Constraints</h4><ul id="css_01_0405__ul1730013491548"><li id="css_01_0405__li1183914449201">Only Elasticsearch 7.6.2 and 7.10.2 clusters support decoupled storage and compute. Archive storage is available only for Elasticsearch 7.10.2 clusters whose image version is 24.3.0.C001 or later.</li></ul>
<ul id="css_01_0405__en-us_topic_0000001223594408_ul1922072711424"><li id="css_01_0405__li22681491413">While an index is being frozen, the system sets the index to the read-only state. Even after the data in the index is dumped to OBS, the index remains read-only and no data can be written into the index.</li><li id="css_01_0405__li6311417277">During index freezing, the data in it can still be queried. After the freezing is complete, the index is closed and then re-opened. After that, the index cannot be queried, and the cluster may be in the <strong id="css_01_0405__b16264135125511">red</strong> status for a short time. The index is restored after being opened.</li><li id="css_01_0405__li739316521373">Dumping the data in an index to OBS consumes network bandwidth.</li><li id="css_01_0405__li133759426222">After an index is frozen, the data in the index is dumped to OBS and the index data on the local disk is deleted. A frozen index has an increased query latency. During aggregation, the latency becomes even longer because the query is complex and a large amount of data needs to be retrieved.</li><li id="css_01_0405__li6269191011106">A frozen index with data already dumped to OBS cannot be unfrozen. That is, a read-only index cannot be rolled back to writable.</li><li id="css_01_0405__li181545384319">In the case of two clusters configured with read/write splitting, the secondary cluster cannot set storage-compute decoupling for indexes synchronized from the primary cluster.</li><li id="css_01_0405__li7742024658">The storage-compute decoupling feature depends on OBS. Therefore, you must comply with the restrictions on OBS bandwidth and QPS. If these restrictions are violated, the performance of queries on OBS will deteriorate. For example, the speed of restoring shards and querying data will become slow.</li></ul>
</div>
<div class="section" id="css_01_0405__section11265153174717"><h4 class="sectiontitle">Accessing a Cluster</h4><ol id="css_01_0405__ol124574417618"><li id="css_01_0405__en-us_topic_0000001223594408_li1274916552817">Log in to the CSS management console.</li><li id="css_01_0405__en-us_topic_0000001223594408_li1274935516811">On the <strong id="css_01_0405__b83771825103117">Clusters</strong> page, locate the target cluster, and click <strong id="css_01_0405__b189171427495">Access Kibana</strong> in the <strong id="css_01_0405__b19896191614314">Operation</strong> column to access Kibana.</li><li id="css_01_0405__en-us_topic_0000001223594408_li927171291011">Click <strong id="css_01_0405__b199439323210">Dev Tools</strong> in the navigation tree on the left.</li></ol>
</div>
<div class="section" id="css_01_0405__section1590610319481"><a name="css_01_0405__section1590610319481"></a><a name="section1590610319481"></a><h4 class="sectiontitle">Freezing an Index</h4><div class="p" id="css_01_0405__p20126131751017">Run the following command to dump the data of a specified index to OBS and set its storage class to Standard:<pre class="screen" id="css_01_0405__en-us_topic_0000001223594408_screen1911145510210">POST ${index_name}/_freeze_low_cost</pre>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="css_01_0405__en-us_topic_0000001223594408_table984014152137" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Request parameters</caption><thead align="left"><tr id="css_01_0405__en-us_topic_0000001223594408_row28415157136"><th align="left" class="cellrowborder" valign="top" width="30%" id="mcps1.3.5.2.2.2.3.1.1"><p id="css_01_0405__en-us_topic_0000001223594408_p1184113155139">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="70%" id="mcps1.3.5.2.2.2.3.1.2"><p id="css_01_0405__en-us_topic_0000001223594408_p17841315131319">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="css_01_0405__en-us_topic_0000001223594408_row10841115171317"><td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.5.2.2.2.3.1.1 "><p id="css_01_0405__en-us_topic_0000001223594408_p138411015161310">index_name</p>
</td>
<td class="cellrowborder" valign="top" width="70%" headers="mcps1.3.5.2.2.2.3.1.2 "><p id="css_01_0405__en-us_topic_0000001223594408_p5841161571312">Name of the index to be frozen.</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<p id="css_01_0405__en-us_topic_0000001223594408_p1499919920218">Information similar to the following is returned:</p>
<pre class="screen" id="css_01_0405__en-us_topic_0000001223594408_screen1714151681511">{
"freeze_uuid": "pdsRgUtSTymVDWR_HoTGFw"
}</pre>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="css_01_0405__en-us_topic_0000001223594408_table19357183012175" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Response parameters</caption><thead align="left"><tr id="css_01_0405__en-us_topic_0000001223594408_row135820307176"><th align="left" class="cellrowborder" valign="top" width="30%" id="mcps1.3.5.5.2.3.1.1"><p id="css_01_0405__en-us_topic_0000001223594408_p1629441171717">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="70%" id="mcps1.3.5.5.2.3.1.2"><p id="css_01_0405__en-us_topic_0000001223594408_p1262954119179">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="css_01_0405__en-us_topic_0000001223594408_row113588303174"><td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.5.5.2.3.1.1 "><p id="css_01_0405__en-us_topic_0000001223594408_p19358153013173">freeze_uuid</p>
</td>
<td class="cellrowborder" valign="top" width="70%" headers="mcps1.3.5.5.2.3.1.2 "><p id="css_01_0405__en-us_topic_0000001223594408_p835813021718">ID of the index freezing task. After an index freezing request is submitted, an asynchronous job is started, and an asynchronous job ID is returned, which can be used to query the progress of the index freezing task.</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="css_01_0405__section1659191294817"><h4 class="sectiontitle">Checking the Progress of an Index Freezing Task</h4><div class="p" id="css_01_0405__p81284244487">Run the following command to check the progress of the index freezing task:<pre class="screen" id="css_01_0405__en-us_topic_0000001223594408_screen04355311249">GET _freeze_low_cost_progress/${freeze_uuid}</pre>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="css_01_0405__en-us_topic_0000001223594408_table815581572510" frame="border" border="1" rules="all"><caption><b>Table 3 </b>Request parameters</caption><thead align="left"><tr id="css_01_0405__en-us_topic_0000001223594408_row191551815152516"><th align="left" class="cellrowborder" valign="top" width="27.6%" id="mcps1.3.6.2.2.2.3.1.1"><p id="css_01_0405__en-us_topic_0000001223594408_p19155415102511">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="72.39999999999999%" id="mcps1.3.6.2.2.2.3.1.2"><p id="css_01_0405__en-us_topic_0000001223594408_p315531513258">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="css_01_0405__en-us_topic_0000001223594408_row1115561517251"><td class="cellrowborder" valign="top" width="27.6%" headers="mcps1.3.6.2.2.2.3.1.1 "><p id="css_01_0405__en-us_topic_0000001223594408_p81551715182520">freeze_uuid</p>
</td>
<td class="cellrowborder" valign="top" width="72.39999999999999%" headers="mcps1.3.6.2.2.2.3.1.2 "><p id="css_01_0405__en-us_topic_0000001223594408_p315651572518">ID of the index freezing task, which is generated during <a href="#css_01_0405__section1590610319481">Freezing an Index</a>.</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<p id="css_01_0405__en-us_topic_0000001223594408_p1831717182714">Information similar to the following is returned:</p>
<pre class="screen" id="css_01_0405__en-us_topic_0000001223594408_screen11317815278">{
"stage" : "STARTED",
"shards_stats" : {
"INIT" : 0,
"FAILURE" : 0,
"DONE" : 0,
"STARTED" : 3,
"ABORTED" : 0
},
"indices" : {
"data1" : [
{
"uuid" : "7OS-G1-tRke2jHZPlckexg",
"index" : {
"name" : "data1",
"index_id" : "4b5PHXJITLaS6AurImfQ9A",
"shard" : 2
},
"start_ms" : 1611972010852,
"end_ms" : -1,
"total_time" : "10.5s",
"total_time_in_millis" : 10505,
"stage" : "STARTED",
"failure" : null,
"size" : {
"total_bytes" : 3211446689,
"finished_bytes" : 222491269,
"percent" : "6.0%"
},
"file" : {
"total_files" : 271,
"finished_files" : 12,
"percent" : "4.0%"
},
"rate_limit" : {
"paused_times" : 1,
"paused_nanos" : 946460970
}
},
{
"uuid" : "7OS-G1-tRke2jHZPlckexg",
"index" : {
"name" : "data1",
"index_id" : "4b5PHXJITLaS6AurImfQ9A",
"shard" : 0
},
"start_ms" : 1611972010998,
"end_ms" : -1,
"total_time" : "10.3s",
"total_time_in_millis" : 10359,
"stage" : "STARTED",
"failure" : null,
"size" : {
"total_bytes" : 3221418186,
"finished_bytes" : 272347118,
"percent" : "8.0%"
},
"file" : {
"total_files" : 372,
"finished_files" : 16,
"percent" : "4.0%"
},
"rate_limit" : {
"paused_times" : 5,
"paused_nanos" : 8269016764
}
},
{
"uuid" : "7OS-G1-tRke2jHZPlckexg",
"index" : {
"name" : "data1",
"index_id" : "4b5PHXJITLaS6AurImfQ9A",
"shard" : 1
},
"start_ms" : 1611972011021,
"end_ms" : -1,
"total_time" : "10.3s",
"total_time_in_millis" : 10336,
"stage" : "STARTED",
"failure" : null,
"size" : {
"total_bytes" : 3220787498,
"finished_bytes" : 305789614,
"percent" : "9.0%"
},
"file" : {
"total_files" : 323,
"finished_files" : 14,
"percent" : "4.0%"
},
"rate_limit" : {
"paused_times" : 3,
"paused_nanos" : 6057933087
}
}
]
}
}</pre>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="css_01_0405__en-us_topic_0000001223594408_table123173132712" frame="border" border="1" rules="all"><caption><b>Table 4 </b>Response parameters</caption><thead align="left"><tr id="css_01_0405__en-us_topic_0000001223594408_row1931718117276"><th align="left" class="cellrowborder" valign="top" width="20.919999999999998%" id="mcps1.3.6.5.2.3.1.1"><p id="css_01_0405__en-us_topic_0000001223594408_p43176122711">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="79.08%" id="mcps1.3.6.5.2.3.1.2"><p id="css_01_0405__en-us_topic_0000001223594408_p03181515273">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="css_01_0405__en-us_topic_0000001223594408_row1931813112279"><td class="cellrowborder" valign="top" width="20.919999999999998%" headers="mcps1.3.6.5.2.3.1.1 "><p id="css_01_0405__en-us_topic_0000001223594408_p19183717193312">stage</p>
</td>
<td class="cellrowborder" valign="top" width="79.08%" headers="mcps1.3.6.5.2.3.1.2 "><p id="css_01_0405__en-us_topic_0000001223594408_p17182111713330">Task status. Its value can be:</p>
<ul id="css_01_0405__en-us_topic_0000001223594408_ul1936411531014"><li id="css_01_0405__en-us_topic_0000001223594408_li5365553918"><strong id="css_01_0405__b211066461144330">INIT</strong>: The instance has just started or is being initialized.</li><li id="css_01_0405__en-us_topic_0000001223594408_li10461355515"><strong id="css_01_0405__b589373444330">FAILURE</strong>: failed</li><li id="css_01_0405__en-us_topic_0000001223594408_li156307587112"><strong id="css_01_0405__b70440472744330">DONE</strong>: complete</li><li id="css_01_0405__en-us_topic_0000001223594408_li4900191723"><strong id="css_01_0405__b163284008044330">STARTED</strong>: started</li><li id="css_01_0405__en-us_topic_0000001223594408_li192761956214"><strong id="css_01_0405__b174277939944330">ABORTED</strong>: canceled. This field is reserved.</li></ul>
</td>
</tr>
<tr id="css_01_0405__en-us_topic_0000001223594408_row13605194920218"><td class="cellrowborder" valign="top" width="20.919999999999998%" headers="mcps1.3.6.5.2.3.1.1 "><p id="css_01_0405__en-us_topic_0000001223594408_p76063495212">shards_stats</p>
</td>
<td class="cellrowborder" valign="top" width="79.08%" headers="mcps1.3.6.5.2.3.1.2 "><p id="css_01_0405__en-us_topic_0000001223594408_p1260618496217">Numbers of shards in each state.</p>
</td>
</tr>
<tr id="css_01_0405__en-us_topic_0000001223594408_row114106538217"><td class="cellrowborder" valign="top" width="20.919999999999998%" headers="mcps1.3.6.5.2.3.1.1 "><p id="css_01_0405__en-us_topic_0000001223594408_p13411753221">indices</p>
</td>
<td class="cellrowborder" valign="top" width="79.08%" headers="mcps1.3.6.5.2.3.1.2 "><p id="css_01_0405__en-us_topic_0000001223594408_p114119534214">Index status details.</p>
</td>
</tr>
</tbody>
</table>
</div>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="css_01_0405__en-us_topic_0000001223594408_table8976322045" frame="border" border="1" rules="all"><caption><b>Table 5 </b>Return values of <strong id="css_01_0405__b158038495844330">indices</strong></caption><thead align="left"><tr id="css_01_0405__en-us_topic_0000001223594408_row6976922742"><th align="left" class="cellrowborder" valign="top" width="28.63%" id="mcps1.3.6.6.2.3.1.1"><p id="css_01_0405__en-us_topic_0000001223594408_p7976162213415">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="71.37%" id="mcps1.3.6.6.2.3.1.2"><p id="css_01_0405__en-us_topic_0000001223594408_p17976922949">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="css_01_0405__en-us_topic_0000001223594408_row169761322344"><td class="cellrowborder" valign="top" width="28.63%" headers="mcps1.3.6.6.2.3.1.1 "><p id="css_01_0405__en-us_topic_0000001223594408_p113672515415">uuid</p>
</td>
<td class="cellrowborder" valign="top" width="71.37%" headers="mcps1.3.6.6.2.3.1.2 "><p id="css_01_0405__en-us_topic_0000001223594408_p936510511246">UUID of the freezing operation</p>
</td>
</tr>
<tr id="css_01_0405__en-us_topic_0000001223594408_row7995042642"><td class="cellrowborder" valign="top" width="28.63%" headers="mcps1.3.6.6.2.3.1.1 "><p id="css_01_0405__en-us_topic_0000001223594408_p179951742344">index</p>
</td>
<td class="cellrowborder" valign="top" width="71.37%" headers="mcps1.3.6.6.2.3.1.2 "><p id="css_01_0405__en-us_topic_0000001223594408_p59951242040">Index and shard information</p>
</td>
</tr>
<tr id="css_01_0405__en-us_topic_0000001223594408_row466317472044"><td class="cellrowborder" valign="top" width="28.63%" headers="mcps1.3.6.6.2.3.1.1 "><p id="css_01_0405__en-us_topic_0000001223594408_p1366410475412">start_ms</p>
</td>
<td class="cellrowborder" valign="top" width="71.37%" headers="mcps1.3.6.6.2.3.1.2 "><p id="css_01_0405__en-us_topic_0000001223594408_p136642479412">Start time</p>
</td>
</tr>
<tr id="css_01_0405__en-us_topic_0000001223594408_row166642471948"><td class="cellrowborder" valign="top" width="28.63%" headers="mcps1.3.6.6.2.3.1.1 "><p id="css_01_0405__en-us_topic_0000001223594408_p9664114719411">end_ms</p>
</td>
<td class="cellrowborder" valign="top" width="71.37%" headers="mcps1.3.6.6.2.3.1.2 "><p id="css_01_0405__en-us_topic_0000001223594408_p126648479418">End time. If no end time is specified, the value <strong id="css_01_0405__b114591834944330">-1</strong> is displayed.</p>
</td>
</tr>
<tr id="css_01_0405__en-us_topic_0000001223594408_row5239171416613"><td class="cellrowborder" valign="top" width="28.63%" headers="mcps1.3.6.6.2.3.1.1 "><p id="css_01_0405__en-us_topic_0000001223594408_p132402014763">total_time</p>
</td>
<td class="cellrowborder" valign="top" width="71.37%" headers="mcps1.3.6.6.2.3.1.2 "><p id="css_01_0405__en-us_topic_0000001223594408_p1224014141961">Time spent</p>
</td>
</tr>
<tr id="css_01_0405__en-us_topic_0000001223594408_row224017141669"><td class="cellrowborder" valign="top" width="28.63%" headers="mcps1.3.6.6.2.3.1.1 "><p id="css_01_0405__en-us_topic_0000001223594408_p122403141566">total_time_in_millis</p>
</td>
<td class="cellrowborder" valign="top" width="71.37%" headers="mcps1.3.6.6.2.3.1.2 "><p id="css_01_0405__en-us_topic_0000001223594408_p132409141669">Time spent, in milliseconds</p>
</td>
</tr>
<tr id="css_01_0405__en-us_topic_0000001223594408_row16240171410611"><td class="cellrowborder" valign="top" width="28.63%" headers="mcps1.3.6.6.2.3.1.1 "><p id="css_01_0405__en-us_topic_0000001223594408_p132404141065">stage</p>
</td>
<td class="cellrowborder" valign="top" width="71.37%" headers="mcps1.3.6.6.2.3.1.2 "><p id="css_01_0405__en-us_topic_0000001223594408_p724010141869">Status of the current shard.</p>
</td>
</tr>
<tr id="css_01_0405__en-us_topic_0000001223594408_row746911174617"><td class="cellrowborder" valign="top" width="28.63%" headers="mcps1.3.6.6.2.3.1.1 "><p id="css_01_0405__en-us_topic_0000001223594408_p1746910171467">failure</p>
</td>
<td class="cellrowborder" valign="top" width="71.37%" headers="mcps1.3.6.6.2.3.1.2 "><p id="css_01_0405__en-us_topic_0000001223594408_p184691017862">Failure cause. If no failure occurs, <strong id="css_01_0405__b166600894544330">null</strong> is returned.</p>
</td>
</tr>
<tr id="css_01_0405__en-us_topic_0000001223594408_row104691417564"><td class="cellrowborder" valign="top" width="28.63%" headers="mcps1.3.6.6.2.3.1.1 "><p id="css_01_0405__en-us_topic_0000001223594408_p346931718610">size.total_bytes</p>
</td>
<td class="cellrowborder" valign="top" width="71.37%" headers="mcps1.3.6.6.2.3.1.2 "><p id="css_01_0405__en-us_topic_0000001223594408_p94698171962">Size of files to be frozen, in bytes</p>
</td>
</tr>
<tr id="css_01_0405__en-us_topic_0000001223594408_row546919178617"><td class="cellrowborder" valign="top" width="28.63%" headers="mcps1.3.6.6.2.3.1.1 "><p id="css_01_0405__en-us_topic_0000001223594408_p1946901713610">size.finished_bytes</p>
</td>
<td class="cellrowborder" valign="top" width="71.37%" headers="mcps1.3.6.6.2.3.1.2 "><p id="css_01_0405__en-us_topic_0000001223594408_p546921710617">Frozen bytes</p>
</td>
</tr>
<tr id="css_01_0405__en-us_topic_0000001223594408_row123010401883"><td class="cellrowborder" valign="top" width="28.63%" headers="mcps1.3.6.6.2.3.1.1 "><p id="css_01_0405__en-us_topic_0000001223594408_p8301124019813">size.percent</p>
</td>
<td class="cellrowborder" valign="top" width="71.37%" headers="mcps1.3.6.6.2.3.1.2 "><p id="css_01_0405__en-us_topic_0000001223594408_p1630116408815">Percentage of frozen bytes</p>
</td>
</tr>
<tr id="css_01_0405__en-us_topic_0000001223594408_row630119401184"><td class="cellrowborder" valign="top" width="28.63%" headers="mcps1.3.6.6.2.3.1.1 "><p id="css_01_0405__en-us_topic_0000001223594408_p19859163599">file.total_bytes</p>
</td>
<td class="cellrowborder" valign="top" width="71.37%" headers="mcps1.3.6.6.2.3.1.2 "><p id="css_01_0405__en-us_topic_0000001223594408_p58594313913">Number of files to be frozen</p>
</td>
</tr>
<tr id="css_01_0405__en-us_topic_0000001223594408_row10302440489"><td class="cellrowborder" valign="top" width="28.63%" headers="mcps1.3.6.6.2.3.1.1 "><p id="css_01_0405__en-us_topic_0000001223594408_p128591237917">file.finished_bytes</p>
</td>
<td class="cellrowborder" valign="top" width="71.37%" headers="mcps1.3.6.6.2.3.1.2 "><p id="css_01_0405__en-us_topic_0000001223594408_p168591731596">Number of frozen files</p>
</td>
</tr>
<tr id="css_01_0405__en-us_topic_0000001223594408_row430274016813"><td class="cellrowborder" valign="top" width="28.63%" headers="mcps1.3.6.6.2.3.1.1 "><p id="css_01_0405__en-us_topic_0000001223594408_p78599312913">file.percent</p>
</td>
<td class="cellrowborder" valign="top" width="71.37%" headers="mcps1.3.6.6.2.3.1.2 "><p id="css_01_0405__en-us_topic_0000001223594408_p14859153499">Percentage of frozen files</p>
</td>
</tr>
<tr id="css_01_0405__en-us_topic_0000001223594408_row67821332892"><td class="cellrowborder" valign="top" width="28.63%" headers="mcps1.3.6.6.2.3.1.1 "><p id="css_01_0405__en-us_topic_0000001223594408_p1534019411299">rate_limit.paused_times</p>
</td>
<td class="cellrowborder" valign="top" width="71.37%" headers="mcps1.3.6.6.2.3.1.2 "><p id="css_01_0405__en-us_topic_0000001223594408_p168651733490">Number of times that freezing is suspended due to rate limiting</p>
</td>
</tr>
<tr id="css_01_0405__en-us_topic_0000001223594408_row137823323910"><td class="cellrowborder" valign="top" width="28.63%" headers="mcps1.3.6.6.2.3.1.1 "><p id="css_01_0405__en-us_topic_0000001223594408_p168661433791">rate_limit.paused_nanos</p>
</td>
<td class="cellrowborder" valign="top" width="71.37%" headers="mcps1.3.6.6.2.3.1.2 "><p id="css_01_0405__en-us_topic_0000001223594408_p3866113314912">Duration of freezing task suspension due to rate limiting, in nanoseconds</p>
</td>
</tr>
</tbody>
</table>
</div>
<p id="css_01_0405__en-us_topic_0000001223594408_p1758184111346">The following parameters are added to a frozen index. For details, see <a href="#css_01_0405__en-us_topic_0000001223594408_table1196310214353">Table 6</a>.</p>
<div class="tablenoborder"><a name="css_01_0405__en-us_topic_0000001223594408_table1196310214353"></a><a name="en-us_topic_0000001223594408_table1196310214353"></a><table cellpadding="4" cellspacing="0" summary="" id="css_01_0405__en-us_topic_0000001223594408_table1196310214353" frame="border" border="1" rules="all"><caption><b>Table 6 </b>Frozen index parameters</caption><thead align="left"><tr id="css_01_0405__en-us_topic_0000001223594408_row189632213354"><th align="left" class="cellrowborder" valign="top" width="37.940000000000005%" id="mcps1.3.6.8.2.3.1.1"><p id="css_01_0405__en-us_topic_0000001223594408_p19963420354">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="62.06%" id="mcps1.3.6.8.2.3.1.2"><p id="css_01_0405__en-us_topic_0000001223594408_p16864914153520">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="css_01_0405__en-us_topic_0000001223594408_row1963229359"><td class="cellrowborder" valign="top" width="37.940000000000005%" headers="mcps1.3.6.8.2.3.1.1 "><p id="css_01_0405__en-us_topic_0000001223594408_p49631428356">index.frozen_low_cost</p>
</td>
<td class="cellrowborder" valign="top" width="62.06%" headers="mcps1.3.6.8.2.3.1.2 "><p id="css_01_0405__en-us_topic_0000001223594408_p18642146353">Whether an index is frozen. The value is <strong id="css_01_0405__b184511171044330">true</strong>.</p>
</td>
</tr>
<tr id="css_01_0405__en-us_topic_0000001223594408_row169632210353"><td class="cellrowborder" valign="top" width="37.940000000000005%" headers="mcps1.3.6.8.2.3.1.1 "><p id="css_01_0405__en-us_topic_0000001223594408_p296319273519">index.blocks.write</p>
</td>
<td class="cellrowborder" valign="top" width="62.06%" headers="mcps1.3.6.8.2.3.1.2 "><p id="css_01_0405__en-us_topic_0000001223594408_p20864114143517">Whether data writing to a frozen index is disallowed. The value is <strong id="css_01_0405__b71243778744330">true</strong>.</p>
</td>
</tr>
<tr id="css_01_0405__en-us_topic_0000001223594408_row16964921359"><td class="cellrowborder" valign="top" width="37.940000000000005%" headers="mcps1.3.6.8.2.3.1.1 "><p id="css_01_0405__en-us_topic_0000001223594408_p1796412220350">index.store.type</p>
</td>
<td class="cellrowborder" valign="top" width="62.06%" headers="mcps1.3.6.8.2.3.1.2 "><p id="css_01_0405__en-us_topic_0000001223594408_p7864181419352">Storage type of an index. The value is <strong id="css_01_0405__b22069174344330">obs</strong>.</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="css_01_0405__section15466184120501"><h4 class="sectiontitle">Querying the Index List Based on Freezing Status</h4><div class="p" id="css_01_0405__p1337418813155">Run the following command to query the index list based on freezing status:<pre class="screen" id="css_01_0405__en-us_topic_0000001223594408_screen111712156459">GET _cat/freeze_indices?stage=${STAGE}</pre>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="css_01_0405__table913682872811" frame="border" border="1" rules="all"><caption><b>Table 7 </b>Request parameters</caption><thead align="left"><tr id="css_01_0405__row1113662882811"><th align="left" class="cellrowborder" valign="top" width="30%" id="mcps1.3.7.2.2.2.3.1.1"><p id="css_01_0405__p113662862818">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="70%" id="mcps1.3.7.2.2.2.3.1.2"><p id="css_01_0405__p41361284287">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="css_01_0405__row15136112818282"><td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.7.2.2.2.3.1.1 "><p id="css_01_0405__p191361428142819">STAGE</p>
</td>
<td class="cellrowborder" valign="top" width="70%" headers="mcps1.3.7.2.2.2.3.1.2 "><p id="css_01_0405__p141361628142812">Index freezing status. The values are as follows:</p>
<ul id="css_01_0405__ul142391912102915"><li id="css_01_0405__li1423901219298"><strong id="css_01_0405__b84584922944330">start</strong>: List of indexes that are being frozen.</li><li id="css_01_0405__li6972240102916"><strong id="css_01_0405__b156926245744330">done</strong>: List of indexes that have been frozen.</li><li id="css_01_0405__li578592013309"><strong id="css_01_0405__b132229728044330">unfreeze</strong>: List of indexes that are not frozen.</li><li id="css_01_0405__li1741173373010">Empty or other values: List of all indexes that are being frozen or have been frozen.</li></ul>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<p id="css_01_0405__en-us_topic_0000001223594408_p16515114484418">Information similar to the following is returned:</p>
<pre class="screen" id="css_01_0405__en-us_topic_0000001223594408_screen1056385184511">green open data2 0bNtxWDtRbOSkS4JYaUgMQ 3 0 5 0 7.9kb 7.9kb
green open data3 oYMLvw31QnyasqUNuyP6RA 3 0 51 0 23.5kb 23.5kb</pre>
<div class="note" id="css_01_0405__en-us_topic_0000001223594408_note1058331715368"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="css_01_0405__p3121118104117">The parameters and return values of this command are the same as those of <b><span class="cmdname" id="css_01_0405__cmdname135172318589">_cat/indices</span></b> of open-source Elasticsearch.</p>
</div></div>
</div>
<div class="section" id="css_01_0405__section147861486269"><h4 class="sectiontitle">Archiving a Frozen Index</h4><div class="note" id="css_01_0405__note45926202713"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="css_01_0405__ul167911514133017"><li id="css_01_0405__li226141813311">Archive storage is available only for Elasticsearch 7.10.2 clusters whose image version is 24.3.0.C001 or later.</li><li id="css_01_0405__li20791121473012">Only indexes that are frozen and closed can be archived.</li><li id="css_01_0405__li6934044121519">If you try to open an archived index directly, the request will time out and the index state will turn RED. Instead, you should first restore the archived index, convert its storage class from Archive to Standard, and then wait for it to open by itself.</li></ul>
</div></div>
<p id="css_01_0405__p278528182617">Run the following command to convert the storage class of a frozen index in OBS from Standard to Archive:</p>
<pre class="screen" id="css_01_0405__screen137851282263">POST {index_name}/_freeze_archive</pre>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="css_01_0405__table7785188202615" frame="border" border="1" rules="all"><caption><b>Table 8 </b>Request parameters</caption><thead align="left"><tr id="css_01_0405__row16785118122611"><th align="left" class="cellrowborder" valign="top" width="30%" id="mcps1.3.8.5.2.3.1.1"><p id="css_01_0405__p13785158132619">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="70%" id="mcps1.3.8.5.2.3.1.2"><p id="css_01_0405__p77851682269">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="css_01_0405__row17785188122615"><td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.8.5.2.3.1.1 "><p id="css_01_0405__p1278578182618">index_name</p>
</td>
<td class="cellrowborder" valign="top" width="70%" headers="mcps1.3.8.5.2.3.1.2 "><p id="css_01_0405__p10785138162612">Name of the index to archive.</p>
</td>
</tr>
</tbody>
</table>
</div>
<p id="css_01_0405__p17855811268">Information similar to the following is returned:</p>
<pre class="screen" id="css_01_0405__screen17785178152612">{
"task_uuid" : "XJTn3hsHSaCjPnvIgv-FFw"
}</pre>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="css_01_0405__table167852832610" frame="border" border="1" rules="all"><caption><b>Table 9 </b>Response parameters</caption><thead align="left"><tr id="css_01_0405__row117859822611"><th align="left" class="cellrowborder" valign="top" width="30%" id="mcps1.3.8.8.2.3.1.1"><p id="css_01_0405__p578510872610">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="70%" id="mcps1.3.8.8.2.3.1.2"><p id="css_01_0405__p167859817263">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="css_01_0405__row37858872614"><td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.8.8.2.3.1.1 "><p id="css_01_0405__p6785883266">task_uuid</p>
</td>
<td class="cellrowborder" valign="top" width="70%" headers="mcps1.3.8.8.2.3.1.2 "><p id="css_01_0405__p1378513802618">ID of the index archiving task. After an index archiving request is submitted, an asynchronous job is started, and an asynchronous job ID is returned, which can be used to query the progress of the index archiving task.</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="css_01_0405__section1948385613354"><h4 class="sectiontitle">Restoring an Archived Index</h4><p id="css_01_0405__p048295693514">Run the following command to convert the storage class of an archived index in OBS from Archive to Standard: Once restored, an archived index will be opened automatically and become searchable while within the restoration validity period.</p>
<pre class="screen" id="css_01_0405__screen1948225617355">POST {index_name}/_freeze_archive_restore
{
"days": 1,
"tier": "EXPEDITED"
}</pre>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="css_01_0405__table5483556163518" frame="border" border="1" rules="all"><caption><b>Table 10 </b>Request parameters</caption><thead align="left"><tr id="css_01_0405__row2482956103515"><th align="left" class="cellrowborder" valign="top" width="30%" id="mcps1.3.9.4.2.3.1.1"><p id="css_01_0405__p104821356123516">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="70%" id="mcps1.3.9.4.2.3.1.2"><p id="css_01_0405__p10482145693512">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="css_01_0405__row164821256183518"><td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.9.4.2.3.1.1 "><p id="css_01_0405__p448211565352">index_name</p>
</td>
<td class="cellrowborder" valign="top" width="70%" headers="mcps1.3.9.4.2.3.1.2 "><p id="css_01_0405__p15482175612357">Name of the index to restore.</p>
</td>
</tr>
<tr id="css_01_0405__row104822566358"><td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.9.4.2.3.1.1 "><p id="css_01_0405__p104822056153519">days</p>
</td>
<td class="cellrowborder" valign="top" width="70%" headers="mcps1.3.9.4.2.3.1.2 "><p id="css_01_0405__p1648225611355">After an archived index is restored, a Standard copy of this index is generated. This parameter specifies for how long (in days) this Standard copy can be retained, that is, the validity period of the restored object. Upon expiration of this period, the file objects in the index transition back to Archive storage, and the index is also closed.</p>
<p id="css_01_0405__p74822056113518">Value range: 1–30</p>
<p id="css_01_0405__p19821744475">Default value: <strong id="css_01_0405__b109290813611053">1</strong></p>
<p id="css_01_0405__p64827569355">Unit: days</p>
</td>
</tr>
<tr id="css_01_0405__row1848235663511"><td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.9.4.2.3.1.1 "><p id="css_01_0405__p184824569356">tier</p>
</td>
<td class="cellrowborder" valign="top" width="70%" headers="mcps1.3.9.4.2.3.1.2 "><p id="css_01_0405__p74822569355">Restoration option. The value can be <strong id="css_01_0405__b2054071116226">EXPEDITED</strong> or <strong id="css_01_0405__b318814159221">STANDARD</strong>, which vary in billing rates.</p>
<ul id="css_01_0405__ul10868122210474"><li id="css_01_0405__li16868162216472"><strong id="css_01_0405__b1314417292236">EXPEDITED</strong>: The restoration takes 5 minutes. The index becomes available automatically after restoration.</li><li id="css_01_0405__li10868172214716"><strong id="css_01_0405__b0466325122411">Standard</strong>: The restoration takes 5 hours. The index becomes available automatically after restoration.</li></ul>
<p id="css_01_0405__p1198335519474">Default value: <strong id="css_01_0405__b1833617422249">EXPEDITED</strong></p>
</td>
</tr>
</tbody>
</table>
</div>
<p id="css_01_0405__p34837561357">Information similar to the following is returned:</p>
<pre class="screen" id="css_01_0405__screen848305693513">{
"task_uuid" : "FA2a5hWgQ66MQ7ENCFKjCg"
}</pre>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="css_01_0405__table94831556143511" frame="border" border="1" rules="all"><caption><b>Table 11 </b>Response parameters</caption><thead align="left"><tr id="css_01_0405__row1648316563358"><th align="left" class="cellrowborder" valign="top" width="30%" id="mcps1.3.9.7.2.3.1.1"><p id="css_01_0405__p164831456173514">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="70%" id="mcps1.3.9.7.2.3.1.2"><p id="css_01_0405__p7483105613355">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="css_01_0405__row124831756113513"><td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.9.7.2.3.1.1 "><p id="css_01_0405__p2048315561351">task_uuid</p>
</td>
<td class="cellrowborder" valign="top" width="70%" headers="mcps1.3.9.7.2.3.1.2 "><p id="css_01_0405__p648325643511">ID of the index restoration task. After an index restoration request is submitted, an asynchronous job is started, and an asynchronous job ID is returned, which can be used to query the progress of the index restoration task.</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="css_01_0405__section1014993835513"><h4 class="sectiontitle">Checking the Progress of an Index Archiving or Restoration Task</h4><p id="css_01_0405__p814803855512">Run the following command to check the status and progress of an index archiving or restoration task:</p>
<pre class="screen" id="css_01_0405__screen171481538115516">GET _freeze_archive_progress/{task_uuid}</pre>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="css_01_0405__table1149163819552" frame="border" border="1" rules="all"><caption><b>Table 12 </b>Request parameters</caption><thead align="left"><tr id="css_01_0405__row16149173885513"><th align="left" class="cellrowborder" valign="top" width="27.6%" id="mcps1.3.10.4.2.3.1.1"><p id="css_01_0405__p71482384558">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="72.39999999999999%" id="mcps1.3.10.4.2.3.1.2"><p id="css_01_0405__p13149123845515">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="css_01_0405__row131493388552"><td class="cellrowborder" valign="top" width="27.6%" headers="mcps1.3.10.4.2.3.1.1 "><p id="css_01_0405__p314919382550">task_uuid</p>
</td>
<td class="cellrowborder" valign="top" width="72.39999999999999%" headers="mcps1.3.10.4.2.3.1.2 "><p id="css_01_0405__p1714973845519">ID of an index archiving or restoration task. This ID is obtained when a frozen index is archived or when an archived index is restored.</p>
</td>
</tr>
</tbody>
</table>
</div>
<p id="css_01_0405__p11491038125513">Information similar to the following is returned:</p>
<pre class="screen" id="css_01_0405__screen2149538115517">{
"id" : "archive:XJTn3hsHSaCjPnvIgv-FFw",
"indices" : [
{
"index_name" : "log-1017", //Index name
"index_uuid" : "HXjhvo9QTwSvjzGBBZBCTw" //Index ID
}
],
"stage" : "ARCHIVE_DONE", //Current status
"type" : "archive", //The value can be archive or restore.
"node" : "css-d0e8-ess-esn-1-1", //The name of the node where the task was executed.
"start" : 1729222604074, //Task start time.
"time_in_millis" : 788, //How long the task lasted.
"files" : {
"total" : 30, //Total number of files.
"success" : 30, //Number of files succeeded.
"incremental" : 0, //Number of incrementally archived files in case the archive is performed multiple times.
"bytes" : 0, //Total number of bytes restored. Only an index restoration task has this parameter.
"fail" : 0 //Failed files.
}
}</pre>
</div>
<div class="section" id="css_01_0405__section94441748185718"><h4 class="sectiontitle">Querying the Index List Based on Archive or Restoration Status</h4><p id="css_01_0405__p184438485579">Run the following command to query the index list based on archive or restoration status:</p>
<pre class="screen" id="css_01_0405__screen4443184815710">GET _cat/archive_indices?stage={STAGE}</pre>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="css_01_0405__table1144434825710" frame="border" border="1" rules="all"><caption><b>Table 13 </b>Request parameters</caption><thead align="left"><tr id="css_01_0405__row8443144805711"><th align="left" class="cellrowborder" valign="top" width="30%" id="mcps1.3.11.4.2.3.1.1"><p id="css_01_0405__p1544317481571">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="70%" id="mcps1.3.11.4.2.3.1.2"><p id="css_01_0405__p64437488571">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="css_01_0405__row1744434814579"><td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.11.4.2.3.1.1 "><p id="css_01_0405__p15443134895719">STAGE</p>
</td>
<td class="cellrowborder" valign="top" width="70%" headers="mcps1.3.11.4.2.3.1.2 "><p id="css_01_0405__p194431348155717">Index archiving or restoration status. The values are as follows:</p>
<ul id="css_01_0405__ul744434816579"><li id="css_01_0405__li84431348175720">ARCHIVE_INIT: The archiving task has been received but has not started yet.</li><li id="css_01_0405__li944318485577">ARCHIVE_STARTED: The archiving task has started.</li><li id="css_01_0405__li8443124812577">ARCHIVE_PARTIAL: The archiving task is partially successful.</li><li id="css_01_0405__li17443848115717">ARCHIVE_DONE: The archiving task is completed.</li><li id="css_01_0405__li14432484576">ARCHIVE_FAILURE: The archiving task failed.</li><li id="css_01_0405__li1244312483578">RESTORE_INIT: The restoration task has been received but has not started yet.</li><li id="css_01_0405__li44438482577">RESTORE_STARTED: The restoration task has started.</li><li id="css_01_0405__li644334816577">RESTORE_PARTIAL: The restoration task is partially successful.</li><li id="css_01_0405__li1443184820574">RESTORE_DONE: The restoration task is successful. The files in the index still take some time to recover.</li><li id="css_01_0405__li6444174814577">RESTORE_FAILURE: The restoration task failed.</li><li id="css_01_0405__li8444154825715">RESTORE_OPENED: The restoration task has been completed, and the index is now available.</li><li id="css_01_0405__li19444174819573">RESTORE_CLOSED: The validity period of the restored index has expired. The index is now closed.</li></ul>
</td>
</tr>
</tbody>
</table>
</div>
<p id="css_01_0405__p144454865715">Information similar to the following is returned:</p>
<pre class="screen" id="css_01_0405__screen04441848155715">health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
close log-105 M0uRAWj_SKydjg0dFzyJow </pre>
<div class="note" id="css_01_0405__note2044464814571"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="css_01_0405__p174446483574">The parameters and return values of this command are the same as those of <b><span class="cmdname" id="css_01_0405__cmdname34440489579">_cat/indices</span></b> of open-source Elasticsearch.</p>
</div></div>
</div>
<div class="section" id="css_01_0405__section724364691115"><h4 class="sectiontitle">Modifying Cache Settings for Cold Data Stored in OBS</h4><p id="css_01_0405__en-us_topic_0000001223594444_p8060118">After data is dumped to OBS, some data is cached to reduce access to OBS and improve cluster query performance. Data that is requested for the first time is retrieved from OBS. The retrieved data is then cached in the cluster memory. In response to subsequent queries, the system searches for data in the cache first.</p>
<p id="css_01_0405__en-us_topic_0000001223594444_p129311346176">Elasticsearch accesses different files using different methods. The cache system supports multi-level cache and uses blocks of different sizes to cache different files. For example, a large number of small blocks are used to cache .fdx and .tip files, while a small number of large blocks are used to cache .fdt files. The cache configuration can be modified based on service requirements. For details about the configuration items, see <a href="#css_01_0405__en-us_topic_0000001223594444_table1151755661711">Table 14</a>.</p>
<div class="tablenoborder"><a name="css_01_0405__en-us_topic_0000001223594444_table1151755661711"></a><a name="en-us_topic_0000001223594444_table1151755661711"></a><table cellpadding="4" cellspacing="0" summary="" id="css_01_0405__en-us_topic_0000001223594444_table1151755661711" frame="border" border="1" rules="all"><caption><b>Table 14 </b>Cache configuration items</caption><thead align="left"><tr id="css_01_0405__en-us_topic_0000001223594444_row17590165611178"><th align="left" class="cellrowborder" valign="top" width="31.47314731473147%" id="mcps1.3.12.4.2.4.1.1"><p id="css_01_0405__en-us_topic_0000001223594444_p15901056101712">Configuration Item</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="12.281228122812282%" id="mcps1.3.12.4.2.4.1.2"><p id="css_01_0405__en-us_topic_0000001223594444_p359014569179">Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="56.24562456245624%" id="mcps1.3.12.4.2.4.1.3"><p id="css_01_0405__en-us_topic_0000001223594444_p1459118561179">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="css_01_0405__en-us_topic_0000001223594444_row1159115568176"><td class="cellrowborder" valign="top" width="31.47314731473147%" headers="mcps1.3.12.4.2.4.1.1 "><p id="css_01_0405__en-us_topic_0000001223594444_p1659110566174">low_cost.obs.blockcache.names</p>
</td>
<td class="cellrowborder" valign="top" width="12.281228122812282%" headers="mcps1.3.12.4.2.4.1.2 "><p id="css_01_0405__en-us_topic_0000001223594444_p14591956111713">Array</p>
</td>
<td class="cellrowborder" valign="top" width="56.24562456245624%" headers="mcps1.3.12.4.2.4.1.3 "><p id="css_01_0405__en-us_topic_0000001223594444_p15591155614178">The cache system supports multi-level cache for data of different access granularities. This configuration lists the names of all caches. If this parameter is not set, the system has a cache named <strong id="css_01_0405__b122288975244330">default</strong>. To customize the configuration, ensure there is a cache named <strong id="css_01_0405__b138034565944330">default</strong>.</p>
<p id="css_01_0405__en-us_topic_0000001223594444_p99572571781">Default value: <strong id="css_01_0405__b3901200144330">default</strong></p>
</td>
</tr>
<tr id="css_01_0405__en-us_topic_0000001223594444_row173321211195613"><td class="cellrowborder" valign="top" width="31.47314731473147%" headers="mcps1.3.12.4.2.4.1.1 "><p id="css_01_0405__en-us_topic_0000001223594444_p83339114563">low_cost.obs.blockcache.&lt;NAME&gt;.type</p>
</td>
<td class="cellrowborder" valign="top" width="12.281228122812282%" headers="mcps1.3.12.4.2.4.1.2 "><p id="css_01_0405__en-us_topic_0000001223594444_p113331811105616">ENUM</p>
</td>
<td class="cellrowborder" valign="top" width="56.24562456245624%" headers="mcps1.3.12.4.2.4.1.3 "><p id="css_01_0405__en-us_topic_0000001223594444_p179446103311">Cache type, which can be <strong id="css_01_0405__b214575424644330">memory</strong> or <strong id="css_01_0405__b139215086944330">file</strong>.</p>
<p id="css_01_0405__en-us_topic_0000001223594444_p11333311185613">If it is set to <strong id="css_01_0405__b58019014244330">memory</strong>, certain memory capacity will be occupied. If it is set to <strong id="css_01_0405__b168851346544330">file</strong>, cache will be stored on disks. You are advised to use ultra-high I/O disks to improve cache performance.</p>
<p id="css_01_0405__en-us_topic_0000001223594444_p1516810101095">Default value: <strong id="css_01_0405__b192161692644330">memory</strong></p>
</td>
</tr>
<tr id="css_01_0405__en-us_topic_0000001223594444_row15911656151712"><td class="cellrowborder" valign="top" width="31.47314731473147%" headers="mcps1.3.12.4.2.4.1.1 "><p id="css_01_0405__en-us_topic_0000001223594444_p1359165616177">low_cost.obs.blockcache.&lt;NAME&gt;.blockshift</p>
</td>
<td class="cellrowborder" valign="top" width="12.281228122812282%" headers="mcps1.3.12.4.2.4.1.2 "><p id="css_01_0405__en-us_topic_0000001223594444_p11591105611712">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="56.24562456245624%" headers="mcps1.3.12.4.2.4.1.3 "><p id="css_01_0405__en-us_topic_0000001223594444_p1059125681719">Size of each block in the cache. Its value is the number of bytes shifted left. For example, if this parameter is set to <strong id="css_01_0405__b80883271944330">16</strong>, the block size is <strong id="css_01_0405__b34221446644330">2</strong><strong id="css_01_0405__b150013803744330"><sup id="css_01_0405__sup56708084044330">16</sup></strong> bytes, that is, 65536 bytes (64 KB).</p>
<p id="css_01_0405__en-us_topic_0000001223594444_p694281611111">Default value: <strong id="css_01_0405__b180221489944330">13</strong> (8 KB)</p>
</td>
</tr>
<tr id="css_01_0405__en-us_topic_0000001223594444_row11591195613171"><td class="cellrowborder" valign="top" width="31.47314731473147%" headers="mcps1.3.12.4.2.4.1.1 "><p id="css_01_0405__en-us_topic_0000001223594444_p18591135614172">low_cost.obs.blockcache.&lt;NAME&gt;.bank.count</p>
</td>
<td class="cellrowborder" valign="top" width="12.281228122812282%" headers="mcps1.3.12.4.2.4.1.2 "><p id="css_01_0405__en-us_topic_0000001223594444_p259115613178">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="56.24562456245624%" headers="mcps1.3.12.4.2.4.1.3 "><p id="css_01_0405__en-us_topic_0000001223594444_p125921256111712">Number of cache partitions.</p>
<p id="css_01_0405__en-us_topic_0000001223594444_p122062613119">Default value: <strong id="css_01_0405__b99401130744330">1</strong></p>
</td>
</tr>
<tr id="css_01_0405__en-us_topic_0000001223594444_row105921256161713"><td class="cellrowborder" valign="top" width="31.47314731473147%" headers="mcps1.3.12.4.2.4.1.1 "><p id="css_01_0405__en-us_topic_0000001223594444_p145938568179">low_cost.obs.blockcache.&lt;NAME&gt;.number.blocks.perbank</p>
</td>
<td class="cellrowborder" valign="top" width="12.281228122812282%" headers="mcps1.3.12.4.2.4.1.2 "><p id="css_01_0405__en-us_topic_0000001223594444_p959385671710">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="56.24562456245624%" headers="mcps1.3.12.4.2.4.1.3 "><p id="css_01_0405__en-us_topic_0000001223594444_p16593155641711">Number of blocks inside each cache partition.</p>
<p id="css_01_0405__en-us_topic_0000001223594444_p1215173411112">Default value: <strong id="css_01_0405__b88843559344330">8192</strong></p>
</td>
</tr>
<tr id="css_01_0405__en-us_topic_0000001223594444_row1059317567175"><td class="cellrowborder" valign="top" width="31.47314731473147%" headers="mcps1.3.12.4.2.4.1.1 "><p id="css_01_0405__en-us_topic_0000001223594444_p18593185618176">low_cost.obs.blockcache. &lt;NAME&gt;.exclude.file.types</p>
</td>
<td class="cellrowborder" valign="top" width="12.281228122812282%" headers="mcps1.3.12.4.2.4.1.2 "><p id="css_01_0405__en-us_topic_0000001223594444_p17593195612172">Array</p>
</td>
<td class="cellrowborder" valign="top" width="56.24562456245624%" headers="mcps1.3.12.4.2.4.1.3 "><p id="css_01_0405__en-us_topic_0000001223594444_p18593356151717">Extensions of files that are not cached. If the extensions of certain files are neither in the <strong id="css_01_0405__b116786104144330">exclude</strong> list nor in the <strong id="css_01_0405__b55391868344330">include</strong> list, they are stored in the default cache.</p>
</td>
</tr>
<tr id="css_01_0405__en-us_topic_0000001223594444_row75931556191711"><td class="cellrowborder" valign="top" width="31.47314731473147%" headers="mcps1.3.12.4.2.4.1.1 "><p id="css_01_0405__en-us_topic_0000001223594444_p6593156171718">low_cost.obs.blockcache. &lt;NAME&gt;.file.types</p>
</td>
<td class="cellrowborder" valign="top" width="12.281228122812282%" headers="mcps1.3.12.4.2.4.1.2 "><p id="css_01_0405__en-us_topic_0000001223594444_p1759314566173">Array</p>
</td>
<td class="cellrowborder" valign="top" width="56.24562456245624%" headers="mcps1.3.12.4.2.4.1.3 "><p id="css_01_0405__en-us_topic_0000001223594444_p3593105631714">Extensions of cached files. If the extensions of certain files are neither in the <strong id="css_01_0405__b109230786544330">exclude</strong> list nor in the <strong id="css_01_0405__b122722239944330">include</strong> list, they are stored in the default cache.</p>
</td>
</tr>
<tr id="css_01_0405__row95749293312"><td class="cellrowborder" valign="top" width="31.47314731473147%" headers="mcps1.3.12.4.2.4.1.1 "><p id="css_01_0405__p1266923663119">index.frozen.obs.max_bytes_per_sec</p>
</td>
<td class="cellrowborder" valign="top" width="12.281228122812282%" headers="mcps1.3.12.4.2.4.1.2 "><p id="css_01_0405__p3669103615319">String</p>
</td>
<td class="cellrowborder" valign="top" width="56.24562456245624%" headers="mcps1.3.12.4.2.4.1.3 "><p id="css_01_0405__p5669436143120">Maximum rate of uploading files to OBS during freezing. It takes effect immediately after you submit configuration.</p>
<p id="css_01_0405__p126691936113117">Default value: <strong id="css_01_0405__b93555508444330">150 MB</strong></p>
</td>
</tr>
<tr id="css_01_0405__row158941929123117"><td class="cellrowborder" valign="top" width="31.47314731473147%" headers="mcps1.3.12.4.2.4.1.1 "><p id="css_01_0405__p8669183653117">low_cost.obs.index.upload.threshold.use.multipart</p>
</td>
<td class="cellrowborder" valign="top" width="12.281228122812282%" headers="mcps1.3.12.4.2.4.1.2 "><p id="css_01_0405__p566993617315">String</p>
</td>
<td class="cellrowborder" valign="top" width="56.24562456245624%" headers="mcps1.3.12.4.2.4.1.3 "><p id="css_01_0405__p46696365315">If the file size exceeds the value of this parameter during freezing, the multipart upload function of OBS is used.</p>
<p id="css_01_0405__p13669123653112">Default value: <strong id="css_01_0405__b110877825044330">1 GB</strong></p>
</td>
</tr>
<tr id="css_01_0405__row1021514302312"><td class="cellrowborder" valign="top" width="31.47314731473147%" headers="mcps1.3.12.4.2.4.1.1 "><p id="css_01_0405__p1567033610318">index.frozen.reader.cache.expire.duration.seconds</p>
</td>
<td class="cellrowborder" valign="top" width="12.281228122812282%" headers="mcps1.3.12.4.2.4.1.2 "><p id="css_01_0405__p1767073633112">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="56.24562456245624%" headers="mcps1.3.12.4.2.4.1.3 "><p id="css_01_0405__p1867053633115">Timeout duration.</p>
<p id="css_01_0405__p18670203693114">To reduce the heap memory occupied by frozen indexes, the reader caches data for a period of time after the index shard is started, and stops caching after it times out.</p>
<p id="css_01_0405__p136705368319">Default value: <strong id="css_01_0405__b90831998844330">300s</strong></p>
</td>
</tr>
<tr id="css_01_0405__row550903063111"><td class="cellrowborder" valign="top" width="31.47314731473147%" headers="mcps1.3.12.4.2.4.1.1 "><p id="css_01_0405__p15670036173112">index.frozen.reader.cache.max.size</p>
</td>
<td class="cellrowborder" valign="top" width="12.281228122812282%" headers="mcps1.3.12.4.2.4.1.2 "><p id="css_01_0405__p36701636173118">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="56.24562456245624%" headers="mcps1.3.12.4.2.4.1.3 "><p id="css_01_0405__p7670436103110">Maximum cache size.</p>
<p id="css_01_0405__p1167053620310">Default value: <strong id="css_01_0405__b10566970444330">100</strong></p>
</td>
</tr>
</tbody>
</table>
</div>
<p id="css_01_0405__en-us_topic_0000001223594444_p66542107195">The following is a common cache configuration. It uses two levels of caches: <strong id="css_01_0405__b46414293019">default</strong> and <strong id="css_01_0405__b142405321507">large</strong>. The <strong id="css_01_0405__b12975125713010">default</strong> cache uses 64-KB blocks and has a total of 30 x 4096 blocks. It is used to cache non-.fdt files. The <strong id="css_01_0405__b194081047216">large</strong> cache uses 2-MB blocks and contains 5 x 1000 blocks. It is used to cache .fdx, .dvd, and .tip files.</p>
<pre class="screen" id="css_01_0405__en-us_topic_0000001223594444_screen196313311802">low_cost.obs.blockcache.names: ["default", "large"]
low_cost.obs.blockcache.default.type: file
low_cost.obs.blockcache.default.blockshift: 16
low_cost.obs.blockcache.default.number.blocks.perbank: 4096
low_cost.obs.blockcache.default.bank.count: 30
low_cost.obs.blockcache.default.exclude.file.types: ["fdt"]
low_cost.obs.blockcache.large.type: file
low_cost.obs.blockcache.large.blockshift: 21
low_cost.obs.blockcache.large.number.blocks.perbank: 1000
low_cost.obs.blockcache.large.bank.count: 5
low_cost.obs.blockcache.large.file.types: ["fdx", "dvd", "tip"]</pre>
</div>
<div class="section" id="css_01_0405__section1517719253485"><h4 class="sectiontitle">Querying the Cache Status of Cold Data Stored in OBS</h4><p id="css_01_0405__p13268578194">When the data of a frozen index is queried for the first time, the data retrieved from OBS is automatically cached by the cluster. You can query the cache status of cold data stored in OBS. You can also reset the cache status when you need to debug cluster performance.</p>
<ol id="css_01_0405__ol1576183744318"><li id="css_01_0405__li7761637144318">Query the cache status of cold data stored in OBS.<ul id="css_01_0405__ul961212915401"><li id="css_01_0405__li12612202944017">Query statistics about cold data caching on all nodes:<pre class="screen" id="css_01_0405__screen13347465385">GET _frozen_stats</pre>
</li><li id="css_01_0405__li261219297402">Query statistics about cold data caching on specified nodes:<pre class="screen" id="css_01_0405__en-us_topic_0000001223594408_screen3226103316442">GET _frozen_stats/${node_id}</pre>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="css_01_0405__en-us_topic_0000001223594408_table188139184612" frame="border" border="1" rules="all"><caption><b>Table 15 </b>Request parameters</caption><thead align="left"><tr id="css_01_0405__en-us_topic_0000001223594408_row16823919464"><th align="left" class="cellrowborder" valign="top" width="30.94%" id="mcps1.3.13.3.1.1.2.2.2.3.1.1"><p id="css_01_0405__en-us_topic_0000001223594408_p1853944615">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="69.06%" id="mcps1.3.13.3.1.1.2.2.2.3.1.2"><p id="css_01_0405__en-us_topic_0000001223594408_p138113934612">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="css_01_0405__en-us_topic_0000001223594408_row148143918469"><td class="cellrowborder" valign="top" width="30.94%" headers="mcps1.3.13.3.1.1.2.2.2.3.1.1 "><p id="css_01_0405__en-us_topic_0000001223594408_p482039174614">node_id</p>
</td>
<td class="cellrowborder" valign="top" width="69.06%" headers="mcps1.3.13.3.1.1.2.2.2.3.1.2 "><p id="css_01_0405__en-us_topic_0000001223594408_p198539114610">Node ID</p>
</td>
</tr>
</tbody>
</table>
</div>
</li></ul>
<p id="css_01_0405__en-us_topic_0000001223594408_p107861228114412">Information similar to the following is returned:</p>
<pre class="screen" id="css_01_0405__en-us_topic_0000001223594408_screen1820145934412">{
"_nodes" : {
"total" : 3, //Total number of nodes
"successful" : 3, //Successful nodes
"failed" : 0 //Failed nodes
},
"cluster_name" : "css-zzz1", //Cluster name
"nodes" : {
"7uwKO38RRoaON37YsXhCYw" : {
"name" : "css-zzz1-ess-esn-2-1", //Node name
"transport_address" : "10.0.0.247:9300", //Node transport address
"host" : "10.0.0.247", //Node host
"ip" : "10.0.0.247", //Node IP address
"block_cache" : {
"default" : {
"type" : "memory", //Cache type. memory indicates in-memory cache.
"block_cache_capacity" : 8192, //Cache capacity
"block_cache_blocksize" : 8192, //Single-block size in the cache, in bytes. In the example, the block size is 8 KB.
"block_cache_size" : 12, //Cache capacity used.
"block_cache_hit" : 14, //Number of cache hits.
"block_cache_miss" : 0, //Number of cache misses.
"block_cache_eviction" : 0, //Number of cache evictions.
"block_cache_store_fail" : 0 //Number of cache storage failures, which occur when the cache is full.
}
},
"obs_stats" : {
"list" : {
"obs_list_count" : 17, //Number of times the OBS list API was called.
"obs_list_ms" : 265, //Total length of time spent calling the OBS list API.
"obs_list_avg_ms" : 15 //Average time spent calling the OBS list API.
},
"get_meta" : {
"obs_get_meta_count" : 79, //Number of times the OBS get metadata API was called.
"obs_get_meta_ms" : 183, //Total length of time spent calling the OBS get metadata API.
"obs_get_meta_avg_ms" : 2 //Average time spent calling the OBS get metadata API.
},
"get_obj" : {
"obs_get_obj_count" : 12, //Number of times the OBS get object API was called.
"obs_get_obj_ms" : 123, //Total length of time spent calling the OBS get object API.
"obs_get_obj_avg_ms" : 10 //Average time spent calling the OBS get object API.
},
"put_obj" : {
"obs_put_obj_count" : 12, //Number of times the OBS put object API was called.
"obs_put_obj_ms" : 2451, //Total length of time spent calling the OBS put object API.
"obs_put_obj_avg_ms" : 204 //Average time spent calling the OBS put object API.
},
"obs_op_total" : {
"obs_op_total_ms" : 3022, //Total length of time spent calling OBS APIs.
"obs_op_total_count" : 120, //Total number of times calling OBS APIs.
"obs_op_avg_ms" : 25 //Average time spent calling OBS APIs.
}
},
"reader_cache" : {
"hit_count" : 0,
"miss_count" : 1,
"load_success_count" : 1,
"load_exception_count" : 0,
"total_load_time" : 291194714,
"eviction_count" : 0
}
},
"73EDpEqoQES749umJqxOzQ" : {
"name" : "css-zzz1-ess-esn-3-1",
"transport_address" : "10.0.0.201:9300",
"host" : "10.0.0.201",
"ip" : "10.0.0.201",
"block_cache" : {
"default" : {
"type" : "memory",
"block_cache_capacity" : 8192,
"block_cache_blocksize" : 8192,
"block_cache_size" : 12,
"block_cache_hit" : 14,
"block_cache_miss" : 0,
"block_cache_eviction" : 0,
"block_cache_store_fail" : 0
}
},
"obs_stats" : {
"list" : {
"obs_list_count" : 17,
"obs_list_ms" : 309,
"obs_list_avg_ms" : 18
},
"get_meta" : {
"obs_get_meta_count" : 79,
"obs_get_meta_ms" : 216,
"obs_get_meta_avg_ms" : 2
},
"get_obj" : {
"obs_get_obj_count" : 12,
"obs_get_obj_ms" : 140,
"obs_get_obj_avg_ms" : 11
},
"put_obj" : {
"obs_put_obj_count" : 12,
"obs_put_obj_ms" : 1081,
"obs_put_obj_avg_ms" : 90
},
"obs_op_total" : {
"obs_op_total_ms" : 1746,
"obs_op_total_count" : 120,
"obs_op_avg_ms" : 14
}
},
"reader_cache" : {
"hit_count" : 0,
"miss_count" : 1,
"load_success_count" : 1,
"load_exception_count" : 0,
"total_load_time" : 367179751,
"eviction_count" : 0
}
},
"EF8WoLCUQbqJl1Pkqo9-OA" : {
"name" : "css-zzz1-ess-esn-1-1",
"transport_address" : "10.0.0.18:9300",
"host" : "10.0.0.18",
"ip" : "10.0.0.18",
"block_cache" : {
"default" : {
"type" : "memory",
"block_cache_capacity" : 8192,
"block_cache_blocksize" : 8192,
"block_cache_size" : 12,
"block_cache_hit" : 14,
"block_cache_miss" : 0,
"block_cache_eviction" : 0,
"block_cache_store_fail" : 0
}
},
"obs_stats" : {
"list" : {
"obs_list_count" : 17,
"obs_list_ms" : 220,
"obs_list_avg_ms" : 12
},
"get_meta" : {
"obs_get_meta_count" : 79,
"obs_get_meta_ms" : 139,
"obs_get_meta_avg_ms" : 1
},
"get_obj" : {
"obs_get_obj_count" : 12,
"obs_get_obj_ms" : 82,
"obs_get_obj_avg_ms" : 6
},
"put_obj" : {
"obs_put_obj_count" : 12,
"obs_put_obj_ms" : 879,
"obs_put_obj_avg_ms" : 73
},
"obs_op_total" : {
"obs_op_total_ms" : 1320,
"obs_op_total_count" : 120,
"obs_op_avg_ms" : 11
}
},
"reader_cache" : {
"hit_count" : 0,
"miss_count" : 1,
"load_success_count" : 1,
"load_exception_count" : 0,
"total_load_time" : 235706838,
"eviction_count" : 0
}
}
}
}
</pre>
</li><li id="css_01_0405__en-us_topic_0000001223594408_li680614341286">Run the following command to reset the cache status:<pre class="screen" id="css_01_0405__en-us_topic_0000001223594408_screen17747182004910">POST _frozen_stats/reset</pre>
<div class="note" id="css_01_0405__en-us_topic_0000001223594408_note86441935185112"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="css_01_0405__en-us_topic_0000001223594408_p1964463519518">This command is used to debug performance issues. If you reset the cache status and then run the cache query command, you can check the accurate cache command status. It is not advisable to use this command during service running.</p>
</div></div>
<p id="css_01_0405__en-us_topic_0000001223594408_p134231015144912">Information similar to the following is returned:</p>
<pre class="screen" id="css_01_0405__en-us_topic_0000001223594408_screen39961919135116">{
"_nodes" : {
"total" : 1,
"successful" : 1,
"failed" : 0
},
"cluster_name" : "Es-0325-007_01",
"nodes" : {
"mqTdk2YRSPyOSXfesREFSg" : {
"result" : "ok"
}
}
}</pre>
</li></ol>
</div>
<div class="section" id="css_01_0405__section196311139511"><h4 class="sectiontitle">Improving Cold Data Query Performance</h4><div class="notice" id="css_01_0405__note817422810105"><span class="noticetitle"><img src="public_sys-resources/notice_3.0-en-us.png"> </span><div class="noticebody"><p id="css_01_0405__p2060852811235">Only Elasticsearch 7.6.2 and 7.10.2 clusters created after February 2023 support this feature.</p>
</div></div>
<p id="css_01_0405__en-us_topic_0000001472612169_p035931615312">When cold data is queried on the <strong id="css_01_0405__b14769145732611">Discover</strong> page of Kibana for the first time, all data needs to be retrieved from OBS because there is no cache. If a large number of documents need to be returned, it takes a long time to retrieve the corresponding time fields and file metadata from OBS. By caching this part of data within the cluster, you can significantly improve query performance. This is how CSS improves the query performance for cold data. Local cache settings are preset. You can modify them as needed. You can also view the local cache settings.</p>
<ol id="css_01_0405__ol11855102814615"><li id="css_01_0405__li17994124613490">Modify local cache settings for cold data.
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="css_01_0405__en-us_topic_0000001472612169_table1549126112114" frame="border" border="1" rules="all"><caption><b>Table 16 </b>Local cache configuration items</caption><thead align="left"><tr id="css_01_0405__en-us_topic_0000001472612169_row1687192652118"><th align="left" class="cellrowborder" valign="top" width="18%" id="mcps1.3.14.4.1.1.2.6.1.1"><p id="css_01_0405__en-us_topic_0000001472612169_p7871326202118">Configuration Item</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="9%" id="mcps1.3.14.4.1.1.2.6.1.2"><p id="css_01_0405__en-us_topic_0000001472612169_p587152611216">Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="9%" id="mcps1.3.14.4.1.1.2.6.1.3"><p id="css_01_0405__en-us_topic_0000001472612169_p14871426142111">scope</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="14.000000000000002%" id="mcps1.3.14.4.1.1.2.6.1.4"><p id="css_01_0405__en-us_topic_0000001472612169_p118862652117">Can Be Changed Dynamically</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.14.4.1.1.2.6.1.5"><p id="css_01_0405__en-us_topic_0000001472612169_p888626112116">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="css_01_0405__en-us_topic_0000001472612169_row8881326122118"><td class="cellrowborder" valign="top" width="18%" headers="mcps1.3.14.4.1.1.2.6.1.1 "><p id="css_01_0405__en-us_topic_0000001472612169_p7887267213">low_cost.local_cache.max.capacity</p>
</td>
<td class="cellrowborder" valign="top" width="9%" headers="mcps1.3.14.4.1.1.2.6.1.2 "><p id="css_01_0405__en-us_topic_0000001472612169_p5888269213">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="9%" headers="mcps1.3.14.4.1.1.2.6.1.3 "><p id="css_01_0405__en-us_topic_0000001472612169_p128820261219">node</p>
</td>
<td class="cellrowborder" valign="top" width="14.000000000000002%" headers="mcps1.3.14.4.1.1.2.6.1.4 "><p id="css_01_0405__en-us_topic_0000001472612169_p888226192111">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.14.4.1.1.2.6.1.5 "><p id="css_01_0405__en-us_topic_0000001472612169_p58882672110">Maximum number of available cold data caches on a node. Each shard corresponds to a cache object.</p>
<p id="css_01_0405__p151154420566">Value range: 10-5000</p>
<p id="css_01_0405__p781318472558">Default value: <strong id="css_01_0405__b129357003644330">500</strong></p>
<div class="note" id="css_01_0405__en-us_topic_0000001472612169_note125431510514"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="css_01_0405__en-us_topic_0000001472612169_ul970310302573"><li id="css_01_0405__en-us_topic_0000001472612169_li2243153417578">If the heap memory usage remains high, you can decrease this value.</li><li id="css_01_0405__en-us_topic_0000001472612169_li3704163011570">If the value of <strong id="css_01_0405__b89798622612">load_overflow_count</strong> keeps increasing rapidly, increase this value.</li></ul>
</div></div>
</td>
</tr>
<tr id="css_01_0405__en-us_topic_0000001472612169_row10881626122117"><td class="cellrowborder" valign="top" width="18%" headers="mcps1.3.14.4.1.1.2.6.1.1 "><p id="css_01_0405__en-us_topic_0000001472612169_p158862642110">index.low_cost.local_cache.threshold</p>
</td>
<td class="cellrowborder" valign="top" width="9%" headers="mcps1.3.14.4.1.1.2.6.1.2 "><p id="css_01_0405__en-us_topic_0000001472612169_p17881226102115">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="9%" headers="mcps1.3.14.4.1.1.2.6.1.3 "><p id="css_01_0405__en-us_topic_0000001472612169_p38819266211">index</p>
</td>
<td class="cellrowborder" valign="top" width="14.000000000000002%" headers="mcps1.3.14.4.1.1.2.6.1.4 "><p id="css_01_0405__en-us_topic_0000001472612169_p188172618215">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.14.4.1.1.2.6.1.5 "><p id="css_01_0405__en-us_topic_0000001472612169_p38852682118">Threshold for enabling the local cache of cold data.</p>
<ul id="css_01_0405__en-us_topic_0000001472612169_ul957011127573"><li id="css_01_0405__en-us_topic_0000001472612169_li417516170578">If in an index, the percentage of fields whose type is <strong id="css_01_0405__b16548641986">date</strong> is less than the value of this parameter, you can enable local cache for cold data of the <strong id="css_01_0405__b1846513293815">date</strong> type. Otherwise, do not use it.</li><li id="css_01_0405__en-us_topic_0000001472612169_li16570412175715">If date fields account for the vast majority of all fields in the current index, you are advised not to use this setting.</li></ul>
<p id="css_01_0405__p11936155485716">Unit: %</p>
<p id="css_01_0405__p5451316195813">Value range: <strong id="css_01_0405__b164357283344330">0</strong> to <strong id="css_01_0405__b88847680544330">100</strong></p>
<p id="css_01_0405__p323973115818">Default value: <strong id="css_01_0405__b729892110911">50</strong></p>
</td>
</tr>
<tr id="css_01_0405__en-us_topic_0000001472612169_row168812617215"><td class="cellrowborder" valign="top" width="18%" headers="mcps1.3.14.4.1.1.2.6.1.1 "><p id="css_01_0405__en-us_topic_0000001472612169_p16891826162111">index.low_cost.local_cache.evict_time</p>
</td>
<td class="cellrowborder" valign="top" width="9%" headers="mcps1.3.14.4.1.1.2.6.1.2 "><p id="css_01_0405__en-us_topic_0000001472612169_p5891265214">String</p>
</td>
<td class="cellrowborder" valign="top" width="9%" headers="mcps1.3.14.4.1.1.2.6.1.3 "><p id="css_01_0405__en-us_topic_0000001472612169_p10898263214">index</p>
</td>
<td class="cellrowborder" valign="top" width="14.000000000000002%" headers="mcps1.3.14.4.1.1.2.6.1.4 "><p id="css_01_0405__en-us_topic_0000001472612169_p148952692110">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.14.4.1.1.2.6.1.5 "><p id="css_01_0405__en-us_topic_0000001472612169_p58992611215">Retention duration for cold data in the local cache. The value is determined based on index.frozen_date (time when the freezing is successful). If index.frozen_date is unavailable, the value is determined based on the index creation time.</p>
<p id="css_01_0405__p1166210301938">Unit: days</p>
<p id="css_01_0405__p428683619318">Value range: 1 to 365 days</p>
<p id="css_01_0405__p18364153910313">Default value: <strong id="css_01_0405__b100210012144330">30d</strong></p>
<div class="note" id="css_01_0405__en-us_topic_0000001472612169_note166021510115213"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="css_01_0405__p176452917317">You are advised to adjust the retention duration based on your disk usage.</p>
</div></div>
</td>
</tr>
</tbody>
</table>
</div>
<ul id="css_01_0405__en-us_topic_0000001472612169_ul1321111971620"><li id="css_01_0405__en-us_topic_0000001472612169_li120101911614">Run the following command to modify <span class="parmname" id="css_01_0405__parmname199002222620"><b>low_cost.local_cache.max.capacity</b></span>:<pre class="screen" id="css_01_0405__en-us_topic_0000001472612169_screen19201219131612">PUT _cluster/settings
{
"persistent": {
"low_cost.local_cache.max.capacity":<em id="css_01_0405__i635615178615">1000</em>
}
}</pre>
</li><li id="css_01_0405__en-us_topic_0000001472612169_li192181915164">Run the following command to modify <span class="parmname" id="css_01_0405__parmname29741347761"><b>index.low_cost.local_cache.threshold</b></span>:<pre class="screen" id="css_01_0405__en-us_topic_0000001472612169_screen182181911615">PUT es_write_pref2-00000000021/_settings
{
"index.low_cost.local_cache.threshold":20
}</pre>
</li><li id="css_01_0405__en-us_topic_0000001472612169_li202110195164">Run the following command to modify <span class="parmname" id="css_01_0405__parmname73105119618"><b>index.low_cost.local_cache.evict_time</b></span>:<pre class="screen" id="css_01_0405__en-us_topic_0000001472612169_screen1321131931615">PUT es_write_pref2-00000000021/_settings
{
"index.low_cost.local_cache.evict_time":"7d"
}</pre>
</li></ul>
</li><li id="css_01_0405__li1285513288464">Query the local cache information for cold data.<ul id="css_01_0405__ul13324271785"><li id="css_01_0405__li153242071484">Query statistics and metrics about cold data caching on all nodes:<pre class="screen" id="css_01_0405__en-us_topic_0000001472612169_screen153096221116">GET /_frozen_stats/local_cache</pre>
</li><li id="css_01_0405__li197593161389">Query statistics and metrics about cold data caching on specified nodes:<pre class="screen" id="css_01_0405__screen2941936888">GET /_frozen_stats/local_cache/{nodeId}</pre>
<p id="css_01_0405__p173925692915"><span class="parmname" id="css_01_0405__parmname5739135611296"><b>{nodeId}</b></span> indicates the node ID.</p>
</li></ul>
<p id="css_01_0405__p1871518248816">Information similar to the following is returned:</p>
<pre class="screen" id="css_01_0405__en-us_topic_0000001472612169_screen85929147148">{
"_nodes" : {
"total" : 1,
"successful" : 1,
"failed" : 0
},
"cluster_name" : "elasticsearch",
"nodes" : {
"6by3lPy1R3m55Dcq3liK8Q" : {
"name" : "node-1",
"transport_address" : "127.0.0.1:9300",
"host" : "127.0.0.1",
"ip" : "127.0.0.1",
"local_cache" : {
"get_stats" : {
"get_total_count" : 562, //Total number of times data was retrieved from the local cold data cache.
"get_hit_count" : 562, //Total number of hits in the local cold data cache.
"get_miss_count" : 0, //Total number of local cold data cache misses.
"get_total_ns" : 43849200, //Total duration for retrieving data from the local cold data cache.
"get_avg_ns" : 78023 //Average duration for retrieving data from the local cold data cache.
},
"load_stats" : {
"load_count" : 2, //Number of times cold data was loaded from the local cache
"load_total_ms" : 29, //Total duration for loading cold data from the local cache
"load_avg_ms" : 14, //Average duration for loading cold data from the local cache
"load_fail_count" : 0, //Number of failures for loading cold data from the local cache
"load_overflow_count" : 0 //Number of times the local cold data cache exceeds the cache pool size.
},
"reload_stats" : {
"reload_count" : 0, //Number of times the local cold data cache was regenerated.
"reload_total_ms" : 0, //Total duration for regenerating the local cold data cache.
"reload_avg_ms" : 0, //Average duration for regenerating the local cold data cache.
"reload_fail_count" : 0 //Number of failures in regenerating the local cold data cache.
},
"init_stats" : {
"init_count" : 0, //Number of times the local cold data cache was initialized.
"init_total_ms" : 0, //Total duration for initializing the local cold data cache.
"init_avg_ms" : 0, //Average duration for initializing the local cold data cache.
"init_fail_count" : 0 //Number of failures in initializing the local cold data cache.
}
}
}
}
}</pre>
</li></ol>
</div>
<div class="section" id="css_01_0405__section7700162112510"><h4 class="sectiontitle">Querying the Real-Time Rates of OBS in Handling Cold Data</h4><div class="notice" id="css_01_0405__note7561195752617"><span class="noticetitle"><img src="public_sys-resources/notice_3.0-en-us.png"> </span><div class="noticebody"><p id="css_01_0405__p375616117358">Only Elasticsearch 7.6.2 and 7.10.2 clusters created after February 2023 support this feature.</p>
</div></div>
<p id="css_01_0405__en-us_topic_0000001520039673_p11877144265613">To help you understand how the storage-compute decoupling plug-in is working with OBS, an API for collecting statistics on the real-time rates of OBS has been added, and the real-time rates are recorded in the index <span class="parmname" id="css_01_0405__parmname10138184773315"><b>.freeze_obs_rate-YYYY.mm.dd</b></span>.</p>
<p id="css_01_0405__p1714431016348">Calculation method: The average OBS operation rates in the last 5 seconds are calculated every 5 seconds.</p>
<p id="css_01_0405__p129965205355">The system index <span class="parmname" id="css_01_0405__parmname14802191053213"><b>.freeze_obs_rate-YYYY.mm.dd</b></span> records statistics on OBS real-time operation rates, helping you understand relevant trends about the OBS that stores cold data. The default retention period of the index is 30 days.</p>
<ol id="css_01_0405__ol175571749182213"><li id="css_01_0405__li1155784919228">Querying the real-time rates of OBS in handling cold data.<ul id="css_01_0405__en-us_topic_0000001520039673_ul111291635191113"><li id="css_01_0405__en-us_topic_0000001520039673_li8129123511112">Run the following command to query the real-time OBS rates on all nodes:<pre class="screen" id="css_01_0405__en-us_topic_0000001520039673_screen390353121220">GET _frozen_stats/obs_rate </pre>
</li><li id="css_01_0405__li16181442911">Run the following command to query the real-time OBS rates on specified nodes:<pre class="screen" id="css_01_0405__screen171111413295">GET _frozen_stats/obs_rate/{nodeId}</pre>
<p id="css_01_0405__p11161472917"><span class="parmname" id="css_01_0405__parmname795013448279"><b>{nodeId}</b></span> indicates the node ID.</p>
</li></ul>
<div class="p" id="css_01_0405__p731112123017">Example response:<pre class="screen" id="css_01_0405__en-us_topic_0000001520039673_screen9421915171813">{
"_nodes" : {
"total" : 1,
"successful" : 1,
"failed" : 0
},
"cluster_name" : "elasticsearch",
"nodes" : {
"dflDvcSwTJ-fkiIlT2zE3A" : {
"name" : "node-1",
"transport_address" : "127.0.0.1:9300",
"host" : "127.0.0.1",
"ip" : "127.0.0.1",
"update_time" : 1671777600482, // Time when the current statistics were updated.
"obs_rate" : {
"list_op_rate" : 0.0, // Rate of OBS <strong id="css_01_0405__b18299052444330">list</strong> operations. Unit: times/s.
"get_meta_op_rate" : 0.0, // Rate of OBS <strong id="css_01_0405__b113445148344330">get meta</strong> operations. Unit: times/s.
"get_obj_op_rate" : 0.0, // Rate of OBS <strong id="css_01_0405__b158655204144330">get</strong> operations. Unit: times/s.
"put_op_rate" : 0.0, // Rate of OBS <strong id="css_01_0405__b186374911744330">put</strong> operations. Unit: times/s.
"obs_total_op_rate" : 0.0, // Rate of all OBS operations. Unit: times/s.
"obs_upload_rate" : "0.0 MB/s", // Data upload rate of OBS, in MB/s.
"obs_download_rate" : "0.0 MB/s" // Data download rate of OBS, in MB/s.
}
}
}
}</pre>
</div>
</li><li id="css_01_0405__li0958410202318">Modify the retention period of the <span class="parmname" id="css_01_0405__en-us_topic_0000001520039673_parmname187591255205916"><b>.freeze_obs_rate-YYYY.mm.dd</b></span> index that stores the OBS real-time rates. The default retention period of indexes is 30 days.<p id="css_01_0405__p3222121910240">Run the following command to change the index retention period to seven days:</p>
<pre class="screen" id="css_01_0405__en-us_topic_0000001520039673_screen1890695511421">PUT _cluster/settings
{
"persistent": {
"low_cost.obs_rate_index.evict_time": "7d"
}
}</pre>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="css_01_0405__en-us_topic_0000001520039673_table89366542411" frame="border" border="1" rules="all"><caption><b>Table 17 </b>Configuration items</caption><thead align="left"><tr id="css_01_0405__en-us_topic_0000001520039673_row1497845416413"><th align="left" class="cellrowborder" valign="top" width="18%" id="mcps1.3.15.6.2.4.2.6.1.1"><p id="css_01_0405__en-us_topic_0000001520039673_p13978135434117">Configuration Item</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="9%" id="mcps1.3.15.6.2.4.2.6.1.2"><p id="css_01_0405__en-us_topic_0000001520039673_p1697813541411">Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="9%" id="mcps1.3.15.6.2.4.2.6.1.3"><p id="css_01_0405__en-us_topic_0000001520039673_p89781354144113">scope</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="14.000000000000002%" id="mcps1.3.15.6.2.4.2.6.1.4"><p id="css_01_0405__en-us_topic_0000001520039673_p79787548411">Can Be Changed Dynamically</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.15.6.2.4.2.6.1.5"><p id="css_01_0405__en-us_topic_0000001520039673_p597816549418">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="css_01_0405__en-us_topic_0000001520039673_row13978354154113"><td class="cellrowborder" valign="top" width="18%" headers="mcps1.3.15.6.2.4.2.6.1.1 "><p id="css_01_0405__en-us_topic_0000001520039673_p1297985411411">low_cost.obs_rate_index.evict_time</p>
</td>
<td class="cellrowborder" valign="top" width="9%" headers="mcps1.3.15.6.2.4.2.6.1.2 "><p id="css_01_0405__en-us_topic_0000001520039673_p18979754144117">String</p>
</td>
<td class="cellrowborder" valign="top" width="9%" headers="mcps1.3.15.6.2.4.2.6.1.3 "><p id="css_01_0405__en-us_topic_0000001520039673_p8979135464120">node</p>
</td>
<td class="cellrowborder" valign="top" width="14.000000000000002%" headers="mcps1.3.15.6.2.4.2.6.1.4 "><p id="css_01_0405__en-us_topic_0000001520039673_p10979135454115">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.15.6.2.4.2.6.1.5 "><p id="css_01_0405__en-us_topic_0000001520039673_p69792544412">The retention period of the <span class="parmname" id="css_01_0405__en-us_topic_0000001520039673_parmname437713506496"><b>.freeze_obs_rate-YYYY.mm.dd</b></span> index.</p>
<ul id="css_01_0405__en-us_topic_0000001520039673_ul8141351174519"><li id="css_01_0405__en-us_topic_0000001520039673_li3141205104512">Value range: 1 to 365 days</li><li id="css_01_0405__en-us_topic_0000001520039673_li1974671514618">Default value: <strong id="css_01_0405__b207089800244330">30d</strong></li><li id="css_01_0405__en-us_topic_0000001520039673_li961615238468">Unit: days</li></ul>
</td>
</tr>
</tbody>
</table>
</div>
</li></ol>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="css_01_0111.html">Enhancing Search Capabilities for Elasticsearch Clusters</a></div>
</div>
</div>
View Git Blame Copy Permalink