yexinru/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
Files
de9f8a7cc5b4e61816cbd4e52cc108c5f9fc3b4c
doc-exports/docs/css/umn/css_01_0379.html
zhengxiu 2125539080 css umn 25.1.0 version 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: zhengxiu <zhengxiu@huawei.com>
Co-committed-by: zhengxiu <zhengxiu@huawei.com>
2025-07-04 09:10:17 +00:00

553 lines
65 KiB
HTML
Raw Blame History

<a name="css_01_0379"></a><a name="css_01_0379"></a>
<h1 class="topictitle1">Elasticsearch Cluster Planning Suggestions</h1>
<div id="body0000001973113041"><p id="css_01_0379__p22234241879">Before creating an Elasticsearch cluster, develop a plan for it, such as whether to deploy the cluster across multiple AZs to improve availability; the node quantity and specifications; the cluster version and security mode; and index sharding, in order to ensure the desired performance and reliability.</p>
<div class="section" id="css_01_0379__section1074611275359"><a name="css_01_0379__section1074611275359"></a><a name="section1074611275359"></a><h4 class="sectiontitle">Planning Cluster AZs</h4><p id="css_01_0379__p579444517285">By deploying a CSS cluster across multiple AZs, you can increase the cluster's availability, lower the likelihood of data loss, and minimize service downtime. You can select two or three different AZs in the same region to deploy a cluster.</p>
<div class="p" id="css_01_0379__p15958205253710">If you select two or three AZs when creating a cluster, CSS automatically enables cross-AZ HA, ensuring cluster nodes distribute evenly across these AZs. For details about the distribution of nodes in different quantities across AZs, see <a href="#css_01_0379__en-us_topic_0000001478394613_table13143175454216">Table 1</a>.<div class="notice" id="css_01_0379__note15310525323"><span class="noticetitle"><img src="public_sys-resources/notice_3.0-en-us.png"> </span><div class="noticebody"><ul id="css_01_0379__en-us_topic_0000001478394613_ul85869454137"><li id="css_01_0379__en-us_topic_0000001478394613_li175461852171311">When creating a multi-AZ cluster, ensure that the number of selected nodes of any type is no less than the number of AZs. Otherwise, multi-AZ cluster deployment will fail.</li><li id="css_01_0379__li128001225131012">When a multi-AZ cluster is deployed, nodes of all types are evenly distributed across different AZs. The difference between node quantities in different AZs does not exceed 1.</li><li id="css_01_0379__li192741747131116">If the number of data nodes plus cold data nodes in a cluster is not an integer multiple of the number of AZs, data in the cluster may be unevenly distributed, affecting data query or write performance.</li></ul>
</div></div>
<div class="tablenoborder"><a name="css_01_0379__en-us_topic_0000001478394613_table13143175454216"></a><a name="en-us_topic_0000001478394613_table13143175454216"></a><table cellpadding="4" cellspacing="0" summary="" id="css_01_0379__en-us_topic_0000001478394613_table13143175454216" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Node quantities and AZ distribution</caption><tbody><tr id="css_01_0379__en-us_topic_0000001478394613_row7221175417423"><th class="firstcol" rowspan="2" valign="top" id="mcps1.3.2.3.3.2.8.1.1"><p id="css_01_0379__en-us_topic_0000001478394613_p162211543426">Node Quantity</p>
</th>
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.3.2.8.1.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p42211754184217">Single AZ</p>
</td>
<td class="cellrowborder" colspan="2" valign="top" headers="mcps1.3.2.3.3.2.8.1.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p15221954114214">Two AZs</p>
</td>
<td class="cellrowborder" colspan="3" valign="top" headers="mcps1.3.2.3.3.2.8.1.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p1422115412421">Three AZs</p>
</td>
</tr>
<tr id="css_01_0379__en-us_topic_0000001478394613_row19221195416424"><td class="cellrowborder" valign="top" headers="mcps1.3.2.3.3.2.8.1.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p6222454144214">AZ1</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.3.2.8.1.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p52228544422">AZ1</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.3.2.8.1.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p10222105434217">AZ2</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.3.2.8.1.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p192221854164219">AZ1</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.3.2.8.1.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p172221654134215">AZ2</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.3.2.8.1.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p1222265410428">AZ3</p>
</td>
</tr>
<tr id="css_01_0379__en-us_topic_0000001478394613_row1722235414214"><th class="firstcol" valign="top" id="mcps1.3.2.3.3.2.8.3.1"><p id="css_01_0379__en-us_topic_0000001478394613_p5222165484215">1</p>
</th>
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.3.2.8.3.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p1222255419428">1</p>
</td>
<td class="cellrowborder" colspan="2" valign="top" headers="mcps1.3.2.3.3.2.8.3.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p122235416424">Not supported</p>
</td>
<td class="cellrowborder" colspan="3" valign="top" headers="mcps1.3.2.3.3.2.8.3.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p1722212549423">Not supported</p>
</td>
</tr>
<tr id="css_01_0379__en-us_topic_0000001478394613_row10222554154219"><th class="firstcol" valign="top" id="mcps1.3.2.3.3.2.8.4.1"><p id="css_01_0379__en-us_topic_0000001478394613_p14222954144212">2</p>
</th>
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.3.2.8.4.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p1222754154211">2</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.3.2.8.4.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p122224543426">1</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.3.2.8.4.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p4222165404213">1</p>
</td>
<td class="cellrowborder" colspan="3" valign="top" headers="mcps1.3.2.3.3.2.8.4.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p1622215420426">Not supported</p>
</td>
</tr>
<tr id="css_01_0379__en-us_topic_0000001478394613_row182224548423"><th class="firstcol" valign="top" width="14.285714285714285%" id="mcps1.3.2.3.3.2.8.5.1"><p id="css_01_0379__en-us_topic_0000001478394613_p10222135484211">3</p>
</th>
<td class="cellrowborder" valign="top" width="14.285714285714285%" headers="mcps1.3.2.3.3.2.8.5.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p52221754134213">3</p>
</td>
<td class="cellrowborder" valign="top" width="14.285714285714285%" headers="mcps1.3.2.3.3.2.8.5.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p16222185474216">2</p>
</td>
<td class="cellrowborder" valign="top" width="14.285714285714285%" headers="mcps1.3.2.3.3.2.8.5.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p102221454104211">1</p>
</td>
<td class="cellrowborder" valign="top" width="14.285714285714285%" headers="mcps1.3.2.3.3.2.8.5.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p222216542422">1</p>
</td>
<td class="cellrowborder" valign="top" width="14.285714285714285%" headers="mcps1.3.2.3.3.2.8.5.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p11222954184219">1</p>
</td>
<td class="cellrowborder" valign="top" width="14.285714285714285%" headers="mcps1.3.2.3.3.2.8.5.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p1223115419429">1</p>
</td>
</tr>
<tr id="css_01_0379__en-us_topic_0000001478394613_row322319542429"><th class="firstcol" valign="top" width="14.285714285714285%" id="mcps1.3.2.3.3.2.8.6.1"><p id="css_01_0379__en-us_topic_0000001478394613_p622385414427">4</p>
</th>
<td class="cellrowborder" valign="top" width="14.285714285714285%" headers="mcps1.3.2.3.3.2.8.6.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p1322315424218">4</p>
</td>
<td class="cellrowborder" valign="top" width="14.285714285714285%" headers="mcps1.3.2.3.3.2.8.6.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p1223195418425">2</p>
</td>
<td class="cellrowborder" valign="top" width="14.285714285714285%" headers="mcps1.3.2.3.3.2.8.6.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p122325417428">2</p>
</td>
<td class="cellrowborder" valign="top" width="14.285714285714285%" headers="mcps1.3.2.3.3.2.8.6.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p1422345464210">2</p>
</td>
<td class="cellrowborder" valign="top" width="14.285714285714285%" headers="mcps1.3.2.3.3.2.8.6.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p1422345444212">1</p>
</td>
<td class="cellrowborder" valign="top" width="14.285714285714285%" headers="mcps1.3.2.3.3.2.8.6.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p322335418425">1</p>
</td>
</tr>
<tr id="css_01_0379__en-us_topic_0000001478394613_row52231854194219"><th class="firstcol" valign="top" width="14.285714285714285%" id="mcps1.3.2.3.3.2.8.7.1"><p id="css_01_0379__en-us_topic_0000001478394613_p1722315410427">...</p>
</th>
<td class="cellrowborder" valign="top" width="14.285714285714285%" headers="mcps1.3.2.3.3.2.8.7.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p182231854164214">...</p>
</td>
<td class="cellrowborder" valign="top" width="14.285714285714285%" headers="mcps1.3.2.3.3.2.8.7.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p19223254204214">...</p>
</td>
<td class="cellrowborder" valign="top" width="14.285714285714285%" headers="mcps1.3.2.3.3.2.8.7.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p2022385434216">...</p>
</td>
<td class="cellrowborder" valign="top" width="14.285714285714285%" headers="mcps1.3.2.3.3.2.8.7.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p52231554194219">...</p>
</td>
<td class="cellrowborder" valign="top" width="14.285714285714285%" headers="mcps1.3.2.3.3.2.8.7.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p1422345413420">...</p>
</td>
<td class="cellrowborder" valign="top" width="14.285714285714285%" headers="mcps1.3.2.3.3.2.8.7.1 "><p id="css_01_0379__en-us_topic_0000001478394613_p172231454104217">...</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="p" id="css_01_0379__p14548205517545">In the case of a multi-AZ deployment, configure the number of replicas in a manner that can better capitalize on the high availability that comes with such as deployment.<ul id="css_01_0379__en-us_topic_0000001478394613_ul1246944134611"><li id="css_01_0379__en-us_topic_0000001478394613_li5246644134611">In a two-AZ deployment, if one AZ becomes unavailable, the other AZ continues to provide services. In this case, at least one replica is required. Elasticsearch uses one replica by default. You can retain the default value if you do not require higher read performance.</li><li id="css_01_0379__en-us_topic_0000001478394613_li55104547186">In the case of a three-AZ deployment, if one AZ becomes unavailable, the other AZs can continue to provide services. In this case, at least one replica is required. Elasticsearch uses one replica by default. If you need more replicas to improve the cluster's ability to handle queries, modify the replica setting to set more replicas.<p id="css_01_0379__en-us_topic_0000001478394613_p1929958164617"><a name="css_01_0379__en-us_topic_0000001478394613_li55104547186"></a><a name="en-us_topic_0000001478394613_li55104547186"></a>For example, you can run the following command to set the number of index replicas:</p>
<p id="css_01_0379__en-us_topic_0000001478394613_p953913552468"><b><span class="cmdname" id="css_01_0379__en-us_topic_0000001478394613_cmdname1553865511464">curl -XPUT http://ip:9200/{index_name}/_settings -d '{"number_of_replicas":2}'</span></b></p>
<p id="css_01_0379__en-us_topic_0000001478394613_p25401755154620">Alternatively, run the following command to specify the number of replicas in the index template:</p>
<p id="css_01_0379__en-us_topic_0000001478394613_p1654015554616"><b><span class="cmdname" id="css_01_0379__en-us_topic_0000001478394613_cmdname154019558467">curl -XPUT http://ip:9200/ _template/templatename -d '{ "template": "*", "settings": {"number_of_replicas": 2}}'</span></b></p>
<p id="css_01_0379__p3772140165512">where, <strong id="css_01_0379__b12561629103713">ip</strong> indicates the private IP address of the cluster, <strong id="css_01_0379__b141852047183714">index_name</strong> indicates the index name, and <strong id="css_01_0379__b280935619376">number_of_replicas</strong> indicates the number of index replicas to change to. In this example, the number of index replicas is changed to 2.</p>
</li></ul>
</div>
<div class="p" id="css_01_0379__p161421538709"><a href="#css_01_0379__table148971114819">Table 2</a> describes the service outage patterns for different variations of a multi-AZ deployment facing the failure of a single AZ.
<div class="tablenoborder"><a name="css_01_0379__table148971114819"></a><a name="table148971114819"></a><table cellpadding="4" cellspacing="0" summary="" id="css_01_0379__table148971114819" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Possible service outage patterns in the face of the failure of a single AZ</caption><thead align="left"><tr id="css_01_0379__row989751419113"><th align="left" class="cellrowborder" valign="top" width="14.62%" id="mcps1.3.2.5.2.2.4.1.1"><p id="css_01_0379__p889718141913">Number of AZs</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="15.43%" id="mcps1.3.2.5.2.2.4.1.2"><p id="css_01_0379__p1489701411118">Number of Master Nodes</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="69.95%" id="mcps1.3.2.5.2.2.4.1.3"><p id="css_01_0379__p1989791412114">Service Outage Patterns and Handling Suggestions</p>
</th>
</tr>
</thead>
<tbody><tr id="css_01_0379__row168977141218"><td class="cellrowborder" valign="top" width="14.62%" headers="mcps1.3.2.5.2.2.4.1.1 "><p id="css_01_0379__p1189710141310">2</p>
</td>
<td class="cellrowborder" valign="top" width="15.43%" headers="mcps1.3.2.5.2.2.4.1.2 "><p id="css_01_0379__p1897514618">0</p>
</td>
<td class="cellrowborder" valign="top" width="69.95%" headers="mcps1.3.2.5.2.2.4.1.3 "><ul id="css_01_0379__ul189811141015"><li id="css_01_0379__li489819142120">If the number of nodes is an even number:<p id="css_01_0379__p98706301833"><a name="css_01_0379__li489819142120"></a><a name="li489819142120"></a>If half of the data nodes are faulty, you need to replace one node in the faulty AZ before a master node can be selected.</p>
</li><li id="css_01_0379__li389819141618">If the number of nodes is an odd number:<ul id="css_01_0379__ul48989149110"><li id="css_01_0379__li19898171419116">If the faulty AZ contains one more node than the normal AZ, you need to replace one node in the faulty AZ before a master node can be selected. For how to replace nodes, contact technical support.</li><li id="css_01_0379__li1389811420112">If the faulty AZ contains one less node than the normal AZ, services will not be interrupted and a master node can be selected.</li></ul>
</li></ul>
</td>
</tr>
<tr id="css_01_0379__row0898114912"><td class="cellrowborder" valign="top" width="14.62%" headers="mcps1.3.2.5.2.2.4.1.1 "><p id="css_01_0379__p6898814612">2</p>
</td>
<td class="cellrowborder" valign="top" width="15.43%" headers="mcps1.3.2.5.2.2.4.1.2 "><p id="css_01_0379__p10898131417116">3</p>
</td>
<td class="cellrowborder" valign="top" width="69.95%" headers="mcps1.3.2.5.2.2.4.1.3 "><p id="css_01_0379__p88981014813">There is a 50% chance of service interruption. When two dedicated master nodes are allocated to one AZ and another master node is allocated to the other AZ:</p>
<ul id="css_01_0379__ul189814140110"><li id="css_01_0379__li7898191413119">If service interruption happens in the AZ with one master node, a master node can be selected from the AZ that has two dedicated master nodes.</li><li id="css_01_0379__li19898161410112">If service interruption happens in the AZ with two dedicated master nodes, master nodes cannot be selected because the remaining AZ has only one dedicated master node. In this case, services will be interrupted and you need to contact technical support.</li></ul>
</td>
</tr>
<tr id="css_01_0379__row1889812144114"><td class="cellrowborder" valign="top" width="14.62%" headers="mcps1.3.2.5.2.2.4.1.1 "><p id="css_01_0379__p689819141516">3</p>
</td>
<td class="cellrowborder" valign="top" width="15.43%" headers="mcps1.3.2.5.2.2.4.1.2 "><p id="css_01_0379__p1389821417117">0</p>
</td>
<td class="cellrowborder" valign="top" width="69.95%" headers="mcps1.3.2.5.2.2.4.1.3 "><p id="css_01_0379__p10898141412119">If you have three AZs and four nodes, one AZ will have two nodes, and the other two will each have one node. If the AZ with two nodes becomes faulty, services will be interrupted. Therefore, you are advised not to configure four nodes when selecting three AZs.</p>
<p id="css_01_0379__p148981141318">There is a small chance of service interruption if you follow this advice.</p>
</td>
</tr>
<tr id="css_01_0379__row5898814310"><td class="cellrowborder" valign="top" width="14.62%" headers="mcps1.3.2.5.2.2.4.1.1 "><p id="css_01_0379__p1189810147110">3</p>
</td>
<td class="cellrowborder" valign="top" width="15.43%" headers="mcps1.3.2.5.2.2.4.1.2 "><p id="css_01_0379__p1589815141416">3</p>
</td>
<td class="cellrowborder" valign="top" width="69.95%" headers="mcps1.3.2.5.2.2.4.1.3 "><p id="css_01_0379__p089820141618">Service interruption does not occur.</p>
</td>
</tr>
</tbody>
</table>
</div>
<div class="note" id="css_01_0379__note14411340111613"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="css_01_0379__p144114403165">You can switch AZs for an existing cluster. For details, see <a href="css_01_0201.html">Switching AZs for an Elasticsearch Cluster</a>.</p>
<div class="p" id="css_01_0379__p3101641181810">You can <strong id="css_01_0379__css_01_0201_en-us_topic_0000001438549444_b12888619193013">Add AZ</strong> or <strong id="css_01_0379__css_01_0201_en-us_topic_0000001438549444_b17504133310308">Migrate AZ</strong>.<ul id="css_01_0379__css_01_0201_en-us_topic_0000001438549444_ul04225613559"><li id="css_01_0379__css_01_0201_en-us_topic_0000001438549444_li1642135611554"><strong id="css_01_0379__css_01_0201_en-us_topic_0000001438549444_b1421516153117">Add AZ</strong>: Add one or two AZs to a single-AZ cluster, or add an AZ to a dual-AZ cluster to improve cluster availability.</li><li id="css_01_0379__css_01_0201_en-us_topic_0000001438549444_li288110413110"><strong id="css_01_0379__css_01_0201_en-us_topic_0000001438549444_b470415993115">Migrate AZ</strong>: Completely migrate data from the current AZ to another AZ that has sufficient resources.</li></ul>
</div>
</div></div>
</div>
</div>
<div class="section" id="css_01_0379__section5809152013345"><h4 class="sectiontitle">Planning the Cluster Version</h4><p id="css_01_0379__p737448103516">When selecting an Elasticsearch cluster version, consider factors such as service requirements, available features, performance, security updates, and long-term support, ensuring that the selected version can meet both current and future needs and provide a stable, secure environment for your data.</p>
<ul id="css_01_0379__en-us_topic_0000001282100129_ul125579175814"><li id="css_01_0379__en-us_topic_0000001282100129_li1655716155811">If you are deploying the Elasticsearch clusters of CSS for the first time, you are advised to use the latest version.</li><li id="css_01_0379__li317701252215">If you are migrating an in-house built or third-party Elasticsearch cluster to CSS without changing the cluster, keep the version of the source cluster.</li><li id="css_01_0379__li243416752416">If you are migrating an in-house built or third-party Elasticsearch cluster to CSS while recoding it, choose Elasticsearch 7.10.2 or 7.6.2.</li></ul>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="css_01_0379__table25351046105517" frame="border" border="1" rules="all"><caption><b>Table 3 </b>Cluster version support</caption><thead align="left"><tr id="css_01_0379__row1753644610558"><th align="left" class="cellrowborder" valign="top" width="20%" id="mcps1.3.3.4.2.5.1.1"><p id="css_01_0379__p1853618461554">Feature</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="20%" id="mcps1.3.3.4.2.5.1.2"><p id="css_01_0379__p135361546165518">Elasticsearch 7.6.2</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="20%" id="mcps1.3.3.4.2.5.1.3"><p id="css_01_0379__p20781175472516">Elasticsearch 7.10.2</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="40%" id="mcps1.3.3.4.2.5.1.4"><p id="css_01_0379__p153684614555">Details</p>
</th>
</tr>
</thead>
<tbody><tr id="css_01_0379__row18536746175519"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.1 "><p id="css_01_0379__p353674615553">Vector search</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.2 "><p id="css_01_0379__p1263711245268">√</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.3 "><p id="css_01_0379__p478175410251">√</p>
</td>
<td class="cellrowborder" valign="top" width="40%" headers="mcps1.3.3.4.2.5.1.4 "><p id="css_01_0379__p19536174635519"><a href="css_01_0117.html">Configuring Vector Search for Elasticsearch Clusters</a></p>
</td>
</tr>
<tr id="css_01_0379__row145361046115514"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.1 "><p id="css_01_0379__p1353613460553">Storage-compute decoupling</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.2 "><p id="css_01_0379__p10636102415261">√</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.3 "><p id="css_01_0379__p97816541254">√</p>
</td>
<td class="cellrowborder" valign="top" width="40%" headers="mcps1.3.3.4.2.5.1.4 "><p id="css_01_0379__p353614635516"><a href="css_01_0405.html">Configuring Storage-Compute Decoupling for an Elasticsearch Cluster</a></p>
</td>
</tr>
<tr id="css_01_0379__row45361446135515"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.1 "><p id="css_01_0379__p12536134614555">Flow Control 2.0</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.2 "><p id="css_01_0379__p2635122414268">√</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.3 "><p id="css_01_0379__p12781854162512">√</p>
</td>
<td class="cellrowborder" valign="top" width="40%" headers="mcps1.3.3.4.2.5.1.4 "><p id="css_01_0379__p3536144675512"><a href="css_01_0406.html">Configuring Flow Control 2.0 for an Elasticsearch Cluster</a></p>
</td>
</tr>
<tr id="css_01_0379__row353620463551"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.1 "><p id="css_01_0379__p10536646125511">Flow Control 1.0</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.2 "><p id="css_01_0379__p927372815268">√</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.3 "><p id="css_01_0379__p578114542253">√</p>
</td>
<td class="cellrowborder" valign="top" width="40%" headers="mcps1.3.3.4.2.5.1.4 "><p id="css_01_0379__p1853613466555"><a href="css_01_0407.html">Configuring Flow Control 1.0 for an Elasticsearch Cluster</a></p>
</td>
</tr>
<tr id="css_01_0379__row9536194615515"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.1 "><p id="css_01_0379__p25361846205520">Large query isolation</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.2 "><p id="css_01_0379__p3630424202615">√</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.3 "><p id="css_01_0379__p107817540254">√</p>
</td>
<td class="cellrowborder" valign="top" width="40%" headers="mcps1.3.3.4.2.5.1.4 "><p id="css_01_0379__p1353644612552"><a href="css_01_0408.html">Configuring Large Query Isolation for an Elasticsearch Cluster</a></p>
</td>
</tr>
<tr id="css_01_0379__row35366465553"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.1 "><p id="css_01_0379__p153616464553">Enhanced aggregation</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.2 "><p id="css_01_0379__p186295244260">x</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.3 "><p id="css_01_0379__p5781155472512">√</p>
</td>
<td class="cellrowborder" valign="top" width="40%" headers="mcps1.3.3.4.2.5.1.4 "><p id="css_01_0379__p453624645510"><a href="css_01_0409.html">Configuring Enhanced Aggregation for an Elasticsearch Cluster</a></p>
</td>
</tr>
<tr id="css_01_0379__row1853694665517"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.1 "><p id="css_01_0379__p35361346195520">Read/write splitting</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.2 "><p id="css_01_0379__p06283244266">√</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.3 "><p id="css_01_0379__p7781154112514">√</p>
</td>
<td class="cellrowborder" valign="top" width="40%" headers="mcps1.3.3.4.2.5.1.4 "><p id="css_01_0379__p13537446175514"><a href="css_01_0410.html">Configuring Read/Write Splitting Between Two Elasticsearch Clusters</a></p>
</td>
</tr>
<tr id="css_01_0379__row1838312223614"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.1 "><p id="css_01_0379__p138482211617">Switchover between hot and cold storage</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.2 "><p id="css_01_0379__p106274244269">√</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.3 "><p id="css_01_0379__p378114545253">√</p>
</td>
<td class="cellrowborder" valign="top" width="40%" headers="mcps1.3.3.4.2.5.1.4 "><p id="css_01_0379__p1338462213611"><a href="css_01_0079.html">Switching Between Hot and Cold Storage for an Elasticsearch Cluster</a></p>
</td>
</tr>
<tr id="css_01_0379__row170416717"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.1 "><p id="css_01_0379__p27112119712">Index recycle bin</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.2 "><p id="css_01_0379__p18626624112615">x</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.3 "><p id="css_01_0379__p1678235452515">√</p>
</td>
<td class="cellrowborder" valign="top" width="40%" headers="mcps1.3.3.4.2.5.1.4 "><p id="css_01_0379__p17711614713"><a href="css_01_0249.html">Configuring an Index Recycle Bin for an Elasticsearch Cluster</a></p>
</td>
</tr>
<tr id="css_01_0379__row153821359152914"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.1 "><p id="css_01_0379__p15382559182910">Enhanced import performance</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.2 "><p id="css_01_0379__p164377205320">x</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.3 "><p id="css_01_0379__p1143792013213">√</p>
</td>
<td class="cellrowborder" valign="top" width="40%" headers="mcps1.3.3.4.2.5.1.4 "><p id="css_01_0379__p18382185912918"><a href="css_01_0397.html">Enhancing the Data Import Performance of Elasticsearch Clusters</a></p>
</td>
</tr>
<tr id="css_01_0379__row101941195306"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.1 "><p id="css_01_0379__p2194819113015">Enhanced cluster kernel monitoring</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.2 "><p id="css_01_0379__p31221447173210">√</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.3 "><p id="css_01_0379__p112294711329">√</p>
</td>
<td class="cellrowborder" valign="top" width="40%" headers="mcps1.3.3.4.2.5.1.4 "><p id="css_01_0379__p19194101911302"><a href="css_01_0427.html">Configuring Kernel Monitoring for an Elasticsearch Cluster</a></p>
</td>
</tr>
<tr id="css_01_0379__row71721503015"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.1 "><p id="css_01_0379__p1917210510302">Index monitoring</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.2 "><p id="css_01_0379__p1387018477323">√</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.3.4.2.5.1.3 "><p id="css_01_0379__p3870134713321">√</p>
</td>
<td class="cellrowborder" valign="top" width="40%" headers="mcps1.3.3.4.2.5.1.4 "><p id="css_01_0379__p1917216563012"><a href="css_01_0428.html">Configuring Index Monitoring for an Elasticsearch Cluster</a></p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="css_01_0379__section1891413413349"><h4 class="sectiontitle">Planning Node Types</h4><p id="css_01_0379__p16211151124119">For an Elasticsearch cluster, the proper planning of different types of nodes is critical to optimizing performance and resource utilization. Before creating a cluster, determine the types of nodes to use based on service requirements, query load, data growth patterns, and performance goals. <a href="#css_01_0379__table1054422922518">Table 4</a> describes the characteristics of different node types and the purposes they are suited for.</p>
<div class="note" id="css_01_0379__note659434312111"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="css_01_0379__ul15590183261512"><li id="css_01_0379__li75901325154">If no master or client nodes were enabled when a cluster was created, you can add them if data nodes become overloaded later at some point. For details, see <a href="css_01_0151.html#css_01_0151__section1195734814511">Adding Master or Client Nodes</a>.</li><li id="css_01_0379__li118015341155">If no cold data nodes were enabled during cluster creation, they cannot be added later, so you have to determine whether to use cold data nodes while creating a cluster.</li></ul>
</div></div>
<div class="tablenoborder"><a name="css_01_0379__table1054422922518"></a><a name="table1054422922518"></a><table cellpadding="4" cellspacing="0" summary="" id="css_01_0379__table1054422922518" frame="border" border="1" rules="all"><caption><b>Table 4 </b>Characteristics and purposes of different types of nodes</caption><thead align="left"><tr id="css_01_0379__row8544729192513"><th align="left" class="cellrowborder" valign="top" width="24%" id="mcps1.3.4.4.2.4.1.1"><p id="css_01_0379__p8544429202510">Node Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="27.96%" id="mcps1.3.4.4.2.4.1.2"><p id="css_01_0379__p95007742818">Node Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="48.04%" id="mcps1.3.4.4.2.4.1.3"><p id="css_01_0379__p7375311162817">Characteristics</p>
</th>
</tr>
</thead>
<tbody><tr id="css_01_0379__row1154415291257"><td class="cellrowborder" valign="top" width="24%" headers="mcps1.3.4.4.2.4.1.1 "><p id="css_01_0379__p7544182922518">Data node (ESS)</p>
</td>
<td class="cellrowborder" valign="top" width="27.96%" headers="mcps1.3.4.4.2.4.1.2 "><p id="css_01_0379__p1544429182518">Data nodes are used to store data. In a cluster that has neither master nor client nodes, data nodes provide the functions of both types of nodes.</p>
</td>
<td class="cellrowborder" valign="top" width="48.04%" headers="mcps1.3.4.4.2.4.1.3 "><p id="css_01_0379__p4375141142816">Data nodes are mandatory for any cluster.</p>
<ul id="css_01_0379__ul4900180174419"><li id="css_01_0379__li1890040184413">If <strong id="css_01_0379__b3718193374312">Master node</strong> and <strong id="css_01_0379__b57181033134310">Client node</strong> are both unselected, data nodes will be used for all of the following purposes: cluster management, data storage, cluster access, and data analysis. To ensure reliability, a cluster should have a least three nodes.</li><li id="css_01_0379__li190013024413">If <strong id="css_01_0379__b1936864134517">Master node</strong> is selected but <strong id="css_01_0379__b14368194104515">Client node</strong> is not, data nodes will be used for data storage, cluster access, and data analysis.</li><li id="css_01_0379__li1649103015152">If <strong id="css_01_0379__b1033785503104625">Master node</strong> is unselected but <strong id="css_01_0379__b793985729104625">Client node</strong> is selected, data nodes will be used for data storage and cluster management.</li><li id="css_01_0379__li119008004413">If <strong id="css_01_0379__b1255825836104625">Master node</strong> and <strong id="css_01_0379__b1138258908104625">Client node</strong> are both selected, data nodes will be used for data storage only.</li></ul>
</td>
</tr>
<tr id="css_01_0379__row1954415292259"><td class="cellrowborder" valign="top" width="24%" headers="mcps1.3.4.4.2.4.1.1 "><p id="css_01_0379__p254462914256">Master node (ess-master)</p>
</td>
<td class="cellrowborder" valign="top" width="27.96%" headers="mcps1.3.4.4.2.4.1.2 "><p id="css_01_0379__p10544629142520">The master node is responsible for cluster management, such as metadata management, index creation and deletion, and shard allocation. It plays a critical role in metadata management, node management, stability guarantee, and cluster operation control for large-scale clusters.</p>
</td>
<td class="cellrowborder" valign="top" width="48.04%" headers="mcps1.3.4.4.2.4.1.3 "><ul id="css_01_0379__ul1076419354211"><li id="css_01_0379__li11764035921">Large-scale cluster: For a cluster that has more than 16 nodes, you are advised to add dedicated master nodes to effectively manage the cluster status and metadata.</li><li id="css_01_0379__li84761961737">Large quantities of indexes and shards: If the number of indexes or shards exceeds 10,000, a master node will have better performance in handling complex cluster management tasks, avoiding impact on the performance of data nodes.</li><li id="css_01_0379__li1197516381832">Better management of cluster nodes: The master node maintains the cluster metadata, including index mapping, settings, and aliases. For a complex cluster structure, a dedicated master node offers better management, including node joining, exiting, and fault detection. The master node plays a critical role in cluster node management.</li><li id="css_01_0379__li1785517228514">Improved cluster stability and reliability: A dedicated master node improves cluster stability and reliability by taking over cluster management responsibilities from data storage and query nodes.</li><li id="css_01_0379__li848418191361">Optimized performance for data nodes: By offloading cluster management tasks from data nodes to master nodes, you can allow data nodes to focus on data processing, which leads to improved performance.</li></ul>
</td>
</tr>
<tr id="css_01_0379__row1954472920256"><td class="cellrowborder" valign="top" width="24%" headers="mcps1.3.4.4.2.4.1.1 "><p id="css_01_0379__p195441029182517">Client node (ess-client)</p>
</td>
<td class="cellrowborder" valign="top" width="27.96%" headers="mcps1.3.4.4.2.4.1.2 "><p id="css_01_0379__p95441829152516">Client nodes receive and coordinate external requests, such as search and write requests. They play an important role in handling high-load queries, complex aggregations, managing a large number of shards, and improving cluster scalability.</p>
</td>
<td class="cellrowborder" valign="top" width="48.04%" headers="mcps1.3.4.4.2.4.1.3 "><ul id="css_01_0379__ul108651826205520"><li id="css_01_0379__li1686582615512">High QPS: In the face of a high queries per second (QPS), a dedicated client node can evenly distribute query requests, reducing the load of data nodes and improving the overall query performance.</li><li id="css_01_0379__li8505152814558">Complex aggregation queries: For complex, compute-intensive aggregation queries, a client node can dedicate to the handling of aggregation results, thus improving the efficiency and response speed of such queries.</li><li id="css_01_0379__li18765197566">Large number of shards: In a cluster with a large number of shards, a client node can effectively coordinate and manage query requests to each shard, improving efficiency in request forwarding and processing.</li><li id="css_01_0379__li238314201574">Reducing the load of data nodes: A client node parses search requests, determines the locations of index shards, and coordinates different nodes to execute searches. This reduces the load of data nodes by allowing them to focus on data storage and indexing.</li><li id="css_01_0379__li5119181319596">Improved cluster scalability: The use of client nodes allows for better cluster scalability and flexibility, enabling supporting for large datasets and more complex query requirements.</li></ul>
</td>
</tr>
<tr id="css_01_0379__row654492915258"><td class="cellrowborder" valign="top" width="24%" headers="mcps1.3.4.4.2.4.1.1 "><p id="css_01_0379__p1544529172515">Cold data node (ess-cold)</p>
</td>
<td class="cellrowborder" valign="top" width="27.96%" headers="mcps1.3.4.4.2.4.1.2 "><p id="css_01_0379__p954513294250">Cold data nodes are used to store query latency-insensitive data in large quantities. They offer an effective way to manage large datasets and cut storage costs.</p>
</td>
<td class="cellrowborder" valign="top" width="48.04%" headers="mcps1.3.4.4.2.4.1.3 "><ul id="css_01_0379__ul1034216399306"><li id="css_01_0379__li3342193919308">Storage of historical data in large quantities: Cold data nodes offer a more cost-effective solution for storing large quantities of historical data that are infrequently accessed but useful for analytical purposes.</li><li id="css_01_0379__li149061052194712">Optimizing hot data performance: By migrating cold data to cold data nodes, you reduce the storage load of hot data nodes, thereby optimizing their query and write performance.</li><li id="css_01_0379__li1477619394814">Insensitivity to query latency: Cold data nodes are a better option for storing data that is insensitive to a high query latency.</li><li id="css_01_0379__li0830650125018">Cost-effectiveness: Cold data nodes usually use large disks that offer inexpensive storage.</li></ul>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="css_01_0379__section462965244115"><h4 class="sectiontitle">Planning Node Storage</h4><ul id="css_01_0379__ul0211104235915"><li id="css_01_0379__li021144217591">Planning <strong id="css_01_0379__b14491830183417">node models</strong><p id="css_01_0379__p1014015765911">CSS supports various ECS models suited for different application needs. Select the appropriate models based on service requirements and performance expectations to achieve a perfect balance between storage performance and costs.</p>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="css_01_0379__table332800124014" frame="border" border="1" rules="all"><caption><b>Table 5 </b>Different node models and the intended application scenarios</caption><thead align="left"><tr id="css_01_0379__row13284010409"><th align="left" class="cellrowborder" valign="top" width="22%" id="mcps1.3.5.2.1.3.2.5.1.1"><p id="css_01_0379__p1932819017407">Node Model</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="12%" id="mcps1.3.5.2.1.3.2.5.1.2"><p id="css_01_0379__p851916015135">Disk Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="16%" id="mcps1.3.5.2.1.3.2.5.1.3"><p id="css_01_0379__p124191343204419">Specifications Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.5.2.1.3.2.5.1.4"><p id="css_01_0379__p19328609401">Recommended Scenario</p>
</th>
</tr>
</thead>
<tbody><tr id="css_01_0379__row1132812064014"><td class="cellrowborder" valign="top" width="22%" headers="mcps1.3.5.2.1.3.2.5.1.1 "><p id="css_01_0379__p203286019407">Computing-intensive</p>
</td>
<td class="cellrowborder" valign="top" width="12%" headers="mcps1.3.5.2.1.3.2.5.1.2 "><p id="css_01_0379__p1351919011132">Cloud drive</p>
</td>
<td class="cellrowborder" valign="top" width="16%" headers="mcps1.3.5.2.1.3.2.5.1.3 "><p id="css_01_0379__p141964304416">vCPUs:Memory = 1:2</p>
</td>
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.5.2.1.3.2.5.1.4 "><p id="css_01_0379__p153281207409">Small-volume searches (less than 100 GB on a single node).</p>
</td>
</tr>
<tr id="css_01_0379__row133295084013"><td class="cellrowborder" valign="top" width="22%" headers="mcps1.3.5.2.1.3.2.5.1.1 "><p id="css_01_0379__p4329509408">General computing</p>
</td>
<td class="cellrowborder" valign="top" width="12%" headers="mcps1.3.5.2.1.3.2.5.1.2 "><p id="css_01_0379__p851920016135">Cloud drive</p>
</td>
<td class="cellrowborder" valign="top" width="16%" headers="mcps1.3.5.2.1.3.2.5.1.3 "><p id="css_01_0379__p14419643144417">vCPUs:Memory = 1:4</p>
</td>
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.5.2.1.3.2.5.1.4 "><p id="css_01_0379__p1232916015404">Medium-scale e-commerce site search, social search, and log search, search and analysis where the data volume on a single node is in the range 100 GB to 1,000 GB.</p>
</td>
</tr>
<tr id="css_01_0379__row1932940154019"><td class="cellrowborder" valign="top" width="22%" headers="mcps1.3.5.2.1.3.2.5.1.1 "><p id="css_01_0379__p1329100174018">Memory-optimized</p>
</td>
<td class="cellrowborder" valign="top" width="12%" headers="mcps1.3.5.2.1.3.2.5.1.2 "><p id="css_01_0379__p851915091315">Cloud drive</p>
</td>
<td class="cellrowborder" valign="top" width="16%" headers="mcps1.3.5.2.1.3.2.5.1.3 "><p id="css_01_0379__p141917431440">vCPUs:Memory = 1:8</p>
</td>
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.5.2.1.3.2.5.1.4 "><p id="css_01_0379__p432919019403">Search and analysis where the data volume on a single node is in the range 100 GB to 2,000 GB.</p>
<p id="css_01_0379__p12329805409">This type of node is a good option for <strong id="css_01_0379__b950611413542">vector search</strong>, as its large memory helps improve cluster performance and stability.</p>
</td>
</tr>
</tbody>
</table>
</div>
</li><li id="css_01_0379__li1820375135920">Planning <strong id="css_01_0379__b185701312375">node specifications</strong><p id="css_01_0379__p2068012141098">Given the expected data handling capacities, it is always preferable to use a smaller number of nodes with larger specifications rather than a larger number of nodes with smaller specifications. For example, a cluster consisting of three nodes each with 32 CPU cores and 64 GB memory is usually better than a cluster consisting of 12 nodes each with 8 CPU cores and 16 GB memory in terms of stability and scalability.</p>
<p id="css_01_0379__p668071416913">The specific advantages are as follows:</p>
<ul id="css_01_0379__ul151981047111016"><li id="css_01_0379__li1868135211105">Cluster stability: High-specs nodes provide more powerful data processing capabilities and larger memory space, leading to higher overall cluster stability.</li><li id="css_01_0379__li194065571014">Improved scalability: When a cluster consisting of high-specs nodes encounters a performance bottleneck, you simply add more of these high-specs nodes. This is easier than increasing the specifications of existing nodes.</li><li id="css_01_0379__li1130634518102">Easier maintenance: A smaller number of nodes means easier maintenance and less complex management.</li></ul>
<p id="css_01_0379__p4607217114">In contrast, when a cluster consisting of low-specs nodes needs extra capacity, usually a vertical scale-up is performed, meaning to increase the specifications of existing nodes. This may entail not only more complex, challenging migration and upgrade processes, but also additional maintenance costs.</p>
<p id="css_01_0379__p1310813591106">To sum up, when planning a cluster, you must fully consider performance, costs, maintenance, and scalability, and choose the node specifications that best suit your needs.</p>
</li><li id="css_01_0379__li12597844308">Planning <strong id="css_01_0379__b4302182119110">storage capacity</strong><p id="css_01_0379__p799613181414">When planning the storage capacity of a CSS cluster, consider the following factors: the original data size, number of data replicas, data bloat rate, and disk usage The following is a recommended formula for determining the needed cluster storage capacity.</p>
<p id="css_01_0379__p12996539140">Storage capacity = Original data size x (1 + Number of replicas) x (1 + Data bloat rate) x (1 + Ratio of reserved space)</p>
<ul id="css_01_0379__ul15455145771717"><li id="css_01_0379__li14750125101812">Original data size: Determine the size of the original data that needs to be stored.</li><li id="css_01_0379__li1461416553172">Number of replicas: The default value is 1.</li><li id="css_01_0379__li8286124188">Data bloat rate: Extra data may be generated due to data indexing. Generally, you are advised to use a 25% data bloat rate.</li><li id="css_01_0379__li5101152113183">Disk usage: Considering the space occupied by the operating system and file system and the space reserved for optimized disk performance and redundancy, you are advised to keep the disk usage under 70%. That is, you need to reserve 30% of the total disk capacity.</li></ul>
<p id="css_01_0379__p1289383418189">A recommended formula is as follows: Cluster storage capacity = Original data size x 2 x 1.25 x 1.3</p>
<p id="css_01_0379__p1397163241810">To put it simply, if the original data size is known, the total storage capacity of the cluster needs to be 3.25 times that. This formula is for quick reference only. You still need to adjust it based on the actual applications and projected data growth rate.</p>
</li></ul>
</div>
<div class="section" id="css_01_0379__section171021159163215"><h4 class="sectiontitle">Planning the Node Quantity</h4><p id="css_01_0379__p5400115183712">Plan the node quantity based on performance requirements and predicted load. <a href="#css_01_0379__table20662829535">Table 6</a> provides a method for calculating the appropriate number of nodes. Following this method helps you ensure cluster performance and stability.</p>
<div class="tablenoborder"><a name="css_01_0379__table20662829535"></a><a name="table20662829535"></a><table cellpadding="4" cellspacing="0" summary="" id="css_01_0379__table20662829535" frame="border" border="1" rules="all"><caption><b>Table 6 </b>Calculating the number of cluster nodes</caption><thead align="left"><tr id="css_01_0379__row17662122115313"><th align="left" class="cellrowborder" valign="top" width="10%" id="mcps1.3.6.3.2.5.1.1"><p id="css_01_0379__p266282115315">Node</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="30%" id="mcps1.3.6.3.2.5.1.2"><p id="css_01_0379__p1066218216536">Performance Baseline</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="30%" id="mcps1.3.6.3.2.5.1.3"><p id="css_01_0379__p186628216539">Formula</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="30%" id="mcps1.3.6.3.2.5.1.4"><p id="css_01_0379__p1858216477512">Example</p>
</th>
</tr>
</thead>
<tbody><tr id="css_01_0379__row26627235314"><td class="cellrowborder" valign="top" width="10%" headers="mcps1.3.6.3.2.5.1.1 "><p id="css_01_0379__p766312115310">Write node</p>
</td>
<td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.6.3.2.5.1.2 "><ul id="css_01_0379__ul3362201101418"><li id="css_01_0379__li20362171118145">For a node that uses cloud disks, the write performance baseline of a single vCPU is 1 MB/s.</li><li id="css_01_0379__li936261119145">For an ultra-high I/O node, the write performance baseline of a single vCPU is 1.5 MB/s.</li></ul>
</td>
<td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.6.3.2.5.1.3 "><p id="css_01_0379__p11663926532">Number of write nodes = Peak traffic/Number of vCPUs per node/Write throughput per vCPU x Number of replicas</p>
</td>
<td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.6.3.2.5.1.4 "><p id="css_01_0379__p19583547105111">If the peak inbound traffic is 100 MB/s and a node has 16 vCPUs and 64 GB memory, 12 nodes (100/16/1 x 2) are needed.</p>
</td>
</tr>
<tr id="css_01_0379__row56635219537"><td class="cellrowborder" valign="top" width="10%" headers="mcps1.3.6.3.2.5.1.1 "><p id="css_01_0379__p566319216536">Query node</p>
</td>
<td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.6.3.2.5.1.2 "><p id="css_01_0379__p14663162185313">It is difficult to evaluate the performance baseline of a single node out of the context of specific application scenarios. The <strong id="css_01_0379__b56941615226">average query response time</strong> (in seconds) is used here to measure the query performance baseline.</p>
</td>
<td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.6.3.2.5.1.3 "><p id="css_01_0379__p4663132165313">Number of query nodes = QPS/(Number of vCPUs per node x 3/2/Average query response time in seconds) x Number of shards</p>
</td>
<td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.6.3.2.5.1.4 "><p id="css_01_0379__p4583124717516">If the query QPS is 1000, the average query response time is 100 ms (0.1s), three index shards are planned, and a node has 16 vCPUs and 64 GB memory, ~12 nodes (1000/(16 x 3/2/0.1) x 3) are needed.</p>
</td>
</tr>
<tr id="css_01_0379__row8693710184411"><td class="cellrowborder" valign="top" width="10%" headers="mcps1.3.6.3.2.5.1.1 "><p id="css_01_0379__p196931810154411">Total number of nodes</p>
</td>
<td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.6.3.2.5.1.2 "><p id="css_01_0379__p669391034411">N/A</p>
</td>
<td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.6.3.2.5.1.3 "><p id="css_01_0379__p46934101444">Total number of nodes = Number of write nodes + Number of query nodes</p>
</td>
<td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.6.3.2.5.1.4 "><p id="css_01_0379__p158311471512">Total number of nodes = Number of write nodes + Number of query nodes = 24</p>
</td>
</tr>
<tr id="css_01_0379__row982415692012"><td class="cellrowborder" colspan="4" valign="top" headers="mcps1.3.6.3.2.5.1.1 mcps1.3.6.3.2.5.1.2 mcps1.3.6.3.2.5.1.3 mcps1.3.6.3.2.5.1.4 "><div class="note" id="css_01_0379__note12125161412208"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="css_01_0379__p21253148205">Here, the total number of nodes refer to the number of data nodes plus that of cold data nodes.</p>
</div></div>
</td>
</tr>
</tbody>
</table>
</div>
<p id="css_01_0379__p5842205192913">In each cluster, the number of nodes supported by each node type varies, depending on the types of nodes used in that cluster. For details, see <a href="#css_01_0379__table144451631608">Table 7</a>.</p>
<div class="tablenoborder"><a name="css_01_0379__table144451631608"></a><a name="table144451631608"></a><table cellpadding="4" cellspacing="0" summary="" id="css_01_0379__table144451631608" frame="border" border="1" rules="all"><caption><b>Table 7 </b>Number of nodes of different types allowed in a single cluster</caption><thead align="left"><tr id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_row1976920435387"><th align="left" class="cellrowborder" valign="top" width="50.529999999999994%" id="mcps1.3.6.5.2.3.1.1"><p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p1976954343811">Node Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="49.47%" id="mcps1.3.6.5.2.3.1.2"><p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p1276920435389">Node Quantity</p>
</th>
</tr>
</thead>
<tbody><tr id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_row1769164393820"><td class="cellrowborder" valign="top" width="50.529999999999994%" headers="mcps1.3.6.5.2.3.1.1 "><p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p18769184333816">ess</p>
</td>
<td class="cellrowborder" valign="top" width="49.47%" headers="mcps1.3.6.5.2.3.1.2 "><p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p127691243123819">ess: 1-32</p>
</td>
</tr>
<tr id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_row177691843183816"><td class="cellrowborder" valign="top" width="50.529999999999994%" headers="mcps1.3.6.5.2.3.1.1 "><p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p776964317385">ess, ess-master</p>
</td>
<td class="cellrowborder" valign="top" width="49.47%" headers="mcps1.3.6.5.2.3.1.2 "><p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p9769194317381">ess: 1-200</p>
<p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p1976911438384">ess-master: an odd number ranging from 3 to 9</p>
</td>
</tr>
<tr id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_row107691743183814"><td class="cellrowborder" valign="top" width="50.529999999999994%" headers="mcps1.3.6.5.2.3.1.1 "><p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p1769443133816">ess, ess-client</p>
</td>
<td class="cellrowborder" valign="top" width="49.47%" headers="mcps1.3.6.5.2.3.1.2 "><p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p157691943163815">ess: 1-32</p>
<p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p1976917431383">ess-client: 1-32</p>
</td>
</tr>
<tr id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_row57692432388"><td class="cellrowborder" valign="top" width="50.529999999999994%" headers="mcps1.3.6.5.2.3.1.1 "><p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p18769343203813">ess, ess-cold</p>
</td>
<td class="cellrowborder" valign="top" width="49.47%" headers="mcps1.3.6.5.2.3.1.2 "><p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p12769194317384">ess: 1-32</p>
<p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p1138220591537">ess-cold: 1-32</p>
</td>
</tr>
<tr id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_row187701743193815"><td class="cellrowborder" valign="top" width="50.529999999999994%" headers="mcps1.3.6.5.2.3.1.1 "><p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p1477054314389">ess, ess-master, ess-client</p>
</td>
<td class="cellrowborder" valign="top" width="49.47%" headers="mcps1.3.6.5.2.3.1.2 "><p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p2770164303816">ess: 1-200</p>
<p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p877020432386">ess-master: an odd number ranging from 3 to 9</p>
<p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p177044313389">ess-client: 1-32</p>
</td>
</tr>
<tr id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_row27701443143811"><td class="cellrowborder" valign="top" width="50.529999999999994%" headers="mcps1.3.6.5.2.3.1.1 "><p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p2770443173816">ess, ess-master, ess-cold</p>
</td>
<td class="cellrowborder" valign="top" width="49.47%" headers="mcps1.3.6.5.2.3.1.2 "><p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p1777074316382">ess: 1-200</p>
<p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p5770124393818">ess-master: an odd number ranging from 3 to 9</p>
<p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p161801250733">ess-cold: 1-32</p>
</td>
</tr>
<tr id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_row67708433382"><td class="cellrowborder" valign="top" width="50.529999999999994%" headers="mcps1.3.6.5.2.3.1.1 "><p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p777018438384">ess, ess-client, ess-cold</p>
</td>
<td class="cellrowborder" valign="top" width="49.47%" headers="mcps1.3.6.5.2.3.1.2 "><p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p077074353816">ess: 1-32</p>
<p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p12770194393813">ess-client: 1-32</p>
<p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p1841425314">ess-cold: 1-32</p>
</td>
</tr>
<tr id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_row7770143153811"><td class="cellrowborder" valign="top" width="50.529999999999994%" headers="mcps1.3.6.5.2.3.1.1 "><p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p577044333818">ess, ess-master, ess-client, ess-cold</p>
</td>
<td class="cellrowborder" valign="top" width="49.47%" headers="mcps1.3.6.5.2.3.1.2 "><p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p1777017437384">ess: 1-200</p>
<p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p077034315388">ess-master: an odd number ranging from 3 to 9</p>
<p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p5770144314386">ess-client: 1-32</p>
<p id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_p1278719254314">ess-cold: 1-32</p>
</td>
</tr>
<tr id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_row14267396512"><td class="cellrowborder" colspan="2" valign="top" headers="mcps1.3.6.5.2.3.1.1 mcps1.3.6.5.2.3.1.2 "><div class="note" id="css_01_0379__css_01_0151_note3723104323311"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_ul12497123683"><li id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_li64981323284"><strong id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_b1543078192814">ess</strong>: data node, which is the default node type that is mandatory for cluster creation. The other three node types are optional.</li><li id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_li6498112311818"><strong id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_b14242174542716">ess-master</strong>: master node</li><li id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_li114982231584"><strong id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_b051753122720">ess-client</strong>: client node</li><li id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_li64984238813"><strong id="css_01_0379__css_01_0151_en-us_topic_0000001285669680_b151201057162715">ess-cold</strong>: cold data node</li></ul>
</div></div>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="css_01_0379__section1997018319358"><h4 class="sectiontitle">Planning a Cluster's Security Mode</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="css_01_0379__en-us_topic_0000001410060261_table198661437165914" frame="border" border="1" rules="all"><caption><b>Table 8 </b>Cluster security modes</caption><thead align="left"><tr id="css_01_0379__css_01_0380_row1882494514112"><th align="left" class="cellrowborder" colspan="2" valign="top" id="mcps1.3.7.2.2.5.1.1"><p id="css_01_0379__css_01_0380_p0824145101119">Cluster Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" id="mcps1.3.7.2.2.5.1.2"><p id="css_01_0379__css_01_0380_p3824545161116">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" id="mcps1.3.7.2.2.5.1.3"><p id="css_01_0379__css_01_0380_p19824184571117">Characteristics</p>
</th>
</tr>
</thead>
<tbody><tr id="css_01_0379__css_01_0380_row382415456118"><td class="cellrowborder" valign="top" width="10%" headers="mcps1.3.7.2.2.5.1.1 "><p id="css_01_0379__p154179414203">Non-security mode cluster</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.7.2.2.5.1.1 "><p id="css_01_0379__css_01_0380_p18824445131110">Cluster for which the security mode is disabled</p>
</td>
<td class="cellrowborder" valign="top" width="35%" headers="mcps1.3.7.2.2.5.1.2 "><p id="css_01_0379__css_01_0380_p12824114561119">With such a cluster, access to the cluster will require no user authentication, and data will be transmitted in plaintext using HTTP. Make sure the customer is in a secure environment, and do not expose the cluster access interface to the public network.</p>
</td>
<td class="cellrowborder" valign="top" width="35%" headers="mcps1.3.7.2.2.5.1.3 "><p id="css_01_0379__css_01_0380_en-us_topic_0000001410060261_p7867123745914">This type of cluster is mostly used for internal services and testing.</p>
<ul id="css_01_0379__css_01_0380_ul87801447174218"><li id="css_01_0379__css_01_0380_li978074716423">Advantage: simple and easy to access.</li><li id="css_01_0379__css_01_0380_li1878011473427">Disadvantage: poor security as anyone can access it.</li></ul>
</td>
</tr>
<tr id="css_01_0379__css_01_0380_row1382494518113"><td class="cellrowborder" rowspan="2" valign="top" width="10%" headers="mcps1.3.7.2.2.5.1.1 "><p id="css_01_0379__p136722051122012">Security-mode cluster</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.7.2.2.5.1.1 "><p id="css_01_0379__css_01_0380_p1482494513117">Cluster in security mode + HTTP</p>
</td>
<td class="cellrowborder" valign="top" width="35%" headers="mcps1.3.7.2.2.5.1.2 "><p id="css_01_0379__css_01_0380_p1282404517113">A security-mode cluster requires user authentication. It supports access control and data encryption, and it uses HTTP to transmit data in plaintext. Make sure the customer is in a secure environment, and do not expose the cluster access interface to the public network.</p>
</td>
<td class="cellrowborder" valign="top" width="35%" headers="mcps1.3.7.2.2.5.1.3 "><p id="css_01_0379__css_01_0380_en-us_topic_0000001410060261_p7867337165912">Access control by user permissions is supported. This type of cluster is suitable for workloads that are particularly performance-demanding.</p>
<ul id="css_01_0379__css_01_0380_ul149328416439"><li id="css_01_0379__css_01_0380_li993294134314">Advantage: User authentication improves cluster security. HTTP-based access ensures high performance of the cluster.</li><li id="css_01_0379__css_01_0380_li109325434320">Disadvantage: The cluster cannot be accessed from the public network.</li></ul>
</td>
</tr>
<tr id="css_01_0379__css_01_0380_row182416454111"><td class="cellrowborder" valign="top" headers="mcps1.3.7.2.2.5.1.1 "><p id="css_01_0379__css_01_0380_p3824445111118">Cluster in security mode + HTTPS</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.7.2.2.5.1.1 "><p id="css_01_0379__css_01_0380_p1375182731616">A security-mode cluster requires user authentication. It supports access control and data encryption, and it uses HTTPS to encrypt communication and enhance data security.</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.7.2.2.5.1.2 "><p id="css_01_0379__css_01_0380_en-us_topic_0000001410060261_p5867153717599">This type of cluster is suitable where there is a high security standard and public network access is required.</p>
<ul id="css_01_0379__css_01_0380_ul1151712715437"><li id="css_01_0379__css_01_0380_li19517167144318">Advantage: User authentication improves cluster security, and HTTPS-based secure communication allows for secure public network access.</li><li id="css_01_0379__css_01_0380_li551713774316">Disadvantage: HTTPS encrypts nearly all information sent between server and client, causing a read performance loss of around 20%.</li></ul>
</td>
</tr>
</tbody>
</table>
</div>
<p id="css_01_0379__en-us_topic_0000001478234873_p16118653165913">To access a security-mode cluster, you need to provide a username and password. CSS supports authentication for the following two types of users:</p>
<ul id="css_01_0379__en-us_topic_0000001478234873_ul15961822191016"><li id="css_01_0379__en-us_topic_0000001478234873_li496622131020">Administrator: The default administrator username is <strong id="css_01_0379__en-us_topic_0000001478234873_b1882413620149">admin</strong>, and the password is the one specified during cluster creation.</li><li id="css_01_0379__en-us_topic_0000001478234873_li111651241101219">Cluster user: created by the cluster administrator on Kibana. For details, see <a href="css_01_0417.html">Creating Users for an Elasticsearch Cluster and Granting Cluster Access</a>.</li></ul>
<div class="note" id="css_01_0379__note280744382218"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="css_01_0379__p28071643152213">You can change the security mode of an existing cluster. For details, see <a href="css_01_0158.html">Changing the Security Mode of an Elasticsearch Cluster</a>.</p>
<p id="css_01_0379__p67371215202410">You have many options when it comes to changing the security mode of a cluster: from non-security mode to security mode, from security mode to non-security mode, and switching between security modes using different web protocols (HTTP or HTTPS).</p>
</div></div>
</div>
<div class="section" id="css_01_0379__section1782085174310"><h4 class="sectiontitle">Planning the Number of Index Shards</h4><p id="css_01_0379__p146924622115">Before importing data to a cluster, carefully consider your service needs and plan the cluster's data structure and distribution in advance. This includes properly designing indexes and deciding on the appropriate number of index shards. To ensure optimal performance and scalability for a cluster, consider following these best practices:</p>
<ul id="css_01_0379__ul17421164342416"><li id="css_01_0379__li1442114437248"><strong id="css_01_0379__b931616278401">The size of a single shard</strong>: Keep the size of each shard between 10 GB and 50 GB. This helps strike a balance between storage efficiency and query performance.</li><li id="css_01_0379__li1042114315247"><strong id="css_01_0379__b32589417419">Total number of shards in a cluster</strong>: To facilitate management and avoid an excessively large scale, make sure the total number of shards in a cluster is less than 30,000. This helps maintain the stability and responsiveness of the cluster.</li><li id="css_01_0379__li1342174342416"><strong id="css_01_0379__b76181122124216">Memory-to-shards ratio</strong>: Limit the number of shards per 1 GB of memory to 20 to 30. This ensures that each shard has sufficient memory resources to respond to indexing and query operations.</li><li id="css_01_0379__li1142224302417"><strong id="css_01_0379__b196761527114410">Number</strong><strong id="css_01_0379__b2676127144416"> of shards per node</strong>: To prevent node overload, keep the number of shards on each node under 1000. This helps to improve node stability.</li><li id="css_01_0379__li17422124342414"><strong id="css_01_0379__b1220810433469">Relationship between the number of index shards and the number of nodes</strong>: For each index, make sure the number of shards is the same as or is an integral multiple of the number of nodes in the cluster. This helps improve load balancing and optimize query and indexing performance.</li></ul>
<p id="css_01_0379__p3470184612114">Following these suggestions, you can plan and manage index shards for a CSS cluster more effectively, improving the cluster's overall performance and maintainability.</p>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="css_01_0207.html">Using Elasticsearch for Data Search</a></div>
</div>
</div>
View Git Blame Copy Permalink