forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
78 lines
8.0 KiB
HTML
78 lines
8.0 KiB
HTML
<a name="mrs_01_1055"></a><a name="mrs_01_1055"></a>
|
|
|
|
<h1 class="topictitle1">Storm Performance Tuning</h1>
|
|
<div id="body1590370637078"><div class="section" id="mrs_01_1055__se7b7ce0409a246f186c78bf28bd4ce2c"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_1055__afa2df4ef67af4af59f1b515438954116">You can modify Storm parameters to improve Storm performance in specific service scenarios.</p>
|
|
<p id="mrs_01_1055__p184721821141115">This section applies to MRS 3.x or later.</p>
|
|
<p id="mrs_01_1055__p11526143493219">Modify the service configuration parameters. For details, see <a href="mrs_01_2125.html">Modifying Cluster Service Configuration Parameters</a>.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_1055__sa60487d3c6204ce9b537d08b887c7867"><h4 class="sectiontitle">Topology Tuning</h4><p id="mrs_01_1055__a6472b009595746ae8bad4ce5f18371e1">This task enables you to optimize topologies to improve efficiency for Storm to process data. It is recommended that topologies be optimized in scenarios with lower reliability requirements.</p>
|
|
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1055__t522a3dd789f8415d92b695f14b1e2198" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Tuning parameters</caption><thead align="left"><tr id="mrs_01_1055__rb30d396d77474993ae130298f3fb69b1"><th align="left" class="cellrowborder" valign="top" width="24.240000000000002%" id="mcps1.3.2.3.2.4.1.1"><p id="mrs_01_1055__ac449e0c054e34d119b556993c4d5b616">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="19.189999999999998%" id="mcps1.3.2.3.2.4.1.2"><p id="mrs_01_1055__a87887049bfe045f2ae364ddae0b9caf6">Default Value</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="56.57%" id="mcps1.3.2.3.2.4.1.3"><p id="mrs_01_1055__ad5e6f55f9e6e4c59b4d45f1a9913c8b9">Scenario</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_1055__raf53ca537a1441c0b02d6995bb5078ee"><td class="cellrowborder" valign="top" width="24.240000000000002%" headers="mcps1.3.2.3.2.4.1.1 "><p id="mrs_01_1055__ad4cddf28c15248f5a58f294e39f07f14">topology.acker.executors</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.189999999999998%" headers="mcps1.3.2.3.2.4.1.2 "><p id="mrs_01_1055__acedf1e69be114057a749bc5425a3e1ad">null</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.57%" headers="mcps1.3.2.3.2.4.1.3 "><p id="mrs_01_1055__ad9f7496694fc45cba860939a37eceef9">Specifies the number of acker executors. If a service application has lower reliability requirements and certain data does not need to be processed, this parameter can be set to <strong id="mrs_01_1055__b2109835104919">null</strong> or <strong id="mrs_01_1055__b1511473544910">0</strong> so that you can set acker off, flow control is weakened, and message delay is not calculated. This improves performance.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1055__r16f9874d2cc44bed88db6b51e4b8e4ab"><td class="cellrowborder" valign="top" width="24.240000000000002%" headers="mcps1.3.2.3.2.4.1.1 "><p id="mrs_01_1055__a9fa1f78ec020418ba0192ca1dc607e94">topology.max.spout.pending</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.189999999999998%" headers="mcps1.3.2.3.2.4.1.2 "><p id="mrs_01_1055__a89356c2fdcda4b73b93b5af4a4427318">null</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.57%" headers="mcps1.3.2.3.2.4.1.3 "><p id="mrs_01_1055__aebe49c1a8b4f47aa9ebe05a0f421605f">Specifies the number of messages cached by spout. The parameter value takes effect only when acker is not <strong id="mrs_01_1055__b1453664324620">0</strong> or <strong id="mrs_01_1055__b6542134394610">null</strong>. Spout adds each message sent to downstream bolt into the pending queue. The message is removed from the queue after downstream bolt processes the message and the processing is confirmed. When the pending queue is full, spout stops sending messages. Increasing the pending value improves the message throughput of spout per second but prolongs the delay.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1055__ra094ccac5b2745e696e6b856d7cf723a"><td class="cellrowborder" valign="top" width="24.240000000000002%" headers="mcps1.3.2.3.2.4.1.1 "><p id="mrs_01_1055__a17e160d77587438fb16b3b9b574969a1">topology.transfer.buffer.size</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.189999999999998%" headers="mcps1.3.2.3.2.4.1.2 "><p id="mrs_01_1055__a88cd668ebde742f19a65df480816a01a">32</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.57%" headers="mcps1.3.2.3.2.4.1.3 "><p id="mrs_01_1055__aee82073ddcb74482934694a2d418de13">Specifies the size of the Distuptor message queue for each worker process. It is recommended that the size be between 4 to 32. Increasing the queue size improves the throughput but may prolong the delay.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1055__r9e5dad2317cc4f56a4701444a3e90b7b"><td class="cellrowborder" valign="top" width="24.240000000000002%" headers="mcps1.3.2.3.2.4.1.1 "><p id="mrs_01_1055__a1b042994e49c4a92907b3cf14dbc8732">RES_CPUSET_PERCENTAGE</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.189999999999998%" headers="mcps1.3.2.3.2.4.1.2 "><p id="mrs_01_1055__aa1655e3f671e46c7b0e9b378c56d2f0b">80</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.57%" headers="mcps1.3.2.3.2.4.1.3 "><p id="mrs_01_1055__a6da219be5f3943d4b792ef483b9407d5">Specifies the percentage of physical CPU resources used by the supervisor role instance (including startup and management worker processes) on each node. Adjust the parameter value based on service volume requirements of the node on which the supervisor exists, to optimize CPU usage.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="mrs_01_1055__s6ca3eebe1a3a4e76bc34541ebd2b0327"><h4 class="sectiontitle">JVM Tuning</h4><p id="mrs_01_1055__a4dbf8e4cf32c472fbd86fd585024f99b">If an application must occupy more memory resources to process a large volume of data and the size of worker memory is greater than 2 GB, the G1 garbage collection algorithm is recommended.</p>
|
|
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1055__t2ea54ebd043d44a6a42c9e42d2cf907e" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Tuning parameters</caption><thead align="left"><tr id="mrs_01_1055__rfb4cc6005b5744cca1ee52a23ee2996c"><th align="left" class="cellrowborder" valign="top" width="24.240000000000002%" id="mcps1.3.3.3.2.4.1.1"><p id="mrs_01_1055__a91ee346ff844466396048e0e2ac504d3">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="19.189999999999998%" id="mcps1.3.3.3.2.4.1.2"><p id="mrs_01_1055__a2a39cb5e13034100b9b03fdde7851966">Default Value</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="56.57%" id="mcps1.3.3.3.2.4.1.3"><p id="mrs_01_1055__a4b2b9d2a65b84272af7b7a7c1befb98c">Scenario</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_1055__r41510a04418d42e99b28fe0d206e2f2b"><td class="cellrowborder" valign="top" width="24.240000000000002%" headers="mcps1.3.3.3.2.4.1.1 "><p id="mrs_01_1055__aa51428f2178541d08f580d8ac1c1189c">WORKER_GC_OPTS</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.189999999999998%" headers="mcps1.3.3.3.2.4.1.2 "><p id="mrs_01_1055__a54fd6f71bafa4b958a4e55ee7be7ef37">-Xms1G -Xmx1G -XX:+UseG1GC -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.57%" headers="mcps1.3.3.3.2.4.1.3 "><p id="mrs_01_1055__en-us_topic_0116526954_p403551114918">If an application must occupy more memory resources to process a large volume of data and the size of worker memory is greater than 2 GB, the G1 garbage collection algorithm is recommended. In this case, change the parameter value to <strong id="mrs_01_1055__b519791234910">-Xms2G -Xmx5G -XX:+UseG1GC</strong>.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1054.html">Performance Tuning</a></div>
|
|
</div>
|
|
</div>
|
|
|