forked from docs/doc-exports
Yang, Tong
3b1f73dece
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
86 lines
14 KiB
HTML
86 lines
14 KiB
HTML
<a name="alm_13002"></a><a name="alm_13002"></a>
|
|
|
|
<h1 class="topictitle1">ALM-13002 ZooKeeper Memory Usage Exceeds the Threshold</h1>
|
|
<div id="body8662426"><div class="section" id="alm_13002__en-us_topic_0191813895_section46652504"><h4 class="sectiontitle">Description</h4><p id="alm_13002__en-us_topic_0191813895_p51759670">The system checks the ZooKeeper service status every 30 seconds. The alarm is generated when the memory usage of a ZooKeeper instance exceeds the threshold (80% of the maximum memory).</p>
|
|
<p id="alm_13002__en-us_topic_0191813895_p63183853">The alarm is cleared when the memory usage is less than the threshold.</p>
|
|
</div>
|
|
<div class="section" id="alm_13002__en-us_topic_0191813895_section17219353"><h4 class="sectiontitle">Attribute</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="alm_13002__en-us_topic_0191813895_table17618432" frame="border" border="1" rules="all"><thead align="left"><tr id="alm_13002__en-us_topic_0191813895_row61813935"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.2.1.4.1.1"><p id="alm_13002__en-us_topic_0191813895_p40872853">Alarm ID</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.2.1.4.1.2"><p id="alm_13002__en-us_topic_0191813895_p22366803">Alarm Severity</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.2.1.4.1.3"><p id="alm_13002__en-us_topic_0191813895_p66880658">Auto Clear</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="alm_13002__en-us_topic_0191813895_row48624250"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.2.1.4.1.1 "><p id="alm_13002__en-us_topic_0191813895_p46250162">13002</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.2.1.4.1.2 "><p id="alm_13002__en-us_topic_0191813895_p55275618">Major</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.2.1.4.1.3 "><p id="alm_13002__en-us_topic_0191813895_p48140083">Yes</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="alm_13002__en-us_topic_0191813895_section20756449"><h4 class="sectiontitle">Parameters</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="alm_13002__en-us_topic_0191813895_table7032652" frame="border" border="1" rules="all"><thead align="left"><tr id="alm_13002__en-us_topic_0191813895_row1472067"><th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.1.3.1.1"><p id="alm_13002__en-us_topic_0191813895_p52128573">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.1.3.1.2"><p id="alm_13002__en-us_topic_0191813895_p61664867">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="alm_13002__en-us_topic_0191813895_row28798367"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="alm_13002__en-us_topic_0191813895_p50966358">ServiceName</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="alm_13002__en-us_topic_0191813895_p34634354">Specifies the service for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="alm_13002__en-us_topic_0191813895_row43273738"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="alm_13002__en-us_topic_0191813895_p15511864">RoleName</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="alm_13002__en-us_topic_0191813895_p48501438">Specifies the role for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="alm_13002__en-us_topic_0191813895_row33859762"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="alm_13002__en-us_topic_0191813895_p58286192">HostName</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="alm_13002__en-us_topic_0191813895_p23561115">Specifies the host for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="alm_13002__en-us_topic_0191813895_row10723444"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="alm_13002__en-us_topic_0191813895_p63292653">Trigger Condition</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="alm_13002__en-us_topic_0191813895_p26431238">Generates an alarm when the actual indicator value exceeds the specified threshold.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="alm_13002__en-us_topic_0191813895_section52590313"><h4 class="sectiontitle">Impact on the System</h4><p id="alm_13002__en-us_topic_0191813895_p60555495">If the available ZooKeeper memory is insufficient, a memory overflow occurs and the service breaks down.</p>
|
|
</div>
|
|
<div class="section" id="alm_13002__en-us_topic_0191813895_section3550770"><h4 class="sectiontitle">Possible Causes</h4><p id="alm_13002__en-us_topic_0191813895_p6048057">The memory usage of the ZooKeeper instance is overused or the memory is inappropriately allocated.</p>
|
|
</div>
|
|
<div class="section" id="alm_13002__en-us_topic_0191813895_section31956933"><h4 class="sectiontitle">Procedure</h4><ol id="alm_13002__en-us_topic_0191813895_ol56619731152326"><li class="tableheading" id="alm_13002__en-us_topic_0191813895_li54360240152326"><span>Check the memory usage.</span><p><ol type="a" id="alm_13002__en-us_topic_0191813895_ol4727180152241"><li id="alm_13002__en-us_topic_0191813895_li19965518">On the MRS cluster details page, choose <strong id="alm_13002__b136624425575">Alarms</strong> > <strong id="alm_13002__b6668144211579">ALM-13002 ZooKeeper Memory Usage Exceeds the Threshold</strong> > <strong id="alm_13002__b166681642185717">Location</strong>. Check the IP address of the instance for which the alarm is generated.</li><li id="alm_13002__en-us_topic_0191813895_li45471939">On the MRS cluster details page, choose <strong id="alm_13002__b14261528103818">Components</strong> > <strong id="alm_13002__b9432182883816">ZooKeeper</strong> > <strong id="alm_13002__b74325280389">Instances</strong> > <strong id="alm_13002__b1443218289386">quorumpeer</strong> (IP address of the instance for which the alarm is generated) > <strong id="alm_13002__b12432182819389">Customize</strong> > <strong id="alm_13002__b843372873815">ZooKeeper Heap And Direct Buffer Resource</strong>. Check the heap memory usage.<div class="note" id="alm_13002__en-us_topic_0191813895_note199815453351"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="alm_13002__en-us_topic_0191813895_p18981545203511">For MRS 1.7.2 or earlier, log in to MRS Manager and choose <strong id="alm_13002__b14498233387">Services</strong> > <strong id="alm_13002__b1449811315387">ZooKeeper</strong> > <strong id="alm_13002__b15498193123812">Instance</strong> > <strong id="alm_13002__b14986317383">quorumpeer</strong> (IP address of the instance for which the alarm is generated) > <strong id="alm_13002__b54993333810">Customize</strong> > <strong id="alm_13002__b15499163123818">ZooKeeper Heap And Direct Buffer Resource</strong>. Check the heap memory usage.</p>
|
|
</div></div>
|
|
</li><li id="alm_13002__en-us_topic_0191813895_li6594274">Check whether the used heap memory of ZooKeeper reaches 80% of the maximum heap memory specified for ZooKeeper.<ul class="subitemlist" id="alm_13002__en-us_topic_0191813895_ul59348466"><li id="alm_13002__en-us_topic_0191813895_li64374149">If yes, go to <a href="#alm_13002__en-us_topic_0191813895_cn_58_42_000001_3_mmccppss_stepb2">1.d</a>.</li><li id="alm_13002__en-us_topic_0191813895_li46923618">If no, go to <a href="#alm_13002__en-us_topic_0191813895_cn_58_42_000001_3_mmccppss_stepb4">1.f</a>.</li></ul>
|
|
</li><li id="alm_13002__en-us_topic_0191813895_cn_58_42_000001_3_mmccppss_stepb2"><a name="alm_13002__en-us_topic_0191813895_cn_58_42_000001_3_mmccppss_stepb2"></a><a name="en-us_topic_0191813895_cn_58_42_000001_3_mmccppss_stepb2"></a>On MRS Manager, choose <strong id="alm_13002__b29071632145819">Services</strong> > <strong id="alm_13002__b169077325585">ZooKeeper</strong> > <strong id="alm_13002__b8907432115813">Configuration</strong> > <strong id="alm_13002__b109086328589">All</strong> > <strong id="alm_13002__b790803255811">quorumpeer</strong> > <strong id="alm_13002__b890863212586">System</strong>. Increase the value of <strong id="alm_13002__b683653410587">-Xmx</strong> in <strong id="alm_13002__b0837173410586">GC_OPTS</strong> as required.</li><li id="alm_13002__en-us_topic_0191813895_li48906009">Check whether the alarm is cleared.<ul class="subitemlist" id="alm_13002__en-us_topic_0191813895_ul37500900"><li id="alm_13002__en-us_topic_0191813895_li1963786">If yes, no further action is required.</li><li id="alm_13002__en-us_topic_0191813895_li17674074">If no, go to <a href="#alm_13002__en-us_topic_0191813895_cn_58_42_000001_3_mmccppss_stepb4">1.f</a>.</li></ul>
|
|
</li><li id="alm_13002__en-us_topic_0191813895_cn_58_42_000001_3_mmccppss_stepb4"><a name="alm_13002__en-us_topic_0191813895_cn_58_42_000001_3_mmccppss_stepb4"></a><a name="en-us_topic_0191813895_cn_58_42_000001_3_mmccppss_stepb4"></a>On the MRS cluster details page, choose <strong id="alm_13002__b83321851195815">Components</strong> > <strong id="alm_13002__b4332115155819">ZooKeeper</strong> > <strong id="alm_13002__b193331151105819">Instances</strong> > <strong id="alm_13002__b6333125115582">quorumpeer</strong> (IP address of the instance for which the alarm is generated) > <strong id="alm_13002__b433315145810">Customize</strong> > <strong id="alm_13002__b4333151115814">ZooKeeper Heap And Direct Buffer Resource</strong>. Check the direct buffer memory usage.<div class="note" id="alm_13002__en-us_topic_0191813895_note1459720271361"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="alm_13002__en-us_topic_0191813895_p14597327193612">For MRS 1.7.2 or earlier, log in to MRS Manager and choose <strong id="alm_13002__b46978499211">Services</strong> > <strong id="alm_13002__b571219494219">ZooKeeper</strong> > <strong id="alm_13002__b471312491622">Instance</strong> > <strong id="alm_13002__b17149491824">quorumpeer</strong> (IP address of the instance for which the alarm is generated) > <strong id="alm_13002__b6715114912218">Customize</strong> > <strong id="alm_13002__b147161149426">ZooKeeper Heap And Direct Buffer Resource</strong>. Check the direct buffer memory usage.</p>
|
|
</div></div>
|
|
</li><li id="alm_13002__en-us_topic_0191813895_li66607344">Check whether the used direct buffer memory of ZooKeeper reaches 80% of the maximum direct buffer memory specified for ZooKeeper.<ul class="subitemlist" id="alm_13002__en-us_topic_0191813895_ul62595191"><li id="alm_13002__en-us_topic_0191813895_li26485815">If yes, go to <a href="#alm_13002__en-us_topic_0191813895_li49457583153150">1.h</a>.</li><li id="alm_13002__en-us_topic_0191813895_li64976233">If no, go to <a href="#alm_13002__en-us_topic_0191813895_li572522141314">2</a>.</li></ul>
|
|
</li><li id="alm_13002__en-us_topic_0191813895_li49457583153150"><a name="alm_13002__en-us_topic_0191813895_li49457583153150"></a><a name="en-us_topic_0191813895_li49457583153150"></a>On the MRS cluster details page, choose <strong id="alm_13002__b8552184502">Components</strong> > <strong id="alm_13002__b85522418019">ZooKeeper</strong> > <strong id="alm_13002__b175537418014">Service Configuration</strong>. Set <strong id="alm_13002__b855374703">Type</strong> to <strong id="alm_13002__b8553174709">All</strong> and choose <strong id="alm_13002__b12553194608">quorumpeer</strong> > <strong id="alm_13002__b1555384702">System</strong>.<p id="alm_13002__en-us_topic_0191813895_p23417023153155">Increase the value of <strong id="alm_13002__b79093211707">-XX:MaxDirectMemorySize</strong> in <strong id="alm_13002__b20914421408">GC_OPTS</strong> as required.</p>
|
|
<div class="note" id="alm_13002__en-us_topic_0191813895_note0851431377"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="alm_13002__en-us_topic_0191813895_p1986173173712">For MRS 1.7.2 or earlier, log in to MRS Manager, choose <strong id="alm_13002__b18243920335">Services</strong> > <strong id="alm_13002__b52582202312">ZooKeeper</strong> > <strong id="alm_13002__b18258220237">Service Configuration</strong>. Set <strong id="alm_13002__b425910201231">Type</strong> to <strong id="alm_13002__b1125911203318">All</strong> and choose <strong id="alm_13002__b22601201332">quorumpeer</strong> > <strong id="alm_13002__b16260132013313">System</strong>.</p>
|
|
</div></div>
|
|
</li><li id="alm_13002__en-us_topic_0191813895_li55925239">Check whether the alarm is cleared.<ul class="subitemlist" id="alm_13002__en-us_topic_0191813895_ul33565108"><li id="alm_13002__en-us_topic_0191813895_li33650520">If yes, no further action is required.</li><li id="alm_13002__en-us_topic_0191813895_li34419228">If no, go to <a href="#alm_13002__en-us_topic_0191813895_li572522141314">2</a>.</li></ul>
|
|
</li></ol>
|
|
</p></li><li id="alm_13002__en-us_topic_0191813895_li572522141314"><a name="alm_13002__en-us_topic_0191813895_li572522141314"></a><a name="en-us_topic_0191813895_li572522141314"></a><span>Collect fault information.</span><p><ol type="a" id="alm_13002__en-us_topic_0191813895_en-us_topic_0191813935_ol6089206913036"><li id="alm_13002__en-us_topic_0191813895_en-us_topic_0191813935_li4478836213036">On MRS Manager, choose <span class="menucascade" id="alm_13002__menucascade353614151511"><b><span class="uicontrol" id="alm_13002__uicontrol1653511516110">System</span></b> > <b><span class="uicontrol" id="alm_13002__uicontrol155358151911">Export Log</span></b></span>.</li><li id="alm_13002__li18574327401">Contact technical support engineers for help. For details, see <a href="https://docs.otc.t-systems.com/en-us/public/learnmore.html" target="_blank" rel="noopener noreferrer">technical support</a>.</li></ol>
|
|
</p></li></ol>
|
|
</div>
|
|
<div class="section" id="alm_13002__en-us_topic_0191813895_section19176948"><h4 class="sectiontitle">Reference</h4><p id="alm_13002__en-us_topic_0191813895_p64983754">None</p>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_0241.html">Alarm Reference (Applicable to Versions Earlier Than MRS 3.x)</a></div>
|
|
</div>
|
|
</div>
|
|
|