forked from docs/doc-exports
Yang, Tong
3b1f73dece
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
89 lines
12 KiB
HTML
89 lines
12 KiB
HTML
<a name="ALM-38002"></a><a name="ALM-38002"></a>
|
|
|
|
<h1 class="topictitle1">ALM-38002 Kafka Heap Memory Usage Exceeds the Threshold</h1>
|
|
<div id="body45831387"><div class="section" id="ALM-38002__s1d68eb52ba164179a0839134e1e6b8ac"><h4 class="sectiontitle">Description</h4><p id="ALM-38002__en-us_topic_0070543587_p6031802">The system checks the Kafka service status every 30 seconds. The alarm is generated when the heap memory usage of a Kafka instance exceeds the threshold (95% of the maximum memory) for 10 consecutive times.</p>
|
|
<p id="ALM-38002__p57993470103641">When the <strong id="ALM-38002__b9575105015161">Trigger Count</strong> is 1, this alarm is cleared when the heap memory usage is less than or equal to the threshold. When the <strong id="ALM-38002__b129675546165">Trigger Count</strong> is greater than 1, this alarm is cleared when the heap memory usage is less than or equal to 90% of the threshold.</p>
|
|
</div>
|
|
<div class="section" id="ALM-38002__s9c85b7560fb34e69918cd364117ce48d"><h4 class="sectiontitle">Attribute</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="ALM-38002__en-us_topic_0070543587_table35107966" frame="border" border="1" rules="all"><thead align="left"><tr id="ALM-38002__en-us_topic_0070543587_row5096677"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.2.1.4.1.1"><p id="ALM-38002__en-us_topic_0070543587_p10177686">Alarm ID</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.2.1.4.1.2"><p id="ALM-38002__en-us_topic_0070543587_p19086203">Alarm Severity</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.2.1.4.1.3"><p id="ALM-38002__en-us_topic_0070543587_p2478588">Automatically Cleared</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="ALM-38002__en-us_topic_0070543587_row66547971"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.2.1.4.1.1 "><p id="ALM-38002__en-us_topic_0070543587_p21676533">38002</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.2.1.4.1.2 "><p id="ALM-38002__en-us_topic_0070543587_p10968764">Major</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.2.1.4.1.3 "><p id="ALM-38002__en-us_topic_0070543587_p16054720">Yes</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="ALM-38002__sbb62fde6ce4b431ea22a183965106fc2"><h4 class="sectiontitle">Parameters</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="ALM-38002__en-us_topic_0070543587_table25363944" frame="border" border="1" rules="all"><thead align="left"><tr id="ALM-38002__en-us_topic_0070543587_row47050234"><th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.1.3.1.1"><p id="ALM-38002__en-us_topic_0070543587_p52972589">Name</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.1.3.1.2"><p id="ALM-38002__en-us_topic_0070543587_p62921288">Meaning</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="ALM-38002__row16305146588"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-38002__p192431315431">Source</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-38002__p692551319435">Specifies the cluster for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="ALM-38002__en-us_topic_0070543587_row63459572"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-38002__en-us_topic_0070543587_p39951704">ServiceName</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-38002__en-us_topic_0070543587_p14862621">Specifies the service name for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="ALM-38002__en-us_topic_0070543587_row66654731"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-38002__en-us_topic_0070543587_p30324164">RoleName</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-38002__en-us_topic_0070543587_p40338225">Specifies the role name for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="ALM-38002__en-us_topic_0070543587_row27499711"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-38002__en-us_topic_0070543587_p12884118">HostName</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-38002__en-us_topic_0070543587_p36980629">Specifies the object (host ID) for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="ALM-38002__en-us_topic_0070543587_row64390212"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-38002__en-us_topic_0070543587_p48224659">Trigger Condition</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-38002__en-us_topic_0070543587_p13883295">Specifies the threshold triggering the alarm. If the current indicator value exceeds this threshold, the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="ALM-38002__s89636580588b4796b43fa40cd637ddb5"><h4 class="sectiontitle">Impact on the System</h4><p id="ALM-38002__en-us_topic_0070543587_p50805071">If the available Kafka heap memory is insufficient, a memory overflow occurs and the service breaks down.</p>
|
|
</div>
|
|
<div class="section" id="ALM-38002__s65504f9cfd624eb190f063a3b472fb02"><h4 class="sectiontitle">Possible Causes</h4><p id="ALM-38002__en-us_topic_0070543587_p21570059">The heap memory of the Kafka instance is overused or the heap memory is inappropriately allocated.</p>
|
|
</div>
|
|
<div class="section" id="ALM-38002__s183422e8c3f84ba58b10458c5e041f07"><h4 class="sectiontitle">Procedure</h4><p class="tableheading" id="ALM-38002__en-us_topic_0070543587_p2344380"><strong id="ALM-38002__b2478093115566">Check heap memory usage.</strong></p>
|
|
<ol id="ALM-38002__ol25961852155611"><li id="ALM-38002__li4567022815563"><span>On the FusionInsight Manager portal, choose <strong id="ALM-38002__b78788146179">O&M </strong>><strong id="ALM-38002__b696261611712"> Alarm </strong>><strong id="ALM-38002__b11963121617170"> Alarms</strong> > <strong id="ALM-38002__b1685715013573">Kafka Heap Memory Usage Exceeds the Threshold</strong> > <strong id="ALM-38002__b1998755015563">Location</strong>. Check the host name of the instance involved in this alarm.</span></li><li id="ALM-38002__li118928315563"><a name="ALM-38002__li118928315563"></a><a name="li118928315563"></a><span>On the FusionInsight Manager portal, choose <strong id="ALM-38002__b250492851316">Cluster</strong> > <em id="ALM-38002__i115046289138">Name of the desired cluster</em> > <strong id="ALM-38002__b105041528171319">Services</strong> > <strong id="ALM-38002__b189741110123">Kafka</strong> > <strong id="ALM-38002__b18504928141319">Instance</strong>. Click the instance for which the alarm is generated to go to the page for the instance. Click the drop-down list in the upper right corner of the chart area, choose <strong id="ALM-38002__b450442871312">Customize</strong> > <strong id="ALM-38002__b2504132831319">Process</strong> > <strong id="ALM-38002__b432495361219">H</strong><strong id="ALM-38002__b03241753141212">eap Memory Usage of Kafka</strong>, and click <strong id="ALM-38002__b3504172818136">OK</strong>.</span></li><li id="ALM-38002__li264106815563"><span>Check whether the used heap memory of Kafka reaches 95% of the maximum heap memory specified for Kafka.</span><p><ul class="subitemlist" id="ALM-38002__ul3011961315563"><li id="ALM-38002__li2922308515563">If yes, go to <a href="#ALM-38002__li1593445465720">4</a>.</li><li id="ALM-38002__li1825970415563">If no, go to <a href="#ALM-38002__li3623590715563">6</a>.</li></ul>
|
|
</p></li></ol>
|
|
<p class="subitemlist" id="ALM-38002__p119128542574"><strong id="ALM-38002__b788405818572">Check the heap memory size configured for Kafka.</strong></p>
|
|
<ol start="4" id="ALM-38002__ol49343541572"><li id="ALM-38002__li1593445465720"><a name="ALM-38002__li1593445465720"></a><a name="li1593445465720"></a><span>On the FusionInsight Manager portal, choose <strong id="ALM-38002__b1793316546576">Cluster</strong> > <em id="ALM-38002__i11933145413574">Name of the desired cluster</em><strong id="ALM-38002__b8933105410572"> </strong>><strong id="ALM-38002__b6933454105713"> Services</strong> > <strong id="ALM-38002__b129337549574">Kafka</strong> > <strong id="ALM-38002__b14933115418578">Configurations</strong> > <strong id="ALM-38002__b993375417574">All</strong> <strong id="ALM-38002__b39330541570">Configurations</strong>> <strong id="ALM-38002__b1193325455717">Broker(Role)</strong> > <strong id="ALM-38002__b169338540572">Environment</strong>. Increase the value of <strong id="ALM-38002__b1693355435710">KAFKA_HEAP_OPTS</strong> by referring to the Note.</span><p><div class="note" id="ALM-38002__note1933205410578"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="ALM-38002__ul11933195415715"><li id="ALM-38002__li139338541577">It is recommended that <strong id="ALM-38002__b149334548571">-Xmx</strong> and <strong id="ALM-38002__b139331954165713">-Xms</strong> be set to the same value.</li><li id="ALM-38002__li893314540575">You are advised to view <strong id="ALM-38002__b0933354195720">Heap Memory Usage of Kafka</strong> by referring to <a href="#ALM-38002__li118928315563">2</a>, and set the value of <strong id="ALM-38002__b12933195411578">KAFKA_HEAP_OPTS</strong> to twice the value of <strong id="ALM-38002__b16933554135710">Heap Memory Used by Kafka.</strong></li></ul>
|
|
</div></div>
|
|
</p></li><li id="ALM-38002__li20934145413573"><span>Check whether the alarm is cleared.</span><p><ul class="subitemlist" id="ALM-38002__ul119341254105715"><li id="ALM-38002__li19341554175715">If yes, no further action is required.</li><li id="ALM-38002__li693445416570">If no, go to <a href="#ALM-38002__li3623590715563">6</a>.</li></ul>
|
|
</p></li></ol>
|
|
<p class="tableheading" id="ALM-38002__p4377467155620"><strong id="ALM-38002__b39397203155620">Collect fault information.</strong></p>
|
|
<ol start="6" id="ALM-38002__ol11156087155625"><li id="ALM-38002__li3623590715563"><a name="ALM-38002__li3623590715563"></a><a name="li3623590715563"></a><span>On the FusionInsight Manager portal, choose <strong id="ALM-38002__b1494713502318">O&M</strong> > <strong id="ALM-38002__b4402193310239">Log </strong>><strong id="ALM-38002__b19403163312320"> Download</strong>.</span></li><li id="ALM-38002__li4419406215563"><span>Select <strong id="ALM-38002__b5768771315563">Kafka</strong> in the required cluster from the <strong id="ALM-38002__b4942737015563">Service</strong> drop-down list.</span></li><li id="ALM-38002__li1145664103113"><span>Click <span><img id="ALM-38002__image1945644173117" src="en-us_image_0269417501.png"></span> in the upper right corner, and set <strong id="ALM-38002__b6456941173117">Start Date</strong> and <strong id="ALM-38002__b11456154113318">End Date</strong> for log collection to 10 minutes ahead of and after the alarm generation time, respectively. Then, click <strong id="ALM-38002__b13456164113319">Download</strong>.</span></li><li id="ALM-38002__li4498283615563"><span>Contact the <span id="ALM-38002__text4614151421417">O&M personnel</span> and send the collected logs.</span></li></ol>
|
|
</div>
|
|
<div class="section" id="ALM-38002__section1529716184534"><h4 class="sectiontitle">Alarm Clearing</h4><p id="ALM-38002__p4677152685316">After the fault is rectified, the system automatically clears this alarm.</p>
|
|
</div>
|
|
<div class="section" id="ALM-38002__s8f598989c0514353b852419f8c1e037e"><h4 class="sectiontitle">Related Information</h4><p id="ALM-38002__en-us_topic_0070543587_p24064686">None</p>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1298.html">Alarm Reference (Applicable to MRS 3.x)</a></div>
|
|
</div>
|
|
</div>
|
|
|