forked from docs/doc-exports
Yang, Tong
2195db241c
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Reviewed-by: Rechenburg, Matthias <matthias.rechenburg@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
85 lines
14 KiB
HTML
85 lines
14 KiB
HTML
<a name="ALM-19009"></a><a name="ALM-19009"></a>
|
|
|
|
<h1 class="topictitle1">ALM-19009 Direct Memory Usage of the HBase Process Exceeds the Threshold</h1>
|
|
<div id="body39746667"><div class="section" id="ALM-19009__s3435379dbdf748c08421e7ac1c26b5e7"><h4 class="sectiontitle">Description</h4><p id="ALM-19009__en-us_topic_0070543523_p2147306">The system checks the HBase service status every 30 seconds. The alarm is generated when the direct memory usage of an HBase service exceeds the threshold (90% of the maximum memory).</p>
|
|
<p id="ALM-19009__en-us_topic_0070543523_p19325759">The alarm is cleared when the direct memory usage is less than the threshold.</p>
|
|
</div>
|
|
<div class="section" id="ALM-19009__sf3d1a2ffedbd4aee8ba8579b3d3aaf5a"><h4 class="sectiontitle">Attribute</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="ALM-19009__en-us_topic_0070543523_table21882681" frame="border" border="1" rules="all"><thead align="left"><tr id="ALM-19009__en-us_topic_0070543523_row59271898"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.2.1.4.1.1"><p id="ALM-19009__en-us_topic_0070543523_p36294438">Alarm ID</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.2.1.4.1.2"><p id="ALM-19009__en-us_topic_0070543523_p54168362">Alarm Severity</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.2.1.4.1.3"><p id="ALM-19009__en-us_topic_0070543523_p25561231">Automatically Cleared</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="ALM-19009__en-us_topic_0070543523_row57193853"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.2.1.4.1.1 "><p id="ALM-19009__en-us_topic_0070543523_p2190491">19009</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.2.1.4.1.2 "><p id="ALM-19009__en-us_topic_0070543523_p43212049">Major</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.2.1.4.1.3 "><p id="ALM-19009__en-us_topic_0070543523_p10515080">Yes</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="ALM-19009__sc79d584ee66b4e159ebce7daa7df7aaf"><h4 class="sectiontitle">Parameters</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="ALM-19009__en-us_topic_0070543523_table46415143" frame="border" border="1" rules="all"><thead align="left"><tr id="ALM-19009__en-us_topic_0070543523_row41820417"><th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.1.3.1.1"><p id="ALM-19009__en-us_topic_0070543523_p32010577">Name</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.1.3.1.2"><p id="ALM-19009__en-us_topic_0070543523_p42719917">Meaning</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="ALM-19009__row73008232103"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-19009__p192431315431">Source</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-19009__p692551319435">Specifies the cluster for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="ALM-19009__en-us_topic_0070543523_row37761278"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-19009__en-us_topic_0070543523_p38764716">ServiceName</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-19009__en-us_topic_0070543523_p52934302">Specifies the service name for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="ALM-19009__en-us_topic_0070543523_row6646677"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-19009__en-us_topic_0070543523_p1509928">RoleName</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-19009__en-us_topic_0070543523_p55195381">Specifies the role name for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="ALM-19009__en-us_topic_0070543523_row26996384"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-19009__en-us_topic_0070543523_p39223476">HostName</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-19009__en-us_topic_0070543523_p22984981">Specifies the object (host ID) for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="ALM-19009__s6b4264f46b514e1a82ce8cb781cf9141"><h4 class="sectiontitle">Impact on the System</h4><p id="ALM-19009__en-us_topic_0070543523_p49844138">If the available HBase direct memory is insufficient, a memory overflow occurs and the service breaks down.</p>
|
|
</div>
|
|
<div class="section" id="ALM-19009__s6336b0fea6144e86932beeacc300ed62"><h4 class="sectiontitle">Possible Causes</h4><p id="ALM-19009__en-us_topic_0070543523_p10843355">The direct memory of the HBase service is overused or the direct memory is inappropriately allocated.</p>
|
|
</div>
|
|
<div class="section" id="ALM-19009__sa3cddf5eca904b619cf9c1f3d8ffaaa1"><h4 class="sectiontitle">Procedure</h4><p class="tableheading" id="ALM-19009__en-us_topic_0070543523_p5896594"><strong id="ALM-19009__b8352718195430">Check direct memory usage.</strong></p>
|
|
<ol id="ALM-19009__ol64515859195437"><li id="ALM-19009__li36906153195425"><span>On the <span id="ALM-19009__text34789336432">MRS</span> Manager portal, click <span class="menucascade" id="ALM-19009__menucascade9622183132519"><b><span class="uicontrol" id="ALM-19009__uicontrol862223115259">O&M</span></b> > <b><span class="uicontrol" id="ALM-19009__uicontrol106221731182512">Alarm</span></b> > <b><span class="uicontrol" id="ALM-19009__uicontrol9622153122518">Alarms</span></b></span> and select the alarm whose <strong id="ALM-19009__b18853005195425">ID</strong> is <strong id="ALM-19009__b35459323195425">19009</strong>. Check the <strong id="ALM-19009__b1955573445015">RoleName</strong> in <strong id="ALM-19009__b052583712505">Location</strong> and confirm the IP address of <strong id="ALM-19009__b1241513413507">HostName</strong>.</span><p><ul class="subitemlist" id="ALM-19009__ul33926845195425"><li id="ALM-19009__li12934386195425">If the role for which the alarm is generated is HMaster, go to <a href="#ALM-19009__li51947016195425">2</a>.</li><li id="ALM-19009__li41052351195425">If the role for which the alarm is generated is RegionServer, go to <a href="#ALM-19009__li17000576195425">3</a>.</li></ul>
|
|
</p></li><li id="ALM-19009__li51947016195425"><a name="ALM-19009__li51947016195425"></a><a name="li51947016195425"></a><span>On the <span id="ALM-19009__text16594185716568">MRS</span> Manager portal, choose <strong id="ALM-19009__b93655375152">Cluster</strong> > <em id="ALM-19009__i175981233194118">Name of the desired cluster</em> ><strong id="ALM-19009__b1236543715159"> Services</strong> > <strong id="ALM-19009__b36608407195425">HBase</strong> > <strong id="ALM-19009__b61040211195425">Instance</strong> and click the HMaster for which the alarm is generated to go to the<strong id="ALM-19009__b14303164441516"> Dashboard </strong>page. Click the drop-down menu in the <strong id="ALM-19009__b653511582416">Chart </strong>area and choose<strong id="ALM-19009__b69658114246"> Customize</strong> > <strong id="ALM-19009__b1785919537166">CPU and Memory</strong> > <strong id="ALM-19009__b5137569195425">HMaster Heap Memory Usage and Direct Memory Usage Statistics</strong> and click <strong id="ALM-19009__b46238124195425">OK</strong> to check whether the used direct memory of the HBase service reaches 90% of the maximum direct memory specified for HBase.</span><p><ul class="subitemlist" id="ALM-19009__ul57967673195425"><li id="ALM-19009__li54300533195425">If yes, go to <a href="#ALM-19009__li30000576195425">4</a>.</li><li id="ALM-19009__li36267014195425">If no, go to <a href="#ALM-19009__li62317418195425">8</a>.</li></ul>
|
|
</p></li><li id="ALM-19009__li17000576195425"><a name="ALM-19009__li17000576195425"></a><a name="li17000576195425"></a><span>On the <span id="ALM-19009__text18748358175617">MRS</span> Manager portal, choose <strong id="ALM-19009__b390175141719">Cluster </strong>> <em id="ALM-19009__i96641338163616">Name of the desired cluster</em> ><strong id="ALM-19009__b1090213518178"> Services</strong> > <strong id="ALM-19009__b46958754195425">HBase</strong> > <strong id="ALM-19009__b19975608195425">Instance</strong> and click the RegionServer for which the alarm is generated to go to the<strong id="ALM-19009__b16371135214195"> Dashboard </strong>page. Click the drop-down menu in the <strong id="ALM-19009__b384164314233">Chart </strong>area and choose<strong id="ALM-19009__b18695040142313"> Customize</strong> > <strong id="ALM-19009__b17229103817176">CPU and Memory </strong>><strong id="ALM-19009__b6230123881716"> RegionServer Heap Memory Usage and Direct Memory Usage Statistics</strong> and click <strong id="ALM-19009__b63465673195425">OK</strong> to check whether the used direct memory of the HBase service reaches 90% of the maximum direct memory specified for HBase.</span><p><ul class="subitemlist" id="ALM-19009__ul24258574195425"><li id="ALM-19009__li40445884195425">If yes, go to <a href="#ALM-19009__li30000576195425">4</a>.</li><li id="ALM-19009__li54891180195425">If no, go to <a href="#ALM-19009__li62317418195425">8</a>.</li></ul>
|
|
</p></li><li id="ALM-19009__li30000576195425"><a name="ALM-19009__li30000576195425"></a><a name="li30000576195425"></a><span>On the <span id="ALM-19009__text01561607574">MRS</span> Manager portal, choose <strong id="ALM-19009__b15857144814172">Cluster </strong>> <em id="ALM-19009__i158151043103617">Name of the desired cluster</em> ><strong id="ALM-19009__b20858248131711"> Services</strong> > <strong id="ALM-19009__b34869409195425">HBase</strong> > <strong id="ALM-19009__b45389225195425">Configurations</strong>, and click <strong id="ALM-19009__b52648615195425">All Configurations</strong>. Choose <strong id="ALM-19009__b4075495195425">HMaster/RegionServer</strong> > <strong id="ALM-19009__b36679462195425">System</strong> and check whether <strong id="ALM-19009__b68231112710">XX:MaxDirectMemorySize</strong> exists in <strong id="ALM-19009__b482313121517">GC_OPTS</strong>.</span><p><ul id="ALM-19009__ul148181324713"><li id="ALM-19009__li1779510368120">If yes, go to <a href="#ALM-19009__li131714294313">5</a>.</li><li id="ALM-19009__li171078461213">If no, go to <a href="#ALM-19009__li336333834">6</a>.</li></ul>
|
|
</p></li><li id="ALM-19009__li131714294313"><a name="ALM-19009__li131714294313"></a><a name="li131714294313"></a><span>On the <span id="ALM-19009__text1876611110578">MRS</span> Manager portal, choose <strong id="ALM-19009__b76331130936">Cluster </strong>> <em id="ALM-19009__i1963318302310">Nameof the desired cluster</em> ><strong id="ALM-19009__b156334301637"> Services</strong> > <strong id="ALM-19009__b1863315303312">HBase</strong> > <strong id="ALM-19009__b66331304316">Configurations</strong>, and click <strong id="ALM-19009__b1063315301335">All Configurations</strong>. Choose <strong id="ALM-19009__b206331730736">HMaster/RegionServer</strong> > <strong id="ALM-19009__b1663316301238">System</strong> and delete <strong id="ALM-19009__b136338304320">XX:MaxDirectMemorySize</strong> from <strong id="ALM-19009__b146331307310">GC_OPTS</strong>.</span></li><li id="ALM-19009__li336333834"><a name="ALM-19009__li336333834"></a><a name="li336333834"></a><span>Check whether the <strong id="ALM-19009__b15780101619320">ALM-19008 Heap Memory Usage of the HBase Process Exceeds the Threshold</strong> alarm is generated.</span><p><p id="ALM-19009__p178201451238">If yes, handle the alarm by referring to <strong id="ALM-19009__b20936134113215">ALM-19008 Heap Memory Usage of the HBase Process Exceeds the Threshold</strong>.</p>
|
|
<p id="ALM-19009__p147673247417">If no, go to <a href="#ALM-19009__li62317418195425">8</a>.</p>
|
|
</p></li><li id="ALM-19009__li13893891195425"><span>Check whether the alarm is cleared.</span><p><ul class="subitemlist" id="ALM-19009__ul3485547195425"><li id="ALM-19009__li1569734195425">If yes, no further action is required.</li><li id="ALM-19009__li60039606195425">If no, go to <a href="#ALM-19009__li62317418195425">8</a>.</li></ul>
|
|
</p></li></ol>
|
|
<p class="tableheading" id="ALM-19009__p31369927195425"><strong id="ALM-19009__b22491650195455">Collect fault information.</strong></p>
|
|
<ol start="8" id="ALM-19009__ol37986748195458"><li id="ALM-19009__li62317418195425"><a name="ALM-19009__li62317418195425"></a><a name="li62317418195425"></a><span>On the <span id="ALM-19009__text11701534577">MRS</span> Manager interface of active and standby clusters, choose <strong id="ALM-19009__b18341057121810">O&M</strong> > <strong id="ALM-19009__b11471205591811">Log </strong>><strong id="ALM-19009__b18471115541813"> Download</strong>.</span></li><li id="ALM-19009__li37385456195425"><span>In the <strong id="ALM-19009__b23985857195425">Service</strong> in the required cluster drop-down list box, select <strong id="ALM-19009__b14546125195425">HBase</strong>.</span></li><li id="ALM-19009__li1145664103113"><span>Click <span><img id="ALM-19009__image1945644173117" src="en-us_image_0000001583087501.png"></span> in the upper right corner, and set <strong id="ALM-19009__b6456941173117">Start Date</strong> and <strong id="ALM-19009__b11456154113318">End Date</strong> for log collection to 10 minutes ahead of and after the alarm generation time, respectively. Then, click <strong id="ALM-19009__b13456164113319">Download</strong>.</span></li><li id="ALM-19009__li48139755195425"><span>Contact the <span id="ALM-19009__text4614151421417">O&M personnel</span> and send the collected fault logs.</span></li></ol>
|
|
</div>
|
|
<div class="section" id="ALM-19009__section1529716184534"><h4 class="sectiontitle">Alarm Clearing</h4><p id="ALM-19009__p4677152685316">After the fault is rectified, the system automatically clears this alarm.</p>
|
|
</div>
|
|
<div class="section" id="ALM-19009__s6541e68162fd43eca31c9a37236f1419"><h4 class="sectiontitle">Related Information</h4><p id="ALM-19009__en-us_topic_0070543523_p27619288">None</p>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1298.html">Alarm Reference (Applicable to MRS 3.x)</a></div>
|
|
</div>
|
|
</div>
|
|
|