forked from docs/doc-exports
Yang, Tong
3b1f73dece
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
95 lines
15 KiB
HTML
95 lines
15 KiB
HTML
<a name="ALM-16007"></a><a name="ALM-16007"></a>
|
|
|
|
<h1 class="topictitle1">ALM-16007 Hive GC Time Exceeds the Threshold</h1>
|
|
<div id="body34872760"><div class="section" id="ALM-16007__s34b86b778d444baabc7fe3d80d14acb3"><h4 class="sectiontitle">Description</h4><p id="ALM-16007__en-us_topic_0070543664_p11859050">The system checks the garbage collection (GC) time of the Hive service every 60 seconds. This alarm is generated when the detected GC time exceeds the threshold (exceeds 12 seconds for three consecutive checks.) To change the threshold, choose <strong id="ALM-16007__b9224441104514"><strong id="ALM-16007__b122242414459">O&M > Alarm > Thresholds ></strong></strong> <em id="ALM-16007__i2226164154520">Name of the desired cluster</em> ><strong id="ALM-16007__b14225134114518"> <strong id="ALM-16007__b152252041194510">Hive</strong></strong>. This alarm is cleared when the Hive GC time is shorter than or equal to the threshold.</p>
|
|
</div>
|
|
<div class="section" id="ALM-16007__s81a942e194bc40faa3b87e3e5f0d1f18"><h4 class="sectiontitle">Attribute</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="ALM-16007__en-us_topic_0070543664_table58071663" frame="border" border="1" rules="all"><thead align="left"><tr id="ALM-16007__en-us_topic_0070543664_row40967500"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.2.1.4.1.1"><p id="ALM-16007__en-us_topic_0070543664_p30033166">Alarm ID</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.2.1.4.1.2"><p id="ALM-16007__en-us_topic_0070543664_p16767397">Alarm Severity</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.2.1.4.1.3"><p id="ALM-16007__en-us_topic_0070543664_p15981945">Automatically Cleared</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="ALM-16007__en-us_topic_0070543664_row19469169"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.2.1.4.1.1 "><p id="ALM-16007__en-us_topic_0070543664_p33498870">16007</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.2.1.4.1.2 "><p id="ALM-16007__en-us_topic_0070543664_p29053935">Major</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.2.1.4.1.3 "><p id="ALM-16007__en-us_topic_0070543664_p4558533">Yes</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="ALM-16007__s244a5390ae224126a71a61ee54348885"><h4 class="sectiontitle">Parameters</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="ALM-16007__en-us_topic_0070543664_table33696863" frame="border" border="1" rules="all"><thead align="left"><tr id="ALM-16007__en-us_topic_0070543664_row55371273"><th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.1.3.1.1"><p id="ALM-16007__en-us_topic_0070543664_p55888158">Name</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.1.3.1.2"><p id="ALM-16007__en-us_topic_0070543664_p30646972">Meaning</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="ALM-16007__row7754103692710"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-16007__p192431315431">Source</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-16007__p692551319435">Specifies the cluster for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="ALM-16007__en-us_topic_0070543664_row66485642"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-16007__en-us_topic_0070543664_p16627888">ServiceName</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-16007__en-us_topic_0070543664_p4681687">Specifies the service name for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="ALM-16007__en-us_topic_0070543664_row42135186"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-16007__en-us_topic_0070543664_p57506909">RoleName</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-16007__en-us_topic_0070543664_p27548028">Specifies the role name for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="ALM-16007__en-us_topic_0070543664_row46605663"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-16007__en-us_topic_0070543664_p16962328">HostName</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-16007__en-us_topic_0070543664_p31771289">Specifies the object (host ID) for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="ALM-16007__row1458813241945"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-16007__en-us_topic_0070543660_p47539882">Trigger Condition</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-16007__en-us_topic_0070543660_p25525229">Specifies the threshold triggering the alarm. If the current indicator value exceeds this threshold, the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="ALM-16007__sb088877b51dc4d32a2ef18052917a05f"><h4 class="sectiontitle">Impact on the System</h4><p id="ALM-16007__en-us_topic_0070543664_p23337611">If the GC time exceeds the threshold, Hive data read and write are affected.</p>
|
|
</div>
|
|
<div class="section" id="ALM-16007__s7d65f31b5c1d430c9208fa5c38f1379a"><h4 class="sectiontitle">Possible Causes</h4><p id="ALM-16007__en-us_topic_0070543664_p11298325">The memory of Hive instances is overused, the heap memory is inappropriately allocated. As a result, GCs occur frequently.</p>
|
|
</div>
|
|
<div class="section" id="ALM-16007__s72d69ebaf5a04e6aae35135bdd272d9d"><h4 class="sectiontitle">Procedure</h4><p class="tableheading" id="ALM-16007__en-us_topic_0070543664_p42749152"><strong id="ALM-16007__b5484643814385">Check the GC time.</strong></p>
|
|
<ol id="ALM-16007__ol37255434143814"><li id="ALM-16007__li5023556414380"><span>On the FusionInsight Manager portal, click <strong id="ALM-16007__b28662750155624">O&M > Alarm > Alarms</strong> and select the alarm whose <strong id="ALM-16007__b3150674214380">Alarm ID</strong> is <strong id="ALM-16007__b1512522314380">16007</strong>. Then check the role name in <strong id="ALM-16007__b14790172183618">Location </strong>and confirm the IP adress of the instance.</span><p><ul class="subitemlist" id="ALM-16007__ul6523405214380"><li id="ALM-16007__li2043387214380">If the role for which the alarm is generated is HiveServer, go to <a href="#ALM-16007__li6180447514380">2</a>.</li><li id="ALM-16007__li4453093014380">If the role for which the alarm is generated is MetaStore, go to <a href="#ALM-16007__li3832089314380">3</a>.</li></ul>
|
|
</p></li><li id="ALM-16007__li6180447514380"><a name="ALM-16007__li6180447514380"></a><a name="li6180447514380"></a><span>On the FusionInsight Manager portal, choose <strong id="ALM-16007__b256842804913">Cluster </strong>><em id="ALM-16007__i1557110281490">Name of the desired cluster </em>><strong id="ALM-16007__b115691628164917"> Services</strong> > <strong id="ALM-16007__b13023504143018">Hive</strong> > <strong id="ALM-16007__b50102677143018">Instance</strong> and click the HiveServer for which the alarm is generated to go to the<strong id="ALM-16007__b14303164441516"> Dashboard </strong>page. Click the drop-down menu in the <strong id="ALM-16007__b9966756057">Chart </strong>area and choose <strong id="ALM-16007__b996617562515">Customize </strong>> <strong id="ALM-16007__b1185302665010">GC</strong>, and select <strong id="ALM-16007__b5759142214380">Garbage Collection (GC) Time of HiveServer</strong> and click <strong id="ALM-16007__b4856075814380">OK</strong> to check whether the GC time is longer than 12 seconds.</span><p><ul class="subitemlist" id="ALM-16007__ul3669332514380"><li id="ALM-16007__li4110733414380">If yes, go to <a href="#ALM-16007__li542936514380">4</a>.</li><li id="ALM-16007__li4135973814380">If no, go to <a href="#ALM-16007__li2731494414380">7</a>.</li></ul>
|
|
</p></li><li id="ALM-16007__li3832089314380"><a name="ALM-16007__li3832089314380"></a><a name="li3832089314380"></a><span>On the FusionInsight Manager portal, choose <strong id="ALM-16007__b352625544914">Cluster </strong>><em id="ALM-16007__i15528145513495">Name of the desired </em><em id="ALM-16007__i12179412165818">cluster</em> ><strong id="ALM-16007__b75271055184910"> Services</strong> > <strong id="ALM-16007__b15134140135514">Hive</strong> > <strong id="ALM-16007__b201351340165520">Instance</strong> and click the MetaStore for which the alarm is generated to go to the<strong id="ALM-16007__b232016134188"> Dashboard </strong>page. Click the drop-down menu in the <strong id="ALM-16007__b426763919817">Chart </strong>area and choose <strong id="ALM-16007__b6267339887">Customize </strong>> <strong id="ALM-16007__b1575673017502">GC</strong>, and select <strong id="ALM-16007__b518088814380">Garbage Collection (GC) Time of MetaStore</strong> and click <strong id="ALM-16007__b4662799214380">OK</strong> to check whether the GC time is longer than 12 seconds.</span><p><ul class="subitemlist" id="ALM-16007__ul6095392514380"><li id="ALM-16007__li1877099214380">If yes, go to <a href="#ALM-16007__li542936514380">4</a>.</li><li id="ALM-16007__li4405536014380">If no, go to <a href="#ALM-16007__li2731494414380">7</a>.</li></ul>
|
|
</p></li></ol>
|
|
<p class="tableheading" id="ALM-16007__p1171441714380"><strong id="ALM-16007__b534043143833">Check the current JVM configuration.</strong></p>
|
|
<ol start="4" id="ALM-16007__ol34397166143848"><li id="ALM-16007__li542936514380"><a name="ALM-16007__li542936514380"></a><a name="li542936514380"></a><span>On the FusionInsight Manager portal, choose <strong id="ALM-16007__b123018251504"><strong id="ALM-16007__b83022565017">Cluster </strong></strong>><em id="ALM-16007__i1333192585014">Name of the desired <em id="ALM-16007__i9730111712589">cluster </em></em>><strong id="ALM-16007__b143117257500"><strong id="ALM-16007__b831125145010"> Services</strong> > <strong id="ALM-16007__b73192515506">Hive</strong> > <strong id="ALM-16007__b43182595016">Configurations > All Configurations</strong></strong>. Choose <strong id="ALM-16007__b3552398514380">HiveServer/MetaStore</strong> > <strong id="ALM-16007__b5128041014380">JVM</strong>. Adjust the value of <strong id="ALM-16007__b5887050714380">-Xmx</strong> in <strong id="ALM-16007__b6007251714380">HIVE_GC_OPTS/METASTORE_GC_OPTS</strong> as the following rules. Click <strong id="ALM-16007__b378174514380">Save</strong>.</span><p><div class="note" id="ALM-16007__note638551412512"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><div class="p" id="ALM-16007__p863882712519">Suggestions for GC parameter settings for the HiveServer:<ul id="ALM-16007__ul15827113432817"><li id="ALM-16007__li188271534122816">When the Hive GC time exceeds the threshold, change the value of <strong id="ALM-16007__b1682193112812">-Xmx</strong> to twice the default value. For example, if <strong id="ALM-16007__b126821431192818">-Xmx</strong> is set to 2 GB by default, change the value of <strong id="ALM-16007__b6682193152813">-Xmx</strong> to 4 GB.</li></ul>
|
|
</div>
|
|
<ul id="ALM-16007__ul743118506284"><li id="ALM-16007__li14431150192816">You are advised to change the value of <strong id="ALM-16007__b1968218317289">-Xms</strong> to set the ratio of <strong id="ALM-16007__b1168233118286">-Xms</strong> and <strong id="ALM-16007__b66821031142810">-Xmx</strong> to 1:2 to avoid performance problems when JVM dynamically.</li></ul>
|
|
<div class="p" id="ALM-16007__p141314122620">Suggestions for GC parameter settings for the MetaServer:<ul id="ALM-16007__ul280712511345"><li id="ALM-16007__li1280765115347">When the Meta GC time exceeds the threshold, change the value of <strong id="ALM-16007__b68071551163418">-Xmx</strong> to twice the default value. For example, if <strong id="ALM-16007__b880715511341">-Xmx</strong> is set to 2 GB by default, change the value of <strong id="ALM-16007__b7807125115347">-Xmx</strong> to 4 GB.</li></ul>
|
|
<ul id="ALM-16007__ul13161155662810"><li id="ALM-16007__li19161165672820">You are advised to change the value of <strong id="ALM-16007__b1716105652813">-Xms</strong> to set the ratio of <strong id="ALM-16007__b816117562284">-Xms</strong> and <strong id="ALM-16007__b1816185619281">-Xmx</strong> to 1:2 to avoid performance problems when JVM dynamically.</li></ul>
|
|
</div>
|
|
</div></div>
|
|
</p></li><li id="ALM-16007__li15313102416318"><span>Click <strong id="ALM-16007__b193941191220">More > Restart Service </strong>to restart the service.</span></li><li id="ALM-16007__li4201503014380"><span>Check whether the alarm is cleared.</span><p><ul class="subitemlist" id="ALM-16007__ul5437149614380"><li id="ALM-16007__li4886429214380">If yes, no further action is required.</li><li id="ALM-16007__li6569360014380">If no, go to <a href="#ALM-16007__li2731494414380">7</a>.</li></ul>
|
|
</p></li></ol>
|
|
<p class="tableheading" id="ALM-16007__p1958141714380"><strong id="ALM-16007__b4006721014393">Collect fault information.</strong></p>
|
|
<ol start="7" id="ALM-16007__ol65276457143859"><li id="ALM-16007__li2731494414380"><a name="ALM-16007__li2731494414380"></a><a name="li2731494414380"></a><span>On the FusionInsight Manager portal of active and standby clusters, choose <strong id="ALM-16007__b39977366113627">O&M</strong> > <strong id="ALM-16007__b24251979113627">Log > Download</strong>.</span></li><li id="ALM-16007__li3268317714380"><span>In the <strong id="ALM-16007__b4450790714380">Service</strong>, select <strong id="ALM-16007__b4876181041210">Hive</strong> in the required cluster.</span></li><li id="ALM-16007__li1145664103113"><span>Click <span><img id="ALM-16007__image1945644173117" src="en-us_image_0269417383.png"></span> in the upper right corner, and set <strong id="ALM-16007__b6456941173117">Start Date</strong> and <strong id="ALM-16007__b11456154113318">End Date</strong> for log collection to 10 minutes ahead of and after the alarm generation time, respectively. Then, click <strong id="ALM-16007__b13456164113319">Download</strong>.</span></li><li id="ALM-16007__li5932781214380"><span>Contact the <span id="ALM-16007__text4614151421417">O&M personnel</span> and send the collected logs.</span></li></ol>
|
|
</div>
|
|
<div class="section" id="ALM-16007__section1529716184534"><h4 class="sectiontitle">Alarm Clearing</h4><p id="ALM-16007__p4677152685316">After the fault is rectified, the system automatically clears this alarm.</p>
|
|
</div>
|
|
<div class="section" id="ALM-16007__sf78347b4005e47619ccd012fecc50255"><h4 class="sectiontitle">Related Information</h4><p id="ALM-16007__en-us_topic_0070543664_p19297076">None</p>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1298.html">Alarm Reference (Applicable to MRS 3.x)</a></div>
|
|
</div>
|
|
</div>
|
|
|