forked from docs/doc-exports
Yang, Tong
3b1f73dece
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
94 lines
16 KiB
HTML
94 lines
16 KiB
HTML
<a name="alm_12052"></a><a name="alm_12052"></a>
|
|
|
|
<h1 class="topictitle1">ALM-12052 Usage of Temporary TCP Ports Exceeds the Threshold</h1>
|
|
<div id="body8662426"><div class="section" id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_section41396267"><h4 class="sectiontitle">Description</h4><p id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_p43212949">The system checks the usage of temporary TCP ports every 30 seconds. This alarm is generated when the usage of temporary TCP ports exceeds the threshold (the default threshold is <strong id="alm_12052__b03851044173210">80%</strong>) for multiple times (the default value is <strong id="alm_12052__b183863445329">5</strong>). </p>
|
|
<p id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_p53372227">You can change the threshold by choosing <strong id="alm_12052__b171022467329">System</strong> > <strong id="alm_12052__b6102164673214">Threshold Configuration</strong> > <strong id="alm_12052__b9103154613216">Host</strong> > <strong id="alm_12052__b910320464329">Network Status</strong> > <strong id="alm_12052__b131031846163214">TCP Ephemeral Port Usage</strong> > <strong id="alm_12052__b8104146183219">TCP Ephemeral Port Usage</strong>.</p>
|
|
<p id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_p4918336211927">If the <strong id="alm_12052__b1260713477321">hit number</strong> is <strong id="alm_12052__b7608847193219">1</strong>, this alarm is cleared when the usage of temporary TCP ports is less than or equal to the threshold. If the <strong id="alm_12052__b76086474325">hit number</strong> is greater than <strong id="alm_12052__b4609154715326">1</strong>, this alarm is cleared when the usage of temporary TCP ports is less than or equal to 90% of the threshold.</p>
|
|
</div>
|
|
<div class="section" id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_section37022085"><h4 class="sectiontitle">Attribute</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_table52321376" frame="border" border="1" rules="all"><thead align="left"><tr id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_row39266406"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.2.1.4.1.1"><p id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_p26462316">Alarm ID</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.2.1.4.1.2"><p id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_p63072822">Alarm Severity</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.2.1.4.1.3"><p id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_p8624940">Auto Clear</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_row27531576"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.2.1.4.1.1 "><p id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_p15465151">12052</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.2.1.4.1.2 "><p id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_p44717739">Major</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.2.1.4.1.3 "><p id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_p65367140">Yes</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_section64763312"><h4 class="sectiontitle">Parameters</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_table60246993" frame="border" border="1" rules="all"><thead align="left"><tr id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_row16742709"><th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.1.3.1.1"><p id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_p13982213">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.1.3.1.2"><p id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_p58817486">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_row66595895"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_p25558376">ServiceName</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_p56962570">Specifies the service for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_row42901084"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_p52435818">RoleName</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_p19442876">Specifies the role for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_row40768163"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_p13886870">HostName</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_p51094723">Specifies the host for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_row57199326"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_p2633810">Trigger Condition</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_p12012051">Specifies the threshold for triggering the alarm.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_section45998902"><h4 class="sectiontitle">Impact on the System</h4><p id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_p33452040">Services on the host fail to establish connections with the external and services are interrupted.</p>
|
|
</div>
|
|
<div class="section" id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_section11336934"><h4 class="sectiontitle">Possible Causes</h4><ul id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_ul25260697"><li id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_li26019688">The temporary ports do not meet service requirements.</li><li id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_li32850608">The system is abnormal.</li></ul>
|
|
</div>
|
|
<div class="section" id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_section34923547"><h4 class="sectiontitle">Procedure</h4><p class="tableheading" id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_p43653554"><strong id="alm_12052__b91891648338">Expand the range of temporary ports.</strong></p>
|
|
<ol id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_ol64201369151254"><li id="alm_12052__en-us_topic_0191813905_li1487713813414"><span>Go to the MRS cluster details page and choose <strong id="alm_12052__b635011515335">Alarms</strong>.</span></li><li id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_li34903479151245"><span>In the real-time alarm list, click the alarm. In the <strong id="alm_12052__b1592216633315">Alarm Details</strong> area, obtain the IP address of the host for which the alarm is generated.</span></li><li id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_li8609526151245"><span>Use PuTTY to log in to the host for which the alarm is generated as user <strong id="alm_12052__b389911713337">omm</strong>.</span></li><li id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_li26282994151245"><span>Run the <strong id="alm_12052__b1578109103314">cat /proc/sys/net/ipv4/ip_local_port_range |cut -f 1</strong> command to obtain the start port number. Run the <strong id="alm_12052__b19781199333">cat /proc/sys/net/ipv4/ip_local_port_range |cut -f 2</strong> command to obtain the end port number. Subtract the start port number from the end port number to obtain the total number of temporary ports. If the total number of temporary ports is less than 28,232, the random port range of the OS is too small. In this case, contact the system administrator to expand the port range.</span></li><li id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_li56782353171842"><span>Run the <strong id="alm_12052__b41691811203320">ss -ant 2>/dev/null | grep -v LISTEN | awk 'NR > 2 {print $4}'|cut -d ':' -f 2 | awk '$1 >"</strong><em id="alm_12052__i1316911114336">start port number</em><strong id="alm_12052__b15170101117335">" {print $1}' | sort -u | wc -l</strong> command to calculate the number of used temporary ports.</span></li><li id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_li913603215207"><span>Calculate the usage of temporary ports using the following formula: Usage of temporary ports = (Number of used temporary ports/Total number of temporary ports) x 100. Check whether the usage exceeds the threshold.</span><p><ul class="subitemlist" id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_ul5941245215260"><li id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_li6495002115260">If yes, go to <a href="#alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_li62811777151245">8</a>.</li><li id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_li4767927915260">If no, go to <a href="#alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_li5574623151245">7</a>.</li></ul>
|
|
</p></li><li id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_li5574623151245"><a name="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_li5574623151245"></a><a name="en-us_topic_0191813905_en-us_topic_0087039394_li5574623151245"></a><span>Wait 5 minutes and check whether the alarm is cleared.</span><p><ul class="subitemlist" id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_ul24923957151245"><li id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_li48547730151245">If yes, no further action is required.</li><li id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_li40052030151245">If no, go to <a href="#alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_li62811777151245">8</a>.</li></ul>
|
|
</p></li></ol>
|
|
<p class="tableheading" id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_p22989023151245"><strong id="alm_12052__b281632113313">Check whether the system environment is normal.</strong></p>
|
|
<ol start="8" id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_ol707043215139"><li id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_li62811777151245"><a name="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_li62811777151245"></a><a name="en-us_topic_0191813905_en-us_topic_0087039394_li62811777151245"></a><span>Run the following command to import the temporary file and view the frequently used ports in the <strong id="alm_12052__b169011032133317">port_result.txt</strong> file: </span><p><p id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_p51428543103538"><strong id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_b2330955514945">netstat -tnp > $BIGDATA_HOME/tmp/port_result.txt</strong></p>
|
|
<pre class="screen" id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_screen60203709103538">netstat -tnp
|
|
|
|
Active Internet connections (w/o servers)
|
|
|
|
Proto Recv Send LocalAddress ForeignAddress State PID/ProgramName tcp 0 0 10-120-85-154:<strong id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_b33196920153428">45433</strong> 10-120-8:25009 CLOSE_WAIT 94237/java
|
|
tcp 0 0 10-120-85-154:45434 10-120-8:25009 CLOSE_WAIT 94237/java
|
|
tcp 0 0 10-120-85-154:45435 10-120-8:25009 CLOSE_WAIT 94237/java
|
|
...</pre>
|
|
</p></li><li id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_li6381664211454"><span>Run the following command to check the processes that occupy a large number of ports:</span><p><p id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_p44662294103538"><strong id="alm_12052__en-us_topic_0191813905_b1531455893614">ps -ef |grep</strong> <i><span class="varname" id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_varname14506244115822">PID</span></i></p>
|
|
<div class="note" id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_note63551870114625"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_ul4800253115836"><li id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_li57319526115839"><em id="alm_12052__i48800352333">PID</em> indicates the process ID of the port queried in <a href="#alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_li62811777151245">8</a>.</li><li id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_li33161866115836">Run the following command to collect information about all processes in the system and check the processes that occupy a large number of ports:<p id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_p66552038104152"><a name="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_li33161866115836"></a><a name="en-us_topic_0191813905_en-us_topic_0087039394_li33161866115836"></a><strong id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_b2974053141023">ps -ef > $BIGDATA_HOME/tmp/ps_result.txt</strong></p>
|
|
</li></ul>
|
|
</div></div>
|
|
</p></li><li id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_li66937120151245"><span>Contact the system administrator to clear the processes that occupy a large number of ports. Wait 5 minutes and check whether the alarm is cleared.</span><p><ul class="subitemlist" id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_ul59650203151245"><li id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_li28435089151245">If yes, no further action is required.</li><li id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_li21540881151245">If no, go to <a href="#alm_12052__en-us_topic_0191813905_li572522141314">11</a>.</li></ul>
|
|
</p></li><li id="alm_12052__en-us_topic_0191813905_li572522141314"><a name="alm_12052__en-us_topic_0191813905_li572522141314"></a><a name="en-us_topic_0191813905_li572522141314"></a><span>Collect fault information.</span><p><ol type="a" id="alm_12052__en-us_topic_0191813905_en-us_topic_0191813935_ol6089206913036"><li id="alm_12052__en-us_topic_0191813905_en-us_topic_0191813935_li4478836213036">On MRS Manager, choose <span class="menucascade" id="alm_12052__menucascade1280815043310"><b><span class="uicontrol" id="alm_12052__uicontrol1807150153318">System</span></b> > <b><span class="uicontrol" id="alm_12052__uicontrol28085503337">Export Log</span></b></span>.</li><li id="alm_12052__li18574327401">Contact technical support engineers for help. For details, see <a href="https://docs.otc.t-systems.com/en-us/public/learnmore.html" target="_blank" rel="noopener noreferrer">technical support</a>.</li></ol>
|
|
</p></li></ol>
|
|
</div>
|
|
<div class="section" id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_section45876468"><h4 class="sectiontitle">Reference</h4><p id="alm_12052__en-us_topic_0191813905_en-us_topic_0087039394_p8752394">None</p>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_0241.html">Alarm Reference (Applicable to Versions Earlier Than MRS 3.x)</a></div>
|
|
</div>
|
|
</div>
|
|
|