forked from docs/doc-exports
Yang, Tong
2195db241c
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Reviewed-by: Rechenburg, Matthias <matthias.rechenburg@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
112 lines
15 KiB
HTML
112 lines
15 KiB
HTML
<a name="admin_guide_000156"></a><a name="admin_guide_000156"></a>
|
|
|
|
<h1 class="topictitle1">Configuring Monitoring Metric Dumping</h1>
|
|
<div id="body1529658735915"><div class="section" id="admin_guide_000156__section43578247145359"><h4 class="sectiontitle">Scenario</h4><p id="admin_guide_000156__p50910430145413">The monitoring data reporting function writes the monitoring data collected in the system into a text file and uploads the file to a specified server in FTP or SFTP mode.</p>
|
|
<p id="admin_guide_000156__p25318521145422">Before using this function, you need to perform related configurations on <span id="admin_guide_000156__text15946118176">MRS</span> Manager.</p>
|
|
</div>
|
|
<div class="section" id="admin_guide_000156__section0517103210717"><h4 class="sectiontitle">Procedure</h4><ol id="admin_guide_000156__ol20084042"><li id="admin_guide_000156__en-us_topic_0046736864_li52358784"><span>Log in to <span id="admin_guide_000156__text1218018193542">MRS</span> Manager.</span></li><li id="admin_guide_000156__en-us_topic_0046736864_li1467012"><span>Choose <strong id="admin_guide_000156__b1133127165112">System</strong> > <strong id="admin_guide_000156__b17328335511">Interconnection</strong> > <strong id="admin_guide_000156__b6535144414397">Upload Performance Data</strong>.</span></li><li id="admin_guide_000156__en-us_topic_0046736864_li13203115"><span>Toggle on <strong id="admin_guide_000156__b950435717399">Upload Performance Data</strong>.</span><p><p id="admin_guide_000156__a9998d9f1221143559618399e739d3f91">The performance data upload service is disabled by default. <span><img id="admin_guide_000156__image18751859141716" src="en-us_image_0000001392414386.png"></span> indicates that the service is enabled.</p>
|
|
</p></li><li id="admin_guide_000156__li46538651"><span>Set the upload parameters according to <a href="#admin_guide_000156__table36700465">Table 1</a>.</span><p>
|
|
<div class="tablenoborder"><a name="admin_guide_000156__table36700465"></a><a name="table36700465"></a><table cellpadding="4" cellspacing="0" summary="" id="admin_guide_000156__table36700465" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Upload parameters</caption><thead align="left"><tr id="admin_guide_000156__row46348368"><th align="left" class="cellrowborder" valign="top" width="23%" id="mcps1.3.2.2.4.2.1.2.3.1.1"><p id="admin_guide_000156__p63230361"><strong id="admin_guide_000156__b1487512025415">Parameter</strong></p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="77%" id="mcps1.3.2.2.4.2.1.2.3.1.2"><p id="admin_guide_000156__p21385622"><strong id="admin_guide_000156__b573814885410">Description</strong></p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="admin_guide_000156__row20862172310518"><td class="cellrowborder" valign="top" width="23%" headers="mcps1.3.2.2.4.2.1.2.3.1.1 "><p id="admin_guide_000156__p208621237520">FTP IP Address Mode</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="77%" headers="mcps1.3.2.2.4.2.1.2.3.1.2 "><p id="admin_guide_000156__p1386217234512">Specifies the server IP address mode. This parameter is mandatory. The value can be <strong id="admin_guide_000156__b611711343815">IPV4</strong> or <strong id="admin_guide_000156__b58591744385">IPV6</strong>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="admin_guide_000156__row54513840"><td class="cellrowborder" valign="top" width="23%" headers="mcps1.3.2.2.4.2.1.2.3.1.1 "><p id="admin_guide_000156__p53544910">FTP IP Address</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="77%" headers="mcps1.3.2.2.4.2.1.2.3.1.2 "><p id="admin_guide_000156__p42170457">Specifies the IP address of the FTP server for storing monitoring files after the monitoring metric data is interconnected. This parameter is mandatory.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="admin_guide_000156__row43989794"><td class="cellrowborder" valign="top" width="23%" headers="mcps1.3.2.2.4.2.1.2.3.1.1 "><p id="admin_guide_000156__p6403579">FTP Port</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="77%" headers="mcps1.3.2.2.4.2.1.2.3.1.2 "><p id="admin_guide_000156__p48927860">Specifies the port for connecting to the FTP server. This parameter is mandatory.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="admin_guide_000156__row37697556"><td class="cellrowborder" valign="top" width="23%" headers="mcps1.3.2.2.4.2.1.2.3.1.1 "><p id="admin_guide_000156__p33603168">FTP Username</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="77%" headers="mcps1.3.2.2.4.2.1.2.3.1.2 "><p id="admin_guide_000156__p37502113">Specifies the username for logging in to the FTP server. This parameter is mandatory.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="admin_guide_000156__row1974705"><td class="cellrowborder" valign="top" width="23%" headers="mcps1.3.2.2.4.2.1.2.3.1.1 "><p id="admin_guide_000156__p25733436">FTP Password</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="77%" headers="mcps1.3.2.2.4.2.1.2.3.1.2 "><p id="admin_guide_000156__p4033584">Specifies the password for logging in to the FTP server. This parameter is mandatory.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="admin_guide_000156__row36302256"><td class="cellrowborder" valign="top" width="23%" headers="mcps1.3.2.2.4.2.1.2.3.1.1 "><p id="admin_guide_000156__p54801619">Save Path</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="77%" headers="mcps1.3.2.2.4.2.1.2.3.1.2 "><p id="admin_guide_000156__p9746163">Specifies the path for storing monitoring files on the FTP server. This parameter is mandatory.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="admin_guide_000156__row20606605"><td class="cellrowborder" valign="top" width="23%" headers="mcps1.3.2.2.4.2.1.2.3.1.1 "><p id="admin_guide_000156__p58522290">Dump Interval (second)</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="77%" headers="mcps1.3.2.2.4.2.1.2.3.1.2 "><p id="admin_guide_000156__p42685064">Specifies the interval at which monitoring files are periodically stored on the FTP server, in seconds. This parameter is mandatory.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="admin_guide_000156__row48621261"><td class="cellrowborder" valign="top" width="23%" headers="mcps1.3.2.2.4.2.1.2.3.1.1 "><p id="admin_guide_000156__p46008046">Dump Mode</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="77%" headers="mcps1.3.2.2.4.2.1.2.3.1.2 "><p id="admin_guide_000156__p35664277">Specifies the protocol used for sending monitoring files. This parameter is mandatory. The value can be <strong id="admin_guide_000156__b10982124418587">SFTP</strong> or <strong id="admin_guide_000156__b6991134835817">FTP</strong>. You are advised to use the SFTP mode based on SSH v2. Otherwise, security risks may be incurred.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="admin_guide_000156__row52543037"><td class="cellrowborder" valign="top" width="23%" headers="mcps1.3.2.2.4.2.1.2.3.1.1 "><p id="admin_guide_000156__p28127589">SFTP Service Public Key</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="77%" headers="mcps1.3.2.2.4.2.1.2.3.1.2 "><p id="admin_guide_000156__p63742213">Specifies the public key of the FTP server. This parameter is optional. It is valid only when <strong id="admin_guide_000156__b42322019145813">Dump Mode</strong> is set to <strong id="admin_guide_000156__b642810223586">SFTP</strong>.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</p></li><li id="admin_guide_000156__li36809010"><span>Click <strong id="admin_guide_000156__b12484757205812">OK</strong>.</span><p><div class="note" id="admin_guide_000156__note62845638"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p class="text" id="admin_guide_000156__p28739834">If the dump mode is SFTP and the public key of the SFTP service is empty, the system displays a security risk warning. You need to evaluate the security risk and then save the configuration.</p>
|
|
</div></div>
|
|
</p></li></ol>
|
|
</div>
|
|
<div class="section" id="admin_guide_000156__section59782784"><h4 class="sectiontitle">Data Format</h4><p id="admin_guide_000156__p46225231">After the configuration is complete, the monitoring data reporting function periodically writes monitoring data in the cluster to text files and reports the files to the corresponding FTP/SFTP service based on the configured reporting period.</p>
|
|
<ul id="admin_guide_000156__ul23719877145613"><li id="admin_guide_000156__li47031161145613">Principles for generating monitoring files<ul id="admin_guide_000156__ul37041913145634"><li id="admin_guide_000156__li9543900">The monitoring metrics are written to files generated every 30, 60, and 300 seconds based on the metric collection period.<p id="admin_guide_000156__p18786238"><a name="admin_guide_000156__li9543900"></a><a name="li9543900"></a>30s: real-time metrics that are collected every 30s by default</p>
|
|
<p id="admin_guide_000156__p34858415">60s: real-time metrics that are collected every 60s by default</p>
|
|
<p id="admin_guide_000156__p45290280">300s: all metrics that are not collected every 30s or 60s</p>
|
|
</li><li id="admin_guide_000156__li4959338">File name format: <em id="admin_guide_000156__i66162102">metric_{</em><em id="admin_guide_000156__i58588010">Interval}_{</em><em id="admin_guide_000156__i57530042">File creation time YYYYMMDDHHMMSS}.log</em><p id="admin_guide_000156__p48008337">Example: <strong id="admin_guide_000156__b14765115517496">metric_60_20160908085915.log</strong></p>
|
|
<p id="admin_guide_000156__p29421855"><strong id="admin_guide_000156__b1315045195012">metric_300_20160908085613.log</strong></p>
|
|
</li></ul>
|
|
</li><li id="admin_guide_000156__li3169475145654">Monitoring file content<ul id="admin_guide_000156__ul34360077"><li id="admin_guide_000156__li40805238">Format of monitoring files:<p id="admin_guide_000156__p31702827"><a name="admin_guide_000156__li40805238"></a><a name="li40805238"></a>"Cluster ID|Cluster name|Displayed name|Service name|Metric ID|Collection time|Collection host@m@Sub-metric|Unit|Metric value", where fields are separated using vertical bars (|). For example:</p>
|
|
<pre class="screen" id="admin_guide_000156__screen44312154339">1|xx1|Host|Host|10000413|2019/06/18 10:05:00|189-66-254-146|KB/s|309.910
|
|
1|xx1|Host|Host|10000413|2019/06/18 10:05:00|189-66-254-152|KB/s|72.870
|
|
2|xx2|Host|Host|10000413|2019/06/18 10:05:00|189-66-254-163|KB/s|100.650</pre>
|
|
<p id="admin_guide_000156__p17792161">Note: The actual files are not in that format.</p>
|
|
</li><li id="admin_guide_000156__li25911725">Interval for uploading monitoring files:<p id="admin_guide_000156__p31878936"><a name="admin_guide_000156__li25911725"></a><a name="li25911725"></a>The interval for uploading monitoring files can be set using the <strong id="admin_guide_000156__b186472381819">Dump Interval (second)</strong> parameter on the page. Currently, the interval can range from <strong id="admin_guide_000156__b578261519588">30</strong> to <strong id="admin_guide_000156__b1673611612588">300</strong>. After the configuration is complete, the system periodically uploads files to the corresponding FTP/SFTP server at the specified interval.</p>
|
|
</li></ul>
|
|
</li><li id="admin_guide_000156__li32408532145726">Monitoring metric description file<ul id="admin_guide_000156__ul8093713145747"><li id="admin_guide_000156__li20077686">Metric set file<p id="admin_guide_000156__p46481447"><a name="admin_guide_000156__li20077686"></a><a name="li20077686"></a>The metric set file <strong id="admin_guide_000156__b204304497911">all-shown-metric-zh_CN</strong> contains detailed information about all metrics. After obtaining the metric IDs from the files reported by the third-party system, you can query details about the metrics from the metric set file.</p>
|
|
<p id="admin_guide_000156__p15679845">Location of the metric set file:</p>
|
|
<p id="admin_guide_000156__p6900880">Active and standby OMS nodes: {<em id="admin_guide_000156__i2317131771117"><span id="admin_guide_000156__text156568322235">MRS</span> installation path</em>} <strong id="admin_guide_000156__b12238162210119">/om-server/om/etc/om/all-shown-metric-zh_CN</strong></p>
|
|
<p id="admin_guide_000156__p62107921">Content of the metric set file:</p>
|
|
<pre class="screen" id="admin_guide_000156__screen22100377">Real-Time Metric ID,5-Minute Metric ID,Metric Name,Metric Collection Period (s),Collected by Default,Service Belonged To,Role Belonged To
|
|
00101,10000101,JobHistoryServer non-heap memory usage,30,false,Mapreduce,JobHistoryServer
|
|
00102,10000102,JobHistoryServer non-heap memory allocation volume,30,false,Mapreduce,JobHistoryServer
|
|
00103,10000103,JobHistoryServer heap memory usage,30,false,Mapreduce,JobHistoryServer
|
|
00104,10000104,JobHistoryServer heap memory allocation volume,30,false,Mapreduce,JobHistoryServer
|
|
00105,10000105,Number of blocked threads,30,false,Mapreduce,JobHistoryServer
|
|
00106,10000106,Number of running threads,30,false,Mapreduce,JobHistoryServer
|
|
00107,10000107,GC time,30,false,Mapreduce,JobHistoryServer
|
|
00110,10000110,JobHistoryServer CPU usage,30,false,Mapreduce,JobHistoryServer
|
|
...</pre>
|
|
</li><li id="admin_guide_000156__li64685665">Field description of critical metrics<p id="admin_guide_000156__p45300074"><a name="admin_guide_000156__li64685665"></a><a name="li64685665"></a><strong id="admin_guide_000156__b1496710398165">Real-Time Metric ID</strong>: indicates the ID of the metric whose collection period is 30s or 60s.</p>
|
|
<p id="admin_guide_000156__p45427372"><strong id="admin_guide_000156__b2261735122010">5-Minute Metric ID</strong>: indicates the ID of a 5-minute (300s) metric.</p>
|
|
<p id="admin_guide_000156__p55738502"><strong id="admin_guide_000156__b234714432218">Metric Collection Period (s)</strong>: indicates the collection period of real-time metrics. The value can be <strong id="admin_guide_000156__b1626332618234">30</strong> or <strong id="admin_guide_000156__b1019820292234">60</strong>.</p>
|
|
<p id="admin_guide_000156__p18524833"><strong id="admin_guide_000156__b1022511451237">Service Belonged To</strong>: indicates the name of the service to which a metric belongs, for example, HDFS and HBase.</p>
|
|
<p id="admin_guide_000156__p24116517"><strong id="admin_guide_000156__b186831817264">Role Belonged To</strong>: indicates the name of the role to which a metric belongs, for example, JobServer and RegionServer.</p>
|
|
</li><li id="admin_guide_000156__li7280824">Description<p id="admin_guide_000156__p65527424"><a name="admin_guide_000156__li7280824"></a><a name="li7280824"></a>For metrics whose collection period is 30s/60s, you can find the corresponding metric description by referring to the first column, that is, <strong id="admin_guide_000156__b1894071752">Real-Time Metric ID</strong>.</p>
|
|
<p id="admin_guide_000156__p6121136">For metrics whose collection period is 300s, you can find the corresponding metric description by referring to the second column, that is, <strong id="admin_guide_000156__b222806555">5-Minute Metric ID</strong>.</p>
|
|
</li></ul>
|
|
</li></ul>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="admin_guide_000153.html">Configuring Interconnections</a></div>
|
|
</div>
|
|
</div>
|
|
|