forked from docs/doc-exports
chenxiaoxiong
f9e2808b7c
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: chenxiaoxiong <chenxiaoxiong@huawei.com> Co-committed-by: chenxiaoxiong <chenxiaoxiong@huawei.com>
141 lines
18 KiB
HTML
141 lines
18 KiB
HTML
<a name="dataartsstudio_01_0457"></a><a name="dataartsstudio_01_0457"></a>
|
|
|
|
<h1 class="topictitle1">MRS MapReduce</h1>
|
|
<div id="body8662426"><div class="section" id="dataartsstudio_01_0457__en-us_topic_0101095234_section44280035173841"><h4 class="sectiontitle">Functions</h4><p id="dataartsstudio_01_0457__en-us_topic_0101095234_p3249577917391">The <span id="dataartsstudio_01_0457__en-us_topic_0101095234_text26719383105149">MRS MapReduce</span> node is used to execute a predefined MapReduce program on MRS.</p>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_01_0457__en-us_topic_0101095234_section1791764173933"><h4 class="sectiontitle">Parameters</h4><p id="dataartsstudio_01_0457__en-us_topic_0101095234_p49252448103256"><a href="#dataartsstudio_01_0457__en-us_topic_0101095234_table3764823994826">Table 1</a> and <a href="#dataartsstudio_01_0457__en-us_topic_0101095234_table58040457102411">Table 2</a> describe the parameters of the <span id="dataartsstudio_01_0457__en-us_topic_0101095234_text73811053165419">MRS MapReduce</span> node.</p>
|
|
|
|
<div class="tablenoborder"><a name="dataartsstudio_01_0457__en-us_topic_0101095234_table3764823994826"></a><a name="en-us_topic_0101095234_table3764823994826"></a><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_01_0457__en-us_topic_0101095234_table3764823994826" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameters of <span id="dataartsstudio_01_0457__text1442104042">MRS MapReduce</span> nodes</caption><thead align="left"><tr id="dataartsstudio_01_0457__en-us_topic_0101095234_row3170822394826"><th align="left" class="cellrowborder" valign="top" width="21.62%" id="mcps1.3.2.3.2.4.1.1"><p id="dataartsstudio_01_0457__en-us_topic_0101095234_p2984581994826">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="10.27%" id="mcps1.3.2.3.2.4.1.2"><p id="dataartsstudio_01_0457__en-us_topic_0101095234_p159227094826">Mandatory</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="68.11%" id="mcps1.3.2.3.2.4.1.3"><p id="dataartsstudio_01_0457__en-us_topic_0101095234_p6186505494826">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dataartsstudio_01_0457__en-us_topic_0101095234_row1991457694826"><td class="cellrowborder" valign="top" width="21.62%" headers="mcps1.3.2.3.2.4.1.1 "><p id="dataartsstudio_01_0457__en-us_topic_0101095234_p246794194826">Node Name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.27%" headers="mcps1.3.2.3.2.4.1.2 "><p id="dataartsstudio_01_0457__en-us_topic_0101095234_p6568554794826">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="68.11%" headers="mcps1.3.2.3.2.4.1.3 "><p id="dataartsstudio_01_0457__en-us_topic_0101095437_p1892909794826">Name of a <span id="dataartsstudio_01_0457__en-us_topic_0099822521_text44323307153939">node</span>. The name must contain 1 to 128 characters, including only letters, numbers, underscores (_), hyphens (-), slashes (/), less-than signs (<), and greater-than signs (>).</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0457__en-us_topic_0101095234_row3614415394826"><td class="cellrowborder" valign="top" width="21.62%" headers="mcps1.3.2.3.2.4.1.1 "><p id="dataartsstudio_01_0457__en-us_topic_0101095234_p4199531294826">MRS Cluster Name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.27%" headers="mcps1.3.2.3.2.4.1.2 "><p id="dataartsstudio_01_0457__en-us_topic_0101095234_p4617707494826">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="68.11%" headers="mcps1.3.2.3.2.4.1.3 "><p id="dataartsstudio_01_0457__en-us_topic_0101095234_p4935552994826">Name of the MRS cluster.</p>
|
|
<div class="p" id="dataartsstudio_01_0457__en-us_topic_0101095234_p14106064105658">To create an MRS cluster, use either of the following methods:<ul id="dataartsstudio_01_0457__en-us_topic_0101095234_en-us_topic_0099822521_ul46080727105259"><li id="dataartsstudio_01_0457__en-us_topic_0101095234_en-us_topic_0099822521_li12073360105259">Click <span><img id="dataartsstudio_01_0457__en-us_topic_0101095234_image898618712510" src="en-us_image_0000002234085912.png"></span>. On the <span class="wintitle" id="dataartsstudio_01_0457__en-us_topic_0101095234_wintitle3126196153112"><b>Clusters</b></span> page, create an MRS cluster.</li><li id="dataartsstudio_01_0457__en-us_topic_0101095234_en-us_topic_0099822521_li24857298105259">Go to the MRS console to create an MRS cluster.</li></ul>
|
|
</div>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0457__en-us_topic_0101095234_row4154658494826"><td class="cellrowborder" valign="top" width="21.62%" headers="mcps1.3.2.3.2.4.1.1 "><p id="dataartsstudio_01_0457__en-us_topic_0101095234_p983015394826">MapReduce Job Name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.27%" headers="mcps1.3.2.3.2.4.1.2 "><p id="dataartsstudio_01_0457__en-us_topic_0101095234_p5804493094826">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="68.11%" headers="mcps1.3.2.3.2.4.1.3 "><p id="dataartsstudio_01_0457__en-us_topic_0177038589_p15602016115115">MRS job name. It can contain a maximum of 64 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed.</p>
|
|
<div class="note" id="dataartsstudio_01_0457__note3401731183519"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0457__en-us_topic_0181380727_p4169117193320">The job name cannot contain Chinese characters or more than 64 characters. If the job name does not meet requirements, the MRS job will fail to be submitted.</p>
|
|
</div></div>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0457__en-us_topic_0101095234_row303485991116"><td class="cellrowborder" valign="top" width="21.62%" headers="mcps1.3.2.3.2.4.1.1 "><p id="dataartsstudio_01_0457__en-us_topic_0101095234_p423174491116">JAR Package</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.27%" headers="mcps1.3.2.3.2.4.1.2 "><p id="dataartsstudio_01_0457__en-us_topic_0101095234_p51613311116">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="68.11%" headers="mcps1.3.2.3.2.4.1.3 "><p id="dataartsstudio_01_0457__en-us_topic_0101095234_p154146861116">Select a JAR package. Before selecting a JAR package, upload the JAR package to the OBS bucket, create a resource on the <span class="wintitle" id="dataartsstudio_01_0457__wintitle9738174974511"><b>Manage Resource</b></span> page, and add the JAR package to the resource management list. For details, see <a href="dataartsstudio_01_0519.html#dataartsstudio_01_0519__en-us_topic_0165312432_section6325757145320">Creating a Resource</a>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0457__en-us_topic_0101095234_row430706211158"><td class="cellrowborder" valign="top" width="21.62%" headers="mcps1.3.2.3.2.4.1.1 "><p id="dataartsstudio_01_0457__en-us_topic_0101095234_p1332771411158">JAR File Parameters</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.27%" headers="mcps1.3.2.3.2.4.1.2 "><p id="dataartsstudio_01_0457__en-us_topic_0101095234_p580307511158">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="68.11%" headers="mcps1.3.2.3.2.4.1.3 "><p id="dataartsstudio_01_0457__en-us_topic_0101095234_p28710511158">Parameters of the JAR package.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0457__en-us_topic_0101095234_row4695054111155"><td class="cellrowborder" valign="top" width="21.62%" headers="mcps1.3.2.3.2.4.1.1 "><p id="dataartsstudio_01_0457__en-us_topic_0101095234_p4489744011155">Input Data Path</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.27%" headers="mcps1.3.2.3.2.4.1.2 "><p id="dataartsstudio_01_0457__en-us_topic_0101095234_p1281406111155">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="68.11%" headers="mcps1.3.2.3.2.4.1.3 "><p id="dataartsstudio_01_0457__en-us_topic_0101095234_p3130601011155">Path where the input data resides.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0457__en-us_topic_0101095234_row633704411335"><td class="cellrowborder" valign="top" width="21.62%" headers="mcps1.3.2.3.2.4.1.1 "><p id="dataartsstudio_01_0457__en-us_topic_0101095234_p4353855211335">Output Data Path</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.27%" headers="mcps1.3.2.3.2.4.1.2 "><p id="dataartsstudio_01_0457__en-us_topic_0101095234_p3696185311335">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="68.11%" headers="mcps1.3.2.3.2.4.1.3 "><p id="dataartsstudio_01_0457__en-us_topic_0101095234_p4112013911335">Path where the output data resides.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
|
|
<div class="tablenoborder"><a name="dataartsstudio_01_0457__en-us_topic_0101095234_table58040457102411"></a><a name="en-us_topic_0101095234_table58040457102411"></a><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_01_0457__en-us_topic_0101095234_table58040457102411" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Advanced parameters</caption><thead align="left"><tr id="dataartsstudio_01_0457__en-us_topic_0099822521_row27216578102411"><th align="left" class="cellrowborder" valign="top" width="28.07%" id="mcps1.3.2.4.2.4.1.1"><p id="dataartsstudio_01_0457__en-us_topic_0099822521_p57059205102411">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="15.659999999999998%" id="mcps1.3.2.4.2.4.1.2"><p id="dataartsstudio_01_0457__en-us_topic_0099822521_p58392901102411">Mandatory</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="56.269999999999996%" id="mcps1.3.2.4.2.4.1.3"><p id="dataartsstudio_01_0457__en-us_topic_0099822521_p32204521102411">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dataartsstudio_01_0457__en-us_topic_0099822521_row51612113175"><td class="cellrowborder" valign="top" width="28.07%" headers="mcps1.3.2.4.2.4.1.1 "><p id="dataartsstudio_01_0457__en-us_topic_0099822521_p416115112178">Node Status Polling Interval (s)</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.659999999999998%" headers="mcps1.3.2.4.2.4.1.2 "><p id="dataartsstudio_01_0457__en-us_topic_0099822521_p101615110176">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.269999999999996%" headers="mcps1.3.2.4.2.4.1.3 "><p id="dataartsstudio_01_0457__en-us_topic_0099822521_p4161191101716">How often the system check completeness of the <span id="dataartsstudio_01_0457__en-us_topic_0099822521_text1526241235118">node</span>. The value ranges from 1 to 60 seconds.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0457__en-us_topic_0099822521_row5101045193916"><td class="cellrowborder" valign="top" width="28.07%" headers="mcps1.3.2.4.2.4.1.1 "><p id="dataartsstudio_01_0457__en-us_topic_0099822521_p147314419397">Max. Node Execution Duration</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.659999999999998%" headers="mcps1.3.2.4.2.4.1.2 "><p id="dataartsstudio_01_0457__en-us_topic_0099822521_p610124511390">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.269999999999996%" headers="mcps1.3.2.4.2.4.1.3 "><p id="dataartsstudio_01_0457__en-us_topic_0099822521_p11011456393">Execution timeout interval for the <span id="dataartsstudio_01_0457__en-us_topic_0099822521_text6930195917115">node</span>. If retry is configured and the execution is not complete within the timeout interval, the <span id="dataartsstudio_01_0457__en-us_topic_0099822521_text2930205914118">node</span> will be executed again.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0457__en-us_topic_0099822521_row58429402102411"><td class="cellrowborder" valign="top" width="28.07%" headers="mcps1.3.2.4.2.4.1.1 "><p id="dataartsstudio_01_0457__en-us_topic_0099822521_p5533912102858">Retry upon Failure</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.659999999999998%" headers="mcps1.3.2.4.2.4.1.2 "><p id="dataartsstudio_01_0457__en-us_topic_0099822521_p45593742102858">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.269999999999996%" headers="mcps1.3.2.4.2.4.1.3 "><p id="dataartsstudio_01_0457__en-us_topic_0099822521_p2105628102858">Whether to re-execute a <span id="dataartsstudio_01_0457__en-us_topic_0099822521_text29185571161243">node</span> if it fails to be executed. Possible values:</p>
|
|
<ul id="dataartsstudio_01_0457__en-us_topic_0099822521_ul18950660102858"><li id="dataartsstudio_01_0457__en-us_topic_0099822521_li36338219102858"><strong id="dataartsstudio_01_0457__en-us_topic_0099822521_b67992121590">Yes</strong>: The <span id="dataartsstudio_01_0457__en-us_topic_0099822521_text19139245161248">node</span> will be re-executed, and the following parameters must be configured:<ul id="dataartsstudio_01_0457__en-us_topic_0099822521_ul58608523102858"><li id="dataartsstudio_01_0457__en-us_topic_0099822521_li17370115484319"><strong id="dataartsstudio_01_0457__en-us_topic_0099822521_b18610151101714">Retry upon Timeout</strong></li><li id="dataartsstudio_01_0457__en-us_topic_0099822521_li44375787102858"><strong id="dataartsstudio_01_0457__en-us_topic_0099822521_b18404183512913">Maximum Retries</strong></li><li id="dataartsstudio_01_0457__en-us_topic_0099822521_li63837764102858"><strong id="dataartsstudio_01_0457__en-us_topic_0099822521_b1484216373917">Retry Interval (seconds)</strong></li></ul>
|
|
</li><li id="dataartsstudio_01_0457__en-us_topic_0099822521_li37668971102858"><strong id="dataartsstudio_01_0457__en-us_topic_0099822521_b0109154414913">No</strong>: The <span id="dataartsstudio_01_0457__en-us_topic_0099822521_text1328324161254">node</span> will not be re-executed. This is the default setting.<div class="note" id="dataartsstudio_01_0457__en-us_topic_0099822521_note845010501289"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0457__en-us_topic_0099822521_p194511050132816">If retry is configured for a job node and the timeout duration is configured, the system allows you to retry a node when the node execution times out.</p>
|
|
<p id="dataartsstudio_01_0457__en-us_topic_0099822521_p1313284812516">If a node is not re-executed when it fails upon timeout, you can go to the <strong id="dataartsstudio_01_0457__en-us_topic_0099822521_b4476194112483">Default Configuration</strong> page to modify this policy.</p>
|
|
<p id="dataartsstudio_01_0457__en-us_topic_0099822521_p15946155074317"><strong id="dataartsstudio_01_0457__en-us_topic_0099822521_b128994567172">Retry upon Timeout</strong> is displayed only when <strong id="dataartsstudio_01_0457__en-us_topic_0099822521_b7899656131716">Retry upon Failure</strong> is set to <strong id="dataartsstudio_01_0457__en-us_topic_0099822521_b1899155620179">Yes</strong>.</p>
|
|
</div></div>
|
|
</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0457__en-us_topic_0099822521_row29541959102411"><td class="cellrowborder" valign="top" width="28.07%" headers="mcps1.3.2.4.2.4.1.1 "><p id="dataartsstudio_01_0457__en-us_topic_0099822521_p13154928102858">Policy for Handling Subsequent Nodes If the Current Node Fails</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.659999999999998%" headers="mcps1.3.2.4.2.4.1.2 "><p id="dataartsstudio_01_0457__en-us_topic_0099822521_p58916261102858">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.269999999999996%" headers="mcps1.3.2.4.2.4.1.3 "><p id="dataartsstudio_01_0457__en-us_topic_0099822521_p7487822102858">Operation that will be performed if the <span id="dataartsstudio_01_0457__en-us_topic_0099822521_text5371194616130">node</span> fails to be executed. Possible values:</p>
|
|
<ul id="dataartsstudio_01_0457__en-us_topic_0099822521_ul281538102858"><li id="dataartsstudio_01_0457__en-us_topic_0099822521_li867222192212"><strong id="dataartsstudio_01_0457__en-us_topic_0099822521_b63511334183">Suspend execution plans of the subsequent nodes</strong>: stops running subsequent nodes. The job instance status is <strong id="dataartsstudio_01_0457__en-us_topic_0099822521_b635117371819">Failed</strong>.</li><li id="dataartsstudio_01_0457__en-us_topic_0099822521_li2533844102858"><strong id="dataartsstudio_01_0457__en-us_topic_0099822521_b31833716587">End the current job execution plan</strong>: stops running the current job. The job instance status is <strong id="dataartsstudio_01_0457__en-us_topic_0099822521_b1521193765820">Failed</strong>.</li><li id="dataartsstudio_01_0457__en-us_topic_0099822521_li22804597102858"><strong id="dataartsstudio_01_0457__en-us_topic_0099822521_b72771223151916">Go to the next node</strong>: ignores the execution failure of the current node. The job instance status is <strong id="dataartsstudio_01_0457__en-us_topic_0099822521_b1727710236199">Failure ignored</strong>.</li><li id="dataartsstudio_01_0457__en-us_topic_0099822521_li1657745411173"><strong id="dataartsstudio_01_0457__en-us_topic_0099822521_b99581526191817">Suspend the current job execution plan</strong>: If the current job instance is in abnormal state, the subsequent nodes of this node and the subsequent job instances that depend on the current job are in waiting state.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0457__en-us_topic_0099822521_row230711409176"><td class="cellrowborder" valign="top" width="28.07%" headers="mcps1.3.2.4.2.4.1.1 "><p id="dataartsstudio_01_0457__en-us_topic_0099822521_p143081740131711">Enable Dry Run</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.659999999999998%" headers="mcps1.3.2.4.2.4.1.2 "><p id="dataartsstudio_01_0457__en-us_topic_0099822521_p030844011714">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.269999999999996%" headers="mcps1.3.2.4.2.4.1.3 "><p id="dataartsstudio_01_0457__en-us_topic_0099822521_p19929151141017">If you select this option, the node will not be executed, and a success message will be returned.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0457__en-us_topic_0099822521_row116622012182617"><td class="cellrowborder" valign="top" width="28.07%" headers="mcps1.3.2.4.2.4.1.1 "><p id="dataartsstudio_01_0457__en-us_topic_0099822521_p1336551322619">Task Groups</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.659999999999998%" headers="mcps1.3.2.4.2.4.1.2 "><p id="dataartsstudio_01_0457__en-us_topic_0099822521_p7365181320261">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.269999999999996%" headers="mcps1.3.2.4.2.4.1.3 "><p id="dataartsstudio_01_0457__en-us_topic_0099822521_p236512139269">Select a task group. If you select a task group, you can control the maximum number of concurrent nodes in the task group in a fine-grained manner in scenarios where a job contains multiple nodes, a data patching task is ongoing, or a job is rerunning.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0441.html">Node Reference</a></div>
|
|
</div>
|
|
</div>
|
|
|