forked from docs/doc-exports
chenxiaoxiong
f9e2808b7c
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: chenxiaoxiong <chenxiaoxiong@huawei.com> Co-committed-by: chenxiaoxiong <chenxiaoxiong@huawei.com>
136 lines
20 KiB
HTML
136 lines
20 KiB
HTML
<a name="dataartsstudio_01_4504"></a><a name="dataartsstudio_01_4504"></a>
|
|
|
|
<h1 class="topictitle1">Python</h1>
|
|
<div id="body0000001135993245"><div class="notice" id="dataartsstudio_01_4504__note344131135112"><span class="noticetitle"><img src="public_sys-resources/notice_3.0-en-us.png"> </span><div class="noticebody"><p id="dataartsstudio_01_4504__p24543118511">Before using a Python node, ensure that the host connected to the node has an environment for executing Python scripts.</p>
|
|
</div></div>
|
|
<div class="section" id="dataartsstudio_01_4504__section1329216425362"><h4 class="sectiontitle">Functions</h4><p id="dataartsstudio_01_4504__p6400174173718">The Python node is used to execute Python statements.</p>
|
|
<p id="dataartsstudio_01_4504__p34038442255">For details about how to use the Python node, see <a href="dataartsstudio_01_0529.html">Developing a Python Job</a>.</p>
|
|
<div class="note" id="dataartsstudio_01_4504__note133681831413"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_4504__p3625236162611">Python nodes support script parameters and job parameters.</p>
|
|
</div></div>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_01_4504__section7394398377"><h4 class="sectiontitle">Parameters</h4><p id="dataartsstudio_01_4504__p1742910459373"><a href="#dataartsstudio_01_4504__en-us_topic_0113842761_table3764823994826">Table 1</a> and <a href="#dataartsstudio_01_4504__table1396374719116">Table 2</a> describe the parameters of the Python node.</p>
|
|
|
|
<div class="tablenoborder"><a name="dataartsstudio_01_4504__en-us_topic_0113842761_table3764823994826"></a><a name="en-us_topic_0113842761_table3764823994826"></a><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_01_4504__en-us_topic_0113842761_table3764823994826" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameters of the Python node</caption><thead align="left"><tr id="dataartsstudio_01_4504__en-us_topic_0113842761_row3170822394826"><th align="left" class="cellrowborder" valign="top" width="21.42%" id="mcps1.3.3.3.2.4.1.1"><p id="dataartsstudio_01_4504__en-us_topic_0113842761_p2984581994826">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="10.13%" id="mcps1.3.3.3.2.4.1.2"><p id="dataartsstudio_01_4504__en-us_topic_0113842761_p159227094826">Mandatory</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="68.45%" id="mcps1.3.3.3.2.4.1.3"><p id="dataartsstudio_01_4504__en-us_topic_0113842761_p6186505494826">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dataartsstudio_01_4504__en-us_topic_0113842761_row3614415394826"><td class="cellrowborder" valign="top" width="21.42%" headers="mcps1.3.3.3.2.4.1.1 "><p id="dataartsstudio_01_4504__en-us_topic_0113842761_p1225314550306">Python Statement or Script</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.13%" headers="mcps1.3.3.3.2.4.1.2 "><p id="dataartsstudio_01_4504__en-us_topic_0113842761_p4617707494826">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="68.45%" headers="mcps1.3.3.3.2.4.1.3 "><p id="dataartsstudio_01_4504__en-us_topic_0113842761_p61023369712">You can select <strong id="dataartsstudio_01_4504__b7710102481910">Python statement</strong> or <strong id="dataartsstudio_01_4504__b7715122411917">Python script</strong>.</p>
|
|
<ul id="dataartsstudio_01_4504__en-us_topic_0113842761_ul1857020154912"><li id="dataartsstudio_01_4504__en-us_topic_0113842761_li1757317196913">Python statement<p id="dataartsstudio_01_4504__en-us_topic_0113842761_p74181856897"><a name="dataartsstudio_01_4504__en-us_topic_0113842761_li1757317196913"></a><a name="en-us_topic_0113842761_li1757317196913"></a>Click the text box under <span class="parmname" id="dataartsstudio_01_4504__en-us_topic_0113842761_parmname116422231331"><b>Python Statement</b></span>. In the displayed <span class="wintitle" id="dataartsstudio_01_4504__en-us_topic_0113842761_wintitle28741019181110"><b>Python Statement</b></span> dialog box, enter the Python statement to be executed.</p>
|
|
</li><li id="dataartsstudio_01_4504__en-us_topic_0113842761_li10570115593">Python script<p id="dataartsstudio_01_4504__en-us_topic_0113842761_p310193912912"><a name="dataartsstudio_01_4504__en-us_topic_0113842761_li10570115593"></a><a name="en-us_topic_0113842761_li10570115593"></a>In <strong id="dataartsstudio_01_4504__b126201314155218">Python Script</strong>, select the Python script to be executed. The Python version is displayed by default, for example, <strong id="dataartsstudio_01_4504__b1062091465210">Python3</strong>. If no script is available, create and develop a script by referring to <a href="dataartsstudio_01_0423.html">Creating a Script</a> and <a href="dataartsstudio_01_4503.html">Developing a Python Script</a>.</p>
|
|
<div class="note" id="dataartsstudio_01_4504__en-us_topic_0113842761_note15529154994813"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dataartsstudio_01_4504__ul18191183193116"><li id="dataartsstudio_01_4504__li111918317318">If you select <strong id="dataartsstudio_01_4504__b187501037112017">Python statement</strong>, the DataArts Factory module cannot parse the parameters contained in the Python statement.</li><li id="dataartsstudio_01_4504__li619123133119">If you select <strong id="dataartsstudio_01_4504__b8438156808">Python script</strong>, the system displays the Python version selected during Python script creation by default.</li><li id="dataartsstudio_01_4504__li8751151513350">For existing jobs, <strong id="dataartsstudio_01_4504__b1284164318527">Python2</strong> is used by default.</li><li id="dataartsstudio_01_4504__li7201184833711">The execution result of a Python node cannot be larger than 30 MB. Otherwise, an error is reported.</li></ul>
|
|
</div></div>
|
|
</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_4504__row16123141615518"><td class="cellrowborder" valign="top" width="21.42%" headers="mcps1.3.3.3.2.4.1.1 "><p id="dataartsstudio_01_4504__p123762216510">Host Connection</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.13%" headers="mcps1.3.3.3.2.4.1.2 "><p id="dataartsstudio_01_4504__p212441612515">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="68.45%" headers="mcps1.3.3.3.2.4.1.3 "><p id="dataartsstudio_01_4504__p1124716052">Select the host where the Python statement is to be executed. Ensure that the host has an environment for executing Python scripts.</p>
|
|
<div class="notice" id="dataartsstudio_01_4504__note9347626175612"><span class="noticetitle"> NOTICE: </span><div class="noticebody"><ul id="dataartsstudio_01_4504__en-us_topic_0000001503446532_ul1732011438531"><li id="dataartsstudio_01_4504__en-us_topic_0000001503446532_li132074355319">The maximum number of shell or Python scripts that can run concurrently on the ECS is determined by the value of <strong id="dataartsstudio_01_4504__en-us_topic_0000001503446532_b744231917314">MaxSessions</strong> in the <strong id="dataartsstudio_01_4504__en-us_topic_0000001503446532_b18801517103811">/etc/ssh/sshd_config</strong> file on the ECS. Set <strong id="dataartsstudio_01_4504__en-us_topic_0000001503446532_b1357831103217">MaxSessions</strong> based on the scheduling frequency of shell or Python scripts.</li><li id="dataartsstudio_01_4504__en-us_topic_0000001503446532_li632094385319">You have the permission to create and execute files in the <strong id="dataartsstudio_01_4504__en-us_topic_0000001503446532_b84586912349">/tmp</strong> directory on the host.</li><li id="dataartsstudio_01_4504__en-us_topic_0000001503446532_li2595194372315">Shell and Python scripts are executed in the <strong id="dataartsstudio_01_4504__en-us_topic_0000001503446532_b17727195642316">/tmp</strong> directory on an ECS. Ensure that the disk space of the <strong id="dataartsstudio_01_4504__en-us_topic_0000001503446532_b15640427112413">/tmp</strong> directory is not used up.</li></ul>
|
|
</div></div>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_4504__row1246614521252"><td class="cellrowborder" valign="top" width="21.42%" headers="mcps1.3.3.3.2.4.1.1 "><p id="dataartsstudio_01_4504__p154660528513">Script Parameters</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.13%" headers="mcps1.3.3.3.2.4.1.2 "><p id="dataartsstudio_01_4504__p1846610521056">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="68.45%" headers="mcps1.3.3.3.2.4.1.3 "><p id="dataartsstudio_01_4504__p1846645219518">Parameter transferred to the script when the Python statement is executed. Parameters are separated by spaces. For example: <strong id="dataartsstudio_01_4504__b458453313510">a b c</strong>. The parameter must be referenced by the Python statement. Otherwise, the parameter is invalid.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_4504__row63090551455"><td class="cellrowborder" valign="top" width="21.42%" headers="mcps1.3.3.3.2.4.1.1 "><p id="dataartsstudio_01_4504__p1419619411610">Interactive Input</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.13%" headers="mcps1.3.3.3.2.4.1.2 "><p id="dataartsstudio_01_4504__p73092556515">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="68.45%" headers="mcps1.3.3.3.2.4.1.3 "><p id="dataartsstudio_01_4504__p1030919551653">Interactive information (passwords, for example) provided during Python statement execution. Interactive parameters are separated by spaces. The Python statement reads parameter values in sequence according to the interaction situation.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_4504__en-us_topic_0113842761_row173916552814"><td class="cellrowborder" valign="top" width="21.42%" headers="mcps1.3.3.3.2.4.1.1 "><p id="dataartsstudio_01_4504__en-us_topic_0113842761_p246794194826">Node Name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.13%" headers="mcps1.3.3.3.2.4.1.2 "><p id="dataartsstudio_01_4504__en-us_topic_0113842761_p6568554794826">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="68.45%" headers="mcps1.3.3.3.2.4.1.3 "><p id="dataartsstudio_01_4504__p14740112265412">Name of the node. The value must consist of 1 to 128 characters and contain only letters, digits, and the following special characters: _-/<>.</p>
|
|
<p id="dataartsstudio_01_4504__p356348122213">By default, the node name is the same as that of the selected script. If you want the node name to be different from the script name, disable this function by referring to <a href="dataartsstudio_01_04501.html#dataartsstudio_01_04501__section67661828112219">Disabling Auto Node Name Change</a>.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
|
|
<div class="tablenoborder"><a name="dataartsstudio_01_4504__table1396374719116"></a><a name="table1396374719116"></a><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_01_4504__table1396374719116" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Advanced parameters</caption><thead align="left"><tr id="dataartsstudio_01_4504__en-us_topic_0099822521_row27216578102411"><th align="left" class="cellrowborder" valign="top" width="28.07%" id="mcps1.3.3.4.2.4.1.1"><p id="dataartsstudio_01_4504__en-us_topic_0099822521_p57059205102411">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="15.659999999999998%" id="mcps1.3.3.4.2.4.1.2"><p id="dataartsstudio_01_4504__en-us_topic_0099822521_p58392901102411">Mandatory</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="56.269999999999996%" id="mcps1.3.3.4.2.4.1.3"><p id="dataartsstudio_01_4504__en-us_topic_0099822521_p32204521102411">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dataartsstudio_01_4504__en-us_topic_0099822521_row51612113175"><td class="cellrowborder" valign="top" width="28.07%" headers="mcps1.3.3.4.2.4.1.1 "><p id="dataartsstudio_01_4504__en-us_topic_0099822521_p416115112178">Node Status Polling Interval (s)</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.659999999999998%" headers="mcps1.3.3.4.2.4.1.2 "><p id="dataartsstudio_01_4504__en-us_topic_0099822521_p101615110176">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.269999999999996%" headers="mcps1.3.3.4.2.4.1.3 "><p id="dataartsstudio_01_4504__en-us_topic_0099822521_p4161191101716">How often the system check completeness of the <span id="dataartsstudio_01_4504__en-us_topic_0099822521_text1526241235118">node</span>. The value ranges from 1 to 60 seconds.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_4504__en-us_topic_0099822521_row5101045193916"><td class="cellrowborder" valign="top" width="28.07%" headers="mcps1.3.3.4.2.4.1.1 "><p id="dataartsstudio_01_4504__en-us_topic_0099822521_p147314419397">Max. Node Execution Duration</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.659999999999998%" headers="mcps1.3.3.4.2.4.1.2 "><p id="dataartsstudio_01_4504__en-us_topic_0099822521_p610124511390">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.269999999999996%" headers="mcps1.3.3.4.2.4.1.3 "><p id="dataartsstudio_01_4504__en-us_topic_0099822521_p11011456393">Execution timeout interval for the <span id="dataartsstudio_01_4504__en-us_topic_0099822521_text6930195917115">node</span>. If retry is configured and the execution is not complete within the timeout interval, the <span id="dataartsstudio_01_4504__en-us_topic_0099822521_text2930205914118">node</span> will be executed again.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_4504__en-us_topic_0099822521_row58429402102411"><td class="cellrowborder" valign="top" width="28.07%" headers="mcps1.3.3.4.2.4.1.1 "><p id="dataartsstudio_01_4504__en-us_topic_0099822521_p5533912102858">Retry upon Failure</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.659999999999998%" headers="mcps1.3.3.4.2.4.1.2 "><p id="dataartsstudio_01_4504__en-us_topic_0099822521_p45593742102858">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.269999999999996%" headers="mcps1.3.3.4.2.4.1.3 "><p id="dataartsstudio_01_4504__en-us_topic_0099822521_p2105628102858">Whether to re-execute a <span id="dataartsstudio_01_4504__en-us_topic_0099822521_text29185571161243">node</span> if it fails to be executed. Possible values:</p>
|
|
<ul id="dataartsstudio_01_4504__en-us_topic_0099822521_ul18950660102858"><li id="dataartsstudio_01_4504__en-us_topic_0099822521_li36338219102858"><strong id="dataartsstudio_01_4504__en-us_topic_0099822521_b67992121590">Yes</strong>: The <span id="dataartsstudio_01_4504__en-us_topic_0099822521_text19139245161248">node</span> will be re-executed, and the following parameters must be configured:<ul id="dataartsstudio_01_4504__en-us_topic_0099822521_ul58608523102858"><li id="dataartsstudio_01_4504__en-us_topic_0099822521_li17370115484319"><strong id="dataartsstudio_01_4504__en-us_topic_0099822521_b18610151101714">Retry upon Timeout</strong></li><li id="dataartsstudio_01_4504__en-us_topic_0099822521_li44375787102858"><strong id="dataartsstudio_01_4504__en-us_topic_0099822521_b18404183512913">Maximum Retries</strong></li><li id="dataartsstudio_01_4504__en-us_topic_0099822521_li63837764102858"><strong id="dataartsstudio_01_4504__en-us_topic_0099822521_b1484216373917">Retry Interval (seconds)</strong></li></ul>
|
|
</li><li id="dataartsstudio_01_4504__en-us_topic_0099822521_li37668971102858"><strong id="dataartsstudio_01_4504__en-us_topic_0099822521_b0109154414913">No</strong>: The <span id="dataartsstudio_01_4504__en-us_topic_0099822521_text1328324161254">node</span> will not be re-executed. This is the default setting.<div class="note" id="dataartsstudio_01_4504__en-us_topic_0099822521_note845010501289"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_4504__en-us_topic_0099822521_p194511050132816">If retry is configured for a job node and the timeout duration is configured, the system allows you to retry a node when the node execution times out.</p>
|
|
<p id="dataartsstudio_01_4504__en-us_topic_0099822521_p1313284812516">If a node is not re-executed when it fails upon timeout, you can go to the <strong id="dataartsstudio_01_4504__en-us_topic_0099822521_b4476194112483">Default Configuration</strong> page to modify this policy.</p>
|
|
<p id="dataartsstudio_01_4504__en-us_topic_0099822521_p15946155074317"><strong id="dataartsstudio_01_4504__en-us_topic_0099822521_b128994567172">Retry upon Timeout</strong> is displayed only when <strong id="dataartsstudio_01_4504__en-us_topic_0099822521_b7899656131716">Retry upon Failure</strong> is set to <strong id="dataartsstudio_01_4504__en-us_topic_0099822521_b1899155620179">Yes</strong>.</p>
|
|
</div></div>
|
|
</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_4504__en-us_topic_0099822521_row29541959102411"><td class="cellrowborder" valign="top" width="28.07%" headers="mcps1.3.3.4.2.4.1.1 "><p id="dataartsstudio_01_4504__en-us_topic_0099822521_p13154928102858">Policy for Handling Subsequent Nodes If the Current Node Fails</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.659999999999998%" headers="mcps1.3.3.4.2.4.1.2 "><p id="dataartsstudio_01_4504__en-us_topic_0099822521_p58916261102858">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.269999999999996%" headers="mcps1.3.3.4.2.4.1.3 "><p id="dataartsstudio_01_4504__en-us_topic_0099822521_p7487822102858">Operation that will be performed if the <span id="dataartsstudio_01_4504__en-us_topic_0099822521_text5371194616130">node</span> fails to be executed. Possible values:</p>
|
|
<ul id="dataartsstudio_01_4504__en-us_topic_0099822521_ul281538102858"><li id="dataartsstudio_01_4504__en-us_topic_0099822521_li867222192212"><strong id="dataartsstudio_01_4504__en-us_topic_0099822521_b63511334183">Suspend execution plans of the subsequent nodes</strong>: stops running subsequent nodes. The job instance status is <strong id="dataartsstudio_01_4504__en-us_topic_0099822521_b635117371819">Failed</strong>.</li><li id="dataartsstudio_01_4504__en-us_topic_0099822521_li2533844102858"><strong id="dataartsstudio_01_4504__en-us_topic_0099822521_b31833716587">End the current job execution plan</strong>: stops running the current job. The job instance status is <strong id="dataartsstudio_01_4504__en-us_topic_0099822521_b1521193765820">Failed</strong>.</li><li id="dataartsstudio_01_4504__en-us_topic_0099822521_li22804597102858"><strong id="dataartsstudio_01_4504__en-us_topic_0099822521_b72771223151916">Go to the next node</strong>: ignores the execution failure of the current node. The job instance status is <strong id="dataartsstudio_01_4504__en-us_topic_0099822521_b1727710236199">Failure ignored</strong>.</li><li id="dataartsstudio_01_4504__en-us_topic_0099822521_li1657745411173"><strong id="dataartsstudio_01_4504__en-us_topic_0099822521_b99581526191817">Suspend the current job execution plan</strong>: If the current job instance is in abnormal state, the subsequent nodes of this node and the subsequent job instances that depend on the current job are in waiting state.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_4504__en-us_topic_0099822521_row230711409176"><td class="cellrowborder" valign="top" width="28.07%" headers="mcps1.3.3.4.2.4.1.1 "><p id="dataartsstudio_01_4504__en-us_topic_0099822521_p143081740131711">Enable Dry Run</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.659999999999998%" headers="mcps1.3.3.4.2.4.1.2 "><p id="dataartsstudio_01_4504__en-us_topic_0099822521_p030844011714">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.269999999999996%" headers="mcps1.3.3.4.2.4.1.3 "><p id="dataartsstudio_01_4504__en-us_topic_0099822521_p19929151141017">If you select this option, the node will not be executed, and a success message will be returned.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_4504__en-us_topic_0099822521_row116622012182617"><td class="cellrowborder" valign="top" width="28.07%" headers="mcps1.3.3.4.2.4.1.1 "><p id="dataartsstudio_01_4504__en-us_topic_0099822521_p1336551322619">Task Groups</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.659999999999998%" headers="mcps1.3.3.4.2.4.1.2 "><p id="dataartsstudio_01_4504__en-us_topic_0099822521_p7365181320261">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.269999999999996%" headers="mcps1.3.3.4.2.4.1.3 "><p id="dataartsstudio_01_4504__en-us_topic_0099822521_p236512139269">Select a task group. If you select a task group, you can control the maximum number of concurrent nodes in the task group in a fine-grained manner in scenarios where a job contains multiple nodes, a data patching task is ongoing, or a job is rerunning.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0441.html">Node Reference</a></div>
|
|
</div>
|
|
</div>
|
|
|