forked from docs/doc-exports
chenxiaoxiong
f9e2808b7c
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: chenxiaoxiong <chenxiaoxiong@huawei.com> Co-committed-by: chenxiaoxiong <chenxiaoxiong@huawei.com>
134 lines
29 KiB
HTML
134 lines
29 KiB
HTML
<a name="dataartsstudio_01_0584"></a><a name="dataartsstudio_01_0584"></a>
|
|
|
|
<h1 class="topictitle1">Obtaining the Output of an SQL Node</h1>
|
|
<div id="body0000001487872810"><p id="dataartsstudio_01_0584__en-us_topic_0000001162343901_en-us_topic_0000001073100734_p2567186349">This section describes how to obtain the output of an SQL node and apply the output to subsequent nodes or judgment in job development.</p>
|
|
<div class="section" id="dataartsstudio_01_0584__en-us_topic_0000001115624138_en-us_topic_0284272823_section6114621155317"><h4 class="sectiontitle">Scenario</h4><p id="dataartsstudio_01_0584__p196541123172010">When you use EL expression <strong id="dataartsstudio_01_0584__b6148130184316">#{Job.getNodeOutput("Name of the previous node")}</strong> to obtain the output of the previous node, the output is a two-dimensional array, for example, <strong id="dataartsstudio_01_0584__b1924313139336">[["Dean",...,"08"],...,["Smith",...,"53"]]</strong>. To obtain the values in the array, use either of the methods provided in <a href="#dataartsstudio_01_0584__table8346195375220">Table 1</a>.</p>
|
|
|
|
<div class="tablenoborder"><a name="dataartsstudio_01_0584__table8346195375220"></a><a name="table8346195375220"></a><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_01_0584__table8346195375220" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Methods for obtaining output values</caption><thead align="left"><tr id="dataartsstudio_01_0584__row93461853125218"><th align="left" class="cellrowborder" valign="top" width="13.458654134586542%" id="mcps1.3.2.3.2.4.1.1"><p id="dataartsstudio_01_0584__p1034665335214">Method</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="45.625437456254375%" id="mcps1.3.2.3.2.4.1.2"><p id="dataartsstudio_01_0584__p296814311927">Key Configuration</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="40.91590840915909%" id="mcps1.3.2.3.2.4.1.3"><p id="dataartsstudio_01_0584__p8346175395210">Application Scenario Requirements</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dataartsstudio_01_0584__row10347115395217"><td class="cellrowborder" valign="top" width="13.458654134586542%" headers="mcps1.3.2.3.2.4.1.1 "><p id="dataartsstudio_01_0584__p434712537526"><a href="#dataartsstudio_01_0584__section14854144844313">Obtaining Output Value Using StringUtil</a></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="45.625437456254375%" headers="mcps1.3.2.3.2.4.1.2 "><p id="dataartsstudio_01_0584__p1896955415416">If the output of the SQL node contains only one field, for example <strong id="dataartsstudio_01_0584__b13966418103620">[["11"]]</strong>, you can use the StringUtil EL expression with an embedded object to split the two-dimensional array and obtain the field value in the output of the previous node.</p>
|
|
<p id="dataartsstudio_01_0584__p940411412213">#{StringUtil.split(StringUtil.split(StringUtil.split(Job.getNodeOutput("<em id="dataartsstudio_01_0584__i909618710"><strong id="dataartsstudio_01_0584__b10519193271">Name of the previous node</strong></em>"),"]")[0],"[")[0],"\\"")[0]}</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.91590840915909%" headers="mcps1.3.2.3.2.4.1.3 "><p id="dataartsstudio_01_0584__p2012213504171">This method is easy to use but has the following requirements on application scenarios:</p>
|
|
<ul id="dataartsstudio_01_0584__ul14492413015"><li id="dataartsstudio_01_0584__li64491415019">The output of the previous SQL node contains only one field, for example, <strong id="dataartsstudio_01_0584__b7811817184216">[["11"]]</strong>.</li><li id="dataartsstudio_01_0584__li7449144206">The output value is a string. The application scenario must support the string data type. For example, if the IF condition needs to be used to judge the size of the output value, the string type is not supported. In this case, this method cannot be used.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0584__row11347453145216"><td class="cellrowborder" valign="top" width="13.458654134586542%" headers="mcps1.3.2.3.2.4.1.1 "><p id="dataartsstudio_01_0584__p11347153105210"><a href="#dataartsstudio_01_0584__section23728149243">Obtaining Output Values Using the For Each Node</a></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="45.625437456254375%" headers="mcps1.3.2.3.2.4.1.2 "><p id="dataartsstudio_01_0584__p3372122216315">Use the For Each node to cyclically obtain the values in the two-dimensional array in the dataset.</p>
|
|
<ul id="dataartsstudio_01_0584__ul14772135510312"><li id="dataartsstudio_01_0584__li1477212558314">For Each node dataset: #{Job.getNodeOutput('<em id="dataartsstudio_01_0584__i38442011179"><strong id="dataartsstudio_01_0584__b198441091874">Name of the previous node</strong></em>')}</li><li id="dataartsstudio_01_0584__li1877375513320">Subjob parameters of the For Each node: #{Loop.current[<em id="dataartsstudio_01_0584__i88420374"><strong id="dataartsstudio_01_0584__b999415181175">Index</strong></em>]}</li></ul>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.91590840915909%" headers="mcps1.3.2.3.2.4.1.3 "><p id="dataartsstudio_01_0584__p115171217142718">This method is applicable to more scenarios, though jobs need to be split into main jobs and subjobs.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_01_0584__section14854144844313"><a name="dataartsstudio_01_0584__section14854144844313"></a><a name="section14854144844313"></a><h4 class="sectiontitle">Obtaining Output Value Using StringUtil</h4><p id="dataartsstudio_01_0584__p15854154834313"><strong id="dataartsstudio_01_0584__b085464813430">Scenario</strong></p>
|
|
<p id="dataartsstudio_01_0584__p15480756428">The StringUtil EL expression with an embedded object is used to split the two-dimensional array result and obtain the output field value of the previous node, which is a string.</p>
|
|
<div class="p" id="dataartsstudio_01_0584__p1817053214519">In this example, the MRS Hive SQL node returns a two-dimensional array that contains a single field. The data sent by the Kafka Client node is defined as the StringUtil EL expression with an embedded object. You can use this expression to split the two-dimensional array and obtain the output field value of the MRS Hive SQL node.<div class="note" id="dataartsstudio_01_0584__note119881363511"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_0584__p3988123612518">To make it easy to view the obtained value, this example uses the Kafka Client node. In practice, you can select a subsequent node type as needed. By using a StringUtil EL expression with an embedded object on the node, you can obtain the data value returned by the previous node.</p>
|
|
</div></div>
|
|
</div>
|
|
<div class="fignone" id="dataartsstudio_01_0584__fig1785564894317"><a name="dataartsstudio_01_0584__fig1785564894317"></a><a name="fig1785564894317"></a><span class="figcap"><b>Figure 1 </b>Example job</span><br><span><img id="dataartsstudio_01_0584__image11741162710101" src="en-us_image_0000002269116113.png"></span></div>
|
|
<div class="p" id="dataartsstudio_01_0584__p178559482437">The key configuration of the Kafka Client node is the <strong id="dataartsstudio_01_0584__b17753747312">Sent Content</strong> parameter. Set it as follows:<pre class="screen" id="dataartsstudio_01_0584__screen487416134292">#{StringUtil.split(StringUtil.split(StringUtil.split(Job.getNodeOutput("count95"),"]")[0],"[")[0],"\\"")[0]}</pre>
|
|
</div>
|
|
<p id="dataartsstudio_01_0584__p385864811439"><strong id="dataartsstudio_01_0584__b1585817482433">Configuration Method</strong></p>
|
|
<ol id="dataartsstudio_01_0584__ol1858648184310"><li id="dataartsstudio_01_0584__li1285864814433"><span>Log in to the <span id="dataartsstudio_01_0584__text15816130436">DataArts Studio</span> console, locate the target <span id="dataartsstudio_01_0584__text981619301937">DataArts Studio</span> instance, and click <strong id="dataartsstudio_01_0584__b12816630630">Access</strong> on the instance card.</span></li><li id="dataartsstudio_01_0584__li18858194811436"><span>Click the <strong id="dataartsstudio_01_0584__b5591936437">Workspaces</strong> tab. In the workspace list, locate the target workspace and click <strong id="dataartsstudio_01_0584__b25973616316">DataArts Factory</strong>.</span></li><li id="dataartsstudio_01_0584__li1755810515125"><span>Create table <strong id="dataartsstudio_01_0584__b16537811416">student_score</strong>. Create a temporary Hive SQL script, select a Hive connection and database, paste the following SQL statement, and run the script. After the script is successfully executed, delete it.</span><p><pre class="screen" id="dataartsstudio_01_0584__screen1532654781414">CREATE TABLE `student_score` (`name` String COMMENT '', `score` INT COMMENT '');
|
|
INSERT INTO
|
|
student_score
|
|
VALUES
|
|
('ZHAO', '90'),
|
|
('QIAN', '88'),
|
|
('SUN', '93'),
|
|
('LI', '94'),
|
|
('ZHOU', '85'),
|
|
('WU', '79'),
|
|
('ZHENG', '87'),
|
|
('WANG', '97'),
|
|
('FENG', '83'),
|
|
('CEHN', '99');</pre>
|
|
</p></li><li id="dataartsstudio_01_0584__li19298135016154"><a name="dataartsstudio_01_0584__li19298135016154"></a><a name="li19298135016154"></a><span>Create the Hive SQL script to be invoked by the MRS Hive SQL node. Create a Hive SQL script named <strong id="dataartsstudio_01_0584__b10791258667">count95</strong>, select a Hive connection and database, paste the following SQL statement, and submit a version.</span><p><pre class="screen" id="dataartsstudio_01_0584__screen1687164311718">--Obtain the number of students whose scores are higher than 95 from the student_score table.--
|
|
SELECT count(*) FROM student_score WHERE score> "95" ; </pre>
|
|
</p></li><li id="dataartsstudio_01_0584__li1085914834318"><span>On the <strong id="dataartsstudio_01_0584__b13352331115620">Develop Job</strong> page, create a data development job. Drag an MRS Hive SQL node and a Kafka Client node and drop them on the canvas. Click and hold <span><img id="dataartsstudio_01_0584__image285914488437" src="en-us_image_0000002234236776.png"></span> to connect the nodes, as shown in <a href="#dataartsstudio_01_0584__fig1785564894317">Figure 1</a>.</span></li><li id="dataartsstudio_01_0584__li138591348124319"><span>Configuring parameters for an MRS Hive SQL node Select the <strong id="dataartsstudio_01_0584__b1518105116917">count95</strong> script submitted in <a href="#dataartsstudio_01_0584__li19298135016154">4</a> for <strong id="dataartsstudio_01_0584__b115781311108">SQL script</strong> and select a Hive connection and database.</span><p><div class="fignone" id="dataartsstudio_01_0584__fig1486119482438"><span class="figcap"><b>Figure 2 </b>Configuring parameters for an MRS Hive SQL node</span><br><span><img id="dataartsstudio_01_0584__image434513214255" src="en-us_image_0000002269196257.png" title="Click to enlarge" class="imgResize"></span></div>
|
|
<p id="dataartsstudio_01_0584__p7144124912324"></p>
|
|
</p></li><li id="dataartsstudio_01_0584__li122011320102513"><span>Configure parameters for the Kafka Client node. Set <strong id="dataartsstudio_01_0584__b19641145771118">Sent Content</strong> to <strong id="dataartsstudio_01_0584__b03521277127">#{StringUtil.split(StringUtil.split(StringUtil.split(Job.getNodeOutput("count95"),"]")[0],"[")[0],"\\"")[0]}</strong> and select a Kafka connection and a topic name.</span><p><div class="fignone" id="dataartsstudio_01_0584__fig620192052511"><span class="figcap"><b>Figure 3 </b>Configuring parameters for the Kafka Client node</span><br><span><img id="dataartsstudio_01_0584__image75873513262" src="en-us_image_0000002234077004.png" title="Click to enlarge" class="imgResize"></span></div>
|
|
<p id="dataartsstudio_01_0584__p85691018113310"></p>
|
|
</p></li><li id="dataartsstudio_01_0584__li7862164884314"><span>After the node configuration is complete, click <strong id="dataartsstudio_01_0584__b18800335191617">Test</strong>. After the job test is successful, right-click the Kafka Client node to view its log. You can find that the two-dimensional array <strong id="dataartsstudio_01_0584__b5583820193615">[["2"]]</strong> returned by the MRS Hive SQL node has been converted to <strong id="dataartsstudio_01_0584__b229903310311">2</strong>.</span><p><div class="note" id="dataartsstudio_01_0584__note840519405364"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_0584__p19406174010361">You can set <strong id="dataartsstudio_01_0584__b179119111818">Sent Content</strong> of the Kafka Client node to <strong id="dataartsstudio_01_0584__b060472733713">#{Job.getNodeOutput("count95")}</strong> and run the job. Then you can view the log of the Kafka Client node to verify that the result returned by the MRS Hive SQL node is two-dimensional array <strong id="dataartsstudio_01_0584__b1775202973818">[["2"]]</strong>.</p>
|
|
</div></div>
|
|
<div class="fignone" id="dataartsstudio_01_0584__fig85342517309"><span class="figcap"><b>Figure 4 </b>Check the Kafka Client node logs.</span><br><span><img id="dataartsstudio_01_0584__image29131815321" src="en-us_image_0000002269196209.png" title="Click to enlarge" class="imgResize"></span></div>
|
|
<p id="dataartsstudio_01_0584__p177944914335"></p>
|
|
</p></li></ol>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_01_0584__section23728149243"><a name="dataartsstudio_01_0584__section23728149243"></a><a name="section23728149243"></a><h4 class="sectiontitle">Obtaining Output Values Using the For Each Node</h4><p id="dataartsstudio_01_0584__en-us_topic_0000001162343901_p48551004391"><strong id="dataartsstudio_01_0584__en-us_topic_0000001162343901_b48556012395">Scenario</strong></p>
|
|
<p id="dataartsstudio_01_0584__en-us_topic_0000001162343901_p1836117710377">You can use the For Each node and the EL expression #{Loop.current[0]} with a Loop embedded object to cyclically obtain the output values of the previous node.</p>
|
|
<div class="p" id="dataartsstudio_01_0584__p18709123119403">In this example, the MRS Hive SQL node returns a two-dimensional array that contains multiple fields. You can use the For Each node which cyclically invokes the subjobs of the Kafka Client node and set <strong id="dataartsstudio_01_0584__b1190471433214">Sent Content</strong> of the Kafka Client node to <strong id="dataartsstudio_01_0584__b11643229143213">#{Loop.current[]}</strong> to obtain the output values of the MRS Hive SQL node.<div class="note" id="dataartsstudio_01_0584__note1671017319408"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_0584__p171053194018">To make it easy to view the obtained values, this example uses the Kafka Client node as the subjob node of the For Each node. In practice, you can select a subjob node type as needed. By using an EL expression with an embedded Loop object on the node, you can obtain the values returned by the previous node of the For Each node.</p>
|
|
</div></div>
|
|
</div>
|
|
<div class="p" id="dataartsstudio_01_0584__p1078327131712">Orchestrate the main job shown in <a href="#dataartsstudio_01_0584__en-us_topic_0000001162343901_fig1639792911135">Figure 5</a>. Key configurations of the For Each node are as follows:<ul id="dataartsstudio_01_0584__en-us_topic_0000001162343901_ul13874125811393"><li id="dataartsstudio_01_0584__en-us_topic_0000001162343901_li487418587394"><strong id="dataartsstudio_01_0584__b1883428988102727">Dataset</strong>: Enter the execution result of the select statement on the Hive SQL node. Use the <strong id="dataartsstudio_01_0584__b1317552119573">#{Job.getNodeOutput("select95")}</strong> expression, where <strong id="dataartsstudio_01_0584__b1817662145713">select95</strong> is the name of the previous node.</li><li id="dataartsstudio_01_0584__en-us_topic_0000001162343901_li2508102204013"><strong id="dataartsstudio_01_0584__b975744322102727">Subjob Parameter Name</strong>: Enter the parameter name defined in the subjob. Transfer the parameter value defined in the main job to the subjob. Set the subjob parameter names to <strong id="dataartsstudio_01_0584__b094181121214">name</strong> and <strong id="dataartsstudio_01_0584__b68774816122">score</strong>, whose values are those in the first and second columns in the dataset, respectively. EL expressions <strong id="dataartsstudio_01_0584__b172905516134">#{Loop.current[0]}</strong> and <strong id="dataartsstudio_01_0584__b1549717546139">#{Loop.current[1]}</strong> are used.</li></ul>
|
|
</div>
|
|
<div class="fignone" id="dataartsstudio_01_0584__en-us_topic_0000001162343901_fig1639792911135"><a name="dataartsstudio_01_0584__en-us_topic_0000001162343901_fig1639792911135"></a><a name="en-us_topic_0000001162343901_fig1639792911135"></a><span class="figcap"><b>Figure 5 </b>Example main job</span><br><span><img id="dataartsstudio_01_0584__image940918519107" src="en-us_image_0000002269116181.png" title="Click to enlarge" class="imgResize"></span></div>
|
|
<p id="dataartsstudio_01_0584__en-us_topic_0000001162343901_p1039742920134">For the subjobs selected for the For Each node, you must set their parameter names so that the main job can identify the parameter definitions.</p>
|
|
<div class="fignone" id="dataartsstudio_01_0584__en-us_topic_0000001162343901_fig517111111225"><a name="dataartsstudio_01_0584__en-us_topic_0000001162343901_fig517111111225"></a><a name="en-us_topic_0000001162343901_fig517111111225"></a><span class="figcap"><b>Figure 6 </b>Example subjob</span><br><span><img id="dataartsstudio_01_0584__image3363165591515" src="en-us_image_0000002234236804.png" title="Click to enlarge" class="imgResize"></span></div>
|
|
<p id="dataartsstudio_01_0584__p1881832712465"></p>
|
|
<p id="dataartsstudio_01_0584__en-us_topic_0000001162343901_p86132236281"><strong id="dataartsstudio_01_0584__en-us_topic_0000001162343901_b132611216205">Configuration Method</strong></p>
|
|
<p id="dataartsstudio_01_0584__en-us_topic_0000001162343901_p937792514371">Developing a Subjob</p>
|
|
<ol id="dataartsstudio_01_0584__en-us_topic_0000001162343901_ol12198621103719"><li id="dataartsstudio_01_0584__en-us_topic_0000001162343901_li193978297137"><span>Log in to the <span id="dataartsstudio_01_0584__en-us_topic_0000001487872810_text15816130436">DataArts Studio</span> console, locate the target <span id="dataartsstudio_01_0584__en-us_topic_0000001487872810_text981619301937">DataArts Studio</span> instance, and click <strong id="dataartsstudio_01_0584__en-us_topic_0000001487872810_b12816630630">Access</strong> on the instance card.</span></li><li id="dataartsstudio_01_0584__en-us_topic_0000001162343901_li7198421163711"><span>Click the <strong id="dataartsstudio_01_0584__en-us_topic_0000001487872810_b5591936437">Workspaces</strong> tab. In the workspace list, locate the target workspace and click <strong id="dataartsstudio_01_0584__en-us_topic_0000001487872810_b25973616316">DataArts Factory</strong>.</span></li><li id="dataartsstudio_01_0584__li880619553241"><span>On the <strong id="dataartsstudio_01_0584__b1220591643102727">Develop Job</strong> page, create a data development subjob named <strong id="dataartsstudio_01_0584__b715720806102727">EL_test_slave</strong>. Select a Kafka Client node, configure job parameters, and orchestrate the job shown in <a href="#dataartsstudio_01_0584__en-us_topic_0000001162343901_fig517111111225">Figure 6</a>.</span><p><p id="dataartsstudio_01_0584__p1328155616246">Set the parameter name to <strong id="dataartsstudio_01_0584__b1030816186249">name</strong> and <strong id="dataartsstudio_01_0584__b8308131815242">score</strong>. This parameter is only used by the For Each node in the main job to identify subjob parameters. You do not need to set the parameter value.</p>
|
|
</p></li><li id="dataartsstudio_01_0584__li17149122752514"><span>Configure parameters for the Kafka Client node. Set <strong id="dataartsstudio_01_0584__b83641058144815">Sent Content</strong> to <strong id="dataartsstudio_01_0584__b1669515414253">${name}: ${score}</strong> and select a Kafka connection and a topic name.</span><p><div class="note" id="dataartsstudio_01_0584__note661562111261"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_0584__en-us_topic_0000001208493414_en-us_topic_0000001162343901_p16691216153">Do not use the <strong id="dataartsstudio_01_0584__en-us_topic_0000001208493414_en-us_topic_0000001162343901_b1137342111119">#{Job.getParam("job_param_name")}</strong> EL expression because this expression can only obtain the values of the parameters configured in the current job, but cannot obtain the parameter values transferred from the parent job or the global variables configured in the workspace. The expression only works for the current job.</p>
|
|
<p id="dataartsstudio_01_0584__en-us_topic_0000001208493414_en-us_topic_0000001162343901_p85351447162819">To obtain the parameter values passed from the parent job and the global variables configured for the workspace, you are advised to use the <strong id="dataartsstudio_01_0584__en-us_topic_0000001208493414_en-us_topic_0000001162343901_b7413133218157">${job_param_name}</strong> expression.</p>
|
|
</div></div>
|
|
<div class="fignone" id="dataartsstudio_01_0584__fig10149102762516"><span class="figcap"><b>Figure 7 </b>Configuring parameters for the Kafka Client node</span><br><span><img id="dataartsstudio_01_0584__image145161910192714" src="en-us_image_0000002234076984.png" title="Click to enlarge" class="imgResize"></span></div>
|
|
<p id="dataartsstudio_01_0584__p14149112792520"></p>
|
|
</p></li><li id="dataartsstudio_01_0584__en-us_topic_0000001162343901_li12781624154514"><span>Submit the subjob after the configuration is complete.</span></li></ol>
|
|
</div>
|
|
<p id="dataartsstudio_01_0584__p3666131316226">Developing a Main Job</p>
|
|
<ol id="dataartsstudio_01_0584__en-us_topic_0000001162343901_ol091619185382"><li id="dataartsstudio_01_0584__li10518295309"><span>Go to the <strong id="dataartsstudio_01_0584__b547519403417">Develop Script</strong> page.</span></li><li id="dataartsstudio_01_0584__li1975917241298"><span>Create table <strong id="dataartsstudio_01_0584__en-us_topic_0000001487872810_b16537811416">student_score</strong>. Create a temporary Hive SQL script, select a Hive connection and database, paste the following SQL statement, and run the script. After the script is successfully executed, delete it.</span><p><pre class="screen" id="dataartsstudio_01_0584__en-us_topic_0000001487872810_screen1532654781414">CREATE TABLE `student_score` (`name` String COMMENT '', `score` INT COMMENT '');
|
|
INSERT INTO
|
|
student_score
|
|
VALUES
|
|
('ZHAO', '90'),
|
|
('QIAN', '88'),
|
|
('SUN', '93'),
|
|
('LI', '94'),
|
|
('ZHOU', '85'),
|
|
('WU', '79'),
|
|
('ZHENG', '87'),
|
|
('WANG', '97'),
|
|
('FENG', '83'),
|
|
('CEHN', '99');</pre>
|
|
</p></li><li id="dataartsstudio_01_0584__li16477457163015"><a name="dataartsstudio_01_0584__li16477457163015"></a><a name="li16477457163015"></a><span>Create the Hive SQL script to be invoked by the MRS Hive SQL node. Create a Hive SQL script named <strong id="dataartsstudio_01_0584__b76761629418">select95</strong>, select a Hive connection and database, paste the following SQL statement, and submit a version.</span><p><pre class="screen" id="dataartsstudio_01_0584__screen18867623143114">--Display the names and scores of students whose scores are higher than 95 in the student_score table.--
|
|
SELECT * FROM student_score WHERE score> "95" ; </pre>
|
|
</p></li><li id="dataartsstudio_01_0584__en-us_topic_0000001162343901_li19198102193716"><span>On the <strong id="dataartsstudio_01_0584__b1470163516594">Develop Job</strong> page, create a data development job named <strong id="dataartsstudio_01_0584__b2470143511594">EL_test_master</strong>. Drag a HIVE SQL node and a For Each node and drop them on the canvas. Click and hold <span><img id="dataartsstudio_01_0584__en-us_topic_0000001162343901_image549216431659" src="en-us_image_0000002234076960.png"></span> to connect the nodes, as shown in <a href="#dataartsstudio_01_0584__en-us_topic_0000001162343901_fig1639792911135">Figure 5</a>.</span></li><li id="dataartsstudio_01_0584__li7797173419332"><span>Configure parameters for the MRS Hive SQL node. Select the <strong id="dataartsstudio_01_0584__b171623115106">select95</strong> script submitted in <a href="#dataartsstudio_01_0584__li16477457163015">3</a> for <strong id="dataartsstudio_01_0584__b15716631161013">SQL script</strong> and select a Hive connection and database.</span><p><div class="fignone" id="dataartsstudio_01_0584__fig07971034173317"><span class="figcap"><b>Figure 8 </b>Configuring parameters for an MRS Hive SQL node</span><br><span><img id="dataartsstudio_01_0584__image152101615357" src="en-us_image_0000002269116161.png" title="Click to enlarge" class="imgResize"></span></div>
|
|
<p id="dataartsstudio_01_0584__p9798193483317"></p>
|
|
</p></li><li id="dataartsstudio_01_0584__en-us_topic_0000001162343901_li18850173864018"><span>Configure properties for the For Each node.</span><p><ul id="dataartsstudio_01_0584__en-us_topic_0000001162343901_ul473126194116"><li id="dataartsstudio_01_0584__en-us_topic_0000001162343901_li1728281115419"><strong id="dataartsstudio_01_0584__b178541020803">Subjob in a Loop</strong>: Select <strong id="dataartsstudio_01_0584__b58544208013">EL_test_slave</strong>, the subjob that has been developed.</li><li id="dataartsstudio_01_0584__en-us_topic_0000001162343901_li67311361414"><strong id="dataartsstudio_01_0584__en-us_topic_0000001487872810_b1883428988102727">Dataset</strong>: Enter the execution result of the select statement on the Hive SQL node. Use the <strong id="dataartsstudio_01_0584__en-us_topic_0000001487872810_b1317552119573">#{Job.getNodeOutput("select95")}</strong> expression, where <strong id="dataartsstudio_01_0584__en-us_topic_0000001487872810_b1817662145713">select95</strong> is the name of the previous node.</li><li id="dataartsstudio_01_0584__en-us_topic_0000001162343901_li1273119674118"><strong id="dataartsstudio_01_0584__en-us_topic_0000001487872810_b975744322102727">Subjob Parameter Name</strong>: Enter the parameter name defined in the subjob. Transfer the parameter value defined in the main job to the subjob. Set the subjob parameter names to <strong id="dataartsstudio_01_0584__en-us_topic_0000001487872810_b094181121214">name</strong> and <strong id="dataartsstudio_01_0584__en-us_topic_0000001487872810_b68774816122">score</strong>, whose values are those in the first and second columns in the dataset, respectively. EL expressions <strong id="dataartsstudio_01_0584__en-us_topic_0000001487872810_b172905516134">#{Loop.current[0]}</strong> and <strong id="dataartsstudio_01_0584__en-us_topic_0000001487872810_b1549717546139">#{Loop.current[1]}</strong> are used.</li></ul>
|
|
<div class="fignone" id="dataartsstudio_01_0584__en-us_topic_0000001162343901_fig176439478428"><span class="figcap"><b>Figure 9 </b>Configuring properties for the For Each node</span><br><span><img id="dataartsstudio_01_0584__image1319113409362" src="en-us_image_0000002269116149.png" title="Click to enlarge" class="imgResize"></span></div>
|
|
<p id="dataartsstudio_01_0584__p1565003914812"></p>
|
|
</p></li><li id="dataartsstudio_01_0584__en-us_topic_0000001162343901_li15453913457"><span>Save the job.</span></li></ol>
|
|
<p id="dataartsstudio_01_0584__en-us_topic_0000001162343901_p116422040164216">Testing the Main Job</p>
|
|
<ol id="dataartsstudio_01_0584__en-us_topic_0000001162343901_ol33632059124213"><li id="dataartsstudio_01_0584__en-us_topic_0000001162343901_li55861024114310"><span>Click <strong id="dataartsstudio_01_0584__b5361107151012">Test</strong> above the main job <strong id="dataartsstudio_01_0584__b93613711107">EL_test_master</strong> canvas to test the job. After the main job is executed, the subjob <strong id="dataartsstudio_01_0584__b1050853215397">EL_test_slave</strong> is cyclically invoked through the For Each node and executed.</span></li><li id="dataartsstudio_01_0584__en-us_topic_0000001162343901_li036495914429"><span>In the navigation pane on the left, choose <strong id="dataartsstudio_01_0584__b11850710171014">Monitor Instance</strong> to view the job execution result.</span></li><li id="dataartsstudio_01_0584__en-us_topic_0000001162343901_li2364659144211"><span>After the job is executed, view the cyclic execution result of the subjob <strong id="dataartsstudio_01_0584__b21045194117">EL_test_slave</strong> on the <strong id="dataartsstudio_01_0584__b88367218115">Monitor Instance</strong> page.</span><p><div class="fignone" id="dataartsstudio_01_0584__en-us_topic_0000001162343901_fig330818168499"><span class="figcap"><b>Figure 10 </b>Execution result of the subjob</span><br><span><img id="dataartsstudio_01_0584__image864043911406" src="en-us_image_0000002234076948.png" title="Click to enlarge" class="imgResize"></span></div>
|
|
<p id="dataartsstudio_01_0584__p158751527511"></p>
|
|
</p></li><li id="dataartsstudio_01_0584__li1622110467415"><span>View the log of the cyclic execution of subjob <strong id="dataartsstudio_01_0584__b145221752184111">EL_test_slave</strong>. The log shows that the output values of the previous node of the For Each node was obtained through the For Each node and the EL expression with a Loop embedded object.</span><p><div class="fignone" id="dataartsstudio_01_0584__fig11583174312425"><span class="figcap"><b>Figure 11 </b>Viewing the log</span><br><span><img id="dataartsstudio_01_0584__image583742011434" src="en-us_image_0000002269116221.png" title="Click to enlarge" class="imgResize"></span></div>
|
|
<p id="dataartsstudio_01_0584__p13521174713511"></p>
|
|
</p></li></ol>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0520.html">Usage Guidance</a></div>
|
|
</div>
|
|
</div>
|
|
|
|
|
|
<script language="JavaScript">
|
|
<!--
|
|
initImageViewer('.imgResize');
|
|
var msg_imageMax = "view original image";
|
|
var msg_imageClose = "close";
|
|
//--></script> |