forked from docs/doc-exports
chenxiaoxiong
f9e2808b7c
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: chenxiaoxiong <chenxiaoxiong@huawei.com> Co-committed-by: chenxiaoxiong <chenxiaoxiong@huawei.com>
632 lines
101 KiB
HTML
632 lines
101 KiB
HTML
<a name="dataartsstudio_01_1436"></a><a name="dataartsstudio_01_1436"></a>
|
|
|
|
<h1 class="topictitle1">Developing a Batch Processing Single-Task SQL Job</h1>
|
|
<div id="body0000001423148402"><p id="dataartsstudio_01_1436__en-us_topic_0099797007_p16460513273">This section describes how to develop and configure a <span id="dataartsstudio_01_1436__text539175713472">job</span>.</p>
|
|
<p id="dataartsstudio_01_1436__p1642415366591">For details about how to develop a batch processing job in single-task mode, see sections <a href="#dataartsstudio_01_1436__section037832452814">Developing an SQL Script</a>, <a href="#dataartsstudio_01_1436__en-us_topic_0099797007_section754991272419">Configuring job parameters</a>, <a href="#dataartsstudio_01_1436__section17103142782618">Data Table</a>, <a href="#dataartsstudio_01_1436__section1462324142616">Testing and Saving the Job</a>, and <a href="#dataartsstudio_01_1436__section2246103584414">Downloading or Dumping a Script Execution Result</a>.</p>
|
|
<div class="section" id="dataartsstudio_01_1436__en-us_topic_0099797007_section536330414847"><h4 class="sectiontitle">Prerequisites</h4><ul id="dataartsstudio_01_1436__ul1836451315143"><li id="dataartsstudio_01_1436__li733054512146">A job has been created. For details, see <a href="dataartsstudio_01_0434.html">Creating a Job</a>.</li><li id="dataartsstudio_01_1436__li1933013935818">You have locked the job. Otherwise, you must click <strong id="dataartsstudio_01_1436__b15322101224910">Lock</strong> so that you can develop the job. A job you create or import is locked by you by default. For details, see the <a href="dataartsstudio_01_0913.html">lock function</a>.</li></ul>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_01_1436__section037832452814"><a name="dataartsstudio_01_1436__section037832452814"></a><a name="section037832452814"></a><h4 class="sectiontitle">Developing an SQL Script</h4><ol id="dataartsstudio_01_1436__ol83781624142817"><li id="dataartsstudio_01_1436__li2848154165110">Log in to the <span id="dataartsstudio_01_1436__en-us_topic_0000001987511677_en-us_topic_0000001127311125_text5574448155719">DataArts Studio</span> console by following the instructions in <a href="dataartsstudio_01_0001.html">Accessing the DataArts Studio Instance Console</a>.</li><li id="dataartsstudio_01_1436__li7378192412813">On the <span id="dataartsstudio_01_1436__en-us_topic_0181092879_text185611381448">DataArts Studio</span> console, locate a workspace and click <strong id="dataartsstudio_01_1436__en-us_topic_0181092879_b65382814249">DataArts Factory</strong>.</li><li id="dataartsstudio_01_1436__li203781243286">In the left navigation pane of DataArts Factory, choose <span class="menucascade" id="dataartsstudio_01_1436__en-us_topic_0181491853_en-us_topic_0099797006_menucascade12634646125010"><b><span class="uicontrol" id="dataartsstudio_01_1436__en-us_topic_0181491853_en-us_topic_0099797006_uicontrol1063494615015">Development</span></b> > <b><span class="uicontrol" id="dataartsstudio_01_1436__en-us_topic_0181491853_en-us_topic_0099797006_uicontrol14634104655019"><span id="dataartsstudio_01_1436__en-us_topic_0181491853_en-us_topic_0099797006_text126341046105013">Develop Job</span></span></b></span>.</li><li id="dataartsstudio_01_1436__li193795246286">In the job directory, double-click the name of a single-task job to access the job development page.</li><li id="dataartsstudio_01_1436__li1249117344416">On the right of the SQL editor, click <strong id="dataartsstudio_01_1436__b44185523518">Basic Info</strong> to configure basic information, properties, and advanced settings of the job. <a href="#dataartsstudio_01_1436__table44429421624">Table 1</a> lists the basic information, <a href="#dataartsstudio_01_1436__table14176135863317">Table 2</a> lists the properties, and <a href="#dataartsstudio_01_1436__table7691319191317">Table 3</a> lists the advanced settings.
|
|
<div class="tablenoborder"><a name="dataartsstudio_01_1436__table44429421624"></a><a name="table44429421624"></a><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_01_1436__table44429421624" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Basic job information</caption><thead align="left"><tr id="dataartsstudio_01_1436__en-us_topic_0181491854_en-us_topic_0099797007_row53625586"><th align="left" class="cellrowborder" valign="top" width="16.3%" id="mcps1.3.4.2.5.5.2.3.1.1"><p id="dataartsstudio_01_1436__en-us_topic_0181491854_en-us_topic_0099797007_p144111518820">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="83.7%" id="mcps1.3.4.2.5.5.2.3.1.2"><p id="dataartsstudio_01_1436__en-us_topic_0181491854_en-us_topic_0099797007_p114495284">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dataartsstudio_01_1436__en-us_topic_0181491854_en-us_topic_0099797007_row124865681"><td class="cellrowborder" valign="top" width="16.3%" headers="mcps1.3.4.2.5.5.2.3.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0181491854_en-us_topic_0099797007_p251957816">Owner</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="83.7%" headers="mcps1.3.4.2.5.5.2.3.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0181491854_en-us_topic_0099797007_p19551159812">An owner configured during job creation is automatically matched. This parameter value can be modified.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0181491854_en-us_topic_0099797007_row53761191048"><td class="cellrowborder" valign="top" width="16.3%" headers="mcps1.3.4.2.5.5.2.3.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0181491854_en-us_topic_0099797007_p83762019643">Executor</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="83.7%" headers="mcps1.3.4.2.5.5.2.3.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0181491854_p1687715635117">This parameter is available when <strong id="dataartsstudio_01_1436__en-us_topic_0181491854_b1597517458487">Scheduling Identities</strong> is set to <strong id="dataartsstudio_01_1436__en-us_topic_0181491854_b15106115794816">Yes</strong>. </p>
|
|
<p id="dataartsstudio_01_1436__en-us_topic_0181491854_en-us_topic_0099797007_p737741920412">User that executes the job. When you enter an executor, the job is executed by the executor. If the executor is left unspecified, the job is executed by the user who submitted the job for startup.</p>
|
|
<div class="note" id="dataartsstudio_01_1436__en-us_topic_0181491854_note165672261718"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_1436__en-us_topic_0181491854_p162860248590">You can configure execution users only after you <span id="dataartsstudio_01_1436__en-us_topic_0181491854_text199891646175014">apply for the whitelist membership</span>. To enable it, contact <span id="dataartsstudio_01_1436__en-us_topic_0181491854_text16120155010504">customer service or technical support</span>.</p>
|
|
</div></div>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0181491854_row1241410054511"><td class="cellrowborder" valign="top" width="16.3%" headers="mcps1.3.4.2.5.5.2.3.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0181491854_p11415100184515">Job Agency</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="83.7%" headers="mcps1.3.4.2.5.5.2.3.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0181491854_p69698248561">This parameter is available when <strong id="dataartsstudio_01_1436__en-us_topic_0181491854_b96086313490">Scheduling Identities</strong> is set to <strong id="dataartsstudio_01_1436__en-us_topic_0181491854_b126097334914">Yes</strong>. </p>
|
|
<p id="dataartsstudio_01_1436__en-us_topic_0181491854_p1641520184510">After an agency is configured, the job interacts with other services as an agency during job execution.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0181491854_en-us_topic_0099797007_row1657451284"><td class="cellrowborder" valign="top" width="16.3%" headers="mcps1.3.4.2.5.5.2.3.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0181491854_en-us_topic_0099797007_p1961175381">Priority</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="83.7%" headers="mcps1.3.4.2.5.5.2.3.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0181491854_en-us_topic_0099797007_p865251818">Priority configured during job creation is automatically matched. This parameter value can be modified.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0181491854_en-us_topic_0099797007_row3112384478"><td class="cellrowborder" valign="top" width="16.3%" headers="mcps1.3.4.2.5.5.2.3.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0181491854_en-us_topic_0099797007_p4121938174720">Execution Timeout</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="83.7%" headers="mcps1.3.4.2.5.5.2.3.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0181491854_en-us_topic_0099797007_p512163844715">Timeout of the job instance. If this parameter is set to 0 or is not set, this parameter does not take effect. If the notification function is enabled for the job and the execution time of the job instance exceeds the preset value, the system sends a specified notification, and the job keeps running.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0181491854_row106615555311"><td class="cellrowborder" valign="top" width="16.3%" headers="mcps1.3.4.2.5.5.2.3.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0181491854_p3662195512316">Exclude Waiting Time from Instance Timeout Duration</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="83.7%" headers="mcps1.3.4.2.5.5.2.3.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0181491854_p13324175911811">Whether to exclude the wait time from the instance execution timeout duration</p>
|
|
<p id="dataartsstudio_01_1436__en-us_topic_0181491854_p46621055834">If you select this option, the time to wait before an instance starts running is excluded from the timeout duration. You can modify this setting in <a href="dataartsstudio_01_04501.html#dataartsstudio_01_04501__section14959925164217">Default Configuration > Exclude Waiting Time from Instance Timeout Duration</a>.</p>
|
|
<p id="dataartsstudio_01_1436__en-us_topic_0181491854_p11770154612810">If you do not select this option, the time to wait before an instance starts running is included in the timeout duration.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0181491854_en-us_topic_0099797007_row2671951686"><td class="cellrowborder" valign="top" width="16.3%" headers="mcps1.3.4.2.5.5.2.3.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0181491854_en-us_topic_0099797007_p107116520814">Custom Parameter</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="83.7%" headers="mcps1.3.4.2.5.5.2.3.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0181491854_en-us_topic_0099797007_p17761952813">Set the name and value of the parameter.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0181491854_row187556312426"><td class="cellrowborder" valign="top" width="16.3%" headers="mcps1.3.4.2.5.5.2.3.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0181491854_p4756143154216">Job Tag</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="83.7%" headers="mcps1.3.4.2.5.5.2.3.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0181491854_p475617310426">Configure job tags to manage jobs by category.</p>
|
|
<p id="dataartsstudio_01_1436__en-us_topic_0181491854_p2488449194213">Click <span class="uicontrol" id="dataartsstudio_01_1436__en-us_topic_0181491854_uicontrol45900149432"><b>Add</b></span> to add a tag to the job. You can also select a tag configured in <a href="dataartsstudio_01_0532.html">Managing Job Tags</a>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0181491854_row76106341013"><td class="cellrowborder" valign="top" width="16.3%" headers="mcps1.3.4.2.5.5.2.3.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0181491854_p46101833102">Job Description</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="83.7%" headers="mcps1.3.4.2.5.5.2.3.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0181491854_p18610836104">Description of the job</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
|
|
<div class="tablenoborder"><a name="dataartsstudio_01_1436__table14176135863317"></a><a name="table14176135863317"></a><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_01_1436__table14176135863317" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Attributes of a batch processing single-task SQL job</caption><thead align="left"><tr id="dataartsstudio_01_1436__row19177115843314"><th align="left" class="cellrowborder" valign="top" width="20.18%" id="mcps1.3.4.2.5.6.2.3.1.1"><p id="dataartsstudio_01_1436__en-us_topic_0104967365_p3475163304919">Property</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="79.82000000000001%" id="mcps1.3.4.2.5.6.2.3.1.2"><p id="dataartsstudio_01_1436__en-us_topic_0104967365_p447543364917">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dataartsstudio_01_1436__en-us_topic_0104967365_row0475533134913"><td class="cellrowborder" colspan="2" valign="top" headers="mcps1.3.4.2.5.6.2.3.1.1 mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__p1366510438577"><strong id="dataartsstudio_01_1436__b1159944153018">DLI SQL properties</strong></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row105195210372"><td class="cellrowborder" valign="top" width="20.18%" headers="mcps1.3.4.2.5.6.2.3.1.1 "><p id="dataartsstudio_01_1436__p95196253712">DLI Data Directory</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="79.82000000000001%" headers="mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__p1132017371098">Select the DLI data directory.</p>
|
|
<ul id="dataartsstudio_01_1436__ul6785173314913"><li id="dataartsstudio_01_1436__li15319168191912">Default DLI data directory <strong id="dataartsstudio_01_1436__b188281246452">dli</strong></li><li id="dataartsstudio_01_1436__li20785833795">Metadata catalog that has been created in LakeFormation associated with DLI.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row817795817339"><td class="cellrowborder" valign="top" width="20.18%" headers="mcps1.3.4.2.5.6.2.3.1.1 "><p id="dataartsstudio_01_1436__p797710514510">Database Name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="79.82000000000001%" headers="mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__p109771551205120">Select a database.</p>
|
|
<p id="dataartsstudio_01_1436__p6964191402417">If you select the default DLI data directory <strong id="dataartsstudio_01_1436__b16716152724520">dli</strong>, select a DLI database and tables.</p>
|
|
<p id="dataartsstudio_01_1436__p104193813108">If you select a metadata catalog that has been created in LakeFormation associated with DLI, select a LakeFormation database and tables.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row8177958203314"><td class="cellrowborder" valign="top" width="20.18%" headers="mcps1.3.4.2.5.6.2.3.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0102588983_p4199531294826">Queue Name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="79.82000000000001%" headers="mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0102588983_p4617707494826">The queue set in the SQL script is selected by default. You can change another one.</p>
|
|
<div class="p" id="dataartsstudio_01_1436__en-us_topic_0102588983_p665784416312">You can create a resource queue using either of the following methods:<ul id="dataartsstudio_01_1436__en-us_topic_0102588983_en-us_topic_0104967365_en-us_topic_0099822521_ul46080727105259"><li id="dataartsstudio_01_1436__en-us_topic_0102588983_en-us_topic_0104967365_en-us_topic_0099822521_li12073360105259">Click <span><img id="dataartsstudio_01_1436__image1268214319231" src="en-us_image_0000002269199617.png"></span>. On the displayed <span class="wintitle" id="dataartsstudio_01_1436__wintitle327612208590"><b>Queue Management</b></span> page of <span id="dataartsstudio_01_1436__text10276172016597">DLI</span>, create a resource queue.</li><li id="dataartsstudio_01_1436__en-us_topic_0102588983_en-us_topic_0104967365_en-us_topic_0099822521_li24857298105259">Go to the <span id="dataartsstudio_01_1436__text11781145445919">DLI</span> console to create a resource queue.</li></ul>
|
|
</div>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row817716586331"><td class="cellrowborder" valign="top" width="20.18%" headers="mcps1.3.4.2.5.6.2.3.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0102588983_p362485410161">Record Dirty Data</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="79.82000000000001%" headers="mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0102588983_p166241543161">Click <span><img id="dataartsstudio_01_1436__image19501153311411" src="en-us_image_0000002234080348.png"></span> to specify whether to record dirty data.</p>
|
|
<ul id="dataartsstudio_01_1436__en-us_topic_0102588983_ul16636134012584"><li id="dataartsstudio_01_1436__en-us_topic_0102588983_li1263616409582">If you select <span><img id="dataartsstudio_01_1436__image95699254911412" src="en-us_image_0000002269205005.png"></span>, dirty data will be recorded.</li><li id="dataartsstudio_01_1436__en-us_topic_0102588983_li14637174045813">If you do not select <span><img id="dataartsstudio_01_1436__image16306552844" src="en-us_image_0000002269205005.png"></span>, dirty data will not be recorded.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row342411916556"><td class="cellrowborder" valign="top" width="20.18%" headers="mcps1.3.4.2.5.6.2.3.1.1 "><p id="dataartsstudio_01_1436__p1598343035014">DLI Environmental Variable</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="79.82000000000001%" headers="mcps1.3.4.2.5.6.2.3.1.2 "><ul id="dataartsstudio_01_1436__ul7362436175216"><li id="dataartsstudio_01_1436__li14363133675219">The environment variable must start with <strong id="dataartsstudio_01_1436__b1451220317227">hoodie.</strong>, <strong id="dataartsstudio_01_1436__b10512133116228">dli.sql.</strong>, <strong id="dataartsstudio_01_1436__b4513143192214">dli.ext.</strong>, <strong id="dataartsstudio_01_1436__b15513163152217">dli.jobs.</strong>, <strong id="dataartsstudio_01_1436__b151313312223">spark.sql.</strong>, or <strong id="dataartsstudio_01_1436__b251343112224">spark.scheduler.pool</strong>.</li><li id="dataartsstudio_01_1436__li17800123120379">If the environment variable is <strong id="dataartsstudio_01_1436__b1185435132217">dli.sql.autoBroadcastJoinThreshold</strong>, the value must be an integer. If the environment variable is <strong id="dataartsstudio_01_1436__b1285163532210">dli.sql.shuffle.partitions</strong>, the value must be a positive integer.</li><li id="dataartsstudio_01_1436__li176775412522">If the key of the environment variable is <strong id="dataartsstudio_01_1436__b11947546510">dli.sql.shuffle.partitions</strong> or <strong id="dataartsstudio_01_1436__b1994713416519">dli.sql.autoBroadcastJoinThreshold</strong>, the environment variable cannot contain the greater than (>) or less than (<) sign.</li><li id="dataartsstudio_01_1436__li828175535418">If a parameter with the same name is configured in both a job and a script, the parameter value configured in the job will overwrite that configured in the script.<div class="note" id="dataartsstudio_01_1436__note17452125172714"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_1436__p9498162810283">When you run a DLI SQL script or test a DLI SQL single-task job in non-scheduling scenarios, the following parameters are enabled by default:</p>
|
|
<ul id="dataartsstudio_01_1436__ul17364192142810"><li id="dataartsstudio_01_1436__li1652173016283"><strong id="dataartsstudio_01_1436__b7635151911143">spark.sql.adaptive.enabled</strong>: Adaptive Query Execution (AQE) is enabled so that Spark can dynamically optimize the query execution plan based on the characteristics of the data being processed and improve the performance by reducing the amount of data to be processed.</li><li id="dataartsstudio_01_1436__li143641221202819"><strong id="dataartsstudio_01_1436__b105225433200">spark.sql.adaptive.join.enabled</strong>: AQE is enabled for join operations. The optimal join algorithm is selected based on the data being processed to improve performance.</li><li id="dataartsstudio_01_1436__li14364142172811"><strong id="dataartsstudio_01_1436__b1126115194211">spark.sql.adaptive.skewedJoin.enabled</strong>: AQE is enabled for skewed join operations. Skewed data can be automatically detected and the join algorithm is optimized accordingly to improve performance.</li><li id="dataartsstudio_01_1436__li103641021172816"><strong id="dataartsstudio_01_1436__b1922825374715">spark.sql.mergeSmallFiles.enabled</strong>: Merging of small files is enabled. Small files can be merged into large ones, improving performance and shortening the processing time. In addition, fewer files need to be read from remote storage, and more local files can be used.</li></ul>
|
|
<p id="dataartsstudio_01_1436__p191538562273">If you do not want to use these functions, you can set the values of the preceding parameters to <strong id="dataartsstudio_01_1436__b16551121215534">false</strong>.</p>
|
|
</div></div>
|
|
</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row0810753115617"><td class="cellrowborder" colspan="2" valign="top" headers="mcps1.3.4.2.5.6.2.3.1.1 mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__p1881120534565"><strong id="dataartsstudio_01_1436__b1581516132516">DWS SQL properties</strong></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row11428155919565"><td class="cellrowborder" valign="top" width="20.18%" headers="mcps1.3.4.2.5.6.2.3.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0101095439_p4199531294826">Data Connection</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="79.82000000000001%" headers="mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__p18253116241">Select a data connection.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row10695152016573"><td class="cellrowborder" valign="top" width="20.18%" headers="mcps1.3.4.2.5.6.2.3.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0101095439_p48321910114651">Database</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="79.82000000000001%" headers="mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__p058612441739">Select a database.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row18157142216573"><td class="cellrowborder" valign="top" width="20.18%" headers="mcps1.3.4.2.5.6.2.3.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0101095439_p4293119145717">Dirty Data Table</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="79.82000000000001%" headers="mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0101095439_p1529317935711">Name of the dirty data table defined in the SQL script.</p>
|
|
<p id="dataartsstudio_01_1436__p4819121171418">The dirty data attributes cannot be edited. They are automatically recommended by the SQL script content.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row3156202255719"><td class="cellrowborder" valign="top" width="20.18%" headers="mcps1.3.4.2.5.6.2.3.1.1 "><p id="dataartsstudio_01_1436__p070155508">Matching Rule</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="79.82000000000001%" headers="mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__p167018515010">Enter a Java regular expression used to match the DWS SQL result. For example, if the expression is (?<=\()(-*\d+?)(?=,) and the SQL result is (1,"error message"), then the matched result is "1".</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row182741423115716"><td class="cellrowborder" valign="top" width="20.18%" headers="mcps1.3.4.2.5.6.2.3.1.1 "><p id="dataartsstudio_01_1436__p11901117004">Failure Matching Value</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="79.82000000000001%" headers="mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__p7901187408">If the matched content equals the set value, the node fails to be executed.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row12274182325711"><td class="cellrowborder" colspan="2" valign="top" headers="mcps1.3.4.2.5.6.2.3.1.1 mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__p167861552349"><strong id="dataartsstudio_01_1436__b11917161667">RDS SQL properties</strong></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row203587244574"><td class="cellrowborder" valign="top" width="20.18%" headers="mcps1.3.4.2.5.6.2.3.1.1 "><p id="dataartsstudio_01_1436__p157861952745">Data Connection</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="79.82000000000001%" headers="mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__p2078613521544">Select a data connection.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row1835742412570"><td class="cellrowborder" valign="top" width="20.18%" headers="mcps1.3.4.2.5.6.2.3.1.1 "><p id="dataartsstudio_01_1436__p178612521240">Database</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="79.82000000000001%" headers="mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__p1478610521644">Select a database.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row9475118151717"><td class="cellrowborder" colspan="2" valign="top" headers="mcps1.3.4.2.5.6.2.3.1.1 mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__p5364729101711"><strong id="dataartsstudio_01_1436__b120125220121">Spark SQL properties</strong></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row554512018178"><td class="cellrowborder" valign="top" width="20.18%" headers="mcps1.3.4.2.5.6.2.3.1.1 "><p id="dataartsstudio_01_1436__p45451720161716">MRS Job Name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="79.82000000000001%" headers="mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__p13583152414227">MRS job name. The system automatically sets this parameter based on the job name.</p>
|
|
<p id="dataartsstudio_01_1436__p554520203172">If the MRS job name is not set and the direct connection mode is selected, the node name can contain only letters, digits, hyphens (-), and underscores (_). A maximum of 64 characters are allowed, and Chinese characters are not allowed.</p>
|
|
<div class="note" id="dataartsstudio_01_1436__note67971855131311"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_1436__p37981255171315">If you select an MRS API data connection, you cannot set the job name.</p>
|
|
</div></div>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row20456922151711"><td class="cellrowborder" valign="top" width="20.18%" headers="mcps1.3.4.2.5.6.2.3.1.1 "><p id="dataartsstudio_01_1436__p2456152211173">Data Connection</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="79.82000000000001%" headers="mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__p15456322131717">Select a data connection.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row12410968428"><td class="cellrowborder" valign="top" width="20.18%" headers="mcps1.3.4.2.5.6.2.3.1.1 "><p id="dataartsstudio_01_1436__p14682157134213">MRS Resource Queue</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="79.82000000000001%" headers="mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__p668217104219">Select a created MRS resource queue.</p>
|
|
<p id="dataartsstudio_01_1436__p66826734210">This parameter is mandatory if <a href="dataartsstudio_01_04501.html#dataartsstudio_01_04501__section146531751579">Whether MRS Resource Queue Is Mandatory</a> is set to <strong id="dataartsstudio_01_1436__en-us_topic_0181380725_b49381423121215">Yes</strong>.</p>
|
|
<div class="note" id="dataartsstudio_01_1436__note93071816685"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_1436__p43087165814">Select a queue you configured in the queue permissions of DataArts Security. If you set multiple resource queues for this node, the resource queue you select here has the highest priority.</p>
|
|
</div></div>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row175522015171911"><td class="cellrowborder" valign="top" width="20.18%" headers="mcps1.3.4.2.5.6.2.3.1.1 "><p id="dataartsstudio_01_1436__p655217155199">Database</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="79.82000000000001%" headers="mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__p855331517192">Select a database. If you select an MRS API connection, you cannot select a database.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row108992018181918"><td class="cellrowborder" valign="top" width="20.18%" headers="mcps1.3.4.2.5.6.2.3.1.1 "><p id="dataartsstudio_01_1436__p18899141881919">Program Parameter</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="79.82000000000001%" headers="mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__p121541735122012">Set program parameters.</p>
|
|
<p id="dataartsstudio_01_1436__p133713301782">The following is an example:</p>
|
|
<p id="dataartsstudio_01_1436__p958211491781">Set <strong id="dataartsstudio_01_1436__b68297003210">Parameter</strong> to <strong id="dataartsstudio_01_1436__b177735251395">--queue</strong> and <strong id="dataartsstudio_01_1436__b1561141933215">Value</strong> to <strong id="dataartsstudio_01_1436__b1469311731010">default_cr</strong>, indicating that a specified queue of the MRS cluster is configured. You can also go to the MRS console, click the name of the MRS cluster and then the <strong id="dataartsstudio_01_1436__b535510452361">Jobs</strong> tab, locate the job, click <strong id="dataartsstudio_01_1436__b15905550103719">More</strong> in the <strong id="dataartsstudio_01_1436__b1177817323815">Operation</strong> column, and select View <strong id="dataartsstudio_01_1436__b19785142114384">Details</strong> to view the job details.</p>
|
|
<div class="note" id="dataartsstudio_01_1436__note3570205812193"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_1436__p1757085851912">Configure optimization parameters such as threads, memory, and vCPUs for the job to optimize resource usage and improve job execution performance. This configuration is unavailable if a Spark proxy connection is used.</p>
|
|
<p id="dataartsstudio_01_1436__p2787204822219">Spark SQL jobs with a single operator and using a connection of the MRS API type support program parameters.</p>
|
|
</div></div>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row9215355172215"><td class="cellrowborder" colspan="2" valign="top" headers="mcps1.3.4.2.5.6.2.3.1.1 mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__p61095213232"><strong id="dataartsstudio_01_1436__b18843859162412">Hive SQL properties</strong></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row782785292214"><td class="cellrowborder" valign="top" width="20.18%" headers="mcps1.3.4.2.5.6.2.3.1.1 "><p id="dataartsstudio_01_1436__p15828145222218">MRS Job Name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="79.82000000000001%" headers="mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__p907513245">MRS job name. The system automatically sets this parameter based on the job name.</p>
|
|
<p id="dataartsstudio_01_1436__p110105117248">If the MRS job name is not set and the direct connection mode is selected, the node name can contain only letters, digits, hyphens (-), and underscores (_). A maximum of 64 characters are allowed, and Chinese characters are not allowed.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row6612115792216"><td class="cellrowborder" valign="top" width="20.18%" headers="mcps1.3.4.2.5.6.2.3.1.1 "><p id="dataartsstudio_01_1436__p36121957182210">Data Connection</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="79.82000000000001%" headers="mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__p190125111245">Select a data connection.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row117941517112410"><td class="cellrowborder" valign="top" width="20.18%" headers="mcps1.3.4.2.5.6.2.3.1.1 "><p id="dataartsstudio_01_1436__p187941017192411">Database</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="79.82000000000001%" headers="mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__p2095122419">Select a database.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row16979911124020"><td class="cellrowborder" valign="top" width="20.18%" headers="mcps1.3.4.2.5.6.2.3.1.1 "><p id="dataartsstudio_01_1436__p9980201115407">MRS Resource Queue</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="79.82000000000001%" headers="mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__p14525749134012">Select a created MRS resource queue.</p>
|
|
<p id="dataartsstudio_01_1436__p1536015140328">This parameter is mandatory if <a href="dataartsstudio_01_04501.html#dataartsstudio_01_04501__section146531751579">Whether MRS Resource Queue Is Mandatory</a> is set to <strong id="dataartsstudio_01_1436__en-us_topic_0181380725_b49381423121215_1">Yes</strong>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row20615620132418"><td class="cellrowborder" valign="top" width="20.18%" headers="mcps1.3.4.2.5.6.2.3.1.1 "><p id="dataartsstudio_01_1436__p8373203172412">Program Parameter</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="79.82000000000001%" headers="mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__p794219173265">Set program parameters.</p>
|
|
<p id="dataartsstudio_01_1436__p1176773017119">The following is an example:</p>
|
|
<p id="dataartsstudio_01_1436__p1376715306118">Set <strong id="dataartsstudio_01_1436__b779111164320">Parameter</strong> to <strong id="dataartsstudio_01_1436__b1076793011117">--hiveconf</strong> and <strong id="dataartsstudio_01_1436__b15792613430">Value</strong> to <strong id="dataartsstudio_01_1436__b17824559141316">mapreduce.job.queuename=</strong><strong id="dataartsstudio_01_1436__b8767183091119">default_cr</strong>, indicating that a specified queue of the MRS cluster is configured. You can also go to the MRS console, click the name of the MRS cluster and then the <strong id="dataartsstudio_01_1436__b1618341710446">Jobs</strong> tab, locate the job, click <strong id="dataartsstudio_01_1436__b101841117204417">More</strong> in the <strong id="dataartsstudio_01_1436__b3184517164416">Operation</strong> column, and select View <strong id="dataartsstudio_01_1436__b18184111784419">Details</strong> to view the job details.</p>
|
|
<div class="note" id="dataartsstudio_01_1436__note129424179265"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_1436__p159431217112611">Configure optimization parameters such as threads, memory, and vCPUs for the job to optimize resource usage and improve job execution performance. This configuration is unavailable if a Hive proxy connection is used.</p>
|
|
<p id="dataartsstudio_01_1436__p63340403241">Hive SQL jobs with a single operator and using a connection of the MRS API type support program parameters.</p>
|
|
</div></div>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row1330113302418"><td class="cellrowborder" colspan="2" valign="top" headers="mcps1.3.4.2.5.6.2.3.1.1 mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__p13538245182519"><strong id="dataartsstudio_01_1436__b13590835141513">Doris SQL properties</strong></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row1697433717246"><td class="cellrowborder" valign="top" width="20.18%" headers="mcps1.3.4.2.5.6.2.3.1.1 "><p id="dataartsstudio_01_1436__p16974137132415">Data Connection</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="79.82000000000001%" headers="mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__p82341924272">Select a data connection.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row92863416253"><td class="cellrowborder" valign="top" width="20.18%" headers="mcps1.3.4.2.5.6.2.3.1.1 "><p id="dataartsstudio_01_1436__p92861541102520">Database</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="79.82000000000001%" headers="mcps1.3.4.2.5.6.2.3.1.2 "><p id="dataartsstudio_01_1436__p1923418210274">Select a database.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
|
|
<div class="tablenoborder"><a name="dataartsstudio_01_1436__table7691319191317"></a><a name="table7691319191317"></a><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_01_1436__table7691319191317" frame="border" border="1" rules="all"><caption><b>Table 3 </b>Advanced parameters</caption><thead align="left"><tr id="dataartsstudio_01_1436__en-us_topic_0099822521_row27216578102411"><th align="left" class="cellrowborder" valign="top" width="21.58%" id="mcps1.3.4.2.5.7.2.4.1.1"><p id="dataartsstudio_01_1436__en-us_topic_0099822521_p57059205102411">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="10.14%" id="mcps1.3.4.2.5.7.2.4.1.2"><p id="dataartsstudio_01_1436__en-us_topic_0099822521_p58392901102411">Mandatory</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="68.28%" id="mcps1.3.4.2.5.7.2.4.1.3"><p id="dataartsstudio_01_1436__en-us_topic_0099822521_p32204521102411">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dataartsstudio_01_1436__en-us_topic_0099822521_row51612113175"><td class="cellrowborder" valign="top" width="21.58%" headers="mcps1.3.4.2.5.7.2.4.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0099822521_p416115112178">Node Status Polling Interval (s)</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.14%" headers="mcps1.3.4.2.5.7.2.4.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0099822521_p101615110176">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="68.28%" headers="mcps1.3.4.2.5.7.2.4.1.3 "><p id="dataartsstudio_01_1436__en-us_topic_0099822521_p4161191101716">How often the system checks whether the <span id="dataartsstudio_01_1436__en-us_topic_0099822521_text1526241235118">node</span> execution is complete. The value ranges from 1 to 60 seconds.</p>
|
|
<p id="dataartsstudio_01_1436__p1328318413543">During the node execution, the system checks whether the node execution is complete at the configured interval.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0099822521_row5101045193916"><td class="cellrowborder" valign="top" width="21.58%" headers="mcps1.3.4.2.5.7.2.4.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0099822521_p147314419397">Max. Node Execution Duration</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.14%" headers="mcps1.3.4.2.5.7.2.4.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0099822521_p610124511390">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="68.28%" headers="mcps1.3.4.2.5.7.2.4.1.3 "><p id="dataartsstudio_01_1436__en-us_topic_0099822521_p11011456393">Execution timeout interval for the <span id="dataartsstudio_01_1436__en-us_topic_0099822521_text380131541112">node</span>. If retry is configured and the execution is not complete within the timeout interval, the <span id="dataartsstudio_01_1436__text2466192632012">node</span> will be executed again.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0099822521_row58429402102411"><td class="cellrowborder" valign="top" width="21.58%" headers="mcps1.3.4.2.5.7.2.4.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0099822521_p5533912102858">Retry upon Failure</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.14%" headers="mcps1.3.4.2.5.7.2.4.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0099822521_p45593742102858">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="68.28%" headers="mcps1.3.4.2.5.7.2.4.1.3 "><p id="dataartsstudio_01_1436__en-us_topic_0099822521_p2105628102858">Whether to re-execute a <span id="dataartsstudio_01_1436__text89561346102019">node</span> if it fails to be executed.</p>
|
|
<ul id="dataartsstudio_01_1436__en-us_topic_0099822521_ul18950660102858"><li id="dataartsstudio_01_1436__en-us_topic_0099822521_li36338219102858"><strong id="dataartsstudio_01_1436__b9374193765318">Yes</strong>: The <span id="dataartsstudio_01_1436__text737493716537">node</span> will be re-executed, and the following parameters must be configured:<ul id="dataartsstudio_01_1436__en-us_topic_0099822521_ul58608523102858"><li id="dataartsstudio_01_1436__li14307346185017">Retry upon Timeout</li><li id="dataartsstudio_01_1436__en-us_topic_0099822521_li44375787102858"><strong id="dataartsstudio_01_1436__b1585633825317">Maximum Retries</strong></li><li id="dataartsstudio_01_1436__en-us_topic_0099822521_li63837764102858"><strong id="dataartsstudio_01_1436__b618565295310">Retry Interval (seconds)</strong></li></ul>
|
|
</li><li id="dataartsstudio_01_1436__en-us_topic_0099822521_li37668971102858"><strong id="dataartsstudio_01_1436__b628165785313">No</strong>: The <span id="dataartsstudio_01_1436__text42819572533">node</span> will not be re-executed. This is the default value.<div class="note" id="dataartsstudio_01_1436__en-us_topic_0099822521_note845010501289"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_1436__en-us_topic_0099822521_p194511050132816">If retry is configured for a job node and the timeout duration is configured, the system allows you to retry a node when the node execution times out.</p>
|
|
<p id="dataartsstudio_01_1436__en-us_topic_0099822521_p1313284812516">If a node is not re-executed when it fails upon timeout, you can go to the <strong id="dataartsstudio_01_1436__b4164194912205">Default Configuration</strong> page to modify this policy.</p>
|
|
<p id="dataartsstudio_01_1436__p1013112593502"><strong id="dataartsstudio_01_1436__b18843191411411">Retry upon Timeout</strong> is displayed only when <strong id="dataartsstudio_01_1436__b313073101410">Retry upon Failure</strong> is set to <strong id="dataartsstudio_01_1436__b758673412140">Yes</strong>.</p>
|
|
</div></div>
|
|
</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0099822521_row29541959102411"><td class="cellrowborder" valign="top" width="21.58%" headers="mcps1.3.4.2.5.7.2.4.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0099822521_p13154928102858">Policy for Handling Subsequent Nodes If the Current <span id="dataartsstudio_01_1436__text385218301026">node</span> Fails</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.14%" headers="mcps1.3.4.2.5.7.2.4.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0099822521_p58916261102858">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="68.28%" headers="mcps1.3.4.2.5.7.2.4.1.3 "><p id="dataartsstudio_01_1436__en-us_topic_0099822521_p7487822102858">Policy for handling subsequent nodes if the current <span id="dataartsstudio_01_1436__en-us_topic_0099822521_text5371194616130">node</span> fails</p>
|
|
<ul id="dataartsstudio_01_1436__en-us_topic_0099822521_ul281538102858"><li id="dataartsstudio_01_1436__en-us_topic_0099822521_li2533844102858"><strong id="dataartsstudio_01_1436__b6842113782215">End the current job execution plan</strong>: Execution of the current job will stop, and the job instance status will become <strong id="dataartsstudio_01_1436__b184210377223">Failed</strong>. If the job is scheduled periodically, subsequent periodic scheduling will run properly.</li><li id="dataartsstudio_01_1436__en-us_topic_0099822521_li22804597102858"><strong id="dataartsstudio_01_1436__b10252526182311">Ignore the failure and set the job execution result to success</strong>: The failure of the current node will be ignored. The job instance status will become <strong id="dataartsstudio_01_1436__b7253192616234">Successful</strong>. If the job is scheduled periodically, subsequent periodic scheduling will run properly.</li></ul>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</li><li id="dataartsstudio_01_1436__li104995178561">Enter one or more SQL statements in the SQL editor.<div class="note" id="dataartsstudio_01_1436__note1357515910162"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dataartsstudio_01_1436__en-us_topic_0181092880_ul193361344111817"><li id="dataartsstudio_01_1436__en-us_topic_0181092880_li53361444184">SQL statements are separated by semicolons (<span class="parmvalue" id="dataartsstudio_01_1436__parmvalue33175618508"><b>;</b></span>). If semicolons are used in other places but not used to separate SQL statements, escape them with backslashes (<span class="parmvalue" id="dataartsstudio_01_1436__parmvalue874411625117"><b>\</b></span>). The following is an example:<pre class="screen" id="dataartsstudio_01_1436__en-us_topic_0181092880_screen857505931613">select 1;
|
|
select * from a where b="dsfa\;"; --example 1\;example 2.</pre>
|
|
</li><li id="dataartsstudio_01_1436__en-us_topic_0181092880_li033093411712">RDS SQL does not support the begin ... commit transaction syntax. If necessary, use the start transaction ... commit transaction syntax.</li><li id="dataartsstudio_01_1436__en-us_topic_0181092880_li873311161613">The script cannot be larger than 16 MB.</li><li id="dataartsstudio_01_1436__en-us_topic_0181092880_li133634441814">The system date obtained by using an SQL statement is different from that obtained by using the database tool. The query result is stored in the database in the YYYY-MM-DD format, but the query result displayed on the page is in the converted format.</li><li id="dataartsstudio_01_1436__en-us_topic_0181092880_li19186161512220">When a user submits a Spark SQL script to MRS, the script is submitted to the tenant queue bound to the user by default. The bound queue is the queue corresponding to tenant role of the user. If there are multiple queues, the system preferentially selects a queue based on the queue priorities. To set a fixed queue for the user to submit scripts, log in to FusionInsight Manager, choose <strong id="dataartsstudio_01_1436__b6351038155113">Tenant Resources</strong> > <strong id="dataartsstudio_01_1436__b10351203818516">Dynamic Resource Plan</strong>, and click the <strong id="dataartsstudio_01_1436__b11351193810514">Global User Policy</strong> tab. For details, see "Managing Global User Policies" in <em id="dataartsstudio_01_1436__i73521338115112">MapReduce Service (MRS) Usage Guide</em>.</li><li id="dataartsstudio_01_1436__li55661519182314">You can click <strong id="dataartsstudio_01_1436__b7881133916515">Check Syntax</strong> to check the syntax of a Spark SQL or Hive SQL script. After the check is complete, you can view the check result in the lower part of the page.</li></ul>
|
|
</div></div>
|
|
<div class="p" id="dataartsstudio_01_1436__p6791111817564">To facilitate script development, <span id="dataartsstudio_01_1436__text439331819172">DataArts Factory</span> provides the following capabilities:<ul id="dataartsstudio_01_1436__ul2335154451820"><li id="dataartsstudio_01_1436__li203351144151815">The script editor supports the following shortcut keys, which improve the script development efficiency:<ul id="dataartsstudio_01_1436__ul56601946112219"><li id="dataartsstudio_01_1436__li14373185914265"><strong id="dataartsstudio_01_1436__b757611711365">F8</strong>: Run a script.</li><li id="dataartsstudio_01_1436__li26191439142715"><strong id="dataartsstudio_01_1436__b17429151083616">F9</strong>: Stop running a script.</li><li id="dataartsstudio_01_1436__li36602464228"><strong id="dataartsstudio_01_1436__b1348125111412">Ctrl</strong> + <strong id="dataartsstudio_01_1436__b183604811811412">/</strong>: Comment out or uncomment the line or code block at the cursor.</li><li id="dataartsstudio_01_1436__li866019461229"><strong id="dataartsstudio_01_1436__b12138436511412">Ctrl</strong> +<strong id="dataartsstudio_01_1436__b198151715511412">Z</strong>: Cancel</li><li id="dataartsstudio_01_1436__li1866015462221"><strong id="dataartsstudio_01_1436__b94073783711412">Ctrl</strong> + <strong id="dataartsstudio_01_1436__b41195112611412">F</strong>: Search</li><li id="dataartsstudio_01_1436__li2066024615229"><strong id="dataartsstudio_01_1436__b173693693311412">Ctrl</strong> + <strong id="dataartsstudio_01_1436__b13456752611412">Shift</strong> + <strong id="dataartsstudio_01_1436__b102570585011412">R</strong>: Replace</li><li id="dataartsstudio_01_1436__li4660144642210"><strong id="dataartsstudio_01_1436__b23216637511412">Ctrl</strong> + <strong id="dataartsstudio_01_1436__b187363332311412">X</strong>: Cut</li><li id="dataartsstudio_01_1436__li3709114892715"><strong id="dataartsstudio_01_1436__b1221361273619">Ctrl</strong> + <strong id="dataartsstudio_01_1436__b172135129366">S</strong>: Save a script.</li><li id="dataartsstudio_01_1436__li186601546192215"><strong id="dataartsstudio_01_1436__b83293256411412">Alt</strong> + mouse dragging: Select columns to edit a block.</li><li id="dataartsstudio_01_1436__li19660146112213"><strong id="dataartsstudio_01_1436__b92551341311412">Ctrl</strong> + mouse click: Select multiple lines to edit or indent them together.</li><li id="dataartsstudio_01_1436__li766034613223"><strong id="dataartsstudio_01_1436__b88752517311412">Ctrl</strong> + <strong id="dataartsstudio_01_1436__b190969742111412">→</strong> (or <strong id="dataartsstudio_01_1436__b172637930411412">←</strong>): Move the cursor rightwards (or leftwards) by word.</li><li id="dataartsstudio_01_1436__li18660546182217"><strong id="dataartsstudio_01_1436__b206004948411412">Ctrl</strong> +<strong id="dataartsstudio_01_1436__b184469080211412"> Home</strong> or <strong id="dataartsstudio_01_1436__b94052854011412">Ctrl</strong> + <strong id="dataartsstudio_01_1436__b3788417411412">End</strong>: Navigate to the beginning or end of the current file.</li><li id="dataartsstudio_01_1436__li15660104662212"><strong id="dataartsstudio_01_1436__b19553018611412">Home</strong> or <strong id="dataartsstudio_01_1436__b139145728611412">End</strong>: Navigate to the beginning or end of the current line.</li><li id="dataartsstudio_01_1436__li1666014682218"><strong id="dataartsstudio_01_1436__b79857623011412">Ctrl</strong> + <strong id="dataartsstudio_01_1436__b62520166111412">Shift</strong> + <strong id="dataartsstudio_01_1436__b17209684411412">L</strong>: Double-click all the same character strings and add cursors to them to implement batch modification.</li><li id="dataartsstudio_01_1436__li1252313711294"><strong id="dataartsstudio_01_1436__b8810102263617">Ctrl</strong> + <strong id="dataartsstudio_01_1436__b16810322183617">D</strong>: Delete a line.</li><li id="dataartsstudio_01_1436__li11649122613018"><strong id="dataartsstudio_01_1436__b954192823617">Shift</strong> + <strong id="dataartsstudio_01_1436__b85511284366">Ctrl</strong> + <strong id="dataartsstudio_01_1436__b455112816362">U</strong>: Unlock a script.</li><li id="dataartsstudio_01_1436__li219431175418"><strong id="dataartsstudio_01_1436__b7625192913369">Ctrl</strong> + <strong id="dataartsstudio_01_1436__b462552923614">Alt</strong> + <strong id="dataartsstudio_01_1436__b10626182953612">K</strong>: Select the word where the cursor resides.</li><li id="dataartsstudio_01_1436__li5134344165412"><strong id="dataartsstudio_01_1436__b9312163143619">Ctrl</strong> + <strong id="dataartsstudio_01_1436__b031263116363">B</strong>: Format</li><li id="dataartsstudio_01_1436__li94225277567"><strong id="dataartsstudio_01_1436__b236336363">Ctrl</strong> + <strong id="dataartsstudio_01_1436__b20393333618">Shift</strong> + <strong id="dataartsstudio_01_1436__b18393316367">Z</strong>: Redo</li><li id="dataartsstudio_01_1436__li615165415612"><strong id="dataartsstudio_01_1436__b5515143413618">Ctrl</strong> + <strong id="dataartsstudio_01_1436__b05153340367">Enter</strong>: Execute the selected line or content.</li><li id="dataartsstudio_01_1436__li143125387582"><strong id="dataartsstudio_01_1436__b1173736173616">Ctrl</strong> + <strong id="dataartsstudio_01_1436__b13737369369">Alt</strong> + <strong id="dataartsstudio_01_1436__b167312365363">F</strong>: Flag</li><li id="dataartsstudio_01_1436__li2713113917215"><strong id="dataartsstudio_01_1436__b18411237163616">Ctrl</strong> + <strong id="dataartsstudio_01_1436__b15411133713362">Shift</strong> + <strong id="dataartsstudio_01_1436__b1441111372363">K</strong>: Search for the previous one.</li><li id="dataartsstudio_01_1436__li67146391428"><strong id="dataartsstudio_01_1436__b11972438113616">Ctrl</strong> + <strong id="dataartsstudio_01_1436__b9972173815362">K</strong>: Search for the next one.</li><li id="dataartsstudio_01_1436__li1371419391024"><strong id="dataartsstudio_01_1436__b4565144043613">Ctrl</strong> + <strong id="dataartsstudio_01_1436__b1956510404365">Backspace</strong>: Delete the word to the left of the cursor.</li><li id="dataartsstudio_01_1436__li87154392213"><strong id="dataartsstudio_01_1436__b189618425367">Ctrl</strong> + <strong id="dataartsstudio_01_1436__b16961442123619">Delete</strong>: Delete the word to the right of the cursor.</li><li id="dataartsstudio_01_1436__li147151039126"><strong id="dataartsstudio_01_1436__b126914432365">Alt</strong> + <strong id="dataartsstudio_01_1436__b6692843203611">Backspace</strong>: Delete all content from the beginning of the line to the cursor.</li><li id="dataartsstudio_01_1436__li971615395218"><strong id="dataartsstudio_01_1436__b2043220457364">Alt</strong> + <strong id="dataartsstudio_01_1436__b194326451360">Delete</strong>: Delete all content from the cursor to the end of the line.</li><li id="dataartsstudio_01_1436__li071615397219"><strong id="dataartsstudio_01_1436__b127254733613">Alt</strong> + <strong id="dataartsstudio_01_1436__b19721547193612">Shift</strong>-<strong id="dataartsstudio_01_1436__b97254716361">Left</strong>: Select all content from the beginning of the line to the cursor.</li><li id="dataartsstudio_01_1436__li871610392213"><strong id="dataartsstudio_01_1436__b1160514813369">Alt</strong> + <strong id="dataartsstudio_01_1436__b6606184883619">Shift</strong>-<strong id="dataartsstudio_01_1436__b0606184815362">Right</strong>: Select all content from the cursor to the end of the line.</li></ul>
|
|
</li><li id="dataartsstudio_01_1436__li5335204418188">System functions<p id="dataartsstudio_01_1436__p17516092224"><a name="dataartsstudio_01_1436__li5335204418188"></a><a name="li5335204418188"></a>To view the functions supported by this type of data connection, click <span class="uicontrol" id="dataartsstudio_01_1436__uicontrol8575175961618"><b>System Functions</b></span> on the right of the editor. You can double-click a function to the editor to use it.</p>
|
|
</li><li id="dataartsstudio_01_1436__li16575145911611">Script parameters<p id="dataartsstudio_01_1436__p657518593168"><a name="dataartsstudio_01_1436__li16575145911611"></a><a name="li16575145911611"></a>Enter script parameters in the SQL statement and click <strong id="dataartsstudio_01_1436__b94874984411412">Parameter Setup</strong> in the right pane of the editor and then click <strong id="dataartsstudio_01_1436__b75758865011412">Update from Script</strong>. You can also directly configure parameters and constants for the job script.</p>
|
|
<p id="dataartsstudio_01_1436__p6575459201615">In the following script example, <em id="dataartsstudio_01_1436__i15521563111412">str1</em> indicates the parameter name. It can contain only letters, digits, hyphens (-), underscores (_), greater-than signs (>), and less-than signs (<), and can contain a maximum of 16 characters. The parameter name must be unique.</p>
|
|
<pre class="screen" id="dataartsstudio_01_1436__screen1357535918167">select ${str1} from data;</pre>
|
|
</li><li id="dataartsstudio_01_1436__li11804121913394">Visualized reading of data tables to generate SQL statements<p id="dataartsstudio_01_1436__p98041219193915"><a name="dataartsstudio_01_1436__li11804121913394"></a><a name="li11804121913394"></a>Click <span class="uicontrol" id="dataartsstudio_01_1436__uicontrol568141819210"><b>Data Tables</b></span> on the right of the editor to display all the tables in the current database or schema. You can select tables and columns and click <strong id="dataartsstudio_01_1436__b9684181727">Generate SQL Statement</strong> in the lower right corner to generate an SQL statement, which you need to manually format.</p>
|
|
</li></ul>
|
|
</div>
|
|
</li><li id="dataartsstudio_01_1436__li1476011035420">(Optional) In the upper part of the editor, click <strong id="dataartsstudio_01_1436__b93456173511412">Format</strong> to format SQL statements.</li><li id="dataartsstudio_01_1436__li1076011035413">In the upper part of the editor, click <span class="uicontrol" id="dataartsstudio_01_1436__uicontrol1864816511360"><b>Execute</b></span>. If you need to execute some SQL statements separately, select the SQL statements first. After executing the SQL statements, view the execution history and result of the script in the lower part of the editor.<div class="note" id="dataartsstudio_01_1436__note2835125114415"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dataartsstudio_01_1436__ul1766452618408"><li id="dataartsstudio_01_1436__li4646143124117">You can click <span class="parmname" id="dataartsstudio_01_1436__parmname101701018162611"><b>View Log</b></span> to view logs of the job.</li><li id="dataartsstudio_01_1436__li17619233372">When viewing the script execution result, you can double-click a field in any row to view the result details. You can copy the field name.</li><li id="dataartsstudio_01_1436__li1836684741610">You can control display of the script execution history by setting <strong id="dataartsstudio_01_1436__en-us_topic_0181092880_b1785141313301">Script Execution History</strong> in <strong id="dataartsstudio_01_1436__en-us_topic_0181092880_b2962161610303">Default Configuration</strong> to <strong id="dataartsstudio_01_1436__en-us_topic_0181092880_b1943110206309">Myself</strong> or <strong id="dataartsstudio_01_1436__en-us_topic_0181092880_b161991225123012">All users</strong>.</li></ul>
|
|
</div></div>
|
|
</li><li id="dataartsstudio_01_1436__li976018016547">Above the editor, click <strong id="dataartsstudio_01_1436__b264955313710">Save</strong> to save the job.</li></ol>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_01_1436__en-us_topic_0099797007_section754991272419"><a name="dataartsstudio_01_1436__en-us_topic_0099797007_section754991272419"></a><a name="en-us_topic_0099797007_section754991272419"></a><h4 class="sectiontitle">Configuring job parameters</h4><p id="dataartsstudio_01_1436__p174671056182519">Click <span class="wintitle" id="dataartsstudio_01_1436__wintitle27421033132618"><b>Parameter Setup</b></span> on the right of the editor and set the parameters described in <a href="#dataartsstudio_01_1436__en-us_topic_0099797007_table20701161192718">Table 4</a>.</p>
|
|
|
|
<div class="tablenoborder"><a name="dataartsstudio_01_1436__en-us_topic_0099797007_table20701161192718"></a><a name="en-us_topic_0099797007_table20701161192718"></a><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_01_1436__en-us_topic_0099797007_table20701161192718" frame="border" border="1" rules="all"><caption><b>Table 4 </b>Job parameter setup</caption><thead align="left"><tr id="dataartsstudio_01_1436__en-us_topic_0099797007_row4717717274"><th align="left" class="cellrowborder" valign="top" width="16.76%" id="mcps1.3.5.3.2.3.1.1"><p id="dataartsstudio_01_1436__en-us_topic_0099797007_p147221618278">Module</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="83.24000000000001%" id="mcps1.3.5.3.2.3.1.2"><p id="dataartsstudio_01_1436__en-us_topic_0099797007_p10726141162720">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dataartsstudio_01_1436__en-us_topic_0099797007_row1296517379302"><td class="cellrowborder" colspan="2" valign="top" headers="mcps1.3.5.3.2.3.1.1 mcps1.3.5.3.2.3.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0099797007_p1397534033012"><strong id="dataartsstudio_01_1436__b79131041394">Variables</strong></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0099797007_row17309182714"><td class="cellrowborder" valign="top" width="16.76%" headers="mcps1.3.5.3.2.3.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0099797007_p473391192713">Add</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="83.24000000000001%" headers="mcps1.3.5.3.2.3.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0099797007_p273671112719">Click <span class="uicontrol" id="dataartsstudio_01_1436__en-us_topic_0099797007_uicontrol673811162720"><b>Add</b></span> and enter the variable parameter name and parameter value in the text boxes.</p>
|
|
<ul id="dataartsstudio_01_1436__en-us_topic_0099797007_ul187409117279"><li id="dataartsstudio_01_1436__en-us_topic_0099797007_li2743151172718">Parameter<p id="dataartsstudio_01_1436__en-us_topic_0099797007_p1674851162719"><a name="dataartsstudio_01_1436__en-us_topic_0099797007_li2743151172718"></a><a name="en-us_topic_0099797007_li2743151172718"></a>Only letters, numbers, periods (.), hyphens (-), and underscores (_) are allowed.</p>
|
|
</li><li id="dataartsstudio_01_1436__en-us_topic_0099797007_li12762111142715">Parameter Value<ul id="dataartsstudio_01_1436__en-us_topic_0099797007_ul157660122715"><li id="dataartsstudio_01_1436__en-us_topic_0099797007_li377818182715">The string type of parameter value is a character string, for example, <strong id="dataartsstudio_01_1436__b1486112357512">str1</strong>.</li><li id="dataartsstudio_01_1436__en-us_topic_0099797007_li678810112717">The numeric type of parameter value is a number or operation expression.</li></ul>
|
|
</li></ul>
|
|
<p id="dataartsstudio_01_1436__en-us_topic_0099797007_p17940172714">After the parameter is configured, it is referenced in the format of <strong id="dataartsstudio_01_1436__b1972743785111">$</strong>{<em id="dataartsstudio_01_1436__i10727193725117">Parameter name</em>} in the <span id="dataartsstudio_01_1436__text137271837145114">job</span>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row1555413219271"><td class="cellrowborder" valign="top" width="16.76%" headers="mcps1.3.5.3.2.3.1.1 "><p id="dataartsstudio_01_1436__p344181110812">Edit Parameter Expression</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="83.24000000000001%" headers="mcps1.3.5.3.2.3.1.2 "><p id="dataartsstudio_01_1436__p15901729135419">Click <span><img id="dataartsstudio_01_1436__image47411049125410" src="en-us_image_0000002269196941.png"></span> next to the parameter value text box. In the displayed dialog box, edit the parameter expression. For more expressions, see <a href="dataartsstudio_01_0494.html">Expression Overview</a>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0099797007_row158016117275"><td class="cellrowborder" valign="top" width="16.76%" headers="mcps1.3.5.3.2.3.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0099797007_p1480620116278">Modifying a Job</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="83.24000000000001%" headers="mcps1.3.5.3.2.3.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0099797007_p68115114273">Change the parameter name or value in the corresponding text boxes.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row1221191403315"><td class="cellrowborder" valign="top" width="16.76%" headers="mcps1.3.5.3.2.3.1.1 "><p id="dataartsstudio_01_1436__p3211514153312">Mask</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="83.24000000000001%" headers="mcps1.3.5.3.2.3.1.2 "><p id="dataartsstudio_01_1436__p92111914193315">If the parameter value is a key, click <span><img id="dataartsstudio_01_1436__image168804391333" src="en-us_image_0000002234237504.png"></span> to mask the value for security purposes.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0099797007_row5824415277"><td class="cellrowborder" valign="top" width="16.76%" headers="mcps1.3.5.3.2.3.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0099797007_p12828171202719">Delete</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="83.24000000000001%" headers="mcps1.3.5.3.2.3.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0099797007_p28314112712">Click <span><img id="dataartsstudio_01_1436__image1742175211346" src="en-us_image_0000002234077652.png"></span> next to the parameter name and value text boxes to delete the job parameter.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0099797007_row12548195220309"><td class="cellrowborder" colspan="2" valign="top" headers="mcps1.3.5.3.2.3.1.1 mcps1.3.5.3.2.3.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0099797007_p1159195517308"><strong id="dataartsstudio_01_1436__en-us_topic_0099797007_b1623716813116">Constant Parameter</strong></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0099797007_row16650721143114"><td class="cellrowborder" valign="top" width="16.76%" headers="mcps1.3.5.3.2.3.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0099797007_p1458714273120">Add</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="83.24000000000001%" headers="mcps1.3.5.3.2.3.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0099797007_p2587142113113">Click <span class="uicontrol" id="dataartsstudio_01_1436__en-us_topic_0099797007_uicontrol658754210311"><b>Add</b></span> and enter the constant parameter name and parameter value in the text boxes.</p>
|
|
<ul id="dataartsstudio_01_1436__en-us_topic_0099797007_ul16587642103115"><li id="dataartsstudio_01_1436__en-us_topic_0099797007_en-us_topic_0099797007_li2743151172718">Parameter<p id="dataartsstudio_01_1436__en-us_topic_0099797007_en-us_topic_0099797007_p1674851162719"><a name="dataartsstudio_01_1436__en-us_topic_0099797007_en-us_topic_0099797007_li2743151172718"></a><a name="en-us_topic_0099797007_en-us_topic_0099797007_li2743151172718"></a>Only letters, numbers, periods (.), hyphens (-), and underscores (_) are allowed.</p>
|
|
</li><li id="dataartsstudio_01_1436__en-us_topic_0099797007_en-us_topic_0099797007_li12762111142715">Parameter Value<ul id="dataartsstudio_01_1436__en-us_topic_0099797007_en-us_topic_0099797007_ul157660122715"><li id="dataartsstudio_01_1436__en-us_topic_0099797007_en-us_topic_0099797007_li377818182715">The string type of parameter value is a character string, for example, <strong id="dataartsstudio_01_1436__b12504153935111">str1</strong>.</li><li id="dataartsstudio_01_1436__en-us_topic_0099797007_en-us_topic_0099797007_li678810112717">The numeric type of parameter value is a number or operation expression.</li></ul>
|
|
</li></ul>
|
|
<p id="dataartsstudio_01_1436__en-us_topic_0099797007_p258818427319">After the parameter is configured, it is referenced in the format of <strong id="dataartsstudio_01_1436__b566412411519">$</strong>{<em id="dataartsstudio_01_1436__i7665104185117">Parameter name</em>} in the <span id="dataartsstudio_01_1436__text18665184135112">job</span>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row2891034963"><td class="cellrowborder" valign="top" width="16.76%" headers="mcps1.3.5.3.2.3.1.1 "><p id="dataartsstudio_01_1436__p1955432162717">Edit Parameter Expression</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="83.24000000000001%" headers="mcps1.3.5.3.2.3.1.2 "><p id="dataartsstudio_01_1436__p1028171911810">Click <span><img id="dataartsstudio_01_1436__image82811915819" src="en-us_image_0000002234237516.png"></span> next to the parameter value text box. In the displayed dialog box, edit the parameter expression. For more expressions, see <a href="dataartsstudio_01_0494.html">Expression Overview</a>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0099797007_row11706164053115"><td class="cellrowborder" valign="top" width="16.76%" headers="mcps1.3.5.3.2.3.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0099797007_p658864203114">Modifying a Job</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="83.24000000000001%" headers="mcps1.3.5.3.2.3.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0099797007_p2588144211314">Modify the parameter name and parameter value in text boxes and save the modifications.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0099797007_row14421933193118"><td class="cellrowborder" valign="top" width="16.76%" headers="mcps1.3.5.3.2.3.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0099797007_p18589164217318">Delete</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="83.24000000000001%" headers="mcps1.3.5.3.2.3.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0099797007_p75897428311">Click <span><img id="dataartsstudio_01_1436__image6758230103520" src="en-us_image_0000002269116869.png"></span> next to the parameter name and value text boxes to delete the job parameter.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row10161330171213"><td class="cellrowborder" colspan="2" valign="top" headers="mcps1.3.5.3.2.3.1.1 mcps1.3.5.3.2.3.1.2 "><p id="dataartsstudio_01_1436__p111481853131015"><strong id="dataartsstudio_01_1436__b670148311">Workspace Environment Variables</strong></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row1873317332121"><td class="cellrowborder" colspan="2" valign="top" headers="mcps1.3.5.3.2.3.1.1 mcps1.3.5.3.2.3.1.2 "><p id="dataartsstudio_01_1436__p1510627151118">View the variables and constants that have been configured in the workspace.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
<p id="dataartsstudio_01_1436__p1323193822219">Click the <strong id="dataartsstudio_01_1436__b19823130173919">Parameter Preview</strong> tab and configure the parameters listed in <a href="#dataartsstudio_01_1436__table1036167182419">Table 5</a>.</p>
|
|
|
|
<div class="tablenoborder"><a name="dataartsstudio_01_1436__table1036167182419"></a><a name="table1036167182419"></a><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_01_1436__table1036167182419" frame="border" border="1" rules="all"><caption><b>Table 5 </b>Job parameter preview</caption><thead align="left"><tr id="dataartsstudio_01_1436__row1361575245"><th align="left" class="cellrowborder" valign="top" width="16.76%" id="mcps1.3.5.5.2.3.1.1"><p id="dataartsstudio_01_1436__p536116710241">Module</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="83.24000000000001%" id="mcps1.3.5.5.2.3.1.2"><p id="dataartsstudio_01_1436__p43615712411">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dataartsstudio_01_1436__row1361579243"><td class="cellrowborder" valign="top" width="16.76%" headers="mcps1.3.5.5.2.3.1.1 "><p id="dataartsstudio_01_1436__p118286375340">Current Time</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="83.24000000000001%" headers="mcps1.3.5.5.2.3.1.2 "><p id="dataartsstudio_01_1436__p182913378348">This parameter is displayed only when <strong id="dataartsstudio_01_1436__b659624173917">Scheduling Type</strong> is set to <strong id="dataartsstudio_01_1436__b8596194183919">Run once</strong>. The default value is the current time.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row19342175682920"><td class="cellrowborder" valign="top" width="16.76%" headers="mcps1.3.5.5.2.3.1.1 "><p id="dataartsstudio_01_1436__p121931610313">Event Triggering Time</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="83.24000000000001%" headers="mcps1.3.5.5.2.3.1.2 "><p id="dataartsstudio_01_1436__p19494849103117">This parameter is displayed only when <strong id="dataartsstudio_01_1436__b511220505398">Scheduling Type</strong> is set to <strong id="dataartsstudio_01_1436__b1411316501399">Event-based</strong>. The default value is the time when an event is triggered.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row19620193410346"><td class="cellrowborder" valign="top" width="16.76%" headers="mcps1.3.5.5.2.3.1.1 "><p id="dataartsstudio_01_1436__p146621315163213">Scheduling Period</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="83.24000000000001%" headers="mcps1.3.5.5.2.3.1.2 "><p id="dataartsstudio_01_1436__p34114314322">This parameter is displayed only when <strong id="dataartsstudio_01_1436__b1135185523920">Scheduling Type</strong> is set to <strong id="dataartsstudio_01_1436__b1613505515394">Run periodically</strong>. The default value is the scheduling period.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row225551163510"><td class="cellrowborder" valign="top" width="16.76%" headers="mcps1.3.5.5.2.3.1.1 "><p id="dataartsstudio_01_1436__p154874171013">Start Time</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="83.24000000000001%" headers="mcps1.3.5.5.2.3.1.2 "><p id="dataartsstudio_01_1436__p2487141712110">This parameter is displayed only when <strong id="dataartsstudio_01_1436__b1135610374012">Scheduling Type</strong> is set to <strong id="dataartsstudio_01_1436__b93562312408">Run periodically</strong>. The value is the configured job execution time.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row782465317297"><td class="cellrowborder" valign="top" width="16.76%" headers="mcps1.3.5.5.2.3.1.1 "><p id="dataartsstudio_01_1436__p2608142318120">Start Time</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="83.24000000000001%" headers="mcps1.3.5.5.2.3.1.2 "><p id="dataartsstudio_01_1436__p1660832310117">This parameter is displayed only when <strong id="dataartsstudio_01_1436__b111116974017">Scheduling Type</strong> is set to <strong id="dataartsstudio_01_1436__b14117994020">Run periodically</strong>. The value is the time when the periodic job scheduling starts.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__row736211772414"><td class="cellrowborder" valign="top" width="16.76%" headers="mcps1.3.5.5.2.3.1.1 "><p id="dataartsstudio_01_1436__p133627752411">Subsequent Instances</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="83.24000000000001%" headers="mcps1.3.5.5.2.3.1.2 "><p id="dataartsstudio_01_1436__p11718104211298">Number of job instances scheduled.</p>
|
|
<ul id="dataartsstudio_01_1436__ul16464763019"><li id="dataartsstudio_01_1436__li1424718304">The default value is <strong id="dataartsstudio_01_1436__b218321610403">1</strong> when <strong id="dataartsstudio_01_1436__b151831716124018">Scheduling Type</strong> is set to <strong id="dataartsstudio_01_1436__b71841516184016">Run once</strong>.</li><li id="dataartsstudio_01_1436__li1443315333013">The default value is <strong id="dataartsstudio_01_1436__b1358614360403">1</strong> when <strong id="dataartsstudio_01_1436__b1758603674013">Scheduling Type</strong> is set to <strong id="dataartsstudio_01_1436__b135861836164012">Event-based</strong>.</li><li id="dataartsstudio_01_1436__li132701419133312">When <strong id="dataartsstudio_01_1436__b11465193918402">Scheduling Type</strong> is set to <strong id="dataartsstudio_01_1436__b164650393407">Run periodically</strong>:<p id="dataartsstudio_01_1436__p1820013301358">If the number of instances exceeds 10, a maximum of 10 instances can be displayed, and the system displays message "A maximum of 10 instances are supported."</p>
|
|
</li></ul>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
<div class="note" id="dataartsstudio_01_1436__note7110121213913"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_1436__p081333017438">In <strong id="dataartsstudio_01_1436__b13732242204014">Parameter Preview</strong>, if a job parameter has a syntax error, the system displays a message.</p>
|
|
<p id="dataartsstudio_01_1436__p111019128395">If a parameter depends on the data generated during job execution, such data cannot be simulated and displayed in <strong id="dataartsstudio_01_1436__b23841944204011">Parameter Preview</strong>.</p>
|
|
</div></div>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_01_1436__section17103142782618"><a name="dataartsstudio_01_1436__section17103142782618"></a><a name="section17103142782618"></a><h4 class="sectiontitle">Data Table</h4><p id="dataartsstudio_01_1436__p1899673218260">You can view tables of Hive SQL, Spark SQL, DLI SQL, Doris SQL, RDS SQL, and DWS SQLsingle-task batch processing jobs. On the <strong id="dataartsstudio_01_1436__b146184116425">Data Tables</strong> slide-out panel, you can select a table name to view the column names, field types, and descriptions in the table.</p>
|
|
<div class="fignone" id="dataartsstudio_01_1436__fig196983205358"><span class="figcap"><b>Figure 1 </b>Viewing a data table</span><br><span><img id="dataartsstudio_01_1436__image1569902073516" src="en-us_image_0000002234240156.png" title="Click to enlarge" class="imgResize"></span></div>
|
|
<p id="dataartsstudio_01_1436__p2325417112519"></p>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_01_1436__section1462324142616"><a name="dataartsstudio_01_1436__section1462324142616"></a><a name="section1462324142616"></a><h4 class="sectiontitle">Testing and Saving the Job</h4><p id="dataartsstudio_01_1436__en-us_topic_0099797007_p14530129204215">After configuring the job, perform the following operations:</p>
|
|
<ol id="dataartsstudio_01_1436__en-us_topic_0099797007_ol154526212465"><li id="dataartsstudio_01_1436__en-us_topic_0099797007_li118622411466"><span>Click <span><img id="dataartsstudio_01_1436__image10657115015817" src="en-us_image_0000002269119545.png"></span> to execute the job.</span><p><div class="note" id="dataartsstudio_01_1436__note203921943175918"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_1436__p1439217439591">You can view the run logs of the job by clicking <span class="parmname" id="dataartsstudio_01_1436__parmname686512415464"><b>View Log</b></span>.</p>
|
|
</div></div>
|
|
</p></li><li id="dataartsstudio_01_1436__en-us_topic_0099797007_li189612332212"><span>After the job is executed, click <span><img id="dataartsstudio_01_1436__image17176172415597" src="en-us_image_0000002234240172.png"></span> to save the job configuration.</span><p><p id="dataartsstudio_01_1436__p76458226213">After the job is saved, a version is automatically generated and displayed in <strong id="dataartsstudio_01_1436__en-us_topic_0181491854_b1186121412414">Versions</strong>. The version can be rolled back. If you save a job multiple times within a minute, only one version is recorded. If the intermediate data is important, you can click <strong id="dataartsstudio_01_1436__en-us_topic_0181491854_b1476819388105712">Save new version</strong> to save and add a version.</p>
|
|
</p></li></ol>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_01_1436__section2246103584414"><a name="dataartsstudio_01_1436__section2246103584414"></a><a name="section2246103584414"></a><h4 class="sectiontitle">Downloading or Dumping Script Execution Results</h4><div class="p" id="dataartsstudio_01_1436__en-us_topic_0181092880_p14694145254015">After a script is executed successfully, you can download or dump the execution result. By default, all users can download and dump the execution results of SQL scripts. If you do not want all users to have this permission, configure the permission by referring to <a href="dataartsstudio_01_04501.html#dataartsstudio_01_04501__section1970845152011">Configuring a Data Export Policy</a>.<ul id="dataartsstudio_01_1436__en-us_topic_0181092880_ul594619010228"><li id="dataartsstudio_01_1436__en-us_topic_0181092880_li123561599219">After executing a script, you can click <strong id="dataartsstudio_01_1436__en-us_topic_0181092880_b016210101483">Download</strong> on the <strong id="dataartsstudio_01_1436__en-us_topic_0181092880_b216214101812">Result</strong> tab page to download a CSV result file to a local path. You can view the download record on the <a href="dataartsstudio_01_1821.html">Download Center</a> page.</li><li id="dataartsstudio_01_1436__en-us_topic_0181092880_li4357359152119">After executing a script, you can click <strong id="dataartsstudio_01_1436__en-us_topic_0181092880_b16939145414916">Dump</strong> on the <strong id="dataartsstudio_01_1436__en-us_topic_0181092880_b169398541917">Result</strong> tab page to dump a CSV and a JSON result file to OBS. For details, see <a href="#dataartsstudio_01_1436__en-us_topic_0181092880_table1192101552416">Table 6</a>.<div class="note" id="dataartsstudio_01_1436__en-us_topic_0181092880_note94071945142212"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dataartsstudio_01_1436__en-us_topic_0181092880_ul114901431491"><li id="dataartsstudio_01_1436__en-us_topic_0181092880_li5777114884920">The dump function is supported only if the OBS service is available.</li><li id="dataartsstudio_01_1436__en-us_topic_0181092880_li8490174312496">Only the execution results of SQL script query statements can be dumped. </li><li id="dataartsstudio_01_1436__en-us_topic_0181092880_li1064142917164">If the execution result of a download or dump SQL statement contains commas (,), newline characters, or other special characters, data may be disordered, the number of rows may increase, or other issues may occur.</li></ul>
|
|
</div></div>
|
|
|
|
<div class="tablenoborder"><a name="dataartsstudio_01_1436__en-us_topic_0181092880_table1192101552416"></a><a name="en-us_topic_0181092880_table1192101552416"></a><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_01_1436__en-us_topic_0181092880_table1192101552416" frame="border" border="1" rules="all"><caption><b>Table 6 </b>Dump parameters</caption><thead align="left"><tr id="dataartsstudio_01_1436__en-us_topic_0181092880_row15922515192417"><th align="left" class="cellrowborder" valign="top" width="18.17%" id="mcps1.3.8.2.2.2.5.2.4.1.1"><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p29222015142417">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="13.200000000000001%" id="mcps1.3.8.2.2.2.5.2.4.1.2"><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p1892241562411">Mandatory</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="68.63%" id="mcps1.3.8.2.2.2.5.2.4.1.3"><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p792271532416">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dataartsstudio_01_1436__en-us_topic_0181092880_row2922515142417"><td class="cellrowborder" valign="top" width="18.17%" headers="mcps1.3.8.2.2.2.5.2.4.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p4922101518241">Data Format</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="13.200000000000001%" headers="mcps1.3.8.2.2.2.5.2.4.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p1292291511248">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="68.63%" headers="mcps1.3.8.2.2.2.5.2.4.1.3 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p10922101532411">Format of the data to be exported. CSV and JSON formats are supported.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0181092880_row15922115202417"><td class="cellrowborder" valign="top" width="18.17%" headers="mcps1.3.8.2.2.2.5.2.4.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p16922515102419">Resource Queue</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="13.200000000000001%" headers="mcps1.3.8.2.2.2.5.2.4.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p992241502418">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="68.63%" headers="mcps1.3.8.2.2.2.5.2.4.1.3 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p199221715192417"><span id="dataartsstudio_01_1436__en-us_topic_0181092880_text6922415192411">DLI</span> queue where the export operation is to be performed. Set this parameter when a DLI or SQL script is created.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0181092880_row16922215182418"><td class="cellrowborder" valign="top" width="18.17%" headers="mcps1.3.8.2.2.2.5.2.4.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p7922715102418">Compression Format</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="13.200000000000001%" headers="mcps1.3.8.2.2.2.5.2.4.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p5922815182415">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="68.63%" headers="mcps1.3.8.2.2.2.5.2.4.1.3 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p592211153241">Format of compression. Set this parameter when a DLI or SQL script is created.</p>
|
|
<ul id="dataartsstudio_01_1436__en-us_topic_0181092880_ul149231715182410"><li id="dataartsstudio_01_1436__en-us_topic_0181092880_li1392316153244">none</li><li id="dataartsstudio_01_1436__en-us_topic_0181092880_li9923161542411">bzip2</li><li id="dataartsstudio_01_1436__en-us_topic_0181092880_li1592361518245">deflate</li><li id="dataartsstudio_01_1436__en-us_topic_0181092880_li1592331542416">gzip</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0181092880_row10923101510246"><td class="cellrowborder" valign="top" width="18.17%" headers="mcps1.3.8.2.2.2.5.2.4.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p192311522419">Storage Path</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="13.200000000000001%" headers="mcps1.3.8.2.2.2.5.2.4.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p13924191519243">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="68.63%" headers="mcps1.3.8.2.2.2.5.2.4.1.3 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p0924715182413">OBS path where the result file is stored. After selecting an OBS path, customize a folder. Then, the system will create it automatically for storing the result file.</p>
|
|
<p id="dataartsstudio_01_1436__en-us_topic_0181092880_p990561775112">You can also go to the <a href="dataartsstudio_01_1821.html">Download Center</a> page to set the default OBS path, which will be automatically set for <strong id="dataartsstudio_01_1436__en-us_topic_0181092880_b2413115781710">Storage Path</strong> in the <strong id="dataartsstudio_01_1436__en-us_topic_0181092880_b15414155715175">Dump Result</strong> dialog box.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0181092880_row19924101518248"><td class="cellrowborder" valign="top" width="18.17%" headers="mcps1.3.8.2.2.2.5.2.4.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p6924715172415">Cover Type</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="13.200000000000001%" headers="mcps1.3.8.2.2.2.5.2.4.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p14924111522419">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="68.63%" headers="mcps1.3.8.2.2.2.5.2.4.1.3 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p3924191562414">If a folder that has the same name as your custom folder exists in the storage path, select a cover type. Set this parameter when a DLI or SQL script is created.</p>
|
|
<ul id="dataartsstudio_01_1436__en-us_topic_0181092880_ul12924161520245"><li id="dataartsstudio_01_1436__en-us_topic_0181092880_li13924121512240"><strong id="dataartsstudio_01_1436__en-us_topic_0181092880_en-us_topic_0104967365_b0982153513287">Overwrite</strong>: The existing folder will be overwritten by the customized folder.</li><li id="dataartsstudio_01_1436__en-us_topic_0181092880_li1292491552413"><strong id="dataartsstudio_01_1436__en-us_topic_0181092880_en-us_topic_0104967365_b1859416222916">Report</strong>: The system reports an error and suspends the export operation.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0181092880_row973516242714"><td class="cellrowborder" valign="top" width="18.17%" headers="mcps1.3.8.2.2.2.5.2.4.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p973642162714">Export Column Name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="13.200000000000001%" headers="mcps1.3.8.2.2.2.5.2.4.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p11736182182717">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="68.63%" headers="mcps1.3.8.2.2.2.5.2.4.1.3 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p20736322273"><strong id="dataartsstudio_01_1436__en-us_topic_0181092880_b1193109141518">Yes</strong>: Column names will be exported.</p>
|
|
<p id="dataartsstudio_01_1436__en-us_topic_0181092880_p1756182916283"><strong id="dataartsstudio_01_1436__en-us_topic_0181092880_b1597083851520">No</strong>: Column names will not be exported.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0181092880_row28617592714"><td class="cellrowborder" valign="top" width="18.17%" headers="mcps1.3.8.2.2.2.5.2.4.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p08720513276">Character Set</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="13.200000000000001%" headers="mcps1.3.8.2.2.2.5.2.4.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p13871756274">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="68.63%" headers="mcps1.3.8.2.2.2.5.2.4.1.3 "><ul id="dataartsstudio_01_1436__en-us_topic_0181092880_ul981282316294"><li id="dataartsstudio_01_1436__en-us_topic_0181092880_li08120235298"><strong id="dataartsstudio_01_1436__en-us_topic_0181092880_b184370324168">UTF-8</strong>: default character set</li><li id="dataartsstudio_01_1436__en-us_topic_0181092880_li9812142312919"><strong id="dataartsstudio_01_1436__en-us_topic_0181092880_b10628856171616">GB2312</strong>: recommended when the data to be exported contains Chinese character sets</li><li id="dataartsstudio_01_1436__en-us_topic_0181092880_li1581213238292"><strong id="dataartsstudio_01_1436__en-us_topic_0181092880_b141634816182">GBK</strong>: expanded based on and compatible with GB2312</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0181092880_row748933615718"><td class="cellrowborder" valign="top" width="18.17%" headers="mcps1.3.8.2.2.2.5.2.4.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p848919365571">Quotation Character</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="13.200000000000001%" headers="mcps1.3.8.2.2.2.5.2.4.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p54891736135711">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="68.63%" headers="mcps1.3.8.2.2.2.5.2.4.1.3 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p1928923312316">This parameter is available and can be set only when <strong id="dataartsstudio_01_1436__en-us_topic_0181092880_b263866162310">Data Format</strong> is <strong id="dataartsstudio_01_1436__en-us_topic_0181092880_b1563926162313">csv</strong>.</p>
|
|
<p id="dataartsstudio_01_1436__en-us_topic_0181092880_p132810532401">Quotation characters are used to identify the beginning and end of text fields when exporting job results, and are used to separate fields.</p>
|
|
<p id="dataartsstudio_01_1436__en-us_topic_0181092880_p1752471112158">Only one character can be set. The default value is double quotation marks (").</p>
|
|
<p id="dataartsstudio_01_1436__en-us_topic_0181092880_p18721118203114">This is mainly used to handle data that contains spaces, special characters, or characters that are the same as the delimiter.</p>
|
|
<p id="dataartsstudio_01_1436__en-us_topic_0181092880_p4471113410395">For details about the examples of using quotation characters and escape characters, see <a href="dataartsstudio_01_0424.html#dataartsstudio_01_0424__section1729219531331">Example of Using Quotation Characters and Escape Characters</a>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0181092880_row542984014571"><td class="cellrowborder" valign="top" width="18.17%" headers="mcps1.3.8.2.2.2.5.2.4.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p1242915403575">Escape Character</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="13.200000000000001%" headers="mcps1.3.8.2.2.2.5.2.4.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p154291840175715">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="68.63%" headers="mcps1.3.8.2.2.2.5.2.4.1.3 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p142620340233">This parameter is available and can be set only when <strong id="dataartsstudio_01_1436__en-us_topic_0181092880_b6559114502313">Data Format</strong> is <strong id="dataartsstudio_01_1436__en-us_topic_0181092880_b5559945122318">csv</strong>.</p>
|
|
<p id="dataartsstudio_01_1436__en-us_topic_0181092880_p061683714223">If special characters, such as quotation marks, need to be included in the exported results, they can be represented using escape characters (backslash \).</p>
|
|
<p id="dataartsstudio_01_1436__en-us_topic_0181092880_p2064024216443">Only one character can be set. The default value is a backslash (\).</p>
|
|
<p id="dataartsstudio_01_1436__en-us_topic_0181092880_p20662252134220">Common scenarios for using escape characters are:</p>
|
|
<ul id="dataartsstudio_01_1436__en-us_topic_0181092880_ul115001721174310"><li id="dataartsstudio_01_1436__en-us_topic_0181092880_li1350082118438">If there is a third quotation mark between two quotation marks, add an escape character before the third quotation mark to prevent the field content from being split.</li><li id="dataartsstudio_01_1436__en-us_topic_0181092880_li11500122120430">If there is already an escape character in the data content, add another escape character before the existing one to avoid the original character being used as an escape character.</li></ul>
|
|
<p id="dataartsstudio_01_1436__en-us_topic_0181092880_p11317121104014">For details about the examples of using quotation characters and escape characters, see <a href="dataartsstudio_01_0424.html#dataartsstudio_01_0424__section1729219531331">Example of Using Quotation Characters and Escape Characters</a>.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</li></ul>
|
|
</div>
|
|
<div class="p" id="dataartsstudio_01_1436__en-us_topic_0181092880_p2785161544717">Download or dump allows you to view more SQL script execution results. <a href="#dataartsstudio_01_1436__en-us_topic_0181092880_table19855813154916">Table 7</a> lists the maximum number of results that you can view, dump, and downloaded for different types of SQL scripts.
|
|
<div class="tablenoborder"><a name="dataartsstudio_01_1436__en-us_topic_0181092880_table19855813154916"></a><a name="en-us_topic_0181092880_table19855813154916"></a><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_01_1436__en-us_topic_0181092880_table19855813154916" frame="border" border="1" rules="all"><caption><b>Table 7 </b>Maximum number of results that you can view, dump, and download</caption><thead align="left"><tr id="dataartsstudio_01_1436__en-us_topic_0181092880_row1985617132494"><th align="left" class="cellrowborder" valign="top" width="16.8%" id="mcps1.3.8.3.2.2.5.1.1"><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p2252193494914">SQL Type</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="28.860000000000003%" id="mcps1.3.8.3.2.2.5.1.2"><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p1925173484917">Maximum Number of Results That You Can View Online</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="26.669999999999998%" id="mcps1.3.8.3.2.2.5.1.3"><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p1625018342499">Maximum Number/Size of Results That Can Be Downloaded</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="27.67%" id="mcps1.3.8.3.2.2.5.1.4"><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p117193171507">Maximum Number/Size of Results That Can Be Dumped</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dataartsstudio_01_1436__en-us_topic_0181092880_row154391225141410"><td class="cellrowborder" valign="top" width="16.8%" headers="mcps1.3.8.3.2.2.5.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p19246203424913">DLI</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="28.860000000000003%" headers="mcps1.3.8.3.2.2.5.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p1324533414492">1,000</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="26.669999999999998%" headers="mcps1.3.8.3.2.2.5.1.3 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p17244634154918">1,000 records, less than 3MB</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="27.67%" headers="mcps1.3.8.3.2.2.5.1.4 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p644012251140">Unlimited</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0181092880_row198568137498"><td class="cellrowborder" valign="top" width="16.8%" headers="mcps1.3.8.3.2.2.5.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p152481134174918">Hive</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="28.860000000000003%" headers="mcps1.3.8.3.2.2.5.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p13247113419495">1,000</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="26.669999999999998%" headers="mcps1.3.8.3.2.2.5.1.3 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p19246173419490">1,000 records, less than 3MB</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="27.67%" headers="mcps1.3.8.3.2.2.5.1.4 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p147201817105019">10,000 records or 3 MB</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0181092880_row168571513114919"><td class="cellrowborder" valign="top" width="16.8%" headers="mcps1.3.8.3.2.2.5.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p18243103418495">GaussDB(DWS)</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="28.860000000000003%" headers="mcps1.3.8.3.2.2.5.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p13242634144920">1,000</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="26.669999999999998%" headers="mcps1.3.8.3.2.2.5.1.3 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p22411834164915">1,000 records, less than 3MB</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="27.67%" headers="mcps1.3.8.3.2.2.5.1.4 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p57206171507">10,000 records or 3 MB</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0181092880_row178571313204920"><td class="cellrowborder" valign="top" width="16.8%" headers="mcps1.3.8.3.2.2.5.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p1723619347495">Spark</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="28.860000000000003%" headers="mcps1.3.8.3.2.2.5.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p723523413491">1,000</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="26.669999999999998%" headers="mcps1.3.8.3.2.2.5.1.3 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p5233134134918">1,000 records, less than 3MB</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="27.67%" headers="mcps1.3.8.3.2.2.5.1.4 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p172017175508">10,000 records or 3 MB</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0181092880_row18740153061717"><td class="cellrowborder" valign="top" width="16.8%" headers="mcps1.3.8.3.2.2.5.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p15740173031716">RDS</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="28.860000000000003%" headers="mcps1.3.8.3.2.2.5.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p07401730121717">1,000</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="26.669999999999998%" headers="mcps1.3.8.3.2.2.5.1.3 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p47401330131715">1,000 records, less than 3MB</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="27.67%" headers="mcps1.3.8.3.2.2.5.1.4 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p4583134581210">Not supported</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_1436__en-us_topic_0181092880_row4753173120536"><td class="cellrowborder" valign="top" width="16.8%" headers="mcps1.3.8.3.2.2.5.1.1 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p6754163114530">Doris</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="28.860000000000003%" headers="mcps1.3.8.3.2.2.5.1.2 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p9754631185319">1,000</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="26.669999999999998%" headers="mcps1.3.8.3.2.2.5.1.3 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p14755193115315">1,000 records, less than 3MB</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="27.67%" headers="mcps1.3.8.3.2.2.5.1.4 "><p id="dataartsstudio_01_1436__en-us_topic_0181092880_p127554313532">1,000 records or 3 MB</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_01_1436__section5176852104915"><h4 class="sectiontitle">Example of Using Quotation Characters and Escape Characters</h4><ul id="dataartsstudio_01_1436__en-us_topic_0181092880_ul20863159563"><li id="dataartsstudio_01_1436__en-us_topic_0181092880_li1548719308297">Usage of quotation characters and escape characters:<ul id="dataartsstudio_01_1436__en-us_topic_0181092880_ul155093317293"><li id="dataartsstudio_01_1436__en-us_topic_0181092880_li7863791668">Quotation character: used to identify and separate fields. The default value is double quotation marks (").</li><li id="dataartsstudio_01_1436__en-us_topic_0181092880_li5863891466">Escape character: If special characters, such as quotation marks, need to be included in the exported results, they can be represented using escape characters (backslash \). The default value is a backslash (\).<ol id="dataartsstudio_01_1436__en-us_topic_0181092880_ol135201946763"><li id="dataartsstudio_01_1436__en-us_topic_0181092880_li25201346266">To prevent the content of a field from being split when there is a third quotation character between two quotation characters, add an escape character before the third quotation character.</li><li id="dataartsstudio_01_1436__en-us_topic_0181092880_li652015468617">If there is already an escape character in the data content, add another escape character before the existing one to avoid the original character being used as an escape character.</li></ol>
|
|
</li></ul>
|
|
</li><li id="dataartsstudio_01_1436__en-us_topic_0181092880_li1360131431315">Example:<p id="dataartsstudio_01_1436__en-us_topic_0181092880_p136572574279"><a name="dataartsstudio_01_1436__en-us_topic_0181092880_li1360131431315"></a><a name="en-us_topic_0181092880_li1360131431315"></a><span><img id="dataartsstudio_01_1436__en-us_topic_0181092880_image12423103965417" src="en-us_image_0000002269198189.png" title="Click to enlarge" class="imgResize"></span></p>
|
|
<p id="dataartsstudio_01_1436__en-us_topic_0181092880_p993542154513"></p>
|
|
<p id="dataartsstudio_01_1436__en-us_topic_0181092880_p911814545351">You can leave <strong id="dataartsstudio_01_1436__en-us_topic_0181092880_b2009355114291">Quotation Character</strong> and <strong id="dataartsstudio_01_1436__en-us_topic_0181092880_b736996277291">Escape Character</strong> empty.</p>
|
|
<p id="dataartsstudio_01_1436__en-us_topic_0181092880_p2843200143620"><span><img id="dataartsstudio_01_1436__en-us_topic_0181092880_image33201336133216" src="en-us_image_0000002234078892.png" title="Click to enlarge" class="imgResize"></span></p>
|
|
<p id="dataartsstudio_01_1436__en-us_topic_0181092880_p105011728145310"></p>
|
|
<p id="dataartsstudio_01_1436__en-us_topic_0181092880_p557571012282">If you leave them empty, the downloaded .csv file contains two rows in Excel.</p>
|
|
<p id="dataartsstudio_01_1436__en-us_topic_0181092880_p8923846103510"><span><img id="dataartsstudio_01_1436__en-us_topic_0181092880_image172461047143513" src="en-us_image_0000002234078900.png"></span></p>
|
|
<p id="dataartsstudio_01_1436__en-us_topic_0181092880_p16580757133820">If you specify both of them, for example, enter double quotation marks ("), the downloaded file is as follows.</p>
|
|
<p id="dataartsstudio_01_1436__en-us_topic_0181092880_p8844617164413"><span><img id="dataartsstudio_01_1436__en-us_topic_0181092880_image2083313537445" src="en-us_image_0000002234238756.png"></span></p>
|
|
</li></ul>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0431.html">Job Development</a></div>
|
|
</div>
|
|
</div>
|
|
|
|
|
|
<script language="JavaScript">
|
|
<!--
|
|
initImageViewer('.imgResize');
|
|
var msg_imageMax = "view original image";
|
|
var msg_imageClose = "close";
|
|
//--></script> |