forked from docs/doc-exports
chenxiaoxiong
f9e2808b7c
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: chenxiaoxiong <chenxiaoxiong@huawei.com> Co-committed-by: chenxiaoxiong <chenxiaoxiong@huawei.com>
63 lines
11 KiB
HTML
63 lines
11 KiB
HTML
<a name="dataartsstudio_01_0524"></a><a name="dataartsstudio_01_0524"></a>
|
|
|
|
<h1 class="topictitle1">Developing a DWS SQL Job</h1>
|
|
<div id="body0000001188180391"><p id="dataartsstudio_01_0524__en-us_topic_0127305016_p96913494251">This section describes how to use the DWS SQL node to develop a job in <span id="dataartsstudio_01_0524__text19943195755">DataArts Factory</span>.</p>
|
|
<div class="section" id="dataartsstudio_01_0524__en-us_topic_0127305016_section722114374269"><h4 class="sectiontitle">Scenario</h4><p id="dataartsstudio_01_0524__en-us_topic_0127305016_p920612407261">This tutorial describes how to develop a DWS job to collect the sales volume of a store on the previous day.</p>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_01_0524__en-us_topic_0127305016_section8658103210277"><h4 class="sectiontitle">Preparing the Environment</h4><ul id="dataartsstudio_01_0524__ul18791114123911"><li id="dataartsstudio_01_0524__li77921545398">Enable DWS and create a DWS cluster for running DWS SQL jobs.</li><li id="dataartsstudio_01_0524__li879294153919">Enable a CDM incremental package. Create a CDM cluster.<p id="dataartsstudio_01_0524__p15328113013112"><a name="dataartsstudio_01_0524__li879294153919"></a><a name="li879294153919"></a>Ensure that the VPC, subnet, and security group of the CDM cluster are the same as those of the DWS cluster so that the two clusters can communicate with each other.</p>
|
|
</li></ul>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_01_0524__en-us_topic_0127305016_section1033111569439"><a name="dataartsstudio_01_0524__en-us_topic_0127305016_section1033111569439"></a><a name="en-us_topic_0127305016_section1033111569439"></a><h4 class="sectiontitle">Creating a DWS Data Connection</h4><p id="dataartsstudio_01_0524__en-us_topic_0127305016_p1728321110444">Before developing a DWS SQL job, you must create a data connection to DWS on the <strong id="dataartsstudio_01_0524__b19516195213123">Manage Data Connections</strong> page of <strong id="dataartsstudio_01_0524__b19516135210126">Management Center</strong>. The data connection name is <span class="parmvalue" id="dataartsstudio_01_0524__parmvalue115161652161213"><b>dws_link</b></span>. For how to create a DWS connections, see <a href="dataartsstudio_01_1300.html">DWS Connection Parameters</a>.</p>
|
|
<p id="dataartsstudio_01_0524__en-us_topic_0127305016_p368212115810">The key parameters are as follows:</p>
|
|
<ul id="dataartsstudio_01_0524__en-us_topic_0127305016_ul22901629175811"><li id="dataartsstudio_01_0524__en-us_topic_0127305016_li4290152995812"><strong id="dataartsstudio_01_0524__b880215458293">Cluster Name</strong>: Select the DWS cluster you have created when preparing the environment.</li><li id="dataartsstudio_01_0524__en-us_topic_0127305016_li756017255599"><strong id="dataartsstudio_01_0524__b287161813113">Agent</strong>: Select the CDM cluster you have created when preparing the environment.</li></ul>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_01_0524__section1318945519560"><h4 class="sectiontitle">Creating a Database</h4><p id="dataartsstudio_01_0524__p136458169578">Create a <strong id="dataartsstudio_01_0524__b2967101823519">gaussdb</strong> database by following the instructions in <a href="dataartsstudio_01_0405.html">Creating a Database</a>.</p>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_01_0524__section19115188573"><h4 class="sectiontitle">Creating Data Tables</h4><p id="dataartsstudio_01_0524__p8295191310576">Create tables <strong id="dataartsstudio_01_0524__b509197864246">trade_log</strong> and <strong id="dataartsstudio_01_0524__b9419979464246">trade_report</strong> in the <strong id="dataartsstudio_01_0524__b9808823516">gaussdb</strong> database. The following is an example script for creating the tables:</p>
|
|
</div>
|
|
<pre class="screen" id="dataartsstudio_01_0524__screen1327218301462">create schema store_sales;
|
|
set current_schema= store_sales;
|
|
drop table if exists trade_log;
|
|
CREATE TABLE trade_log
|
|
(
|
|
sn VARCHAR(16),
|
|
trade_time DATE,
|
|
trade_count INTEGER(8)
|
|
);
|
|
set current_schema= store_sales;
|
|
drop table if exists trade_report;
|
|
CREATE TABLE trade_report
|
|
(
|
|
rq DATE,
|
|
trade_total INTEGER(8)
|
|
);</pre>
|
|
<div class="section" id="dataartsstudio_01_0524__en-us_topic_0127305016_section17888155820591"><a name="dataartsstudio_01_0524__en-us_topic_0127305016_section17888155820591"></a><a name="en-us_topic_0127305016_section17888155820591"></a><h4 class="sectiontitle">Developing a DWS SQL Script</h4><p id="dataartsstudio_01_0524__en-us_topic_0127305016_p442265564612">Choose <strong id="dataartsstudio_01_0524__b814914715383">Development</strong> > <strong id="dataartsstudio_01_0524__b191558479384">Develop Script</strong> and create a DWS SQL script named <strong id="dataartsstudio_01_0524__b9155144716388">dws_sql</strong>. Enter an SQL statement in the editor to collect the sales amount of the previous day.</p>
|
|
<div class="fignone" id="dataartsstudio_01_0524__en-us_topic_0127305016_fig693875618223"><a name="dataartsstudio_01_0524__en-us_topic_0127305016_fig693875618223"></a><a name="en-us_topic_0127305016_fig693875618223"></a><span class="figcap"><b>Figure 1 </b>Developing a script</span><br><span><img id="dataartsstudio_01_0524__image573112011189" src="en-us_image_0000002234238460.png" title="Click to enlarge" class="imgResize"></span></div>
|
|
<p id="dataartsstudio_01_0524__en-us_topic_0127305016_p720511441417">Key notes:</p>
|
|
<ul id="dataartsstudio_01_0524__en-us_topic_0127305016_ul173451537418"><li id="dataartsstudio_01_0524__en-us_topic_0127305016_li12291173894212">The script development area in <a href="#dataartsstudio_01_0524__en-us_topic_0127305016_fig693875618223">Figure 1</a> is a temporary debugging area. After you close the script tab, the development area will be cleared. You can click <strong id="dataartsstudio_01_0524__b71487134020">Submit</strong> to save and submit a script version.</li><li id="dataartsstudio_01_0524__en-us_topic_0127305016_li5345753144114"><strong id="dataartsstudio_01_0524__b176554247407">Connection</strong>: Select the data connection created in <a href="#dataartsstudio_01_0524__en-us_topic_0127305016_section1033111569439">Creating a DWS Data Connection</a>.</li></ul>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_01_0524__section8560636173110"><h4 class="sectiontitle">Developing a DWS SQL Job</h4><p id="dataartsstudio_01_0524__en-us_topic_0127305016_p161077163440">After developing the DWS SQL script, create a job for periodically executing the DWS SQL script.</p>
|
|
<ol id="dataartsstudio_01_0524__en-us_topic_0127305016_ol127703343322"><li id="dataartsstudio_01_0524__en-us_topic_0127305016_li171929505617"><span>Create a batch job named <span class="parmvalue" id="dataartsstudio_01_0524__en-us_topic_0127305016_parmvalue07227285255"><b>job_dws_sql</b></span>.</span></li><li id="dataartsstudio_01_0524__en-us_topic_0127305016_li1340320144265"><span>Go to the job development page, drag the DWS SQL node to the canvas, and click the node to configure its properties.</span><p><div class="fignone" id="dataartsstudio_01_0524__en-us_topic_0127305016_fig1736311111259"><span class="figcap"><b>Figure 2 </b>Configuring properties for the DWS SQL node</span><br><span><img id="dataartsstudio_01_0524__image1372115812110" src="en-us_image_0000002234238444.png" title="Click to enlarge" class="imgResize"></span></div>
|
|
<p id="dataartsstudio_01_0524__en-us_topic_0127305016_p7797149164316">Key properties:</p>
|
|
<ul id="dataartsstudio_01_0524__en-us_topic_0127305016_ul679955184413"><li id="dataartsstudio_01_0524__en-us_topic_0127305016_li107992054445"><strong id="dataartsstudio_01_0524__b10856111615557">SQL script</strong>: Associate with the <span class="parmvalue" id="dataartsstudio_01_0524__en-us_topic_0127305016_parmvalue108540513528"><b>dws_sql</b></span> script developed in <a href="#dataartsstudio_01_0524__en-us_topic_0127305016_section17888155820591">Developing a DWS SQL Script</a>.</li><li id="dataartsstudio_01_0524__en-us_topic_0127305016_li176364615511"><strong id="dataartsstudio_01_0524__b5901712195714">Data Connection</strong>: Select the data connection configured in the <span class="parmvalue" id="dataartsstudio_01_0524__en-us_topic_0127305016_parmvalue1347790155613"><b>dws_sql</b></span> script. The data connection can be changed.</li><li id="dataartsstudio_01_0524__en-us_topic_0127305016_li174789611565"><strong id="dataartsstudio_01_0524__b91161984582">Database</strong>: Select the database configured in the <span class="parmvalue" id="dataartsstudio_01_0524__en-us_topic_0127305016_parmvalue8759112817562"><b>dws_sql</b></span> script. The database can be changed.</li><li id="dataartsstudio_01_0524__li113919456713"><strong id="dataartsstudio_01_0524__b35081650175819">Script Parameter</strong>: Obtain the value of <strong id="dataartsstudio_01_0524__b17816815145913">yesterday</strong> using the following EL expression:<pre class="screen" id="dataartsstudio_01_0524__screen2216238104">#{Job.getYesterday("yyyy-MM-dd")}</pre>
|
|
</li><li id="dataartsstudio_01_0524__en-us_topic_0127305016_li1155613339568"><strong id="dataartsstudio_01_0524__b8558143815593">Node Name</strong>: The name of the <span class="parmvalue" id="dataartsstudio_01_0524__en-us_topic_0127305016_parmvalue26051152105616"><b>dws_sql</b></span> script is displayed by default. The name can be changed.</li></ul>
|
|
</p></li><li id="dataartsstudio_01_0524__en-us_topic_0127305016_li2067134781615"><span>After configuring the job, click <span><img id="dataartsstudio_01_0524__image1953015565143" src="en-us_image_0000002269197897.png"></span> to test it.</span></li><li id="dataartsstudio_01_0524__en-us_topic_0127305016_li23861329209"><span>If the test is successful, click the blank area on the canvas and then the <strong id="dataartsstudio_01_0524__b10521507294246">Scheduling Setup</strong> tab on the right. On the displayed page, configure the scheduling policy.</span><p><div class="fignone" id="dataartsstudio_01_0524__en-us_topic_0127305016_fig18819172118"><span class="figcap"><b>Figure 3 </b>Configuring the scheduling policy</span><br><span><img id="dataartsstudio_01_0524__image919610407453" src="en-us_image_0000002269197901.png" title="Click to enlarge" class="imgResize"></span></div>
|
|
<p id="dataartsstudio_01_0524__en-us_topic_0127305016_p20815183910257"></p>
|
|
<p id="dataartsstudio_01_0524__en-us_topic_0127305016_p188221245123318">Parameter descriptions:</p>
|
|
<p id="dataartsstudio_01_0524__p17140559201614">From Aug 6 to Aug 31 in 2021, the job was executed once at 02:00 every day.</p>
|
|
</p></li><li id="dataartsstudio_01_0524__en-us_topic_0127305016_li849302173615"><span>Click <strong id="dataartsstudio_01_0524__b134819351547">Submit</strong> and then <strong id="dataartsstudio_01_0524__b13273491747">Execute</strong>. The job will be executed automatically every day.</span></li></ol>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0520.html">Usage Guidance</a></div>
|
|
</div>
|
|
</div>
|
|
|
|
|
|
<script language="JavaScript">
|
|
<!--
|
|
initImageViewer('.imgResize');
|
|
var msg_imageMax = "view original image";
|
|
var msg_imageClose = "close";
|
|
//--></script> |