forked from docs/doc-exports
Su, Xiaomeng
76a5b1ee83
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: Su, Xiaomeng <suxiaomeng1@huawei.com> Co-committed-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
130 lines
14 KiB
HTML
130 lines
14 KiB
HTML
<a name="dli_08_0249"></a><a name="dli_08_0249"></a>
|
|
|
|
<h1 class="topictitle1">DDS Sink Stream</h1>
|
|
<div id="body1574393104340"><div class="section" id="dli_08_0249__en-us_topic_0132966325_section848675393917"><h4 class="sectiontitle">Function</h4><p id="dli_08_0249__en-us_topic_0132966325_p1691125017526">DLI outputs the job output data to Document Database Service (DDS).</p>
|
|
<p id="dli_08_0249__p05451046134219">DDS is compatible with the MongoDB protocol and is secure, highly available, reliable, scalable, and easy to use. It provides DB instance creation, scaling, redundancy, backup, restoration, monitoring, and alarm reporting functions with just a few clicks on the DDS console.</p>
|
|
<p id="dli_08_0249__p7871212439">For more information about DDS, see the <em id="dli_08_0249__i382819113916">Document Database Service User Guide</em>.</p>
|
|
</div>
|
|
<div class="section" id="dli_08_0249__en-us_topic_0132966325_section1773605853811"><h4 class="sectiontitle">Prerequisites</h4><ul id="dli_08_0249__en-us_topic_0132966325_ul62257516404"><li id="dli_08_0249__en-us_topic_0132966325_li6425436163814">Ensure that you have created a DDS instance on DDS using your account.<p id="dli_08_0249__en-us_topic_0132966325_p823517563510"><a name="dli_08_0249__en-us_topic_0132966325_li6425436163814"></a><a name="en-us_topic_0132966325_li6425436163814"></a>For details about how to create a DDS instance, see <span class="filepath" id="dli_08_0249__en-us_topic_0132966325_filepath1550681242511"><b>Buying a DDS DB Instance</b></span> in the <em id="dli_08_0249__en-us_topic_0132966325_i57313451126">Document Database Service Getting Started</em>.</p>
|
|
</li><li id="dli_08_0249__en-us_topic_0132966325_li964131974210">Currently, only cluster instances with SSL authentication disabled are supported. Replica set and single node instances are not supported.</li><li id="dli_08_0249__en-us_topic_0132966325_li1380511910163">In this scenario, jobs must run on the dedicated queue of DLI. Ensure that the dedicated queue of DLI has been created.</li><li id="dli_08_0249__en-us_topic_0132966325_li19621034131214">Ensure that a datasource connection has been set up between the DLI dedicated queue and the DDS cluster, and security group rules have been configured based on the site requirements.<p id="dli_08_0249__en-us_topic_0132966325_p162011132813"><a name="dli_08_0249__en-us_topic_0132966325_li19621034131214"></a><a name="en-us_topic_0132966325_li19621034131214"></a>For details about how to create an enhanced datasource connection, see <span class="filepath" id="dli_08_0249__en-us_topic_0132966325_filepath13726182511815"><b>Enhanced Datasource Connections</b></span> in the <i><cite id="dli_08_0249__cite534516e564224b13bad1de4471da3139181527">Data Lake Insight User Guide</cite></i>.</p>
|
|
<p id="dli_08_0249__en-us_topic_0132966325_p133355395811">For details about how to configure security group rules, see <span class="filepath" id="dli_08_0249__filepath78235157113"><b>Security Group</b></span> in the <em id="dli_08_0249__i082519153111">Virtual Private Cloud User Guide</em>.</p>
|
|
</li></ul>
|
|
</div>
|
|
<div class="section" id="dli_08_0249__en-us_topic_0132966325_section5292631103113"><h4 class="sectiontitle">Syntax</h4><div class="codecoloring" codetype="Sql" id="dli_08_0249__screen7216237195713"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
|
|
<span class="normal">2</span>
|
|
<span class="normal">3</span>
|
|
<span class="normal">4</span>
|
|
<span class="normal">5</span>
|
|
<span class="normal">6</span>
|
|
<span class="normal">7</span>
|
|
<span class="normal">8</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="n">SINK</span><span class="w"> </span><span class="n">STREAM</span><span class="w"> </span><span class="n">stream_id</span><span class="w"> </span><span class="p">(</span><span class="n">attr_name</span><span class="w"> </span><span class="n">attr_type</span><span class="w"> </span><span class="p">(</span><span class="s1">','</span><span class="w"> </span><span class="n">attr_name</span><span class="w"> </span><span class="n">attr_type</span><span class="p">)</span><span class="o">*</span><span class="w"> </span><span class="p">)</span>
|
|
<span class="w"> </span><span class="k">WITH</span><span class="w"> </span><span class="p">(</span>
|
|
<span class="w"> </span><span class="k">type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">"dds"</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">username</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">""</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">password</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">""</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">db_url</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">""</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">field_names</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">""</span>
|
|
<span class="w"> </span><span class="p">);</span>
|
|
</pre></div></td></tr></table></div>
|
|
|
|
</div>
|
|
</div>
|
|
<div class="section" id="dli_08_0249__section937816374584"><h4 class="sectiontitle">Keywords</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_08_0249__en-us_topic_0132966325_table1648420306385" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Keywords</caption><thead align="left"><tr id="dli_08_0249__en-us_topic_0132966325_row248543016387"><th align="left" class="cellrowborder" valign="top" width="20.03%" id="mcps1.3.4.2.2.4.1.1"><p id="dli_08_0249__en-us_topic_0132966325_p3485130143810">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="9.47%" id="mcps1.3.4.2.2.4.1.2"><p id="dli_08_0249__en-us_topic_0132966325_p168561350102814">Mandatory</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="70.5%" id="mcps1.3.4.2.2.4.1.3"><p id="dli_08_0249__en-us_topic_0132966325_p148513023812">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dli_08_0249__en-us_topic_0132966325_row19485193073810"><td class="cellrowborder" valign="top" width="20.03%" headers="mcps1.3.4.2.2.4.1.1 "><p id="dli_08_0249__en-us_topic_0132966325_p3485330133817">type</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="9.47%" headers="mcps1.3.4.2.2.4.1.2 "><p id="dli_08_0249__en-us_topic_0132966325_p08567508283">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="70.5%" headers="mcps1.3.4.2.2.4.1.3 "><p id="dli_08_0249__en-us_topic_0132966325_p13485123063812">Output channel type. <strong id="dli_08_0249__en-us_topic_0132966325_b9291024182219">dds</strong> indicates that data is exported to DDS.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_0249__en-us_topic_0132966325_row9485143010387"><td class="cellrowborder" valign="top" width="20.03%" headers="mcps1.3.4.2.2.4.1.1 "><p id="dli_08_0249__en-us_topic_0132966325_p1748515305387">username</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="9.47%" headers="mcps1.3.4.2.2.4.1.2 "><p id="dli_08_0249__en-us_topic_0132966325_p1385665012811">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="70.5%" headers="mcps1.3.4.2.2.4.1.3 "><p id="dli_08_0249__en-us_topic_0132966325_p548513301385">Username for connecting to a database.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_0249__en-us_topic_0132966325_row6485163083810"><td class="cellrowborder" valign="top" width="20.03%" headers="mcps1.3.4.2.2.4.1.1 "><p id="dli_08_0249__en-us_topic_0132966325_p1248511308389">password</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="9.47%" headers="mcps1.3.4.2.2.4.1.2 "><p id="dli_08_0249__en-us_topic_0132966325_p78566504284">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="70.5%" headers="mcps1.3.4.2.2.4.1.3 "><p id="dli_08_0249__en-us_topic_0132966325_p148573019388">Password for connecting to a database.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_0249__en-us_topic_0132966325_row648533013820"><td class="cellrowborder" valign="top" width="20.03%" headers="mcps1.3.4.2.2.4.1.1 "><p id="dli_08_0249__en-us_topic_0132966325_p248510306381">db_url</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="9.47%" headers="mcps1.3.4.2.2.4.1.2 "><p id="dli_08_0249__en-us_topic_0132966325_p10856145082816">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="70.5%" headers="mcps1.3.4.2.2.4.1.3 "><p id="dli_08_0249__en-us_topic_0132966325_p163661321115512">DDS instance access address, for example, <strong id="dli_08_0249__en-us_topic_0132966325_b1271217104239">ip1:port,ip2:port/database/collection</strong>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_0249__en-us_topic_0132966325_row2661452184316"><td class="cellrowborder" valign="top" width="20.03%" headers="mcps1.3.4.2.2.4.1.1 "><p id="dli_08_0249__en-us_topic_0132966325_p356311206563">field_names</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="9.47%" headers="mcps1.3.4.2.2.4.1.2 "><p id="dli_08_0249__en-us_topic_0132966325_p17561720125617">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="70.5%" headers="mcps1.3.4.2.2.4.1.3 "><p id="dli_08_0249__en-us_topic_0132966325_p155597202562">Key of the data field to be inserted. The format is <strong id="dli_08_0249__b1735244184912">f1,f2,f3</strong>. Ensure that the key corresponds to the data column in the sink stream.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_0249__en-us_topic_0132966325_row123510295489"><td class="cellrowborder" valign="top" width="20.03%" headers="mcps1.3.4.2.2.4.1.1 "><p id="dli_08_0249__en-us_topic_0132966325_p185581120185611">batch_insert_data_num</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="9.47%" headers="mcps1.3.4.2.2.4.1.2 "><p id="dli_08_0249__en-us_topic_0132966325_p19557132015610">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="70.5%" headers="mcps1.3.4.2.2.4.1.3 "><p id="dli_08_0249__en-us_topic_0132966325_p16555020135610">Amount of data to be written in batches at a time. The value must be a positive integer. The default value is <strong id="dli_08_0249__en-us_topic_0132966325_b842352706193359">10</strong>.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="dli_08_0249__en-us_topic_0132966325_section171023793119"><h4 class="sectiontitle">Example</h4><p id="dli_08_0249__en-us_topic_0132966325_p12360185814444">Output data in the <strong id="dli_08_0249__en-us_topic_0132966325_b7200175817230">qualified_cars</strong> stream to the <strong id="dli_08_0249__en-us_topic_0132966325_b1452211428248">collectionTest</strong> DDS DB.</p>
|
|
<div class="codecoloring" codetype="Sql" id="dli_08_0249__en-us_topic_0132966325_screen15458151114815"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal"> 1</span>
|
|
<span class="normal"> 2</span>
|
|
<span class="normal"> 3</span>
|
|
<span class="normal"> 4</span>
|
|
<span class="normal"> 5</span>
|
|
<span class="normal"> 6</span>
|
|
<span class="normal"> 7</span>
|
|
<span class="normal"> 8</span>
|
|
<span class="normal"> 9</span>
|
|
<span class="normal">10</span>
|
|
<span class="normal">11</span>
|
|
<span class="normal">12</span>
|
|
<span class="normal">13</span>
|
|
<span class="normal">14</span>
|
|
<span class="normal">15</span>
|
|
<span class="normal">16</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="n">SINK</span><span class="w"> </span><span class="n">STREAM</span><span class="w"> </span><span class="n">qualified_cars</span><span class="w"> </span><span class="p">(</span>
|
|
<span class="w"> </span><span class="n">car_id</span><span class="w"> </span><span class="n">STRING</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">car_owner</span><span class="w"> </span><span class="n">STRING</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">car_age</span><span class="w"> </span><span class="nb">INT</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">average_speed</span><span class="w"> </span><span class="nb">INT</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">total_miles</span><span class="w"> </span><span class="nb">INT</span>
|
|
<span class="p">)</span>
|
|
<span class="w"> </span><span class="k">WITH</span><span class="w"> </span><span class="p">(</span>
|
|
<span class="w"> </span><span class="k">type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">"dds"</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">region</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">"xxx"</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">db_url</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">"192.168.0.8:8635,192.168.0.130:8635/dbtest/collectionTest"</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">username</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">"xxxxxxxxxx"</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">password</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">"xxxxxxxxxx"</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">field_names</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">"car_id,car_owner,car_age,average_speed,total_miles"</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">batch_insert_data_num</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">"10"</span>
|
|
<span class="w"> </span><span class="p">);</span>
|
|
</pre></div></td></tr></table></div>
|
|
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_08_0240.html">Creating a Sink Stream</a></div>
|
|
</div>
|
|
</div>
|
|
|