vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/dli/sqlreference/dli_08_0261.html
Su, Xiaomeng 76a5b1ee83 dli_sqlreference_20240227 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
Co-committed-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
2024-03-27 22:02:33 +00:00

219 lines
21 KiB
HTML
Raw Blame History

<a name="dli_08_0261"></a><a name="dli_08_0261"></a>
<h1 class="topictitle1">Creating an RDS Table</h1>
<div id="body1574393121912"><p id="dli_08_0261__p7798335163614">Create an RDS/DWS table to connect to the source stream.</p>
<p id="dli_08_0261__p174595520164">For details about the JOIN syntax, see <a href="dli_08_0106.html">JOIN</a>.</p>
<div class="section" id="dli_08_0261__section149656154515"><h4 class="sectiontitle">Prerequisites</h4><ul id="dli_08_0261__en-us_topic_0111499975_ul810551215398"><li id="dli_08_0261__en-us_topic_0111499975_li6425436163814">Ensure that you have created a PostgreSQL or MySQL RDS instance in RDS.<p id="dli_08_0261__en-us_topic_0111499975_p104771013203812"><a name="dli_08_0261__en-us_topic_0111499975_li6425436163814"></a><a name="en-us_topic_0111499975_li6425436163814"></a>For details about how to create an RDS instance, see <span class="filepath" id="dli_08_0261__filepath102767631292142"><b>Creating an Instance</b></span> in the <em id="dli_08_0261__i25507952392142">Relational Database Service User Guide</em>.</p>
</li><li id="dli_08_0261__en-us_topic_0111499975_li14111191212396">In this scenario, jobs must run on the dedicated queue of DLI. Therefore, DLI must interconnect with the enhanced datasource connection that has been connected with RDS instance. You can also set the security group rules as required.<p id="dli_08_0261__p94091618103916"><a name="dli_08_0261__en-us_topic_0111499975_li14111191212396"></a><a name="en-us_topic_0111499975_li14111191212396"></a>For details about how to create an enhanced datasource connection, see <span class="filepath" id="dli_08_0261__dli_08_0249_en-us_topic_0132966325_filepath13726182511815"><b>Enhanced Datasource Connections</b></span> in the <i><cite id="dli_08_0261__dli_08_0249_cite534516e564224b13bad1de4471da3139181527">Data Lake Insight User Guide</cite></i>.</p>
<p id="dli_08_0261__p11454122131813">For details about how to configure security group rules, see <span class="filepath" id="dli_08_0261__dli_08_0249_filepath78235157113"><b>Security Group</b></span> in the <em id="dli_08_0261__dli_08_0249_i082519153111">Virtual Private Cloud User Guide</em>.</p>
</li></ul>
</div>
<div class="section" id="dli_08_0261__section1691193353316"><h4 class="sectiontitle">Syntax</h4><div class="codecoloring" codetype="Sql" id="dli_08_0261__screen5122423123018"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal"> 1</span>
<span class="normal"> 2</span>
<span class="normal"> 3</span>
<span class="normal"> 4</span>
<span class="normal"> 5</span>
<span class="normal"> 6</span>
<span class="normal"> 7</span>
<span class="normal"> 8</span>
<span class="normal"> 9</span>
<span class="normal">10</span>
<span class="normal">11</span>
<span class="normal">12</span>
<span class="normal">13</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">table_id</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="n">car_id</span><span class="w"> </span><span class="n">STRING</span><span class="p">,</span>
<span class="w"> </span><span class="n">car_owner</span><span class="w"> </span><span class="n">STRING</span><span class="p">,</span>
<span class="w"> </span><span class="n">car_brand</span><span class="w"> </span><span class="n">STRING</span><span class="p">,</span>
<span class="w"> </span><span class="n">car_price</span><span class="w"> </span><span class="nb">INT</span>
<span class="p">)</span>
<span class="w"> </span><span class="k">WITH</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="k">type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;rds&quot;</span><span class="p">,</span>
<span class="w"> </span><span class="n">username</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;&quot;</span><span class="p">,</span>
<span class="w"> </span><span class="n">password</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;&quot;</span><span class="p">,</span>
<span class="w"> </span><span class="n">db_url</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;&quot;</span><span class="p">,</span>
<span class="w"> </span><span class="k">table_name</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;&quot;</span>
<span class="w"> </span><span class="p">);</span>
</pre></div></td></tr></table></div>
</div>
<p id="dli_08_0261__p549285215166"></p>
</div>
<div class="section" id="dli_08_0261__section52043281918"><h4 class="sectiontitle">Keywords</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_08_0261__table115011952171611" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Keywords</caption><thead align="left"><tr id="dli_08_0261__row7534452171619"><th align="left" class="cellrowborder" valign="top" width="16.444444444444446%" id="mcps1.3.5.2.2.4.1.1"><p id="dli_08_0261__p145430522169">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="12.202020202020202%" id="mcps1.3.5.2.2.4.1.2"><p id="dli_08_0261__p955415216164">Mandatory</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="71.35353535353536%" id="mcps1.3.5.2.2.4.1.3"><p id="dli_08_0261__p1056555281615">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_08_0261__row8575105261616"><td class="cellrowborder" valign="top" width="16.444444444444446%" headers="mcps1.3.5.2.2.4.1.1 "><p id="dli_08_0261__p12361417173119">type</p>
</td>
<td class="cellrowborder" valign="top" width="12.202020202020202%" headers="mcps1.3.5.2.2.4.1.2 "><p id="dli_08_0261__p18364176319">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="71.35353535353536%" headers="mcps1.3.5.2.2.4.1.3 "><p id="dli_08_0261__p1346177318">Output channel type. Value <strong id="dli_08_0261__en-us_topic_0111499975_b842352706193629">rds</strong> indicates that data is stored to RDS.</p>
</td>
</tr>
<tr id="dli_08_0261__row11645852191610"><td class="cellrowborder" valign="top" width="16.444444444444446%" headers="mcps1.3.5.2.2.4.1.1 "><p id="dli_08_0261__p163281720316">username</p>
</td>
<td class="cellrowborder" valign="top" width="12.202020202020202%" headers="mcps1.3.5.2.2.4.1.2 "><p id="dli_08_0261__p1932917123113">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="71.35353535353536%" headers="mcps1.3.5.2.2.4.1.3 "><p id="dli_08_0261__p53061718317">Username for connecting to a database.</p>
</td>
</tr>
<tr id="dli_08_0261__row166821052141616"><td class="cellrowborder" valign="top" width="16.444444444444446%" headers="mcps1.3.5.2.2.4.1.1 "><p id="dli_08_0261__p92915173319">password</p>
</td>
<td class="cellrowborder" valign="top" width="12.202020202020202%" headers="mcps1.3.5.2.2.4.1.2 "><p id="dli_08_0261__p11415122419323">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="71.35353535353536%" headers="mcps1.3.5.2.2.4.1.3 "><p id="dli_08_0261__p426417113117">Password for connecting to a database.</p>
</td>
</tr>
<tr id="dli_08_0261__row1871675211610"><td class="cellrowborder" valign="top" width="16.444444444444446%" headers="mcps1.3.5.2.2.4.1.1 "><p id="dli_08_0261__p225131743113">db_url</p>
</td>
<td class="cellrowborder" valign="top" width="12.202020202020202%" headers="mcps1.3.5.2.2.4.1.2 "><p id="dli_08_0261__p1042702413215">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="71.35353535353536%" headers="mcps1.3.5.2.2.4.1.3 "><p id="dli_08_0261__p14859300387">Database connection address, for example, <strong id="dli_08_0261__b26002101492142">{database_type}://ip:port/database</strong>.</p>
<p id="dli_08_0261__p193531452515">Currently, two types of database connections are supported: MySQL and PostgreSQL.</p>
<ul id="dli_08_0261__ul18228175913311"><li id="dli_08_0261__li122812591335">MySQL: 'mysql://ip:port/database'</li><li id="dli_08_0261__li112281159133316">PostgreSQL: 'postgresql://ip:port/database'<div class="note" id="dli_08_0261__note19207195162811"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_08_0261__p62071951142817">To create a DWS dimension table, set the database connection address to a DWS database address. If the DWS database version is later than 8.1.0, the open-source PostgreSQL driver cannot be used for connection. You need to use the GaussDB driver for connection.</p>
</div></div>
</li></ul>
</td>
</tr>
<tr id="dli_08_0261__row177441952131614"><td class="cellrowborder" valign="top" width="16.444444444444446%" headers="mcps1.3.5.2.2.4.1.1 "><p id="dli_08_0261__p17218170316">table_name</p>
</td>
<td class="cellrowborder" valign="top" width="12.202020202020202%" headers="mcps1.3.5.2.2.4.1.2 "><p id="dli_08_0261__p34341024123211">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="71.35353535353536%" headers="mcps1.3.5.2.2.4.1.3 "><p id="dli_08_0261__p418191793112">Indicates the name of the database table for data query.</p>
</td>
</tr>
<tr id="dli_08_0261__row69411319206"><td class="cellrowborder" valign="top" width="16.444444444444446%" headers="mcps1.3.5.2.2.4.1.1 "><p id="dli_08_0261__p18171317143114">db_columns</p>
</td>
<td class="cellrowborder" valign="top" width="12.202020202020202%" headers="mcps1.3.5.2.2.4.1.2 "><p id="dli_08_0261__p1916171716312">No</p>
</td>
<td class="cellrowborder" valign="top" width="71.35353535353536%" headers="mcps1.3.5.2.2.4.1.3 "><p id="dli_08_0261__p810959165112">Indicates the mapping of stream attribute fields between the sink stream and database table. This parameter is mandatory when the stream attribute fields in the sink stream do not match those in the database table. The parameter value is in the format of dbtable_attr1,dbtable_attr2,dbtable_attr3.</p>
</td>
</tr>
<tr id="dli_08_0261__row189061849441"><td class="cellrowborder" valign="top" width="16.444444444444446%" headers="mcps1.3.5.2.2.4.1.1 "><p id="dli_08_0261__p189351517175514">cache_max_num</p>
</td>
<td class="cellrowborder" valign="top" width="12.202020202020202%" headers="mcps1.3.5.2.2.4.1.2 "><p id="dli_08_0261__p1893516176555">No</p>
</td>
<td class="cellrowborder" valign="top" width="71.35353535353536%" headers="mcps1.3.5.2.2.4.1.3 "><p id="dli_08_0261__p29351617135515">Indicates the maximum number of cached query results. The default value is <strong id="dli_08_0261__b151515220092142">32768</strong>.</p>
</td>
</tr>
<tr id="dli_08_0261__row4905049143"><td class="cellrowborder" valign="top" width="16.444444444444446%" headers="mcps1.3.5.2.2.4.1.1 "><p id="dli_08_0261__p1079162475510">cache_time</p>
</td>
<td class="cellrowborder" valign="top" width="12.202020202020202%" headers="mcps1.3.5.2.2.4.1.2 "><p id="dli_08_0261__p1799247551">No</p>
</td>
<td class="cellrowborder" valign="top" width="71.35353535353536%" headers="mcps1.3.5.2.2.4.1.3 "><p id="dli_08_0261__p179524185520">Indicates the maximum duration for caching database query results in the memory. The unit is millisecond. The default value is <strong id="dli_08_0261__b165025644892142">10000</strong>. The value <strong id="dli_08_0261__b33645444492142">0</strong> indicates that caching is disabled.</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="dli_08_0261__section13376151217347"><h4 class="sectiontitle">Example</h4><div class="p" id="dli_08_0261__p88543522162">The RDS table is used to connect to the source stream.<div class="codecoloring" codetype="Sql" id="dli_08_0261__screen68579527168"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal"> 1</span>
<span class="normal"> 2</span>
<span class="normal"> 3</span>
<span class="normal"> 4</span>
<span class="normal"> 5</span>
<span class="normal"> 6</span>
<span class="normal"> 7</span>
<span class="normal"> 8</span>
<span class="normal"> 9</span>
<span class="normal">10</span>
<span class="normal">11</span>
<span class="normal">12</span>
<span class="normal">13</span>
<span class="normal">14</span>
<span class="normal">15</span>
<span class="normal">16</span>
<span class="normal">17</span>
<span class="normal">18</span>
<span class="normal">19</span>
<span class="normal">20</span>
<span class="normal">21</span>
<span class="normal">22</span>
<span class="normal">23</span>
<span class="normal">24</span>
<span class="normal">25</span>
<span class="normal">26</span>
<span class="normal">27</span>
<span class="normal">28</span>
<span class="normal">29</span>
<span class="normal">30</span>
<span class="normal">31</span>
<span class="normal">32</span>
<span class="normal">33</span>
<span class="normal">34</span>
<span class="normal">35</span>
<span class="normal">36</span>
<span class="normal">37</span>
<span class="normal">38</span>
<span class="normal">39</span>
<span class="normal">40</span>
<span class="normal">41</span>
<span class="normal">42</span>
<span class="normal">43</span>
<span class="normal">44</span>
<span class="normal">45</span>
<span class="normal">46</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">SOURCE</span><span class="w"> </span><span class="n">STREAM</span><span class="w"> </span><span class="n">car_infos</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="n">car_id</span><span class="w"> </span><span class="n">STRING</span><span class="p">,</span>
<span class="w"> </span><span class="n">car_owner</span><span class="w"> </span><span class="n">STRING</span><span class="p">,</span>
<span class="w"> </span><span class="n">car_brand</span><span class="w"> </span><span class="n">STRING</span><span class="p">,</span>
<span class="w"> </span><span class="n">car_price</span><span class="w"> </span><span class="nb">INT</span>
<span class="p">)</span>
<span class="w"> </span><span class="k">WITH</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="k">type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;dis&quot;</span><span class="p">,</span>
<span class="w"> </span><span class="n">region</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;&quot;</span><span class="p">,</span>
<span class="w"> </span><span class="n">channel</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;dliinput&quot;</span><span class="p">,</span>
<span class="w"> </span><span class="n">encode</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;csv&quot;</span><span class="p">,</span>
<span class="w"> </span><span class="n">field_delimiter</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;,&quot;</span>
<span class="w"> </span><span class="p">);</span>
<span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">db_info</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="n">car_id</span><span class="w"> </span><span class="n">STRING</span><span class="p">,</span>
<span class="w"> </span><span class="n">car_owner</span><span class="w"> </span><span class="n">STRING</span><span class="p">,</span>
<span class="w"> </span><span class="n">car_brand</span><span class="w"> </span><span class="n">STRING</span><span class="p">,</span>
<span class="w"> </span><span class="n">car_price</span><span class="w"> </span><span class="nb">INT</span>
<span class="p">)</span>
<span class="w"> </span><span class="k">WITH</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="k">type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;rds&quot;</span><span class="p">,</span>
<span class="w"> </span><span class="n">username</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;root&quot;</span><span class="p">,</span>
<span class="w"> </span><span class="n">password</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;******&quot;</span><span class="p">,</span>
<span class="w"> </span><span class="n">db_url</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;postgresql://192.168.0.0:2000/test1&quot;</span><span class="p">,</span>
<span class="w"> </span><span class="k">table_name</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;car&quot;</span>
<span class="p">);</span>
<span class="k">CREATE</span><span class="w"> </span><span class="n">SINK</span><span class="w"> </span><span class="n">STREAM</span><span class="w"> </span><span class="n">audi_cheaper_than_30w</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="n">car_id</span><span class="w"> </span><span class="n">STRING</span><span class="p">,</span>
<span class="w"> </span><span class="n">car_owner</span><span class="w"> </span><span class="n">STRING</span><span class="p">,</span>
<span class="w"> </span><span class="n">car_brand</span><span class="w"> </span><span class="n">STRING</span><span class="p">,</span>
<span class="w"> </span><span class="n">car_price</span><span class="w"> </span><span class="nb">INT</span>
<span class="p">)</span>
<span class="w"> </span><span class="k">WITH</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="k">type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;dis&quot;</span><span class="p">,</span>
<span class="w"> </span><span class="n">region</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;&quot;</span><span class="p">,</span>
<span class="w"> </span><span class="n">channel</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;dlioutput&quot;</span><span class="p">,</span>
<span class="w"> </span><span class="n">partition_key</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;car_owner&quot;</span><span class="p">,</span>
<span class="w"> </span><span class="n">encode</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;csv&quot;</span><span class="p">,</span>
<span class="w"> </span><span class="n">field_delimiter</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;,&quot;</span>
<span class="w"> </span><span class="p">);</span>
<span class="k">INSERT</span><span class="w"> </span><span class="k">INTO</span><span class="w"> </span><span class="n">audi_cheaper_than_30w</span>
<span class="k">SELECT</span><span class="w"> </span><span class="n">a</span><span class="p">.</span><span class="n">car_id</span><span class="p">,</span><span class="w"> </span><span class="n">b</span><span class="p">.</span><span class="n">car_owner</span><span class="p">,</span><span class="w"> </span><span class="n">b</span><span class="p">.</span><span class="n">car_brand</span><span class="p">,</span><span class="w"> </span><span class="n">b</span><span class="p">.</span><span class="n">car_price</span><span class="w"> </span>
<span class="k">FROM</span><span class="w"> </span><span class="n">car_infos</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="k">join</span><span class="w"> </span><span class="n">db_info</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="n">b</span><span class="w"> </span><span class="k">on</span><span class="w"> </span><span class="n">a</span><span class="p">.</span><span class="n">car_id</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">b</span><span class="p">.</span><span class="n">car_id</span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
</div>
<div class="note" id="dli_08_0261__note13797314164316"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_08_0261__p379716147437">To create a DWS dimension table, set the database connection address to a DWS database address. If the DWS database version is later than 8.1.0, the open-source PostgreSQL driver cannot be used for connection. You need to use the GaussDB driver for connection.</p>
</div></div>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_08_0259.html">Creating a Dimension Table</a></div>
</div>
</div>
View Git Blame Copy Permalink