vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/dws/dev/dws_04_0265.html
Lu, Huayi e6fa411af0 DWS DEV 830.201 version 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Lu, Huayi <luhuayi@huawei.com>
Co-committed-by: Lu, Huayi <luhuayi@huawei.com>
2024-05-16 07:24:04 +00:00

61 lines
11 KiB
HTML
Raw Blame History

<a name="EN-US_TOPIC_0000001188163652"></a><a name="EN-US_TOPIC_0000001188163652"></a>
<h1 class="topictitle1">Creating a GDS Foreign Table</h1>
<div id="body8662426"><div class="section" id="EN-US_TOPIC_0000001188163652__sb6c038ea4d104257ba56d44d0697211c"><h4 class="sectiontitle">Procedure</h4><ol id="EN-US_TOPIC_0000001188163652__ob04b061e899b4a459cbaeb61d4373f58"><li id="EN-US_TOPIC_0000001188163652__l269bbf077f94485fbec6fa526af7310b"><span>Set the <strong id="EN-US_TOPIC_0000001188163652__b1636814815596">location</strong> parameter for the foreign table based on the path planned in <a href="dws_04_0263.html">Planning Data Export</a>.</span><p><ul id="EN-US_TOPIC_0000001188163652__u13a1ba9375a34a82a244d30f70e1cb77"><li id="EN-US_TOPIC_0000001188163652__ld2edb8497c7f4ef99dabb792c654e944"><strong id="EN-US_TOPIC_0000001188163652__en-us_topic_0117443420_b50455901182733">Remote</strong> mode<div class="p" id="EN-US_TOPIC_0000001188163652__a9e71bbe344844742aa577de84e58ff67">Set the <strong id="EN-US_TOPIC_0000001188163652__en-us_topic_0085031883_b84235270616473">location</strong> parameter to the URL of the directory that stores the data files.<ul id="EN-US_TOPIC_0000001188163652__u2ffa4cb35e8e4823b5cdc91404fe12be"><li id="EN-US_TOPIC_0000001188163652__le44e0f13dd6049c6bcf57534772c202f">You do not need to specify a file name in the URL.</li><li id="EN-US_TOPIC_0000001188163652__l3bc8a38b9c844a92bde257833b2bd5cc">When the number of data sources exported is fewer than the available paths, files are created for the surplus paths without writing any data.</li></ul>
</div>
<p id="EN-US_TOPIC_0000001188163652__a7c7244983b224759a7b9117058196c15">For example:</p>
<p id="EN-US_TOPIC_0000001188163652__a20924e35e2d34373884b165e742b165c">The IP address of the GDS data server is 192.168.0.90. The listening port number set during GDS startup is 5000. The directory for storing data files is <strong id="EN-US_TOPIC_0000001188163652__b116461751194818">/output_data</strong>.</p>
<p id="EN-US_TOPIC_0000001188163652__ae3bcaef4d30c4752bdc2e7af90cfc19c">In this case, set the <strong id="EN-US_TOPIC_0000001188163652__en-us_topic_0085031883_b842352706164814">location</strong> parameter to <strong id="EN-US_TOPIC_0000001188163652__en-us_topic_0085031883_b842352706164819">gsfs://192.168.0.90:5000/</strong>.</p>
<div class="note" id="EN-US_TOPIC_0000001188163652__note1223714021313"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="EN-US_TOPIC_0000001188163652__ul154581019389"><li id="EN-US_TOPIC_0000001188163652__li16545131063814"><strong id="EN-US_TOPIC_0000001188163652__b653465018378">location</strong> can be set to a subdirectory, for example, <strong id="EN-US_TOPIC_0000001188163652__b1153435073720">gsfs://192.168.0.90:5000/2019/11/</strong>, so that the same table can be exported to different directories by date.</li><li id="EN-US_TOPIC_0000001188163652__li5545161033815">In the current version, when an <strong id="EN-US_TOPIC_0000001188163652__b1328765512544">export</strong> task is executed, the system checks whether the <strong id="EN-US_TOPIC_0000001188163652__b9440172115511">/output_data/2019/11</strong> directory exists. If the directory does not exist, the system creates it. During the export, files are written to this directory. In this way, you do not need to manually run the <strong id="EN-US_TOPIC_0000001188163652__b975213117560">mkdir -p /output_data/2019/11</strong> command after creating or modifying a foreign table.</li></ul>
</div></div>
</li></ul>
</p></li><li id="EN-US_TOPIC_0000001188163652__li166411122912"><span>Set data format parameters in the foreign table based on the planned data file formats. For details about format parameters, see data format parameters.</span></li><li id="EN-US_TOPIC_0000001188163652__l3e1f6c916ea74f329b3cb9ddb3854f3d"><span>Create a GDS foreign table based on the parameter settings in the preceding steps. For details about how to create a foreign table, see CREATE FOREIGN TABLE (for GDS Import and Export).</span></li></ol>
</div>
<div class="section" id="EN-US_TOPIC_0000001188163652__sd142b490bc4a49c1a5aa819b99c233b5"><h4 class="sectiontitle">Example</h4><ul id="EN-US_TOPIC_0000001188163652__uf217e50c39df43d2b45cc2cdebe94d97"><li id="EN-US_TOPIC_0000001188163652__l40a428c8886d4b2b90d4b9adff37e79b"><strong id="EN-US_TOPIC_0000001188163652__b162216291117">Example</strong>: Create the GDS foreign table <strong id="EN-US_TOPIC_0000001188163652__b962292912113">foreign_tpcds_reasons</strong> for the source data. Data is to be exported as CSV files.<p id="EN-US_TOPIC_0000001188163652__a79af719de77f49ffbd16255a57090b5b">Data export mode settings are as follows:</p>
<p id="EN-US_TOPIC_0000001188163652__ad8bd9a10399a4cc29951693389c0eb6d">The data server resides on the same intranet as the cluster. The IP address of the data server is 192.168.0.90. Data is to be exported as CSV files. The <strong id="EN-US_TOPIC_0000001188163652__b1419611159714">Remote</strong> mode is selected for parallel data export.</p>
<p id="EN-US_TOPIC_0000001188163652__a8ae7a2de86554665865ea2620d4612ec">Assume that the directory for storing data files is <strong id="EN-US_TOPIC_0000001188163652__b842352706165544">/output_data/</strong> and the GDS listening port is 5000 when GDS is started. Therefore, the <strong id="EN-US_TOPIC_0000001188163652__b842352706165555">location</strong> parameter is set to <strong id="EN-US_TOPIC_0000001188163652__b84235270616561">gsfs://192.168.0.90:5000/</strong>.</p>
<p id="EN-US_TOPIC_0000001188163652__ade5787e614d644d2a11c795a848b55ca">Data format parameter settings are as follows:</p>
<ul id="EN-US_TOPIC_0000001188163652__u8469b25260f749778d5f7053d5257e05"><li id="EN-US_TOPIC_0000001188163652__lad48220ae4724ae9aeb3cb7a017855fc"><strong id="EN-US_TOPIC_0000001188163652__b842352706165627">format</strong> is set to <strong id="EN-US_TOPIC_0000001188163652__b842352706165631">CSV</strong>.</li><li id="EN-US_TOPIC_0000001188163652__lb63983d208594b72a79610d2032e27fe"><strong id="EN-US_TOPIC_0000001188163652__b842352706143434">encoding</strong> is set to <strong id="EN-US_TOPIC_0000001188163652__b842352706161148">UTF-8</strong>.</li><li id="EN-US_TOPIC_0000001188163652__ld35611b1c8a6474aba52cffba76dd4fd"><strong id="EN-US_TOPIC_0000001188163652__b203601326243">delimiter</strong> is set to <strong id="EN-US_TOPIC_0000001188163652__b236114261413">E'\x08'</strong>.</li><li id="EN-US_TOPIC_0000001188163652__l1b2b670a4b214b84b528160da5b9171d"><strong id="EN-US_TOPIC_0000001188163652__b842352706165714">quote</strong> is set to <strong id="EN-US_TOPIC_0000001188163652__b842352706165717">E'\x1b'</strong>.</li><li id="EN-US_TOPIC_0000001188163652__lacfa868b047c4e8b8df8670ab6cfc4b3"><strong id="EN-US_TOPIC_0000001188163652__b842352706165729">null</strong> is set to an empty string without quotation marks.</li><li id="EN-US_TOPIC_0000001188163652__lad4e684aa2484af88440f51e0aba7e61"><strong id="EN-US_TOPIC_0000001188163652__b41714265341">escape</strong> is set to the same value as that of <strong id="EN-US_TOPIC_0000001188163652__b19176126203410">quote</strong> by default.</li><li id="EN-US_TOPIC_0000001188163652__l784d9d6163644ab9b3608e57f292abcc"><strong id="EN-US_TOPIC_0000001188163652__b842352706165753">header</strong> is set to <strong id="EN-US_TOPIC_0000001188163652__b842352706171243">false</strong>, indicating that the first row is identified as a data row in an exported file.</li><li id="EN-US_TOPIC_0000001188163652__l0d718ba9a7624dd1af983dc18e4b3df5"><strong id="EN-US_TOPIC_0000001188163652__b842352706165812">EOL</strong> is set to <strong id="EN-US_TOPIC_0000001188163652__b842352706165817">0X0A</strong>.</li></ul>
<div class="p" id="EN-US_TOPIC_0000001188163652__a2937dea056934d2d93d48d57dba53e7c">The foreign table is created using the following statement:<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188163652__s7629129ccb4d4b11925230261ac93012"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal"> 1</span>
<span class="normal"> 2</span>
<span class="normal"> 3</span>
<span class="normal"> 4</span>
<span class="normal"> 5</span>
<span class="normal"> 6</span>
<span class="normal"> 7</span>
<span class="normal"> 8</span>
<span class="normal"> 9</span>
<span class="normal">10</span>
<span class="normal">11</span>
<span class="normal">12</span>
<span class="normal">13</span>
<span class="normal">14</span>
<span class="normal">15</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">FOREIGN</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">foreign_tpcds_reasons</span>
<span class="p">(</span>
<span class="w"> </span><span class="n">r_reason_sk</span><span class="w"> </span><span class="nb">integer</span><span class="w"> </span><span class="k">not</span><span class="w"> </span><span class="k">null</span><span class="p">,</span>
<span class="w"> </span><span class="n">r_reason_id</span><span class="w"> </span><span class="nb">char</span><span class="p">(</span><span class="mi">16</span><span class="p">)</span><span class="w"> </span><span class="k">not</span><span class="w"> </span><span class="k">null</span><span class="p">,</span>
<span class="w"> </span><span class="n">r_reason_desc</span><span class="w"> </span><span class="nb">char</span><span class="p">(</span><span class="mi">100</span><span class="p">)</span>
<span class="p">)</span><span class="w"> </span>
<span class="n">SERVER</span><span class="w"> </span><span class="n">gsmpp_server</span><span class="w"> </span>
<span class="k">OPTIONS</span><span class="w"> </span><span class="p">(</span><span class="k">LOCATION</span><span class="w"> </span><span class="s1">'gsfs://192.168.0.90:5000/'</span><span class="p">,</span><span class="w"> </span>
<span class="n">FORMAT</span><span class="w"> </span><span class="s1">'CSV'</span><span class="p">,</span>
<span class="k">DELIMITER</span><span class="w"> </span><span class="n">E</span><span class="s1">'\x08'</span><span class="p">,</span>
<span class="n">QUOTE</span><span class="w"> </span><span class="n">E</span><span class="s1">'\x1b'</span><span class="p">,</span><span class="w"> </span>
<span class="k">NULL</span><span class="w"> </span><span class="s1">''</span><span class="p">,</span><span class="w"> </span>
<span class="n">EOL</span><span class="w"> </span><span class="s1">'0x0a'</span>
<span class="p">)</span>
<span class="k">WRITE</span><span class="w"> </span><span class="k">ONLY</span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
</div>
</li></ul>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dws_04_0261.html">Using GDS to Export Data to a Remote Server</a></div>
</div>
</div>
View Git Blame Copy Permalink