yexinru/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
Files
03e77f3e5b0080047fdcb72efdfd3333dd3bcf92
doc-exports/docs/dws/dev/dws_04_1075.html
luhuayi 177cd61a57 DWS DEVG 910.211 version 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: luhuayi <luhuayi@huawei.com>
Co-committed-by: luhuayi <luhuayi@huawei.com>
2025-05-05 07:44:03 +00:00

57 lines
9.9 KiB
HTML
Raw Blame History

<a name="EN-US_TOPIC_0000001764650240"></a><a name="EN-US_TOPIC_0000001764650240"></a>
<h1 class="topictitle1">Querying a Hudi Foreign Table</h1>
<div id="body0000001541696786"><p id="EN-US_TOPIC_0000001764650240__p8060118">You can query data in a Hudi foreign table. By default, it gives you a real-time view. You can set parameters to query the incremental data.</p>
<div class="section" id="EN-US_TOPIC_0000001764650240__section1751874692118"><h4 class="sectiontitle">Querying Incremental Data</h4><p id="EN-US_TOPIC_0000001764650240__p335024842119">You can set incremental query parameters to implement incremental query.</p>
<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001764650240__screen12259128142113"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">SET</span><span class="w"> </span><span class="n">hoodie</span><span class="p">.</span><span class="k">SCHEMA</span><span class="p">.</span><span class="n">FOREIGN_TABLE</span><span class="p">.</span><span class="n">consume</span><span class="p">.</span><span class="k">mode</span><span class="o">=</span><span class="n">incremental</span><span class="p">;</span>
<span class="k">SET</span><span class="w"> </span><span class="n">hoodie</span><span class="p">.</span><span class="k">SCHEMA</span><span class="p">.</span><span class="n">FOREIGN_TABLE</span><span class="p">.</span><span class="n">consume</span><span class="p">.</span><span class="k">start</span><span class="p">.</span><span class="k">timestamp</span><span class="o">=</span><span class="n">start_timestamp</span><span class="p">;</span>
<span class="k">SET</span><span class="w"> </span><span class="n">hoodie</span><span class="p">.</span><span class="k">SCHEMA</span><span class="p">.</span><span class="n">FOREIGN_TABLE</span><span class="p">.</span><span class="n">consume</span><span class="p">.</span><span class="n">ending</span><span class="p">.</span><span class="k">timestamp</span><span class="o">=</span><span class="n">end_timestamp</span><span class="p">;</span>
<span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="k">SCHEMA</span><span class="p">.</span><span class="n">FOREIGN_TABLE</span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
</div>
<p id="EN-US_TOPIC_0000001764650240__p1260122811215">Example:</p>
<p id="EN-US_TOPIC_0000001764650240__p726042817217">Query the incremental data of the MOR hudi foreign table <strong id="EN-US_TOPIC_0000001764650240__b13295155473310">public.rtd_mfdt_int_currency_ft</strong> from <strong id="EN-US_TOPIC_0000001764650240__b572993183415">20221207164617</strong> to <strong id="EN-US_TOPIC_0000001764650240__b882514643419">20221207170234</strong>. Where:</p>
<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001764650240__screen11214155194410"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">SET</span><span class="w"> </span><span class="n">hoodie</span><span class="p">.</span><span class="k">public</span><span class="p">.</span><span class="n">rtd_mfdt_int_currency_ft</span><span class="p">.</span><span class="n">consume</span><span class="p">.</span><span class="k">mode</span><span class="o">=</span><span class="n">incremental</span><span class="p">;</span>
<span class="k">SET</span><span class="w"> </span><span class="n">hoodie</span><span class="p">.</span><span class="k">public</span><span class="p">.</span><span class="n">rtd_mfdt_int_currency_ft</span><span class="p">.</span><span class="n">consume</span><span class="p">.</span><span class="k">start</span><span class="p">.</span><span class="k">timestamp</span><span class="o">=</span><span class="mi">20221207164617</span><span class="p">;</span>
<span class="k">SET</span><span class="w"> </span><span class="n">hoodie</span><span class="p">.</span><span class="k">public</span><span class="p">.</span><span class="n">rtd_mfdt_int_currency_ft</span><span class="p">.</span><span class="n">consume</span><span class="p">.</span><span class="n">ending</span><span class="p">.</span><span class="k">timestamp</span><span class="o">=</span><span class="mi">20221207170234</span><span class="p">;</span>
<span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="k">public</span><span class="p">.</span><span class="n">rtd_mfdt_int_currency_ft</span><span class="w"> </span><span class="k">where</span><span class="w"> </span><span class="n">_hoodie_commit_time</span><span class="o">&gt;</span><span class="mi">20221207164617</span><span class="w"> </span><span class="k">and</span><span class="w"> </span><span class="n">_hoodie_commit_time</span><span class="o">&lt;=</span><span class="mi">20221207170234</span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
<div class="section" id="EN-US_TOPIC_0000001764650240__section3958184172219"><h4 class="sectiontitle">Querying the Configured Incremental Parameters</h4><div class="p" id="EN-US_TOPIC_0000001764650240__p16369720236">You can use the following function to check the incremental parameter configuration.<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001764650240__screen20361174233"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">pg_show_custom_settings</span><span class="p">();</span>
</pre></div></td></tr></table></div>
</div>
</div>
</div>
<div class="section" id="EN-US_TOPIC_0000001764650240__section14942101819231"><h4 class="sectiontitle">Querying the Properties of a Hudi Foreign Table (hoodie.properties)</h4><div class="p" id="EN-US_TOPIC_0000001764650240__p04749932418">Run the following command to query the <strong id="EN-US_TOPIC_0000001764650240__b644811326399">hoodie.properties</strong> of the Hudi data on OBS:<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001764650240__screen1247499182411"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">hudi_get_options</span><span class="p">(</span><span class="s1">'SCHEMA.FOREIGN_TABLE'</span><span class="p">);</span>
</pre></div></td></tr></table></div>
</div>
</div>
<p id="EN-US_TOPIC_0000001764650240__p11352814192516">Example: Query the hudi properties of the OBS foreign table <strong id="EN-US_TOPIC_0000001764650240__b2892011373">rtd_mfdt_int_unit_ft</strong> in the current schema.</p>
<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001764650240__screen87878433459"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">hudi_get_options</span><span class="p">(</span><span class="s1">'rtd_mfdt_int_unit_ft'</span><span class="p">);</span>
</pre></div></td></tr></table></div>
</div>
</div>
<div class="section" id="EN-US_TOPIC_0000001764650240__section146962035153217"><h4 class="sectiontitle">Querying the Maximum Timeline of a Hudi Foreign Table</h4><p id="EN-US_TOPIC_0000001764650240__p56115343219">Run the following command to query the maximum timeline of the hudi data on OBS, that is, the latest submitted data:</p>
<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001764650240__screen561175333215"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">hudi_get_max_commit</span><span class="p">(</span><span class="s1">'SCHEMA.FOREIGN_TABLE'</span><span class="p">);</span>
</pre></div></td></tr></table></div>
</div>
<p id="EN-US_TOPIC_0000001764650240__p166195312326">Example: Query the maximum timeline of the OBS foreign table <strong id="EN-US_TOPIC_0000001764650240__b1723711482409">rtd_mfdt_int_unit_ft</strong> in the current schema.</p>
<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001764650240__screen1392461054610"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">hudi_get_max_commit</span><span class="p">(</span><span class="s1">'rtd_mfdt_int_unit_ft'</span><span class="p">);</span>
</pre></div></td></tr></table></div>
</div>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dws_04_1069.html">SQL on Hudi</a></div>
</div>
</div>
View Git Blame Copy Permalink