forked from docs/doc-exports
chenxiaoxiong
f9e2808b7c
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: chenxiaoxiong <chenxiaoxiong@huawei.com> Co-committed-by: chenxiaoxiong <chenxiaoxiong@huawei.com>
67 lines
10 KiB
HTML
67 lines
10 KiB
HTML
<a name="dataartsstudio_01_0287"></a><a name="dataartsstudio_01_0287"></a>
|
||
|
||
<h1 class="topictitle1">To MRS ClickHouse</h1>
|
||
<div id="body8662426"><p id="dataartsstudio_01_0287__en-us_topic_0000001377168373_p32842430161426">If the destination link of a job is an <a href="dataartsstudio_01_0285.html">MRS ClickHouse link</a>, configure the destination job parameters based on <a href="#dataartsstudio_01_0287__en-us_topic_0000001377168373_table31823995163953">Table 1</a>.</p>
|
||
<div class="note" id="dataartsstudio_01_0287__en-us_topic_0000001377168373_note15456150184112"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_0287__en-us_topic_0000001377168373_p831512385244">If the source link of the job is an MRS ClickHouse, DWS, or Hive link:</p>
|
||
<ul id="dataartsstudio_01_0287__en-us_topic_0000001377168373_ul10206105610256"><li id="dataartsstudio_01_0287__en-us_topic_0000001377168373_li142071856132518">If the int or float fields are null, set the field type to <strong id="dataartsstudio_01_0287__en-us_topic_0000001377168373_b846841716269">nullable()</strong> when creating an MRS ClickHouse table. Otherwise, the value written to MRS ClickHouse is <strong id="dataartsstudio_01_0287__en-us_topic_0000001377168373_b2630103010261">0</strong>.</li><li id="dataartsstudio_01_0287__en-us_topic_0000001377168373_li1207456202510">Check whether the destination table engine is ReplicatedMergeTree. This engine has a deduplication mechanism, in which the data to be deduplicated cannot be predicted accurately. If this engine is used, ensure that data is unique. Otherwise, non-unique data will be ignored and not written, or ReplicatedMergeTree will be replaced by other types of table engines such as MergeTree.</li></ul>
|
||
</div></div>
|
||
|
||
<div class="tablenoborder"><a name="dataartsstudio_01_0287__en-us_topic_0000001377168373_table31823995163953"></a><a name="en-us_topic_0000001377168373_table31823995163953"></a><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_01_0287__en-us_topic_0000001377168373_table31823995163953" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="dataartsstudio_01_0287__en-us_topic_0000001377168373_row18653487163953"><th align="left" class="cellrowborder" valign="top" width="18.28%" id="mcps1.3.3.2.4.1.1"><p id="dataartsstudio_01_0287__en-us_topic_0000001377168373_p15314298163953">Parameter</p>
|
||
</th>
|
||
<th align="left" class="cellrowborder" valign="top" width="62.34%" id="mcps1.3.3.2.4.1.2"><p id="dataartsstudio_01_0287__en-us_topic_0000001377168373_p32498630163953">Description</p>
|
||
</th>
|
||
<th align="left" class="cellrowborder" valign="top" width="19.380000000000003%" id="mcps1.3.3.2.4.1.3"><p id="dataartsstudio_01_0287__en-us_topic_0000001377168373_p15143370163953">Example Value</p>
|
||
</th>
|
||
</tr>
|
||
</thead>
|
||
<tbody><tr id="dataartsstudio_01_0287__en-us_topic_0000001377168373_row1928353163953"><td class="cellrowborder" valign="top" width="18.28%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0287__en-us_topic_0000001377168373_p28108739161524">Schema/Tablespace</p>
|
||
</td>
|
||
<td class="cellrowborder" valign="top" width="62.34%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0287__en-us_topic_0000001377168373_p17702619155928">Click the icon next to the text box to select a schema or tablespace.</p>
|
||
</td>
|
||
<td class="cellrowborder" valign="top" width="19.380000000000003%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0287__en-us_topic_0000001377168373_p64647157163953">schema</p>
|
||
</td>
|
||
</tr>
|
||
<tr id="dataartsstudio_01_0287__en-us_topic_0000001377168373_row11687830163953"><td class="cellrowborder" valign="top" width="18.28%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0287__en-us_topic_0000001377168373_p23518636161524">Table Name</p>
|
||
</td>
|
||
<td class="cellrowborder" valign="top" width="62.34%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0287__en-us_topic_0000001377168373_p17984995155931">Destination table name.</p>
|
||
<p id="dataartsstudio_01_0287__en-us_topic_0000001377168373_p63592058163953">Click the icon next to the text box. The dialog box for selecting the table is displayed.</p>
|
||
<p id="dataartsstudio_01_0287__en-us_topic_0000001377168373_p1210244910548">This parameter can be configured as a macro variable of date and time and a path name can contain multiple macro variables. When the macro variable of date and time works with a scheduled job, the incremental data can be synchronized periodically. </p>
|
||
<div class="note" id="dataartsstudio_01_0287__en-us_topic_0000001377168373_note45791757134910"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0287__en-us_topic_0000001377168373_en-us_topic_0108275319_p52974484409">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0287__en-us_topic_0000001377168373_en-us_topic_0108275319_text9997118203">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0287__en-us_topic_0000001377168373_en-us_topic_0108275319_i799871152012">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0287__en-us_topic_0000001377168373_en-us_topic_0108275319_i0998101192020">Offset</em>) rather than (<em id="dataartsstudio_01_0287__en-us_topic_0000001377168373_en-us_topic_0108275319_i299821132018">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0287__en-us_topic_0000001377168373_en-us_topic_0108275319_i129981917200">Offset</em>).</p>
|
||
</div></div>
|
||
</td>
|
||
<td class="cellrowborder" valign="top" width="19.380000000000003%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0287__en-us_topic_0000001377168373_p50683068163953">table</p>
|
||
</td>
|
||
</tr>
|
||
<tr id="dataartsstudio_01_0287__en-us_topic_0000001377168373_row43042085151531"><td class="cellrowborder" valign="top" width="18.28%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0287__en-us_topic_0000001377168373_p21324079151542">Clear Data Before Import</p>
|
||
</td>
|
||
<td class="cellrowborder" valign="top" width="62.34%" headers="mcps1.3.3.2.4.1.2 "><div class="p" id="dataartsstudio_01_0287__en-us_topic_0000001377168373_p230515232919">Whether to clear the data in the destination table before data import. The options are as follows:<ul id="dataartsstudio_01_0287__en-us_topic_0000001377168373_ul33051852142915"><li id="dataartsstudio_01_0287__en-us_topic_0000001377168373_li1530565292910"><strong id="dataartsstudio_01_0287__en-us_topic_0000001377168373_b131910562314">Do not clear</strong>: The data in the destination table is not cleared before data import. The imported data is just added to the table.</li><li id="dataartsstudio_01_0287__en-us_topic_0000001377168373_li8305185252910"><strong id="dataartsstudio_01_0287__en-us_topic_0000001377168373_b18868161162314">Clear all data</strong>: All data is cleared from the destination table before data import.</li><li id="dataartsstudio_01_0287__en-us_topic_0000001377168373_li1530545282913"><strong id="dataartsstudio_01_0287__en-us_topic_0000001377168373_b1532931642316">Clear part of data</strong>: Part of the data in the destination table is cleared before data import. If you select <strong id="dataartsstudio_01_0287__en-us_topic_0000001377168373_b17330121618238">Clear part of data</strong>, you must configure <span class="parmname" id="dataartsstudio_01_0287__en-us_topic_0000001377168373_parmname73301316112315"><b>WHERE Clause</b></span> to specify which part will be deleted.</li></ul>
|
||
</div>
|
||
</td>
|
||
<td class="cellrowborder" valign="top" width="19.380000000000003%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0287__en-us_topic_0000001377168373_p1285911311267">Clear part of data</p>
|
||
</td>
|
||
</tr>
|
||
<tr id="dataartsstudio_01_0287__en-us_topic_0000001377168373_row4795289210"><td class="cellrowborder" valign="top" width="18.28%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0287__en-us_topic_0000001377168373_p580615813716">Whether On Cluster</p>
|
||
</td>
|
||
<td class="cellrowborder" valign="top" width="62.34%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0287__en-us_topic_0000001377168373_p15823653195618">This parameter is displayed when <strong id="dataartsstudio_01_0287__en-us_topic_0000001377168373_b973087105720">Clear Data Before Import</strong> is set to <strong id="dataartsstudio_01_0287__en-us_topic_0000001377168373_b23062414573">Clear part of data</strong> or <strong id="dataartsstudio_01_0287__en-us_topic_0000001377168373_b135035377573">Clear all data</strong>. If this parameter is set to <strong id="dataartsstudio_01_0287__en-us_topic_0000001377168373_b015155755710">Yes</strong>, all or part of data on all the nodes in the cluster will be cleared.</p>
|
||
</td>
|
||
<td class="cellrowborder" valign="top" width="19.380000000000003%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0287__en-us_topic_0000001377168373_p148016281921">Yes</p>
|
||
</td>
|
||
</tr>
|
||
<tr id="dataartsstudio_01_0287__en-us_topic_0000001377168373_row11284228135717"><td class="cellrowborder" valign="top" width="18.28%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0287__en-us_topic_0000001377168373_p1628472814572">WHERE Clause</p>
|
||
</td>
|
||
<td class="cellrowborder" valign="top" width="62.34%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0287__en-us_topic_0000001377168373_p4231527102515">If <strong id="dataartsstudio_01_0287__en-us_topic_0000001377168373_b32291025102312">Clear Data Before Import</strong> is set to <strong id="dataartsstudio_01_0287__en-us_topic_0000001377168373_b8229162522310">Clear part of data</strong>, data in the destination table will be deleted based on the WHERE clause after the configuration is complete and before the import starts.</p>
|
||
</td>
|
||
<td class="cellrowborder" valign="top" width="19.380000000000003%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0287__en-us_topic_0000001377168373_p198925356258">age > 18 and age <= 60</p>
|
||
</td>
|
||
</tr>
|
||
</tbody>
|
||
</table>
|
||
</div>
|
||
</div>
|
||
<div>
|
||
<div class="familylinks">
|
||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0061.html">Configuring CDM Destination Job Parameters</a></div>
|
||
</div>
|
||
</div>
|
||
|