yexinru/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
Files
a41a4e0331fbc0beff63bb28fa59228cc9a751df
doc-exports/docs/dataartsstudio/umn/dataartsstudio_01_0064.html
chenxiaoxiong f9e2808b7c DataArts UMN 20250810 version 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: chenxiaoxiong <chenxiaoxiong@huawei.com>
Co-committed-by: chenxiaoxiong <chenxiaoxiong@huawei.com>
2025-09-02 10:44:13 +00:00

90 lines
13 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<a name="dataartsstudio_01_0064"></a><a name="dataartsstudio_01_0064"></a>
<h1 class="topictitle1">To HBase/CloudTable</h1>
<div id="body8662426"><p id="dataartsstudio_01_0064__en-us_topic_0108275309_p21313215367">If the destination link of a job is an <a href="dataartsstudio_01_0039.html">HBase link</a> or <a href="dataartsstudio_01_0027.html">CloudTable link</a>, configure the destination job parameters based on <a href="#dataartsstudio_01_0064__en-us_topic_0108275309_table5046103815165">Table 1</a>.</p>
<div class="tablenoborder"><a name="dataartsstudio_01_0064__en-us_topic_0108275309_table5046103815165"></a><a name="en-us_topic_0108275309_table5046103815165"></a><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_01_0064__en-us_topic_0108275309_table5046103815165" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="dataartsstudio_01_0064__en-us_topic_0108275309_row585315215165"><th align="left" class="cellrowborder" valign="top" width="18.17%" id="mcps1.3.2.2.4.1.1"><p id="dataartsstudio_01_0064__en-us_topic_0108275309_p1626397215165">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="61.83%" id="mcps1.3.2.2.4.1.2"><p id="dataartsstudio_01_0064__en-us_topic_0108275309_p4231334915165">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="20%" id="mcps1.3.2.2.4.1.3"><p id="dataartsstudio_01_0064__en-us_topic_0108275309_p482921015165">Example Value</p>
</th>
</tr>
</thead>
<tbody><tr id="dataartsstudio_01_0064__en-us_topic_0108275309_row4012116315165"><td class="cellrowborder" valign="top" width="18.17%" headers="mcps1.3.2.2.4.1.1 "><p id="dataartsstudio_01_0064__en-us_topic_0108275309_p2858877215165">Table Name</p>
</td>
<td class="cellrowborder" valign="top" width="61.83%" headers="mcps1.3.2.2.4.1.2 "><p id="dataartsstudio_01_0064__en-us_topic_0108275309_p3398923015165">Name of the HBase table to which data will be written. If you want to create an HBase table, you can copy the field names from the migration source. Click the icon next to the text box. The dialog box for selecting the table is displayed.</p>
<p id="dataartsstudio_01_0064__en-us_topic_0108275309_p1210244910548">This parameter can be configured as a macro variable of date and time and a path name can contain multiple macro variables. When the macro variable of date and time works with a scheduled job, the incremental data can be synchronized periodically. </p>
<div class="note" id="dataartsstudio_01_0064__en-us_topic_0108275309_note45791757134910"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0064__en-us_topic_0108275309_en-us_topic_0108275319_p52974484409">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0064__en-us_topic_0108275309_en-us_topic_0108275319_text9997118203">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0064__en-us_topic_0108275309_en-us_topic_0108275319_i799871152012">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0064__en-us_topic_0108275309_en-us_topic_0108275319_i0998101192020">Offset</em>) rather than (<em id="dataartsstudio_01_0064__en-us_topic_0108275309_en-us_topic_0108275319_i299821132018">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0064__en-us_topic_0108275309_en-us_topic_0108275319_i129981917200">Offset</em>).</p>
</div></div>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.2.2.4.1.3 "><p id="dataartsstudio_01_0064__en-us_topic_0108275309_p166427315165">TBL_2</p>
</td>
</tr>
<tr id="dataartsstudio_01_0064__en-us_topic_0108275309_row14607720151241"><td class="cellrowborder" valign="top" width="18.17%" headers="mcps1.3.2.2.4.1.1 "><p id="dataartsstudio_01_0064__en-us_topic_0108275309_p9799599151241">Clear Data Before Import</p>
</td>
<td class="cellrowborder" valign="top" width="61.83%" headers="mcps1.3.2.2.4.1.2 "><div class="p" id="dataartsstudio_01_0064__en-us_topic_0108275309_p49350536142510">Whether the data in the destination table is cleared before data import. The options are as follows:<ul id="dataartsstudio_01_0064__en-us_topic_0108275309_en-us_topic_0108275481_ua1819b93796044cd9f46f9dfb3e78f6b"><li id="dataartsstudio_01_0064__en-us_topic_0108275309_en-us_topic_0108275481_l6ab161004da642a19cc2c8de077ee24e"><strong id="dataartsstudio_01_0064__en-us_topic_0108275309_en-us_topic_0108275481_b842352706155658">Yes</strong>: The data is cleared.</li><li id="dataartsstudio_01_0064__en-us_topic_0108275309_en-us_topic_0108275481_l8659f1f4120241ee86b1c992eed6a627"><strong id="dataartsstudio_01_0064__en-us_topic_0108275309_en-us_topic_0108275481_b842352706155739">No</strong>: The data is not cleared. Instead, it will be added to the existing table.</li></ul>
</div>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.2.2.4.1.3 "><p id="dataartsstudio_01_0064__en-us_topic_0108275309_p4879287151241">Yes</p>
</td>
</tr>
<tr id="dataartsstudio_01_0064__en-us_topic_0108275309_row456095318558"><td class="cellrowborder" valign="top" width="18.17%" headers="mcps1.3.2.2.4.1.1 "><p id="dataartsstudio_01_0064__en-us_topic_0108275309_p63059528297">Auto Table Creation</p>
</td>
<td class="cellrowborder" valign="top" width="61.83%" headers="mcps1.3.2.2.4.1.2 "><div class="p" id="dataartsstudio_01_0064__en-us_topic_0108275309_p63051752162912">This parameter is displayed only when the source is a relational database. The options are as follows:<ul id="dataartsstudio_01_0064__en-us_topic_0108275309_ul1730585217292"><li id="dataartsstudio_01_0064__en-us_topic_0108275309_li19305135216298"><strong id="dataartsstudio_01_0064__en-us_topic_0108275309_b44220465811">Non-auto creation</strong>: CDM will not automatically create a table.</li><li id="dataartsstudio_01_0064__en-us_topic_0108275309_li193052523291"><strong id="dataartsstudio_01_0064__en-us_topic_0108275309_b18160157125812">Auto creation</strong>: If the destination database does not contain the table specified by <span class="parmname" id="dataartsstudio_01_0064__en-us_topic_0108275309_parmname416020795820"><b>Table Name</b></span>, CDM will automatically create the table. If the table specified by <span class="parmname" id="dataartsstudio_01_0064__en-us_topic_0108275309_parmname1455129155819"><b>Table Name</b></span> already exists, no table is created and data is written to the existing table.<div class="note" id="dataartsstudio_01_0064__en-us_topic_0108275309_note12473622171920"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0064__en-us_topic_0108275309_p747422271913">The automatically created HBase table contains the column family and coprocessor information. For other attributes, default values are retained.</p>
</div></div>
</li></ul>
</div>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.2.2.4.1.3 "><p id="dataartsstudio_01_0064__en-us_topic_0108275309_p2305115252910">Non-auto creation</p>
</td>
</tr>
<tr id="dataartsstudio_01_0064__en-us_topic_0108275309_row222701815165"><td class="cellrowborder" valign="top" width="18.17%" headers="mcps1.3.2.2.4.1.1 "><p id="dataartsstudio_01_0064__en-us_topic_0108275309_p3385126816013">Rowkey Delimiter</p>
</td>
<td class="cellrowborder" valign="top" width="61.83%" headers="mcps1.3.2.2.4.1.2 "><p id="dataartsstudio_01_0064__en-us_topic_0108275309_p5345668716027">(Optional) Used to combine multiple columns as a rowkey. Spaces are used by default.</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.2.2.4.1.3 "><p id="dataartsstudio_01_0064__en-us_topic_0108275309_p322507154548">,</p>
</td>
</tr>
<tr id="dataartsstudio_01_0064__en-us_topic_0108275309_row29420512145148"><td class="cellrowborder" valign="top" width="18.17%" headers="mcps1.3.2.2.4.1.1 "><p id="dataartsstudio_01_0064__en-us_topic_0108275309_p22891711145148">Rowkey Data Redundancy</p>
</td>
<td class="cellrowborder" valign="top" width="61.83%" headers="mcps1.3.2.2.4.1.2 "><p id="dataartsstudio_01_0064__en-us_topic_0108275309_p42289313145148">(Optional) Whether to write the rowkey data into HBase columns. The default value is <span class="parmvalue" id="dataartsstudio_01_0064__en-us_topic_0108275309_parmvalue187011458319"><b>No</b></span>.</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.2.2.4.1.3 "><p id="dataartsstudio_01_0064__en-us_topic_0108275309_p2882352145148">No</p>
</td>
</tr>
<tr id="dataartsstudio_01_0064__en-us_topic_0108275309_row12086944145153"><td class="cellrowborder" valign="top" width="18.17%" headers="mcps1.3.2.2.4.1.1 "><p id="dataartsstudio_01_0064__en-us_topic_0108275309_p46874800145153">Compression Format</p>
</td>
<td class="cellrowborder" valign="top" width="61.83%" headers="mcps1.3.2.2.4.1.2 "><div class="p" id="dataartsstudio_01_0064__en-us_topic_0108275309_p38762485145153">(Optional) Compression format used in creating an HBase table. The default value is <strong id="dataartsstudio_01_0064__en-us_topic_0108275309_b842352706153247">None</strong>.<ul id="dataartsstudio_01_0064__en-us_topic_0108275309_ul59671921145439"><li id="dataartsstudio_01_0064__en-us_topic_0108275309_li46824973145439"><strong id="dataartsstudio_01_0064__en-us_topic_0108275309_b17883112035119">None</strong>: The files are not compressed.</li><li id="dataartsstudio_01_0064__en-us_topic_0108275309_li4700383414557"><strong id="dataartsstudio_01_0064__en-us_topic_0108275309_b8423527061507_1">Snappy</strong>: The files are compressed in snappy format.</li><li id="dataartsstudio_01_0064__en-us_topic_0108275309_li35929781145739"><strong id="dataartsstudio_01_0064__en-us_topic_0108275309_b8423527061507_3">gzip</strong>: The files are compressed in gzip format.</li></ul>
</div>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.2.2.4.1.3 "><p id="dataartsstudio_01_0064__en-us_topic_0108275309_p52753603145153">None</p>
</td>
</tr>
<tr id="dataartsstudio_01_0064__en-us_topic_0108275309_row5532119162618"><td class="cellrowborder" valign="top" width="18.17%" headers="mcps1.3.2.2.4.1.1 "><p id="dataartsstudio_01_0064__en-us_topic_0108275309_p8532121910267">Write WAL</p>
</td>
<td class="cellrowborder" valign="top" width="61.83%" headers="mcps1.3.2.2.4.1.2 "><p id="dataartsstudio_01_0064__en-us_topic_0108275309_p791673125616">Whether to enable Write Ahead Log (WAL) of HBase. The options are as follows:</p>
<ul id="dataartsstudio_01_0064__en-us_topic_0108275309_ul1899132517107"><li id="dataartsstudio_01_0064__en-us_topic_0108275309_li1199114259100"><strong id="dataartsstudio_01_0064__en-us_topic_0108275309_b19848123910429">Yes</strong>: If the HBase server breaks down after the function is enabled, you can replay the operations that have not been performed in WAL.</li><li id="dataartsstudio_01_0064__en-us_topic_0108275309_li1382192814103"><strong id="dataartsstudio_01_0064__en-us_topic_0108275309_b33115124515">No</strong>: If you set this parameter to <strong id="dataartsstudio_01_0064__en-us_topic_0108275309_b0248101364512">No</strong>, the write performance is improved. However, if the HBase server breaks down, data may be lost.</li></ul>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.2.2.4.1.3 "><p id="dataartsstudio_01_0064__en-us_topic_0108275309_p11532151914268">No</p>
</td>
</tr>
<tr id="dataartsstudio_01_0064__en-us_topic_0108275309_row13472310174"><td class="cellrowborder" valign="top" width="18.17%" headers="mcps1.3.2.2.4.1.1 "><p id="dataartsstudio_01_0064__en-us_topic_0108275309_p6474110971">Match Data Type</p>
</td>
<td class="cellrowborder" valign="top" width="61.83%" headers="mcps1.3.2.2.4.1.2 "><ul id="dataartsstudio_01_0064__en-us_topic_0108275309_ul4509123194219"><li id="dataartsstudio_01_0064__en-us_topic_0108275309_li125091936429"><strong id="dataartsstudio_01_0064__en-us_topic_0108275309_b13322273410">Yes</strong>: Data of the Short, Int, Long, Float, Double, and Decimal columns in the source database is converted into Byte[] arrays (binary) and written into HBase. Other types of data are written as character strings. If several types of data mentioned above are combined as rowkeys, they will be written as character strings.<p id="dataartsstudio_01_0064__en-us_topic_0108275309_p12731115251120">This function saves storage space. In specific scenarios, the rowkey distribution is evener.</p>
</li><li id="dataartsstudio_01_0064__en-us_topic_0108275309_li18571121654717"><strong id="dataartsstudio_01_0064__en-us_topic_0108275309_b10475161854218">No</strong>: All types of data in the source database are written into HBase as character strings.</li></ul>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.2.2.4.1.3 "><p id="dataartsstudio_01_0064__en-us_topic_0108275309_p24742101873">No</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0061.html">Configuring CDM Destination Job Parameters</a></div>
</div>
</div>
View Git Blame Copy Permalink