yexinru/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
Files
0b3a43371a4bcc99577594de0a685249d676a567
doc-exports/docs/dataartsstudio/umn/dataartsstudio_01_0113.html
chenxiaoxiong f9e2808b7c DataArts UMN 20250810 version 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: chenxiaoxiong <chenxiaoxiong@huawei.com>
Co-committed-by: chenxiaoxiong <chenxiaoxiong@huawei.com>
2025-09-02 10:44:13 +00:00

24 lines
11 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<a name="dataartsstudio_01_0113"></a><a name="dataartsstudio_01_0113"></a>
<h1 class="topictitle1">Incremental Migration of Relational Databases</h1>
<div id="body32001227"><p id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_p1788415308396">CDM supports incremental migration of relational databases. After a full migration is complete, data in a specified period can be incrementally migrated. For example, data added on the previous day can be exported at 00:00:00 every day.</p>
<ul id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_ul6804182054719"><li id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_li128043209471"><strong id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_b1175450191213">Migrating incremental data within a specified period of time</strong><ul id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_ul10203433816"><li id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_li13925106183815">Application scenarios: The source end is a relational database. The destination end can be of any type.</li><li id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_li13489212387">Key configurations: <a href="#dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_section7506134317385">WHERE Clause</a> and Schedule Execution</li><li id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_li187871726113818">Prerequisites: The data table contains a date and time field or timestamp field.</li></ul>
</li></ul>
<p id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_p5741738859">In incremental migration, only the specified data is written to the data table. The existing records are not updated or deleted.</p>
<div class="note" id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_note45791757134910"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_en-us_topic_0108275319_p52974484409">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_en-us_topic_0108275319_text9997118203">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_en-us_topic_0108275319_i799871152012">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_en-us_topic_0108275319_i0998101192020">Offset</em>) rather than (<em id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_en-us_topic_0108275319_i299821132018">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_en-us_topic_0108275319_i129981917200">Offset</em>).</p>
</div></div>
<div class="section" id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_section7506134317385"><a name="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_section7506134317385"></a><a name="en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_section7506134317385"></a><h4 class="sectiontitle">WHERE Clause</h4><ul id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_ul1143972116011"><li id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_li1443916213016">Parameter position: When creating a table/file migration job, if the source end is a relational database, the <span class="parmname" id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_en-us_topic_0108275332_parmname18631437121010"><b>Where Clause</b></span> parameter is available in the advanced attributes of <strong id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_en-us_topic_0108275332_b7971124819916">Source Job Configuration</strong>.</li><li id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_li169491530407">Parameter principle: Set <span class="parmname" id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_parmname1764617178213"><b>WHERE Clause</b></span> to an SQL statement, for example, <strong id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_b17428183711215">age &gt; 18 and age &lt;= 60</strong>, CDM exports only the data that meets the SQL statement requirement. If <span class="parmname" id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_parmname85861449132115"><b>WHERE Clause</b></span> is not specified, the entire table is exported.<p id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_p157092344115"><span class="parmname" id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_parmname18259290204"><b>Where Clause</b></span> can be set to <a href="dataartsstudio_01_0114.html">macro variables of date and time</a>. When the data table contains the <strong id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_b7613175031112">date</strong> or <strong id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_b9749125371119">timestamp</strong> field, <span class="parmname" id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_parmname1674416112206"><b>Where Clause</b></span> and Schedule Execution can be used together to extract data of a specified date.</p>
</li><li id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_li11219540244">Example configurations:<p id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_p37968427413"><a name="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_li11219540244"></a><a name="en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_li11219540244"></a>Suppose that the database table contains column <strong id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_b10711183732212">DS</strong> indicating the time, the value type of the column is <span class="uicontrol" id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_uicontrol171173719225"><b>varchar(30)</b></span>, and the inserted time format is similar to <em id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_i78557319242">2017-xx-xx</em>. See <a href="#dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_fig14550053112127">Figure 1</a>. Set the parameters as follows:</p>
<div class="fignone" id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_fig14550053112127"><a name="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_fig14550053112127"></a><a name="en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_fig14550053112127"></a><span class="figcap"><b>Figure 1 </b>Table data</span><br><span><img id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_en-us_topic_0108275294_image37703684112127" src="en-us_image_0000002234240460.png"></span></div>
<ol id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_ol1992716204618"><li id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_li19271206619"><strong id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_b845917206133">WHERE Clause</strong>: Set this parameter to <strong id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_b1774104141310">DS='${dateformat(yyyy-MM-dd,-1,DAY)}'</strong>.</li><li id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_li10752836881">Scheduling job execution: Set <strong id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_b1317172165020">Cycle (days)</strong> to <strong id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_b562885114273">1</strong> and <strong id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_b6659181919284">Start Time</strong> to <strong id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_b6644721132816">00:00:00</strong>.</li></ol>
<p id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_p12687339580">In this way, all data generated on the previous day can be exported at 00:00:00 every day. <strong id="dataartsstudio_01_0113__en-us_topic_0000001151779454_en-us_topic_0000001151619238_en-us_topic_0108275332_b1099155517333">WHERE Clause</strong> can be configured to various <a href="dataartsstudio_01_0114.html">macro variables of date and time</a>. You can use the macro variables of date and time and scheduled jobs with specified cycle of minutes, hours, days, weeks, or months together to automatically export data at a specific time.</p>
</li></ul>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0111.html">Incremental Migration</a></div>
</div>
</div>
View Git Blame Copy Permalink