yexinru/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
Files
main
doc-exports/docs/dataartsstudio/umn/dataartsstudio_01_0048.html
chenxiaoxiong f9e2808b7c DataArts UMN 20250810 version 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: chenxiaoxiong <chenxiaoxiong@huawei.com>
Co-committed-by: chenxiaoxiong <chenxiaoxiong@huawei.com>
2025-09-02 10:44:13 +00:00

295 lines
45 KiB
HTML
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<a name="dataartsstudio_01_0048"></a><a name="dataartsstudio_01_0048"></a>
<h1 class="topictitle1">From OBS</h1>
<div id="body8662426"><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p6222841216210">If the source link of a job is an <a href="dataartsstudio_01_0045.html">OBS link</a>, configure the source job parameters based on <a href="#dataartsstudio_01_0048__en-us_topic_0108275319_table31823995163953">Table 1</a>.</p>
<p id="dataartsstudio_01_0048__en-us_topic_0108275319_p58828939103847">Advanced attributes are optional and not displayed by default. You can click <span class="uicontrol" id="dataartsstudio_01_0048__en-us_topic_0108275319_uicontrol212596276811411"><b>Show Advanced Attributes</b></span> to display them.</p>
<div class="tablenoborder"><a name="dataartsstudio_01_0048__en-us_topic_0108275319_table31823995163953"></a><a name="en-us_topic_0108275319_table31823995163953"></a><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_01_0048__en-us_topic_0108275319_table31823995163953" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row18653487163953"><th align="left" class="cellrowborder" valign="top" width="14.530000000000005%" id="mcps1.3.3.2.5.1.1"><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p2993976910371">Category</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="19.660000000000004%" id="mcps1.3.3.2.5.1.2"><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p15314298163953">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="49.150000000000006%" id="mcps1.3.3.2.5.1.3"><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p32498630163953">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="16.660000000000004%" id="mcps1.3.3.2.5.1.4"><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p15143370163953">Example Value</p>
</th>
</tr>
</thead>
<tbody><tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row34313000163953"><td class="cellrowborder" rowspan="9" valign="top" width="14.530000000000005%" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p52608321103750">Basic parameters</p>
</td>
<td class="cellrowborder" valign="top" width="19.660000000000004%" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p46804376161524">Bucket Name</p>
</td>
<td class="cellrowborder" valign="top" width="49.150000000000006%" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p46074171163953">Name of the bucket from which data will be migrated</p>
</td>
<td class="cellrowborder" valign="top" width="16.660000000000004%" headers="mcps1.3.3.2.5.1.4 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p41020337163953">BUCKET_2</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row1928353163953"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p28108739161524">Source Directory/File</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p4100102710298">This parameter is available only when <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b103174111711">Pull List File</strong> is set to <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b175956151974">No</strong>.</p>
<p id="dataartsstudio_01_0048__en-us_topic_0108275319_p855434665810">Directory or file path from which data will be extracted. You can enter a maximum of 50 file paths. By default, the file paths are separated by vertical bars (|). You can also customize a file separator. </p>
<p id="dataartsstudio_01_0048__en-us_topic_0108275319_p49679878163953">Directory from which data is to be migrated. All files (including all nested subdirectories and their subfiles) in the directory will be migrated.</p>
<p id="dataartsstudio_01_0048__en-us_topic_0108275319_p6705959695715">This parameter can be configured as a macro variable of date and time and a path name can contain multiple macro variables. When the macro variable of date and time works with a scheduled job, the incremental data can be synchronized periodically. </p>
<div class="note" id="dataartsstudio_01_0048__en-us_topic_0108275319_note0371124034016"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_p52974484409">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_text9997118203">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i799871152012">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i0998101192020">Offset</em>) rather than (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i299821132018">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i129981917200">Offset</em>).</p>
</div></div>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p64647157163953">FROM/example.csv</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row11687830163953"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p23518636161524">File Format</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><div class="p" id="dataartsstudio_01_0048__en-us_topic_0108275319_p27142413114140">Format in which CDM parses data. The options are as follows:<ul id="dataartsstudio_01_0048__en-us_topic_0108275319_ul41690379112856"><li id="dataartsstudio_01_0048__en-us_topic_0108275319_li922214112856"><strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b842352706102048_1">CSV</strong>: Source files will be migrated to tables after being converted to CSV format.</li><li id="dataartsstudio_01_0048__en-us_topic_0108275319_li2377599103550"><strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b842352706111710">Binary</strong>: Files (even not in binary format) will be transferred directly. It is used for file copy.</li><li id="dataartsstudio_01_0048__en-us_topic_0108275319_li7653780143848"><strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b842352706102048_3">JSON</strong>: Source files will be migrated to tables after being converted to JSON format.</li></ul>
</div>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p50683068163953">CSV</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row192010569556"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p172732585211">Pull List File</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p1784616256013">This parameter is displayed only when <span class="parmname" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmname3216104418176"><b>File Format</b></span> is set to <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue102161044101717"><b>Binary</b></span>.</p>
<p id="dataartsstudio_01_0048__en-us_topic_0108275319_p133836141792">If the pull list file function is enabled, the content of a file (such as a .txt file) in an OBS bucket can be read as the list of files to be migrated. The content in the file must be the absolute path of the file to be migrated (rather than a directory). For example, the content is as follows:</p>
<p id="dataartsstudio_01_0048__en-us_topic_0108275319_p395515141267">/052101/DAY20211110.data</p>
<p id="dataartsstudio_01_0048__en-us_topic_0108275319_p1468212404274">/052101/DAY20211111.data</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p1420212569555">Yes</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row14791112175616"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p583413915521">OBS Link of List File</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p1779718175122">This parameter is available only when <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b089055752017">Pull List File</strong> is set to <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b11890155762018">Yes</strong>. You can select the OBS link where the list file is located.</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p5625141693519">OBS_test_link</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row1657612546553"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p1491784155215">OBS Bucket of entries files</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p15767548553">This parameter is available only when <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b1264844132316">Pull List File</strong> is set to <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b8651544122316">Yes</strong>. It indicates the name of the OBS bucket where the list file is located.</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p557625435518">01</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row158561452195514"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p096517320540">Path/Directory of entries files</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p1350591217157">This parameter is available only when <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b162629390245">Pull List File</strong> is set to <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b19262839162414">Yes</strong>. It indicates the absolute path or directory of the list file in the OBS bucket.</p>
<p id="dataartsstudio_01_0048__en-us_topic_0108275319_p17199150132416">You are advised to select the absolute path of the file. If you select a directory, files in subdirectories can also be migrated. However, if the number of files in the directory is too large, the cluster memory may become insufficient.</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p5857452105518">/0521/Lists.txt</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row14158172014415"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p141589201416">JSON Type</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p151581520164119">This parameter is displayed only when <span class="parmname" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmname538960128"><b>File Format</b></span> is set to <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue1158646714"><b>JSON</b></span>. Type of a JSON object stored in a JSON file. The options are <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue961121311247"><b>JSON object</b></span> and <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue7205191992416"><b>JSON array</b></span>.</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p715815208415">JSON object</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row2424075414516"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p6269270114516">JSON Reference Node</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p4494402114516">This parameter is used only when <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b7740192016326">File Format</strong> is set to <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b1873531717321">JSON</strong> and <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b8581441173618">JSON Type</strong> is set to <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b145140132376">JSON Object</strong>. CDM parses the data under the JSON node. If the node's corresponding data is a JSON array, the system will extract data from the array in the same pattern. Use periods (.) to separate multi-layer nested JSON nodes.</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p1658709814516">data.list</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row4164236716479"><td class="cellrowborder" rowspan="24" valign="top" width="14.530000000000005%" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p199034103754">Advanced attributes</p>
<p id="dataartsstudio_01_0048__en-us_topic_0108275319_p1567819318531"></p>
</td>
<td class="cellrowborder" valign="top" width="19.660000000000004%" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p5746360161524">Line Separator</p>
</td>
<td class="cellrowborder" valign="top" width="49.150000000000006%" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p44379962142358">Lind feed character in a file. By default, the system automatically identifies <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue1628141519144315"><b>\n</b></span>, <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue946796237144315"><b>\r</b></span>, and <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue936618015144315"><b>\r\n</b></span>. This parameter is displayed only when <span class="parmname" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmname1101376798112032"><b>File Format</b></span> is set to <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue86583942112056"><b>CSV</b></span>.</p>
</td>
<td class="cellrowborder" valign="top" width="16.660000000000004%" headers="mcps1.3.3.2.5.1.4 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p3841367016479">\n</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row52292662164713"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p3584363161524">Field Delimiter</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p14005657142512">Character used to separate fields in the file. To set the <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b842352706113647">Tab</strong> key as the delimiter, set this parameter to <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue1470440010114039"><b>\t</b></span>. This parameter is displayed only when <span class="parmname" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmname1357494714213"><b>File Format</b></span> is set to <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue1257416476212"><b>CSV</b></span>.</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p13273001164713">,</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row20306111763516"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p630614173358">Use Quote Character</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p113238488438">If you set this parameter to <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue18865347172213"><b>Yes</b></span>, the field delimiters in the encircling symbol are regarded as a part of the string value. Currently, the default encircling symbol of CDM is <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b1466910403251">"</strong>.</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p6306101773520">No</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row182118115134"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p11212171111315">Using Escape Char</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p321241201311">If you select <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275374_b817410365189">Yes</strong>, the backslash (\) in the data row is used as an escape character. If you select <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275374_parmvalue11635949293"><b>No</b></span>, the backslash (\) in the CSV file will not be escaped. CSV supports only the backslash (\) as the escape character.</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p1521215191315">Yes</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row64482558153555"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p55704729153555">Use RE to Separate Fields</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p15789213153555">Whether to use regular expressions to separate fields. If you set this parameter to <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue171050633914118"><b>Yes</b></span>, <span class="parmname" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmname146402768141132"><b>Field Delimiter</b></span> becomes invalid. This parameter is displayed only when <span class="parmname" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmname816213311308"><b>File Format</b></span> is set to <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue13162233303"><b>CSV</b></span>.</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p3857840153555">Yes</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row10816719153653"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p3739051153653">Regular Expression</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p34427711153653">Regular expression used to separate fields. </p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p37181236153653">^(\d.*\d) (\w*) \[(.*)\] ([\w\.]*) (\w.*).*</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row18185115712363"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p101860576360">Use First N Rows as Header</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p1186155710366">This parameter is displayed only when <span class="parmname" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmname1938690813"><b>File Format</b></span> is set to <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue1483588303"><b>CSV</b></span>. When you migrate a CSV file to a table, CDM writes all data to the table by default. If you set this parameter to <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue1968432313314"><b>Yes</b></span>, CDM uses the first N rows of the CSV file as the heading row and does not write the row to the destination table.</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p11186257103611">No</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row1988284318534"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p9883643125310">The Number of Header Rows</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p4884443195318">This parameter is available when <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b15338132023514">Use First N Rows as Header</strong> is set to <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b676442343515">Yes</strong>. It specifies the number of header rows to be skipped during data extraction.</p>
<div class="note" id="dataartsstudio_01_0048__en-us_topic_0108275319_note977211117215"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p17722117217">The number of header rows cannot be empty. The value is an integer from 1 to 99.</p>
</div></div>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p9884184311534">1</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row953412513497"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p16535142524912">Extract first row as columns</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p6535192519494">This parameter is available when <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b1769252914354">Use First N Rows as Header</strong> is set to <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b7692729113512">Yes</strong>. It specifies whether to parse the first row of the header as a column name. The column name is displayed in the source field during field mapping configuration.</p>
<div class="note" id="dataartsstudio_01_0048__en-us_topic_0108275319_note1880120359501"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dataartsstudio_01_0048__en-us_topic_0108275319_ul7627182525112"><li id="dataartsstudio_01_0048__en-us_topic_0108275319_li136279257517">If the number of header rows is greater than 1, only the first row of the header can be parsed as the column name.</li><li id="dataartsstudio_01_0048__en-us_topic_0108275319_li7627725185119">The column name cannot contain the ampersand (&amp;). Otherwise, the job migration fails. If the column name contains the ampersand (&amp;), you must change it in the CSV file to ensure successful migration.</li></ul>
</div></div>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p10535182519499">Yes</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row20619738164716"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p59975518161524">Encoding Type</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p33988295142526">Encoding type, for example, <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue830270147115123"><b>UTF-8</b></span> or <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue354317202115128"><b>GBK</b></span>. You can set the encoding type for text files only. This parameter is invalid when <span class="parmname" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmname1134287303145943"><b>File Format</b></span> is set to <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue1589373374145950"><b>Binary</b></span>.</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p35014356164716">GBK</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row61444624154013"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p10958685154013">Compression Format</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><div class="p" id="dataartsstudio_01_0048__en-us_topic_0108275319_p35560870154156">The options are as follows:<ul id="dataartsstudio_01_0048__en-us_topic_0108275319_ul2275301315423"><li id="dataartsstudio_01_0048__en-us_topic_0108275319_li6351528115423"><strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b84235270615655_1">NONE</strong>: Files in all formats can be transferred.</li><li id="dataartsstudio_01_0048__en-us_topic_0108275319_li2191882915426"><strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b161191081482">GZIP</strong>: Only files in gzip format can be transferred.</li><li id="dataartsstudio_01_0048__en-us_topic_0108275319_li146183210612"><strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b84235270615655_3">ZIP</strong>: Only files in Zip format can be transferred.</li><li id="dataartsstudio_01_0048__en-us_topic_0108275319_li1754311673810"><strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b16279301073">TAR.GZ</strong>: Files in TAR.GZ format are transferred.</li></ul>
</div>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p10683184763014">NONE</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row2888152523513"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p24911467163">Compressed File Suffix</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p4301193717336">This parameter is displayed when <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b56981326292">Compression Format</strong> is not <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b869819261797">NONE</strong>.</p>
<p id="dataartsstudio_01_0048__en-us_topic_0108275319_p13187191714515">This parameter specifies the extension of the files to be decompressed. The decompression operation is performed only when the file name extension is used in a batch of files. Otherwise, files are transferred in the original format. If you enter <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b19098768612106">*</strong> or leave the parameter blank, all files are decompressed.</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p549646101615">*</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row64460251145943"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p53897812145943">Start Job by Marker File</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p3646613145943">Whether to start a job by a marker file. A job is only started if there is a marker file for starting the job in the source path. If there is no marker file, the job will be suspended for a period of time specified by <span class="parmname" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmname3785430292"><b>Suspension Period</b></span>.</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p26940231145943">No</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row252817516562"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p7528205145616">Marker File</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p952895115563">Name of the marker file for starting a job. If you specify a marker file, the migration job is executed only when the marker file exists in the source path. The marker file will not be migrated.</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p35282051195618">ok.txt</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row15452159135710"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p245210975717">Suspension Period</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p15247819764">Waiting period for a marker file. If you set <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b6652183325420">Start Job by Marker File</strong> to <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b365273313548">Yes</strong> but there is no marker file in the source path, the job fails when the suspension period times out.</p>
<p id="dataartsstudio_01_0048__en-us_topic_0108275319_p149111137114">If you set this parameter to <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b197346115010">0</strong> and there is no marker file in the source path, the job will fail immediately.</p>
<p id="dataartsstudio_01_0048__en-us_topic_0108275319_p104529915719">Unit: second</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p145220945719">10</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row1881335641920"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p581315631913">File Separator</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p8813175612198">File separator. If you enter multiple file paths in <span class="parmname" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmname1857611301331"><b>Source Directory/Files</b></span>, CDM uses the file separator to identify files. The default value is <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b957623014318">|</strong>.</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p118131563194">|</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row310973315583"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p4550940175816">Filter Type</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p0275103016610">Only paths or files that meet the filtering conditions are transferred. The options are <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b1123610303127">None</strong>, <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b924293071215">Wildcard</strong>, and <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b9242530131215">Regex</strong>. </p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p15111183311585">Wildcard</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row3022091315196"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p3197490715196">Directory Filter</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p48744974143114">If you set <span class="parmname" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmname9651163720211"><b>Filter Type</b></span> to <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue1199413443210"><b>Wildcard</b></span> or <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b17182166142119">Regex</strong>, enter a wildcard character to filter paths. The paths that meet the filtering condition are migrated. You can configure multiple paths separated by commas (,).</p>
<div class="note" id="dataartsstudio_01_0048__en-us_topic_0108275319_note10792171911399"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_p52974484409_1">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_text9997118203_1">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i799871152012_1">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i0998101192020_1">Offset</em>) rather than (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i299821132018_1">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i129981917200_1">Offset</em>).</p>
</div></div>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p708014715196">*input</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row4511408015199"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p3036188415199">File Filter</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p21968542143129">If you set <span class="parmname" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmname15716259342"><b>Filter Type</b></span> to <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue561362193810"><b>Wildcard</b></span> or <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue1759949153411"><b>Regex</b></span>, you can enter a wildcard character to search for files in a specified path. The files that meet the search criteria are migrated. You can configure multiple files separated by commas (,).</p>
<div class="note" id="dataartsstudio_01_0048__en-us_topic_0108275319_note12752123203916"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_p52974484409_2">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_text9997118203_2">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i799871152012_2">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i0998101192020_2">Offset</em>) rather than (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i299821132018_2">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i129981917200_2">Offset</em>).</p>
</div></div>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p2943506815199">*.csv,*.txt</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row2033617812558"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p333615875518">Time Filter</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p133361388559">If you select <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b113651633184010">Yes</strong>, files are transferred based on their modification time.</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p163364885515">Yes</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row1699311282615"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p109931712152614">Minimum Timestamp</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p299381213269">If you set <span class="parmname" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmname8132151819611"><b>Filter Type</b></span> to <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue113211184612"><b>Time Filter</b></span>, and specify a point in time for this parameter, only the files modified at or after the specified time are transferred. The time format must be <em id="dataartsstudio_01_0048__en-us_topic_0108275319_i16132918961">yyyy-MM-dd HH:mm:ss</em>.</p>
<p id="dataartsstudio_01_0048__en-us_topic_0108275319_p1365975611912">This parameter can be set to a macro variable of date and time. For example, <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b97861314103">${timestamp(dateformat(yyyy-MM-dd HH:mm:ss,-90,DAY))}</strong> indicates that only files generated within the latest 90 days are migrated.</p>
<div class="note" id="dataartsstudio_01_0048__en-us_topic_0108275319_note139020491241"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p52974484409">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0048__en-us_topic_0108275319_text9997118203">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_i799871152012">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_i0998101192020">Offset</em>) rather than (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_i299821132018">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_i129981917200">Offset</em>).</p>
</div></div>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p499321214262">2019-06-01 00:00:00</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row288815535588"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p7890553205812">Maximum Timestamp</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p176752141020">If you set <span class="parmname" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmname32123244113"><b>Filter Type</b></span> to <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue161911324415"><b>Time Filter</b></span>, and specify a point in time for this parameter, only the files modified before the specified time are transferred. The time format must be <em id="dataartsstudio_01_0048__en-us_topic_0108275319_i4192321411">yyyy-MM-dd HH:mm:ss</em>.</p>
<p id="dataartsstudio_01_0048__en-us_topic_0108275319_p3469261931">This parameter can be set to a macro variable of date and time. For example, <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b1521616538112">${timestamp(dateformat(yyyy-MM-dd HH:mm:ss))}</strong> indicates that only the files whose modification time is earlier than the current time are migrated.</p>
<div class="note" id="dataartsstudio_01_0048__en-us_topic_0108275319_note45791757134910"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_p52974484409_3">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_text9997118203_3">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i799871152012_3">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i0998101192020_3">Offset</em>) rather than (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i299821132018_3">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i129981917200_3">Offset</em>).</p>
</div></div>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p1389085325813">2019-07-01 00:00:00</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row76375302097"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p131273401892">Disregard Non-existent Path or File</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p16127540398">If this is set to <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b109132417614">Yes</strong>, the job can be successfully executed even if the source path does not exist.</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p712717406915">No</p>
</td>
</tr>
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row59691240202911"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p39711540162910">MD5 File Extension</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p2534111595912">This parameter is displayed only when <span class="parmname" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmname16121112171713"><b>File Format</b></span> is set to <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue1812612217176"><b>Binary</b></span>.</p>
<p id="dataartsstudio_01_0048__en-us_topic_0108275319_p224712441078">This parameter is used to check whether the files extracted by CDM are consistent with source files. </p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p18971124019297">.md5</p>
</td>
</tr>
</tbody>
</table>
</div>
<div class="note" id="dataartsstudio_01_0048__en-us_topic_0108275319_note8164156255"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ol id="dataartsstudio_01_0048__en-us_topic_0108275319_ol1210120121054"><li id="dataartsstudio_01_0048__en-us_topic_0108275319_li18101612653">CDM supports incremental file migration (by skipping repeated files), but does not support resumable transfer.<p id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275404_p53495242813"><a name="dataartsstudio_01_0048__en-us_topic_0108275319_li18101612653"></a><a name="en-us_topic_0108275319_li18101612653"></a>For example, if three files are to be migrated and the second file fails to be migrated due to the network fault. When the migration task is started again, the first file is skipped. The second file, however, cannot be migrated from the point where the fault occurs, but can only be migrated again.</p>
</li><li id="dataartsstudio_01_0048__en-us_topic_0108275319_li1841413442513">During file migration, a single task supports millions of files. If there are too many files in the directory to be migrated, you are advised to split the files into different directories and create multiple tasks.</li></ol>
</div></div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0047.html">Configuring CDM Source Job Parameters</a></div>
</div>
</div>
View Git Blame Copy Permalink