doc-exports/docs/dataartsstudio/umn/dataartsstudio_01_0046.html

<a name="dataartsstudio_01_0046"></a><a name="dataartsstudio_01_0046"></a>

<h1 class="topictitle1">Table/File Migration Jobs</h1>
<div id="body8662426"><div class="section" id="dataartsstudio_01_0046__en-us_topic_0108275458_sf6122b57876a439ca9c6d77777c1dc48"><h4 class="sectiontitle">Scenario</h4><p id="dataartsstudio_01_0046__en-us_topic_0108275458_a6a05a4793fa04de799e48799f0efc9bb">CDM supports table and file migration between homogeneous or heterogeneous data sources. For details about supported data sources, see <a href="dataartsstudio_01_0215.html">Supported Data Sources</a>.</p>
</div>
<div class="section" id="dataartsstudio_01_0046__en-us_topic_0108275458_section10128104561720"><h4 class="sectiontitle">Constraints</h4><ul id="dataartsstudio_01_0046__en-us_topic_0108275458_ul968318429312"><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li1368415425313">The dirty data recording function depends on OBS.</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li11684164214319">The JSON file of a job to be imported cannot exceed 1 MB.</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li9213343163817">The size of a file to be transferred cannot exceed 1 TB.</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li15271158217">Field names of the source and destination parameters cannot contain ampersands (&amp;) or number signs (%).</li></ul>
</div>
<div class="section" id="dataartsstudio_01_0046__en-us_topic_0108275458_sfbab7a6f896a46d1be21cf46c8a3b8c3"><h4 class="sectiontitle">Prerequisites</h4><ul id="dataartsstudio_01_0046__en-us_topic_0108275458_u18c5f21ca2024c828bd83b7762870046"><li id="dataartsstudio_01_0046__en-us_topic_0108275458_l0f09c327f05f45e0a7aa0f61f0fe8d35">A link has been created. For details, see <a href="dataartsstudio_01_0024.html">Creating a Link Between CDM and a Data Source</a>.</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_l5038f44ed94b4d5c92d5b6d53eced26e">The CDM cluster can communicate with the data source.</li></ul>
</div>
<div class="section" id="dataartsstudio_01_0046__en-us_topic_0108275458_s466f96b7ba414d42aa49f9a9cfeec1c7"><h4 class="sectiontitle">Procedure</h4><ol id="dataartsstudio_01_0046__en-us_topic_0108275458_od1b27a0a1cf748ac921a0b7a3274f193"><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li4336350316596"><span>Log in to the management console and choose <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b532585162814">Service List</strong> &gt; <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b1854045714289">Cloud Data Migration</strong>. In the left navigation pane, choose <span class="uicontrol" id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0108275370_uicontrol1030131521017"><b>Cluster Management</b></span>. Locate the target cluster and click <span class="uicontrol" id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0108275370_uicontrol11301115191010"><b>Job Management</b></span>.</span></li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li5835621161046"><span>Choose <span class="menucascade" id="dataartsstudio_01_0046__en-us_topic_0108275458_menucascade568733910590"><b><span class="uicontrol" id="dataartsstudio_01_0046__en-us_topic_0108275458_uicontrol12687103910597">Table/File Migration</span></b> &gt; <b><span class="uicontrol" id="dataartsstudio_01_0046__en-us_topic_0108275458_uicontrol26951539135920">Create Job</span></b></span>. The page for configuring the job is displayed.</span><p><div class="fignone" id="dataartsstudio_01_0046__en-us_topic_0108275458_fig49328951161541"><span class="figcap"><b>Figure 1 </b>Creating a migration job</span><br><span><img id="dataartsstudio_01_0046__en-us_topic_0108275458_image1244683055511" src="en-us_image_0000002269124649.png" title="Click to enlarge" class="imgResize"></span></div>
</p></li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li3067061016183"><span>Select the source and destination links.</span><p><ul id="dataartsstudio_01_0046__en-us_topic_0108275458_ul4045703616187"><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li2686484716187"><strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b19943164510327">Job Name</strong>: Enter a string consisting of 1 to 240 characters. The name can contain digits, letters, hyphens (-), underscores (_), and periods (.), and cannot start with a hyphen (-) or period (.). An example value is <span class="uicontrol" id="dataartsstudio_01_0046__en-us_topic_0108275458_uicontrol15943945133216"><b>oracle2rds_t</b></span>.</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li12187027161823"><strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b19849040183714">Source Link Name</strong>: Select the data source from which data will be exported.</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li36813026161831"><strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b5481165413714">Destination Link Name</strong>: Select the data source to which data will be imported.</li></ul>
</p></li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li41220264163159"><span>Configure the source link parameters. </span><p><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p2311114515556">The parameters vary with data sources. For details about the job parameters of other types of data sources, see <a href="#dataartsstudio_01_0046__en-us_topic_0108275458_table14973632102118">Table 1</a> and <a href="#dataartsstudio_01_0046__en-us_topic_0108275458_table64470284115654">Table 2</a>.</p>

<div class="tablenoborder"><a name="dataartsstudio_01_0046__en-us_topic_0108275458_table14973632102118"></a><a name="en-us_topic_0108275458_table14973632102118"></a><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_01_0046__en-us_topic_0108275458_table14973632102118" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Source link parameter description</caption><thead align="left"><tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row20487414102118"><th align="left" class="cellrowborder" valign="top" width="30.303030303030305%" id="mcps1.3.4.2.4.2.2.2.4.1.1"><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p37157660102118">Migration Source</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="31.313131313131315%" id="mcps1.3.4.2.4.2.2.2.4.1.2"><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p56980450102118">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="38.38383838383839%" id="mcps1.3.4.2.4.2.2.2.4.1.3"><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p52013737102118">Parameter Settings</p>
</th>
</tr>
</thead>
<tbody><tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row54909470102118"><td class="cellrowborder" valign="top" width="30.303030303030305%" headers="mcps1.3.4.2.4.2.2.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p7944859135113">OBS</p>
</td>
<td class="cellrowborder" valign="top" width="31.313131313131315%" headers="mcps1.3.4.2.4.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p13140187143019">Data can be extracted in CSV, JSON, or binary format. Data extracted in binary format is free from file resolution, which ensures high performance and is more suitable for file migration.</p>
</td>
<td class="cellrowborder" valign="top" width="38.38383838383839%" headers="mcps1.3.4.2.4.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p62279142102118">For details, see <a href="dataartsstudio_01_0048.html">From OBS</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row22465714102118"><td class="cellrowborder" valign="top" width="30.303030303030305%" headers="mcps1.3.4.2.4.2.2.2.4.1.1 "><ul id="dataartsstudio_01_0046__en-us_topic_0108275458_ul512454321014"><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li4124164341012">MRS HDFS</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li1096124382610">FusionInsight HDFS</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li11203104617261">Apache HDFS</li></ul>
</td>
<td class="cellrowborder" valign="top" width="31.313131313131315%" headers="mcps1.3.4.2.4.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p26488425102118">HDFS data can be exported in CSV, Parquet, or binary format and can be compressed in multiple formats.</p>
</td>
<td class="cellrowborder" valign="top" width="38.38383838383839%" headers="mcps1.3.4.2.4.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p65187718102118">For details, see <a href="dataartsstudio_01_0049.html">From HDFS</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row39360206102118"><td class="cellrowborder" valign="top" width="30.303030303030305%" headers="mcps1.3.4.2.4.2.2.2.4.1.1 "><ul id="dataartsstudio_01_0046__en-us_topic_0108275458_ul156301658131120"><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li1463020586118">MRS HBase</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li16465142745414">FusionInsight HBase</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li3140010105513">Apache HBase</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li1421880131219">CloudTable Service</li></ul>
</td>
<td class="cellrowborder" valign="top" width="31.313131313131315%" headers="mcps1.3.4.2.4.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p7403316102118">Data can be exported from MRS, FusionInsight HD, open source Apache Hadoop HBase, or CloudTable. You need to know all column families and field names of HBase tables.</p>
</td>
<td class="cellrowborder" valign="top" width="38.38383838383839%" headers="mcps1.3.4.2.4.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p62797755102118">For details, see <a href="dataartsstudio_01_0050.html">From HBase/CloudTable</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row44356635102118"><td class="cellrowborder" valign="top" width="30.303030303030305%" headers="mcps1.3.4.2.4.2.2.2.4.1.1 "><ul id="dataartsstudio_01_0046__en-us_topic_0108275458_ul116941458135714"><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li869415895712">MRS Hive</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li76941658115719">FusionInsight Hive</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li1969411585571">Apache Hive</li></ul>
</td>
<td class="cellrowborder" valign="top" width="31.313131313131315%" headers="mcps1.3.4.2.4.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p39851941102118">Data can be exported from Hive through the JDBC API.</p>
<p id="dataartsstudio_01_0046__en-us_topic_0108275458_p487151910118">If the data source is Hive, CDM will automatically partition data using the Hive data partitioning file.</p>
</td>
<td class="cellrowborder" valign="top" width="38.38383838383839%" headers="mcps1.3.4.2.4.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p6781823102118">For details, see <a href="dataartsstudio_01_0051.html">From Hive</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row8872613128"><td class="cellrowborder" valign="top" width="30.303030303030305%" headers="mcps1.3.4.2.4.2.2.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p887219132217">DLI</p>
</td>
<td class="cellrowborder" valign="top" width="31.313131313131315%" headers="mcps1.3.4.2.4.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p28722133215">Data can be exported from DLI.</p>
</td>
<td class="cellrowborder" valign="top" width="38.38383838383839%" headers="mcps1.3.4.2.4.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p178721131421">For details, see <a href="dataartsstudio_01_0120.html">From DLI</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row39564001104154"><td class="cellrowborder" valign="top" width="30.303030303030305%" headers="mcps1.3.4.2.4.2.2.2.4.1.1 "><ul id="dataartsstudio_01_0046__en-us_topic_0108275458_ul59703381316"><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li1897020361315">FTP</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li207811767135">SFTP</li></ul>
</td>
<td class="cellrowborder" valign="top" width="31.313131313131315%" headers="mcps1.3.4.2.4.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p2330157104154">FTP and SFTP data can be exported in CSV, JSON, or binary format.</p>
</td>
<td class="cellrowborder" valign="top" width="38.38383838383839%" headers="mcps1.3.4.2.4.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p3963018311534">For details, see <a href="dataartsstudio_01_0052.html">From FTP/SFTP</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row480315388201"><td class="cellrowborder" valign="top" width="30.303030303030305%" headers="mcps1.3.4.2.4.2.2.2.4.1.1 "><ul id="dataartsstudio_01_0046__en-us_topic_0108275458_ul1324644112014"><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li122494472013">HTTP</li></ul>
</td>
<td class="cellrowborder" valign="top" width="31.313131313131315%" headers="mcps1.3.4.2.4.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p980323822015">These connectors are used to read files with an HTTP/HTTPS URL, such as reading public files on the third-party object storage system and web disks.</p>
<p id="dataartsstudio_01_0046__en-us_topic_0108275458_p16901632183216"><span id="dataartsstudio_01_0046__en-us_topic_0108275458_ph5212204733414">Currently, data can only be exported from the HTTP URLs.</span></p>
</td>
<td class="cellrowborder" valign="top" width="38.38383838383839%" headers="mcps1.3.4.2.4.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p5803238112017">For details, see <a href="dataartsstudio_01_0053.html">From HTTP</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row12163112115437"><td class="cellrowborder" valign="top" width="30.303030303030305%" headers="mcps1.3.4.2.4.2.2.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p63344322411">Data Warehouse Service</p>
</td>
<td class="cellrowborder" valign="top" width="31.313131313131315%" headers="mcps1.3.4.2.4.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p88061232142516">Data can be exported from DWS.</p>
</td>
<td class="cellrowborder" valign="top" width="38.38383838383839%" headers="mcps1.3.4.2.4.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p123394322416">For details, see <a href="dataartsstudio_01_0387.html">From DWS</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row313116920266"><td class="cellrowborder" valign="top" width="30.303030303030305%" headers="mcps1.3.4.2.4.2.2.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p213111914269">SAP HANA</p>
</td>
<td class="cellrowborder" valign="top" width="31.313131313131315%" headers="mcps1.3.4.2.4.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p1413216962613">Data can be exported from SAP HANA.</p>
</td>
<td class="cellrowborder" valign="top" width="38.38383838383839%" headers="mcps1.3.4.2.4.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p11322932618">For details, see <a href="dataartsstudio_01_0388.html">From SAP HANA</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row21323854102118"><td class="cellrowborder" valign="top" width="30.303030303030305%" headers="mcps1.3.4.2.4.2.2.2.4.1.1 "><ul id="dataartsstudio_01_0046__en-us_topic_0108275458_ul112531201645"><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li6758151451717">RDS for PostgreSQL</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li177761855151710">RDS for SQL Server</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li158791137101820">Microsoft SQL Server</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li19117331238">PostgreSQL</li></ul>
</td>
<td class="cellrowborder" valign="top" width="31.313131313131315%" headers="mcps1.3.4.2.4.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p463840251598">Data can be exported from the cloud database services.</p>
<p id="dataartsstudio_01_0046__en-us_topic_0108275458_p1387115620216">The non-cloud databases can be those created in the on-premises data center or deployed on ECSs, or database services on the third-party clouds.</p>
</td>
<td class="cellrowborder" valign="top" width="38.38383838383839%" headers="mcps1.3.4.2.4.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p32124759102118">When data is exported from these data sources, CDM uses the JDBC API to extract data. The job parameters for the migration source are the same. For details, see <a href="dataartsstudio_01_0054.html">From PostgreSQL/SQL Server</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row11107101611422"><td class="cellrowborder" valign="top" width="30.303030303030305%" headers="mcps1.3.4.2.4.2.2.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p13108151613428">MySQL</p>
</td>
<td class="cellrowborder" valign="top" width="31.313131313131315%" headers="mcps1.3.4.2.4.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p8108201611428">Data can be exported from a MySQL database.</p>
</td>
<td class="cellrowborder" valign="top" width="38.38383838383839%" headers="mcps1.3.4.2.4.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p11108116134216">For details, see <a href="dataartsstudio_01_1254.html">From MySQL</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row1434473812273"><td class="cellrowborder" valign="top" width="30.303030303030305%" headers="mcps1.3.4.2.4.2.2.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p12344163872719">Oracle</p>
</td>
<td class="cellrowborder" valign="top" width="31.313131313131315%" headers="mcps1.3.4.2.4.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p18344143892715">Data can be exported from an Oracle database.</p>
</td>
<td class="cellrowborder" valign="top" width="38.38383838383839%" headers="mcps1.3.4.2.4.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p20345193852715">For details, see <a href="dataartsstudio_01_1255.html">From Oracle</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row1775461612278"><td class="cellrowborder" valign="top" width="30.303030303030305%" headers="mcps1.3.4.2.4.2.2.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p4754181632712">Database Sharding</p>
</td>
<td class="cellrowborder" valign="top" width="31.313131313131315%" headers="mcps1.3.4.2.4.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p1175412165271">Data can be exported from a shard.</p>
</td>
<td class="cellrowborder" valign="top" width="38.38383838383839%" headers="mcps1.3.4.2.4.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p1375521662715">For details, see <a href="dataartsstudio_01_1256.html">From a Database Shard</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row27158404104251"><td class="cellrowborder" valign="top" width="30.303030303030305%" headers="mcps1.3.4.2.4.2.2.2.4.1.1 "><ul id="dataartsstudio_01_0046__en-us_topic_0108275458_ul1772220420226"><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li11722442192210">MongoDB</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li18680174482212">Document Database Service</li></ul>
</td>
<td class="cellrowborder" valign="top" width="31.313131313131315%" headers="mcps1.3.4.2.4.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p6555507410438">Data can be exported from MongoDB or DDS.</p>
<div class="note" id="dataartsstudio_01_0046__en-us_topic_0108275458_note97718010442"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p1245563174417">MongoDB and DDS data sources with SSL enabled are not supported.</p>
</div></div>
</td>
<td class="cellrowborder" valign="top" width="38.38383838383839%" headers="mcps1.3.4.2.4.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p836075410438">For details, see <a href="dataartsstudio_01_0055.html">From MongoDB/DDS</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row47473059115023"><td class="cellrowborder" valign="top" width="30.303030303030305%" headers="mcps1.3.4.2.4.2.2.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p1976313184715">Redis</p>
</td>
<td class="cellrowborder" valign="top" width="31.313131313131315%" headers="mcps1.3.4.2.4.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p18506388115023">Data can be exported from open source Redis.</p>
</td>
<td class="cellrowborder" valign="top" width="38.38383838383839%" headers="mcps1.3.4.2.4.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p22622455115023">For details, see <a href="dataartsstudio_01_0056.html">From Redis</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row52458452113617"><td class="cellrowborder" valign="top" width="30.303030303030305%" headers="mcps1.3.4.2.4.2.2.2.4.1.1 "><ul id="dataartsstudio_01_0046__en-us_topic_0108275458_ul33901539135620"><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li93906394564">Apache Kafka</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li1887824012566">DMS Kafka</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li0956648102517">MRS Kafka</li></ul>
</td>
<td class="cellrowborder" valign="top" width="31.313131313131315%" headers="mcps1.3.4.2.4.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p515913955515">Data can only be exported to Cloud Search Service (CSS).</p>
</td>
<td class="cellrowborder" valign="top" width="38.38383838383839%" headers="mcps1.3.4.2.4.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p6805099113617">For details, see <a href="dataartsstudio_01_0058.html">From Kafka/DMS Kafka</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row335911454241"><td class="cellrowborder" valign="top" width="30.303030303030305%" headers="mcps1.3.4.2.4.2.2.2.4.1.1 "><ul id="dataartsstudio_01_0046__en-us_topic_0108275458_ul67101657696"><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li27102057697">Cloud Search Service</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li1024217020107">Elasticsearch</li></ul>
</td>
<td class="cellrowborder" valign="top" width="31.313131313131315%" headers="mcps1.3.4.2.4.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p16769009252">Data can be exported from CSS or Elasticsearch.</p>
</td>
<td class="cellrowborder" valign="top" width="38.38383838383839%" headers="mcps1.3.4.2.4.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p73597452248">For details, see <a href="dataartsstudio_01_0059.html">From Elasticsearch or CSS</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row8841361299"><td class="cellrowborder" valign="top" width="30.303030303030305%" headers="mcps1.3.4.2.4.2.2.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p11841036112919">MRS Hudi</p>
</td>
<td class="cellrowborder" valign="top" width="31.313131313131315%" headers="mcps1.3.4.2.4.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p7873121513304">Data can be exported from MRS Hudi.</p>
</td>
<td class="cellrowborder" valign="top" width="38.38383838383839%" headers="mcps1.3.4.2.4.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p1017649123011">For details, see <a href="dataartsstudio_01_0185.html">From MRS Hudi</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row1323133213294"><td class="cellrowborder" valign="top" width="30.303030303030305%" headers="mcps1.3.4.2.4.2.2.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p152321323291">MRS ClickHouse</p>
</td>
<td class="cellrowborder" valign="top" width="31.313131313131315%" headers="mcps1.3.4.2.4.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p182321832182912">Data can be exported from MRS ClickHouse.</p>
</td>
<td class="cellrowborder" valign="top" width="38.38383838383839%" headers="mcps1.3.4.2.4.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p19841159143019">For details, see <a href="dataartsstudio_01_0286.html">From MRS ClickHouse</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row743072914213"><td class="cellrowborder" valign="top" width="30.303030303030305%" headers="mcps1.3.4.2.4.2.2.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p143022918213">LogHub (SLS)</p>
</td>
<td class="cellrowborder" valign="top" width="31.313131313131315%" headers="mcps1.3.4.2.4.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p9691849172119">Data can be exported from LogHub (SLS).</p>
</td>
<td class="cellrowborder" valign="top" width="38.38383838383839%" headers="mcps1.3.4.2.4.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p1431172914213">For details, see <a href="dataartsstudio_01_0289.html">From LogHub (SLS)</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row154741259191"><td class="cellrowborder" valign="top" width="30.303030303030305%" headers="mcps1.3.4.2.4.2.2.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p84750591496">ShenTong database</p>
</td>
<td class="cellrowborder" valign="top" width="31.313131313131315%" headers="mcps1.3.4.2.4.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p174751059498">Data can be exported from a ShenTong database.</p>
</td>
<td class="cellrowborder" valign="top" width="38.38383838383839%" headers="mcps1.3.4.2.4.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p04758595919">For details, see <a href="dataartsstudio_01_0291.html">From a ShenTong Database</a>.</p>
</td>
</tr>
</tbody>
</table>
</div>
</p></li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li15809741103429"><span>Configure job parameters for the migration destination based on <a href="#dataartsstudio_01_0046__en-us_topic_0108275458_table64470284115654">Table 2</a>.</span><p>
<div class="tablenoborder"><a name="dataartsstudio_01_0046__en-us_topic_0108275458_table64470284115654"></a><a name="en-us_topic_0108275458_table64470284115654"></a><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_01_0046__en-us_topic_0108275458_table64470284115654" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Parameter description</caption><thead align="left"><tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row45377303115654"><th align="left" class="cellrowborder" valign="top" width="30%" id="mcps1.3.4.2.5.2.1.2.4.1.1"><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p51682937115654">Migration Destination</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="32%" id="mcps1.3.4.2.5.2.1.2.4.1.2"><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p25568392115654">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="38%" id="mcps1.3.4.2.5.2.1.2.4.1.3"><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p57773867115654">Parameter Settings</p>
</th>
</tr>
</thead>
<tbody><tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row49171659115654"><td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.4.2.5.2.1.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p535265412917">OBS</p>
</td>
<td class="cellrowborder" valign="top" width="32%" headers="mcps1.3.4.2.5.2.1.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p22947104115654">Files (even in a large volume) can be batch migrated to OBS in CSV or binary format.</p>
</td>
<td class="cellrowborder" valign="top" width="38%" headers="mcps1.3.4.2.5.2.1.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p46776101115654">For details, see <a href="dataartsstudio_01_0062.html">To OBS</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row30767814115654"><td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.4.2.5.2.1.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p781061192411">MRS HDFS</p>
</td>
<td class="cellrowborder" valign="top" width="32%" headers="mcps1.3.4.2.5.2.1.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p4170495115654">You can select a compression format when importing data to HDFS.</p>
</td>
<td class="cellrowborder" valign="top" width="38%" headers="mcps1.3.4.2.5.2.1.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p2265830115654">For details, see <a href="dataartsstudio_01_0063.html">To HDFS</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row49314570115654"><td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.4.2.5.2.1.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p19736393115">MRS HBase</p>
<p id="dataartsstudio_01_0046__en-us_topic_0108275458_p18920163861119">CloudTable Service</p>
</td>
<td class="cellrowborder" valign="top" width="32%" headers="mcps1.3.4.2.5.2.1.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p21063719115654">Data can be imported to HBase. The compression algorithm can be set when a new HBase table is created.</p>
</td>
<td class="cellrowborder" valign="top" width="38%" headers="mcps1.3.4.2.5.2.1.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p28439664115654">For details, see <a href="dataartsstudio_01_0064.html">To HBase/CloudTable</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row21911446115654"><td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.4.2.5.2.1.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p148116122410">MRS Hive</p>
</td>
<td class="cellrowborder" valign="top" width="32%" headers="mcps1.3.4.2.5.2.1.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p15499359184015">Data can be rapidly imported to MRS Hive.</p>
</td>
<td class="cellrowborder" valign="top" width="38%" headers="mcps1.3.4.2.5.2.1.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p44977602115654">For details, see <a href="dataartsstudio_01_0066.html">To Hive</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row24726917115654"><td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.4.2.5.2.1.2.4.1.1 "><ul id="dataartsstudio_01_0046__en-us_topic_0108275458_ul122829305514"><li id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0108275458_li6758151451717">MySQL</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0108275458_li177761855151710">SQL Server</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0108275458_li158791137101820">PostgreSQL</li></ul>
</td>
<td class="cellrowborder" valign="top" width="32%" headers="mcps1.3.4.2.5.2.1.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p7801438103712">Data can be imported to cloud database services.</p>
</td>
<td class="cellrowborder" valign="top" width="38%" headers="mcps1.3.4.2.5.2.1.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p42602429115654">For details about how to use the JDBC API to import data, see <a href="dataartsstudio_01_0068.html">To MySQL/SQL Server/PostgreSQL</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row82542011196"><td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.4.2.5.2.1.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p12250209194">DWS</p>
</td>
<td class="cellrowborder" valign="top" width="32%" headers="mcps1.3.4.2.5.2.1.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p125162071919">Data can be imported to DWS.</p>
</td>
<td class="cellrowborder" valign="top" width="38%" headers="mcps1.3.4.2.5.2.1.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p313134381917">For details, see <a href="dataartsstudio_01_1251.html">To DWS</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row1831151712189"><td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.4.2.5.2.1.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p78321217131820">Oracle</p>
</td>
<td class="cellrowborder" valign="top" width="32%" headers="mcps1.3.4.2.5.2.1.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p20832617161817">Data can be imported to an Oracle database.</p>
</td>
<td class="cellrowborder" valign="top" width="38%" headers="mcps1.3.4.2.5.2.1.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p1519705120184">For details, see <a href="dataartsstudio_01_0252.html">To Oracle</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row19298174814813"><td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.4.2.5.2.1.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p11293402196">DLI</p>
</td>
<td class="cellrowborder" valign="top" width="32%" headers="mcps1.3.4.2.5.2.1.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p131297401199">Data can be imported to DLI.</p>
</td>
<td class="cellrowborder" valign="top" width="38%" headers="mcps1.3.4.2.5.2.1.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p3129124013193">For details, see <a href="dataartsstudio_01_0072.html">To DLI</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row111657241073"><td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.4.2.5.2.1.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p9166324575">Elasticsearchor Cloud Search Service (CSS)</p>
</td>
<td class="cellrowborder" valign="top" width="32%" headers="mcps1.3.4.2.5.2.1.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p1938871217143">Data can be imported to CSS.</p>
</td>
<td class="cellrowborder" valign="top" width="38%" headers="mcps1.3.4.2.5.2.1.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p138871291414">For details, see <a href="dataartsstudio_01_0071.html">To Elasticsearch/CSS</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row11241165114710"><td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.4.2.5.2.1.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p1643122410488">MRS Hudi</p>
</td>
<td class="cellrowborder" valign="top" width="32%" headers="mcps1.3.4.2.5.2.1.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p51664116487">Data can be rapidly imported to MRS Hudi.</p>
</td>
<td class="cellrowborder" valign="top" width="38%" headers="mcps1.3.4.2.5.2.1.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p5643152414813">For details, see <a href="dataartsstudio_01_0186.html">To MRS Hudi</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row1068465474713"><td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.4.2.5.2.1.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p88572054195610">MRS ClickHouse</p>
</td>
<td class="cellrowborder" valign="top" width="32%" headers="mcps1.3.4.2.5.2.1.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p385716543561">Data can be rapidly imported to MRS ClickHouse.</p>
</td>
<td class="cellrowborder" valign="top" width="38%" headers="mcps1.3.4.2.5.2.1.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p17857185425610">For details, see <a href="dataartsstudio_01_0287.html">To MRS ClickHouse</a>.</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row1936710301871"><td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.4.2.5.2.1.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p1536763015718">MongoDB</p>
</td>
<td class="cellrowborder" valign="top" width="32%" headers="mcps1.3.4.2.5.2.1.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p517256176">Data can be rapidly imported to MongoDB.</p>
<div class="note" id="dataartsstudio_01_0046__en-us_topic_0108275458_note42617246444"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p62616249444">MongoDB data sources with SSL enabled are not supported.</p>
</div></div>
</td>
<td class="cellrowborder" valign="top" width="38%" headers="mcps1.3.4.2.5.2.1.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p16367123017715">For details, see <a href="dataartsstudio_01_0332.html">To MongoDB</a>.</p>
</td>
</tr>
</tbody>
</table>
</div>
</p></li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li15816706172612"><span>After the parameters are configured, click <span class="uicontrol" id="dataartsstudio_01_0046__en-us_topic_0108275458_uicontrol1502934244163158"><b>Next</b></span>. The <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b1949110574419">Map Field</strong> tab page is displayed.</span><p><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p59531677153626">If files are migrated between FTP, SFTP, OBS, and HDFS and the migration source's <span class="parmname" id="dataartsstudio_01_0046__en-us_topic_0108275458_parmname23038418337"><b>File Format</b></span> is set to <span class="parmvalue" id="dataartsstudio_01_0046__en-us_topic_0108275458_parmvalue23036413312"><b>Binary</b></span>, files will be directly transferred, free from field mapping.</p>
<p id="dataartsstudio_01_0046__en-us_topic_0108275458_p173294813511">In other scenarios, CDM automatically maps fields of the source table and the destination table. You need to check whether the mapping and time format are correct. For example, check whether the source field type can be converted into the destination field type.</p>
<div class="fignone" id="dataartsstudio_01_0046__en-us_topic_0108275458_fig53755635105617"><span class="figcap"><b>Figure 2 </b>Field mapping</span><br><span><img id="dataartsstudio_01_0046__en-us_topic_0108275458_image189429103305" src="en-us_image_0000002269204745.png" title="Click to enlarge" class="imgResize"></span></div>
<div class="note" id="dataartsstudio_01_0046__en-us_topic_0108275458_note122049565355"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dataartsstudio_01_0046__en-us_topic_0108275458_ul13523111833918"><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li7523101810391">If the fields from the source and destination do not match, you can drag the fields to make adjustments.</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li1365631672">On the <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b1746601711534">Map Field</strong> tab page, if CDM fails to obtain all columns by obtaining sample values (for example, when data is exported from HBase, CloudTable, or MongoDB, or when data is migrated from SFTP/FTP to DLI, there is a high probability that CDM failed to obtain all columns), you can click <span><img id="dataartsstudio_01_0046__en-us_topic_0108275458_image6467131715539" src="en-us_image_0000002269124641.png"></span> and select <span class="uicontrol" id="dataartsstudio_01_0046__en-us_topic_0108275458_uicontrol15467171745317"><b>Add a new field</b></span> to add new fields to ensure that the data imported to the migration destination is complete.</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li141838319368">When a relational database, Hive, DLI, or MRS Hudi is used as the migration source, sample values cannot be obtained.</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li74551022173518">On the <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b17862739124512">Map Field</strong> page, you can click <span><img id="dataartsstudio_01_0046__en-us_topic_0108275458_image166022617368" src="en-us_image_0000002269204741.png"></span> to add custom constants, variables, and expressions.</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li89695335298">Column names are displayed when the source of the migration job is OBS, CSV files are to be migrated, and parameter <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b139861043105519">Extract first row as columns</strong> is set to <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b16616212578">Yes</strong>.</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li1193920413558">When SQLServer is the destination, fields of the timestamp type cannot be written. You must change their type (for example, to datatime) so that they can be written.</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li87621645192015">When Hive serves as the source, data of the array and map types can be read.</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li17558825153711">Field mapping is not involved when the binary format is used to migrate files to files.</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li16330152017146">If the data is imported to DWS, you need to select the distribution columns in the destination fields. You are advised to select the distribution columns according to the following principles:<ol type="a" id="dataartsstudio_01_0046__en-us_topic_0108275458_ol9123174710293"><li id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0109007834_li15695951123715">Use the primary key as the distribution column.</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0109007834_li12711145112373">If multiple data segments are combined as primary keys, specify all primary keys as the distribution column.</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0109007834_li133234368451">In the scenario where no primary key is available, if no distribution column is selected, DWS uses the first column as the distribution column by default. As a result, data skew risks exist.</li></ol>
</li></ul>
</div></div>
</p></li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li41291161113719"><span>CDM supports field conversion. Click <span><img id="dataartsstudio_01_0046__en-us_topic_0108275458_image4692102813412" src="en-us_image_0000002234085460.png"></span> and then click <span class="uicontrol" id="dataartsstudio_01_0046__en-us_topic_0108275458_uicontrol106926281648"><b>Create Converter</b></span>.</span><p><div class="fignone" id="dataartsstudio_01_0046__en-us_topic_0108275458_fig46727271114434"><span class="figcap"><b>Figure 3 </b>Creating a converter</span><br><span><img id="dataartsstudio_01_0046__en-us_topic_0108275458_image155705360417" src="en-us_image_0000002269116385.png" title="Click to enlarge" class="imgResize"></span></div>
<p id="dataartsstudio_01_0046__en-us_topic_0108275458_p1920512451662"></p>
<p id="dataartsstudio_01_0046__en-us_topic_0108275458_p53634377144513">CDM supports the following converters:</p>
<ul id="dataartsstudio_01_0046__en-us_topic_0108275458_ul22005412144624"><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li25661764154523"><strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b72061249184514">Anonymization</strong>: hides key data in the character string.<div class="p" id="dataartsstudio_01_0046__en-us_topic_0108275458_p6195028154529">For example, if you want to convert <span class="uicontrol" id="dataartsstudio_01_0046__en-us_topic_0108275458_uicontrol50382807103247"><b>12345678910</b></span> to <span class="uicontrol" id="dataartsstudio_01_0046__en-us_topic_0108275458_uicontrol112904859310337"><b>123****8910</b></span>, configure the parameters as follows:<ul id="dataartsstudio_01_0046__en-us_topic_0108275458_ul6274915715419"><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li5987403115417">Set <span class="parmname" id="dataartsstudio_01_0046__en-us_topic_0108275458_parmname53421842154226"><b>Reserve Start Length</b></span> to <span class="parmvalue" id="dataartsstudio_01_0046__en-us_topic_0108275458_parmvalue27259213154244"><b>3</b></span>.</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li26752523154331">Set <span class="parmname" id="dataartsstudio_01_0046__en-us_topic_0108275458_parmname19718213192110"><b>Reserve End Length</b></span> to <span class="parmvalue" id="dataartsstudio_01_0046__en-us_topic_0108275458_parmvalue18719413172114"><b>4</b></span>.</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li18671023154340">Set <span class="parmname" id="dataartsstudio_01_0046__en-us_topic_0108275458_parmname9497941154351"><b>Replace Character</b></span> to <span class="parmvalue" id="dataartsstudio_01_0046__en-us_topic_0108275458_parmvalue1367489715442"><b>*</b></span>.</li></ul>
</div>
</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li38024799144624"><strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b14962111210489">Trim</strong> automatically deletes the spaces before and after the character string.</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li65128478144624"><strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b185111345011">Reverse string</strong> automatically reverses a character string. For example, reverse <span class="uicontrol" id="dataartsstudio_01_0046__en-us_topic_0108275458_uicontrol172521419124911"><b>ABC</b></span> into <span class="uicontrol" id="dataartsstudio_01_0046__en-us_topic_0108275458_uicontrol18822172816493"><b>CBA</b></span>.</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li30976826144624"><strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b15716143754416">Replace string</strong> replaces the specified character string.</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li31453073144624"><strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b273917502513">Expression conversion</strong> uses the JSP expression language (EL) to convert the current field or a row of data. </li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li511652291517"><strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b13831151272">Remove line break</strong> deletes the newline characters, such as \n, \r, and \r\n from the field.</li></ul>
<div class="note" id="dataartsstudio_01_0046__en-us_topic_0108275458_note323817186386"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p1323812184388">If <span class="parmname" id="dataartsstudio_01_0046__en-us_topic_0108275458_parmname1141826113814"><b>Use SQL Statement</b></span> is set to <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b14501129112118">Yes</strong> in the source job configuration, converters cannot be created.</p>
</div></div>
</p></li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li62792427103741"><span>Click <span class="uicontrol" id="dataartsstudio_01_0046__en-us_topic_0108275458_uicontrol38635611526"><b>Next</b></span>, set job parameters, and click <span class="uicontrol" id="dataartsstudio_01_0046__en-us_topic_0108275458_uicontrol17872012135317"><b>Show Advanced Attributes</b></span> to display and configure optional parameters.</span><p><div class="fignone" id="dataartsstudio_01_0046__en-us_topic_0108275458_fig18649183519455"><span class="figcap"><b>Figure 4 </b>Task parameters</span><br><span><img id="dataartsstudio_01_0046__en-us_topic_0108275458_image1347513115911" src="en-us_image_0000002234085456.png" title="Click to enlarge" class="imgResize"></span></div>
<div class="p" id="dataartsstudio_01_0046__en-us_topic_0108275458_p1952722104238"><a href="#dataartsstudio_01_0046__en-us_topic_0108275458_table62790900104257">Table 3</a> describes related parameters.
<div class="tablenoborder"><a name="dataartsstudio_01_0046__en-us_topic_0108275458_table62790900104257"></a><a name="en-us_topic_0108275458_table62790900104257"></a><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_01_0046__en-us_topic_0108275458_table62790900104257" frame="border" border="1" rules="all"><caption><b>Table 3 </b>Parameter description</caption><thead align="left"><tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row65473362104257"><th align="left" class="cellrowborder" valign="top" width="36%" id="mcps1.3.4.2.8.2.2.2.2.4.1.1"><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p15679301104257">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="38%" id="mcps1.3.4.2.8.2.2.2.2.4.1.2"><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p62063881104257">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="26%" id="mcps1.3.4.2.8.2.2.2.2.4.1.3"><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p61118428104257">Example Value</p>
</th>
</tr>
</thead>
<tbody><tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row275513314335"><td class="cellrowborder" valign="top" width="36%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p0755133114331">Retry upon Failure</p>
</td>
<td class="cellrowborder" valign="top" width="38%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p11755143118333">You can select <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b6893102920167">Retry 3 times</strong> or <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b11894103710166">Never</strong>.</p>
<p id="dataartsstudio_01_0046__en-us_topic_0108275458_p13649152833516">You are advised to configure automatic retry for only file migration jobs or database migration jobs with <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b19831131519189">Import to Staging Table</strong> enabled to avoid data inconsistency caused by repeated data writes.</p>
<div class="note" id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0099822521_note69071033105815"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p19198172795213">If you want to set parameters in <span id="dataartsstudio_01_0046__en-us_topic_0108275458_text113039492972317">DataArts Studio</span> DataArts Factory to schedule the CDM migration job, do not configure this parameter. Instead, set parameter <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b177969842572317">Retry upon Failure</strong> for the CDM node in DataArts Factory.</p>
</div></div>
</td>
<td class="cellrowborder" valign="top" width="26%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p13755031183312">Never</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row3631134372219"><td class="cellrowborder" valign="top" width="36%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p863154310228">Job</p>
</td>
<td class="cellrowborder" valign="top" width="38%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p163116432221">Select a group where the job resides. The default group is <span class="parmvalue" id="dataartsstudio_01_0046__en-us_topic_0108275458_parmvalue1229163714324"><b>DEFAULT</b></span>. On the <span class="wintitle" id="dataartsstudio_01_0046__en-us_topic_0108275458_wintitle950817428406"><b>Job Management</b></span> page, jobs can be displayed, started, or exported by group.</p>
</td>
<td class="cellrowborder" valign="top" width="26%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p19632114332214">DEFAULT</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row30305653162435"><td class="cellrowborder" valign="top" width="36%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p38838811162435">Schedule Execution</p>
</td>
<td class="cellrowborder" valign="top" width="38%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p58936023162435">If you select <span class="parmvalue" id="dataartsstudio_01_0046__en-us_topic_0108275458_parmvalue99175653111"><b>Yes</b></span>, you can set the start time, cycle, and validity period of a job. For details, see <a href="dataartsstudio_01_0082.html">Configuring a Scheduled CDM Job</a>.</p>
<div class="note" id="dataartsstudio_01_0046__en-us_topic_0108275458_note202879594918"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p32879515497">If you use <span id="dataartsstudio_01_0046__en-us_topic_0108275458_text63418818567">DataArts Studio</span> DataArts Factory to schedule the CDM migration job and configure this parameter, both configurations take effect. To ensure unified service logic and avoid scheduling conflicts, enable job scheduling in DataArts Factory and do not configure a scheduled task for the job in DataArts Migration.</p>
</div></div>
</td>
<td class="cellrowborder" valign="top" width="26%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p9088526162435">No</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row16101193015491"><td class="cellrowborder" valign="top" width="36%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p62157876104257">Concurrent Extractors</p>
</td>
<td class="cellrowborder" valign="top" width="38%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p15574751086">Maximum number of threads of the job for reading data from the source</p>
<div class="note" id="dataartsstudio_01_0046__en-us_topic_0108275458_note710791615819"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p1810771616814">The number of concurrent threads may be less than or equal to the value of this parameter for some data sources that do not support concurrent extraction, for example, CSS and ClickHouse.</p>
</div></div>
<div class="p" id="dataartsstudio_01_0046__en-us_topic_0108275458_p208324115207">CDM migrates data through data migration jobs. It works in the following way:<ol type="a" id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0000001287646722_en-us_topic_0000001340006573_ol1095184992318"><li id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0000001287646722_en-us_topic_0000001340006573_li79541847152411">When data migration jobs are submitted, CDM splits each job into multiple tasks based on the <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0000001287646722_en-us_topic_0000001340006573_b49451957175511">Concurrent Extractors</strong> parameter in the job configuration.<div class="note" id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0000001287646722_en-us_topic_0000001340006573_note636713355310"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0000001287646722_en-us_topic_0000001340006573_p1036843195312">Jobs for different data sources may be split based on different dimensions. Some jobs may not be split based on the <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0000001287646722_en-us_topic_0000001340006573_b258662735711">Concurrent Extractors</strong> parameter.</p>
</div></div>
</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0000001287646722_en-us_topic_0000001340006573_li587861564119">CDM submits the tasks to the running pool in sequence. Tasks (defined by <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0000001287646722_en-us_topic_0000001340006573_b169912243">Maximum Concurrent Extractors</strong>) run concurrently. Excess tasks are queued.</li></ol>
</div>
<p id="dataartsstudio_01_0046__en-us_topic_0108275458_p16182131102016">By setting appropriate values for this parameter and the <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b445331412266">Maximum Concurrent Extractors</strong> parameter, you can accelerate migration.</p>
<p id="dataartsstudio_01_0046__en-us_topic_0108275458_p9445165115227">Configure the number of concurrent extractors based on the following rules:</p>
<ol type="a" id="dataartsstudio_01_0046__en-us_topic_0108275458_ol144051381820"><li id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0000001287646722_li852255615323">When data is to be migrated to files, CDM does not support multiple concurrent tasks. In this case, set a single process to extract data.</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0000001287646722_li12522145616329">If each row of the table contains less than or equal to 1 MB data, data can be extracted concurrently. If each row contains more than 1 MB data, it is recommended that data be extracted in a single thread.</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0000001287646722_li13833942349">Set <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0000001287646722_b16578109101816">Concurrent Extractors</strong> for a job based on <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0000001287646722_b1648672014182">Maximum Concurrent Extractors</strong> for the cluster. It is recommended that <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0000001287646722_b317975281813">Concurrent Extractors</strong> is less than <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0000001287646722_b148781559111818">Maximum Concurrent Extractors</strong>.</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0000001287646722_li97288231260">If the destination is DLI, you are advised to set the number of concurrent extractors to 1. Otherwise, data may fail to be written.</li></ol>
<p id="dataartsstudio_01_0046__en-us_topic_0108275458_p1864412105196">The maximum number of concurrent extractors for a cluster varies depending on the CDM cluster flavor. You are advised to set the maximum number of concurrent extractors to twice the number of vCPUs of the CDM cluster. For example, the maximum number of concurrent extractors for a cluster with 8 vCPUs and 16 GB memory is 16.</p>
</td>
<td class="cellrowborder" valign="top" width="26%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p64368425104257">1</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row29830202153334"><td class="cellrowborder" valign="top" width="36%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p46457431153343">Concurrent Loaders</p>
</td>
<td class="cellrowborder" valign="top" width="38%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p26508935153334">Number of Loaders to be concurrently executed</p>
<p id="dataartsstudio_01_0046__en-us_topic_0108275458_p5573660510500">This parameter is displayed only when HBase or Hive serves as the destination data source.</p>
</td>
<td class="cellrowborder" valign="top" width="26%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p66849029153334">3</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row131711452152920"><td class="cellrowborder" valign="top" width="36%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p2172205214291">Number of split retries</p>
</td>
<td class="cellrowborder" valign="top" width="38%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p151729525297">Number of retries when a split fails to be executed. Value <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b153875872072317">0</strong> indicates that no retry will be performed.</p>
</td>
<td class="cellrowborder" valign="top" width="26%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p2017295222911">0</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row52808246153537"><td class="cellrowborder" valign="top" width="36%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p49609538153537">Write Dirty Data</p>
</td>
<td class="cellrowborder" valign="top" width="38%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p58949638153537">Whether to record dirty data. By default, this parameter is set to <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b842352706144759">No</strong>.</p>
<p id="dataartsstudio_01_0046__en-us_topic_0108275458_p1818710351886">Dirty data in CDM refers to the data in invalid format. If the source data contains dirty data, you are advised to enable this function. Otherwise, the migration job may fail.</p>
<div class="note" id="dataartsstudio_01_0046__en-us_topic_0108275458_note16899135215715"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p34221353105718">Dirty data can only be written to OBS paths. Therefore, this parameter is available only when an OBS link is available.</p>
</div></div>
</td>
<td class="cellrowborder" valign="top" width="26%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p10191403153537">Yes</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row9411130214937"><td class="cellrowborder" valign="top" width="36%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p24104098214937">Write Dirty Data Link</p>
</td>
<td class="cellrowborder" valign="top" width="38%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p62681934215229">This parameter is displayed only when <span class="parmname" id="dataartsstudio_01_0046__en-us_topic_0108275458_parmname128344056714491"><b>Write Dirty Data</b></span> is set to <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b842352706144918">Yes</strong>.</p>
<p id="dataartsstudio_01_0046__en-us_topic_0108275458_p6274906214937">You can only select an OBS link.</p>
</td>
<td class="cellrowborder" valign="top" width="26%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p38505354214937">obs_link</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row22967022215317"><td class="cellrowborder" valign="top" width="36%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p48389470215317">OBS Bucket</p>
</td>
<td class="cellrowborder" valign="top" width="38%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p59616852215319">This parameter is displayed only when <span class="parmname" id="dataartsstudio_01_0046__en-us_topic_0108275458_parmname1913284710145041"><b>Write Dirty Data Link</b></span> is a link to OBS.</p>
<p id="dataartsstudio_01_0046__en-us_topic_0108275458_p4238450021543">Name of the OBS bucket to which the dirty data will be written.</p>
</td>
<td class="cellrowborder" valign="top" width="26%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p58389960215317">dirtydata</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row39573864104257"><td class="cellrowborder" valign="top" width="36%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p51366450104257">Dirty Data Directory</p>
</td>
<td class="cellrowborder" valign="top" width="38%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p1077787153637">This parameter is displayed only when <span class="parmname" id="dataartsstudio_01_0046__en-us_topic_0108275458_parmname425130106"><b>Write Dirty Data</b></span> is set to <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b621023625">Yes</strong>.</p>
<p id="dataartsstudio_01_0046__en-us_topic_0108275458_p67041813104257">Dirty data is stored in the directory for storing dirty data on OBS. Dirty data is saved only when this parameter is configured.</p>
<p id="dataartsstudio_01_0046__en-us_topic_0108275458_p2836431711824">You can go to this directory to query data that fails to be processed or is filtered out during job execution, and check the source data that does not meet conversion or cleaning rules.</p>
</td>
<td class="cellrowborder" valign="top" width="26%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p61677752104257">/user/dirtydir</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row45605180153618"><td class="cellrowborder" valign="top" width="36%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p46055690153628">Max. Error Records in a Single Shard</p>
</td>
<td class="cellrowborder" valign="top" width="38%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p53760732164518">This parameter is displayed only when <span class="parmname" id="dataartsstudio_01_0046__en-us_topic_0108275458_parmname1992686731"><b>Write Dirty Data</b></span> is set to <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b2129701034">Yes</strong>.</p>
<p id="dataartsstudio_01_0046__en-us_topic_0108275458_p39523408153628">When the number of error records of a single map exceeds the upper limit, the job will automatically terminate and the imported data cannot be rolled back. You are advised to use a temporary table as the destination table. After the data is imported, rename the table or combine it into the final data table.</p>
</td>
<td class="cellrowborder" valign="top" width="26%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p47279450153628">0</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row19733841121020"><td class="cellrowborder" valign="top" width="36%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p77331741111017">Throttling</p>
</td>
<td class="cellrowborder" valign="top" width="38%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p9733841141015">Enabling throttling reduces the read pressure on the source. It controls the CDM transmission rate, not the NIC traffic.</p>
<div class="note" id="dataartsstudio_01_0046__en-us_topic_0108275458_note1563743510311"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dataartsstudio_01_0046__en-us_topic_0108275458_ul38741549183318"><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li1387444915337">Throttling can be enabled for non-binary file migration jobs.</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li14874349113317">To configure throttling for multiple jobs, multiply the rate by the number of concurrent jobs.</li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li7631637172316">Throttling is not supported for binary transmission between files.</li></ul>
</div></div>
</td>
<td class="cellrowborder" valign="top" width="26%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p97331141121017">Yes</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row21696388103"><td class="cellrowborder" valign="top" width="36%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p21691238101013">byteRate(MB/s)</p>
</td>
<td class="cellrowborder" valign="top" width="38%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p10169153814101">Maximum read/write speed of the job</p>
<p id="dataartsstudio_01_0046__en-us_topic_0108275458_p3491247112316">Throttling can be enabled for a job for migrating data to Hive, DLI, JDBC, OBS, or HDFS. If multiple concurrent jobs are allowed, the actual maximum speed can be calculated by the value of this parameter multiplied by the number of concurrent jobs.</p>
<div class="note" id="dataartsstudio_01_0046__en-us_topic_0108275458_note196441985017"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p156561925013">The rate is an integer greater than 1.</p>
</div></div>
</td>
<td class="cellrowborder" valign="top" width="26%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p016973810108">20</p>
</td>
</tr>
<tr id="dataartsstudio_01_0046__en-us_topic_0108275458_row1366919745711"><td class="cellrowborder" valign="top" width="36%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.1 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p667087165713">Intermediate Queue Cache Size (MB)</p>
</td>
<td class="cellrowborder" valign="top" width="38%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.2 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p19522123315719">Amount of data that the intermediate queue can cache. The value ranges from 1 to 500. The default value is <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b168021713171016">64</strong>.</p>
<p id="dataartsstudio_01_0046__en-us_topic_0108275458_p0670976571">If the amount of data of a row exceeds the value of this parameter, the migration may fail. If the value of this parameter is too large, the cluster may not run properly. Set an appropriate value for this parameter and use the default value (<strong id="dataartsstudio_01_0046__en-us_topic_0108275458_b1393616195124">64</strong>) unless otherwise specified.</p>
</td>
<td class="cellrowborder" valign="top" width="26%" headers="mcps1.3.4.2.8.2.2.2.2.4.1.3 "><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p136701776574">64</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
</p></li><li id="dataartsstudio_01_0046__en-us_topic_0108275458_li11856310104641"><span>Click <span class="uicontrol" id="dataartsstudio_01_0046__en-us_topic_0108275458_uicontrol2062714387567"><b>Save</b></span> or <span class="uicontrol" id="dataartsstudio_01_0046__en-us_topic_0108275458_uicontrol845585111566"><b>Save and Run</b></span>. On the displayed page, you can view the job status.</span><p><div class="note" id="dataartsstudio_01_0046__en-us_topic_0108275458_note132213013118"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_0046__en-us_topic_0108275458_p12221302118">The job status can be <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0108275443_b203902057137">New</strong>, <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0108275443_b439065101315">Pending</strong>, <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0108275443_b12390125191314">Booting</strong>, <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0108275443_b13919511137">Running</strong>, <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0108275443_b83911758134">Failed</strong>, <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0108275443_b73915516134">Succeeded</strong>, or <strong id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0108275443_b01905183139">Stopped</strong>.</p>
<p id="dataartsstudio_01_0046__en-us_topic_0108275458_p167371224626"><span class="uicontrol" id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0108275443_uicontrol1285813274203"><b>Pending</b></span> indicates that the job is waiting to be scheduled by the system, and <span class="uicontrol" id="dataartsstudio_01_0046__en-us_topic_0108275458_en-us_topic_0108275443_uicontrol6778143782017"><b>Booting</b></span> indicates that the data to be migrated is being analyzed.</p>
</div></div>
</p></li></ol>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0081.html">Creating a Job in a CDM Cluster</a></div>
</div>
</div>


<script language="JavaScript">
<!--
initImageViewer('.imgResize');
var msg_imageMax = "view original image";
var msg_imageClose = "close";
//--></script>