forked from docs/doc-exports
chenxiaoxiong
f9e2808b7c
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: chenxiaoxiong <chenxiaoxiong@huawei.com> Co-committed-by: chenxiaoxiong <chenxiaoxiong@huawei.com>
165 lines
17 KiB
HTML
165 lines
17 KiB
HTML
<a name="dataartsstudio_01_0072"></a><a name="dataartsstudio_01_0072"></a>
|
|
|
|
<h1 class="topictitle1">To DLI</h1>
|
|
<div id="body8662426"><p id="dataartsstudio_01_0072__en-us_topic_0108275441_p32842430161426">If the destination link of a job is a <a href="dataartsstudio_01_0036.html">DLI link</a>, configure the destination job parameters based on <a href="#dataartsstudio_01_0072__en-us_topic_0108275441_table5046103815165">Table 1</a>.</p>
|
|
<div class="caution" id="dataartsstudio_01_0072__en-us_topic_0108275441_note13118121416502"><span class="cautiontitle"><img src="public_sys-resources/caution_3.0-en-us.png"> </span><div class="cautionbody"><p id="dataartsstudio_01_0072__en-us_topic_0108275441_p151191614105011">When data is migrated to DLI using CDM, DLI generates data files in the <i><span class="varname" id="dataartsstudio_01_0072__en-us_topic_0108275441_varname13941194493013">dli-trans*</span></i> temporary OBS bucket. Therefore, you need to grant the user who uses the AK/SK the permissions to read and write the <i><span class="varname" id="dataartsstudio_01_0072__en-us_topic_0108275441_varname494144415308">dli-trans*</span></i> bucket and create directories. Otherwise, the migration will fail. For details about how to add permission policies for temporary bucket <i><span class="varname" id="dataartsstudio_01_0072__en-us_topic_0108275441_varname1939285783010">dli-trans*</span></i>, see <a href="#dataartsstudio_01_0072__en-us_topic_0108275441_section4208185214711">Adding an Authorization Policy for the dli-trans* Temporary Bucket</a>.</p>
|
|
</div></div>
|
|
|
|
<div class="tablenoborder"><a name="dataartsstudio_01_0072__en-us_topic_0108275441_table5046103815165"></a><a name="en-us_topic_0108275441_table5046103815165"></a><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_01_0072__en-us_topic_0108275441_table5046103815165" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="dataartsstudio_01_0072__en-us_topic_0108275441_row585315215165"><th align="left" class="cellrowborder" valign="top" width="17.169999999999998%" id="mcps1.3.3.2.4.1.1"><p id="dataartsstudio_01_0072__en-us_topic_0108275441_p1626397215165">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="63.77%" id="mcps1.3.3.2.4.1.2"><p id="dataartsstudio_01_0072__en-us_topic_0108275441_p4231334915165">Description</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="19.06%" id="mcps1.3.3.2.4.1.3"><p id="dataartsstudio_01_0072__en-us_topic_0108275441_p482921015165">Example Value</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dataartsstudio_01_0072__en-us_topic_0108275441_row61343144519"><td class="cellrowborder" valign="top" width="17.169999999999998%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0072__en-us_topic_0108275441_p21341141357">Resource Queue</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="63.77%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0072__en-us_topic_0108275441_p713417141954">Resource queue to which the destination table belongs</p>
|
|
<p id="dataartsstudio_01_0072__en-us_topic_0108275441_p11113521577">The default queue of DLI cannot be used for migration jobs. You need to create a SQL queue in DLI.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.06%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0072__en-us_topic_0108275441_p131341314654">cdm</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0072__en-us_topic_0108275441_row4012116315165"><td class="cellrowborder" valign="top" width="17.169999999999998%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0072__en-us_topic_0108275441_p2858877215165">Database Name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="63.77%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0072__en-us_topic_0108275441_p188281713155918">Name of the database to which data will be written</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.06%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0072__en-us_topic_0108275441_p166427315165">dli</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0072__en-us_topic_0108275441_row660232720546"><td class="cellrowborder" valign="top" width="17.169999999999998%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0072__en-us_topic_0108275441_p5602172785412">Table Name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="63.77%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0072__en-us_topic_0108275441_p8823111311598">Name of the table to which data will be written</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.06%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0072__en-us_topic_0108275441_p116025271541">car_detail</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0072__en-us_topic_0108275441_row1084214304541"><td class="cellrowborder" valign="top" width="17.169999999999998%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0072__en-us_topic_0108275441_p88425300545">Clear Data Before Import</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="63.77%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0072__en-us_topic_0108275441_p98148138591">Whether to clear data in the destination table before data import</p>
|
|
<p id="dataartsstudio_01_0072__en-us_topic_0108275441_p5902591216">If this parameter is set to <strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b1817603012718">Yes</strong>, data in the destination table will be cleared before the task is started.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.06%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0072__en-us_topic_0108275441_p1842930105416">No</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0072__en-us_topic_0108275441_row86721040184610"><td class="cellrowborder" valign="top" width="17.169999999999998%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0072__en-us_topic_0108275441_p19788058124611">Convert empty strings to null</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="63.77%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0072__en-us_topic_0108275441_p2673440174619">If this parameter is set to <strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b6308125465912">Yes</strong>, an empty string is regarded as null.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.06%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0072__en-us_topic_0108275441_p567384012466">No</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0072__en-us_topic_0108275441_row17275122711113"><td class="cellrowborder" valign="top" width="17.169999999999998%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0072__en-us_topic_0108275441_p3276182721115">Data Clearing Mode</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="63.77%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0072__en-us_topic_0108275441_p7159142101419">This parameter is available when <strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b55041131983">Clear Data Before Import</strong> is set to <strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b85041231781">Yes</strong>.</p>
|
|
<p id="dataartsstudio_01_0072__en-us_topic_0108275441_p42761827191114"><strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b86314297815">TRUNCATE</strong>: deletes standard data.</p>
|
|
<p id="dataartsstudio_01_0072__en-us_topic_0108275441_p127952561123"><strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b115572421080">INSERT_OVERWRITE</strong>: overwrites existing data with inserted data.</p>
|
|
<div class="note" id="dataartsstudio_01_0072__en-us_topic_0108275441_note2022043718378"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0072__en-us_topic_0108275441_p202201237163712">If the source link is a Kafka link and <strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b1966128162719">Clear Data Before Import</strong> is set to <strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b896662872720">Yes</strong>, <strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b139670280272">INSERT_OVERWRITE</strong> is unavailable.</p>
|
|
</div></div>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.06%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0072__en-us_topic_0108275441_p127615279115">TRUNCATE</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0072__en-us_topic_0108275441_row7651152211319"><td class="cellrowborder" valign="top" width="17.169999999999998%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0072__en-us_topic_0108275441_p2651122181312">Partition</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="63.77%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0072__en-us_topic_0108275441_p6622439171414">This parameter is available when <strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b17867156141017">Clear Data Before Import</strong> is set to <strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b178678564104">Yes</strong>.</p>
|
|
<p id="dataartsstudio_01_0072__en-us_topic_0108275441_p196521522121320">When you enter partitions, data in these partitions will be cleared.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.06%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0072__en-us_topic_0108275441_p12652112231311">year=2020,location=sun</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_01_0072__en-us_topic_0108275441_section4208185214711"><a name="dataartsstudio_01_0072__en-us_topic_0108275441_section4208185214711"></a><a name="en-us_topic_0108275441_section4208185214711"></a><h4 class="sectiontitle">Adding an Authorization Policy for the <i><span class="varname" id="dataartsstudio_01_0072__en-us_topic_0108275441_varname206618501348">dli-trans*</span></i> Temporary Bucket</h4><ol id="dataartsstudio_01_0072__en-us_topic_0108275441_ol1691979818"><li id="dataartsstudio_01_0072__en-us_topic_0108275441_li18291646181214"><span>Log in to the IAM console.</span></li><li id="dataartsstudio_01_0072__en-us_topic_0108275441_li155018156819"><span>In the navigation pane, choose <strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b182211471230">Permissions</strong> > <strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b0228154711312">Policies/Roles</strong> and click <strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b522814475315">Create Custom Policy</strong> in the upper right corner.</span><p><div class="fignone" id="dataartsstudio_01_0072__en-us_topic_0108275441_fig6217451719"><span class="figcap"><b>Figure 1 </b>Creating a custom policy</span><br><span><img id="dataartsstudio_01_0072__en-us_topic_0108275441_image6872564356" src="en-us_image_0000002269114785.png" title="Click to enlarge" class="imgResize"></span></div>
|
|
</p></li><li id="dataartsstudio_01_0072__en-us_topic_0108275441_li61017525541"><span>On the <strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b109708177422">Create Custom Policy</strong> page, select <strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b1497121714219">JSON</strong> for <strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b697111712429">Policy View</strong> and create custom policy <strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b841217459568">obs</strong><strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b19873131815578">_dli-trans</strong>.</span><p><pre class="screen" id="dataartsstudio_01_0072__en-us_topic_0108275441_screen191821233115320">{
|
|
"Version": "1.1",
|
|
"Statement": [
|
|
{
|
|
"Effect": "Allow",
|
|
"Action": [
|
|
"obs:object:GetObject",
|
|
"obs:object:DeleteObjectVersion",
|
|
"obs:bucket:GetBucketLocation",
|
|
"obs:object:GetAccessLabel",
|
|
"obs:bucket:PutEncryptionConfiguration",
|
|
"obs:bucket:PutBucketStoragePolicy",
|
|
"obs:object:DeleteAccessLabel",
|
|
"obs:bucket:PutBucketCustomDomainConfiguration",
|
|
"obs:bucket:GetLifecycleConfiguration",
|
|
"obs:bucket:PutBucketInventoryConfiguration",
|
|
"obs:bucket:DeleteDirectColdAccessConfiguration",
|
|
"obs:object:AbortMultipartUpload",
|
|
"obs:bucket:PutBucketLogging",
|
|
"obs:bucket:DeleteBucketWebsite",
|
|
"obs:object:DeleteObject",
|
|
"obs:bucket:PutBucketVersioning",
|
|
"obs:bucket:GetBucketWebsite",
|
|
"obs:bucket:GetBucketLogging",
|
|
"obs:bucket:DeleteBucketCustomDomainConfiguration",
|
|
"obs:object:PutObject",
|
|
"obs:object:RestoreObject",
|
|
"obs:bucket:PutReplicationConfiguration",
|
|
"obs:bucket:GetBucketQuota",
|
|
"obs:object:GetObjectVersionAcl",
|
|
"obs:bucket:DeleteBucket",
|
|
"obs:bucket:CreateBucket",
|
|
"obs:bucket:GetDirectColdAccessConfiguration",
|
|
"obs:bucket:PutDirectColdAccessConfiguration",
|
|
"obs:bucket:GetBucketAcl",
|
|
"obs:bucket:GetBucketVersioning",
|
|
"obs:bucket:GetBucketInventoryConfiguration",
|
|
"obs:bucket:GetBucketStoragePolicy",
|
|
"obs:bucket:GetEncryptionConfiguration",
|
|
"obs:bucket:PutBucketCORS",
|
|
"obs:bucket:PutBucketTagging",
|
|
"obs:bucket:GetBucketTagging",
|
|
"obs:bucket:PutLifecycleConfiguration",
|
|
"obs:bucket:GetBucketCustomDomainConfiguration",
|
|
"obs:object:ListMultipartUploadParts",
|
|
"obs:object:ModifyObjectMetaData",
|
|
"obs:bucket:ListBucketVersions",
|
|
"obs:bucket:PutBucketQuota",
|
|
"obs:object:PutAccessLabel",
|
|
"obs:bucket:ListBucket",
|
|
"obs:bucket:GetBucketCORS",
|
|
"obs:bucket:DeleteBucketInventoryConfiguration",
|
|
"obs:object:GetObjectVersion",
|
|
"obs:bucket:PutBucketWebsite",
|
|
"obs:bucket:DeleteReplicationConfiguration",
|
|
"obs:object:GetObjectAcl",
|
|
"obs:bucket:GetBucketNotification",
|
|
"obs:bucket:PutBucketNotification",
|
|
"obs:bucket:GetReplicationConfiguration",
|
|
"obs:bucket:GetBucketPolicy",
|
|
"obs:bucket:DeleteBucketTagging",
|
|
"obs:bucket:GetBucketStorage"
|
|
],
|
|
"Resource": [
|
|
"OBS:*:*:object:*",
|
|
"OBS:*:*:bucket:dli-trans*"
|
|
]
|
|
}
|
|
]
|
|
}</pre>
|
|
<div class="fignone" id="dataartsstudio_01_0072__en-us_topic_0108275441_fig5410103910579"><span class="figcap"><b>Figure 2 </b>Creating custom policy <strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b9531185014425">obs</strong><strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b145311350144211">_dli-trans</strong></span><br><span><img id="dataartsstudio_01_0072__en-us_topic_0108275441_image38706532576" src="en-us_image_0000002234075564.png" title="Click to enlarge" class="imgResize"></span></div>
|
|
<p id="dataartsstudio_01_0072__en-us_topic_0108275441_p621182518916"></p>
|
|
</p></li><li id="dataartsstudio_01_0072__en-us_topic_0108275441_li585471612815"><span>Click <strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b152221038164313">OK</strong>.</span></li><li id="dataartsstudio_01_0072__en-us_topic_0108275441_li16729115395815"><span>In the navigation pane, choose <strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b15495821124414">User Groups</strong>, locate the user group to which the DLI link user using the AK/SK belongs, and click <strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b1019945613459">Authorize</strong> to assign the custom <strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b255420451929">obs</strong><strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b355416451029">_dli-trans</strong> policy to the user.</span><p><div class="fignone" id="dataartsstudio_01_0072__en-us_topic_0108275441_fig19120206269"><span class="figcap"><b>Figure 3 </b>Assigning the custom <strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b1665171415620">obs</strong><strong id="dataartsstudio_01_0072__en-us_topic_0108275441_b166521416613">_dli-trans</strong> policy to a user group</span><br><span><img id="dataartsstudio_01_0072__en-us_topic_0108275441_image192471129362" src="en-us_image_0000002269114801.png" title="Click to enlarge" class="imgResize"></span></div>
|
|
<p id="dataartsstudio_01_0072__en-us_topic_0108275441_p1382093816711"></p>
|
|
</p></li></ol>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0061.html">Configuring CDM Destination Job Parameters</a></div>
|
|
</div>
|
|
</div>
|
|
|
|
|
|
<script language="JavaScript">
|
|
<!--
|
|
initImageViewer('.imgResize');
|
|
var msg_imageMax = "view original image";
|
|
var msg_imageClose = "close";
|
|
//--></script> |