forked from docs/doc-exports
DataArts UMN 20250810 version
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: chenxiaoxiong <chenxiaoxiong@huawei.com> Co-committed-by: chenxiaoxiong <chenxiaoxiong@huawei.com>
This commit is contained in:
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@ -1,9 +1,9 @@
|
||||
<a name="dataartsstudio_01_0001"></a><a name="dataartsstudio_01_0001"></a>
|
||||
|
||||
<h1 class="topictitle1">Accessing the <span id="text2083152211518">DataArts Studio</span> Instance Console</h1>
|
||||
<div id="body0000001853601317"><div class="section" id="dataartsstudio_01_0001__section133514245719"><h4 class="sectiontitle">Prerequisites</h4><p id="dataartsstudio_01_0001__p16738144416585">A <span id="dataartsstudio_01_0001__text736833275210">DataArts Studio</span> instance has been obtained. For details, see <a href="dataartsstudio_01_0115.html">Creating a DataArts Studio Basic Package</a>.</p>
|
||||
<div id="body0000001987431545"><div class="section" id="dataartsstudio_01_0001__section133514245719"><h4 class="sectiontitle">Prerequisites</h4><p id="dataartsstudio_01_0001__p16738144416585">You have obtained a <span id="dataartsstudio_01_0001__text736833275210">DataArts Studio</span> instance. For details, see <a href="dataartsstudio_01_0115.html">Creating a DataArts Studio Instance</a>.</p>
|
||||
</div>
|
||||
<div class="section" id="dataartsstudio_01_0001__section185393313315"><h4 class="sectiontitle">Procedure</h4><ol id="dataartsstudio_01_0001__ol165206015246"><li id="dataartsstudio_01_0001__en-us_topic_0181028490_li586523263418"><span>Log in to the <span id="dataartsstudio_01_0001__text1122912916583">cloud</span> console. Choose <span id="dataartsstudio_01_0001__text1771592614358">DataArts Studio</span> from the service list in the upper left corner to access the <span id="dataartsstudio_01_0001__text1421955017286">DataArts Studio</span> instance console.</span><p><ul id="dataartsstudio_01_0001__ul32482719575"><li id="dataartsstudio_01_0001__li3244279575">If there are multiple <span id="dataartsstudio_01_0001__text626003945410">DataArts Studio</span> instances in the current region, the instance list is displayed by default. Locate an instance and click <strong id="dataartsstudio_01_0001__b15891858174611">Access</strong> to access the <span id="dataartsstudio_01_0001__text166763199566">DataArts Studio</span> console.<p id="dataartsstudio_01_0001__p1584233632220"></p>
|
||||
<div class="section" id="dataartsstudio_01_0001__section185393313315"><h4 class="sectiontitle">Procedure</h4><ol id="dataartsstudio_01_0001__ol165206015246"><li id="dataartsstudio_01_0001__en-us_topic_0181028490_li586523263418"><span>Log in to the <span id="dataartsstudio_01_0001__text1183620919483">cloud</span> console. Choose <span id="dataartsstudio_01_0001__text048473563711">DataArts Studio</span> from the service list in the upper left corner.</span><p><ul id="dataartsstudio_01_0001__ul32482719575"><li id="dataartsstudio_01_0001__li3244279575">If there are multiple <span id="dataartsstudio_01_0001__text626003945410">DataArts Studio</span> instances in the current region, the instance list is displayed by default. Locate an instance and click <strong id="dataartsstudio_01_0001__b1236110421376">Access</strong> to access the <span id="dataartsstudio_01_0001__text166763199566">DataArts Studio</span> console homepage.<p id="dataartsstudio_01_0001__p1584233632220"></p>
|
||||
<p id="dataartsstudio_01_0001__p79261641112316"></p>
|
||||
</li><li id="dataartsstudio_01_0001__li182432755712">If there is only one <span id="dataartsstudio_01_0001__text787861295711">DataArts Studio</span> instance in the current region, the <span id="dataartsstudio_01_0001__text1560072112575">DataArts Studio</span> console homepage is displayed by default.<p id="dataartsstudio_01_0001__p278692623011"></p>
|
||||
<p id="dataartsstudio_01_0001__p313020184610"></p>
|
||||
@ -14,7 +14,7 @@
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_12_0002.html">Preparations</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_1028.html">Creating and Configuring a DataArts Studio Instance</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
@ -1,13 +0,0 @@
|
||||
<a name="dataartsstudio_01_0003"></a><a name="dataartsstudio_01_0003"></a>
|
||||
|
||||
<h1 class="topictitle1">Preparations</h1>
|
||||
<div id="body8662426"><p id="dataartsstudio_01_0003__p19331838114818">Before using <span id="dataartsstudio_01_0003__text14567613773">DataArts Studio</span>, you need to <span id="dataartsstudio_01_0003__text18331103854810">create</span> a <span id="dataartsstudio_01_0003__text1331538114813">a cloud platform account</span>, <span id="dataartsstudio_01_0003__text13331143824812">create</span> a <span id="dataartsstudio_01_0003__text18331143812480">DataArts Studio</span> instance, access the DataArts Studio instance console, manage workspaces, and authorize users to use <span id="dataartsstudio_01_0003__text16819177115812">DataArts Studio</span>.</p>
|
||||
<p id="dataartsstudio_01_0003__en-us_topic_0197575849_p0579429102216">The following figure shows the preparations and operations, which will be described in detail in later sections.</p>
|
||||
<div class="fignone" id="dataartsstudio_01_0003__fig7123194914174"><span class="figcap"><b>Figure 1 </b>Preparations for using <span id="dataartsstudio_01_0003__text112324991718">DataArts Studio</span></span><br><span><img id="dataartsstudio_01_0003__image12982120271" src="en-us_image_0000002270844674.png"></span></div>
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_12_0002.html">Preparations</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@ -1,27 +1,29 @@
|
||||
<a name="dataartsstudio_01_0004"></a><a name="dataartsstudio_01_0004"></a>
|
||||
|
||||
<h1 class="topictitle1">Creating an IAM User and Assigning <span id="text13250174381611">DataArts Studio</span> Permissions</h1>
|
||||
<div id="body8662426"><div class="p" id="dataartsstudio_01_0004__en-us_topic_0223009156_p988518487488">Identity and Access Management (IAM) can be used for fine-grained permissions management on your <span id="dataartsstudio_01_0004__text189401823191311">DataArts Studio</span> resources. With IAM, you can:<ul id="dataartsstudio_01_0004__ul20307133063417"><li id="dataartsstudio_01_0004__en-us_topic_0223009156_li588684820486">Create IAM users for employees based on your enterprise's organizational structure. Each IAM user will have their own security credentials for accessing <span id="dataartsstudio_01_0004__text2090111011530">DataArts Studio</span> resources.</li><li id="dataartsstudio_01_0004__en-us_topic_0223009156_li38863484487">Assign users only the permissions required to perform a task.</li><li id="dataartsstudio_01_0004__en-us_topic_0223009156_li17886174824816">Entrust a <span id="dataartsstudio_01_0004__text685775312213">a cloud platform account</span> or cloud service to perform efficient O&M on your <span id="dataartsstudio_01_0004__text885745314210">DataArts Studio</span> resources.</li></ul>
|
||||
<h1 class="topictitle1">Creating an IAM User and Assigning <span id="text83374331400">DataArts Studio</span> Permissions</h1>
|
||||
<div id="body0000002024014390"><div class="p" id="dataartsstudio_01_0004__en-us_topic_0223009156_p988518487488">Identity and Access Management (IAM) can be used for fine-grained permissions management on your <span id="dataartsstudio_01_0004__text1142013904015">DataArts Studio</span> resources. With IAM, you can:<ul id="dataartsstudio_01_0004__ul20307133063417"><li id="dataartsstudio_01_0004__en-us_topic_0223009156_li588684820486">Create IAM users for employees based on your enterprise's organizational structure. Each IAM user will have their own security credentials for accessing <span id="dataartsstudio_01_0004__text16979149134014">DataArts Studio</span> resources.</li><li id="dataartsstudio_01_0004__en-us_topic_0223009156_li38863484487">Grant users only the permissions required to perform a given task based on their job responsibilities.</li><li id="dataartsstudio_01_0004__en-us_topic_0223009156_li17886174824816">Entrust a <span id="dataartsstudio_01_0004__text9957155617403">cloud account</span> or cloud service to perform efficient O&M on your <span id="dataartsstudio_01_0004__text595785624020">DataArts Studio</span> resources.</li></ul>
|
||||
</div>
|
||||
<p id="dataartsstudio_01_0004__en-us_topic_0223009156_p7886148114812">If you do not require individual IAM users for permissions management, skip this section.</p>
|
||||
<div class="section" id="dataartsstudio_01_0004__en-us_topic_0223009156_section149106163317"><h4 class="sectiontitle">Background</h4><ul id="dataartsstudio_01_0004__en-us_topic_0223009156_ul4345266339"><li id="dataartsstudio_01_0004__en-us_topic_0223009156_li19891163562811">Before assigning permissions to a user group, you need to understand the <span id="dataartsstudio_01_0004__text256471362515">DataArts Studio</span> permission system and select permissions as needed. </li></ul>
|
||||
</div>
|
||||
<div class="section" id="dataartsstudio_01_0004__section3404192611368"><h4 class="sectiontitle">Notes and Constraints</h4><ul id="dataartsstudio_01_0004__ul5958194983813"><li id="dataartsstudio_01_0004__li42343448248">The DARTS User role provides permissions related to instances, workspaces, and dependent services. Workspace roles provide service operation permissions in workspaces.</li><li id="dataartsstudio_01_0004__li11959194993815">IAM provides the following two authorization mechanisms. <span id="dataartsstudio_01_0004__en-us_topic_0222691153_text98649123311">DataArts Studio</span> supports IAM roles but not IAM policies.<ul id="dataartsstudio_01_0004__ul4577145434117"><li id="dataartsstudio_01_0004__en-us_topic_0222691153_en-us_topic_0216589782_li18737181101111"><strong id="dataartsstudio_01_0004__b42311658171511">IAM roles</strong>: IAM initially provides a coarse-grained authorization mechanism to define permissions based on users' job responsibilities. Only a limited number of service-level roles are available for authorization. However, traditional IAM roles are not an ideal choice for fine-grained authorization and secure access control.</li><li id="dataartsstudio_01_0004__en-us_topic_0222691153_li144587474243"><strong id="dataartsstudio_01_0004__b12674181818163">IAM policies</strong>: A type of fine-grained authorization mechanism that defines permissions required to perform operations on specific cloud resources under certain conditions. This type of authorization is more flexible and meets the principle of least privilege (PoLP).</li></ul>
|
||||
<p id="dataartsstudio_01_0004__en-us_topic_0223009156_p7886148114812">If you do not require individual IAM users for permissions management, skip this topic.</p>
|
||||
<p id="dataartsstudio_01_0004__en-us_topic_0223009156_p1886124811488">This section describes the procedure for granting permissions. See <a href="#dataartsstudio_01_0004__en-us_topic_0223009156_section9135134520119">Procedure</a> for details.</p>
|
||||
<div class="section" id="dataartsstudio_01_0004__en-us_topic_0223009156_section149106163317"><h4 class="sectiontitle">Context</h4><ul id="dataartsstudio_01_0004__en-us_topic_0223009156_ul4345266339"><li id="dataartsstudio_01_0004__en-us_topic_0223009156_li19891163562811">Before assigning permissions to a user group, you need to understand the permission system of so that you can select the permissions that meet your needs. <div class="fignone" id="dataartsstudio_01_0004__fig168744210348"><span class="figcap"><b>Figure 1 </b>Permission system</span><br><span><img class="imgResize" id="dataartsstudio_01_0004__en-us_topic_0000001492262982_image1913610193343" src="en-us_image_0000002391028724.png" title="Click to enlarge"></span></div>
|
||||
</li></ul>
|
||||
</div>
|
||||
<div class="section" id="dataartsstudio_01_0004__en-us_topic_0223009156_section9135134520119"><h4 class="sectiontitle">Procedure</h4><div class="fignone" id="dataartsstudio_01_0004__fig320863593815"><span class="figcap"><b>Figure 1 </b>Authorization process</span><br><span><img class="eddx" id="dataartsstudio_01_0004__image52081635103816" src="en-us_image_0000002270921874.png"></span></div>
|
||||
<ol id="dataartsstudio_01_0004__ol5977748151916"><li id="dataartsstudio_01_0004__li39771248101915"><a name="dataartsstudio_01_0004__li39771248101915"></a><a name="li39771248101915"></a><span>Create a user group and assign a system role to the group.</span><p><p id="dataartsstudio_01_0004__p55201136510">Log in to the IAM console using <span id="dataartsstudio_01_0004__text1831511164010">a cloud platform account</span>, create a user group and assign a <span id="dataartsstudio_01_0004__text1570132531">DataArts Studio</span> system role to the group. For example, the system role can be DARTS Administrator or DARTS User.</p>
|
||||
<p id="dataartsstudio_01_0004__en-us_topic_0223009156_p108313320205">For details, see "User Groups and Authorization" > "Creating a User Group and Assigning Permissions" in <em id="dataartsstudio_01_0004__i944914123332">Identity and Access Management User Guide</em>.</p>
|
||||
<div class="note" id="dataartsstudio_01_0004__en-us_topic_0223009156_note52501597263"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dataartsstudio_01_0004__en-us_topic_0223009156_ul114668532278"><li id="dataartsstudio_01_0004__li1834013912412">When configuring <span id="dataartsstudio_01_0004__text129169401166">DataArts Studio</span> permissions for a user group, enter DARTS in the search box to search for the permissions and select the permissions to be granted to the user group, for example, <strong id="dataartsstudio_01_0004__b129161940161613">DARTS User</strong>.</li><li id="dataartsstudio_01_0004__li2866162654415"><span id="dataartsstudio_01_0004__text129123284613">DataArts Studio</span> is a project-level service deployed in specific physical regions. If you select <strong id="dataartsstudio_01_0004__b827913464817">All resources</strong> for <strong id="dataartsstudio_01_0004__b350973420578">Scope</strong>, the permission takes effect in all projects of all regions. If you select <strong id="dataartsstudio_01_0004__b1819632815583">Region-specific projects</strong> for <strong id="dataartsstudio_01_0004__b16349103655811">Scope</strong>, the permission takes effect only for a specified project. When accessing <span id="dataartsstudio_01_0004__text5352191557">DataArts Studio</span>, the IAM user must switch to the region where they have been assigned the required permissions.</li></ul>
|
||||
<div class="section" id="dataartsstudio_01_0004__section3404192611368"><h4 class="sectiontitle">Notes and Constraints</h4><ul id="dataartsstudio_01_0004__ul5958194983813"><li id="dataartsstudio_01_0004__li42343448248">The <span id="dataartsstudio_01_0004__text680510344508">DARTS</span> User system role provides the permissions related to instances, workspaces, and dependent services. The operation permissions in workspaces are provided by workspace roles.</li><li id="dataartsstudio_01_0004__li84281638144015"><div class="p" id="dataartsstudio_01_0004__p1899684094017"><a name="dataartsstudio_01_0004__li84281638144015"></a><a name="li84281638144015"></a>IAM provides the following two authorization mechanisms: Note that <span id="dataartsstudio_01_0004__en-us_topic_0000002059932461_text98649123311">DataArts Studio</span> supports only the IAM role-based authorization and does not support the IAM policy-based authorization.<ul id="dataartsstudio_01_0004__en-us_topic_0000002059932461_ul16474158122614"><li id="dataartsstudio_01_0004__en-us_topic_0000002059932461_en-us_topic_0216589782_li18737181101111"><strong id="dataartsstudio_01_0004__en-us_topic_0000002059932461_b1448919334131">IAM Roles</strong>: IAM initially provides a coarse-grained authorization mechanism to define permissions based on users' job responsibilities. Only a limited number of service-level roles are available. However, traditional IAM roles are not an ideal choice for fine-grained authorization and secure access control.</li><li id="dataartsstudio_01_0004__en-us_topic_0000002059932461_li144587474243"><strong id="dataartsstudio_01_0004__en-us_topic_0000002059932461_b203801928144">IAM Policies</strong>: A type of fine-grained authorization mechanism that defines permissions required to perform operations on specific cloud resources under certain conditions. This type of authorization is more flexible and is ideal for least privilege access.</li></ul>
|
||||
</div>
|
||||
</li></ul>
|
||||
</div>
|
||||
<div class="section" id="dataartsstudio_01_0004__en-us_topic_0223009156_section9135134520119"><a name="dataartsstudio_01_0004__en-us_topic_0223009156_section9135134520119"></a><a name="en-us_topic_0223009156_section9135134520119"></a><h4 class="sectiontitle">Procedure</h4><ol id="dataartsstudio_01_0004__ol5977748151916"><li id="dataartsstudio_01_0004__li39771248101915"><a name="dataartsstudio_01_0004__li39771248101915"></a><a name="li39771248101915"></a><span>Create a user group and assign a system role to the group.</span><p><p id="dataartsstudio_01_0004__p55201136510">Log in to the IAM console using a <span id="dataartsstudio_01_0004__text132541145165011">cloud account</span>, create a user group and assign a <span id="dataartsstudio_01_0004__text3254145145017">DataArts Studio</span> system role to the group. For example, the system role can be <span id="dataartsstudio_01_0004__text325594535015">DARTS</span> Administrator or <span id="dataartsstudio_01_0004__text1225564525015">DARTS</span> User.</p>
|
||||
<p id="dataartsstudio_01_0004__en-us_topic_0223009156_p108313320205">For details, see "User Groups and Authorization" > "Creating a User Group and Assigning Permissions" in <em id="dataartsstudio_01_0004__i452011553416">Identity and Access Management User Guide</em>.</p>
|
||||
<div class="note" id="dataartsstudio_01_0004__en-us_topic_0223009156_note52501597263"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dataartsstudio_01_0004__en-us_topic_0223009156_ul114668532278"><li id="dataartsstudio_01_0004__li1834013912412">When configuring <span id="dataartsstudio_01_0004__text16819177115812">DataArts Studio</span> permissions for a user group, enter <strong id="dataartsstudio_01_0004__b78475552518"><span id="dataartsstudio_01_0004__text58471555135114">DARTS</span></strong> in the search box to search for the permissions and select the permissions to be granted to the user group, for example, <strong id="dataartsstudio_01_0004__b108471855135116"><span id="dataartsstudio_01_0004__text9847195514512">DARTS</span> User</strong>.</li><li id="dataartsstudio_01_0004__li2866162654415"><span id="dataartsstudio_01_0004__text195812112424">DataArts Studio</span> is a project-level service deployed in specific physical regions. If you select <strong id="dataartsstudio_01_0004__b189870159422">All resources</strong> for <strong id="dataartsstudio_01_0004__b29871515134214">Scope</strong>, the permission takes effect in all projects of all regions. If you select <strong id="dataartsstudio_01_0004__b1298715159424">Region-specific projects</strong> for <strong id="dataartsstudio_01_0004__b129871515104216">Scope</strong>, the permission takes effect only for a specified project. When accessing <span id="dataartsstudio_01_0004__text199751520174210">DataArts Studio</span>, the IAM user must switch to the region where they have been assigned the required permissions.</li></ul>
|
||||
</div></div>
|
||||
</p></li><li id="dataartsstudio_01_0004__li797716482198"><span>Create a user and add it to the user group.</span><p><p id="dataartsstudio_01_0004__p178485310194">Create users on the IAM console and add them to the group created in <a href="#dataartsstudio_01_0004__li39771248101915">step 1</a>.</p>
|
||||
<p id="dataartsstudio_01_0004__p108285546197">For details, see "IAM Users" > "Creating an IAM User" in <em id="dataartsstudio_01_0004__i9320185761913">Identity and Access Management User Guide</em>.</p>
|
||||
<div class="note" id="dataartsstudio_01_0004__note127591741175411"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_0004__p676134155414">An IAM user can pass the authentication and access <span id="dataartsstudio_01_0004__text9760148191813">DataArts Studio</span> through an API or SDK only if <strong id="dataartsstudio_01_0004__b10760204891816">Programmatic access</strong> is selected for <strong id="dataartsstudio_01_0004__b16760134811188">Access Type</strong> during the creation of the IAM user.</p>
|
||||
</p></li><li id="dataartsstudio_01_0004__li797716482198"><span>Create a user and add the user to the user group.</span><p><p id="dataartsstudio_01_0004__p178485310194">Create users on the IAM console and add them to the group created in <a href="#dataartsstudio_01_0004__li39771248101915">Step 1</a>.</p>
|
||||
<p id="dataartsstudio_01_0004__p108285546197">For details, see "IAM Users" > "Creating an IAM User" in <em id="dataartsstudio_01_0004__i82181740154214">Identity and Access Management User Guide</em>.</p>
|
||||
<div class="note" id="dataartsstudio_01_0004__note127591741175411"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_0004__p676134155414">An IAM user can pass the authentication and access <span id="dataartsstudio_01_0004__en-us_topic_0183235763_text57841510161910">DataArts Studio</span> through an API or SDK only if <strong id="dataartsstudio_01_0004__en-us_topic_0183235763_b1834015273813">Programmatic access</strong> is selected for <strong id="dataartsstudio_01_0004__en-us_topic_0183235763_b146218599421">Access Type</strong> during the creation of the IAM user.</p>
|
||||
</div></div>
|
||||
</p></li><li id="dataartsstudio_01_0004__li585115271418"><span>Create a custom workspace role for DARTS User, add it as a workspace member, and assign a role to the member.</span><p><p id="dataartsstudio_01_0004__p1579162432010"><span id="dataartsstudio_01_0004__text554811241949">DataArts Studio</span> workspace roles determine the permissions of DARTS User in a workspace. There are five preset roles: admin, developer, deployer, operator, and viewer. If these preset roles can meet your requirements, you do not need to create a custom workspace role. Instead, you can add the user as a workspace member and assign a preset role to the member. Otherwise, you need to create a custom role, add the users as a workspace member, and assign the custom role to the member. For details about how to create a custom workspace role, see <a href="dataartsstudio_01_0120.html">(Optional) Defining a Workspace Role</a>. For details about how to add a workspace member and assign a role to the member, see <a href="dataartsstudio_01_0117.html">Adding a Member and Assigning a Role</a>.</p>
|
||||
<p id="dataartsstudio_01_0004__p142198484228">For details about the permissions of the roles, see "<span id="dataartsstudio_01_0004__text1921247151618">DataArts Studio</span> Permissions" in <em id="dataartsstudio_01_0004__i1392847161611">Service Overview</em>.</p>
|
||||
</p></li><li id="dataartsstudio_01_0004__li585115271418"><span>Create a custom workspace role for <span id="dataartsstudio_01_0004__text399512291527">DARTS</span> User, add it as a workspace member, and assign a role to the member.</span><p><p id="dataartsstudio_01_0004__p1579162432010"><span id="dataartsstudio_01_0004__text153877412521">DataArts Studio</span> workspace roles determine the permissions of <span id="dataartsstudio_01_0004__text16388144135217">DARTS</span> User in a workspace. There are five preset roles: admin, developer, deployer, operator, and viewer. If these preset roles can meet your requirements, you do not need to create a custom workspace role. Instead, you can add the user as a workspace member and assign a preset role to the member. Otherwise, you need to create a custom role, add the users as a workspace member, and assign a custom role to the member. For details about how to create a custom workspace role, see <a href="dataartsstudio_01_0107.html">(Optional) Defining a Workspace Role</a>For details about how to add a workspace member and assign a role, see <a href="dataartsstudio_01_0117.html">Adding Workspace Members and Assigning Roles</a>.</p>
|
||||
<p id="dataartsstudio_01_0004__p142198484228">For details about the permissions of the roles, see "Permissions" in <em id="dataartsstudio_01_0004__i13174413430">Service Overview</em>.</p>
|
||||
</p></li><li id="dataartsstudio_01_0004__en-us_topic_0000001489537442_li1177513202816"><span>Log in to the console and verify permissions.</span><p><p id="dataartsstudio_01_0004__en-us_topic_0000001489537442_p1317741312289">Log in to the console using the created user and verify permissions of the user.</p>
|
||||
<ul id="dataartsstudio_01_0004__en-us_topic_0000001489537442_ul1692751312242"><li id="dataartsstudio_01_0004__en-us_topic_0000001489537442_li205729227246">Choose <strong id="dataartsstudio_01_0004__b10786936161912">Service List</strong> > <strong id="dataartsstudio_01_0004__b7825945191914">DataArts Studio</strong>. Locate a <span id="dataartsstudio_01_0004__text1421955017286">DataArts Studio</span> instance and click <strong id="dataartsstudio_01_0004__b15374182913214">Access</strong>. Check whether the workspace list is displayed.</li><li id="dataartsstudio_01_0004__en-us_topic_0000001489537442_li1857212212418">Access a service module (for example, Management Center) to which your current user has been added and check whether you can perform the operations allowed for the workspace role assigned to you.</li></ul>
|
||||
<ul id="dataartsstudio_01_0004__en-us_topic_0000001489537442_ul1692751312242"><li id="dataartsstudio_01_0004__en-us_topic_0000001489537442_li205729227246">Choose <strong id="dataartsstudio_01_0004__b1659761818434">Service List</strong> > <strong id="dataartsstudio_01_0004__b459816187431">DataArts Studio</strong>. Locate a <span id="dataartsstudio_01_0004__text1559871816439">DataArts Studio</span> instance and click <strong id="dataartsstudio_01_0004__b1459801804314">Access</strong>. Check whether the workspace list is displayed.</li><li id="dataartsstudio_01_0004__en-us_topic_0000001489537442_li1857212212418">Access a service module (for example, Management Center) to which your current user has been added and check whether you can perform the operations allowed for the workspace role assigned to you.</li></ul>
|
||||
</p></li></ol>
|
||||
</div>
|
||||
</div>
|
||||
@ -31,3 +33,10 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
<script language="JavaScript">
|
||||
<!--
|
||||
initImageViewer('.imgResize');
|
||||
var msg_imageMax = "view original image";
|
||||
var msg_imageClose = "close";
|
||||
//--></script>
|
||||
234
docs/dataartsstudio/umn/dataartsstudio_01_0005.html
Normal file
234
docs/dataartsstudio/umn/dataartsstudio_01_0005.html
Normal file
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@ -1,28 +0,0 @@
|
||||
<a name="dataartsstudio_01_0006"></a><a name="dataartsstudio_01_0006"></a>
|
||||
|
||||
<h1 class="topictitle1">(Optional) Obtaining Authentication Information</h1>
|
||||
<div id="body8662426"><p id="dataartsstudio_01_0006__p9577341160">When using <span id="dataartsstudio_01_0006__text198211624115318">DataArts Studio</span>, for example, when creating OBS links, calling APIs, or locating issues, you may need to obtain information such as access keys, project IDs, and endpoints. This section describes how to obtain such information.</p>
|
||||
<div class="section" id="dataartsstudio_01_0006__en-us_topic_0183643042_en-us_topic_0183643042_section6679105711399"><h4 class="sectiontitle">Obtaining an Access Key</h4><div class="p" id="dataartsstudio_01_0006__en-us_topic_0000001129241845_p157171154134613">To obtain an access key, perform the following steps:<ol id="dataartsstudio_01_0006__en-us_topic_0000001129241845_en-us_topic_0183643042_ol535183018588"><li id="dataartsstudio_01_0006__en-us_topic_0000001129241845_en-us_topic_0183643042_li1535103025819">Log in to the TICS console, move the cursor to the username in the upper right corner, and select <strong id="dataartsstudio_01_0006__b1698346784111913">My Credentials</strong> from the drop-down list.</li><li id="dataartsstudio_01_0006__en-us_topic_0000001129241845_en-us_topic_0183643042_li173533018584">On the <strong id="dataartsstudio_01_0006__b1957659294111913">My Credentials</strong> page, choose <span class="menucascade" id="dataartsstudio_01_0006__menucascade1220767440111913"><b><span class="uicontrol" id="dataartsstudio_01_0006__uicontrol760472201111913">Access Keys</span></b> > <b><span class="uicontrol" id="dataartsstudio_01_0006__uicontrol1320938428111913">Create Access Key</span></b></span>. See <a href="#dataartsstudio_01_0006__en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615">Figure 1</a>.<div class="fignone" id="dataartsstudio_01_0006__en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615"><a name="dataartsstudio_01_0006__en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615"></a><a name="en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615"></a><span class="figcap"><b>Figure 1 </b>Clicking Create Access Key</span><br><span><img id="dataartsstudio_01_0006__en-us_topic_0000001129241845_image20389043111611" src="en-us_image_0000002270844902.png"></span></div>
|
||||
</li><li id="dataartsstudio_01_0006__en-us_topic_0000001129241845_en-us_topic_0183643042_li1535530185815">Click <strong id="dataartsstudio_01_0006__b21601729101619">OK</strong> and save the access key file when prompted. The access key file is saved to the default download location in your browser. Open the <strong id="dataartsstudio_01_0006__b361492671616">credentials.csv</strong> file to view the access key with an AK and SK.<div class="note" id="dataartsstudio_01_0006__en-us_topic_0000001129241845_note4554158201"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dataartsstudio_01_0006__en-us_topic_0000001129241845_ul25541081906"><li id="dataartsstudio_01_0006__en-us_topic_0000001129241845_li175541819018">Only two access keys can be added for each user.</li><li id="dataartsstudio_01_0006__en-us_topic_0000001129241845_li3554138305">For security purposes, access keys are automatically downloaded only when they are generated for the first time and cannot be obtained from the console later. Keep them properly.</li></ul>
|
||||
</div></div>
|
||||
</li></ol>
|
||||
</div>
|
||||
</div>
|
||||
<div class="section" id="dataartsstudio_01_0006__section134096463399"><a name="dataartsstudio_01_0006__section134096463399"></a><a name="section134096463399"></a><h4 class="sectiontitle">Obtaining a Project ID and Account ID</h4><div class="p" id="dataartsstudio_01_0006__en-us_topic_0000001129537317_en-us_topic_0183643042_en-us_topic_0080205343_p491327619141">A project is a group of tenant resources, and an account ID corresponds to the current account. The IAM ID corresponds to the current user. You can view the project IDs, account IDs, and user IDs in different regions on the corresponding pages.<ol id="dataartsstudio_01_0006__en-us_topic_0000001129537317_ol6260200121819"><li id="dataartsstudio_01_0006__en-us_topic_0000001129537317_li19260803185">Register with and log in to the management console.</li><li id="dataartsstudio_01_0006__en-us_topic_0000001129537317_li142601019186">Hover the cursor on the username in the upper right corner and select <span class="uicontrol" id="dataartsstudio_01_0006__en-us_topic_0000001129537317_uicontrol0260130191818"><b>My Credentials</b></span> from the drop-down list.</li><li id="dataartsstudio_01_0006__en-us_topic_0000001129537317_li1026113031818">On the <strong id="dataartsstudio_01_0006__b1933425520111913">API Credentials</strong> page, obtain the account name, account ID, IAM username, and IAM user ID, and obtain the project ID from the project list.</li></ol>
|
||||
</div>
|
||||
</div>
|
||||
<div class="section" id="dataartsstudio_01_0006__section7883936162812"><h4 class="sectiontitle">Obtaining the <span id="dataartsstudio_01_0006__en-us_topic_0000001150701582_text1341912581632">DataArts Studio</span> Instance ID and Workspace ID</h4><div class="p" id="dataartsstudio_01_0006__en-us_topic_0000001150701582_en-us_topic_0183643042_en-us_topic_0080205343_p491327619141">You can obtain them from the URI on the <span id="dataartsstudio_01_0006__en-us_topic_0000001150701582_text973312442920">DataArts Studio</span> console.<ol id="dataartsstudio_01_0006__en-us_topic_0000001150701582_ol6260200121819"><li id="dataartsstudio_01_0006__en-us_topic_0000001150701582_li554111518432">On the homepage of the <span id="dataartsstudio_01_0006__en-us_topic_0000001150701582_en-us_topic_0187412481_text88021121155819">DataArts Studio</span> console, select a workspace, and click a module, for example, <strong id="dataartsstudio_01_0006__b336323133915">Management Center</strong>.</li><li id="dataartsstudio_01_0006__en-us_topic_0000001150701582_li142601019186">On the <strong id="dataartsstudio_01_0006__b4975726133910">Management Center</strong> page, obtain the values of <strong id="dataartsstudio_01_0006__b3975126143917">instanceId</strong> and <strong id="dataartsstudio_01_0006__b9975182603910">workspace</strong> in the browser address bar, which are the <span id="dataartsstudio_01_0006__en-us_topic_0000001150701582_text1545312442213">DataArts Studio</span> instance ID and workspace ID, respectively.<div class="p" id="dataartsstudio_01_0006__en-us_topic_0000001150701582_p187071943102319">As shown in <a href="#dataartsstudio_01_0006__en-us_topic_0000001150701582_fig394214118237">Figure 2</a>, the instance ID is <strong id="dataartsstudio_01_0006__b1545854719111913">6b88…2688</strong>, and the workspace ID is <strong id="dataartsstudio_01_0006__b267920851111913">1dd3bc…d93f0</strong>.<div class="fignone" id="dataartsstudio_01_0006__en-us_topic_0000001150701582_fig394214118237"><a name="dataartsstudio_01_0006__en-us_topic_0000001150701582_fig394214118237"></a><a name="en-us_topic_0000001150701582_fig394214118237"></a><span class="figcap"><b>Figure 2 </b>Obtaining the instance ID and workspace ID</span><br><span><img id="dataartsstudio_01_0006__image95718161852" src="en-us_image_0000002270956052.png"></span></div>
|
||||
</div>
|
||||
</li></ol>
|
||||
</div>
|
||||
</div>
|
||||
<div class="section" id="dataartsstudio_01_0006__section16737727193510"><h4 class="sectiontitle">Obtaining an Endpoint</h4><p id="dataartsstudio_01_0006__en-us_topic_0000001089114743_p15821453115417">An endpoint is the <strong id="dataartsstudio_01_0006__en-us_topic_0000001089114743_b1282153175410">request address</strong> for calling an API. Endpoints vary depending on services and regions.</p>
|
||||
<p id="dataartsstudio_01_0006__en-us_topic_0000001089114743_p27115277328">You can obtain endpoints from <a href="https://docs.otc.t-systems.com/en-us/endpoint/index.html" target="_blank" rel="noopener noreferrer">Regions and Endpoints</a>.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_12_0002.html">Preparations</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@ -5,15 +5,17 @@
|
||||
</div>
|
||||
<div>
|
||||
<ul class="ullinks">
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0005_0.html">Data Sources</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0005.html">Data Sources Supported by DataArts Studio</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0009.html">Managing Data Connections</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1299.html">Creating a DataArts Studio Data Connection</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0010.html">Migrating Resources</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0009.html">Configuring DataArts Studio Data Connection Parameters</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_5105.html">Configuring Environment Isolation for a Workspace in Enterprise Mode</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0010.html">Configuring DataArts Studio Resource Migration</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0350.html">Tutorials</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_5105.html">Configuring Environment Isolation for a DataArts Studio Workspace in Enterprise Mode</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0350.html">Typical Scenarios for Using Management Center</a></strong><br>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
|
||||
@ -1,36 +1,42 @@
|
||||
<a name="dataartsstudio_01_0009"></a><a name="dataartsstudio_01_0009"></a>
|
||||
|
||||
<h1 class="topictitle1">Managing Data Connections</h1>
|
||||
<h1 class="topictitle1">Configuring <span id="text11850151613114">DataArts Studio</span> Data Connection Parameters</h1>
|
||||
<div id="body8662426"></div>
|
||||
<div>
|
||||
<ul class="ullinks">
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1299.html">Creating a Data Connection</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1300.html">DWS Connection Parameters</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1300.html">Configuring a DWS Connection</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1301.html">DLI Connection Parameters</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1301.html">Configuring a DLI Connection</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1306.html">MRS Hive Connection Parameters</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1306.html">Configuring an MRS Hive Connection</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1307.html">MRS HBase Connection Parameters</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1307.html">Configuring an MRS HBase Connection</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1308.html">MRS Kafka Connection Parameters</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1308.html">Configuring an MRS Kafka Connection</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1314.html">MRS Spark Connection Parameters</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1314.html">Configuring an MRS Spark Connection</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1309.html">MRS ClickHouse Connection Parameters</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1309.html">Configuring an MRS ClickHouse Connection</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1310.html">MRS Hetu Connection Parameters</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1310.html">Configuring an MRS Hetu Connection</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1311.html">MRS Impala Connection Parameters</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1311.html">Configuring an MRS Impala Connection</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1390.html">Doris Connection Parameters</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1390.html">Configuring an MRS Doris Connection</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1321.html">OpenSource ClickHouse Connection Parameters</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1303.html">Configuring an RDS Connection</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1303.html">RDS Connection Parameters</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1302.html">Configuring an Oracle Connection</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1302.html">Oracle Connection Parameters</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1305.html">Configuring a Host Connection</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1305.html">Host Connection Parameters</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1620.html">Rest Client Connection Parameters</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1509.html">Redis Connection Parameters</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1510.html">SAP HANA Connection Parameters</a></strong><br>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
|
||||
@ -1,32 +1,31 @@
|
||||
<a name="dataartsstudio_01_0010"></a><a name="dataartsstudio_01_0010"></a>
|
||||
|
||||
<h1 class="topictitle1">Migrating Resources</h1>
|
||||
<h1 class="topictitle1">Configuring <span id="text654542017149">DataArts Studio</span> Resource Migration</h1>
|
||||
<div id="body1564469667050"><p id="dataartsstudio_01_0010__p118481119122413">To migrate resources in one workspace to another, you can use the resource migration function provided by <span id="dataartsstudio_01_0010__text1084871952418">DataArts Studio</span>.</p>
|
||||
<p id="dataartsstudio_01_0010__p1418713612214">Resources that can be migrated include the data connections in Management Center.</p>
|
||||
<div class="section" id="dataartsstudio_01_0010__section13321703214"><h4 class="sectiontitle">Prerequisites</h4><ul id="dataartsstudio_01_0010__ul1468814513610"><li id="dataartsstudio_01_0010__li1871417481229">Resources can be imported from OBS or a local path.</li><li id="dataartsstudio_01_0010__li18688165861">There are resources that can be migrated. For details on how to create data connections, see <a href="dataartsstudio_01_0009.html">Managing Data Connections</a>.</li></ul>
|
||||
<div class="p" id="dataartsstudio_01_0010__p19944775552">Resources can be imported from OBS or a local path. Resources that can be migrated include the following service data:<ul id="dataartsstudio_01_0010__ul2017875755515"><li id="dataartsstudio_01_0010__li166451919573">Data connections created in Management Center</li><li id="dataartsstudio_01_0010__li1619512355599">CDM jobs created in DataArts Migration, including the links in jobs</li><li id="dataartsstudio_01_0010__li135318184562">Scripts and jobs that have been submitted in DataArts Factory. By default, when jobs are exported, their dependent scripts and resources are not exported.</li></ul>
|
||||
</div>
|
||||
<div class="section" id="dataartsstudio_01_0010__section197713411677"><h4 class="sectiontitle">Constraints</h4><ul id="dataartsstudio_01_0010__ul8306105215815"><li id="dataartsstudio_01_0010__li29038123813">Only an exported .zip file can be imported. During the import, the system verifies the resources in the file.</li><li id="dataartsstudio_01_0010__li138111155474">For security concerns, passwords of connections are not exported when the connections are exported. You need to enter the passwords when importing the connections.</li><li id="dataartsstudio_01_0010__li15824726191112">The size of the file to be imported from an OBS bucket or local path cannot exceed 10 MB.</li></ul>
|
||||
<div class="section" id="dataartsstudio_01_0010__section197713411677"><h4 class="sectiontitle">Constraints</h4><ul id="dataartsstudio_01_0010__ul8306105215815"><li id="dataartsstudio_01_0010__li29038123813">Only an exported .zip file can be imported. During the import, the system verifies the resources in the file.</li><li id="dataartsstudio_01_0010__li138111155474">For security concerns, passwords of connections are not exported when the connections are exported. You need to enter the passwords when importing the connections.</li><li id="dataartsstudio_01_0010__li15824726191112">The file to be imported from an OBS bucket or local path cannot be larger than 10 MB.</li></ul>
|
||||
</div>
|
||||
<div class="section" id="dataartsstudio_01_0010__section84691754394"><h4 class="sectiontitle">Exporting a Resource</h4><ol id="dataartsstudio_01_0010__ol27771443151310"><li class="msonormal" id="dataartsstudio_01_0010__li19275175611217"><span>Log in to the <span id="dataartsstudio_01_0010__en-us_topic_0000001397651462_text5574448155719">DataArts Studio</span> console by following the instructions in <a href="dataartsstudio_01_0001.html">Accessing the DataArts Studio Instance Console</a>.</span></li><li class="msonormal" id="dataartsstudio_01_0010__li11777174316135"><span>On the <span id="dataartsstudio_01_0010__en-us_topic_0000001503129708_text82024413513">DataArts Studio</span> console, locate a workspace and click <strong id="dataartsstudio_01_0010__en-us_topic_0000001503129708_b820215416513">Management Center</strong>.</span></li><li id="dataartsstudio_01_0010__li1777715436133"><span>In the navigation pane, choose <strong id="dataartsstudio_01_0010__b89281489484">Migrate Resources</strong>.</span><p><div class="fignone" id="dataartsstudio_01_0010__fig112561281915"><span class="figcap"><b>Figure 1 </b>Migrating Resources</span><br><span><img id="dataartsstudio_01_0010__image1235516539128" src="en-us_image_0000002305439969.png"></span></div>
|
||||
<div class="section" id="dataartsstudio_01_0010__section84691754394"><h4 class="sectiontitle">Exporting a Resource</h4><ol id="dataartsstudio_01_0010__ol27771443151310"><li id="dataartsstudio_01_0010__li2848154165110"><span>Log in to the <span id="dataartsstudio_01_0010__en-us_topic_0000001987511677_en-us_topic_0000001127311125_text5574448155719">DataArts Studio</span> console by following the instructions in <a href="dataartsstudio_01_0001.html">Accessing the DataArts Studio Instance Console</a>.</span></li><li class="msonormal" id="dataartsstudio_01_0010__li11777174316135"><span>On the <span id="dataartsstudio_01_0010__en-us_topic_0000001503129708_text82024413513">DataArts Studio</span> console, locate a workspace and click <strong id="dataartsstudio_01_0010__en-us_topic_0000001503129708_b820215416513">Management Center</strong>.</span></li><li id="dataartsstudio_01_0010__li1777715436133"><span>In the navigation pane, choose <strong id="dataartsstudio_01_0010__b89281489484">Migrate Resources</strong>.</span><p><div class="fignone" id="dataartsstudio_01_0010__fig112561281915"><span class="figcap"><b>Figure 1 </b>Migrating Resources</span><br><span><img id="dataartsstudio_01_0010__image37672010183115" src="en-us_image_0000002269115061.png" title="Click to enlarge" class="imgResize"></span></div>
|
||||
<p id="dataartsstudio_01_0010__p1880613432136"></p>
|
||||
</p></li><li id="dataartsstudio_01_0010__li1677712439132"><span>Click <span class="wintitle" id="dataartsstudio_01_0010__wintitle5751111319203"><b>Create Export Task</b></span> to configure the file name and the OBS path for saving the file. If OBS is unavailable, you only need to set the file name.</span><p><div class="fignone" id="dataartsstudio_01_0010__fig173183613715"><span class="figcap"><b>Figure 2 </b>Export Task</span><br><span><img id="dataartsstudio_01_0010__image1360771419536" src="en-us_image_0000002305406933.png"></span></div>
|
||||
</p></li><li id="dataartsstudio_01_0010__li1677712439132"><span>Click <span class="wintitle" id="dataartsstudio_01_0010__wintitle277211311988"><b>Export Task</b></span> to configure the file name and the OBS path for saving the file. If OBS is unavailable, you only need to set the file name.</span><p><div class="fignone" id="dataartsstudio_01_0010__fig173183613715"><span class="figcap"><b>Figure 2 </b>Export Task</span><br><span><img id="dataartsstudio_01_0010__image1360771419536" src="en-us_image_0000002234238624.png" title="Click to enlarge" class="imgResize"></span></div>
|
||||
<p id="dataartsstudio_01_0010__p1897784316235"></p>
|
||||
</p></li><li id="dataartsstudio_01_0010__li117772430138"><span>Click <span class="uicontrol" id="dataartsstudio_01_0010__uicontrol1681291711214"><b>Next</b></span> and select the resources to export.</span><p><p id="dataartsstudio_01_0010__p14589150145312"></p>
|
||||
</p></li><li id="dataartsstudio_01_0010__li10777124381318"><span>Click <strong id="dataartsstudio_01_0010__b1671194112214">Next</strong> and wait until the export is complete. The resource package is exported to the OBS path you have set.</span><p><div class="fignone" id="dataartsstudio_01_0010__fig115243158274"><span class="figcap"><b>Figure 3 </b>Export completed</span><br><span><img id="dataartsstudio_01_0010__image726804155317" src="en-us_image_0000002305406921.png"></span></div>
|
||||
</p></li><li id="dataartsstudio_01_0010__li10777124381318"><span>Click <strong id="dataartsstudio_01_0010__b1671194112214">Next</strong> and wait until the export is complete. The resource package is exported to the OBS path you have set.</span><p><div class="fignone" id="dataartsstudio_01_0010__fig115243158274"><span class="figcap"><b>Figure 3 </b>Export completed</span><br><span><img id="dataartsstudio_01_0010__image726804155317" src="en-us_image_0000002269198077.png" title="Click to enlarge" class="imgResize"></span></div>
|
||||
<p id="dataartsstudio_01_0010__p4442736153913">If no result is displayed in 1 minute, the export fails. Try again. If the failure persists, contact <span id="dataartsstudio_01_0010__text144183817145">the customer service or technical support</span>.</p>
|
||||
</p></li><li id="dataartsstudio_01_0010__li1377714301310"><span>After the export is complete, you can click <strong id="dataartsstudio_01_0010__b10365745162217">Download</strong> in the row of the corresponding migration task to download the exported resource package.</span><p><div class="fignone" id="dataartsstudio_01_0010__fig1756933011329"><span class="figcap"><b>Figure 4 </b>Downloading the exported result</span><br><span><img id="dataartsstudio_01_0010__image158429583322" src="en-us_image_0000002305406917.png"></span></div>
|
||||
</p></li><li id="dataartsstudio_01_0010__li1377714301310"><span>After the export is complete, you can click <strong id="dataartsstudio_01_0010__b10365745162217">Download</strong> in the row of the corresponding migration task to download the exported resource package.</span><p><div class="fignone" id="dataartsstudio_01_0010__fig1756933011329"><span class="figcap"><b>Figure 4 </b>Downloading the exported result</span><br><span><img id="dataartsstudio_01_0010__image158429583322" src="en-us_image_0000002234078784.png" title="Click to enlarge" class="imgResize"></span></div>
|
||||
<p id="dataartsstudio_01_0010__p877675153316"></p>
|
||||
</p></li></ol>
|
||||
</div>
|
||||
<div class="section" id="dataartsstudio_01_0010__section3797132414396"><h4 class="sectiontitle">Importing a Resource</h4><ol id="dataartsstudio_01_0010__ol2078921718166"><li class="msonormal" id="dataartsstudio_01_0010__li7243143919166"><span>Log in to the <span id="dataartsstudio_01_0010__en-us_topic_0000001397651462_text5574448155719_1">DataArts Studio</span> console by following the instructions in <a href="dataartsstudio_01_0001.html">Accessing the DataArts Studio Instance Console</a>.</span></li><li class="msonormal" id="dataartsstudio_01_0010__li1724323910164"><span>On the <span id="dataartsstudio_01_0010__en-us_topic_0000001503129708_text82024413513_1">DataArts Studio</span> console, locate a workspace and click <strong id="dataartsstudio_01_0010__en-us_topic_0000001503129708_b820215416513_1">Management Center</strong>.</span></li><li id="dataartsstudio_01_0010__li1489119478163"><span>In the navigation pane, choose <strong id="dataartsstudio_01_0010__en-us_topic_0182554812_b89281489484">Migrate Resources</strong>.</span><p><div class="fignone" id="dataartsstudio_01_0010__en-us_topic_0182554812_fig112561281915"><span class="figcap"><b>Figure 5 </b>Migrating Resources</span><br><span><img id="dataartsstudio_01_0010__en-us_topic_0182554812_image1235516539128" src="en-us_image_0000002305439969.png"></span></div>
|
||||
<div class="section" id="dataartsstudio_01_0010__section3797132414396"><h4 class="sectiontitle">Importing a Resource</h4><ol id="dataartsstudio_01_0010__ol2078921718166"><li id="dataartsstudio_01_0010__li1489119478163"><span>In the navigation pane, choose <strong id="dataartsstudio_01_0010__en-us_topic_0182554812_b89281489484">Migrate Resources</strong>.</span><p><div class="fignone" id="dataartsstudio_01_0010__en-us_topic_0182554812_fig112561281915"><span class="figcap"><b>Figure 5 </b>Migrating Resources</span><br><span><img id="dataartsstudio_01_0010__en-us_topic_0182554812_image37672010183115" src="en-us_image_0000002269115061.png" title="Click to enlarge" class="imgResize"></span></div>
|
||||
<p id="dataartsstudio_01_0010__en-us_topic_0182554812_p1880613432136"></p>
|
||||
</p></li><li id="dataartsstudio_01_0010__li5789191718162"><span>Click <span class="wintitle" id="dataartsstudio_01_0010__wintitle03856552527"><b>Create Import Task</b></span>. On the displayed page, select an import mode and set the OBS bucket and path or local path that stores resources. The resource to be imported must be a .zip file exported from the console.</span><p><div class="fignone" id="dataartsstudio_01_0010__fig96755416444"><span class="figcap"><b>Figure 6 </b>Configuring the path that stores the resources to be imported</span><br><span><img id="dataartsstudio_01_0010__image525261911410" src="en-us_image_0000002270847054.png"></span></div>
|
||||
</p></li><li id="dataartsstudio_01_0010__li5789191718162"><span>Click <span class="wintitle" id="dataartsstudio_01_0010__wintitle164412318101"><b>Import File</b></span>. On the displayed page, select an import mode and set the OBS bucket and path or local path that stores resources. The resource to be imported must be a .zip file exported from the console.</span><p><div class="fignone" id="dataartsstudio_01_0010__fig96755416444"><span class="figcap"><b>Figure 6 </b>Configuring the path that stores the resources to be imported</span><br><span><img id="dataartsstudio_01_0010__image525261911410" src="en-us_image_0000002234238636.png" title="Click to enlarge" class="imgResize"></span></div>
|
||||
<p id="dataartsstudio_01_0010__p14635161511617"></p>
|
||||
</p></li><li id="dataartsstudio_01_0010__li1994717161716"><span>Click <strong id="dataartsstudio_01_0010__b17293532182716">Create Import Task</strong> and upload a .zip resource file that you have exported.</span></li><li id="dataartsstudio_01_0010__li278991720168"><span>Click <span class="uicontrol" id="dataartsstudio_01_0010__uicontrol8778518297"><b>Next</b></span> and select the resources to import.</span><p><p id="dataartsstudio_01_0010__p0141152812535"></p>
|
||||
</p></li><li id="dataartsstudio_01_0010__li878961741610"><span>If you select <strong id="dataartsstudio_01_0010__b1151742615270">DataSource</strong>, click <span class="uicontrol" id="dataartsstudio_01_0010__uicontrol2080213318373"><b>Next</b></span> to configure a data connection.</span><p><div class="fignone" id="dataartsstudio_01_0010__fig13935181811416"><span class="figcap"><b>Figure 7 </b>Configuring a data connection</span><br><span><img id="dataartsstudio_01_0010__image135113918593" src="en-us_image_0000002305406925.png"></span></div>
|
||||
</p></li><li id="dataartsstudio_01_0010__li107894175164"><span>Click <span class="uicontrol" id="dataartsstudio_01_0010__en-us_topic_0182554812_uicontrol133851455165217"><b>Next</b></span> and wait until the import task is delivered. When the import task is delivered successfully, the system displays message "Import task started."</span><p><div class="fignone" id="dataartsstudio_01_0010__fig1353193313612"><span class="figcap"><b>Figure 8 </b>Import task started</span><br><span><img id="dataartsstudio_01_0010__image1433519459184" src="en-us_image_0000002305439965.png"></span></div>
|
||||
</p></li><li id="dataartsstudio_01_0010__li1994717161716"><span>Click <span class="wintitle" id="dataartsstudio_01_0010__wintitle1779152014174"><b>Import File</b></span> and upload resources. a .zip resource file that you have exported.</span></li><li id="dataartsstudio_01_0010__li278991720168"><span>Click <span class="uicontrol" id="dataartsstudio_01_0010__uicontrol8778518297"><b>Next</b></span> and select the resources to import.</span><p><p id="dataartsstudio_01_0010__p0141152812535"></p>
|
||||
</p></li><li id="dataartsstudio_01_0010__li878961741610"><span>If you select <strong id="dataartsstudio_01_0010__b1151742615270">DataSource</strong>, click <span class="uicontrol" id="dataartsstudio_01_0010__uicontrol2080213318373"><b>Next</b></span> to configure a data connection.</span><p><div class="fignone" id="dataartsstudio_01_0010__fig13935181811416"><span class="figcap"><b>Figure 7 </b>Configuring a data connection</span><br><span><img id="dataartsstudio_01_0010__image135113918593" src="en-us_image_0000002269198065.png" title="Click to enlarge" class="imgResize"></span></div>
|
||||
</p></li><li id="dataartsstudio_01_0010__li107894175164"><span>Click <span class="uicontrol" id="dataartsstudio_01_0010__en-us_topic_0182554812_uicontrol133851455165217"><b>Next</b></span> and wait until the import task is delivered. When the import task is delivered successfully, the system displays message "Import task started."</span><p><div class="fignone" id="dataartsstudio_01_0010__fig1353193313612"><span class="figcap"><b>Figure 8 </b>Import task started</span><br><span><img id="dataartsstudio_01_0010__image1433519459184" src="en-us_image_0000002234238656.png" title="Click to enlarge" class="imgResize"></span></div>
|
||||
<p id="dataartsstudio_01_0010__p515218324354"></p>
|
||||
</p></li><li id="dataartsstudio_01_0010__li578971731610"><span>Click <strong id="dataartsstudio_01_0010__b20842182352310">OK</strong>. You can view the import result in the resource migration task list.</span><p><div class="p" id="dataartsstudio_01_0010__p1025084483014">Subtasks that fail are marked in red. You can click their names to view the failure causes.<div class="fignone" id="dataartsstudio_01_0010__fig19832184619289"><span class="figcap"><b>Figure 9 </b>Viewing the import result</span><br><span><img id="dataartsstudio_01_0010__image07738567282" src="en-us_image_0000002305406929.png"></span></div>
|
||||
</p></li><li id="dataartsstudio_01_0010__li578971731610"><span>Click <strong id="dataartsstudio_01_0010__b20842182352310">OK</strong>. You can view the import result in the resource migration task list.</span><p><div class="p" id="dataartsstudio_01_0010__p1025084483014">Subtasks that fail are marked in red. You can click their names to view the failure causes.<div class="fignone" id="dataartsstudio_01_0010__fig19832184619289"><span class="figcap"><b>Figure 9 </b>Viewing the import result</span><br><span><img id="dataartsstudio_01_0010__image07738567282" src="en-us_image_0000002269198105.png" title="Click to enlarge" class="imgResize"></span></div>
|
||||
</div>
|
||||
<p id="dataartsstudio_01_0010__p2904906357"></p>
|
||||
</p></li></ol>
|
||||
@ -38,3 +37,10 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
<script language="JavaScript">
|
||||
<!--
|
||||
initImageViewer('.imgResize');
|
||||
var msg_imageMax = "view original image";
|
||||
var msg_imageClose = "close";
|
||||
//--></script>
|
||||
@ -1,20 +1,22 @@
|
||||
<a name="dataartsstudio_01_0012"></a><a name="dataartsstudio_01_0012"></a>
|
||||
|
||||
<h1 class="topictitle1">DataArts Migration</h1>
|
||||
<h1 class="topictitle1">DataArts Migration (CDM Jobs)</h1>
|
||||
<div id="body1563878452040"></div>
|
||||
<div>
|
||||
<ul class="ullinks">
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0013.html">Overview</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0015.html">Constraints</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0015.html">Notes and Constraints</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0215.html">Supported Data Sources</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0017.html">Managing Clusters</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0017.html">Creating and Managing a CDM Cluster</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0023.html">Managing Links</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0023.html">Creating a Link in a CDM Cluster</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0081.html">Managing Jobs</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0081.html">Creating a Job in a CDM Cluster</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0114.html">Using Macro Variables of Date and Time</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0018.html">Improving Migration Performance</a></strong><br>
|
||||
</li>
|
||||
|
||||
@ -4,7 +4,7 @@
|
||||
<div id="body1563878452040"><p id="dataartsstudio_01_0013__p4422519145022">DataArts Migration is an efficient and easy-to-use data integration service. Based on the big data migration to the cloud and intelligent data lake solutions, CDM provides easy-to-use migration capabilities and can integrate various types of data sources into the data lake, which simplifies data source migration and integration and improves efficiency for you.</p>
|
||||
<p id="dataartsstudio_01_0013__p1159813425254">In this document, DataArts Migration refers to Cloud Data Migration (CDM).</p>
|
||||
<div class="section" id="dataartsstudio_01_0013__en-us_topic_0108275398_section69302054123512"><h4 class="sectiontitle">Introduction to CDM</h4><p id="dataartsstudio_01_0013__en-us_topic_0108275398_p52164437151344">CDM uses a distributed compute framework and concurrent processing techniques to help you migrate enterprise data in batches without any downtime and rapidly build desired data structures.</p>
|
||||
<div class="fignone" id="dataartsstudio_01_0013__fig168071431636"><span class="figcap"><b>Figure 1 </b>CDM</span><br><span><img id="dataartsstudio_01_0013__image345113512419" src="en-us_image_0000002270791492.png"></span></div>
|
||||
<div class="fignone" id="dataartsstudio_01_0013__fig168071431636"><span class="figcap"><b>Figure 1 </b>CDM</span><br><span><img id="dataartsstudio_01_0013__image345113512419" src="en-us_image_0000002234076228.png" title="Click to enlarge" class="imgResize"></span></div>
|
||||
</div>
|
||||
<div class="section" id="dataartsstudio_01_0013__en-us_topic_0108275398_section8948312133415"><h4 class="sectiontitle">Functions</h4><ul id="dataartsstudio_01_0013__en-us_topic_0108275398_ul20174521145718"><li id="dataartsstudio_01_0013__en-us_topic_0108275398_li51745219571"><strong id="dataartsstudio_01_0013__en-us_topic_0108275398_b13757734033">Table/file/entire DB migration</strong><p id="dataartsstudio_01_0013__en-us_topic_0108275398_p3767173325717">Tables or files can be migrated in batches. An entire database can be migrated between homogeneous and heterogeneous databases. A job can migrate hundreds of tables.</p>
|
||||
</li><li id="dataartsstudio_01_0013__en-us_topic_0108275398_li123531673310"><strong id="dataartsstudio_01_0013__en-us_topic_0108275398_b13373124715415">Incremental data migration</strong><p id="dataartsstudio_01_0013__en-us_topic_0108275398_p666246937">CDM supports incremental migration of files, relational databases, and HBase/CloudTable, as well as with WHERE clauses and macro variables of date and time.</p>
|
||||
@ -17,12 +17,19 @@
|
||||
</div>
|
||||
<div class="section" id="dataartsstudio_01_0013__section102647379456"><h4 class="sectiontitle">Migration Principles</h4><p class="msonormal" id="dataartsstudio_01_0013__en-us_topic_0163295208_p22430675">When a tenant uses CDM, the CDM system provisions a fully-managed CDM instance in the tenant's VPC. The instance allows only console and RESTful API access. Therefore the tenant cannot access the instance through other interfaces (such as SSH). This ensures data isolation between CDM tenants, prevents data leakage, and ensures transmission security during data migration between different cloud services in a VPC. Tenants can also use the VPN to migrate data from the on-premises data center to cloud services to ensure migration security.</p>
|
||||
<p id="dataartsstudio_01_0013__en-us_topic_0163295208_p1908150154719">CDM works in push-pull mode. CDM pulls data from the migration source and pushes the data to the migration destination. Data access operations are initiated by CDM. SSL will be used if the data source (such as RDS) supports it. During the migration, the usernames and passwords of the migration source and destination are required. Such information is stored in the database of the CDM instance. Protecting such information is critical to ensure CDM security.</p>
|
||||
<div class="fignone" id="dataartsstudio_01_0013__en-us_topic_0163295208_fig913012503618"><span class="figcap"><b>Figure 2 </b>Migration principles</span><br><span><img id="dataartsstudio_01_0013__en-us_topic_0163295208_image9828541123015" src="en-us_image_0000002270791496.png"></span></div>
|
||||
<div class="fignone" id="dataartsstudio_01_0013__en-us_topic_0163295208_fig913012503618"><span class="figcap"><b>Figure 2 </b>Migration principles</span><br><span><img id="dataartsstudio_01_0013__en-us_topic_0163295208_image9828541123015" src="en-us_image_0000002269115445.png" title="Click to enlarge" class="imgResize"></span></div>
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0012.html">DataArts Migration</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0012.html">DataArts Migration (CDM Jobs)</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
<script language="JavaScript">
|
||||
<!--
|
||||
initImageViewer('.imgResize');
|
||||
var msg_imageMax = "view original image";
|
||||
var msg_imageClose = "close";
|
||||
//--></script>
|
||||
File diff suppressed because it is too large
Load Diff
@ -1,11 +1,11 @@
|
||||
<a name="dataartsstudio_01_0015"></a><a name="dataartsstudio_01_0015"></a>
|
||||
|
||||
<h1 class="topictitle1">Constraints</h1>
|
||||
<div id="body8662426"><div class="section" id="dataartsstudio_01_0015__en-us_topic_0108275404_section41741324807"><h4 class="sectiontitle"><strong id="dataartsstudio_01_0015__en-us_topic_0108275404_b1491912352113">CDM System Constraints</strong></h4><ol id="dataartsstudio_01_0015__en-us_topic_0108275404_ol387012301638"><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li1687013307318">You cannot modify the flavor of an existing cluster. If you require a higher flavor, create a cluster with your desired flavor.</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li348111171418">Arm CDM clusters do not support agents. The CDM cluster version (Arm or x86) is determined by the architecture of underlying resources.</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li15870183017315">CDM does not support the function of controlling the data migration speed. Therefore, do not perform data migration during peak hours. </li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li102008075610">During data migration, CDM imposes pressure on the data source. You are advised to create a database account for data migration and configure an account policy to reduce the resource consumption of the data source. For example, you can configure a policy to delete the connections of the account when the CPU usage exceeds 30% to prevent impact on services.</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li114851637462">The baseline and maximum bandwidths of the NIC of the cdm.large CDM instance is 0.8 Gbit/s and 3 Gbit/s, respectively. The theoretical maximum volume of data that can be transmitted per instance per day is about 8 TB. Similarly, the baseline and maximum bandwidths of the NIC of the cdm.xlarge instance are 4 Gbit/s and 10 Gbit/s, respectively, and the theoretical maximum volume of data that can be transmitted per instance per day is about 40 TB. The baseline and maximum bandwidths of the NIC of the cdm.4xlarge instance is 36 Gbit/s and 40 Gbit/s, respectively, and the theoretical maximum volume of data that can be transmitted per instance per day is about 360 TB. You can use multiple CDM instances if you want faster data transfer.<p id="dataartsstudio_01_0015__en-us_topic_0108275404_en-us_topic_0108275404_p1551615411466"><a name="dataartsstudio_01_0015__en-us_topic_0108275404_li114851637462"></a><a name="en-us_topic_0108275404_li114851637462"></a>The actual amount of data that can be migrated in a day depends on the data source type, the read and write performance of the source and destination, and the actual available bandwidth. Typically you can migrate as much as 8 TB per day (large file migration to OBS) using the cdm.large instance. It is recommended that you test the speed with a small amount of data before migration.</p>
|
||||
<h1 class="topictitle1">Notes and Constraints</h1>
|
||||
<div id="body8662426"><div class="section" id="dataartsstudio_01_0015__en-us_topic_0108275404_section41741324807"><h4 class="sectiontitle"><strong id="dataartsstudio_01_0015__en-us_topic_0108275404_b1491912352113">CDM System Notes and Constraints</strong></h4><ol id="dataartsstudio_01_0015__en-us_topic_0108275404_ol387012301638"><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li1687013307318">You cannot modify the flavor of an existing cluster. If you require a higher flavor, create a cluster with your desired flavor.</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li348111171418">The CDM cluster version (Arm or x86) is determined by the architecture of underlying resources.</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li15870183017315">CDM does not support the function of controlling the data migration speed. Therefore, do not perform data migration during peak hours. </li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li102008075610">During data migration, CDM imposes pressure on the data source. You are advised to create a database account for data migration and configure an account policy to reduce the resource consumption of the data source. For example, you can configure a policy to delete the connections of the account when the CPU usage exceeds 30% to prevent impact on services.</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li114851637462">The baseline and maximum bandwidths of the NIC of the cdm.large CDM instance is 0.8 Gbit/s and 3 Gbit/s, respectively. The theoretical maximum volume of data that can be transmitted per instance per day is about 8 TB. Similarly, the baseline and maximum bandwidths of the NIC of the cdm.xlarge instance are 4 Gbit/s and 10 Gbit/s, respectively, and the theoretical maximum volume of data that can be transmitted per instance per day is about 40 TB. The baseline and maximum bandwidths of the NIC of the cdm.4xlarge instance is 36 Gbit/s and 40 Gbit/s, respectively, and the theoretical maximum volume of data that can be transmitted per instance per day is about 360 TB. You can use multiple CDM instances if you want faster data transfer.<p id="dataartsstudio_01_0015__en-us_topic_0108275404_en-us_topic_0108275404_p1551615411466"><a name="dataartsstudio_01_0015__en-us_topic_0108275404_li114851637462"></a><a name="en-us_topic_0108275404_li114851637462"></a>The actual amount of data that can be migrated in a day depends on the data source type, the read and write performance of the source and destination, and the actual available bandwidth. Typically you can migrate as much as 8 TB per day (large file migration to OBS) using the cdm.large instance. It is recommended that you test the speed with a small amount of data before migration.</p>
|
||||
</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li758482013815">CDM supports incremental file migration (by skipping repeated files), but does not support resumable transfer.<p id="dataartsstudio_01_0015__en-us_topic_0108275404_p53495242813"><a name="dataartsstudio_01_0015__en-us_topic_0108275404_li758482013815"></a><a name="en-us_topic_0108275404_li758482013815"></a>For example, if three files are to be migrated and the second file fails to be migrated due to the network fault. When the migration task is started again, the first file is skipped. The second file, however, cannot be migrated from the point where the fault occurs, but can only be migrated again.</p>
|
||||
</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li195974311420">During file migration, a single task supports millions of files. If there are too many files in the directory to be migrated, you are advised to split the files into different directories and create multiple tasks.</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li10481165219410">You can export links and jobs configured on CDM to a local directory. To ensure password security, CDM does not export the link password of the corresponding data source. Therefore, before importing job configurations to CDM, you need to manually input the password in the exported JSON file or configure the password in the import dialog box.</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li374145718417">The cluster cannot automatically upgrade to a new version. You need to use the job export and import functions to upgrade the cluster to the new version.</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li448111114516">If OBS is unavailable, CDM does not automatically back up users' job configurations. You need to export and back up configuration data using the export function.</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li11946809112"><span id="dataartsstudio_01_0015__en-us_topic_0108275404_ph207651219381">If VPC peering connection is configured, the peer VPC subnet may overlap with the CDM management network. As a result, data sources in the peer VPC cannot be accessed. You are advised to use the public network for cross-VPC data migration, or contact the administrator to add specific routes to the VPC peering connection in the CDM background.</span></li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li193610315569">If the destination of a CDM job is a DWS or NewSQL database, constraints of the source end, such as the primary key and unique index, cannot be migrated together.</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li9139811175311">When performing a CDM job, ensure that the JSON file formats of the two clusters are the same so that jobs can be imported from the source cluster to the destination cluster.</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li122981410751">If a running job is interrupted unexpectedly, the data that has been written to the destination will not be deleted. You must manually delete the data if needed.</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li19577619321">The size of a file to be transferred cannot exceed 1 TB.</li></ol>
|
||||
</div>
|
||||
<div class="section" id="dataartsstudio_01_0015__en-us_topic_0108275404_section18141628181814"><h4 class="sectiontitle"><strong id="dataartsstudio_01_0015__en-us_topic_0108275404_b129314716715">General Constraints on Database Migration</strong></h4><ol id="dataartsstudio_01_0015__en-us_topic_0108275404_ol12131857191813"><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li31317570181">CDM is mainly used for batch migration. It supports only limited incremental migration but does not support real-time incremental migration. </li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li192216218208">The entire DB migration of CDM supports only data table migration but not migration of database objects such as stored procedures, triggers, functions, and views. <p id="dataartsstudio_01_0015__en-us_topic_0108275404_p867452492013"><a name="dataartsstudio_01_0015__en-us_topic_0108275404_li192216218208"></a><a name="en-us_topic_0108275404_li192216218208"></a>CDM applies only to scenarios where databases are migrated to the cloud at a time, including homogeneous and heterogeneous database migrations. CDM is not applicable to data synchronization, for example, disaster recovery and real-time synchronization.</p>
|
||||
<div class="section" id="dataartsstudio_01_0015__en-us_topic_0108275404_section18141628181814"><h4 class="sectiontitle"><strong id="dataartsstudio_01_0015__en-us_topic_0108275404_b129314716715">General Notes and Constraints on Database Migration</strong></h4><ol id="dataartsstudio_01_0015__en-us_topic_0108275404_ol12131857191813"><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li31317570181">CDM is mainly used for batch migration. It supports only limited incremental migration but does not support real-time incremental migration. </li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li192216218208">The entire DB migration of CDM supports only data table migration but not migration of database objects such as stored procedures, triggers, functions, and views. <p id="dataartsstudio_01_0015__en-us_topic_0108275404_p867452492013"><a name="dataartsstudio_01_0015__en-us_topic_0108275404_li192216218208"></a><a name="en-us_topic_0108275404_li192216218208"></a>CDM applies only to scenarios where databases are migrated to the cloud at a time, including homogeneous and heterogeneous database migrations. CDM is not applicable to data synchronization, for example, disaster recovery and real-time synchronization.</p>
|
||||
</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li1431291119575">If CDM fails to migrate an entire database or table, the data that has been imported to the target table will not be rolled back automatically. If you want to perform migration in transaction mode, configure the <strong id="dataartsstudio_01_0015__en-us_topic_0108275404_b925962425814">Import to Staging Table</strong> parameter to enable a rollback upon a migration failure.<p id="dataartsstudio_01_0015__en-us_topic_0108275404_p187345158577">In extreme cases, the created stage table or temporary table cannot be automatically deleted. You need to manually clear the table (the name of the stage table ends with <span class="uicontrol" id="dataartsstudio_01_0015__en-us_topic_0108275404_uicontrol3737124419572"><b>_cdm_stage</b></span>), for example, <strong id="dataartsstudio_01_0015__en-us_topic_0108275404_b117791013173914">cdmtet_cdm_stage</strong>).</p>
|
||||
</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li6930205232919">If CDM needs to access data sources in the on-premises data center (for example, the on-premises MySQL database), the data sources must support Internet access and the CDM instances must be bound with elastic IP addresses. In this case, the security practice is to configure the firewall or security policies to allow only the EIPs of the CDM instances to access the local data sources.</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li15135574183">Only common data types are supported, including character strings, digits, and dates. Object types are limited. If objects are too large, migration cannot be performed.</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li18131570188">Only the GBK and UTF-8 character sets are supported.</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li1534605245718">A field name cannot contain & or %.</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li149911049104017">jdbc2hive and hive2jdbc entire DB migration is implemented by field name mapping, and is unavailable if the source and destination field names are inconsistent.</li></ol>
|
||||
</div>
|
||||
@ -14,7 +14,7 @@
|
||||
</div>
|
||||
<div class="section" id="dataartsstudio_01_0015__en-us_topic_0108275404_section1443661143314"><h4 class="sectiontitle"><strong id="dataartsstudio_01_0015__en-us_topic_0108275404_b88331756193">Constraints on FusionInsight HD and Apache Hadoop</strong></h4><p id="dataartsstudio_01_0015__en-us_topic_0108275404_p102661957113315">If the FusionInsight HD and Apache Hadoop data sources are deployed in the on-premises data center, CDM must access all nodes in the cluster for reading and writing the Hadoop files. Therefore, the network access must be enabled for each node.</p>
|
||||
</div>
|
||||
<div class="section" id="dataartsstudio_01_0015__en-us_topic_0108275404_section10533131115352"><h4 class="sectiontitle"><strong id="dataartsstudio_01_0015__en-us_topic_0108275404_b626717411286">Constraints on DWS and FusionInsight LibrA</strong></h4><ol id="dataartsstudio_01_0015__en-us_topic_0108275404_ol123656567357"><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li1036535610357"><span id="dataartsstudio_01_0015__en-us_topic_0108275404_ph896020457565">If the DWS primary key or table contains only one field, the field type must be a common character string, value, or date. When data is migrated from another database to DWS, if automatic table creation is selected, the primary key must be of the following types. If no primary key is set, at least one of the following fields must be set. Otherwise, the table cannot be created and the CDM job fails.</span><ul id="dataartsstudio_01_0015__en-us_topic_0108275404_ul19587148144414"><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li178061658447">INTEGER TYPES: TINYINT, SMALLINT, INT, BIGINT, NUMERIC/DECIMAL</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li13513127191516">CHARACTER TYPES: CHAR, BPCHAR, VARCHAR, VARCHAR2, NVARCHAR2, TEXT</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li17722183661616">DATA/TIME TYPES: DATE, TIME, TIMETZ, TIMESTAMP, TIMESTAMPTZ, INTERVAL, SMALLDATETIME</li></ul>
|
||||
<div class="section" id="dataartsstudio_01_0015__en-us_topic_0108275404_section10533131115352"><h4 class="sectiontitle"><strong id="dataartsstudio_01_0015__en-us_topic_0108275404_b18848182520554">Constraints on GaussDB(DWS)</strong></h4><ol id="dataartsstudio_01_0015__en-us_topic_0108275404_ol123656567357"><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li1036535610357"><span id="dataartsstudio_01_0015__en-us_topic_0108275404_ph896020457565">If the DWS primary key or table contains only one field, the field type must be a common character string, value, or date. When data is migrated from another database to DWS, if automatic table creation is selected, the primary key must be of the following types. If no primary key is set, at least one of the following fields must be set. Otherwise, the table cannot be created and the CDM job fails.</span><ul id="dataartsstudio_01_0015__en-us_topic_0108275404_ul19587148144414"><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li178061658447">INTEGER TYPES: TINYINT, SMALLINT, INT, BIGINT, NUMERIC/DECIMAL</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li13513127191516">CHARACTER TYPES: CHAR, BPCHAR, VARCHAR, VARCHAR2, NVARCHAR2, TEXT</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li17722183661616">DATA/TIME TYPES: DATE, TIME, TIMETZ, TIMESTAMP, TIMESTAMPTZ, INTERVAL, SMALLDATETIME</li></ul>
|
||||
<div class="note" id="dataartsstudio_01_0015__en-us_topic_0108275404_note14604639135010"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_0015__en-us_topic_0108275404_p160433915506">For clusters of version 2.9.1.200 or earlier, the NVARCHAR2 data type is not supported for DWS.</p>
|
||||
</div></div>
|
||||
</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li1987905110177">In DWS, the character string <strong id="dataartsstudio_01_0015__en-us_topic_0108275404_b4284165153511">''</strong> is null. A null character string cannot be inserted into a field with non-null constraints. This is inconsistent with the MySQL behavior. MySQL does not consider that <strong id="dataartsstudio_01_0015__en-us_topic_0108275404_b9349145113618">''</strong> is null. Migration from MySQL to DWS may fail due to the preceding reason.</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li1270919093612">When the Gauss Data Service (GDS) mode is used to quickly import data to DWS, you need to configure a security group or firewall policy to allow DataNodes of DWS or FusionInsight LibrA to access port 25000 of the CDM IP address.</li><li id="dataartsstudio_01_0015__en-us_topic_0108275404_li18441396234">When data is imported to DWS in GDS mode, CDM automatically creates a foreign table for data import. The table name ends with a universally unique identifier (UUID), for example, <strong id="dataartsstudio_01_0015__en-us_topic_0108275404_b1738395918464">cdmtest_aecf3f8n0z73dsl72d0d1dk4lcir8cd</strong>. If a job fails, it will be automatically deleted. In extreme cases, you may need to manually delete it.</li></ol>
|
||||
@ -70,7 +70,7 @@ STORED AS TEXTFILE;</pre>
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0012.html">DataArts Migration</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0012.html">DataArts Migration (CDM Jobs)</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
@ -1,27 +1,27 @@
|
||||
<a name="dataartsstudio_01_0017"></a><a name="dataartsstudio_01_0017"></a>
|
||||
|
||||
<h1 class="topictitle1">Managing Clusters</h1>
|
||||
<h1 class="topictitle1">Creating and Managing a CDM Cluster</h1>
|
||||
<div id="body8662426"></div>
|
||||
<div>
|
||||
<ul class="ullinks">
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0576.html">Creating a CDM Cluster</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0020_0.html">Binding or Unbinding an EIP</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0586.html">Binding or Unbinding an EIP</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0578.html">Restarting a Cluster</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0578.html">Restarting a CDM Cluster</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0579.html">Deleting a Cluster</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0579.html">Deleting a CDM Cluster</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0022.html">Downloading Cluster Logs</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0022.html">Downloading CDM Cluster Logs</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0021_0.html">Viewing Basic Cluster Information and Modifying Cluster Configurations</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0587.html">Viewing and Modifying CDM Cluster Configurations</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0121.html">Viewing Metrics</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0121.html">Managing and Viewing CDM Metrics</a></strong><br>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0012.html">DataArts Migration</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0012.html">DataArts Migration (CDM Jobs)</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
@ -15,7 +15,7 @@
|
||||
</ul>
|
||||
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0012.html">DataArts Migration</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0012.html">DataArts Migration (CDM Jobs)</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
@ -1,20 +0,0 @@
|
||||
<a name="dataartsstudio_01_0020_0"></a><a name="dataartsstudio_01_0020_0"></a>
|
||||
|
||||
<h1 class="topictitle1">Binding or Unbinding an EIP</h1>
|
||||
<div id="body8662426"><div class="section" id="dataartsstudio_01_0020_0__en-us_topic_0108275471_section6082040915224"><h4 class="sectiontitle">Scenario</h4><p id="dataartsstudio_01_0020_0__en-us_topic_0108275471_p742920411259">After creating a CDM cluster, you can bind an EIP to or unbind an EIP from the cluster. </p>
|
||||
<p id="dataartsstudio_01_0020_0__en-us_topic_0108275471_p12141738191218">If CDM needs to access a local or Internet data source, or a cloud service in another VPC, bind an EIP to the CDM cluster or use a NAT gateway to enable the CDM cluster to share the EIP with ECSs to access the Internet.</p>
|
||||
<div class="note" id="dataartsstudio_01_0020_0__en-us_topic_0108275471_note12930114533313"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_0020_0__en-us_topic_0108275471_p1693084519332">If SSL encryption is configured for the access channel of a local data source, CDM cannot connect to the data source using the EIP.</p>
|
||||
</div></div>
|
||||
</div>
|
||||
<div class="section" id="dataartsstudio_01_0020_0__en-us_topic_0108275471_section5781841515252"><h4 class="sectiontitle">Prerequisites</h4><ul id="dataartsstudio_01_0020_0__en-us_topic_0108275471_ul3448175015190"><li id="dataartsstudio_01_0020_0__en-us_topic_0108275471_li2479787415190">You have created a CDM cluster.</li><li id="dataartsstudio_01_0020_0__en-us_topic_0108275471_li3269623515192">Your EIP quota is sufficient.</li></ul>
|
||||
</div>
|
||||
<div class="section" id="dataartsstudio_01_0020_0__en-us_topic_0108275471_section1674818115330"><h4 class="sectiontitle">Procedure</h4><ol id="dataartsstudio_01_0020_0__en-us_topic_0108275471_ol4602013152046"><li id="dataartsstudio_01_0020_0__en-us_topic_0108275471_li418244016821"><span id="dataartsstudio_01_0020_0__en-us_topic_0108275471_en-us_topic_0000001397651462_p119011953152312">Log in to the <span id="dataartsstudio_01_0020_0__en-us_topic_0108275471_en-us_topic_0000001397651462_text5574448155719">DataArts Studio</span> console by following the instructions in <a href="dataartsstudio_01_0001.html">Accessing the DataArts Studio Instance Console</a>. On the <span id="dataartsstudio_01_0020_0__en-us_topic_0108275471_text185611381448">DataArts Studio</span> console, locate a workspace and click <strong id="dataartsstudio_01_0020_0__en-us_topic_0108275471_b16600914612">DataArts Migration</strong> to access the CDM console.</span></li><li id="dataartsstudio_01_0020_0__en-us_topic_0108275471_li0189204285419"><span>Bind an EIP to or unbind an EIP from a cluster.</span><p><ul id="dataartsstudio_01_0020_0__en-us_topic_0108275471_ul10803195193010"><li id="dataartsstudio_01_0020_0__en-us_topic_0108275471_li13803175193019">Binding an EIP: In the <strong id="dataartsstudio_01_0020_0__en-us_topic_0108275471_b2015317258420">Operation</strong> column, click <span class="uicontrol" id="dataartsstudio_01_0020_0__en-us_topic_0108275471_uicontrol1415352524218"><b>Bind EIP</b></span>. The <strong id="dataartsstudio_01_0020_0__en-us_topic_0108275471_b10153182516423">Bind EIP</strong> dialog box is displayed.</li><li id="dataartsstudio_01_0020_0__en-us_topic_0108275471_li241312592392">Unbinding an EIP: In the <strong id="dataartsstudio_01_0020_0__en-us_topic_0108275471_b298793728141839">Operation</strong> column, choose <span class="menucascade" id="dataartsstudio_01_0020_0__en-us_topic_0108275471_menucascade323504945141742"><b><span class="uicontrol" id="dataartsstudio_01_0020_0__en-us_topic_0108275471_uicontrol2130511470141742">More</span></b> > <b><span class="uicontrol" id="dataartsstudio_01_0020_0__en-us_topic_0108275471_uicontrol2037634173141742">Unbind EIP</span></b></span>.</li></ul>
|
||||
</p></li><li id="dataartsstudio_01_0020_0__en-us_topic_0108275471_li33887024151727"><span>Click <span class="uicontrol" id="dataartsstudio_01_0020_0__en-us_topic_0108275471_uicontrol917090064153628"><b>Yes</b></span>.</span></li></ol>
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0017.html">Managing Clusters</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@ -1,27 +0,0 @@
|
||||
<a name="dataartsstudio_01_0021_0"></a><a name="dataartsstudio_01_0021_0"></a>
|
||||
|
||||
<h1 class="topictitle1">Viewing Basic Cluster Information and Modifying Cluster Configurations</h1>
|
||||
<div id="body8662426"><div class="section" id="dataartsstudio_01_0021_0__en-us_topic_0108275372_section6082040915224"><h4 class="sectiontitle">Scenario</h4><p id="dataartsstudio_01_0021_0__en-us_topic_0108275372_p13184211124520">After creating a CDM cluster, you can view its basic information and modify its configurations.</p>
|
||||
<ul id="dataartsstudio_01_0021_0__en-us_topic_0108275372_ul19578141918455"><li id="dataartsstudio_01_0021_0__en-us_topic_0108275372_li1177617222457">You can view the following basic cluster information:<ul id="dataartsstudio_01_0021_0__en-us_topic_0108275372_ul34505418485"><li id="dataartsstudio_01_0021_0__en-us_topic_0108275372_li49551248144815">Cluster information: cluster version, creation time, project ID, instance ID, and cluster ID</li><li id="dataartsstudio_01_0021_0__en-us_topic_0108275372_li13938113919488">Instance configuration: cluster flavor, CPU, and memory</li><li id="dataartsstudio_01_0021_0__en-us_topic_0108275372_li1691813454916">Network configuration</li></ul>
|
||||
</li><li id="dataartsstudio_01_0021_0__en-us_topic_0108275372_li357841917454">You can modify the following cluster configurations:<ul id="dataartsstudio_01_0021_0__en-us_topic_0108275372_ul12517341719"><li id="dataartsstudio_01_0021_0__en-us_topic_0108275372_li65133411112">Notification: If a CDM migration job (only table/file migration) fails or the EIP is abnormal, CDM sends an SMS or email notification to the user. </li><li id="dataartsstudio_01_0021_0__en-us_topic_0108275372_li1451163419115">User isolation: determines whether other users can operate the migration jobs or links in the cluster.<ul id="dataartsstudio_01_0021_0__en-us_topic_0108275372_ul185117349120"><li id="dataartsstudio_01_0021_0__en-us_topic_0108275372_li1751534315">If this function is enabled, migration jobs and links in the cluster are isolated. Other IAM users of the <span id="dataartsstudio_01_0021_0__en-us_topic_0108275372_text145133410115">a cloud platform account</span> cannot operate the jobs and links.</li><li id="dataartsstudio_01_0021_0__en-us_topic_0108275372_li55114348115">If this function is disabled, migration jobs and links in the cluster can be shared by users. All IAM users with the required permission in the <span id="dataartsstudio_01_0021_0__en-us_topic_0108275372_text1943484611425">a cloud platform account</span> can view and perform operations on the jobs and links in the cluster.<p id="dataartsstudio_01_0021_0__en-us_topic_0108275372_p1651153414114">After disabling <strong id="dataartsstudio_01_0021_0__en-us_topic_0108275372_b996710202447">User Isolation</strong>, restart the cluster VM for the settings to take effect.</p>
|
||||
</li></ul>
|
||||
</li></ul>
|
||||
</li></ul>
|
||||
</div>
|
||||
<div class="section" id="dataartsstudio_01_0021_0__en-us_topic_0108275372_section5781841515252"><h4 class="sectiontitle">Prerequisites</h4><p id="dataartsstudio_01_0021_0__en-us_topic_0108275372_p33513853111130">You have created a CDM cluster.</p>
|
||||
</div>
|
||||
<div class="section" id="dataartsstudio_01_0021_0__en-us_topic_0108275372_section1674818115330"><h4 class="sectiontitle">Viewing Basic Cluster Information</h4><ol id="dataartsstudio_01_0021_0__en-us_topic_0108275372_ol4602013152046"><li id="dataartsstudio_01_0021_0__en-us_topic_0108275372_li35890017143134"><span id="dataartsstudio_01_0021_0__en-us_topic_0108275372_en-us_topic_0108275471_en-us_topic_0000001397651462_p119011953152312">Log in to the <span id="dataartsstudio_01_0021_0__en-us_topic_0108275372_en-us_topic_0108275471_en-us_topic_0000001397651462_text5574448155719">DataArts Studio</span> console by following the instructions in <a href="dataartsstudio_01_0001.html">Accessing the DataArts Studio Instance Console</a>. On the <span id="dataartsstudio_01_0021_0__en-us_topic_0108275372_en-us_topic_0108275471_text185611381448">DataArts Studio</span> console, locate a workspace and click <strong id="dataartsstudio_01_0021_0__en-us_topic_0108275372_en-us_topic_0108275471_b16600914612">DataArts Migration</strong> to access the CDM console.</span></li><li id="dataartsstudio_01_0021_0__en-us_topic_0108275372_li16142765310"><span>Click the cluster name to view its basic information.</span><p><p id="dataartsstudio_01_0021_0__en-us_topic_0108275372_p1988954512340"></p>
|
||||
</p></li></ol>
|
||||
</div>
|
||||
<div class="section" id="dataartsstudio_01_0021_0__en-us_topic_0108275372_section8304145665316"><h4 class="sectiontitle">Modifying Cluster Configurations</h4><ol id="dataartsstudio_01_0021_0__en-us_topic_0108275372_ol1999165225516"><li id="dataartsstudio_01_0021_0__en-us_topic_0108275372_li16425756115512"><span id="dataartsstudio_01_0021_0__en-us_topic_0108275372_en-us_topic_0108275471_en-us_topic_0000001397651462_p119011953152312_1">Log in to the <span id="dataartsstudio_01_0021_0__en-us_topic_0108275372_en-us_topic_0108275471_en-us_topic_0000001397651462_text5574448155719_1">DataArts Studio</span> console by following the instructions in <a href="dataartsstudio_01_0001.html">Accessing the DataArts Studio Instance Console</a>. On the <span id="dataartsstudio_01_0021_0__en-us_topic_0108275372_en-us_topic_0108275471_text185611381448_1">DataArts Studio</span> console, locate a workspace and click <strong id="dataartsstudio_01_0021_0__en-us_topic_0108275372_en-us_topic_0108275471_b16600914612_1">DataArts Migration</strong> to access the CDM console.</span></li></ol><ol start="2" id="dataartsstudio_01_0021_0__en-us_topic_0108275372_ol893465814539"><li id="dataartsstudio_01_0021_0__en-us_topic_0108275372_li2934258175315"><span>Click the name of a cluster and click the <span class="uicontrol" id="dataartsstudio_01_0021_0__en-us_topic_0108275372_uicontrol175133228101"><b>Cluster Configuration</b></span> tab to modify <strong id="dataartsstudio_01_0021_0__en-us_topic_0108275372_b1651322291018">Notification</strong> and <strong id="dataartsstudio_01_0021_0__en-us_topic_0108275372_b25131022191018">User Isolation</strong> configuration.</span><p><p id="dataartsstudio_01_0021_0__en-us_topic_0108275372_p164740175220"></p>
|
||||
</p></li><li id="dataartsstudio_01_0021_0__en-us_topic_0108275372_li16934258135318"><span>Click <span class="uicontrol" id="dataartsstudio_01_0021_0__en-us_topic_0108275372_uicontrol191388974921518"><b>Save</b></span>. The <strong id="dataartsstudio_01_0021_0__en-us_topic_0108275372_b148848311621518">Cluster Management</strong> page is displayed.</span></li><li id="dataartsstudio_01_0021_0__en-us_topic_0108275372_li12934195818537"><span>If <strong id="dataartsstudio_01_0021_0__en-us_topic_0108275372_b119183067121518">User Isolation</strong> is disabled, choose <span class="menucascade" id="dataartsstudio_01_0021_0__en-us_topic_0108275372_menucascade54419951821518"><b><span class="uicontrol" id="dataartsstudio_01_0021_0__en-us_topic_0108275372_uicontrol82627482221518">More</span></b> > <b><span class="uicontrol" id="dataartsstudio_01_0021_0__en-us_topic_0108275372_uicontrol11172047021518">Restart</span></b></span> in the <strong id="dataartsstudio_01_0021_0__en-us_topic_0108275372_b23573550421518">Operation</strong> column to restart the cluster VM for the settings to take effect.</span><p><div class="fignone" id="dataartsstudio_01_0021_0__en-us_topic_0108275372_fig159341758205317"><span class="figcap"><b>Figure 1 </b>Restarting a cluster</span><br><span><img id="dataartsstudio_01_0021_0__en-us_topic_0108275372_image139341458125315" src="en-us_image_0000002305441289.png"></span></div>
|
||||
<ul id="dataartsstudio_01_0021_0__en-us_topic_0108275372_ul293411589531"><li id="dataartsstudio_01_0021_0__en-us_topic_0108275372_li199341458205313"><strong id="dataartsstudio_01_0021_0__en-us_topic_0108275372_b664218019278">Restart CDM service process</strong>: Only the CDM service process is restarted. The cluster VM will not be restarted.</li><li id="dataartsstudio_01_0021_0__en-us_topic_0108275372_li49341958125318"><strong id="dataartsstudio_01_0021_0__en-us_topic_0108275372_b84235270615490">VM restart</strong>: The service process will be interrupted and VMs in the cluster will be restarted.</li></ul>
|
||||
</p></li><li id="dataartsstudio_01_0021_0__en-us_topic_0108275372_li1093410587539"><span>Select <span class="uicontrol" id="dataartsstudio_01_0021_0__en-us_topic_0108275372_uicontrol1876751918166"><b>VM restart</b></span> and click <span class="uicontrol" id="dataartsstudio_01_0021_0__en-us_topic_0108275372_uicontrol1010382481610"><b>Yes</b></span>.</span></li></ol>
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0017.html">Managing Clusters</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@ -1,17 +1,24 @@
|
||||
<a name="dataartsstudio_01_0022"></a><a name="dataartsstudio_01_0022"></a>
|
||||
|
||||
<h1 class="topictitle1">Downloading Cluster Logs</h1>
|
||||
<h1 class="topictitle1">Downloading CDM Cluster Logs</h1>
|
||||
<div id="body8662426"><div class="section" id="dataartsstudio_01_0022__en-us_topic_0108275415_section6082040915224"><h4 class="sectiontitle">Scenario</h4><p id="dataartsstudio_01_0022__en-us_topic_0108275415_p533529515256">This section describes how to obtain cluster logs to view the job running history and locate job failure causes.</p>
|
||||
</div>
|
||||
<div class="section" id="dataartsstudio_01_0022__en-us_topic_0108275415_section5781841515252"><h4 class="sectiontitle">Prerequisites</h4><p id="dataartsstudio_01_0022__en-us_topic_0108275415_p33513853111130">You have created a CDM cluster.</p>
|
||||
</div>
|
||||
<div class="section" id="dataartsstudio_01_0022__en-us_topic_0108275415_section1674818115330"><h4 class="sectiontitle">Procedure</h4><ol id="dataartsstudio_01_0022__en-us_topic_0108275415_ol6681758181820"><li id="dataartsstudio_01_0022__en-us_topic_0108275415_li35890017143134"><span id="dataartsstudio_01_0022__en-us_topic_0108275415_en-us_topic_0108275471_en-us_topic_0000001397651462_p119011953152312">Log in to the <span id="dataartsstudio_01_0022__en-us_topic_0108275415_en-us_topic_0108275471_en-us_topic_0000001397651462_text5574448155719">DataArts Studio</span> console by following the instructions in <a href="dataartsstudio_01_0001.html">Accessing the DataArts Studio Instance Console</a>. On the <span id="dataartsstudio_01_0022__en-us_topic_0108275415_en-us_topic_0108275471_text185611381448">DataArts Studio</span> console, locate a workspace and click <strong id="dataartsstudio_01_0022__en-us_topic_0108275415_en-us_topic_0108275471_b16600914612">DataArts Migration</strong> to access the CDM console.</span></li></ol><ol start="2" id="dataartsstudio_01_0022__en-us_topic_0108275415_ol4602013152046"><li id="dataartsstudio_01_0022__en-us_topic_0108275415_li4368748394623"><span>Locate the row that contains a cluster, click <strong id="dataartsstudio_01_0022__en-us_topic_0108275415_b1674594319426">More</strong> in the <strong id="dataartsstudio_01_0022__en-us_topic_0108275415_b715115534425">Operation</strong> column, and select <strong id="dataartsstudio_01_0022__en-us_topic_0108275415_b1425998431">Download Log</strong> from the drop-down list.</span><p><div class="fignone" id="dataartsstudio_01_0022__en-us_topic_0108275415_fig159341758205317"><span class="figcap"><b>Figure 1 </b>Download Log</span><br><span><img id="dataartsstudio_01_0022__en-us_topic_0108275415_image1359985320200" src="en-us_image_0000002270790548.png"></span></div>
|
||||
<div class="section" id="dataartsstudio_01_0022__en-us_topic_0108275415_section1674818115330"><h4 class="sectiontitle">Procedure</h4><ol id="dataartsstudio_01_0022__en-us_topic_0108275415_ol6681758181820"><li id="dataartsstudio_01_0022__en-us_topic_0108275415_li35890017143134"><span id="dataartsstudio_01_0022__en-us_topic_0108275415_en-us_topic_0108275471_en-us_topic_0000001397651462_p119011953152312">Log in to the <span id="dataartsstudio_01_0022__en-us_topic_0108275415_en-us_topic_0108275471_text1118012943810">DataArts Studio</span> console by following the instructions in <a href="dataartsstudio_01_0001.html">Accessing the DataArts Studio Instance Console</a>. On the <span id="dataartsstudio_01_0022__en-us_topic_0108275415_en-us_topic_0108275471_text185611381448">DataArts Studio</span> console, locate a workspace and click <strong id="dataartsstudio_01_0022__en-us_topic_0108275415_en-us_topic_0108275471_b16600914612">DataArts Migration</strong> to access the CDM console.</span></li></ol><ol start="2" id="dataartsstudio_01_0022__en-us_topic_0108275415_ol4602013152046"><li id="dataartsstudio_01_0022__en-us_topic_0108275415_li4368748394623"><span>Locate the row that contains a cluster, click <strong id="dataartsstudio_01_0022__en-us_topic_0108275415_b1674594319426">More</strong> in the <strong id="dataartsstudio_01_0022__en-us_topic_0108275415_b715115534425">Operation</strong> column, and select <strong id="dataartsstudio_01_0022__en-us_topic_0108275415_b1425998431">Download Log</strong> from the drop-down list.</span><p><div class="fignone" id="dataartsstudio_01_0022__en-us_topic_0108275415_fig159341758205317"><span class="figcap"><b>Figure 1 </b>Download Log</span><br><span><img id="dataartsstudio_01_0022__en-us_topic_0108275415_image1495344714" src="en-us_image_0000002234236680.png" title="Click to enlarge" class="imgResize"></span></div>
|
||||
</p></li><li id="dataartsstudio_01_0022__en-us_topic_0108275415_li209131722132014"><span>In the displayed dialog box, click <strong id="dataartsstudio_01_0022__en-us_topic_0108275415_b01985196448">OK</strong> to download logs to a local PC.</span></li></ol>
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0017.html">Managing Clusters</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0017.html">Creating and Managing a CDM Cluster</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
<script language="JavaScript">
|
||||
<!--
|
||||
initImageViewer('.imgResize');
|
||||
var msg_imageMax = "view original image";
|
||||
var msg_imageClose = "close";
|
||||
//--></script>
|
||||
@ -1,73 +1,21 @@
|
||||
<a name="dataartsstudio_01_0023"></a><a name="dataartsstudio_01_0023"></a>
|
||||
|
||||
<h1 class="topictitle1">Managing Links</h1>
|
||||
<h1 class="topictitle1">Creating a Link in a CDM Cluster</h1>
|
||||
<div id="body8662426"></div>
|
||||
<div>
|
||||
<ul class="ullinks">
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0024.html">Creating Links</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0024.html">Creating a Link Between CDM and a Data Source</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0132.html">Managing Drivers</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0157.html">Configuring Link Parameters</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0128.html">Managing Agents</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0132.html">Uploading a CDM Link Driver</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1096.html">Managing Cluster Configurations</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0045.html">Link to OBS</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0044.html">Link to PostgreSQL/SQLServer</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0226.html">Link to DWS</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1211.html">Link to an RDS for MySQL/MySQL Database</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1212.html">Link to an Oracle Database</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0036.html">Link to DLI</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0026.html">Link to Hive</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0039.html">Link to HBase</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0040.html">Link to HDFS</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0028.html">Link to an FTP or SFTP Server</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0032.html">Link to Redis</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0031.html">Link to DDS</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0027.html">Link to CloudTable</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0030.html">Link to MongoDB</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_004501.html">Link to Cassandra</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0033.html">Link to Kafka</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0038.html">Link to DMS Kafka</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0035.html">Link to CSS</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1380.html">Link to Elasticsearch</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0227.html">Link to SAP HANA</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1214.html">Link to a Database Shard</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0184.html">Link to MRS Hudi</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0285.html">Link to MRS ClickHouse</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0290.html">Link to a ShenTong Database</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0037.html">Link to CloudTable OpenTSDB</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1215.html">Link to Doris</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1096.html">Creating a Hadoop Cluster Configuration</a></strong><br>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0012.html">DataArts Migration</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0012.html">DataArts Migration (CDM Jobs)</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@ -1,8 +1,8 @@
|
||||
<a name="dataartsstudio_01_0025"></a><a name="dataartsstudio_01_0025"></a>
|
||||
|
||||
<h1 class="topictitle1">How Migration Jobs Work</h1>
|
||||
<div id="en-us_topic_0000001340006573_body0000001340006573"><div class="section" id="dataartsstudio_01_0025__en-us_topic_0000001340006573_section14018854920"><h4 class="sectiontitle">Data Migration Model</h4><p id="dataartsstudio_01_0025__en-us_topic_0000001340006573_p111321034164314"><a href="#dataartsstudio_01_0025__en-us_topic_0000001340006573_fig165851910185417">Figure 1</a> shows the simplified migration model used by CDM.</p>
|
||||
<div class="fignone" id="dataartsstudio_01_0025__en-us_topic_0000001340006573_fig165851910185417"><a name="dataartsstudio_01_0025__en-us_topic_0000001340006573_fig165851910185417"></a><a name="en-us_topic_0000001340006573_fig165851910185417"></a><span class="figcap"><b>Figure 1 </b>Migration model used by CDM</span><br><span><img class="eddx" id="dataartsstudio_01_0025__en-us_topic_0000001340006573_image8585010145411" src="en-us_image_0000002270847386.png"></span></div>
|
||||
<div id="body8662426"><div class="section" id="dataartsstudio_01_0025__en-us_topic_0000001340006573_section14018854920"><h4 class="sectiontitle">Data Migration Model</h4><p id="dataartsstudio_01_0025__en-us_topic_0000001340006573_p111321034164314"><a href="#dataartsstudio_01_0025__en-us_topic_0000001340006573_fig165851910185417">Figure 1</a> shows the simplified migration model used by CDM.</p>
|
||||
<div class="fignone" id="dataartsstudio_01_0025__en-us_topic_0000001340006573_fig165851910185417"><a name="dataartsstudio_01_0025__en-us_topic_0000001340006573_fig165851910185417"></a><a name="en-us_topic_0000001340006573_fig165851910185417"></a><span class="figcap"><b>Figure 1 </b>Migration model used by CDM</span><br><span><img class="imgResize" id="dataartsstudio_01_0025__en-us_topic_0000001340006573_image8585010145411" src="en-us_image_0000002269117481.png" title="Click to enlarge"></span></div>
|
||||
<div class="p" id="dataartsstudio_01_0025__en-us_topic_0000001340006573_p731010268319">CDM migrates data through data migration jobs. It works in the following way:<ol id="dataartsstudio_01_0025__en-us_topic_0000001340006573_ol1095184992318"><li id="dataartsstudio_01_0025__en-us_topic_0000001340006573_li79541847152411">When data migration jobs are submitted, CDM splits each job into multiple tasks based on the <strong id="dataartsstudio_01_0025__en-us_topic_0000001340006573_b49451957175511">Concurrent Extractors</strong> parameter in the job configuration.<div class="note" id="dataartsstudio_01_0025__en-us_topic_0000001340006573_note636713355310"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_0025__en-us_topic_0000001340006573_p1036843195312">Jobs for different data sources may be split based on different dimensions. Some jobs may not be split based on the <strong id="dataartsstudio_01_0025__en-us_topic_0000001340006573_b258662735711">Concurrent Extractors</strong> parameter.</p>
|
||||
</div></div>
|
||||
</li><li id="dataartsstudio_01_0025__en-us_topic_0000001340006573_li587861564119">CDM submits the tasks to the running pool in sequence. Tasks (defined by <strong id="dataartsstudio_01_0025__en-us_topic_0000001340006573_b169912243">Maximum Concurrent Extractors</strong>) run concurrently. Excess tasks are queued.</li></ol>
|
||||
@ -74,3 +74,10 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
<script language="JavaScript">
|
||||
<!--
|
||||
initImageViewer('.imgResize');
|
||||
var msg_imageMax = "view original image";
|
||||
var msg_imageClose = "close";
|
||||
//--></script>
|
||||
@ -1,13 +1,14 @@
|
||||
<a name="dataartsstudio_01_0026"></a><a name="dataartsstudio_01_0026"></a>
|
||||
|
||||
<h1 class="topictitle1">Link to Hive</h1>
|
||||
<h1 class="topictitle1">Hive Link Parameters</h1>
|
||||
<div id="body8662426"><div class="p" id="dataartsstudio_01_0026__en-us_topic_0108618545_p98197496459">CDM supports the following Hive data sources:<ul id="dataartsstudio_01_0026__en-us_topic_0108618545_ul1998193319335"><li id="dataartsstudio_01_0026__en-us_topic_0108618545_li189813313319"><a href="#dataartsstudio_01_0026__en-us_topic_0108618545_section6335145774617">MRS Hive</a></li><li id="dataartsstudio_01_0026__en-us_topic_0108618545_li46813410343"><a href="#dataartsstudio_01_0026__en-us_topic_0108618545_section999517914488">FusionInsight Hive</a></li><li id="dataartsstudio_01_0026__en-us_topic_0108618545_li88038432344"><a href="#dataartsstudio_01_0026__en-us_topic_0108618545_section9585729145218">Apache Hive</a><div class="note" id="dataartsstudio_01_0026__en-us_topic_0108618545_note8867140110"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0108275292_p167832113185">Do not change the password or user when the job is running. If you do so, the password will not take effect immediately and the job will fail.</p>
|
||||
</div></div>
|
||||
</li></ul>
|
||||
</div>
|
||||
<div class="section" id="dataartsstudio_01_0026__en-us_topic_0108618545_section6335145774617"><a name="dataartsstudio_01_0026__en-us_topic_0108618545_section6335145774617"></a><a name="en-us_topic_0108618545_section6335145774617"></a><h4 class="sectiontitle">MRS Hive</h4><p id="dataartsstudio_01_0026__en-us_topic_0108618545_p18541618205320">You can view a table during field mapping only when you have the permission to access the table connected to MRS Hive.</p>
|
||||
<p id="dataartsstudio_01_0026__en-us_topic_0108618545_p396911259314">MRS Hive links apply to the MapReduce Service (MRS) on <span id="dataartsstudio_01_0026__en-us_topic_0108618545_text97221338133">cloud</span>. <a href="#dataartsstudio_01_0026__en-us_topic_0108618545_table111803819501">Table 1</a> describes related parameters.</p>
|
||||
<div class="note" id="dataartsstudio_01_0026__en-us_topic_0108618545_note0738159182614"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dataartsstudio_01_0026__en-us_topic_0108618545_ul268415325415"><li id="dataartsstudio_01_0026__en-us_topic_0108618545_li116841636546">Before creating an MRS Hive link, you need to add an authenticated Kerberos user on MRS and log in to the MRS management page to change the initial password. Then use the new user to create an MRS link.</li><li id="dataartsstudio_01_0026__en-us_topic_0108618545_li36846319542">To connect to an MRS 2.<em id="dataartsstudio_01_0026__en-us_topic_0108618545_i20573572105247">x</em> cluster, create a CDM cluster of version 2.<em id="dataartsstudio_01_0026__en-us_topic_0108618545_i364849425105247">x</em> first. CDM 1.8.<em id="dataartsstudio_01_0026__en-us_topic_0108618545_i208804587212">x</em> clusters cannot connect to MRS 2.<em id="dataartsstudio_01_0026__en-us_topic_0108618545_i16492183142213">x</em> clusters.</li><li id="dataartsstudio_01_0026__en-us_topic_0108618545_li3684133125414">Currently, the Hive link obtains the <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_b19726957135315">core-site.xml</strong> configuration information from MRS HDFS. Therefore, if MRS Hive uses OBS as the underlying storage system, configure the AK/SK of OBS on MRS HDFS before creating the Hive link.</li><li id="dataartsstudio_01_0026__en-us_topic_0108618545_li6383241205811">Ensure that the MRS cluster and the <span id="dataartsstudio_01_0026__en-us_topic_0108618545_text1932919212411">DataArts Studio</span> instance can communicate with each other. The following requirements must be met for network interconnection:<ul id="dataartsstudio_01_0026__en-us_topic_0108618545_ul1715617232281"><li id="dataartsstudio_01_0026__en-us_topic_0108618545_li17205293217">If the CDM cluster in the <span id="dataartsstudio_01_0026__en-us_topic_0108618545_text6263192520410">DataArts Studio</span> instance and the MRS cluster are in different regions, a public network or a dedicated connection is required. If the Internet is used for communication, ensure that an EIP has been bound to the CDM cluster, and the MRS cluster can access the Internet and the port has been enabled in the firewall rule.</li><li id="dataartsstudio_01_0026__en-us_topic_0108618545_li13205893215">If the CDM cluster in the <span id="dataartsstudio_01_0026__en-us_topic_0108618545_text13585787189">DataArts Studio</span> instance and the cloud service are in the same region, VPC, subnet, and security group, they can communicate with each other by default. If they are in the same VPC but in different subnets or security groups, you must configure routing rules and security group rules. For details about how to configure routing rules, see "Adding a Custom Route" in <em id="dataartsstudio_01_0026__en-us_topic_0108618545_i145879819184">Virtual Private Cloud (VPC) Usage Guide</em>. For details about how to configure security group rules, see <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_b18590686186">Security Group</strong> > <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_b259016851813">Adding a Security Group Rule</strong> in <em id="dataartsstudio_01_0026__en-us_topic_0108618545_i14591198181815">Virtual Private Cloud (VPC) Usage Guide</em>.</li><li id="dataartsstudio_01_0026__en-us_topic_0108618545_li20156162311283">The MRS cluster and the <span id="dataartsstudio_01_0026__en-us_topic_0108618545_text13313028144319">DataArts Studio</span> workspace belong to the same enterprise project. If they do not, you can modify the enterprise project of the workspace.</li></ul>
|
||||
<div class="note" id="dataartsstudio_01_0026__en-us_topic_0108618545_note0738159182614"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dataartsstudio_01_0026__en-us_topic_0108618545_ul268415325415"><li id="dataartsstudio_01_0026__en-us_topic_0108618545_li3509922112118"><p id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001624214396_p1463312815216"><a name="dataartsstudio_01_0026__en-us_topic_0108618545_li3509922112118"></a><a name="en-us_topic_0108618545_li3509922112118"></a>MRS clusters whose Kerberos encryption type is <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001624214396_b742132751912">aes256-sha2,aes128-sha2</strong> are not supported, and only MRS clusters whose Kerberos encryption type is <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001624214396_b1977394971913">aes256-sha1,aes128-sha1</strong> are supported.</p>
|
||||
</li><li id="dataartsstudio_01_0026__en-us_topic_0108618545_li116841636546">Before creating an MRS Hive link, you need to add an authenticated Kerberos user on MRS and log in to the MRS management page to change the initial password. Then use the new user to create an MRS link.</li><li id="dataartsstudio_01_0026__en-us_topic_0108618545_li36846319542">To connect to an MRS 2.<em id="dataartsstudio_01_0026__en-us_topic_0108618545_i20573572105247">x</em> cluster, create a CDM cluster of version 2.<em id="dataartsstudio_01_0026__en-us_topic_0108618545_i364849425105247">x</em> first. CDM 1.8.<em id="dataartsstudio_01_0026__en-us_topic_0108618545_i208804587212">x</em> clusters cannot connect to MRS 2.<em id="dataartsstudio_01_0026__en-us_topic_0108618545_i16492183142213">x</em> clusters.</li><li id="dataartsstudio_01_0026__en-us_topic_0108618545_li3684133125414">Currently, the Hive link obtains the <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_b19726957135315">core-site.xml</strong> configuration information from MRS HDFS. Therefore, if MRS Hive uses OBS as the underlying storage system, configure the AK/SK of OBS on MRS HDFS before creating the Hive link.</li><li id="dataartsstudio_01_0026__en-us_topic_0108618545_li6383241205811">Ensure that the MRS cluster and the <span id="dataartsstudio_01_0026__en-us_topic_0108618545_text1932919212411">DataArts Studio</span> instance can communicate with each other. The following requirements must be met for network interconnection:<ul id="dataartsstudio_01_0026__en-us_topic_0108618545_ul1715617232281"><li id="dataartsstudio_01_0026__en-us_topic_0108618545_li17205293217">If the CDM cluster in the <span id="dataartsstudio_01_0026__en-us_topic_0108618545_text6263192520410">DataArts Studio</span> instance and the MRS cluster are in different regions, a public network or a dedicated connection is required. If the Internet is used for communication, ensure that an EIP has been bound to the CDM cluster, and the MRS cluster can access the Internet and the port has been enabled in the firewall rule.</li><li id="dataartsstudio_01_0026__en-us_topic_0108618545_li13205893215">If the CDM cluster in the <span id="dataartsstudio_01_0026__en-us_topic_0108618545_text7368163316234">DataArts Studio</span> instance and the MRS cluster are in the same region, VPC, subnet, and security group, they can communicate with each other by default. If they are in the same VPC but in different subnets or security groups, you must configure routing rules and security group rules. For details about how to configure routing rules, see "Adding Routes to a Route Table" in <em id="dataartsstudio_01_0026__en-us_topic_0108618545_i143699331238">Virtual Private Cloud (VPC) x.x.x User Guide</em> in <em id="dataartsstudio_01_0026__en-us_topic_0108618545_i1936923332316">Virtual Private Cloud (VPC) x.x.x Usage Guide</em>. For details about how to configure security group rules, see "Security Group" > "Adding a Security Group Rule" in <em id="dataartsstudio_01_0026__en-us_topic_0108618545_i1137003313234">Virtual Private Cloud (VPC) x.x.x User Guide</em> in <em id="dataartsstudio_01_0026__en-us_topic_0108618545_i1537014331235">Virtual Private Cloud (VPC) x.x.x Usage Guide</em>.</li><li id="dataartsstudio_01_0026__en-us_topic_0108618545_li20156162311283">The MRS cluster and the <span id="dataartsstudio_01_0026__en-us_topic_0108618545_text13313028144319">DataArts Studio</span> workspace belong to the same enterprise project. If they do not, you can modify the enterprise project of the workspace.</li></ul>
|
||||
</li></ul>
|
||||
</div></div>
|
||||
|
||||
@ -28,7 +29,7 @@
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0026__en-us_topic_0108618545_row16181208165013"><td class="cellrowborder" valign="top" width="16.72%" headers="mcps1.3.2.5.2.4.1.1 "><p id="dataartsstudio_01_0026__en-us_topic_0108618545_p21818818506">Manager IP</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="62.1%" headers="mcps1.3.2.5.2.4.1.2 "><div class="p" id="dataartsstudio_01_0026__en-us_topic_0108618545_p218148165010">Floating IP address of MRS Manager. Click <span class="uicontrol" id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0108275286_uicontrol578864992012"><b>Select</b></span> next to the <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0108275286_b2078834920203">Manager IP</strong> text box to select an MRS cluster. CDM automatically fills in the authentication information.<div class="note" id="dataartsstudio_01_0026__note1340014473413"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0026__en-us_topic_0182566327_p116249795810"><span id="dataartsstudio_01_0026__en-us_topic_0182566327_text7355181825814">DataArts Studio</span> does not support MRS clusters whose Kerberos encryption type is <strong id="dataartsstudio_01_0026__en-us_topic_0182566327_b143479402717">aes256-sha2,aes128-sha2</strong>, and only supports MRS clusters whose Kerberos encryption type is <strong id="dataartsstudio_01_0026__en-us_topic_0182566327_b19532103219813">aes256-sha1,aes128-sha1</strong>.</p>
|
||||
<td class="cellrowborder" valign="top" width="62.1%" headers="mcps1.3.2.5.2.4.1.2 "><div class="p" id="dataartsstudio_01_0026__en-us_topic_0108618545_p218148165010">Floating IP address of MRS Manager. Click <span class="uicontrol" id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0108275286_uicontrol578864992012"><b>Select</b></span> next to the <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0108275286_b2078834920203">Manager IP</strong> text box to select an MRS cluster. CDM automatically fills in the authentication information.<div class="note" id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0108275286_note1340014473413"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0108275286_en-us_topic_0182566327_p116249795810"><span id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0108275286_en-us_topic_0182566327_text7355181825814">DataArts Studio</span> does not support MRS clusters whose Kerberos encryption type is <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0108275286_en-us_topic_0182566327_b143479402717">aes256-sha2,aes128-sha2</strong>, and only supports MRS clusters whose Kerberos encryption type is <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0108275286_en-us_topic_0182566327_b19532103219813">aes256-sha1,aes128-sha1</strong>.</p>
|
||||
</div></div>
|
||||
</div>
|
||||
</td>
|
||||
@ -77,7 +78,7 @@
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0026__en-us_topic_0108618545_row167980166457"><td class="cellrowborder" valign="top" width="16.72%" headers="mcps1.3.2.5.2.4.1.1 "><p id="dataartsstudio_01_0026__en-us_topic_0108618545_p2798416154516">ldapUsername</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="62.1%" headers="mcps1.3.2.5.2.4.1.2 "><p id="dataartsstudio_01_0026__en-us_topic_0108618545_p228323461719">This parameter is mandatory when <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_b194501221306">Enable ldap</strong> is enabled.</p>
|
||||
<td class="cellrowborder" valign="top" width="62.1%" headers="mcps1.3.2.5.2.4.1.2 "><p id="dataartsstudio_01_0026__en-us_topic_0108618545_p228323461719">This parameter is mandatory when <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_b2055819378149">Enable ldap</strong> is enabled.</p>
|
||||
<p id="dataartsstudio_01_0026__en-us_topic_0108618545_p720114494318">Enter the username configured when LDAP authentication was enabled for MRS Hive.</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="21.18%" headers="mcps1.3.2.5.2.4.1.3 "><p id="dataartsstudio_01_0026__en-us_topic_0108618545_p879841613453">-</p>
|
||||
@ -85,7 +86,7 @@
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0026__en-us_topic_0108618545_row203131218453"><td class="cellrowborder" valign="top" width="16.72%" headers="mcps1.3.2.5.2.4.1.1 "><p id="dataartsstudio_01_0026__en-us_topic_0108618545_p173191212454">ldapPassword</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="62.1%" headers="mcps1.3.2.5.2.4.1.2 "><p id="dataartsstudio_01_0026__en-us_topic_0108618545_p737215376176">This parameter is mandatory when <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_b16816610157">Enable ldap</strong> is enabled.</p>
|
||||
<td class="cellrowborder" valign="top" width="62.1%" headers="mcps1.3.2.5.2.4.1.2 "><p id="dataartsstudio_01_0026__en-us_topic_0108618545_p737215376176">This parameter is mandatory when <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_b7346406146">Enable ldap</strong> is enabled.</p>
|
||||
<p id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0108618545_p1431412184516">Enter the password configured when LDAP authentication was enabled for MRS Hive.</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="21.18%" headers="mcps1.3.2.5.2.4.1.3 "><p id="dataartsstudio_01_0026__en-us_topic_0108618545_p1531412164511">-</p>
|
||||
@ -102,7 +103,7 @@
|
||||
</td>
|
||||
<td class="cellrowborder" rowspan="2" valign="top" width="62.1%" headers="mcps1.3.2.5.2.4.1.2 "><p id="dataartsstudio_01_0026__en-us_topic_0108618545_p595472512338">This parameter is mandatory when <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_b193391985111223">OBS storage support</strong> is enabled. The account corresponding to the AK/SK pair must have the OBS Buckets Viewer permission. Otherwise, OBS cannot be accessed and the "403 AccessDenied" error is reported.</p>
|
||||
<p id="dataartsstudio_01_0026__en-us_topic_0108618545_p14128105533915">You need to create an access key for the current account and obtain an AK/SK pair.</p>
|
||||
<ol id="dataartsstudio_01_0026__en-us_topic_0108618545_ol1361418377715"><li id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_li1535103025819">Log in to the management console, move the cursor to the username in the upper right corner, and select <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b1586023610364">My Credentials</strong> from the drop-down list.</li><li id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_li173533018584">On the <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b149721128172418">My Credentials</strong> page, choose <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b15803656132414">Access Keys</strong>, and click <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b2452152020258">Create Access Key</strong>. See <a href="#dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615">Figure 1</a>.<div class="fignone" id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615"><a name="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615"></a><a name="en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615"></a><span class="figcap"><b>Figure 1 </b>Clicking Create Access Key</span><br><span><img id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_image20389043111611" src="en-us_image_0000002270789428.png"></span></div>
|
||||
<ol id="dataartsstudio_01_0026__en-us_topic_0108618545_ol1361418377715"><li id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_li1535103025819">Log in to the management console, move the cursor to the username in the upper right corner, and select <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b1586023610364">My Credentials</strong> from the drop-down list.</li><li id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_li173533018584">On the <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b149721128172418">My Credentials</strong> page, choose <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b15803656132414">Access Keys</strong>, and click <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b2452152020258">Create Access Key</strong>. See <a href="#dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615">Figure 1</a>.<div class="fignone" id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615"><a name="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615"></a><a name="en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615"></a><span class="figcap"><b>Figure 1 </b>Clicking Create Access Key</span><br><span><img id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_image20389043111611" src="en-us_image_0000002269194761.png" title="Click to enlarge" class="imgResize"></span></div>
|
||||
</li><li id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_li1535530185815">Click <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b8340319122819">OK</strong> and save the access key file as prompted. The access key file will be saved to your browser's configured download location. Open the <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b1940852512813">credentials.csv</strong> file to view <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b191574128718">Access Key Id</strong> and <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b175769149718">Secret Access Key</strong>.<div class="note" id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_note4554158201"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_ul25541081906"><li id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_li175541819018">Only two access keys can be added for each user.</li><li id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_li3554138305">To ensure access key security, the access key is automatically downloaded only when it is generated for the first time and cannot be obtained from the management console later. Keep them properly.</li></ul>
|
||||
</div></div>
|
||||
</li></ol>
|
||||
@ -233,8 +234,8 @@
|
||||
</td>
|
||||
<td class="cellrowborder" rowspan="2" valign="top" width="62.1%" headers="mcps1.3.3.4.2.4.1.2 "><p id="dataartsstudio_01_0026__en-us_topic_0108618545_p1953012815409">This parameter is mandatory when <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_b5498744161517">OBS storage support</strong> is enabled. The account corresponding to the AK/SK pair must have the OBS Buckets Viewer permission. Otherwise, OBS cannot be accessed and the "403 AccessDenied" error is reported.</p>
|
||||
<p id="dataartsstudio_01_0026__en-us_topic_0108618545_p8530122816408">You need to create an access key for the current account and obtain an AK/SK pair.</p>
|
||||
<ol id="dataartsstudio_01_0026__en-us_topic_0108618545_ol175313280409"><li id="dataartsstudio_01_0026__li66755233401">Log in to the management console, move the cursor to the username in the upper right corner, and select <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b1586023610364_1">My Credentials</strong> from the drop-down list.</li><li id="dataartsstudio_01_0026__li156752239403">On the <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b149721128172418_1">My Credentials</strong> page, choose <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b15803656132414_1">Access Keys</strong>, and click <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b2452152020258_1">Create Access Key</strong>. See <a href="#dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615">Figure 1</a>.<div class="fignone" id="dataartsstudio_01_0026__fig1667652312405"><span class="figcap"><b>Figure 2 </b>Clicking Create Access Key</span><br><span><img id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_image20389043111611_1" src="en-us_image_0000002270789428.png"></span></div>
|
||||
</li><li id="dataartsstudio_01_0026__li176766238405">Click <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b8340319122819_1">OK</strong> and save the access key file as prompted. The access key file will be saved to your browser's configured download location. Open the <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b1940852512813_1">credentials.csv</strong> file to view <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b191574128718_1">Access Key Id</strong> and <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b175769149718_1">Secret Access Key</strong>.<div class="note" id="dataartsstudio_01_0026__note16761323164012"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dataartsstudio_01_0026__ul667619237407"><li id="dataartsstudio_01_0026__li12676182317408">Only two access keys can be added for each user.</li><li id="dataartsstudio_01_0026__li106761723124014">To ensure access key security, the access key is automatically downloaded only when it is generated for the first time and cannot be obtained from the management console later. Keep them properly.</li></ul>
|
||||
<ol id="dataartsstudio_01_0026__en-us_topic_0108618545_ol175313280409"><li id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_li1535103025819_1">Log in to the management console, move the cursor to the username in the upper right corner, and select <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b1586023610364_1">My Credentials</strong> from the drop-down list.</li><li id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_li173533018584_1">On the <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b149721128172418_1">My Credentials</strong> page, choose <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b15803656132414_1">Access Keys</strong>, and click <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b2452152020258_1">Create Access Key</strong>. See <a href="#dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615_1">Figure 2</a>.<div class="fignone" id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615_1"><a name="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615_1"></a><a name="en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615_1"></a><span class="figcap"><b>Figure 2 </b>Clicking Create Access Key</span><br><span><img id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_image20389043111611_1" src="en-us_image_0000002269194761.png" title="Click to enlarge" class="imgResize"></span></div>
|
||||
</li><li id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_li1535530185815_1">Click <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b8340319122819_1">OK</strong> and save the access key file as prompted. The access key file will be saved to your browser's configured download location. Open the <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b1940852512813_1">credentials.csv</strong> file to view <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b191574128718_1">Access Key Id</strong> and <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b175769149718_1">Secret Access Key</strong>.<div class="note" id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_note4554158201_1"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_ul25541081906_1"><li id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_li175541819018_1">Only two access keys can be added for each user.</li><li id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_li3554138305_1">To ensure access key security, the access key is automatically downloaded only when it is generated for the first time and cannot be obtained from the management console later. Keep them properly.</li></ul>
|
||||
</div></div>
|
||||
</li></ol>
|
||||
</td>
|
||||
@ -343,8 +344,8 @@
|
||||
</td>
|
||||
<td class="cellrowborder" rowspan="2" valign="top" width="62.1%" headers="mcps1.3.4.4.2.4.1.2 "><p id="dataartsstudio_01_0026__en-us_topic_0108618545_p1063093184112">This parameter is mandatory when <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_b2163125219151">OBS storage support</strong> is enabled. The account corresponding to the AK/SK pair must have the OBS Buckets Viewer permission. Otherwise, OBS cannot be accessed and the "403 AccessDenied" error is reported.</p>
|
||||
<p id="dataartsstudio_01_0026__en-us_topic_0108618545_p1463019317413">You need to create an access key for the current account and obtain an AK/SK pair.</p>
|
||||
<ol id="dataartsstudio_01_0026__en-us_topic_0108618545_ol1463010324116"><li id="dataartsstudio_01_0026__li13685723174016">Log in to the management console, move the cursor to the username in the upper right corner, and select <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b1586023610364_2">My Credentials</strong> from the drop-down list.</li><li id="dataartsstudio_01_0026__li166861234408">On the <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b149721128172418_2">My Credentials</strong> page, choose <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b15803656132414_2">Access Keys</strong>, and click <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b2452152020258_2">Create Access Key</strong>. See <a href="#dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615">Figure 1</a>.<div class="fignone" id="dataartsstudio_01_0026__fig17686162394010"><span class="figcap"><b>Figure 3 </b>Clicking Create Access Key</span><br><span><img id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_image20389043111611_2" src="en-us_image_0000002270789428.png"></span></div>
|
||||
</li><li id="dataartsstudio_01_0026__li1668652364015">Click <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b8340319122819_2">OK</strong> and save the access key file as prompted. The access key file will be saved to your browser's configured download location. Open the <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b1940852512813_2">credentials.csv</strong> file to view <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b191574128718_2">Access Key Id</strong> and <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b175769149718_2">Secret Access Key</strong>.<div class="note" id="dataartsstudio_01_0026__note1768632304018"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dataartsstudio_01_0026__ul1768632319408"><li id="dataartsstudio_01_0026__li176861236400">Only two access keys can be added for each user.</li><li id="dataartsstudio_01_0026__li1568682318407">To ensure access key security, the access key is automatically downloaded only when it is generated for the first time and cannot be obtained from the management console later. Keep them properly.</li></ul>
|
||||
<ol id="dataartsstudio_01_0026__en-us_topic_0108618545_ol1463010324116"><li id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_li1535103025819_2">Log in to the management console, move the cursor to the username in the upper right corner, and select <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b1586023610364_2">My Credentials</strong> from the drop-down list.</li><li id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_li173533018584_2">On the <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b149721128172418_2">My Credentials</strong> page, choose <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b15803656132414_2">Access Keys</strong>, and click <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b2452152020258_2">Create Access Key</strong>. See <a href="#dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615_2">Figure 3</a>.<div class="fignone" id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615_2"><a name="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615_2"></a><a name="en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615_2"></a><span class="figcap"><b>Figure 3 </b>Clicking Create Access Key</span><br><span><img id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_image20389043111611_2" src="en-us_image_0000002269194761.png" title="Click to enlarge" class="imgResize"></span></div>
|
||||
</li><li id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_en-us_topic_0183643042_li1535530185815_2">Click <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b8340319122819_2">OK</strong> and save the access key file as prompted. The access key file will be saved to your browser's configured download location. Open the <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b1940852512813_2">credentials.csv</strong> file to view <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b191574128718_2">Access Key Id</strong> and <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_b175769149718_2">Secret Access Key</strong>.<div class="note" id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_note4554158201_2"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_ul25541081906_2"><li id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_li175541819018_2">Only two access keys can be added for each user.</li><li id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0000001129241845_li3554138305_2">To ensure access key security, the access key is automatically downloaded only when it is generated for the first time and cannot be obtained from the management console later. Keep them properly.</li></ul>
|
||||
</div></div>
|
||||
</li></ol>
|
||||
</td>
|
||||
@ -359,7 +360,7 @@
|
||||
<tr id="dataartsstudio_01_0026__en-us_topic_0108618545_row5325131312537"><td class="cellrowborder" valign="top" width="16.72%" headers="mcps1.3.4.4.2.4.1.1 "><p id="dataartsstudio_01_0026__en-us_topic_0108618545_p133251513195314">Run Mode</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="62.1%" headers="mcps1.3.4.4.2.4.1.2 "><p id="dataartsstudio_01_0026__en-us_topic_0108618545_p13251913175310">This parameter is used only when the Hive version is <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_b19361205321217">HIVE_3_X</strong>. Possible values are:</p>
|
||||
<ul id="dataartsstudio_01_0026__en-us_topic_0108618545_ul83512207444"><li id="dataartsstudio_01_0026__li166881723154013"><strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0108618545_b1095020874814_1">EMBEDDED</strong>: The link instance runs with CDM. This mode delivers better performance.</li><li id="dataartsstudio_01_0026__li136881237406"><strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0108618545_b1353409697105248_1">Standalone</strong>: The link instance runs in an independent process. If CDM needs to connect to multiple Hadoop data sources (MRS, Hadoop, or CloudTable) with both Kerberos and Simple authentication modes, <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0108618545_b9202184312114_1">Standalone</strong> prevails.<div class="note" id="dataartsstudio_01_0026__note3688823164014"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0026__p1668812324019">The <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0108618545_b1798836155817_1">STANDALONE</strong> mode is used to solve the version conflict problem. If the connector versions of the source and destination ends of the same link are different, a JAR file conflict occurs. In this case, you need to place the source or destination end in the STANDALONE process to prevent the migration failure caused by the conflict.</p>
|
||||
<ul id="dataartsstudio_01_0026__en-us_topic_0108618545_ul83512207444"><li id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0108618545_li188444317283_1"><strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0108618545_b1095020874814_1">EMBEDDED</strong>: The link instance runs with CDM. This mode delivers better performance.</li><li id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0108618545_li118451236288_1"><strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0108618545_b1353409697105248_1">Standalone</strong>: The link instance runs in an independent process. If CDM needs to connect to multiple Hadoop data sources (MRS, Hadoop, or CloudTable) with both Kerberos and Simple authentication modes, <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0108618545_b9202184312114_1">Standalone</strong> prevails.<div class="note" id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0108618545_note169619581157_1"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0108618545_p1184511312281_1">The <strong id="dataartsstudio_01_0026__en-us_topic_0108618545_en-us_topic_0108618545_b1798836155817_1">STANDALONE</strong> mode is used to solve the version conflict problem. If the connector versions of the source and destination ends of the same link are different, a JAR file conflict occurs. In this case, you need to place the source or destination end in the STANDALONE process to prevent the migration failure caused by the conflict.</p>
|
||||
</div></div>
|
||||
</li></ul>
|
||||
</td>
|
||||
@ -398,7 +399,14 @@
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0023.html">Managing Links</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0157.html">Configuring Link Parameters</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
<script language="JavaScript">
|
||||
<!--
|
||||
initImageViewer('.imgResize');
|
||||
var msg_imageMax = "view original image";
|
||||
var msg_imageClose = "close";
|
||||
//--></script>
|
||||
@ -1,6 +1,6 @@
|
||||
<a name="dataartsstudio_01_0027"></a><a name="dataartsstudio_01_0027"></a>
|
||||
|
||||
<h1 class="topictitle1">Link to CloudTable</h1>
|
||||
<h1 class="topictitle1">CloudTable Link Parameters</h1>
|
||||
<div id="body8662426"><p id="dataartsstudio_01_0027__en-us_topic_0108275355_p59722913113957">When connecting CDM to CloudTable, configure the parameters as described in <a href="#dataartsstudio_01_0027__en-us_topic_0108275355_table34037531171418">Table 1</a>.</p>
|
||||
<div class="note" id="dataartsstudio_01_0027__en-us_topic_0108275355_note8867140110"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_0027__en-us_topic_0108275355_en-us_topic_0108275292_p167832113185">Do not change the password or user when the job is running. If you do so, the password will not take effect immediately and the job will fail.</p>
|
||||
</div></div>
|
||||
@ -80,7 +80,7 @@
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0023.html">Managing Links</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0157.html">Configuring Link Parameters</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
<a name="dataartsstudio_01_0028"></a><a name="dataartsstudio_01_0028"></a>
|
||||
|
||||
<h1 class="topictitle1">Link to an FTP or SFTP Server</h1>
|
||||
<h1 class="topictitle1">FTP/SFTP Link Parameters</h1>
|
||||
<div id="body8662426"><p id="dataartsstudio_01_0028__en-us_topic_0108275400_p1616721175217">The FTP/SFTP link is used to migrate files from the on-premises file server or ECS to a database.</p>
|
||||
<div class="note" id="dataartsstudio_01_0028__en-us_topic_0108275400_note193108573515"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dataartsstudio_01_0028__en-us_topic_0108275400_ul103927387116"><li id="dataartsstudio_01_0028__en-us_topic_0108275400_li18637205191114">Only FTP servers running Linux are supported.</li><li id="dataartsstudio_01_0028__en-us_topic_0108275400_li13978165615117">Do not change the password or user when the job is running. If you do so, the password will not take effect immediately and the job will fail.</li></ul>
|
||||
</div></div>
|
||||
@ -48,6 +48,14 @@
|
||||
<td class="cellrowborder" valign="top" width="20.62%" headers="mcps1.3.3.2.2.4.1.3 "><p id="dataartsstudio_01_0028__en-us_topic_0108275400_p8552168171418">-</p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0028__en-us_topic_0108275400_row51011378115"><td class="cellrowborder" valign="top" width="18.060000000000002%" headers="mcps1.3.3.2.2.4.1.1 "><p id="dataartsstudio_01_0028__en-us_topic_0108275400_p8101671319">FTP File Name controlEncoding</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="61.31999999999999%" headers="mcps1.3.3.2.2.4.1.2 "><p id="dataartsstudio_01_0028__en-us_topic_0108275400_p3637732918">This parameter is available for a FTP link.</p>
|
||||
<p id="dataartsstudio_01_0028__en-us_topic_0108275400_p1210116716117">It indicates the controlEncoding file name encoding configuration of ftp-client. The value can be <strong id="dataartsstudio_01_0028__en-us_topic_0108275400_b1456831618164">ISO-8859-1</strong> or <strong id="dataartsstudio_01_0028__en-us_topic_0108275400_b289182021611">UFT8</strong>. The default value is <strong id="dataartsstudio_01_0028__en-us_topic_0108275400_b7771112618168">ISO-8859-1</strong>.</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="20.62%" headers="mcps1.3.3.2.2.4.1.3 "><p id="dataartsstudio_01_0028__en-us_topic_0108275400_p121011713113">ISO-8859-1</p>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
@ -55,7 +63,7 @@
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0023.html">Managing Links</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0157.html">Configuring Link Parameters</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
@ -1,9 +1,9 @@
|
||||
<a name="dataartsstudio_01_0030"></a><a name="dataartsstudio_01_0030"></a>
|
||||
|
||||
<h1 class="topictitle1">Link to MongoDB</h1>
|
||||
<h1 class="topictitle1">MongoDB Link Parameters</h1>
|
||||
<div id="body8662426"><p id="dataartsstudio_01_0030__en-us_topic_0108275382_p868619161103">This link is used to transfer data from a third-party cloud MongoDB service or MongoDB created in the on-premises data center or ECS to a big data platform.</p>
|
||||
<p id="dataartsstudio_01_0030__en-us_topic_0108275382_p1471816353573">When connecting CDM to an on-premises MongoDB database, configure the parameters as described in <a href="#dataartsstudio_01_0030__en-us_topic_0108275382_table34037531171418">Table 1</a>.</p>
|
||||
<div class="note" id="dataartsstudio_01_0030__en-us_topic_0108275382_note8867140110"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_0030__en-us_topic_0108275382_en-us_topic_0108275292_p167832113185">Do not change the password or user when the job is running. If you do so, the password will not take effect immediately and the job will fail.</p>
|
||||
<div class="note" id="dataartsstudio_01_0030__en-us_topic_0108275382_note8867140110"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dataartsstudio_01_0030__en-us_topic_0108275382_ul1673418379363"><li id="dataartsstudio_01_0030__en-us_topic_0108275382_li1513715717348">MongoDB data sources with SSL enabled are not supported.</li><li id="dataartsstudio_01_0030__en-us_topic_0108275382_li9154151010387">Do not change the password or user when a job is running. If you do so, the password will not take effect immediately and the job will fail.</li></ul>
|
||||
</div></div>
|
||||
|
||||
<div class="tablenoborder"><a name="dataartsstudio_01_0030__en-us_topic_0108275382_table34037531171418"></a><a name="en-us_topic_0108275382_table34037531171418"></a><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_01_0030__en-us_topic_0108275382_table34037531171418" frame="border" border="1" rules="all"><caption><b>Table 1 </b>MongoDB link parameters</caption><thead align="left"><tr id="dataartsstudio_01_0030__en-us_topic_0108275382_row56630393171418"><th align="left" class="cellrowborder" valign="top" width="19.18%" id="mcps1.3.4.2.4.1.1"><p id="dataartsstudio_01_0030__en-us_topic_0108275382_p23659124171418">Parameter</p>
|
||||
@ -72,7 +72,7 @@
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0023.html">Managing Links</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0157.html">Configuring Link Parameters</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
@ -1,9 +1,9 @@
|
||||
<a name="dataartsstudio_01_0031"></a><a name="dataartsstudio_01_0031"></a>
|
||||
|
||||
<h1 class="topictitle1">Link to DDS</h1>
|
||||
<h1 class="topictitle1">DDS Link Parameters</h1>
|
||||
<div id="body8662426"><p id="dataartsstudio_01_0031__en-us_topic_0173586864_p175011321165416">The DDS link is used to synchronize data from Document Database Service (DDS) on the cloud to a big data platform.</p>
|
||||
<p id="dataartsstudio_01_0031__en-us_topic_0173586864_p1471816353573">When connecting CDM to DDS, configure the parameters as described in <a href="#dataartsstudio_01_0031__en-us_topic_0173586864_table34037531171418">Table 1</a>.</p>
|
||||
<div class="note" id="dataartsstudio_01_0031__en-us_topic_0173586864_note8867140110"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_0031__en-us_topic_0173586864_en-us_topic_0108275292_p167832113185">Do not change the password or user when the job is running. If you do so, the password will not take effect immediately and the job will fail.</p>
|
||||
<div class="note" id="dataartsstudio_01_0031__en-us_topic_0173586864_note8867140110"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dataartsstudio_01_0031__en-us_topic_0173586864_ul550618517341"><li id="dataartsstudio_01_0031__en-us_topic_0173586864_li1513715717348">DDS data sources with SSL enabled are not supported.</li><li id="dataartsstudio_01_0031__en-us_topic_0173586864_li450615553411">Do not change the password or user when a job is running. If you do so, the password will not take effect immediately and the job will fail.</li></ul>
|
||||
</div></div>
|
||||
|
||||
<div class="tablenoborder"><a name="dataartsstudio_01_0031__en-us_topic_0173586864_table34037531171418"></a><a name="en-us_topic_0173586864_table34037531171418"></a><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_01_0031__en-us_topic_0173586864_table34037531171418" frame="border" border="1" rules="all"><caption><b>Table 1 </b>DDS link parameters</caption><thead align="left"><tr id="dataartsstudio_01_0031__en-us_topic_0173586864_row56630393171418"><th align="left" class="cellrowborder" valign="top" width="18.509999999999998%" id="mcps1.3.4.2.4.1.1"><p id="dataartsstudio_01_0031__en-us_topic_0173586864_p23659124171418">Parameter</p>
|
||||
@ -64,7 +64,7 @@
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0023.html">Managing Links</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0157.html">Configuring Link Parameters</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
<a name="dataartsstudio_01_0032"></a><a name="dataartsstudio_01_0032"></a>
|
||||
|
||||
<h1 class="topictitle1">Link to Redis</h1>
|
||||
<h1 class="topictitle1">Redis Link Parameters</h1>
|
||||
<div id="body8662426"><p id="dataartsstudio_01_0032__en-us_topic_0108275391_p1746173925518">The Redis link is applicable to data migration of Redis created in the local data center or ECS. It is used to load data in the database or files to Redis.</p>
|
||||
<p id="dataartsstudio_01_0032__en-us_topic_0108275391_p75081211182211">Links to Redis data encrypted using SSL are not supported.</p>
|
||||
<p id="dataartsstudio_01_0032__en-us_topic_0108275391_p840683413016">When connecting CDM to an on-premises Redis database, configure the parameters as described in <a href="#dataartsstudio_01_0032__en-us_topic_0108275391_table34037531171418">Table 1</a>.</p>
|
||||
@ -84,7 +84,7 @@
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0023.html">Managing Links</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0157.html">Configuring Link Parameters</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
<a name="dataartsstudio_01_0033"></a><a name="dataartsstudio_01_0033"></a>
|
||||
|
||||
<h1 class="topictitle1">Link to Kafka</h1>
|
||||
<h1 class="topictitle1">Kafka Link Parameters</h1>
|
||||
<div id="body8662426"><div class="section" id="dataartsstudio_01_0033__en-us_topic_0108275385_section8886729122818"><h4 class="sectiontitle">MRS Kafka</h4><p id="dataartsstudio_01_0033__en-us_topic_0108275385_p1822602317293">When connecting CDM to Kafka of MRS, configure the parameters as described in <a href="#dataartsstudio_01_0033__en-us_topic_0108275385_table3104110173020">Table 1</a>.</p>
|
||||
<div class="note" id="dataartsstudio_01_0033__en-us_topic_0108275385_note8867140110"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_0033__en-us_topic_0108275385_en-us_topic_0108275292_p167832113185">Do not change the password or user when the job is running. If you do so, the password will not take effect immediately and the job will fail.</p>
|
||||
</div></div>
|
||||
@ -22,7 +22,7 @@
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0033__en-us_topic_0108275385_row11045013017"><td class="cellrowborder" valign="top" width="18.95%" headers="mcps1.3.1.4.2.4.1.1 "><p id="dataartsstudio_01_0033__en-us_topic_0108275385_p121041208302">Manager IP</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="61.42999999999999%" headers="mcps1.3.1.4.2.4.1.2 "><div class="p" id="dataartsstudio_01_0033__en-us_topic_0108275385_p218148165010">Floating IP address of MRS Manager. Click <span class="uicontrol" id="dataartsstudio_01_0033__en-us_topic_0108275385_en-us_topic_0108275286_uicontrol578864992012"><b>Select</b></span> next to the <strong id="dataartsstudio_01_0033__en-us_topic_0108275385_en-us_topic_0108275286_b2078834920203">Manager IP</strong> text box to select an MRS cluster. CDM automatically fills in the authentication information.<div class="note" id="dataartsstudio_01_0033__note1340014473413"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0033__en-us_topic_0182566327_p116249795810"><span id="dataartsstudio_01_0033__en-us_topic_0182566327_text7355181825814">DataArts Studio</span> does not support MRS clusters whose Kerberos encryption type is <strong id="dataartsstudio_01_0033__en-us_topic_0182566327_b143479402717">aes256-sha2,aes128-sha2</strong>, and only supports MRS clusters whose Kerberos encryption type is <strong id="dataartsstudio_01_0033__en-us_topic_0182566327_b19532103219813">aes256-sha1,aes128-sha1</strong>.</p>
|
||||
<td class="cellrowborder" valign="top" width="61.42999999999999%" headers="mcps1.3.1.4.2.4.1.2 "><div class="p" id="dataartsstudio_01_0033__en-us_topic_0108275385_p218148165010">Floating IP address of MRS Manager. Click <span class="uicontrol" id="dataartsstudio_01_0033__en-us_topic_0108275385_en-us_topic_0108275286_uicontrol578864992012"><b>Select</b></span> next to the <strong id="dataartsstudio_01_0033__en-us_topic_0108275385_en-us_topic_0108275286_b2078834920203">Manager IP</strong> text box to select an MRS cluster. CDM automatically fills in the authentication information.<div class="note" id="dataartsstudio_01_0033__en-us_topic_0108275385_en-us_topic_0108275286_note1340014473413"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0033__en-us_topic_0108275385_en-us_topic_0108275286_en-us_topic_0182566327_p116249795810"><span id="dataartsstudio_01_0033__en-us_topic_0108275385_en-us_topic_0108275286_en-us_topic_0182566327_text7355181825814">DataArts Studio</span> does not support MRS clusters whose Kerberos encryption type is <strong id="dataartsstudio_01_0033__en-us_topic_0108275385_en-us_topic_0108275286_en-us_topic_0182566327_b143479402717">aes256-sha2,aes128-sha2</strong>, and only supports MRS clusters whose Kerberos encryption type is <strong id="dataartsstudio_01_0033__en-us_topic_0108275385_en-us_topic_0108275286_en-us_topic_0182566327_b19532103219813">aes256-sha1,aes128-sha1</strong>.</p>
|
||||
</div></div>
|
||||
</div>
|
||||
</td>
|
||||
@ -64,9 +64,9 @@
|
||||
|
||||
<div class="tablenoborder"><a name="dataartsstudio_01_0033__en-us_topic_0108275385_table1990636915212"></a><a name="en-us_topic_0108275385_table1990636915212"></a><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_01_0033__en-us_topic_0108275385_table1990636915212" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Parameter description</caption><thead align="left"><tr id="dataartsstudio_01_0033__en-us_topic_0108275385_row4257242215212"><th align="left" class="cellrowborder" valign="top" width="18.95%" id="mcps1.3.2.4.2.4.1.1"><p id="dataartsstudio_01_0033__en-us_topic_0108275385_p3100089215212"><strong id="dataartsstudio_01_0033__en-us_topic_0108275385_en-us_topic_0065840554_b84235270692541">Parameter</strong></p>
|
||||
</th>
|
||||
<th align="left" class="cellrowborder" valign="top" width="61.42999999999999%" id="mcps1.3.2.4.2.4.1.2"><p id="dataartsstudio_01_0033__en-us_topic_0108275385_p2804435515212"><strong id="dataartsstudio_01_0033__en-us_topic_0108275385_b37959942">Description</strong></p>
|
||||
<th align="left" class="cellrowborder" valign="top" width="61.42999999999999%" id="mcps1.3.2.4.2.4.1.2"><p id="dataartsstudio_01_0033__en-us_topic_0108275385_p2804435515212"><strong id="dataartsstudio_01_0033__en-us_topic_0108275385_b1351820074">Description</strong></p>
|
||||
</th>
|
||||
<th align="left" class="cellrowborder" valign="top" width="19.62%" id="mcps1.3.2.4.2.4.1.3"><p id="dataartsstudio_01_0033__en-us_topic_0108275385_p5700024515212"><strong id="dataartsstudio_01_0033__en-us_topic_0108275385_b1153592201">Example Value</strong></p>
|
||||
<th align="left" class="cellrowborder" valign="top" width="19.62%" id="mcps1.3.2.4.2.4.1.3"><p id="dataartsstudio_01_0033__en-us_topic_0108275385_p5700024515212"><strong id="dataartsstudio_01_0033__en-us_topic_0108275385_b1501787170">Example Value</strong></p>
|
||||
</th>
|
||||
</tr>
|
||||
</thead>
|
||||
@ -92,7 +92,7 @@
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0023.html">Managing Links</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0157.html">Configuring Link Parameters</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
<a name="dataartsstudio_01_0035"></a><a name="dataartsstudio_01_0035"></a>
|
||||
|
||||
<h1 class="topictitle1">Link to CSS</h1>
|
||||
<h1 class="topictitle1">CSS Link Parameters</h1>
|
||||
<div id="body8662426"><p id="dataartsstudio_01_0035__en-us_topic_0108275341_p3341958135318">Cloud Search Service (CSS) is a fully hosted distributed search service powered by open-source Elasticsearch. CSS links can be used to migrate log files and database records to CSS for search and analysis using Elasticsearch.</p>
|
||||
<div class="note" id="dataartsstudio_01_0035__en-us_topic_0108275341_note1213983418198"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dataartsstudio_01_0035__en-us_topic_0108275341_ul161881059148"><li id="dataartsstudio_01_0035__en-us_topic_0108275341_li894710289414">Do not change the password or user when the job is running. If you do so, the password will not take effect immediately and the job will fail.</li></ul>
|
||||
</div></div>
|
||||
@ -52,7 +52,7 @@
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0035__en-us_topic_0108275341_row83091712451"><td class="cellrowborder" valign="top" width="21.85%" headers="mcps1.3.4.2.4.1.1 "><p id="dataartsstudio_01_0035__en-us_topic_0108275341_p591311246511">HTTPS Access</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="55.17999999999999%" headers="mcps1.3.4.2.4.1.2 "><p id="dataartsstudio_01_0035__en-us_topic_0108275341_p663711151553">This parameter is displayed when <strong id="dataartsstudio_01_0035__en-us_topic_0108275341_b539290683">Security Mode Authentication</strong> is set to <strong id="dataartsstudio_01_0035__en-us_topic_0108275341_b1208862950">Yes</strong>. This parameter specifies whether to enable HTTPS access. HTTPS access is more secure than HTTP access.</p>
|
||||
<td class="cellrowborder" valign="top" width="55.17999999999999%" headers="mcps1.3.4.2.4.1.2 "><p id="dataartsstudio_01_0035__en-us_topic_0108275341_p663711151553">This parameter is displayed when <strong id="dataartsstudio_01_0035__en-us_topic_0108275341_b1537589756">Security Mode Authentication</strong> is set to <strong id="dataartsstudio_01_0035__en-us_topic_0108275341_b1749991604">Yes</strong>. This parameter specifies whether to enable HTTPS access. HTTPS access is more secure than HTTP access.</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="22.97%" headers="mcps1.3.4.2.4.1.3 "><p id="dataartsstudio_01_0035__en-us_topic_0108275341_p86372015554">Yes</p>
|
||||
</td>
|
||||
@ -63,7 +63,7 @@
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0023.html">Managing Links</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0157.html">Configuring Link Parameters</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@ -1,6 +1,6 @@
|
||||
<a name="dataartsstudio_01_0037"></a><a name="dataartsstudio_01_0037"></a>
|
||||
|
||||
<h1 class="topictitle1">Link to CloudTable OpenTSDB</h1>
|
||||
<h1 class="topictitle1">CloudTable OpenTSDB Link Parameters</h1>
|
||||
<div id="body8662426"><p id="dataartsstudio_01_0037__en-us_topic_0133463138_p493133413817">When connecting CDM to CloudTable OpenTSDB, configure the parameters as described in <a href="#dataartsstudio_01_0037__en-us_topic_0133463138_table22075105144748">Table 1</a>.</p>
|
||||
|
||||
<div class="tablenoborder"><a name="dataartsstudio_01_0037__en-us_topic_0133463138_table22075105144748"></a><a name="en-us_topic_0133463138_table22075105144748"></a><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_01_0037__en-us_topic_0133463138_table22075105144748" frame="border" border="1" rules="all"><caption><b>Table 1 </b>CloudTable OpenTSDB link parameters</caption><thead align="left"><tr id="dataartsstudio_01_0037__en-us_topic_0133463138_row19905440144748"><th align="left" class="cellrowborder" valign="top" width="20.07%" id="mcps1.3.2.2.4.1.1"><p id="dataartsstudio_01_0037__en-us_topic_0133463138_p1727937144748">Parameter</p>
|
||||
@ -36,7 +36,7 @@
|
||||
<tr id="dataartsstudio_01_0037__en-us_topic_0133463138_row1392892065413"><td class="cellrowborder" valign="top" width="20.07%" headers="mcps1.3.2.2.4.1.1 "><p id="dataartsstudio_01_0037__en-us_topic_0133463138_p1016019285015">Project ID</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="58.19%" headers="mcps1.3.2.2.4.1.2 "><p id="dataartsstudio_01_0037__en-us_topic_0133463138_p1637616424613">Project ID in the region where CloudTable resides</p>
|
||||
<div class="p" id="dataartsstudio_01_0037__en-us_topic_0133463138_p1668482916813">A project is a group of tenant resources, and an account ID corresponds to the current account. The IAM ID corresponds to the current user. You can view the project IDs, account IDs, and user IDs in different regions on the corresponding pages.<ol id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129537317_ol6260200121819"><li id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129537317_li19260803185">Register with and log in to the management console.</li><li id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129537317_li142601019186">Hover the cursor on the username in the upper right corner and select <span class="uicontrol" id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129537317_uicontrol0260130191818"><b>My Credentials</b></span> from the drop-down list.</li><li id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129537317_li1026113031818">On the <strong id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129537317_b0801614153013">API Credentials</strong> page, obtain the account name, account ID, IAM username, and IAM user ID, and obtain the project ID from the project list.</li></ol>
|
||||
<div class="p" id="dataartsstudio_01_0037__en-us_topic_0133463138_p1668482916813">A project is a group of tenant resources, and an account ID corresponds to the current account. The IAM ID corresponds to the current user. You can view the project IDs, account IDs, and user IDs in different regions on the corresponding pages.<ol id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129537317_ol6260200121819"><li id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129537317_li19260803185">Register with and log in to the management console.</li><li id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129537317_li142601019186">Hover the cursor on the username in the upper right corner and select <span class="uicontrol" id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129537317_uicontrol0260130191818"><b>My Credentials</b></span> from the drop-down list.</li><li id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129537317_li1026113031818">On the <strong id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129537317_b588761635018">API Credentials</strong> page, obtain the account name, account ID, IAM username, and IAM user ID, and obtain the project and its ID from the project list.</li></ol>
|
||||
</div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="21.740000000000002%" headers="mcps1.3.2.2.4.1.3 "><p id="dataartsstudio_01_0037__en-us_topic_0133463138_p11608245019">-</p>
|
||||
@ -53,7 +53,7 @@
|
||||
</td>
|
||||
<td class="cellrowborder" rowspan="2" valign="top" width="58.19%" headers="mcps1.3.2.2.4.1.2 "><p id="dataartsstudio_01_0037__en-us_topic_0133463138_p181471423104">AK and SK for accessing CloudTable.</p>
|
||||
<p id="dataartsstudio_01_0037__en-us_topic_0133463138_p2396698815212">You need to create an access key for the current account and obtain an AK/SK pair.</p>
|
||||
<ol id="dataartsstudio_01_0037__en-us_topic_0133463138_ol1361418377715"><li id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_en-us_topic_0183643042_li1535103025819">Log in to the management console, move the cursor to the username in the upper right corner, and select <strong id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_b1586023610364">My Credentials</strong> from the drop-down list.</li><li id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_en-us_topic_0183643042_li173533018584">On the <strong id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_b149721128172418">My Credentials</strong> page, choose <strong id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_b15803656132414">Access Keys</strong>, and click <strong id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_b2452152020258">Create Access Key</strong>. See <a href="#dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615">Figure 1</a>.<div class="fignone" id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615"><a name="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615"></a><a name="en-us_topic_0133463138_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615"></a><span class="figcap"><b>Figure 1 </b>Clicking Create Access Key</span><br><span><img id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_image20389043111611" src="en-us_image_0000002270789428.png"></span></div>
|
||||
<ol id="dataartsstudio_01_0037__en-us_topic_0133463138_ol1361418377715"><li id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_en-us_topic_0183643042_li1535103025819">Log in to the management console, move the cursor to the username in the upper right corner, and select <strong id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_b1586023610364">My Credentials</strong> from the drop-down list.</li><li id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_en-us_topic_0183643042_li173533018584">On the <strong id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_b149721128172418">My Credentials</strong> page, choose <strong id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_b15803656132414">Access Keys</strong>, and click <strong id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_b2452152020258">Create Access Key</strong>. See <a href="#dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615">Figure 1</a>.<div class="fignone" id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615"><a name="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615"></a><a name="en-us_topic_0133463138_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615"></a><span class="figcap"><b>Figure 1 </b>Clicking Create Access Key</span><br><span><img id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_image20389043111611" src="en-us_image_0000002269194761.png" title="Click to enlarge" class="imgResize"></span></div>
|
||||
</li><li id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_en-us_topic_0183643042_li1535530185815">Click <strong id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_b8340319122819">OK</strong> and save the access key file as prompted. The access key file will be saved to your browser's configured download location. Open the <strong id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_b1940852512813">credentials.csv</strong> file to view <strong id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_b191574128718">Access Key Id</strong> and <strong id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_b175769149718">Secret Access Key</strong>.<div class="note" id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_note4554158201"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_ul25541081906"><li id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_li175541819018">Only two access keys can be added for each user.</li><li id="dataartsstudio_01_0037__en-us_topic_0133463138_en-us_topic_0000001129241845_li3554138305">To ensure access key security, the access key is automatically downloaded only when it is generated for the first time and cannot be obtained from the management console later. Keep them properly.</li></ul>
|
||||
</div></div>
|
||||
</li></ol>
|
||||
@ -72,7 +72,14 @@
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0023.html">Managing Links</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0157.html">Configuring Link Parameters</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
<script language="JavaScript">
|
||||
<!--
|
||||
initImageViewer('.imgResize');
|
||||
var msg_imageMax = "view original image";
|
||||
var msg_imageClose = "close";
|
||||
//--></script>
|
||||
@ -1,6 +1,6 @@
|
||||
<a name="dataartsstudio_01_0038"></a><a name="dataartsstudio_01_0038"></a>
|
||||
|
||||
<h1 class="topictitle1">Link to DMS Kafka</h1>
|
||||
<h1 class="topictitle1">DMS Kafka Link Parameters</h1>
|
||||
<div id="body8662426"><p id="dataartsstudio_01_0038__en-us_topic_0143085538_p493133413817">When connecting CDM to DMS Kafka, configure the parameters as described in <a href="#dataartsstudio_01_0038__en-us_topic_0143085538_table22075105144748">Table 1</a>.</p>
|
||||
<div class="note" id="dataartsstudio_01_0038__en-us_topic_0143085538_note8867140110"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_0038__en-us_topic_0143085538_en-us_topic_0108275292_p167832113185">Do not change the password or user when the job is running. If you do so, the password will not take effect immediately and the job will fail.</p>
|
||||
</div></div>
|
||||
@ -72,7 +72,7 @@
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0023.html">Managing Links</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0157.html">Configuring Link Parameters</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
@ -1,12 +1,16 @@
|
||||
<a name="dataartsstudio_01_0039"></a><a name="dataartsstudio_01_0039"></a>
|
||||
|
||||
<h1 class="topictitle1">Link to HBase</h1>
|
||||
<h1 class="topictitle1">HBase Link Parameters</h1>
|
||||
<div id="body8662426"><div class="p" id="dataartsstudio_01_0039__en-us_topic_0108604188_p396911259314">CDM supports the following HBase data sources:<ul id="dataartsstudio_01_0039__en-us_topic_0108604188_ul1998193319335"><li id="dataartsstudio_01_0039__en-us_topic_0108604188_li189813313319"><a href="#dataartsstudio_01_0039__en-us_topic_0108604188_section127371747103314">MRS HBase</a></li><li id="dataartsstudio_01_0039__en-us_topic_0108604188_li46813410343"><a href="#dataartsstudio_01_0039__en-us_topic_0108604188_section1851831133416">FusionInsight HBase</a></li><li id="dataartsstudio_01_0039__en-us_topic_0108604188_li88038432344"><a href="#dataartsstudio_01_0039__en-us_topic_0108604188_section690499173512">Apache HBase</a><div class="note" id="dataartsstudio_01_0039__en-us_topic_0108604188_note8867140110"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108275292_p167832113185">Do not change the password or user when the job is running. If you do so, the password will not take effect immediately and the job will fail.</p>
|
||||
</div></div>
|
||||
</li></ul>
|
||||
</div>
|
||||
<div class="section" id="dataartsstudio_01_0039__en-us_topic_0108604188_section127371747103314"><a name="dataartsstudio_01_0039__en-us_topic_0108604188_section127371747103314"></a><a name="en-us_topic_0108604188_section127371747103314"></a><h4 class="sectiontitle">MRS HBase</h4><p id="dataartsstudio_01_0039__en-us_topic_0108604188_p124412208341">When connecting CDM to HBase of MRS, configure the parameters as described in <a href="#dataartsstudio_01_0039__en-us_topic_0108604188_table6441152003419">Table 1</a>.</p>
|
||||
<div class="note" id="dataartsstudio_01_0039__en-us_topic_0108604188_note0738159182614"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dataartsstudio_01_0039__en-us_topic_0108604188_ul268415325415"><li id="dataartsstudio_01_0039__en-us_topic_0108604188_li116841636546">Before creating an MRS link, you need to add an authenticated Kerberos user on MRS and log in to the MRS management page to change the initial password. Then use the new user to create an MRS link.</li><li id="dataartsstudio_01_0039__en-us_topic_0108604188_li36846319542">To connect to an MRS 2.<em id="dataartsstudio_01_0039__en-us_topic_0108604188_i1155152111254">x</em> cluster, create a CDM cluster of version 2.<em id="dataartsstudio_01_0039__en-us_topic_0108604188_i115512112257">x</em> first. CDM 1.8.<em id="dataartsstudio_01_0039__en-us_topic_0108604188_i0707423162514">x</em> clusters cannot connect to MRS 2.<em id="dataartsstudio_01_0039__en-us_topic_0108604188_i0707423142520">x</em> clusters.</li><li id="dataartsstudio_01_0039__en-us_topic_0108604188_li132471615165418">If the connection fails after you select a cluster, check whether the MRS cluster can communicate with the CDM instance which functions as the agent. They can communicate with each other in the following scenarios:<ul id="dataartsstudio_01_0039__ul9600145218485"><li id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0000001503606428_li17205293217">If the CDM cluster in the <span id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0000001503606428_text15319922125112">DataArts Studio</span> instance and the MRS cluster are in different regions, a public network or a dedicated connection is required. If the Internet is used for communication, ensure that an EIP has been bound to the CDM cluster, and the MRS cluster can access the Internet and the port has been enabled in the firewall rule.</li><li id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0000001503606428_li13205893215">If the CDM cluster in the <span id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0000001503606428_text867019165375">DataArts Studio</span> instance and the cloud service are in the same region, VPC, subnet, and security group, they can communicate with each other by default. If they are in the same VPC but in different subnets or security groups, you must configure routing rules and security group rules. For details about how to configure routing rules, see "Adding a Custom Route" in <em id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0000001503606428_i1267271614375">Virtual Private Cloud (VPC) Usage Guide</em>. For details about how to configure security group rules, see "Security Group" > "Adding a Security Group Rule" in <em id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0000001503606428_i15675816103713">Virtual Private Cloud (VPC) Usage Guide</em>.</li><li id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0000001503606428_li20156162311283">The MRS cluster and the <span id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0000001503606428_text4156132310284">DataArts Studio</span> workspace belong to the same enterprise project. If they do not, you can modify the enterprise project of the workspace.</li></ul>
|
||||
<div class="note" id="dataartsstudio_01_0039__en-us_topic_0108604188_note0738159182614"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dataartsstudio_01_0039__en-us_topic_0108604188_ul268415325415"><li id="dataartsstudio_01_0039__en-us_topic_0108604188_li3509922112118"><p id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0000001624214396_p1463312815216"><a name="dataartsstudio_01_0039__en-us_topic_0108604188_li3509922112118"></a><a name="en-us_topic_0108604188_li3509922112118"></a>MRS clusters whose Kerberos encryption type is <strong id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0000001624214396_b742132751912">aes256-sha2,aes128-sha2</strong> are not supported, and only MRS clusters whose Kerberos encryption type is <strong id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0000001624214396_b1977394971913">aes256-sha1,aes128-sha1</strong> are supported.</p>
|
||||
</li><li id="dataartsstudio_01_0039__en-us_topic_0108604188_li116841636546">Before creating an MRS link, you need to add an authenticated Kerberos user on MRS and log in to the MRS management page to change the initial password. Then use the new user to create an MRS link.</li><li id="dataartsstudio_01_0039__en-us_topic_0108604188_li36846319542">To connect to an MRS 2.<em id="dataartsstudio_01_0039__en-us_topic_0108604188_i1155152111254">x</em> cluster, create a CDM cluster of version 2.<em id="dataartsstudio_01_0039__en-us_topic_0108604188_i115512112257">x</em> first. CDM 1.8.<em id="dataartsstudio_01_0039__en-us_topic_0108604188_i0707423162514">x</em> clusters cannot connect to MRS 2.<em id="dataartsstudio_01_0039__en-us_topic_0108604188_i0707423142520">x</em> clusters.</li><li id="dataartsstudio_01_0039__en-us_topic_0108604188_li132471615165418"><div class="p" id="dataartsstudio_01_0039__en-us_topic_0108604188_p134241819125419"><a name="dataartsstudio_01_0039__en-us_topic_0108604188_li132471615165418"></a><a name="en-us_topic_0108604188_li132471615165418"></a>If the connection fails after you select a cluster, check whether the MRS cluster can communicate with the CDM instance which functions as the agent. They can communicate with each other in the following scenarios:<ul id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0000001503606428_ul1715617232281"><li id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0000001503606428_li17205293217">If the CDM cluster in the <span id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0000001503606428_text15319922125112">DataArts Studio</span> instance and the MRS cluster are in different regions, a public network or a dedicated connection is required. If the Internet is used for communication, ensure that an EIP has been bound to the CDM cluster, and the MRS cluster can access the Internet and the port has been enabled in the firewall rule.</li><li id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0000001503606428_li13205893215">If the CDM cluster in the <span id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0000001503606428_text151663810818">DataArts Studio</span> instance and the MRS cluster are in the same region, VPC, subnet, and security group, they can communicate with each other by default. If they are in the same VPC but in different subnets or security groups, you must configure routing rules and security group rules. For details about how to configure routing rules, see "Adding Routes to a Route Table" in <em id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0000001503606428_i1317838485">Virtual Private Cloud (VPC) x.x.x User Guide</em> in <em id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0000001503606428_i13171138184">Virtual Private Cloud (VPC) x.x.x Usage Guide</em>. For details about how to configure security group rules, see "Security Group" > "Adding a Security Group Rule" in Virtual Private Cloud (VPC) x.x.x User Guide in Virtual Private Cloud (VPC) x.x.x Usage Guide.</li><li id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0000001503606428_li20156162311283">The MRS cluster and the <span id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0000001503606428_text4156132310284">DataArts Studio</span> workspace belong to the same enterprise project. If they do not, you can modify the enterprise project of the workspace.</li></ul>
|
||||
<div class="note" id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0000001503606428_note030793314500"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0000001503606428_p130813395015">If an agent is connected to multiple MRS clusters and one of the MRS clusters is deleted or abnormal, connections to the other MRS clusters will be affected. Therefore, you are advised to connect an agent to only one MRS cluster.</p>
|
||||
</div></div>
|
||||
</div>
|
||||
</li></ul>
|
||||
</div></div>
|
||||
|
||||
@ -27,7 +31,7 @@
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0039__en-us_topic_0108604188_row2456122053415"><td class="cellrowborder" valign="top" width="21.520000000000003%" headers="mcps1.3.2.4.2.4.1.1 "><p id="dataartsstudio_01_0039__en-us_topic_0108604188_p13456172019344">Manager IP</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="59.20000000000001%" headers="mcps1.3.2.4.2.4.1.2 "><div class="p" id="dataartsstudio_01_0039__en-us_topic_0108604188_p218148165010">Floating IP address of MRS Manager. Click <span class="uicontrol" id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108275286_uicontrol578864992012"><b>Select</b></span> next to the <strong id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108275286_b2078834920203">Manager IP</strong> text box to select an MRS cluster. CDM automatically fills in the authentication information.<div class="note" id="dataartsstudio_01_0039__note1340014473413"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0039__en-us_topic_0182566327_p116249795810"><span id="dataartsstudio_01_0039__en-us_topic_0182566327_text7355181825814">DataArts Studio</span> does not support MRS clusters whose Kerberos encryption type is <strong id="dataartsstudio_01_0039__en-us_topic_0182566327_b143479402717">aes256-sha2,aes128-sha2</strong>, and only supports MRS clusters whose Kerberos encryption type is <strong id="dataartsstudio_01_0039__en-us_topic_0182566327_b19532103219813">aes256-sha1,aes128-sha1</strong>.</p>
|
||||
<td class="cellrowborder" valign="top" width="59.20000000000001%" headers="mcps1.3.2.4.2.4.1.2 "><div class="p" id="dataartsstudio_01_0039__en-us_topic_0108604188_p218148165010">Floating IP address of MRS Manager. Click <span class="uicontrol" id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108275286_uicontrol578864992012"><b>Select</b></span> next to the <strong id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108275286_b2078834920203">Manager IP</strong> text box to select an MRS cluster. CDM automatically fills in the authentication information.<div class="note" id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108275286_note1340014473413"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108275286_en-us_topic_0182566327_p116249795810"><span id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108275286_en-us_topic_0182566327_text7355181825814">DataArts Studio</span> does not support MRS clusters whose Kerberos encryption type is <strong id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108275286_en-us_topic_0182566327_b143479402717">aes256-sha2,aes128-sha2</strong>, and only supports MRS clusters whose Kerberos encryption type is <strong id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108275286_en-us_topic_0182566327_b19532103219813">aes256-sha1,aes128-sha1</strong>.</p>
|
||||
</div></div>
|
||||
</div>
|
||||
</td>
|
||||
@ -149,7 +153,7 @@
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0039__en-us_topic_0108604188_row15485153518397"><td class="cellrowborder" valign="top" width="21.740000000000002%" headers="mcps1.3.3.3.2.4.1.1 "><p id="dataartsstudio_01_0039__en-us_topic_0108604188_p1562410194013">Authentication Method</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="59.64%" headers="mcps1.3.3.3.2.4.1.2 "><div class="p" id="dataartsstudio_01_0039__en-us_topic_0108604188_p128501224175618">Authentication method used for accessing the cluster:<ul id="dataartsstudio_01_0039__ul154428246403"><li id="dataartsstudio_01_0039__li19442102414409"><strong id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108275286_b101916542026_1">SIMPLE</strong>: Select this for non-security mode.</li><li id="dataartsstudio_01_0039__li15442112419400"><strong id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108275286_b1615131211222_1">KERBEROS</strong>: Select this for security mode.</li></ul>
|
||||
<td class="cellrowborder" valign="top" width="59.64%" headers="mcps1.3.3.3.2.4.1.2 "><div class="p" id="dataartsstudio_01_0039__en-us_topic_0108604188_p128501224175618">Authentication method used for accessing the cluster:<ul id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108275286_ul193521539183819_1"><li id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108275286_li11352039203811_1"><strong id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108275286_b101916542026_1">SIMPLE</strong>: Select this for non-security mode.</li><li id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108275286_li53521239113811_1"><strong id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108275286_b1615131211222_1">KERBEROS</strong>: Select this for security mode.</li></ul>
|
||||
</div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="18.62%" headers="mcps1.3.3.3.2.4.1.3 "><p id="dataartsstudio_01_0039__en-us_topic_0108604188_p106241018407">Kerberos</p>
|
||||
@ -219,7 +223,7 @@
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0039__en-us_topic_0108604188_row49250918352"><td class="cellrowborder" valign="top" width="20.4%" headers="mcps1.3.4.3.2.4.1.1 "><p id="dataartsstudio_01_0039__en-us_topic_0108604188_p189268910357">Authentication Method</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="56.86%" headers="mcps1.3.4.3.2.4.1.2 "><div class="p" id="dataartsstudio_01_0039__en-us_topic_0108604188_p106241129145614">Authentication method used for accessing the cluster:<ul id="dataartsstudio_01_0039__ul16445172414012"><li id="dataartsstudio_01_0039__li1144514240408"><strong id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108275286_b101916542026_2">SIMPLE</strong>: Select this for non-security mode.</li><li id="dataartsstudio_01_0039__li044592414407"><strong id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108275286_b1615131211222_2">KERBEROS</strong>: Select this for security mode.</li></ul>
|
||||
<td class="cellrowborder" valign="top" width="56.86%" headers="mcps1.3.4.3.2.4.1.2 "><div class="p" id="dataartsstudio_01_0039__en-us_topic_0108604188_p106241129145614">Authentication method used for accessing the cluster:<ul id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108275286_ul193521539183819_2"><li id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108275286_li11352039203811_2"><strong id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108275286_b101916542026_2">SIMPLE</strong>: Select this for non-security mode.</li><li id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108275286_li53521239113811_2"><strong id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108275286_b1615131211222_2">KERBEROS</strong>: Select this for security mode.</li></ul>
|
||||
</div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="22.74%" headers="mcps1.3.4.3.2.4.1.3 "><p id="dataartsstudio_01_0039__en-us_topic_0108604188_p109305910356">Kerberos</p>
|
||||
@ -244,7 +248,7 @@
|
||||
<tr id="dataartsstudio_01_0039__en-us_topic_0108604188_row71073549237"><td class="cellrowborder" valign="top" width="20.4%" headers="mcps1.3.4.3.2.4.1.1 "><p id="dataartsstudio_01_0039__en-us_topic_0108604188_p174517362557">Run Mode</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="56.86%" headers="mcps1.3.4.3.2.4.1.2 "><p id="dataartsstudio_01_0039__en-us_topic_0108604188_p693111135543">Run mode of the HBase link. This parameter is used only when the HBase version is <strong id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108604188_b13390124110407_1">HBASE_2_X</strong>.</p>
|
||||
<ul id="dataartsstudio_01_0039__en-us_topic_0108604188_ul1358073519445"><li id="dataartsstudio_01_0039__li144616244406"><strong id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108618545_b1095020874814_1">EMBEDDED</strong>: The link instance runs with CDM. This mode delivers better performance.</li><li id="dataartsstudio_01_0039__li1446102474019"><strong id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108618545_b1353409697105248_1">Standalone</strong>: The link instance runs in an independent process. If CDM needs to connect to multiple Hadoop data sources (MRS, Hadoop, or CloudTable) with both Kerberos and Simple authentication modes, <strong id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108618545_b9202184312114_1">Standalone</strong> prevails.<div class="note" id="dataartsstudio_01_0039__note13446324134018"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0039__p14461824164015">The <strong id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108618545_b1798836155817_1">STANDALONE</strong> mode is used to solve the version conflict problem. If the connector versions of the source and destination ends of the same link are different, a JAR file conflict occurs. In this case, you need to place the source or destination end in the STANDALONE process to prevent the migration failure caused by the conflict.</p>
|
||||
<ul id="dataartsstudio_01_0039__en-us_topic_0108604188_ul1358073519445"><li id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108618545_li188444317283_1"><strong id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108618545_b1095020874814_1">EMBEDDED</strong>: The link instance runs with CDM. This mode delivers better performance.</li><li id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108618545_li118451236288_1"><strong id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108618545_b1353409697105248_1">Standalone</strong>: The link instance runs in an independent process. If CDM needs to connect to multiple Hadoop data sources (MRS, Hadoop, or CloudTable) with both Kerberos and Simple authentication modes, <strong id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108618545_b9202184312114_1">Standalone</strong> prevails.<div class="note" id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108618545_note169619581157_1"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108618545_p1184511312281_1">The <strong id="dataartsstudio_01_0039__en-us_topic_0108604188_en-us_topic_0108618545_b1798836155817_1">STANDALONE</strong> mode is used to solve the version conflict problem. If the connector versions of the source and destination ends of the same link are different, a JAR file conflict occurs. In this case, you need to place the source or destination end in the STANDALONE process to prevent the migration failure caused by the conflict.</p>
|
||||
</div></div>
|
||||
</li></ul>
|
||||
</td>
|
||||
@ -274,7 +278,7 @@
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0023.html">Managing Links</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0157.html">Configuring Link Parameters</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
@ -1,12 +1,16 @@
|
||||
<a name="dataartsstudio_01_0040"></a><a name="dataartsstudio_01_0040"></a>
|
||||
|
||||
<h1 class="topictitle1">Link to HDFS</h1>
|
||||
<h1 class="topictitle1">HDFS Link Parameters</h1>
|
||||
<div id="body8662426"><div class="p" id="dataartsstudio_01_0040__en-us_topic_0108275286_p396911259314">CDM supports the following HDFS data sources:<ul id="dataartsstudio_01_0040__en-us_topic_0108275286_ul1998193319335"><li id="dataartsstudio_01_0040__en-us_topic_0108275286_li189813313319"><a href="#dataartsstudio_01_0040__en-us_topic_0108275286_section127371747103314">MRS HDFS</a></li><li id="dataartsstudio_01_0040__en-us_topic_0108275286_li46813410343"><a href="#dataartsstudio_01_0040__en-us_topic_0108275286_section1851831133416">FusionInsight HDFS</a></li><li id="dataartsstudio_01_0040__en-us_topic_0108275286_li88038432344"><a href="#dataartsstudio_01_0040__en-us_topic_0108275286_section690499173512">Apache HDFS</a><div class="note" id="dataartsstudio_01_0040__en-us_topic_0108275286_note8867140110"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108275292_p167832113185">Do not change the password or user when the job is running. If you do so, the password will not take effect immediately and the job will fail.</p>
|
||||
</div></div>
|
||||
</li></ul>
|
||||
</div>
|
||||
<div class="section" id="dataartsstudio_01_0040__en-us_topic_0108275286_section127371747103314"><a name="dataartsstudio_01_0040__en-us_topic_0108275286_section127371747103314"></a><a name="en-us_topic_0108275286_section127371747103314"></a><h4 class="sectiontitle">MRS HDFS</h4><p id="dataartsstudio_01_0040__en-us_topic_0108275286_p124412208341">When connecting CDM to HDFS of MRS, configure the parameters as described in <a href="#dataartsstudio_01_0040__en-us_topic_0108275286_table6441152003419">Table 1</a>.</p>
|
||||
<div class="note" id="dataartsstudio_01_0040__en-us_topic_0108275286_note5285102815136"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_ul268415325415"><li id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_li116841636546">Before creating an MRS link, you need to add an authenticated Kerberos user on MRS and log in to the MRS management page to change the initial password. Then use the new user to create an MRS link.</li><li id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_li36846319542">To connect to an MRS 2.<em id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_i1155152111254">x</em> cluster, create a CDM cluster of version 2.<em id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_i115512112257">x</em> first. CDM 1.8.<em id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_i0707423162514">x</em> clusters cannot connect to MRS 2.<em id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_i0707423142520">x</em> clusters.</li><li id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_li132471615165418">If the connection fails after you select a cluster, check whether the MRS cluster can communicate with the CDM instance which functions as the agent. They can communicate with each other in the following scenarios:<ul id="dataartsstudio_01_0040__ul9471834174919"><li id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_en-us_topic_0000001503606428_li17205293217">If the CDM cluster in the <span id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_en-us_topic_0000001503606428_text15319922125112">DataArts Studio</span> instance and the MRS cluster are in different regions, a public network or a dedicated connection is required. If the Internet is used for communication, ensure that an EIP has been bound to the CDM cluster, and the MRS cluster can access the Internet and the port has been enabled in the firewall rule.</li><li id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_en-us_topic_0000001503606428_li13205893215">If the CDM cluster in the <span id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_en-us_topic_0000001503606428_text867019165375">DataArts Studio</span> instance and the cloud service are in the same region, VPC, subnet, and security group, they can communicate with each other by default. If they are in the same VPC but in different subnets or security groups, you must configure routing rules and security group rules. For details about how to configure routing rules, see "Adding a Custom Route" in <em id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_en-us_topic_0000001503606428_i1267271614375">Virtual Private Cloud (VPC) Usage Guide</em>. For details about how to configure security group rules, see "Security Group" > "Adding a Security Group Rule" in <em id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_en-us_topic_0000001503606428_i15675816103713">Virtual Private Cloud (VPC) Usage Guide</em>.</li><li id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_en-us_topic_0000001503606428_li20156162311283">The MRS cluster and the <span id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_en-us_topic_0000001503606428_text4156132310284">DataArts Studio</span> workspace belong to the same enterprise project. If they do not, you can modify the enterprise project of the workspace.</li></ul>
|
||||
<div class="note" id="dataartsstudio_01_0040__en-us_topic_0108275286_note5285102815136"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_ul268415325415"><li id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_li3509922112118"><p id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_en-us_topic_0000001624214396_p1463312815216"><a name="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_li3509922112118"></a><a name="en-us_topic_0108275286_en-us_topic_0108604188_li3509922112118"></a>MRS clusters whose Kerberos encryption type is <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_en-us_topic_0000001624214396_b742132751912">aes256-sha2,aes128-sha2</strong> are not supported, and only MRS clusters whose Kerberos encryption type is <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_en-us_topic_0000001624214396_b1977394971913">aes256-sha1,aes128-sha1</strong> are supported.</p>
|
||||
</li><li id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_li116841636546">Before creating an MRS link, you need to add an authenticated Kerberos user on MRS and log in to the MRS management page to change the initial password. Then use the new user to create an MRS link.</li><li id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_li36846319542">To connect to an MRS 2.<em id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_i1155152111254">x</em> cluster, create a CDM cluster of version 2.<em id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_i115512112257">x</em> first. CDM 1.8.<em id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_i0707423162514">x</em> clusters cannot connect to MRS 2.<em id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_i0707423142520">x</em> clusters.</li><li id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_li132471615165418"><div class="p" id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_p134241819125419"><a name="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_li132471615165418"></a><a name="en-us_topic_0108275286_en-us_topic_0108604188_li132471615165418"></a>If the connection fails after you select a cluster, check whether the MRS cluster can communicate with the CDM instance which functions as the agent. They can communicate with each other in the following scenarios:<ul id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_en-us_topic_0000001503606428_ul1715617232281"><li id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_en-us_topic_0000001503606428_li17205293217">If the CDM cluster in the <span id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_en-us_topic_0000001503606428_text15319922125112">DataArts Studio</span> instance and the MRS cluster are in different regions, a public network or a dedicated connection is required. If the Internet is used for communication, ensure that an EIP has been bound to the CDM cluster, and the MRS cluster can access the Internet and the port has been enabled in the firewall rule.</li><li id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_en-us_topic_0000001503606428_li13205893215">If the CDM cluster in the <span id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_en-us_topic_0000001503606428_text151663810818">DataArts Studio</span> instance and the MRS cluster are in the same region, VPC, subnet, and security group, they can communicate with each other by default. If they are in the same VPC but in different subnets or security groups, you must configure routing rules and security group rules. For details about how to configure routing rules, see "Adding Routes to a Route Table" in <em id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_en-us_topic_0000001503606428_i1317838485">Virtual Private Cloud (VPC) x.x.x User Guide</em> in <em id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_en-us_topic_0000001503606428_i13171138184">Virtual Private Cloud (VPC) x.x.x Usage Guide</em>. For details about how to configure security group rules, see "Security Group" > "Adding a Security Group Rule" in Virtual Private Cloud (VPC) x.x.x User Guide in Virtual Private Cloud (VPC) x.x.x Usage Guide.</li><li id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_en-us_topic_0000001503606428_li20156162311283">The MRS cluster and the <span id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_en-us_topic_0000001503606428_text4156132310284">DataArts Studio</span> workspace belong to the same enterprise project. If they do not, you can modify the enterprise project of the workspace.</li></ul>
|
||||
<div class="note" id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_en-us_topic_0000001503606428_note030793314500"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108604188_en-us_topic_0000001503606428_p130813395015">If an agent is connected to multiple MRS clusters and one of the MRS clusters is deleted or abnormal, connections to the other MRS clusters will be affected. Therefore, you are advised to connect an agent to only one MRS cluster.</p>
|
||||
</div></div>
|
||||
</div>
|
||||
</li></ul>
|
||||
</div></div>
|
||||
|
||||
@ -27,7 +31,7 @@
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0040__en-us_topic_0108275286_row2456122053415"><td class="cellrowborder" valign="top" width="20.51%" headers="mcps1.3.2.4.2.4.1.1 "><p id="dataartsstudio_01_0040__en-us_topic_0108275286_p13456172019344">Manager IP</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="57.720000000000006%" headers="mcps1.3.2.4.2.4.1.2 "><div class="p" id="dataartsstudio_01_0040__en-us_topic_0108275286_p1345602010341">Floating IP address of MRS Manager. Click <span class="uicontrol" id="dataartsstudio_01_0040__en-us_topic_0108275286_uicontrol578864992012"><b>Select</b></span> next to the <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b2078834920203">Manager IP</strong> text box to select an MRS cluster. CDM automatically fills in the authentication information.<div class="note" id="dataartsstudio_01_0040__note1340014473413"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0040__en-us_topic_0182566327_p116249795810"><span id="dataartsstudio_01_0040__en-us_topic_0182566327_text7355181825814">DataArts Studio</span> does not support MRS clusters whose Kerberos encryption type is <strong id="dataartsstudio_01_0040__en-us_topic_0182566327_b143479402717">aes256-sha2,aes128-sha2</strong>, and only supports MRS clusters whose Kerberos encryption type is <strong id="dataartsstudio_01_0040__en-us_topic_0182566327_b19532103219813">aes256-sha1,aes128-sha1</strong>.</p>
|
||||
<td class="cellrowborder" valign="top" width="57.720000000000006%" headers="mcps1.3.2.4.2.4.1.2 "><div class="p" id="dataartsstudio_01_0040__en-us_topic_0108275286_p1345602010341">Floating IP address of MRS Manager. Click <span class="uicontrol" id="dataartsstudio_01_0040__en-us_topic_0108275286_uicontrol578864992012"><b>Select</b></span> next to the <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b2078834920203">Manager IP</strong> text box to select an MRS cluster. CDM automatically fills in the authentication information.<div class="note" id="dataartsstudio_01_0040__en-us_topic_0108275286_note1340014473413"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0182566327_p116249795810"><span id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0182566327_text7355181825814">DataArts Studio</span> does not support MRS clusters whose Kerberos encryption type is <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0182566327_b143479402717">aes256-sha2,aes128-sha2</strong>, and only supports MRS clusters whose Kerberos encryption type is <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0182566327_b19532103219813">aes256-sha1,aes128-sha1</strong>.</p>
|
||||
</div></div>
|
||||
</div>
|
||||
</td>
|
||||
@ -61,17 +65,17 @@
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0040__en-us_topic_0108275286_row10211145833913"><td class="cellrowborder" valign="top" width="20.51%" headers="mcps1.3.2.4.2.4.1.1 "><p id="dataartsstudio_01_0040__en-us_topic_0108275286_p721185823920">Run Mode</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="57.720000000000006%" headers="mcps1.3.2.4.2.4.1.2 "><div class="p" id="dataartsstudio_01_0040__en-us_topic_0108275286_p206752056145718">Run mode of the HDFS link. The options are as follows:<ul id="dataartsstudio_01_0040__en-us_topic_0108275286_ul1747017597574"><li id="dataartsstudio_01_0040__en-us_topic_0108275286_li6848185317587"><strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b03725315400">EMBEDDED</strong>: The link instance runs with CDM. This mode delivers better performance.</li><li id="dataartsstudio_01_0040__en-us_topic_0108275286_li1261464125915"><strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b1859873319585">STANDALONE</strong>: The link instance runs in an independent process. If CDM needs to connect to multiple Hadoop data sources (MRS, Hadoop, or CloudTable) with both Kerberos and Simple authentication modes, select <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b9202184312114">STANDALONE</strong> or configure different agents.<p id="dataartsstudio_01_0040__en-us_topic_0108275286_p8792217159">Note: The STANDALONE mode is used to solve the version conflict problem. If the connector versions of the source and destination ends of the same link are different, a JAR file conflict occurs. In this case, you need to place the source or destination end in the STANDALONE process to prevent the migration failure caused by the conflict.</p>
|
||||
</li><li id="dataartsstudio_01_0040__en-us_topic_0108275286_li893446123812"><strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b181003145318">Agent</strong>: The link instance runs on an agent.</li></ul>
|
||||
<td class="cellrowborder" valign="top" width="57.720000000000006%" headers="mcps1.3.2.4.2.4.1.2 "><div class="p" id="dataartsstudio_01_0040__en-us_topic_0108275286_p206752056145718">Run mode of the HDFS link. The options are as follows:<ul id="dataartsstudio_01_0040__en-us_topic_0108275286_ul1747017597574"><li id="dataartsstudio_01_0040__en-us_topic_0108275286_li6848185317587"><strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b03725315400">EMBEDDED</strong>: The link instance runs with CDM. This mode delivers better performance.</li><li id="dataartsstudio_01_0040__en-us_topic_0108275286_li1261464125915"><strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b1859873319585">STANDALONE</strong>: The link instance runs in an independent process. If you want to connect CDM to multiple Hadoop data sources (MRS, Hadoop, or CloudTable), and both <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b14923332895">KERBEROS</strong> and <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b119234321899">SIMPLE</strong> authentication modes are available, you must select <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b119235321591">STANDALONE</strong> for this parameter.<p id="dataartsstudio_01_0040__en-us_topic_0108275286_p8792217159">Note: The STANDALONE mode is used to solve the version conflict problem. If the connector versions of the source and destination ends of the same link are different, a JAR file conflict occurs. In this case, you need to place the source or destination end in the STANDALONE process to prevent the migration failure caused by the conflict.</p>
|
||||
</li></ul>
|
||||
</div>
|
||||
<p id="dataartsstudio_01_0040__en-us_topic_0108275286_p192253613346">If <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b36013583118">Agent</strong> is not used, and the CDM cluster connects to two or more clusters with Kerberos authentication enabled and the same realm, only one cluster can be connected in <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b17602145812113">EMBEDDED</strong> mode, and the other clusters must be in <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b1460355817113">STANDALONE</strong> mode.</p>
|
||||
<p id="dataartsstudio_01_0040__en-us_topic_0108275286_p192253613346">If a CDM cluster connects to two or more clusters with Kerberos authentication enabled and the same realm, only one cluster can be connected in <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b114811532911">EMBEDDED</strong> mode, and the other clusters must be connected in <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b13148353791">STANDALONE</strong> mode.</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="21.77%" headers="mcps1.3.2.4.2.4.1.3 "><p id="dataartsstudio_01_0040__en-us_topic_0108275286_p1021155811394">STANDALONE</p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0040__en-us_topic_0108275286_row1946581311414"><td class="cellrowborder" valign="top" width="20.51%" headers="mcps1.3.2.4.2.4.1.1 "><p id="dataartsstudio_01_0040__en-us_topic_0108275286_p1999031319386">Agent</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="57.720000000000006%" headers="mcps1.3.2.4.2.4.1.2 "><p id="dataartsstudio_01_0040__en-us_topic_0108275286_p5412193413515">Click <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b64938181244">Select</strong> and select the agent created in <a href="dataartsstudio_01_0128.html#dataartsstudio_01_0128__en-us_topic_0207402273_en-us_topic_0191978474_section1072083564713">Connecting to an Agent</a>. This parameter is displayed when <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b25494136401">Run Mode</strong> is set to <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b1081748104013">Agent</strong>.</p>
|
||||
<td class="cellrowborder" valign="top" width="57.720000000000006%" headers="mcps1.3.2.4.2.4.1.2 "><p id="dataartsstudio_01_0040__en-us_topic_0108275286_p1651731719256"><span id="dataartsstudio_01_0040__en-us_topic_0108275286_text79507435388">The agent function will be unavailable soon and does not need to be configured.</span></p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="21.77%" headers="mcps1.3.2.4.2.4.1.3 "><p id="dataartsstudio_01_0040__en-us_topic_0108275286_p1499013137381">-</p>
|
||||
</td>
|
||||
@ -159,8 +163,8 @@
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0040__en-us_topic_0108275286_row129026915554"><td class="cellrowborder" valign="top" width="20.74%" headers="mcps1.3.3.3.2.4.1.1 "><p id="dataartsstudio_01_0040__en-us_topic_0108275286_p1418522311554">Run Mode</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="57.97%" headers="mcps1.3.3.3.2.4.1.2 "><div class="p" id="dataartsstudio_01_0040__en-us_topic_0108275286_p1990239195518">Run mode of the HDFS link. The options are as follows:<ul id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108275286_ul1747017597574"><li id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108275286_li6848185317587"><strong id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108275286_b03725315400">EMBEDDED</strong>: The link instance runs with CDM. This mode delivers better performance.</li><li id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108275286_li1261464125915"><strong id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108275286_b1859873319585">STANDALONE</strong>: The link instance runs in an independent process. If CDM needs to connect to multiple Hadoop data sources (MRS, Hadoop, or CloudTable) with both Kerberos and Simple authentication modes, select <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108275286_b9202184312114">STANDALONE</strong> or configure different agents.<p id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108275286_p8792217159">Note: The STANDALONE mode is used to solve the version conflict problem. If the connector versions of the source and destination ends of the same link are different, a JAR file conflict occurs. In this case, you need to place the source or destination end in the STANDALONE process to prevent the migration failure caused by the conflict.</p>
|
||||
</li><li id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108275286_li893446123812"><strong id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108275286_b181003145318">Agent</strong>: The link instance runs on an agent.</li></ul>
|
||||
<td class="cellrowborder" valign="top" width="57.97%" headers="mcps1.3.3.3.2.4.1.2 "><div class="p" id="dataartsstudio_01_0040__en-us_topic_0108275286_p1990239195518">Run mode of the HDFS link. The options are as follows:<ul id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108275286_ul1747017597574"><li id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108275286_li6848185317587"><strong id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108275286_b03725315400">EMBEDDED</strong>: The link instance runs with CDM. This mode delivers better performance.</li><li id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108275286_li1261464125915"><strong id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108275286_b1859873319585">STANDALONE</strong>: The link instance runs in an independent process. If you want to connect CDM to multiple Hadoop data sources (MRS, Hadoop, or CloudTable), and both <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108275286_b14923332895">KERBEROS</strong> and <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108275286_b119234321899">SIMPLE</strong> authentication modes are available, you must select <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108275286_b119235321591">STANDALONE</strong> for this parameter.<p id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108275286_p8792217159">Note: The STANDALONE mode is used to solve the version conflict problem. If the connector versions of the source and destination ends of the same link are different, a JAR file conflict occurs. In this case, you need to place the source or destination end in the STANDALONE process to prevent the migration failure caused by the conflict.</p>
|
||||
</li></ul>
|
||||
</div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="21.29%" headers="mcps1.3.3.3.2.4.1.3 "><p id="dataartsstudio_01_0040__en-us_topic_0108275286_p18902892555">STANDALONE</p>
|
||||
@ -168,7 +172,7 @@
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0040__en-us_topic_0108275286_row5249204817429"><td class="cellrowborder" valign="top" width="20.74%" headers="mcps1.3.3.3.2.4.1.1 "><p id="dataartsstudio_01_0040__en-us_topic_0108275286_p10250104844219">Agent</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="57.97%" headers="mcps1.3.3.3.2.4.1.2 "><p id="dataartsstudio_01_0040__en-us_topic_0108275286_p1425064819422">Click <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108275286_b64938181244">Select</strong> and select the agent created in <a href="dataartsstudio_01_0128.html#dataartsstudio_01_0128__en-us_topic_0207402273_en-us_topic_0191978474_section1072083564713">Connecting to an Agent</a>. This parameter is displayed when <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108275286_b25494136401">Run Mode</strong> is set to <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_en-us_topic_0108275286_b1081748104013">Agent</strong>.</p>
|
||||
<td class="cellrowborder" valign="top" width="57.97%" headers="mcps1.3.3.3.2.4.1.2 "><p id="dataartsstudio_01_0040__en-us_topic_0108275286_p938620154211"><span id="dataartsstudio_01_0040__en-us_topic_0108275286_text123812014213">The agent function will be unavailable soon and does not need to be configured.</span></p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="21.29%" headers="mcps1.3.3.3.2.4.1.3 "><p id="dataartsstudio_01_0040__en-us_topic_0108275286_p13250648124212">-</p>
|
||||
</td>
|
||||
@ -227,8 +231,8 @@
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0040__en-us_topic_0108275286_row693151312543"><td class="cellrowborder" valign="top" width="20.96%" headers="mcps1.3.4.3.2.4.1.1 "><p id="dataartsstudio_01_0040__en-us_topic_0108275286_p174517362557">Run Mode</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="58.08%" headers="mcps1.3.4.3.2.4.1.2 "><div class="p" id="dataartsstudio_01_0040__en-us_topic_0108275286_p693111135543">Run mode of the HDFS link. The options are as follows:<ul id="dataartsstudio_01_0040__en-us_topic_0108275286_ul118812510251"><li id="dataartsstudio_01_0040__en-us_topic_0108275286_li88811453254"><strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b1130034924">EMBEDDED</strong>: The link instance runs with CDM. This mode delivers better performance.</li><li id="dataartsstudio_01_0040__en-us_topic_0108275286_li1688117511256"><strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b174476832">STANDALONE</strong>: The link instance runs in an independent process. If CDM needs to connect to multiple Hadoop data sources (MRS, Hadoop, or CloudTable) with both Kerberos and Simple authentication modes, select <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b525219538">STANDALONE</strong> or configure different agents.<p id="dataartsstudio_01_0040__en-us_topic_0108275286_p17881145172520">Note: The STANDALONE mode is used to solve the version conflict problem. If the connector versions of the source and destination ends of the same link are different, a JAR file conflict occurs. In this case, you need to place the source or destination end in the STANDALONE process to prevent the migration failure caused by the conflict.</p>
|
||||
</li><li id="dataartsstudio_01_0040__en-us_topic_0108275286_li10881125162514"><strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b174245713138">Agent</strong>: The link instance runs on an agent. For Apache HDFS, you can select <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b36952016141417">Agent</strong> only if <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b109381449141416">Authentication Method</strong> is set to <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b777195813149">SIMPLE</strong>.</li></ul>
|
||||
<td class="cellrowborder" valign="top" width="58.08%" headers="mcps1.3.4.3.2.4.1.2 "><div class="p" id="dataartsstudio_01_0040__en-us_topic_0108275286_p693111135543">Run mode of the HDFS link. The options are as follows:<ul id="dataartsstudio_01_0040__en-us_topic_0108275286_ul118812510251"><li id="dataartsstudio_01_0040__en-us_topic_0108275286_li88811453254"><strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b1097960987">EMBEDDED</strong>: The link instance runs with CDM. This mode delivers better performance.</li><li id="dataartsstudio_01_0040__en-us_topic_0108275286_li1688117511256"><strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b877378848">STANDALONE</strong>: The link instance runs in an independent process. If you want to connect CDM to multiple Hadoop data sources (MRS, Hadoop, or CloudTable), and both <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b116412551913">KERBEROS</strong> and <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b15164155513917">SIMPLE</strong> authentication modes are available, you must select <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b616419552919">STANDALONE</strong> for this parameter.<p id="dataartsstudio_01_0040__en-us_topic_0108275286_p17881145172520">Note: The STANDALONE mode is used to solve the version conflict problem. If the connector versions of the source and destination ends of the same link are different, a JAR file conflict occurs. In this case, you need to place the source or destination end in the STANDALONE process to prevent the migration failure caused by the conflict.</p>
|
||||
</li></ul>
|
||||
</div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="20.96%" headers="mcps1.3.4.3.2.4.1.3 "><p id="dataartsstudio_01_0040__en-us_topic_0108275286_p15931141318544">STANDALONE</p>
|
||||
@ -245,7 +249,7 @@
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0040__en-us_topic_0108275286_row13173134416"><td class="cellrowborder" valign="top" width="20.96%" headers="mcps1.3.4.3.2.4.1.1 "><p id="dataartsstudio_01_0040__en-us_topic_0108275286_p20303117444">Agent</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="58.08%" headers="mcps1.3.4.3.2.4.1.2 "><p id="dataartsstudio_01_0040__en-us_topic_0108275286_p461134915213">This parameter is required when <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b91971643153316">Authentication Method</strong> is set to <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b1069295119333">SIMPLE</strong> and <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b728710263410">Run Mode</strong> is set to <strong id="dataartsstudio_01_0040__en-us_topic_0108275286_b1672531316346">Agent</strong>. Select the agent created in <a href="dataartsstudio_01_0128.html#dataartsstudio_01_0128__en-us_topic_0207402273_en-us_topic_0191978474_section1072083564713">Connecting to an Agent</a>.</p>
|
||||
<td class="cellrowborder" valign="top" width="58.08%" headers="mcps1.3.4.3.2.4.1.2 "><p id="dataartsstudio_01_0040__en-us_topic_0108275286_p201134017433"><span id="dataartsstudio_01_0040__en-us_topic_0108275286_text1611360164310">The agent function will be unavailable soon and does not need to be configured.</span></p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="20.96%" headers="mcps1.3.4.3.2.4.1.3 "><p id="dataartsstudio_01_0040__en-us_topic_0108275286_p123193116445">-</p>
|
||||
</td>
|
||||
@ -272,7 +276,7 @@
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0023.html">Managing Links</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0157.html">Configuring Link Parameters</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
@ -1,7 +1,7 @@
|
||||
<a name="dataartsstudio_01_0044"></a><a name="dataartsstudio_01_0044"></a>
|
||||
|
||||
<h1 class="topictitle1">Link to PostgreSQL/SQLServer</h1>
|
||||
<div id="body8662426"><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p18117455259"><a href="#dataartsstudio_01_0044__en-us_topic_0108275292_table5321744015490">Table 1</a> lists the parameters for creating a link to PostgreSQL/SQLServer.</p>
|
||||
<h1 class="topictitle1">PostgreSQL/SQLServer Link Parameters</h1>
|
||||
<div id="body8662426"><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p18117455259"><a href="#dataartsstudio_01_0044__en-us_topic_0108275292_table5321744015490">Table 1</a> lists the parameters for creating a link to PostgreSQL/SQLServer. Greenplum, Kingbase, and GaussDB can be connected through the PostgreSQL connector. The source and destination data sources supported by migration jobs are the same as those for PostgreSQL.</p>
|
||||
<div class="note" id="dataartsstudio_01_0044__en-us_topic_0108275292_note36461362017"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p167832113185">Do not change the password or user when the job is running. If you do so, the password will not take effect immediately and the job will fail.</p>
|
||||
</div></div>
|
||||
|
||||
@ -60,16 +60,31 @@
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0044__en-us_topic_0108275292_row15561131344619"><td class="cellrowborder" valign="top" width="16.5%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p815422212356">Use Agent</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="61.550000000000004%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p1615418228357">Whether to extract data from the data source through an agent</p>
|
||||
<td class="cellrowborder" valign="top" width="61.550000000000004%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p1651731719256"><span id="dataartsstudio_01_0044__en-us_topic_0108275292_text79507435388">The agent function will be unavailable soon and does not need to be configured.</span></p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="21.95%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p1415442293513">Yes</p>
|
||||
<td class="cellrowborder" valign="top" width="21.95%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p4141154315252">-</p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0044__en-us_topic_0108275292_row19539167464"><td class="cellrowborder" valign="top" width="16.5%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p18412133418352">Agent</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="61.550000000000004%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p5412193413515">Click <strong id="dataartsstudio_01_0044__en-us_topic_0108275292_b17397163613819">Select</strong> and select the agent created in <a href="dataartsstudio_01_0128.html">Managing Agents</a>.</p>
|
||||
<td class="cellrowborder" valign="top" width="61.550000000000004%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p179409111394"><span id="dataartsstudio_01_0044__en-us_topic_0108275292_text1940816391">The agent function will be unavailable soon and does not need to be configured.</span></p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="21.95%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p1241253413354">-</p>
|
||||
<td class="cellrowborder" valign="top" width="21.95%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p1570915504257">-</p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0044__en-us_topic_0108275292_row8884162619471"><td class="cellrowborder" valign="top" width="16.5%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p17885182674711">Driver Class Name</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="61.550000000000004%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p4827321125813">Class name of the uploaded driver</p>
|
||||
<p id="dataartsstudio_01_0044__en-us_topic_0108275292_p138851626184713">Select <strong id="dataartsstudio_01_0044__en-us_topic_0108275292_b940112541176">org.postgresql.Driver</strong> or <strong id="dataartsstudio_01_0044__en-us_topic_0108275292_b233171489">com.kingbase8.Driver</strong>.</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="21.95%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p788552614713">-</p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0044__en-us_topic_0108275292_row5943122520489"><td class="cellrowborder" valign="top" width="16.5%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p237624617348">Reference Sign</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="61.550000000000004%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p5825824617348">(Optional) Delimiter between the names of the referenced tables or columns. For details, see the product documentation of the corresponding database.</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="21.95%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p2129746617348">"</p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0044__en-us_topic_0108275292_row1983251620289"><td class="cellrowborder" valign="top" width="16.5%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p1483391622812">Driver Version</p>
|
||||
@ -87,20 +102,36 @@
|
||||
<td class="cellrowborder" valign="top" width="21.95%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p4890496717340">1000</p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0044__en-us_topic_0108275292_row1837555074812"><td class="cellrowborder" valign="top" width="16.5%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p1989555882818">Commit Size</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="61.550000000000004%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p1792313318298">(Optional) Displayed when you click <strong id="dataartsstudio_01_0044__en-us_topic_0108275292_b6203026665952">Show Advanced Attributes</strong>.</p>
|
||||
<p id="dataartsstudio_01_0044__en-us_topic_0108275292_p13923635293">Number of records submitted each time. Set this parameter based on the destination and data size of the job. If the value is too large or too small, the job execution time may be affected.</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="21.95%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p18345554102810">10000</p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0044__en-us_topic_0108275292_row8648150174913"><td class="cellrowborder" valign="top" width="16.5%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p46177674141711">SSL Encryption</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="61.550000000000004%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p1472085442710">Whether to connect to the database in SSL mode</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="21.95%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p42313489141711">Yes</p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0044__en-us_topic_0108275292_row6497846117345"><td class="cellrowborder" valign="top" width="16.5%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p2876397817345">Link Attributes</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="61.550000000000004%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p4818086317345">(Optional) Click <span class="uicontrol" id="dataartsstudio_01_0044__en-us_topic_0108275292_uicontrol143481412163910"><b>Add</b></span> to add the JDBC connector attributes of multiple specified data sources. For details, see the JDBC connector document of the corresponding database.</p>
|
||||
<div class="p" id="dataartsstudio_01_0044__en-us_topic_0108275292_p3176513115010">The following are some examples:<ul id="dataartsstudio_01_0044__en-us_topic_0108275292_ul1284813922912"><li id="dataartsstudio_01_0044__en-us_topic_0108275292_li67681857202918"><strong id="dataartsstudio_01_0044__en-us_topic_0108275292_b777456143615">connectTimeout=60</strong> and <strong id="dataartsstudio_01_0044__en-us_topic_0108275292_b99571823616">socketTimeout=300</strong>: When a large amount of data needs to be migrated or the entire table is retrieved using query statements, the migration fails due to connection timeout. In this case, you can customize the connection timeout interval (s) and socket timeout interval (s) to prevent failures caused by timeout.</li><li id="dataartsstudio_01_0044__en-us_topic_0108275292_li836212252319"><strong id="dataartsstudio_01_0044__en-us_topic_0108275292_b0417150361">useCursorFetch=false</strong>: By default, <strong id="dataartsstudio_01_0044__en-us_topic_0108275292_b732421619483">useCursorFetch</strong> is enabled, indicating that the JDBC connector communicates with relational databases using a binary protocol. Some third-party systems may have compatibility issues, causing migration time conversion errors. In this case, you can disable this function.</li><li id="dataartsstudio_01_0044__en-us_topic_0108275292_li9948037164220"><strong id="dataartsstudio_01_0044__en-us_topic_0108275292_b147221356437">trustServerCertificate=true</strong>: A PKIX error may be reported during the creation of a secure connection. You are advised to set this parameter to <strong id="dataartsstudio_01_0044__en-us_topic_0108275292_b7836144363212">true</strong>.</li></ul>
|
||||
<div class="p" id="dataartsstudio_01_0044__en-us_topic_0108275292_p3176513115010">The following are some examples:<ul id="dataartsstudio_01_0044__en-us_topic_0108275292_ul1284813922912"><li id="dataartsstudio_01_0044__en-us_topic_0108275292_li67681857202918"><strong id="dataartsstudio_01_0044__en-us_topic_0108275292_b777456143615">connectTimeout=60</strong> and <strong id="dataartsstudio_01_0044__en-us_topic_0108275292_b99571823616">socketTimeout=300</strong>: When a large amount of data needs to be migrated or the entire table is retrieved using query statements, the migration fails due to connection timeout. In this case, you can customize the connection timeout interval (s) and socket timeout interval (s) to prevent failures caused by timeout.</li><li id="dataartsstudio_01_0044__en-us_topic_0108275292_li836212252319"><strong id="dataartsstudio_01_0044__en-us_topic_0108275292_b0417150361">useCursorFetch=false</strong>: By default, <strong id="dataartsstudio_01_0044__en-us_topic_0108275292_b732421619483">useCursorFetch</strong> is enabled, indicating that the JDBC connector communicates with relational databases using a binary protocol. Some third-party systems may have compatibility issues, causing migration time conversion errors. In this case, you can disable this function.</li><li id="dataartsstudio_01_0044__en-us_topic_0108275292_li9948037164220"><strong id="dataartsstudio_01_0044__en-us_topic_0108275292_b147221356437">trustServerCertificate=true</strong>: A PKIX error may be reported during the creation of a secure connection. You are advised to set this parameter to <strong id="dataartsstudio_01_0044__en-us_topic_0108275292_b7836144363212">true</strong>.</li><li id="dataartsstudio_01_0044__en-us_topic_0108275292_li1844193243612"><strong id="dataartsstudio_01_0044__en-us_topic_0108275292_b15258154317367">sslmode=require</strong>: The link to PostgreSQL may fail when SSL authentication is enabled. Set this parameter to <strong id="dataartsstudio_01_0044__en-us_topic_0108275292_b951815332219">require</strong>.</li></ul>
|
||||
</div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="21.95%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p1033584517345">sslmode=require</p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0044__en-us_topic_0108275292_row5885315517348"><td class="cellrowborder" valign="top" width="16.5%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p237624617348">Reference Sign</p>
|
||||
<tr id="dataartsstudio_01_0044__en-us_topic_0108275292_row1582665634819"><td class="cellrowborder" valign="top" width="16.5%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p1513318543325">Link Secret Attributes</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="61.550000000000004%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p5825824617348">(Optional) Delimiter between the names of the referenced tables or columns. For details, see the product documentation of the corresponding database.</p>
|
||||
<td class="cellrowborder" valign="top" width="61.550000000000004%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p17107122911366">(Optional) Displayed when you click <strong id="dataartsstudio_01_0044__en-us_topic_0108275292_b12345115718813">Show Advanced Attributes</strong>.</p>
|
||||
<p id="dataartsstudio_01_0044__en-us_topic_0108275292_p11321154183220">Custom secret attributes of the link</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="21.95%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p2129746617348">"</p>
|
||||
<td class="cellrowborder" valign="top" width="21.95%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0044__en-us_topic_0108275292_p927481593614">sk=09fUgD5WOF1L6f</p>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
@ -109,7 +140,7 @@
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0023.html">Managing Links</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0157.html">Configuring Link Parameters</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
<a name="dataartsstudio_01_0045"></a><a name="dataartsstudio_01_0045"></a>
|
||||
|
||||
<h1 class="topictitle1">Link to OBS</h1>
|
||||
<h1 class="topictitle1">OBS Link Parameters</h1>
|
||||
<div id="body8662426"><p id="dataartsstudio_01_0045__en-us_topic_0108275368_p10847147142919">When connecting CDM to the destination OBS bucket, you need to add the read and write permissions to the destination OBS bucket, and file authentication is not required.</p>
|
||||
<div class="note" id="dataartsstudio_01_0045__en-us_topic_0108275368_note18752146121711"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dataartsstudio_01_0045__en-us_topic_0108275368_ul1797192611414"><li id="dataartsstudio_01_0045__en-us_topic_0108275368_li189717265416">If the CDM cluster and OBS bucket are not in the same region, the CDM cluster cannot access the OBS bucket.</li><li id="dataartsstudio_01_0045__en-us_topic_0108275368_li894710289414">Do not change the password or user when the job is running. If you do so, the password will not take effect immediately and the job will fail.</li></ul>
|
||||
</div></div>
|
||||
@ -27,7 +27,7 @@
|
||||
<div class="note" id="dataartsstudio_01_0045__en-us_topic_0108275368_note136619272103"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0108275368_ul1797192611414"><li id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0108275368_li189717265416">If the CDM cluster and OBS bucket are not in the same region, the CDM cluster cannot access the OBS bucket.</li><li id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0108275368_li894710289414">Do not change the password or user when the job is running. If you do so, the password will not take effect immediately and the job will fail.</li></ul>
|
||||
</div></div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="20.630000000000003%" headers="mcps1.3.3.2.2.4.1.3 "><p id="dataartsstudio_01_0045__en-us_topic_0108275368_p89819474596">-</p>
|
||||
<td class="cellrowborder" valign="top" width="20.630000000000003%" headers="mcps1.3.3.2.2.4.1.3 "><p id="dataartsstudio_01_0045__en-us_topic_0108275368_p1671302612411">obs.myregion.mycloud.com</p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0045__en-us_topic_0108275368_row55420632152843"><td class="cellrowborder" valign="top" width="16.5%" headers="mcps1.3.3.2.2.4.1.1 "><p id="dataartsstudio_01_0045__en-us_topic_0108275368_p18947563152843">Port</p>
|
||||
@ -48,7 +48,7 @@
|
||||
</td>
|
||||
<td class="cellrowborder" rowspan="2" valign="top" width="62.870000000000005%" headers="mcps1.3.3.2.2.4.1.2 "><p id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0108275368_p41664200165510">AK and SK are used to log in to the OBS server.</p>
|
||||
<p id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0108275368_p1660302262810">You need to create an access key for the current account and obtain an AK/SK pair.</p>
|
||||
<div class="p" id="dataartsstudio_01_0045__en-us_topic_0108275368_p13152215453">To obtain an access key, perform the following steps:<ol id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_en-us_topic_0183643042_ol535183018588"><li id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_en-us_topic_0183643042_li1535103025819">Log in to the management console, move the cursor to the username in the upper right corner, and select <strong id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_b1586023610364">My Credentials</strong> from the drop-down list.</li><li id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_en-us_topic_0183643042_li173533018584">On the <strong id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_b149721128172418">My Credentials</strong> page, choose <strong id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_b15803656132414">Access Keys</strong>, and click <strong id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_b2452152020258">Create Access Key</strong>. See <a href="#dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615">Figure 1</a>.<div class="fignone" id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615"><a name="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615"></a><a name="en-us_topic_0108275368_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615"></a><span class="figcap"><b>Figure 1 </b>Clicking Create Access Key</span><br><span><img id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_image20389043111611" src="en-us_image_0000002270789428.png"></span></div>
|
||||
<div class="p" id="dataartsstudio_01_0045__en-us_topic_0108275368_p13152215453">To obtain an access key, perform the following steps:<ol id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_en-us_topic_0183643042_ol535183018588"><li id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_en-us_topic_0183643042_li1535103025819">Log in to the management console, move the cursor to the username in the upper right corner, and select <strong id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_b1586023610364">My Credentials</strong> from the drop-down list.</li><li id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_en-us_topic_0183643042_li173533018584">On the <strong id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_b149721128172418">My Credentials</strong> page, choose <strong id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_b15803656132414">Access Keys</strong>, and click <strong id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_b2452152020258">Create Access Key</strong>. See <a href="#dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615">Figure 1</a>.<div class="fignone" id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615"><a name="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615"></a><a name="en-us_topic_0108275368_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615"></a><span class="figcap"><b>Figure 1 </b>Clicking Create Access Key</span><br><span><img id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_image20389043111611" src="en-us_image_0000002269194761.png" title="Click to enlarge" class="imgResize"></span></div>
|
||||
</li><li id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_en-us_topic_0183643042_li1535530185815">Click <strong id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_b8340319122819">OK</strong> and save the access key file as prompted. The access key file will be saved to your browser's configured download location. Open the <strong id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_b1940852512813">credentials.csv</strong> file to view <strong id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_b191574128718">Access Key Id</strong> and <strong id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_b175769149718">Secret Access Key</strong>.<div class="note" id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_note4554158201"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_ul25541081906"><li id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_li175541819018">Only two access keys can be added for each user.</li><li id="dataartsstudio_01_0045__en-us_topic_0108275368_en-us_topic_0000001129241845_li3554138305">To ensure access key security, the access key is automatically downloaded only when it is generated for the first time and cannot be obtained from the management console later. Keep them properly.</li></ul>
|
||||
</div></div>
|
||||
</li></ol>
|
||||
@ -62,6 +62,17 @@
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.2.4.1.2 "><p id="dataartsstudio_01_0045__en-us_topic_0108275368_p12963499163911">-</p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0045__en-us_topic_0108275368_row8667141016424"><td class="cellrowborder" valign="top" width="16.5%" headers="mcps1.3.3.2.2.4.1.1 "><p id="dataartsstudio_01_0045__en-us_topic_0108275368_p1045442813819">Link Attributes</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="62.870000000000005%" headers="mcps1.3.3.2.2.4.1.2 "><p id="dataartsstudio_01_0045__en-us_topic_0108275368_p17107122911366">(Optional) Displayed when you click <strong id="dataartsstudio_01_0045__en-us_topic_0108275368_b1767864611596">Show Advanced Attributes</strong>.</p>
|
||||
<p id="dataartsstudio_01_0045__en-us_topic_0108275368_p1745422833816">You can click <strong id="dataartsstudio_01_0045__en-us_topic_0108275368_b16321251014">Add</strong> to add custom attributes for the link.</p>
|
||||
<p id="dataartsstudio_01_0045__en-us_topic_0108275368_p175581752154217">Only <strong id="dataartsstudio_01_0045__en-us_topic_0108275368_b18361521317">connectionTimeout</strong>, <strong id="dataartsstudio_01_0045__en-us_topic_0108275368_b17544202411116">socketTimeout</strong>, and <strong id="dataartsstudio_01_0045__en-us_topic_0108275368_b104564301316">idleConnectionTime</strong> are supported.</p>
|
||||
<p id="dataartsstudio_01_0045__en-us_topic_0108275368_p6831185816432">The following are some examples:</p>
|
||||
<ul id="dataartsstudio_01_0045__en-us_topic_0108275368_ul194522368447"><li id="dataartsstudio_01_0045__en-us_topic_0108275368_li2452173634419"><strong id="dataartsstudio_01_0045__en-us_topic_0108275368_b127531648223">socketTimeout</strong>: timeout interval for data transmission at the socket layer, in milliseconds</li><li id="dataartsstudio_01_0045__en-us_topic_0108275368_li15452163611448"><strong id="dataartsstudio_01_0045__en-us_topic_0108275368_b221017542045">connectionTimeout</strong>: timeout interval for establishing an HTTP/HTTPS connection, in milliseconds</li></ul>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="20.630000000000003%" headers="mcps1.3.3.2.2.4.1.3 "><p id="dataartsstudio_01_0045__en-us_topic_0108275368_p245419286386">-</p>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
@ -69,7 +80,14 @@
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0023.html">Managing Links</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0157.html">Configuring Link Parameters</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
<script language="JavaScript">
|
||||
<!--
|
||||
initImageViewer('.imgResize');
|
||||
var msg_imageMax = "view original image";
|
||||
var msg_imageClose = "close";
|
||||
//--></script>
|
||||
@ -1,7 +1,7 @@
|
||||
<a name="dataartsstudio_01_004501"></a><a name="dataartsstudio_01_004501"></a>
|
||||
|
||||
<h1 class="topictitle1">Link to Cassandra</h1>
|
||||
<div id="body8662426"><div class="note" id="dataartsstudio_01_004501__en-us_topic_0000001086742916_note8867140110"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_004501__en-us_topic_0000001086742916_en-us_topic_0108275292_p167832113185">Do not change the password or user when the job is running. If you do so, the password will not take effect immediately and the job will fail.</p>
|
||||
<h1 class="topictitle1">Cassandra Link Parameters</h1>
|
||||
<div id="body8662426"><div class="note" id="dataartsstudio_01_004501__en-us_topic_0000001086742916_note8867140110"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dataartsstudio_01_004501__en-us_topic_0000001086742916_ul15986163910193"><li id="dataartsstudio_01_004501__en-us_topic_0000001086742916_li199861139151920">Cassandra is not supported in version 2.9.3.300 or later.</li><li id="dataartsstudio_01_004501__en-us_topic_0000001086742916_li998619395194">Do not change the password or user when a job is running. If you do so, the password will not take effect immediately and the job will fail.</li></ul>
|
||||
</div></div>
|
||||
|
||||
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_01_004501__en-us_topic_0000001086742916_table34037531171418" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="dataartsstudio_01_004501__en-us_topic_0000001086742916_row56630393171418"><th align="left" class="cellrowborder" valign="top" width="19.18%" id="mcps1.3.2.2.4.1.1"><p id="dataartsstudio_01_004501__en-us_topic_0000001086742916_p23659124171418">Parameter</p>
|
||||
@ -69,7 +69,7 @@
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0023.html">Managing Links</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0157.html">Configuring Link Parameters</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@ -1,6 +1,6 @@
|
||||
<a name="dataartsstudio_01_0047"></a><a name="dataartsstudio_01_0047"></a>
|
||||
|
||||
<h1 class="topictitle1">Source Job Parameters</h1>
|
||||
<h1 class="topictitle1">Configuring CDM Source Job Parameters</h1>
|
||||
<div id="body8662426"></div>
|
||||
<div>
|
||||
<ul class="ullinks">
|
||||
@ -12,7 +12,7 @@
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0051.html">From Hive</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0120_0.html">From DLI</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0120.html">From DLI</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0052.html">From FTP/SFTP</a></strong><br>
|
||||
</li>
|
||||
@ -44,14 +44,16 @@
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0286.html">From MRS ClickHouse</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0289.html">From LogHub (SLS)</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0291.html">From a ShenTong Database</a></strong><br>
|
||||
</li>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_0136.html">From Doris</a></strong><br>
|
||||
<li class="ulchildlink"><strong><a href="dataartsstudio_01_1629.html">From YASHAN</a></strong><br>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0081.html">Managing Jobs</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0081.html">Creating a Job in a CDM Cluster</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
@ -29,7 +29,7 @@
|
||||
<p id="dataartsstudio_01_0048__en-us_topic_0108275319_p855434665810">Directory or file path from which data will be extracted. You can enter a maximum of 50 file paths. By default, the file paths are separated by vertical bars (|). You can also customize a file separator. </p>
|
||||
<p id="dataartsstudio_01_0048__en-us_topic_0108275319_p49679878163953">Directory from which data is to be migrated. All files (including all nested subdirectories and their subfiles) in the directory will be migrated.</p>
|
||||
<p id="dataartsstudio_01_0048__en-us_topic_0108275319_p6705959695715">This parameter can be configured as a macro variable of date and time and a path name can contain multiple macro variables. When the macro variable of date and time works with a scheduled job, the incremental data can be synchronized periodically. </p>
|
||||
<div class="note" id="dataartsstudio_01_0048__en-us_topic_0108275319_note0371124034016"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_p52974484409">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_text1929717485405">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i2814743133910">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i118151243173917">Offset</em>) rather than (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i3816184313913">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i1981664317393">Offset</em>).</p>
|
||||
<div class="note" id="dataartsstudio_01_0048__en-us_topic_0108275319_note0371124034016"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_p52974484409">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_text9997118203">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i799871152012">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i0998101192020">Offset</em>) rather than (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i299821132018">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i129981917200">Offset</em>).</p>
|
||||
</div></div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p64647157163953">FROM/example.csv</p>
|
||||
@ -77,7 +77,7 @@
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row14158172014415"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p141589201416">JSON Type</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p151581520164119">This parameter is displayed only when <span class="parmname" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmname98832165"><b>File Format</b></span> is set to <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue407007853"><b>JSON</b></span>. Type of a JSON object stored in a JSON file. The options are <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue961121311247"><b>JSON object</b></span> and <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue7205191992416"><b>JSON array</b></span>.</p>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p151581520164119">This parameter is displayed only when <span class="parmname" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmname538960128"><b>File Format</b></span> is set to <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue1158646714"><b>JSON</b></span>. Type of a JSON object stored in a JSON file. The options are <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue961121311247"><b>JSON object</b></span> and <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue7205191992416"><b>JSON array</b></span>.</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p715815208415">JSON object</p>
|
||||
</td>
|
||||
@ -136,7 +136,7 @@
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row18185115712363"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p101860576360">Use First N Rows as Header</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p1186155710366">This parameter is displayed only when <span class="parmname" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmname369680571"><b>File Format</b></span> is set to <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue1230851433"><b>CSV</b></span>. When you migrate a CSV file to a table, CDM writes all data to the table by default. If you set this parameter to <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue1968432313314"><b>Yes</b></span>, CDM uses the first N rows of the CSV file as the heading row and does not write the row to the destination table.</p>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p1186155710366">This parameter is displayed only when <span class="parmname" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmname1938690813"><b>File Format</b></span> is set to <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue1483588303"><b>CSV</b></span>. When you migrate a CSV file to a table, CDM writes all data to the table by default. If you set this parameter to <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue1968432313314"><b>Yes</b></span>, CDM uses the first N rows of the CSV file as the heading row and does not write the row to the destination table.</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p11186257103611">No</p>
|
||||
</td>
|
||||
@ -222,7 +222,7 @@
|
||||
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row3022091315196"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p3197490715196">Directory Filter</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p48744974143114">If you set <span class="parmname" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmname9651163720211"><b>Filter Type</b></span> to <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue1199413443210"><b>Wildcard</b></span> or <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b17182166142119">Regex</strong>, enter a wildcard character to filter paths. The paths that meet the filtering condition are migrated. You can configure multiple paths separated by commas (,).</p>
|
||||
<div class="note" id="dataartsstudio_01_0048__en-us_topic_0108275319_note10792171911399"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_p52974484409_1">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_text1929717485405_1">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i2814743133910_1">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i118151243173917_1">Offset</em>) rather than (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i3816184313913_1">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i1981664317393_1">Offset</em>).</p>
|
||||
<div class="note" id="dataartsstudio_01_0048__en-us_topic_0108275319_note10792171911399"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_p52974484409_1">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_text9997118203_1">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i799871152012_1">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i0998101192020_1">Offset</em>) rather than (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i299821132018_1">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i129981917200_1">Offset</em>).</p>
|
||||
</div></div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p708014715196">*input</p>
|
||||
@ -231,7 +231,7 @@
|
||||
<tr id="dataartsstudio_01_0048__en-us_topic_0108275319_row4511408015199"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p3036188415199">File Filter</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p21968542143129">If you set <span class="parmname" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmname15716259342"><b>Filter Type</b></span> to <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue561362193810"><b>Wildcard</b></span> or <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue1759949153411"><b>Regex</b></span>, you can enter a wildcard character to search for files in a specified path. The files that meet the search criteria are migrated. You can configure multiple files separated by commas (,).</p>
|
||||
<div class="note" id="dataartsstudio_01_0048__en-us_topic_0108275319_note12752123203916"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_p52974484409_2">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_text1929717485405_2">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i2814743133910_2">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i118151243173917_2">Offset</em>) rather than (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i3816184313913_2">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i1981664317393_2">Offset</em>).</p>
|
||||
<div class="note" id="dataartsstudio_01_0048__en-us_topic_0108275319_note12752123203916"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_p52974484409_2">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_text9997118203_2">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i799871152012_2">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i0998101192020_2">Offset</em>) rather than (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i299821132018_2">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i129981917200_2">Offset</em>).</p>
|
||||
</div></div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p2943506815199">*.csv,*.txt</p>
|
||||
@ -248,7 +248,7 @@
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p299381213269">If you set <span class="parmname" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmname8132151819611"><b>Filter Type</b></span> to <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue113211184612"><b>Time Filter</b></span>, and specify a point in time for this parameter, only the files modified at or after the specified time are transferred. The time format must be <em id="dataartsstudio_01_0048__en-us_topic_0108275319_i16132918961">yyyy-MM-dd HH:mm:ss</em>.</p>
|
||||
<p id="dataartsstudio_01_0048__en-us_topic_0108275319_p1365975611912">This parameter can be set to a macro variable of date and time. For example, <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b97861314103">${timestamp(dateformat(yyyy-MM-dd HH:mm:ss,-90,DAY))}</strong> indicates that only files generated within the latest 90 days are migrated.</p>
|
||||
<div class="note" id="dataartsstudio_01_0048__en-us_topic_0108275319_note139020491241"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p52974484409">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0048__en-us_topic_0108275319_text1929717485405">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_i2814743133910">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_i118151243173917">Offset</em>) rather than (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_i3816184313913">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_i1981664317393">Offset</em>).</p>
|
||||
<div class="note" id="dataartsstudio_01_0048__en-us_topic_0108275319_note139020491241"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p52974484409">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0048__en-us_topic_0108275319_text9997118203">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_i799871152012">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_i0998101192020">Offset</em>) rather than (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_i299821132018">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_i129981917200">Offset</em>).</p>
|
||||
</div></div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p499321214262">2019-06-01 00:00:00</p>
|
||||
@ -258,7 +258,7 @@
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p176752141020">If you set <span class="parmname" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmname32123244113"><b>Filter Type</b></span> to <span class="parmvalue" id="dataartsstudio_01_0048__en-us_topic_0108275319_parmvalue161911324415"><b>Time Filter</b></span>, and specify a point in time for this parameter, only the files modified before the specified time are transferred. The time format must be <em id="dataartsstudio_01_0048__en-us_topic_0108275319_i4192321411">yyyy-MM-dd HH:mm:ss</em>.</p>
|
||||
<p id="dataartsstudio_01_0048__en-us_topic_0108275319_p3469261931">This parameter can be set to a macro variable of date and time. For example, <strong id="dataartsstudio_01_0048__en-us_topic_0108275319_b1521616538112">${timestamp(dateformat(yyyy-MM-dd HH:mm:ss))}</strong> indicates that only the files whose modification time is earlier than the current time are migrated.</p>
|
||||
<div class="note" id="dataartsstudio_01_0048__en-us_topic_0108275319_note45791757134910"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_p52974484409_3">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_text1929717485405_3">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i2814743133910_3">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i118151243173917_3">Offset</em>) rather than (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i3816184313913_3">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i1981664317393_3">Offset</em>).</p>
|
||||
<div class="note" id="dataartsstudio_01_0048__en-us_topic_0108275319_note45791757134910"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_p52974484409_3">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_text9997118203_3">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i799871152012_3">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i0998101192020_3">Offset</em>) rather than (<em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i299821132018_3">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0048__en-us_topic_0108275319_en-us_topic_0108275319_i129981917200_3">Offset</em>).</p>
|
||||
</div></div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0048__en-us_topic_0108275319_p1389085325813">2019-07-01 00:00:00</p>
|
||||
@ -288,7 +288,7 @@
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0047.html">Source Job Parameters</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0047.html">Configuring CDM Source Job Parameters</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
@ -28,7 +28,7 @@
|
||||
<p id="dataartsstudio_01_0049__en-us_topic_0108275442_p3398923015165">Directory or file path from which data will be extracted.</p>
|
||||
<p id="dataartsstudio_01_0049__en-us_topic_0108275442_p49679878163953">Directory from which data is to be migrated. All files (including all nested subdirectories and their subfiles) in the directory will be migrated.</p>
|
||||
<p id="dataartsstudio_01_0049__en-us_topic_0108275442_p1210244910548">This parameter can be configured as a macro variable of date and time and a path name can contain multiple macro variables. When the macro variable of date and time works with a scheduled job, the incremental data can be synchronized periodically. </p>
|
||||
<div class="note" id="dataartsstudio_01_0049__en-us_topic_0108275442_note45791757134910"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_p52974484409">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_text1929717485405">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i2814743133910">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i118151243173917">Offset</em>) rather than (<em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i3816184313913">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i1981664317393">Offset</em>).</p>
|
||||
<div class="note" id="dataartsstudio_01_0049__en-us_topic_0108275442_note45791757134910"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_p52974484409">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_text9997118203">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i799871152012">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i0998101192020">Offset</em>) rather than (<em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i299821132018">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i129981917200">Offset</em>).</p>
|
||||
</div></div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.3 "><p id="dataartsstudio_01_0049__en-us_topic_0108275442_p166427315165">/user/cdm/</p>
|
||||
@ -91,7 +91,7 @@
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0049__en-us_topic_0108275442_row06816265114"><td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.1 "><p id="dataartsstudio_01_0049__en-us_topic_0108275442_p116811622519">Use First Row as Header</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.2 "><p id="dataartsstudio_01_0049__en-us_topic_0108275442_p1468115213518">This parameter is displayed only when <span class="parmname" id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_parmname369680571"><b>File Format</b></span> is set to <span class="parmvalue" id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_parmvalue1230851433"><b>CSV</b></span>. When you migrate a CSV file to a table, CDM writes all data to the table by default. If you set this parameter to <span class="parmvalue" id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_parmvalue1968432313314"><b>Yes</b></span>, CDM uses the first N rows of the CSV file as the heading row and does not write the row to the destination table.</p>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.2 "><p id="dataartsstudio_01_0049__en-us_topic_0108275442_p1468115213518">This parameter is displayed only when <span class="parmname" id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_parmname1938690813"><b>File Format</b></span> is set to <span class="parmvalue" id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_parmvalue1483588303"><b>CSV</b></span>. When you migrate a CSV file to a table, CDM writes all data to the table by default. If you set this parameter to <span class="parmvalue" id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_parmvalue1968432313314"><b>Yes</b></span>, CDM uses the first N rows of the CSV file as the heading row and does not write the row to the destination table.</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.3 "><p id="dataartsstudio_01_0049__en-us_topic_0108275442_p368113211517">No</p>
|
||||
</td>
|
||||
@ -120,7 +120,7 @@
|
||||
<tr id="dataartsstudio_01_0049__en-us_topic_0108275442_row994336215165"><td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.1 "><p id="dataartsstudio_01_0049__en-us_topic_0108275442_p64217335145116">Directory Filter</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.2 "><p id="dataartsstudio_01_0049__en-us_topic_0108275442_p34221671145116">If you set <span class="parmname" id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_parmname9651163720211"><b>Filter Type</b></span> to <span class="parmvalue" id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_parmvalue1199413443210"><b>Wildcard</b></span> or <strong id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_b17182166142119">Regex</strong>, enter a wildcard character to filter paths. The paths that meet the filtering condition are migrated. You can configure multiple paths separated by commas (,).</p>
|
||||
<div class="note" id="dataartsstudio_01_0049__en-us_topic_0108275442_note363856144214"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_p52974484409_1">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_text1929717485405_1">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i2814743133910_1">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i118151243173917_1">Offset</em>) rather than (<em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i3816184313913_1">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i1981664317393_1">Offset</em>).</p>
|
||||
<div class="note" id="dataartsstudio_01_0049__en-us_topic_0108275442_note363856144214"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_p52974484409_1">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_text9997118203_1">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i799871152012_1">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i0998101192020_1">Offset</em>) rather than (<em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i299821132018_1">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i129981917200_1">Offset</em>).</p>
|
||||
</div></div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.3 "><p id="dataartsstudio_01_0049__en-us_topic_0108275442_p20491984145116">*input</p>
|
||||
@ -129,7 +129,7 @@
|
||||
<tr id="dataartsstudio_01_0049__en-us_topic_0108275442_row851377714512"><td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.1 "><p id="dataartsstudio_01_0049__en-us_topic_0108275442_p40488609145116">File Filter</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.2 "><p id="dataartsstudio_01_0049__en-us_topic_0108275442_p58351869145116">If you set <span class="parmname" id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_parmname15716259342"><b>Filter Type</b></span> to <span class="parmvalue" id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_parmvalue561362193810"><b>Wildcard</b></span> or <span class="parmvalue" id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_parmvalue1759949153411"><b>Regex</b></span>, you can enter a wildcard character to search for files in a specified path. The files that meet the search criteria are migrated. You can configure multiple files separated by commas (,).</p>
|
||||
<div class="note" id="dataartsstudio_01_0049__en-us_topic_0108275442_note937081116422"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_p52974484409_2">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_text1929717485405_2">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i2814743133910_2">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i118151243173917_2">Offset</em>) rather than (<em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i3816184313913_2">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i1981664317393_2">Offset</em>).</p>
|
||||
<div class="note" id="dataartsstudio_01_0049__en-us_topic_0108275442_note937081116422"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_p52974484409_2">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_text9997118203_2">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i799871152012_2">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i0998101192020_2">Offset</em>) rather than (<em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i299821132018_2">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i129981917200_2">Offset</em>).</p>
|
||||
</div></div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.3 "><p id="dataartsstudio_01_0049__en-us_topic_0108275442_p28880974145116">*.csv</p>
|
||||
@ -146,7 +146,7 @@
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.2 "><p id="dataartsstudio_01_0049__en-us_topic_0108275442_p18311447203911">If you set <span class="parmname" id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_parmname8132151819611"><b>Filter Type</b></span> to <span class="parmvalue" id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_parmvalue113211184612"><b>Time Filter</b></span>, and specify a point in time for this parameter, only the files modified at or after the specified time are transferred. The time format must be <em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i16132918961">yyyy-MM-dd HH:mm:ss</em>.</p>
|
||||
<p id="dataartsstudio_01_0049__en-us_topic_0108275442_p12144111817144">This parameter can be set to a macro variable of date and time. For example, <strong id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_b97861314103">${timestamp(dateformat(yyyy-MM-dd HH:mm:ss,-90,DAY))}</strong> indicates that only files generated within the latest 90 days are migrated.</p>
|
||||
<div class="note" id="dataartsstudio_01_0049__en-us_topic_0108275442_note6622111464219"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_p52974484409_3">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_text1929717485405_3">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i2814743133910_3">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i118151243173917_3">Offset</em>) rather than (<em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i3816184313913_3">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i1981664317393_3">Offset</em>).</p>
|
||||
<div class="note" id="dataartsstudio_01_0049__en-us_topic_0108275442_note6622111464219"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_p52974484409_3">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_text9997118203_3">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i799871152012_3">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i0998101192020_3">Offset</em>) rather than (<em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i299821132018_3">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i129981917200_3">Offset</em>).</p>
|
||||
</div></div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.3 "><p id="dataartsstudio_01_0049__en-us_topic_0108275442_p1683154713395">2019-07-01 00:00:00</p>
|
||||
@ -156,7 +156,7 @@
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.2 "><p id="dataartsstudio_01_0049__en-us_topic_0108275442_p34921611171416">If you set <span class="parmname" id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_parmname32123244113"><b>Filter Type</b></span> to <span class="parmvalue" id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_parmvalue161911324415"><b>Time Filter</b></span>, and specify a point in time for this parameter, only the files modified before the specified time are transferred. The time format must be <em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i4192321411">yyyy-MM-dd HH:mm:ss</em>.</p>
|
||||
<p id="dataartsstudio_01_0049__en-us_topic_0108275442_p13254806156">This parameter can be set to a macro variable of date and time. For example, <strong id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_b1521616538112">${timestamp(dateformat(yyyy-MM-dd HH:mm:ss))}</strong> indicates that only the files whose modification time is earlier than the current time are migrated.</p>
|
||||
<div class="note" id="dataartsstudio_01_0049__en-us_topic_0108275442_note107252176423"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_p52974484409_4">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_text1929717485405_4">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i2814743133910_4">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i118151243173917_4">Offset</em>) rather than (<em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i3816184313913_4">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i1981664317393_4">Offset</em>).</p>
|
||||
<div class="note" id="dataartsstudio_01_0049__en-us_topic_0108275442_note107252176423"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_p52974484409_4">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_text9997118203_4">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i799871152012_4">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i0998101192020_4">Offset</em>) rather than (<em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i299821132018_4">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275319_i129981917200_4">Offset</em>).</p>
|
||||
</div></div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.3 "><p id="dataartsstudio_01_0049__en-us_topic_0108275442_p8442749101515">2019-07-30 00:00:00</p>
|
||||
@ -181,7 +181,7 @@
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0049__en-us_topic_0108275442_row8357848162917"><td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.1 "><p id="dataartsstudio_01_0049__en-us_topic_0108275442_p2560191135113">DEK</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.2 "><p id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275442_p13560410513">This parameter is displayed only when <span class="parmname" id="dataartsstudio_01_0049__en-us_topic_0108275442_parmname21474223411287"><b>Encryption</b></span> is set to <span class="parmvalue" id="dataartsstudio_01_0049__en-us_topic_0108275442_parmvalue73098374311287"><b>AES-256-GCM</b></span>. The key consists of 64 hexadecimal numbers and must be the same as the <span class="parmname" id="dataartsstudio_01_0049__en-us_topic_0108275442_parmname187575420211287"><b>DEK</b></span> configured during encryption. If the encryption and decryption keys are inconsistent, the system does not report an exception, but the decrypted data is incorrect.</p>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.2 "><p id="dataartsstudio_01_0049__en-us_topic_0108275442_en-us_topic_0108275442_p13560410513">This parameter is displayed only when <span class="parmname" id="dataartsstudio_01_0049__en-us_topic_0108275442_parmname358922962013"><b>Encryption</b></span> is set to <span class="parmvalue" id="dataartsstudio_01_0049__en-us_topic_0108275442_parmvalue1958922962018"><b>AES-256-GCM</b></span>. The key consists of 64 hexadecimal numbers and must be the same as the <span class="parmname" id="dataartsstudio_01_0049__en-us_topic_0108275442_parmname135891029132013"><b>DEK</b></span> configured during encryption. If the encryption and decryption keys are inconsistent, the system does not report an exception, but the decrypted data is incorrect.</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.3 "><p id="dataartsstudio_01_0049__en-us_topic_0108275442_p10560111155111">DD0AE00DFECD78BF051BCFDA25BD4E320DB0A7AC75A1F3FC3D3C56A457DCDC1B</p>
|
||||
</td>
|
||||
@ -207,7 +207,7 @@
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0047.html">Source Job Parameters</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0047.html">Configuring CDM Source Job Parameters</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
@ -22,7 +22,7 @@
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="42.230000000000004%" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0050__en-us_topic_0108275276_p3398923015165">Name of the HBase table that data will be exported from</p>
|
||||
<p id="dataartsstudio_01_0050__en-us_topic_0108275276_p1210244910548">This parameter can be configured as a macro variable of date and time and a path name can contain multiple macro variables. When the macro variable of date and time works with a scheduled job, the incremental data can be synchronized periodically. </p>
|
||||
<div class="note" id="dataartsstudio_01_0050__en-us_topic_0108275276_note1391794455811"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_p52974484409">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_text1929717485405">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_i2814743133910">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_i118151243173917">Offset</em>) rather than (<em id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_i3816184313913">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_i1981664317393">Offset</em>).</p>
|
||||
<div class="note" id="dataartsstudio_01_0050__en-us_topic_0108275276_note1391794455811"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_p52974484409">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_text9997118203">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_i799871152012">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_i0998101192020">Offset</em>) rather than (<em id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_i299821132018">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_i129981917200">Offset</em>).</p>
|
||||
</div></div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="20.4%" headers="mcps1.3.3.2.5.1.4 "><p id="dataartsstudio_01_0050__en-us_topic_0108275276_p166427315165">TBL_2</p>
|
||||
@ -55,7 +55,7 @@
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0050__en-us_topic_0108275276_p091005834018">(Optional) <span id="dataartsstudio_01_0050__en-us_topic_0108275276_ph5654161619427">Start time (including the value) for extracting data. The format is <em id="dataartsstudio_01_0050__en-us_topic_0108275276_i38211611191413">yyyy-MM-dd HH:mm:ss</em>. Only the data generated at the specified time and later is extracted.</span></p>
|
||||
<p id="dataartsstudio_01_0050__en-us_topic_0108275276_p116711580415">This parameter can be set to a macro variable of date and time. When the macro variable of date and time works with a scheduled job, the incremental data can be synchronized periodically. </p>
|
||||
<div class="note" id="dataartsstudio_01_0050__en-us_topic_0108275276_note45791757134910"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_p52974484409_1">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_text1929717485405_1">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_i2814743133910_1">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_i118151243173917_1">Offset</em>) rather than (<em id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_i3816184313913_1">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_i1981664317393_1">Offset</em>).</p>
|
||||
<div class="note" id="dataartsstudio_01_0050__en-us_topic_0108275276_note45791757134910"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_p52974484409_1">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_text9997118203_1">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_i799871152012_1">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_i0998101192020_1">Offset</em>) rather than (<em id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_i299821132018_1">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_i129981917200_1">Offset</em>).</p>
|
||||
</div></div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0050__en-us_topic_0108275276_p17670173812526">2019-01-01 20:00:00</p>
|
||||
@ -65,7 +65,7 @@
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0050__en-us_topic_0108275276_p2700041145210">(Optional) <span id="dataartsstudio_01_0050__en-us_topic_0108275276_ph5201142410429">End time (excluding the value) for extracting data. The format is <em id="dataartsstudio_01_0050__en-us_topic_0108275276_i1268214522145">yyyy-MM-dd HH:mm:ss</em>. Only the data generated before the time point is extracted.</span></p>
|
||||
<p id="dataartsstudio_01_0050__en-us_topic_0108275276_p92671211101710">This parameter can be set to a macro variable of date and time. </p>
|
||||
<div class="note" id="dataartsstudio_01_0050__en-us_topic_0108275276_note11381953104418"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_p52974484409_2">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_text1929717485405_2">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_i2814743133910_2">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_i118151243173917_2">Offset</em>) rather than (<em id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_i3816184313913_2">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_i1981664317393_2">Offset</em>).</p>
|
||||
<div class="note" id="dataartsstudio_01_0050__en-us_topic_0108275276_note11381953104418"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_p52974484409_2">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_text9997118203_2">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_i799871152012_2">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_i0998101192020_2">Offset</em>) rather than (<em id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_i299821132018_2">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0050__en-us_topic_0108275276_en-us_topic_0108275319_i129981917200_2">Offset</em>).</p>
|
||||
</div></div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0050__en-us_topic_0108275276_p17007416523">2019-02-01 20:00:00</p>
|
||||
@ -77,7 +77,7 @@
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0047.html">Source Job Parameters</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0047.html">Configuring CDM Source Job Parameters</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
@ -26,7 +26,7 @@
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.2 "><p id="dataartsstudio_01_0051__en-us_topic_0108275424_p17984995155931">Hive table name. Click the icon next to the text box. The dialog box for selecting the table is displayed.</p>
|
||||
<p id="dataartsstudio_01_0051__en-us_topic_0108275424_p1210244910548">This parameter can be configured as a macro variable of date and time and a path name can contain multiple macro variables. When the macro variable of date and time works with a scheduled job, the incremental data can be synchronized periodically. </p>
|
||||
<div class="note" id="dataartsstudio_01_0051__en-us_topic_0108275424_note116874368587"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_p52974484409">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_text1929717485405">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_i2814743133910">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_i118151243173917">Offset</em>) rather than (<em id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_i3816184313913">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_i1981664317393">Offset</em>).</p>
|
||||
<div class="note" id="dataartsstudio_01_0051__en-us_topic_0108275424_note116874368587"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_p52974484409">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_text9997118203">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_i799871152012">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_i0998101192020">Offset</em>) rather than (<em id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_i299821132018">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_i129981917200">Offset</em>).</p>
|
||||
</div></div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.3 "><p id="dataartsstudio_01_0051__en-us_topic_0108275424_p50683068163953">TBL_E</p>
|
||||
@ -35,7 +35,9 @@
|
||||
<tr id="dataartsstudio_01_0051__en-us_topic_0108275424_row962165734014"><td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.1 "><p id="dataartsstudio_01_0051__en-us_topic_0108275424_p96355710402">Read Mode</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.2 "><p id="dataartsstudio_01_0051__en-us_topic_0108275424_p1656393534412">Two read modes are available: HDFS and JDBC. By default, the HDFS mode is used. If you do not need to use the WHERE condition to filter data or add new fields on the field mapping page, select the HDFS mode.</p>
|
||||
<ul id="dataartsstudio_01_0051__en-us_topic_0108275424_ul169421201417"><li id="dataartsstudio_01_0051__en-us_topic_0108275424_li3378113716448">The HDFS mode shows good performance, but in this mode, you cannot use the WHERE condition to filter data or add new fields on the field mapping page.</li><li id="dataartsstudio_01_0051__en-us_topic_0108275424_li13942122018412">The HDFS mode allows you to use the WHERE condition to filter data or add new fields on the field mapping page.</li></ul>
|
||||
<ul id="dataartsstudio_01_0051__en-us_topic_0108275424_ul169421201417"><li id="dataartsstudio_01_0051__en-us_topic_0108275424_li3378113716448">The HDFS mode shows good performance, but in this mode, you cannot use the WHERE condition to filter data or add new fields on the field mapping page.</li><li id="dataartsstudio_01_0051__en-us_topic_0108275424_li13942122018412">The HDFS mode allows you to use the WHERE condition to filter data or add new fields on the field mapping page.<div class="note" id="dataartsstudio_01_0051__en-us_topic_0108275424_note10801173103210"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0051__en-us_topic_0108275424_p16801133183213">If the migration source is Hive and JDBC is used to read data, CDM does not support concurrency. That is, <strong id="dataartsstudio_01_0051__en-us_topic_0108275424_b63018315255">Concurrent Extractors</strong> can only be set to <strong id="dataartsstudio_01_0051__en-us_topic_0108275424_b757413336254">1</strong>.</p>
|
||||
</div></div>
|
||||
</li></ul>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.3 "><p id="dataartsstudio_01_0051__en-us_topic_0108275424_p2631557154010">HDFS</p>
|
||||
</td>
|
||||
@ -50,8 +52,8 @@
|
||||
<tr id="dataartsstudio_01_0051__en-us_topic_0108275424_row1186522594314"><td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.1 "><p id="dataartsstudio_01_0051__en-us_topic_0108275424_p1186632594313">SQL Statement</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.2 "><p id="dataartsstudio_01_0051__en-us_topic_0108275424_p5220739135">When <span class="parmname" id="dataartsstudio_01_0051__en-us_topic_0108275424_parmname346133185611"><b>Use SQL Statement</b></span> is set to <span class="parmvalue" id="dataartsstudio_01_0051__en-us_topic_0108275424_parmvalue623353520561"><b>Yes</b></span>, enter an SQL statement here. CDM exports data based on the SQL statement.</p>
|
||||
<div class="note" id="dataartsstudio_01_0051__en-us_topic_0108275424_note1624112587564"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dataartsstudio_01_0051__en-us_topic_0108275424_ul95781925125319"><li id="dataartsstudio_01_0051__en-us_topic_0108275424_li15578132545313">SQL statements can only be used to query data. Join and nesting are supported, but multiple query statements are not allowed, for example, <strong id="dataartsstudio_01_0051__en-us_topic_0108275424_b438026484112425">select * from table a; select * from table b</strong>.</li><li id="dataartsstudio_01_0051__en-us_topic_0108275424_li115789258538">With statements are not supported.</li><li id="dataartsstudio_01_0051__en-us_topic_0108275424_li15578152525319">Comments, such as <strong id="dataartsstudio_01_0051__en-us_topic_0108275424_b694493195112425">--</strong> and <strong id="dataartsstudio_01_0051__en-us_topic_0108275424_b574213437112425">/*</strong>, are not supported.</li><li id="dataartsstudio_01_0051__en-us_topic_0108275424_li115785251536">Addition, deletion, and modification operations are not supported, including but not limited to the following:<ul id="dataartsstudio_01_0051__en-us_topic_0108275424_ul185084210539"><li id="dataartsstudio_01_0051__en-us_topic_0108275424_li1623929145315">load data</li><li id="dataartsstudio_01_0051__en-us_topic_0108275424_li15832356155310">delete from</li><li id="dataartsstudio_01_0051__en-us_topic_0108275424_li144890175411">alter table</li><li id="dataartsstudio_01_0051__en-us_topic_0108275424_li17896205419">create table</li><li id="dataartsstudio_01_0051__en-us_topic_0108275424_li7715510105412">drop table</li><li id="dataartsstudio_01_0051__en-us_topic_0108275424_li548061418543">into outfile</li></ul>
|
||||
</li></ul>
|
||||
<div class="note" id="dataartsstudio_01_0051__en-us_topic_0108275424_note1624112587564"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275324_en-us_topic_0108275424_ul95781925125319"><li id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275324_en-us_topic_0108275424_li15578132545313">SQL statements can only be used to query data. Join and nesting are supported, but multiple query statements are not allowed, for example, <strong id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275324_b143067633772251">select * from table a; select * from table b</strong>.</li><li id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275324_en-us_topic_0108275424_li115789258538">With statements are not supported.</li><li id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275324_en-us_topic_0108275424_li15578152525319">Comments, such as <strong id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275324_b148619239372251">--</strong> and <strong id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275324_b187717313372251">/*</strong>, are not supported.</li><li id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275324_en-us_topic_0108275424_li115785251536">Addition, deletion, and modification operations are not supported, including but not limited to the following:<ul id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275324_en-us_topic_0108275424_ul185084210539"><li id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275324_en-us_topic_0108275424_li1623929145315">load data</li><li id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275324_en-us_topic_0108275424_li15832356155310">delete from</li><li id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275324_en-us_topic_0108275424_li144890175411">alter table</li><li id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275324_en-us_topic_0108275424_li17896205419">create table</li><li id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275324_en-us_topic_0108275424_li7715510105412">drop table</li><li id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275324_en-us_topic_0108275424_li548061418543">into outfile</li></ul>
|
||||
</li><li id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275324_li1517424293819">If the SQL statement is too long, the request fails to be delivered. If you continue to create a job, the system displays an error message indicating that the request is incorrect. In this case, you need to simplify or clear the SQL statement and try again.</li></ul>
|
||||
</div></div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.3 "><p id="dataartsstudio_01_0051__en-us_topic_0108275424_p6220439735">select id,name from sqoop.user;</p>
|
||||
@ -64,7 +66,7 @@
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="41.91%" headers="mcps1.3.2.2.5.1.3 "><p id="dataartsstudio_01_0051__en-us_topic_0108275424_p1219435334515">This parameter is displayed when you select the HDFS read mode and click <strong id="dataartsstudio_01_0051__en-us_topic_0108275424_b1454933863319">Show Advanced Attributes</strong>.</p>
|
||||
<p id="dataartsstudio_01_0051__en-us_topic_0108275424_p1020335414148">This parameter indicates extracting the partition of a specified value. The attribute name is the partition name. You can configure multiple values (separated by spaces) or a field value range. The time macro function is supported. </p>
|
||||
<div class="note" id="dataartsstudio_01_0051__en-us_topic_0108275424_note1855834135914"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_p52974484409_1">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_text1929717485405_1">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_i2814743133910_1">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_i118151243173917_1">Offset</em>) rather than (<em id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_i3816184313913_1">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_i1981664317393_1">Offset</em>).</p>
|
||||
<div class="note" id="dataartsstudio_01_0051__en-us_topic_0108275424_note1855834135914"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_p52974484409_1">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_text9997118203_1">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_i799871152012_1">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_i0998101192020_1">Offset</em>) rather than (<em id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_i299821132018_1">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_i129981917200_1">Offset</em>).</p>
|
||||
</div></div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="20.61%" headers="mcps1.3.2.2.5.1.4 "><ul id="dataartsstudio_01_0051__en-us_topic_0108275424_ul16185646318"><li id="dataartsstudio_01_0051__en-us_topic_0108275424_li187594277316">Attribute value in the single-value or multi-value filtering scenario:<p id="dataartsstudio_01_0051__en-us_topic_0108275424_p88421435130"><a name="dataartsstudio_01_0051__en-us_topic_0108275424_li187594277316"></a><a name="en-us_topic_0108275424_li187594277316"></a>${dateformat(yyyyMMdd, -1, DAY)} ${dateformat(yyyyMMdd)}</p>
|
||||
@ -77,7 +79,7 @@
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.2 "><p id="dataartsstudio_01_0051__en-us_topic_0108275424_p1213520181469">This parameter is displayed when you select the JDBC read mode and click <strong id="dataartsstudio_01_0051__en-us_topic_0108275424_b1066165863415">Show Advanced Attributes</strong>.</p>
|
||||
<p id="dataartsstudio_01_0051__en-us_topic_0108275424_p976143054712">This parameter indicates the WHERE clause to be extracted. If this parameter is not set, the entire table is extracted. If the table to be migrated does not contain the fields specified by the WHERE clause, the migration will fail.</p>
|
||||
<p id="dataartsstudio_01_0051__en-us_topic_0108275424_p18740131613716">You can set a date macro variable to extract data generated on a specific date. </p>
|
||||
<div class="note" id="dataartsstudio_01_0051__en-us_topic_0108275424_note45791757134910"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_p52974484409_2">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_text1929717485405_2">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_i2814743133910_2">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_i118151243173917_2">Offset</em>) rather than (<em id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_i3816184313913_2">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_i1981664317393_2">Offset</em>).</p>
|
||||
<div class="note" id="dataartsstudio_01_0051__en-us_topic_0108275424_note45791757134910"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_p52974484409_2">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_text9997118203_2">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_i799871152012_2">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_i0998101192020_2">Offset</em>) rather than (<em id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_i299821132018_2">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0051__en-us_topic_0108275424_en-us_topic_0108275319_i129981917200_2">Offset</em>).</p>
|
||||
</div></div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.3 "><p id="dataartsstudio_01_0051__en-us_topic_0108275424_p8135518144617">age > 18 and age <= 60</p>
|
||||
@ -91,7 +93,7 @@
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0047.html">Source Job Parameters</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0047.html">Configuring CDM Source Job Parameters</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
@ -21,7 +21,7 @@
|
||||
<td class="cellrowborder" valign="top" width="54.39000000000001%" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0052__en-us_topic_0108275374_p2876132645914">Directory or file path from which data will be extracted. You can enter a maximum of 50 file paths. By default, the file paths are separated by vertical bars (|). You can also customize a file separator. </p>
|
||||
<p id="dataartsstudio_01_0052__en-us_topic_0108275374_p49679878163953">Directory from which data is to be migrated. All files (including all nested subdirectories and their subfiles) in the directory will be migrated.</p>
|
||||
<p id="dataartsstudio_01_0052__en-us_topic_0108275374_p1210244910548">This parameter can be configured as a macro variable of date and time and a path name can contain multiple macro variables. When the macro variable of date and time works with a scheduled job, the incremental data can be synchronized periodically. </p>
|
||||
<div class="note" id="dataartsstudio_01_0052__en-us_topic_0108275374_note45791757134910"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_p52974484409">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_text1929717485405">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i2814743133910">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i118151243173917">Offset</em>) rather than (<em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i3816184313913">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i1981664317393">Offset</em>).</p>
|
||||
<div class="note" id="dataartsstudio_01_0052__en-us_topic_0108275374_note45791757134910"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_p52974484409">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_text9997118203">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i799871152012">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i0998101192020">Offset</em>) rather than (<em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i299821132018">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i129981917200">Offset</em>).</p>
|
||||
</div></div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="17.4%" headers="mcps1.3.3.2.5.1.4 "><p id="dataartsstudio_01_0052__en-us_topic_0108275374_p166427315165">/ftp/a.csv|/ftp/b.txt</p>
|
||||
@ -39,7 +39,7 @@
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0052__en-us_topic_0108275374_row8316154014433"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0052__en-us_topic_0108275374_p631674094316">JSON Type</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0052__en-us_topic_0108275374_p231620409435">This parameter is displayed only when <span class="parmname" id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_parmname98832165"><b>File Format</b></span> is set to <span class="parmvalue" id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_parmvalue407007853"><b>JSON</b></span>. Type of a JSON object stored in a JSON file. The options are <span class="parmvalue" id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_parmvalue961121311247"><b>JSON object</b></span> and <span class="parmvalue" id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_parmvalue7205191992416"><b>JSON array</b></span>.</p>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0052__en-us_topic_0108275374_p231620409435">This parameter is displayed only when <span class="parmname" id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_parmname538960128"><b>File Format</b></span> is set to <span class="parmvalue" id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_parmvalue1158646714"><b>JSON</b></span>. Type of a JSON object stored in a JSON file. The options are <span class="parmvalue" id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_parmvalue961121311247"><b>JSON object</b></span> and <span class="parmvalue" id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_parmvalue7205191992416"><b>JSON array</b></span>.</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0052__en-us_topic_0108275374_p10316340104318">JSON object</p>
|
||||
</td>
|
||||
@ -105,7 +105,7 @@
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0052__en-us_topic_0108275374_row15301324194018"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0052__en-us_topic_0108275374_p101860576360">Use First Row as Header</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0052__en-us_topic_0108275374_p23062415408">This parameter is displayed only when <span class="parmname" id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_parmname369680571"><b>File Format</b></span> is set to <span class="parmvalue" id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_parmvalue1230851433"><b>CSV</b></span>. When you migrate a CSV file to a table, CDM writes all data to the table by default. If you set this parameter to <span class="parmvalue" id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_parmvalue1968432313314"><b>Yes</b></span>, CDM uses the first N rows of the CSV file as the heading row and does not write the row to the destination table.</p>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0052__en-us_topic_0108275374_p23062415408">This parameter is displayed only when <span class="parmname" id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_parmname1938690813"><b>File Format</b></span> is set to <span class="parmvalue" id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_parmvalue1483588303"><b>CSV</b></span>. When you migrate a CSV file to a table, CDM writes all data to the table by default. If you set this parameter to <span class="parmvalue" id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_parmvalue1968432313314"><b>Yes</b></span>, CDM uses the first N rows of the CSV file as the heading row and does not write the row to the destination table.</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0052__en-us_topic_0108275374_p1630112484014">Yes</p>
|
||||
</td>
|
||||
@ -173,7 +173,7 @@
|
||||
<tr id="dataartsstudio_01_0052__en-us_topic_0108275374_row994336215165"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0052__en-us_topic_0108275374_p58820566161328">Directory Filter</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0052__en-us_topic_0108275374_p4672372415942">If you set <span class="parmname" id="dataartsstudio_01_0052__en-us_topic_0108275374_parmname107276441218"><b>Filter Type</b></span> to <span class="parmvalue" id="dataartsstudio_01_0052__en-us_topic_0108275374_parmvalue1172824417216"><b>Wildcard</b></span> or <strong id="dataartsstudio_01_0052__en-us_topic_0108275374_b2088912183229">Regex</strong>, enter a wildcard character to filter paths. The paths that meet the filtering condition are migrated. You can configure multiple paths separated by commas (,).</p>
|
||||
<div class="note" id="dataartsstudio_01_0052__en-us_topic_0108275374_note6254846184217"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_p52974484409_1">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_text1929717485405_1">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i2814743133910_1">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i118151243173917_1">Offset</em>) rather than (<em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i3816184313913_1">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i1981664317393_1">Offset</em>).</p>
|
||||
<div class="note" id="dataartsstudio_01_0052__en-us_topic_0108275374_note6254846184217"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_p52974484409_1">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_text9997118203_1">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i799871152012_1">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i0998101192020_1">Offset</em>) rather than (<em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i299821132018_1">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i129981917200_1">Offset</em>).</p>
|
||||
</div></div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0052__en-us_topic_0108275374_p45766974161328">*input,*out</p>
|
||||
@ -182,7 +182,7 @@
|
||||
<tr id="dataartsstudio_01_0052__en-us_topic_0108275374_row56738186161317"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.1 "><p id="dataartsstudio_01_0052__en-us_topic_0108275374_p11018662161328">File Filter</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0052__en-us_topic_0108275374_p3740142115942">If you set <span class="parmname" id="dataartsstudio_01_0052__en-us_topic_0108275374_parmname94201846112210"><b>Filter Type</b></span> to <span class="parmvalue" id="dataartsstudio_01_0052__en-us_topic_0108275374_parmvalue14421046172219"><b>Wildcard</b></span> or <strong id="dataartsstudio_01_0052__en-us_topic_0108275374_b1042184622217">Regex</strong>, enter a wildcard character to filter paths. The files that meet the filtering condition are migrated. You can configure multiple files separated by commas (,).</p>
|
||||
<div class="note" id="dataartsstudio_01_0052__en-us_topic_0108275374_note66064498428"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_p52974484409_2">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_text1929717485405_2">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i2814743133910_2">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i118151243173917_2">Offset</em>) rather than (<em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i3816184313913_2">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i1981664317393_2">Offset</em>).</p>
|
||||
<div class="note" id="dataartsstudio_01_0052__en-us_topic_0108275374_note66064498428"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_p52974484409_2">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_text9997118203_2">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i799871152012_2">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i0998101192020_2">Offset</em>) rather than (<em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i299821132018_2">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i129981917200_2">Offset</em>).</p>
|
||||
</div></div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0052__en-us_topic_0108275374_p17196199161328">*.csv</p>
|
||||
@ -199,7 +199,7 @@
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0052__en-us_topic_0108275374_p18311447203911">If you set <span class="parmname" id="dataartsstudio_01_0052__en-us_topic_0108275374_parmname161102721312"><b>Time Filter</b></span> to <span class="parmvalue" id="dataartsstudio_01_0052__en-us_topic_0108275374_parmvalue31118781318"><b>Yes</b></span>, you can specify a point in time for <strong id="dataartsstudio_01_0052__en-us_topic_0108275374_b41114710138">Minimum Timestamp</strong>, and then only the files modified at or after the specified time are transferred. The time format must be <em id="dataartsstudio_01_0052__en-us_topic_0108275374_i161111479132">yyyy-MM-dd HH:mm:ss</em>.</p>
|
||||
<p id="dataartsstudio_01_0052__en-us_topic_0108275374_p12144111817144">This parameter can be set to a macro variable of date and time. For example, <strong id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_b97861314103">${timestamp(dateformat(yyyy-MM-dd HH:mm:ss,-90,DAY))}</strong> indicates that only files generated within the latest 90 days are migrated.</p>
|
||||
<div class="note" id="dataartsstudio_01_0052__en-us_topic_0108275374_note9951165212427"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_p52974484409_3">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_text1929717485405_3">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i2814743133910_3">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i118151243173917_3">Offset</em>) rather than (<em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i3816184313913_3">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i1981664317393_3">Offset</em>).</p>
|
||||
<div class="note" id="dataartsstudio_01_0052__en-us_topic_0108275374_note9951165212427"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_p52974484409_3">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_text9997118203_3">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i799871152012_3">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i0998101192020_3">Offset</em>) rather than (<em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i299821132018_3">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i129981917200_3">Offset</em>).</p>
|
||||
</div></div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0052__en-us_topic_0108275374_p1683154713395">2019-07-01 00:00:00</p>
|
||||
@ -209,7 +209,7 @@
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.2 "><p id="dataartsstudio_01_0052__en-us_topic_0108275374_p34921611171416">If you set <span class="parmname" id="dataartsstudio_01_0052__en-us_topic_0108275374_parmname73341232250"><b>Time Filter</b></span> to <span class="parmvalue" id="dataartsstudio_01_0052__en-us_topic_0108275374_parmvalue1933582312514"><b>Yes</b></span>, you can specify a point in time for <strong id="dataartsstudio_01_0052__en-us_topic_0108275374_b2335202310253">Maximum Timestamp</strong>, and then only the files modified before the specified time are transferred. The time format must be <em id="dataartsstudio_01_0052__en-us_topic_0108275374_i0335182312250">yyyy-MM-dd HH:mm:ss</em>.</p>
|
||||
<p id="dataartsstudio_01_0052__en-us_topic_0108275374_p13254806156">This parameter can be set to a macro variable of date and time. For example, <strong id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_b1521616538112">${timestamp(dateformat(yyyy-MM-dd HH:mm:ss))}</strong> indicates that only the files whose modification time is earlier than the current time are migrated.</p>
|
||||
<div class="note" id="dataartsstudio_01_0052__en-us_topic_0108275374_note5911455114218"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_p52974484409_4">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_text1929717485405_4">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i2814743133910_4">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i118151243173917_4">Offset</em>) rather than (<em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i3816184313913_4">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i1981664317393_4">Offset</em>).</p>
|
||||
<div class="note" id="dataartsstudio_01_0052__en-us_topic_0108275374_note5911455114218"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_p52974484409_4">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_text9997118203_4">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i799871152012_4">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i0998101192020_4">Offset</em>) rather than (<em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i299821132018_4">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0052__en-us_topic_0108275374_en-us_topic_0108275319_i129981917200_4">Offset</em>).</p>
|
||||
</div></div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.5.1.3 "><p id="dataartsstudio_01_0052__en-us_topic_0108275374_p8442749101515">2019-07-30 00:00:00</p>
|
||||
@ -260,7 +260,7 @@
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0047.html">Source Job Parameters</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0047.html">Configuring CDM Source Job Parameters</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
@ -49,9 +49,9 @@
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0053__en-us_topic_0114711714_row1497845915165"><td class="cellrowborder" valign="top" width="14.531453145314533%" headers="mcps1.3.2.2.4.1.1 "><p id="dataartsstudio_01_0053__en-us_topic_0114711714_p529563715165">File Format</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="64.41644164416441%" headers="mcps1.3.2.2.4.1.2 "><p id="dataartsstudio_01_0053__p440214575817">Format used for transmitting data. The CSV and JSON formats are supported for migration to tables, and the binary format is supported for file migration.</p>
|
||||
<td class="cellrowborder" valign="top" width="64.41644164416441%" headers="mcps1.3.2.2.4.1.2 "><p id="dataartsstudio_01_0053__en-us_topic_0114711714_p440214575817">Format used for transmitting data. The CSV and JSON formats are supported for migration to tables, and the binary format is supported for file migration.</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="21.052105210521052%" headers="mcps1.3.2.2.4.1.3 "><p id="dataartsstudio_01_0053__p3753014815165">Binary</p>
|
||||
<td class="cellrowborder" valign="top" width="21.052105210521052%" headers="mcps1.3.2.2.4.1.3 "><p id="dataartsstudio_01_0053__en-us_topic_0114711714_p3753014815165">Binary</p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0053__en-us_topic_0114711714_row195259453167"><td class="cellrowborder" valign="top" width="14.531453145314533%" headers="mcps1.3.2.2.4.1.1 "><p id="dataartsstudio_01_0053__en-us_topic_0114711714_p5526745191614">Compression Format</p>
|
||||
@ -98,13 +98,20 @@
|
||||
<td class="cellrowborder" valign="top" width="21.052105210521052%" headers="mcps1.3.2.2.4.1.3 "><p id="dataartsstudio_01_0053__en-us_topic_0114711714_p267718319321">.md5</p>
|
||||
</td>
|
||||
</tr>
|
||||
<tr id="dataartsstudio_01_0053__en-us_topic_0114711714_row03971236135910"><td class="cellrowborder" valign="top" width="14.531453145314533%" headers="mcps1.3.2.2.4.1.1 "><p id="dataartsstudio_01_0053__en-us_topic_0114711714_p102221939115916">Query Parameter</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="64.41644164416441%" headers="mcps1.3.2.2.4.1.2 "><p id="dataartsstudio_01_0053__en-us_topic_0114711714_p939703610592">If this parameter is set to <strong id="dataartsstudio_01_0053__en-us_topic_0114711714_b71231711116">Yes</strong>, the name of the object to be uploaded is a string with the query parameter removed.</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" width="21.052105210521052%" headers="mcps1.3.2.2.4.1.3 "><p id="dataartsstudio_01_0053__en-us_topic_0114711714_p17397183655915">No</p>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0047.html">Source Job Parameters</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0047.html">Configuring CDM Source Job Parameters</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
@ -25,8 +25,8 @@
|
||||
<tr id="dataartsstudio_01_0054__en-us_topic_0108275324_row1122012394320"><td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.1 "><p id="dataartsstudio_01_0054__en-us_topic_0108275324_p92201739139">SQL Statement</p>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.2 "><p id="dataartsstudio_01_0054__en-us_topic_0108275324_p5220739135">When <span class="parmname" id="dataartsstudio_01_0054__en-us_topic_0108275324_parmname13725155517269"><b>Use SQL Statement</b></span> is set to <span class="parmvalue" id="dataartsstudio_01_0054__en-us_topic_0108275324_parmvalue545821122713"><b>Yes</b></span>, enter an SQL statement here. CDM exports data based on the SQL statement.</p>
|
||||
<div class="note" id="dataartsstudio_01_0054__en-us_topic_0108275324_note1624112587564"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dataartsstudio_01_0054__en-us_topic_0108275324_ul95781925125319"><li id="dataartsstudio_01_0054__en-us_topic_0108275324_li15578132545313">SQL statements can only be used to query data. Join and nesting are supported, but multiple query statements are not allowed, for example, <strong id="dataartsstudio_01_0054__en-us_topic_0108275324_b143067633772251">select * from table a; select * from table b</strong>.</li><li id="dataartsstudio_01_0054__en-us_topic_0108275324_li115789258538">With statements are not supported.</li><li id="dataartsstudio_01_0054__en-us_topic_0108275324_li15578152525319">Comments, such as <strong id="dataartsstudio_01_0054__en-us_topic_0108275324_b148619239372251">--</strong> and <strong id="dataartsstudio_01_0054__en-us_topic_0108275324_b187717313372251">/*</strong>, are not supported.</li><li id="dataartsstudio_01_0054__en-us_topic_0108275324_li115785251536">Addition, deletion, and modification operations are not supported, including but not limited to the following:<ul id="dataartsstudio_01_0054__en-us_topic_0108275324_ul185084210539"><li id="dataartsstudio_01_0054__en-us_topic_0108275324_li1623929145315">load data</li><li id="dataartsstudio_01_0054__en-us_topic_0108275324_li15832356155310">delete from</li><li id="dataartsstudio_01_0054__en-us_topic_0108275324_li144890175411">alter table</li><li id="dataartsstudio_01_0054__en-us_topic_0108275324_li17896205419">create table</li><li id="dataartsstudio_01_0054__en-us_topic_0108275324_li7715510105412">drop table</li><li id="dataartsstudio_01_0054__en-us_topic_0108275324_li548061418543">into outfile</li></ul>
|
||||
</li></ul>
|
||||
<div class="note" id="dataartsstudio_01_0054__en-us_topic_0108275324_note1624112587564"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275424_ul95781925125319"><li id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275424_li15578132545313">SQL statements can only be used to query data. Join and nesting are supported, but multiple query statements are not allowed, for example, <strong id="dataartsstudio_01_0054__en-us_topic_0108275324_b143067633772251">select * from table a; select * from table b</strong>.</li><li id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275424_li115789258538">With statements are not supported.</li><li id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275424_li15578152525319">Comments, such as <strong id="dataartsstudio_01_0054__en-us_topic_0108275324_b148619239372251">--</strong> and <strong id="dataartsstudio_01_0054__en-us_topic_0108275324_b187717313372251">/*</strong>, are not supported.</li><li id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275424_li115785251536">Addition, deletion, and modification operations are not supported, including but not limited to the following:<ul id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275424_ul185084210539"><li id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275424_li1623929145315">load data</li><li id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275424_li15832356155310">delete from</li><li id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275424_li144890175411">alter table</li><li id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275424_li17896205419">create table</li><li id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275424_li7715510105412">drop table</li><li id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275424_li548061418543">into outfile</li></ul>
|
||||
</li><li id="dataartsstudio_01_0054__en-us_topic_0108275324_li1517424293819">If the SQL statement is too long, the request fails to be delivered. If you continue to create a job, the system displays an error message indicating that the request is incorrect. In this case, you need to simplify or clear the SQL statement and try again.</li></ul>
|
||||
</div></div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.3 "><p id="dataartsstudio_01_0054__en-us_topic_0108275324_p6220439735">select id,name from sqoop.user;</p>
|
||||
@ -48,7 +48,7 @@
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.2 "><p id="dataartsstudio_01_0054__en-us_topic_0108275324_p6187019816562">Name of the table from which data will be extracted. This parameter is displayed when <strong id="dataartsstudio_01_0054__en-us_topic_0108275324_b1452052643818">Use SQL Statement</strong> is set to <strong id="dataartsstudio_01_0054__en-us_topic_0108275324_b5520526113810">No</strong>. Click the icon next to the text box to go to the page for selecting the table or directly enter a table name.</p>
|
||||
<p id="dataartsstudio_01_0054__en-us_topic_0108275324_p49990086164128">If the desired table is not displayed, check whether the table exists or whether the login account has the permission to query metadata.</p>
|
||||
<p id="dataartsstudio_01_0054__en-us_topic_0108275324_p1386724520614">This parameter can be configured as a macro variable of date and time and a path name can contain multiple macro variables. When the macro variable of date and time works with a scheduled job, the incremental data can be synchronized periodically. </p>
|
||||
<div class="note" id="dataartsstudio_01_0054__en-us_topic_0108275324_note13940218801"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275319_p52974484409">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275319_text1929717485405">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275319_i2814743133910">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275319_i118151243173917">Offset</em>) rather than (<em id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275319_i3816184313913">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275319_i1981664317393">Offset</em>).</p>
|
||||
<div class="note" id="dataartsstudio_01_0054__en-us_topic_0108275324_note13940218801"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275319_p52974484409">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275319_text9997118203">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275319_i799871152012">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275319_i0998101192020">Offset</em>) rather than (<em id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275319_i299821132018">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275319_i129981917200">Offset</em>).</p>
|
||||
</div></div>
|
||||
<div class="note" id="dataartsstudio_01_0054__en-us_topic_0108275324_note5966122551119"><span class="notetitle"> NOTE: </span><div class="notebody"><div class="p" id="dataartsstudio_01_0054__en-us_topic_0108275324_p2522183912523">The table name can contain wildcard characters (*), which is used to export all tables whose names start with a certain prefix or end with a certain suffix. The number and types of fields in the tables must be the same. The examples are as follows:<ul id="dataartsstudio_01_0054__en-us_topic_0108275324_ul12793194217522"><li id="dataartsstudio_01_0054__en-us_topic_0108275324_li2079344215522"><strong id="dataartsstudio_01_0054__en-us_topic_0108275324_b9764102344710">table*</strong> indicates that all tables whose names starting with <span class="uicontrol" id="dataartsstudio_01_0054__en-us_topic_0108275324_uicontrol5206217536"><b>table</b></span> are exported.</li><li id="dataartsstudio_01_0054__en-us_topic_0108275324_li7991290534"><strong id="dataartsstudio_01_0054__en-us_topic_0108275324_b11991169185314">*table</strong> indicates that all tables whose names ending with <span class="uicontrol" id="dataartsstudio_01_0054__en-us_topic_0108275324_uicontrol16951252175715"><b>table</b></span> are exported.</li><li id="dataartsstudio_01_0054__en-us_topic_0108275324_li1999159155314"><strong id="dataartsstudio_01_0054__en-us_topic_0108275324_b5991189165316">*table</strong><strong id="dataartsstudio_01_0054__en-us_topic_0108275324_b3991209185319">*</strong> indicates that all tables whose names containing <span class="uicontrol" id="dataartsstudio_01_0054__en-us_topic_0108275324_uicontrol221153614538"><b>table</b></span> are exported.</li></ul>
|
||||
</div>
|
||||
@ -75,7 +75,7 @@
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.2 "><p id="dataartsstudio_01_0054__en-us_topic_0108275324_p26422060162021">WHERE clause used to specify the data extraction range. This parameter is displayed when <strong id="dataartsstudio_01_0054__en-us_topic_0108275324_b1488615598483">Use SQL Statement</strong> is set to <strong id="dataartsstudio_01_0054__en-us_topic_0108275324_b35361936496">No</strong>. If this parameter is not set, the entire table is extracted.</p>
|
||||
<p id="dataartsstudio_01_0054__en-us_topic_0108275324_p18740131613716">You can set a date macro variable to extract data generated on a specific date. </p>
|
||||
<div class="note" id="dataartsstudio_01_0054__en-us_topic_0108275324_note45791757134910"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275319_p52974484409_1">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275319_text1929717485405_1">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275319_i2814743133910_1">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275319_i118151243173917_1">Offset</em>) rather than (<em id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275319_i3816184313913_1">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275319_i1981664317393_1">Offset</em>).</p>
|
||||
<div class="note" id="dataartsstudio_01_0054__en-us_topic_0108275324_note45791757134910"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275319_p52974484409_1">If you have configured a macro variable of date and time and schedule a CDM job through <span id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275319_text9997118203_1">DataArts Studio DataArts Factory</span>, the system replaces the macro variable of date and time with (<em id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275319_i799871152012_1">Planned start time of the data development job</em> – <em id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275319_i0998101192020_1">Offset</em>) rather than (<em id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275319_i299821132018_1">Actual start time of the CDM job</em> – <em id="dataartsstudio_01_0054__en-us_topic_0108275324_en-us_topic_0108275319_i129981917200_1">Offset</em>).</p>
|
||||
</div></div>
|
||||
</td>
|
||||
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.5.1.3 "><p id="dataartsstudio_01_0054__en-us_topic_0108275324_p60448746162021">DS='${dateformat(yyyy-MM-dd,-1,DAY)}'</p>
|
||||
@ -139,7 +139,7 @@
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0047.html">Source Job Parameters</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0047.html">Configuring CDM Source Job Parameters</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
@ -48,7 +48,7 @@
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0047.html">Source Job Parameters</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0047.html">Configuring CDM Source Job Parameters</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
@ -61,7 +61,7 @@
|
||||
</div>
|
||||
<div>
|
||||
<div class="familylinks">
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0047.html">Source Job Parameters</a></div>
|
||||
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0047.html">Configuring CDM Source Job Parameters</a></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user