forked from docs/doc-exports
chenxiaoxiong
f9e2808b7c
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: chenxiaoxiong <chenxiaoxiong@huawei.com> Co-committed-by: chenxiaoxiong <chenxiaoxiong@huawei.com>
113 lines
16 KiB
HTML
113 lines
16 KiB
HTML
<a name="dataartsstudio_01_0184"></a><a name="dataartsstudio_01_0184"></a>
|
|
|
|
<h1 class="topictitle1">MRS Hudi Link Parameters</h1>
|
|
<div id="body8662426"><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p15819545445"><a href="#dataartsstudio_01_0184__en-us_topic_0000001351701141_table121614820218">Table 1</a> describes the MRS Hudi link parameters.</p>
|
|
<div class="note" id="dataartsstudio_01_0184__en-us_topic_0000001351701141_note8867140110"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0108275292_p167832113185">Do not change the password or user when the job is running. If you do so, the password will not take effect immediately and the job will fail.</p>
|
|
</div></div>
|
|
|
|
<div class="tablenoborder"><a name="dataartsstudio_01_0184__en-us_topic_0000001351701141_table121614820218"></a><a name="en-us_topic_0000001351701141_table121614820218"></a><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_01_0184__en-us_topic_0000001351701141_table121614820218" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Hudi link parameters</caption><thead align="left"><tr id="dataartsstudio_01_0184__en-us_topic_0000001351701141_row63349812215"><th align="left" class="cellrowborder" valign="top" width="16.509999999999998%" id="mcps1.3.3.2.4.1.1"><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p13351181028">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="56.25%" id="mcps1.3.3.2.4.1.2"><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p6335781725">Description</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="27.24%" id="mcps1.3.3.2.4.1.3"><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p18335118329">Example Value</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dataartsstudio_01_0184__en-us_topic_0000001351701141_row2033548625"><td class="cellrowborder" valign="top" width="16.509999999999998%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p1433568722">Name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.25%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p1133510814210">Link name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="27.24%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p433518814214">Hudilink</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0184__en-us_topic_0000001351701141_row83351181529"><td class="cellrowborder" valign="top" width="16.509999999999998%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p7335108223">Manager IP</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.25%" headers="mcps1.3.3.2.4.1.2 "><div class="p" id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p218148165010">Floating IP address of MRS Manager. Click <span class="uicontrol" id="dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0108275286_uicontrol578864992012"><b>Select</b></span> next to the <strong id="dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0108275286_b2078834920203">Manager IP</strong> text box to select an MRS cluster. CDM automatically fills in the authentication information.<div class="note" id="dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0108275286_note1340014473413"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0108275286_en-us_topic_0182566327_p116249795810"><span id="dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0108275286_en-us_topic_0182566327_text7355181825814">DataArts Studio</span> does not support MRS clusters whose Kerberos encryption type is <strong id="dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0108275286_en-us_topic_0182566327_b143479402717">aes256-sha2,aes128-sha2</strong>, and only supports MRS clusters whose Kerberos encryption type is <strong id="dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0108275286_en-us_topic_0182566327_b19532103219813">aes256-sha1,aes128-sha1</strong>.</p>
|
|
</div></div>
|
|
</div>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="27.24%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p718110815509">127.0.0.1</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0184__en-us_topic_0000001351701141_row17336168225"><td class="cellrowborder" valign="top" width="16.509999999999998%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p20336888213">Authentication Method</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.25%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p633658428">Authentication method used for accessing MRS</p>
|
|
<ul id="dataartsstudio_01_0184__en-us_topic_0000001351701141_ul14273425329"><li id="dataartsstudio_01_0184__en-us_topic_0000001351701141_li2027318255219"><strong id="dataartsstudio_01_0184__en-us_topic_0000001351701141_b95789612772246">SIMPLE</strong>: Select this for non-security mode.</li><li id="dataartsstudio_01_0184__en-us_topic_0000001351701141_li227317251727"><strong id="dataartsstudio_01_0184__en-us_topic_0000001351701141_b173812918172246">KERBEROS</strong>: Select this for security mode.</li></ul>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="27.24%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p5336481422">KERBEROS</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0184__en-us_topic_0000001351701141_row1733617815215"><td class="cellrowborder" valign="top" width="16.509999999999998%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p1336581214">Account</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.25%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p233612817216">Username for logging in to MRS Manager</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="27.24%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p53361989210">cdm</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0184__en-us_topic_0000001351701141_row203373818217"><td class="cellrowborder" valign="top" width="16.509999999999998%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p17337138321">Password</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.25%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p113371681721">Password for logging in to MRS Manager</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="27.24%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p1233713817215">-</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0184__en-us_topic_0000001351701141_row73371284220"><td class="cellrowborder" valign="top" width="16.509999999999998%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p113371482025">OBS storage support</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.25%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p18337208827">Whether to support OBS storage. If the Hudi table data is stored in OBS, you need to enable this function.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="27.24%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p19337381926">Yes</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0184__en-us_topic_0000001351701141_row53371281127"><td class="cellrowborder" valign="top" width="16.509999999999998%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p15337881224">AK</p>
|
|
</td>
|
|
<td class="cellrowborder" rowspan="2" valign="top" width="56.25%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p1958111399553">This parameter is available when <strong id="dataartsstudio_01_0184__en-us_topic_0000001351701141_b888232816269">OBS storage support</strong> is set to <strong id="dataartsstudio_01_0184__en-us_topic_0000001351701141_b470617450269">Yes</strong>.</p>
|
|
<p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p173371581120">AK and SK are used to log in to the OBS server.</p>
|
|
<p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p11337681920">You need to create an access key for the current account and obtain an AK/SK pair.</p>
|
|
<div class="p" id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p13152215453">To obtain an access key, perform the following steps:<ol id="dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0000001129241845_en-us_topic_0183643042_ol535183018588"><li id="dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0000001129241845_en-us_topic_0183643042_li1535103025819">Log in to the management console, move the cursor to the username in the upper right corner, and select <strong id="dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0000001129241845_b1586023610364">My Credentials</strong> from the drop-down list.</li><li id="dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0000001129241845_en-us_topic_0183643042_li173533018584">On the <strong id="dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0000001129241845_b149721128172418">My Credentials</strong> page, choose <strong id="dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0000001129241845_b15803656132414">Access Keys</strong>, and click <strong id="dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0000001129241845_b2452152020258">Create Access Key</strong>. See <a href="#dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615">Figure 1</a>.<div class="fignone" id="dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615"><a name="dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615"></a><a name="en-us_topic_0000001351701141_en-us_topic_0000001129241845_en-us_topic_0183643042_fig1552229194615"></a><span class="figcap"><b>Figure 1 </b>Clicking Create Access Key</span><br><span><img id="dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0000001129241845_image20389043111611" src="en-us_image_0000002269194761.png" title="Click to enlarge" class="imgResize"></span></div>
|
|
</li><li id="dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0000001129241845_en-us_topic_0183643042_li1535530185815">Click <strong id="dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0000001129241845_b8340319122819">OK</strong> and save the access key file as prompted. The access key file will be saved to your browser's configured download location. Open the <strong id="dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0000001129241845_b1940852512813">credentials.csv</strong> file to view <strong id="dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0000001129241845_b191574128718">Access Key Id</strong> and <strong id="dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0000001129241845_b175769149718">Secret Access Key</strong>.<div class="note" id="dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0000001129241845_note4554158201"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0000001129241845_ul25541081906"><li id="dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0000001129241845_li175541819018">Only two access keys can be added for each user.</li><li id="dataartsstudio_01_0184__en-us_topic_0000001351701141_en-us_topic_0000001129241845_li3554138305">To ensure access key security, the access key is automatically downloaded only when it is generated for the first time and cannot be obtained from the management console later. Keep them properly.</li></ul>
|
|
</div></div>
|
|
</li></ol>
|
|
</div>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="27.24%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p133371083211">-</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0184__en-us_topic_0000001351701141_row18338081720"><td class="cellrowborder" valign="top" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p13381782020">SK</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p193381812213">-</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0184__en-us_topic_0000001351701141_row1371655917295"><td class="cellrowborder" valign="top" width="16.509999999999998%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p631111818337">OBS Test Path</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.25%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p202041270541">This parameter is available when <strong id="dataartsstudio_01_0184__en-us_topic_0000001351701141_b102592058182610">OBS storage support</strong> is set to <strong id="dataartsstudio_01_0184__en-us_topic_0000001351701141_b825910585263">Yes</strong>.</p>
|
|
<p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p1717195916295">Enter a complete file path. The permission to access the path will be verified through the metadata query API.</p>
|
|
<div class="note" id="dataartsstudio_01_0184__en-us_topic_0000001351701141_note65991855143116"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dataartsstudio_01_0184__en-us_topic_0000001351701141_ul244217243311"><li id="dataartsstudio_01_0184__en-us_topic_0000001351701141_li644215263313">For object storage, the path must be accurate to object, for example, <span class="parmvalue" id="dataartsstudio_01_0184__en-us_topic_0000001351701141_parmvalue675216516291"><b>obs://bucket/dir/test.txt</b></span>. Otherwise, a 404 error occurs.</li><li id="dataartsstudio_01_0184__en-us_topic_0000001351701141_li17442142103311">For a parallel file system, the path must be accurate to directory, for example, <span class="parmvalue" id="dataartsstudio_01_0184__en-us_topic_0000001351701141_parmvalue28871145133218"><b>obs://bucket/dir</b></span>.</li></ul>
|
|
</div></div>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="27.24%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p137171159192913">obs://bucket/dir/test.txt</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0184__en-us_topic_0000001351701141_row113381819212"><td class="cellrowborder" valign="top" width="16.509999999999998%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p73381682026">Hive Properties</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.25%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p1833813816213">Names of the tables to be integrated. Use commas (,) to separate multiple table names. This parameter is mandatory and cannot contain spaces.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="27.24%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0184__en-us_topic_0000001351701141_p13389817212">-</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0157.html">Configuring Link Parameters</a></div>
|
|
</div>
|
|
</div>
|
|
|
|
|
|
<script language="JavaScript">
|
|
<!--
|
|
initImageViewer('.imgResize');
|
|
var msg_imageMax = "view original image";
|
|
var msg_imageClose = "close";
|
|
//--></script> |