forked from docs/doc-exports
chenxiaoxiong
f9e2808b7c
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: chenxiaoxiong <chenxiaoxiong@huawei.com> Co-committed-by: chenxiaoxiong <chenxiaoxiong@huawei.com>
110 lines
14 KiB
HTML
110 lines
14 KiB
HTML
<a name="dataartsstudio_01_0538"></a><a name="dataartsstudio_01_0538"></a>
|
|
|
|
<h1 class="topictitle1">Kafka Client</h1>
|
|
<div id="body8662426"><div class="section" id="dataartsstudio_01_0538__en-us_topic_0235670394_section44280035173841"><h4 class="sectiontitle">Functions</h4><p id="dataartsstudio_01_0538__en-us_topic_0235670394_p1931416566377">The Kafka Client node is used to send data to Kafka topics.</p>
|
|
<p id="dataartsstudio_01_0538__p777773321720">You can learn more about how to use the Kafka Client node in .</p>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_01_0538__en-us_topic_0235670394_section5516152714271"><h4 class="sectiontitle">Parameters</h4><p id="dataartsstudio_01_0538__en-us_topic_0235670394_p49252448103256"><a href="#dataartsstudio_01_0538__en-us_topic_0235670394_table3764823994826">Table 1</a> describes the parameters of the Kafka Client node.</p>
|
|
|
|
<div class="tablenoborder"><a name="dataartsstudio_01_0538__en-us_topic_0235670394_table3764823994826"></a><a name="en-us_topic_0235670394_table3764823994826"></a><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_01_0538__en-us_topic_0235670394_table3764823994826" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameters of Kafka Client nodes</caption><thead align="left"><tr id="dataartsstudio_01_0538__en-us_topic_0235670394_row3170822394826"><th align="left" class="cellrowborder" valign="top" width="21.740000000000002%" id="mcps1.3.2.3.2.4.1.1"><p id="dataartsstudio_01_0538__en-us_topic_0235670394_p2984581994826">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="10.27%" id="mcps1.3.2.3.2.4.1.2"><p id="dataartsstudio_01_0538__en-us_topic_0235670394_p159227094826">Mandatory</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="67.99%" id="mcps1.3.2.3.2.4.1.3"><p id="dataartsstudio_01_0538__en-us_topic_0235670394_p6186505494826">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dataartsstudio_01_0538__en-us_topic_0235670394_row13745168174815"><td class="cellrowborder" valign="top" width="21.740000000000002%" headers="mcps1.3.2.3.2.4.1.1 "><p id="dataartsstudio_01_0538__en-us_topic_0235670394_p77451280481">Data Connection</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.27%" headers="mcps1.3.2.3.2.4.1.2 "><p id="dataartsstudio_01_0538__en-us_topic_0235670394_p1274618815488">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="67.99%" headers="mcps1.3.2.3.2.4.1.3 "><p id="dataartsstudio_01_0538__en-us_topic_0235670394_p13746281480">Select the MRS Kafka connection created in the management center.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0538__en-us_topic_0235670394_row1669516214485"><td class="cellrowborder" valign="top" width="21.740000000000002%" headers="mcps1.3.2.3.2.4.1.1 "><p id="dataartsstudio_01_0538__en-us_topic_0235670394_p1169572112481">Topic Name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.27%" headers="mcps1.3.2.3.2.4.1.2 "><p id="dataartsstudio_01_0538__en-us_topic_0235670394_p5695172113486">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="67.99%" headers="mcps1.3.2.3.2.4.1.3 "><p id="dataartsstudio_01_0538__en-us_topic_0235670394_p146951721144811">Select the topic to which data is to be uploaded. If there are multiple partitions, data is sent to partition 0 by default.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0538__en-us_topic_0235670394_row1991457694826"><td class="cellrowborder" valign="top" width="21.740000000000002%" headers="mcps1.3.2.3.2.4.1.1 "><p id="dataartsstudio_01_0538__en-us_topic_0235670394_p246794194826">Node Name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.27%" headers="mcps1.3.2.3.2.4.1.2 "><p id="dataartsstudio_01_0538__en-us_topic_0235670394_p6568554794826">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="67.99%" headers="mcps1.3.2.3.2.4.1.3 "><p id="dataartsstudio_01_0538__en-us_topic_0235670394_p1892909794826">Name of a <span id="dataartsstudio_01_0538__en-us_topic_0099822521_text44323307153939">node</span>. The name must contain 1 to 128 characters, including only letters, numbers, underscores (_), hyphens (-), slashes (/), less-than signs (<), and greater-than signs (>).</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0538__en-us_topic_0235670394_row303485991116"><td class="cellrowborder" valign="top" width="21.740000000000002%" headers="mcps1.3.2.3.2.4.1.1 "><p id="dataartsstudio_01_0538__en-us_topic_0235670394_p47481414237">Text</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="10.27%" headers="mcps1.3.2.3.2.4.1.2 "><p id="dataartsstudio_01_0538__en-us_topic_0235670394_p51613311116">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="67.99%" headers="mcps1.3.2.3.2.4.1.3 "><p id="dataartsstudio_01_0538__en-us_topic_0235670394_p2602055553">Text content sent to Kafka. You can directly enter text or click <span><img id="dataartsstudio_01_0538__en-us_topic_0235670394_image176782036121314" src="en-us_image_0000002269122377.png"></span> to use the EL expression.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_01_0538__en-us_topic_0235670394_table16592383540" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Advanced parameters</caption><thead align="left"><tr id="dataartsstudio_01_0538__en-us_topic_0099822521_row9846111555118"><th align="left" class="cellrowborder" valign="top" width="28.07%" id="mcps1.3.3.2.4.1.1"><p id="dataartsstudio_01_0538__en-us_topic_0099822521_p2846515195115">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="15.659999999999998%" id="mcps1.3.3.2.4.1.2"><p id="dataartsstudio_01_0538__en-us_topic_0099822521_p108461215185110">Mandatory</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="56.269999999999996%" id="mcps1.3.3.2.4.1.3"><p id="dataartsstudio_01_0538__en-us_topic_0099822521_p1484719153511">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dataartsstudio_01_0538__en-us_topic_0099822521_row18847141515512"><td class="cellrowborder" valign="top" width="28.07%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0538__en-us_topic_0099822521_p2847181535113">Max. Node Execution Duration</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.659999999999998%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0538__en-us_topic_0099822521_p15847171511512">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.269999999999996%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0538__en-us_topic_0099822521_p1884761565119">Execution timeout interval for the <span id="dataartsstudio_01_0538__en-us_topic_0099822521_text1344611820218">node</span>. If retry is configured and the execution is not complete within the timeout interval, the <span id="dataartsstudio_01_0538__en-us_topic_0099822521_text8447488212">node</span> will be executed again.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0538__en-us_topic_0099822521_row19847181555112"><td class="cellrowborder" valign="top" width="28.07%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0538__en-us_topic_0099822521_p12847815125117">Retry upon Failure</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.659999999999998%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0538__en-us_topic_0099822521_p8847161516511">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.269999999999996%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0538__en-us_topic_0099822521_p684761514516">Whether to re-execute a <span id="dataartsstudio_01_0538__en-us_topic_0099822521_text68471415185118">node</span> if it fails to be executed. Possible values:</p>
|
|
<ul id="dataartsstudio_01_0538__en-us_topic_0099822521_ul18479151514"><li id="dataartsstudio_01_0538__en-us_topic_0099822521_li148481915205115"><strong id="dataartsstudio_01_0538__en-us_topic_0099822521_b692668954">Yes</strong>: The <span id="dataartsstudio_01_0538__en-us_topic_0099822521_text184861512517">node</span> will be re-executed, and the following parameters must be configured:<ul id="dataartsstudio_01_0538__en-us_topic_0099822521_ul284811151511"><li id="dataartsstudio_01_0538__en-us_topic_0099822521_li1927319416429"><strong id="dataartsstudio_01_0538__en-us_topic_0099822521_b11288652181717">Retry upon Timeout</strong></li><li id="dataartsstudio_01_0538__en-us_topic_0099822521_li1584811515119"><strong id="dataartsstudio_01_0538__en-us_topic_0099822521_b1150205942">Maximum Retries</strong></li><li id="dataartsstudio_01_0538__en-us_topic_0099822521_li1184841512511"><strong id="dataartsstudio_01_0538__en-us_topic_0099822521_b910983280">Retry Interval (seconds)</strong></li></ul>
|
|
</li><li id="dataartsstudio_01_0538__en-us_topic_0099822521_li1884851535115"><strong id="dataartsstudio_01_0538__en-us_topic_0099822521_b8133175784614">No</strong>: The <span id="dataartsstudio_01_0538__en-us_topic_0099822521_text5848215145116">node</span> will not be re-executed. This is the default setting.<div class="note" id="dataartsstudio_01_0538__en-us_topic_0099822521_note112261354122511"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dataartsstudio_01_0538__en-us_topic_0099822521_p1722635418254">If retry is configured for a job node and the timeout duration is configured, the system allows you to retry a node when the node execution times out.</p>
|
|
<p id="dataartsstudio_01_0538__en-us_topic_0099822521_p1690217201682">If a node is not re-executed when it fails upon timeout, you can go to the <strong id="dataartsstudio_01_0538__en-us_topic_0099822521_b14924182735216">Default Configuration</strong> page to modify this policy.</p>
|
|
<p id="dataartsstudio_01_0538__en-us_topic_0099822521_p979555414426"><strong id="dataartsstudio_01_0538__en-us_topic_0099822521_b11436145811174">Retry upon Timeout</strong> is displayed only when <strong id="dataartsstudio_01_0538__en-us_topic_0099822521_b64361558201713">Retry upon Failure</strong> is set to <strong id="dataartsstudio_01_0538__en-us_topic_0099822521_b34366589176">Yes</strong>.</p>
|
|
</div></div>
|
|
</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0538__en-us_topic_0099822521_row148481015115110"><td class="cellrowborder" valign="top" width="28.07%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0538__en-us_topic_0099822521_p168481315165110">Policy for Handling Subsequent Nodes If the Current Node Fails</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.659999999999998%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0538__en-us_topic_0099822521_p7848181515114">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.269999999999996%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0538__en-us_topic_0099822521_p1848915165110">Operation that will be performed if the <span id="dataartsstudio_01_0538__en-us_topic_0099822521_text184871517513">node</span> fails to be executed. Possible values:</p>
|
|
<ul id="dataartsstudio_01_0538__en-us_topic_0099822521_ul684811155518"><li id="dataartsstudio_01_0538__en-us_topic_0099822521_en-us_topic_0099822521_li867222192212"><strong id="dataartsstudio_01_0538__en-us_topic_0099822521_en-us_topic_0099822521_b63511334183">Suspend execution plans of the subsequent nodes</strong>: stops running subsequent nodes. The job instance status is <strong id="dataartsstudio_01_0538__en-us_topic_0099822521_en-us_topic_0099822521_b635117371819">Failed</strong>.</li><li id="dataartsstudio_01_0538__en-us_topic_0099822521_en-us_topic_0099822521_li2533844102858"><strong id="dataartsstudio_01_0538__en-us_topic_0099822521_en-us_topic_0099822521_b31833716587">End the current job execution plan</strong>: stops running the current job. The job instance status is <strong id="dataartsstudio_01_0538__en-us_topic_0099822521_en-us_topic_0099822521_b1521193765820">Failed</strong>.</li><li id="dataartsstudio_01_0538__en-us_topic_0099822521_en-us_topic_0099822521_li22804597102858"><strong id="dataartsstudio_01_0538__en-us_topic_0099822521_en-us_topic_0099822521_b72771223151916">Go to the next node</strong>: ignores the execution failure of the current node. The job instance status is <strong id="dataartsstudio_01_0538__en-us_topic_0099822521_en-us_topic_0099822521_b1727710236199">Failure ignored</strong>.</li><li id="dataartsstudio_01_0538__en-us_topic_0099822521_en-us_topic_0099822521_li1657745411173"><strong id="dataartsstudio_01_0538__en-us_topic_0099822521_en-us_topic_0099822521_b99581526191817">Suspend the current job execution plan</strong>: If the current job instance is in abnormal state, the subsequent nodes of this node and the subsequent job instances that depend on the current job are in waiting state.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0538__en-us_topic_0099822521_row443311414209"><td class="cellrowborder" valign="top" width="28.07%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0538__en-us_topic_0099822521_p6433154115208">Enable Dry Run</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.659999999999998%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0538__en-us_topic_0099822521_p134343417207">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.269999999999996%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0538__en-us_topic_0099822521_p9477175317207">If you select this option, the node will not be executed, and a success message will be returned.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_01_0538__en-us_topic_0099822521_row127470182515"><td class="cellrowborder" valign="top" width="28.07%" headers="mcps1.3.3.2.4.1.1 "><p id="dataartsstudio_01_0538__en-us_topic_0099822521_p1027414042510">Task Groups</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.659999999999998%" headers="mcps1.3.3.2.4.1.2 "><p id="dataartsstudio_01_0538__en-us_topic_0099822521_p1827419013254">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="56.269999999999996%" headers="mcps1.3.3.2.4.1.3 "><p id="dataartsstudio_01_0538__en-us_topic_0099822521_p4881175711254">Select a task group. If you select a task group, you can control the maximum number of concurrent nodes in the task group in a fine-grained manner in scenarios where a job contains multiple nodes, a data patching task is ongoing, or a job is rerunning.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_01_0441.html">Node Reference</a></div>
|
|
</div>
|
|
</div>
|
|
|