vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/mrs/component-operation-guide-lts/mrs_01_24278.html
Yang, Tong 3f5759eed2 MRS comp-lts 2.0.38.SP20 version 
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com>
Co-authored-by: Yang, Tong <yangtong2@huawei.com>
Co-committed-by: Yang, Tong <yangtong2@huawei.com>
2023-01-19 17:08:45 +00:00

97 lines
14 KiB
HTML
Raw Blame History

<a name="mrs_01_24278"></a><a name="mrs_01_24278"></a>
<h1 class="topictitle1">SET/RESET</h1>
<div id="body8662426"><div class="section" id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_s50a79a00d0464249907123eb396c627d"><h4 class="sectiontitle">Function</h4><p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_abcdff905c48a4d1ba817c07c37eb0594">This command is used to dynamically add, update, display, or reset Hudi parameters without restarting the driver.</p>
</div>
<div class="section" id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_s9b445ccf2739404e9d1a2c450c384942"><h4 class="sectiontitle">Syntax</h4><ul id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_u2ec7ae48b172458bba5e89058447b65f"><li id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_l20f3887ef7d54aa5ae9e3a3661035bb2">Add or update a parameter value:<p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_a0d0b28fb0c0c47f2b890f811a14e50df"><a name="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_l20f3887ef7d54aa5ae9e3a3661035bb2"></a><a name="en-us_topic_0000001173789316_en-us_topic_0265792808_l20f3887ef7d54aa5ae9e3a3661035bb2"></a><b><span class="cmdname" id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_cmdname1591111815208">SET</span></b> <em id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_i635693810208">parameter_name</em>=<em id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_i18357103842015">parameter_value</em></p>
<p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_aabda752be0034bb39a17b7881220df39">This command is used to add or update the value of <span class="parmname" id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_pce42f0de34c3428a9d4afe1054f2af40"><b>parameter_name</b></span>.</p>
</li><li id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_l865e38fbb6c44488be77115c15e14dd3">Display a parameter value:<p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_a5a26a713773d4516b8459566824f8471"><a name="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_l865e38fbb6c44488be77115c15e14dd3"></a><a name="en-us_topic_0000001173789316_en-us_topic_0265792808_l865e38fbb6c44488be77115c15e14dd3"></a><b><span class="cmdname" id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_cmdname15406142211206">SET</span></b> <em id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_i134418449207">parameter_name</em></p>
<p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_a4d59a4074c2249539048f9e27b29d09a">This command is used to display the value of <span class="parmname" id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_p58306084187846ecb00c1f37bc170d14"><b>parameter_name</b></span>.</p>
</li><li id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_lb4eedc6845604429a935466adb2fa27d">Display session parameters:<p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_a4791c317a73a4fc7ac8ce79d19d3d764"><a name="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_lb4eedc6845604429a935466adb2fa27d"></a><a name="en-us_topic_0000001173789316_en-us_topic_0265792808_lb4eedc6845604429a935466adb2fa27d"></a><b><span class="cmdname" id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_cmdname132228287208">SET</span></b></p>
<p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_a073f7cd8dd9d4ff987c342b0418aadb6">This command is used to display all supported session parameters.</p>
</li><li id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_lbfd00437a4194245862d6acec4d66d05">Display session parameters along with usage details:<p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_a26af39c6446b4f08aa394cd67feb3456"><a name="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_lbfd00437a4194245862d6acec4d66d05"></a><a name="en-us_topic_0000001173789316_en-us_topic_0265792808_lbfd00437a4194245862d6acec4d66d05"></a><b><span class="cmdname" id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_cmdname3359134982014">SET</span></b> -v</p>
<p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_a629308ec728349e8b3606dd9149e0eb3">This command is used to display all supported session parameters and their usage details.</p>
</li><li id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_l9ff5ee54ac83483c9d20dc3e5057e45a">Reset parameter values:<p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_a308855b1ecd44412b6e0bf9bcb6e1bb1"><a name="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_l9ff5ee54ac83483c9d20dc3e5057e45a"></a><a name="en-us_topic_0000001173789316_en-us_topic_0265792808_l9ff5ee54ac83483c9d20dc3e5057e45a"></a><b><span class="cmdname" id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_cmdname3577852172020">RESET</span></b></p>
<p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_aae371db389174e7d80a91bb7a823afc1">This command is used to reset all session parameters.</p>
</li></ul>
</div>
<div class="section" id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_s874354d5c2a643c8a61ff676853754ef"><h4 class="sectiontitle">Parameter Description</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_t24d72ee171974750bc56763d1b4994a5" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameters</caption><thead align="left"><tr id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_r150e9b8c930f47638dfd7ab7afed270a"><th align="left" class="cellrowborder" valign="top" width="30%" id="mcps1.3.3.2.2.3.1.1"><p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_a48feb400d1724c45a4759914f57d2957">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="70%" id="mcps1.3.3.2.2.3.1.2"><p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_ae50041b6ac77460e8b78b145e0dff052">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_rfa4122e4f5134925af32bf60b6394a53"><td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.3.2.2.3.1.1 "><p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_a275c70eab8094d72ab6f7bb2a70c1568">parameter_name</p>
</td>
<td class="cellrowborder" valign="top" width="70%" headers="mcps1.3.3.2.2.3.1.2 "><p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_a3dbd111d045d41fb82801b02e1546e1d">Name of the parameter to be dynamically added, updated, or displayed.</p>
</td>
</tr>
<tr id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_r69351225935747b7bdc190f228e8d95c"><td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.3.2.2.3.1.1 "><p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_aee5eaf277eb743bfb0baa0e9b8f3b957">parameter_value</p>
</td>
<td class="cellrowborder" valign="top" width="70%" headers="mcps1.3.3.2.2.3.1.2 "><p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_a10b8717fda9746ef8f5250e2d0b806fd">New value to be set for <span class="parmname" id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_p822304b1443746c09165f377151414dc"><b>parameter_name</b></span>.</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="mrs_01_24278__en-us_topic_0000001173789316_section660371444514"><h4 class="sectiontitle">Precautions</h4><p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_a94c9d5eb89fe497192bb75f4022dcaa3">The following table lists the properties to be used in the <strong id="mrs_01_24278__en-us_topic_0000001173789316_b129802131411">SET</strong> or <strong id="mrs_01_24278__en-us_topic_0000001173789316_b196376101416">RESET</strong> commands.</p>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_t5f527eeacdb14c8b80cb886d6455dbb2" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Properties</caption><thead align="left"><tr id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_r0083b193af5b422788ee292d7c8940f3"><th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.4.3.2.3.1.1"><p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_a43cd6d0ad30e43fea40bee82c301e047">Property</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.4.3.2.3.1.2"><p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_ad04c8cc0a4d944e3b32242aad6cbf5e8">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_r3caf4a439c25496a921a6f356c03a774"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_a1060bf80618e45439f569794bc2a4fab">hoodie.insert.shuffle.parallelism</p>
</td>
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_a7f98d77ec4164767a342fec2b20c23eb">Degree of parallelism (DOP) of Spark shuffle for writing data in insert mode.</p>
</td>
</tr>
<tr id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_rc182a826cf074195aec37efe577153dd"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_a1e883799c9a14764bd316bdc0d210c39">hoodie.upsert.shuffle.parallelism</p>
</td>
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="mrs_01_24278__en-us_topic_0000001173789316_p1740317125414">DOP of Spark shuffle for writing data in upsert mode.</p>
</td>
</tr>
<tr id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_re82042d924364770b71e4949a06ea6ad"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_a86ebfc82a1084bd8bcc5692b5474023b">hoodie.delete.shuffle.parallelism</p>
</td>
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="mrs_01_24278__en-us_topic_0000001173789316_p11168142635413">DOP of Spark shuffle for deleting data in delete mode.</p>
</td>
</tr>
<tr id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_r452b376a1a8648098f4dffccf53c82bd"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_ad475b4ad3002477680b38d3f0cba579a">hoodie.sql.insert.mode</p>
</td>
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_a2a332e2d3ab2476b9df82211f95cf3f9">Insert mode. The value can be strict, non-strict, or upsert.</p>
</td>
</tr>
<tr id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_r34262402716140a99c0bf3d1de66bdd5"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_a1b6bd244db0e409680cfe2ced2797ad0">hoodie.sql.bulk.insert.enable</p>
</td>
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_a27d8c585d9eb4841955c468a31261c3d">Whether to enable bulk insert.</p>
</td>
</tr>
<tr id="mrs_01_24278__en-us_topic_0000001173789316_row62701446132"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="mrs_01_24278__en-us_topic_0000001173789316_p1627020414137">spark.sql.hive.convertMetastoreParquet</p>
</td>
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="mrs_01_24278__en-us_topic_0000001173789316_p1270194141312">Converts the parquet table into a data source table for reading. If the provider of Hudi is Hive and Spark SQL or Spark Beeline is used to read data, set this parameter to <strong id="mrs_01_24278__en-us_topic_0000001173789316_b638191115492">false</strong>.</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_s0e7b22986c084aebb0a4c962c202121f"><h4 class="sectiontitle">Examples</h4><ul id="mrs_01_24278__en-us_topic_0000001173789316_ul974141984711"><li id="mrs_01_24278__en-us_topic_0000001173789316_li147411119154717">Add or Update command:<pre class="screen" id="mrs_01_24278__en-us_topic_0000001173789316_screen103041471016">set hoodie.insert.shuffle.parallelism = 100;
set hoodie.upsert.shuffle.parallelism = 100;
set hoodie.delete.shuffle.parallelism = 100;</pre>
</li></ul>
<ul id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_u48677368522e476b9e079b83e2b5aea8"><li id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_lad81debf083840bd884c6925144866e2">Reset command:<pre class="screen" id="mrs_01_24278__en-us_topic_0000001173789316_screen168681911514">RESET</pre>
</li></ul>
</div>
<div class="section" id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_s1bb5666182f149baa0f41ec36ebc85fc"><h4 class="sectiontitle">System Response</h4><ul id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_u0f27508b68204df7bf2e912db202d649"><li id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_lfabc560ae0614945a1ea107cf852d692">You can view the success result in driver logs.</li><li id="mrs_01_24278__en-us_topic_0000001173789316_en-us_topic_0265792808_l9834e6956bf24a05a9a5873fb69fc7f4">You can view the failure result on the UI.</li></ul>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_24272.html">DML</a></div>
</div>
</div>
View Git Blame Copy Permalink