forked from docs/doc-exports
yangtong
c285e88a17
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: yangtong <yangtong2@huawei.com> Co-committed-by: yangtong <yangtong2@huawei.com>
47 lines
13 KiB
HTML
47 lines
13 KiB
HTML
<a name="admin_guide_000413"></a><a name="admin_guide_000413"></a>
|
|
|
|
<h1 class="topictitle1">Configuring ClickHouse SQL Inspection</h1>
|
|
<div id="body0000002007715621"><div class="section" id="admin_guide_000413__en-us_topic_0000001674340529_section24814312328"><h4 class="sectiontitle">Scenario</h4><p id="admin_guide_000413__en-us_topic_0000001674340529_p54811033326">You can configure rules for ClickHouse SQL inspection on MRS Manager and configure rule parameters as you need.</p>
|
|
</div>
|
|
<div class="section" id="admin_guide_000413__en-us_topic_0000001674340529_section1148112353213"><h4 class="sectiontitle">Prerequisites</h4><ul id="admin_guide_000413__en-us_topic_0000001674340529_ul529883419233"><li id="admin_guide_000413__en-us_topic_0000001674340529_li192982034172313">The cluster client that contains the ClickHouse service has been installed in the <strong id="admin_guide_000413__en-us_topic_0000001674340529_b148431533172620">/opt/hadoopclient</strong> directory.</li><li id="admin_guide_000413__en-us_topic_0000001674340529_li829814346234">The ClickHouse logical cluster is running properly.</li><li id="admin_guide_000413__en-us_topic_0000001674340529_li83361852483">For clusters with Kerberos authentication enabled, you need to create a service user who has the permission to operate the ClickHouse table. For example, create a human-machine user <strong id="admin_guide_000413__b17559195818252">clickhouseuser</strong>.</li><li id="admin_guide_000413__en-us_topic_0000001674340529_li59711633109">A tenant associated with the ClickHouse service has been created and associated with the ClickHouse service user. For details, see <a href="admin_guide_000100.html">Creating Tenants</a></li></ul>
|
|
</div>
|
|
<div class="section" id="admin_guide_000413__en-us_topic_0000001674340529_section1248116313328"><h4 class="sectiontitle">Constraints</h4><ul id="admin_guide_000413__en-us_topic_0000001674340529_ul1544612292247"><li id="admin_guide_000413__en-us_topic_0000001674340529_li044611299247">The default dynamic validity period of a rule is 1 minute.</li><li id="admin_guide_000413__en-us_topic_0000001674340529_li54461029162415">Interception and blocking rules will interrupt SQL queries, so you need to set parameters of these rules properly based on the site requirements.</li><li id="admin_guide_000413__en-us_topic_0000001674340529_li156151152173517">After configuring ClickHouse rules, you need to log in to the client again for the rules to take effect.</li></ul>
|
|
</div>
|
|
<div class="section" id="admin_guide_000413__en-us_topic_0000001674340529_section73635164318"><h4 class="sectiontitle">Procedure</h4><ol id="admin_guide_000413__en-us_topic_0000001674340529_ol1748143173218"><li id="admin_guide_000413__en-us_topic_0000001674340529_li86364538219"><span>Log in to MRS Manager, click <strong id="admin_guide_000413__en-us_topic_0000001674340529_b165782133282">Cluster</strong>, and choose <strong id="admin_guide_000413__en-us_topic_0000001674340529_b157813139282">SQL Inspector</strong>. The <strong id="admin_guide_000413__en-us_topic_0000001674340529_b165781613182815">SQL Inspector</strong> page is displayed.</span></li><li id="admin_guide_000413__en-us_topic_0000001674340529_li7650134142117"><span>Add rules for ClickHouse by referring to <a href="admin_guide_000409.html">Adding an SQL Inspection</a>.</span><p><p id="admin_guide_000413__en-us_topic_0000001674340529_p2791339182117">For details about the rules supported by the ClickHouse SQL engine, see <a href="admin_guide_000409.html#admin_guide_000409__en-us_topic_0000001662442869_section19510043143814">MRS SQL Inspection Rules</a>.</p>
|
|
<p id="admin_guide_000413__en-us_topic_0000001674340529_p121087251228">For example, add a rule whose ID is <strong id="admin_guide_000413__en-us_topic_0000001674340529_b5626204392810">static_0008</strong> and checks whether a SQL statement executes the cluster-level table update operation. If so, the system displays a hint.</p>
|
|
<div class="fignone" id="admin_guide_000413__en-us_topic_0000001674340529_fig172016218254"><span class="figcap"><b>Figure 1 </b>Adding a ClickHouse SQL inspection rule</span><br><span><img id="admin_guide_000413__en-us_topic_0000001674340529_image1288911304114" src="en-us_image_0000002007717733.png"></span></div>
|
|
</p></li><li id="admin_guide_000413__en-us_topic_0000001674340529_li174811636328"><span>Log in to the node where the ClickHouse client is installed and run the following command to switch to the client installation directory.</span><p><p id="admin_guide_000413__en-us_topic_0000001674340529_p74811338327"><strong id="admin_guide_000413__en-us_topic_0000001674340529_b24818312325">cd /opt/hadoopclient</strong></p>
|
|
<p id="admin_guide_000413__en-us_topic_0000001674340529_p5224132317279">Run the following command to set environment variables:</p>
|
|
<p id="admin_guide_000413__en-us_topic_0000001674340529_p5481113133213"><strong id="admin_guide_000413__en-us_topic_0000001674340529_b1648183183214">source bigdata_env</strong></p>
|
|
</p></li><li id="admin_guide_000413__en-us_topic_0000001674340529_li1910333418475"><span>If the current cluster is in security mode (Kerberos authentication is enabled), run the following command to authenticate the current user. The current user must have the permission to create ClickHouse tables. If the current cluster is in normal mode (Kerberos authentication is disabled), skip this step.</span><p><p id="admin_guide_000413__en-us_topic_0000001674340529_p55652016339"><strong id="admin_guide_000413__en-us_topic_0000001674340529_b7806161163212">kinit</strong> <em id="admin_guide_000413__en-us_topic_0000001674340529_i1980621163216">Component service user</em></p>
|
|
<p id="admin_guide_000413__en-us_topic_0000001674340529_p23600116552">Example: <strong id="admin_guide_000413__en-us_topic_0000001674340529_b194011121325">kinit clickhouseuser</strong></p>
|
|
</p></li><li id="admin_guide_000413__en-us_topic_0000001674340529_li179997576401"><span>Use the ClickHouse client to connect to the ClickHouse server.</span><p><p id="admin_guide_000413__en-us_topic_0000001674340529_p1152917119414">Security mode</p>
|
|
<p id="admin_guide_000413__en-us_topic_0000001674340529_p114906514387"><strong id="admin_guide_000413__en-us_topic_0000001674340529_b1437142514323">clickhouse client --host </strong><em id="admin_guide_000413__en-us_topic_0000001674340529_i12371825173218">IP address of the ClickHouseServer instance</em><strong id="admin_guide_000413__en-us_topic_0000001674340529_b18372725133210"> --port </strong><strong id="admin_guide_000413__en-us_topic_0000001674340529_b19337142913310">9440</strong> <strong id="admin_guide_000413__en-us_topic_0000001674340529_b0372102511321">--secure</strong></p>
|
|
<p id="admin_guide_000413__en-us_topic_0000001674340529_p0402121264114">Normal clusters:</p>
|
|
<p id="admin_guide_000413__en-us_topic_0000001674340529_p749055117382"><strong id="admin_guide_000413__en-us_topic_0000001674340529_b14167737113310">clickhouse client --host </strong><em id="admin_guide_000413__en-us_topic_0000001674340529_i131681137143314">IP address of the ClickHouseServer instance</em><strong id="admin_guide_000413__en-us_topic_0000001674340529_b1168183719333">--user </strong><em id="admin_guide_000413__en-us_topic_0000001674340529_i116818373336">Username</em><strong id="admin_guide_000413__en-us_topic_0000001674340529_b816843763318"> --password</strong><strong id="admin_guide_000413__en-us_topic_0000001674340529_b151691337203311"> --port </strong><strong id="admin_guide_000413__en-us_topic_0000001674340529_b15169163773318">9000</strong></p>
|
|
<p id="admin_guide_000413__en-us_topic_0000001674340529_p1749055163813"><em id="admin_guide_000413__en-us_topic_0000001674340529_i153891007347">Enter the password.</em></p>
|
|
</p></li><li id="admin_guide_000413__en-us_topic_0000001674340529_li1540074617348"><span>Run the following statements to create a data table:</span><p><p id="admin_guide_000413__en-us_topic_0000001674340529_p252217482407"><strong id="admin_guide_000413__en-us_topic_0000001674340529_b2831123043519">CREATE DATABASE </strong><em id="admin_guide_000413__en-us_topic_0000001674340529_i783393043513">cktest</em><strong id="admin_guide_000413__en-us_topic_0000001674340529_b882543513520"> ON CLUSTER </strong><em id="admin_guide_000413__en-us_topic_0000001674340529_i168262035143511">default_cluster</em><strong id="admin_guide_000413__en-us_topic_0000001674340529_b148251835103520">;</strong></p>
|
|
<p id="admin_guide_000413__en-us_topic_0000001674340529_p1097644634010"><strong id="admin_guide_000413__en-us_topic_0000001674340529_b982071217365">CREATE TABLE </strong><em id="admin_guide_000413__en-us_topic_0000001674340529_i1182241223611">cktest.test2</em><strong id="admin_guide_000413__en-us_topic_0000001674340529_b196150199364"> ON CLUSTER </strong><em id="admin_guide_000413__en-us_topic_0000001674340529_i8616619163618">default_cluster</em><strong id="admin_guide_000413__en-us_topic_0000001674340529_b197471351468"> ( `EventDate` DateTime, `CounterID` UInt32, `UserID` UInt32, `ver` UInt16 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/</strong><em id="admin_guide_000413__en-us_topic_0000001674340529_i17487511617">cktest/test2</em><strong id="admin_guide_000413__en-us_topic_0000001674340529_b1674812510614">', '{replica}')</strong> <strong id="admin_guide_000413__en-us_topic_0000001674340529_b148581417161018">PARTITION BY toYYYYMM(EventDate) ORDER BY (EventDate, intHash32(UserID));</strong></p>
|
|
<p id="admin_guide_000413__en-us_topic_0000001674340529_p5469141554110"><strong id="admin_guide_000413__en-us_topic_0000001674340529_b872175112361">CREATE TABLE </strong><em id="admin_guide_000413__en-us_topic_0000001674340529_i4208652193611">cktest.test2_dir</em><strong id="admin_guide_000413__en-us_topic_0000001674340529_b292357103611"> ON CLUSTER </strong><em id="admin_guide_000413__en-us_topic_0000001674340529_i8937574369">default_cluster</em><strong id="admin_guide_000413__en-us_topic_0000001674340529_b766357183718"> as </strong><em id="admin_guide_000413__en-us_topic_0000001674340529_i16645793713">cktest.test2</em> <strong id="admin_guide_000413__en-us_topic_0000001674340529_b1094518237374">ENGINE = Distributed(</strong><em id="admin_guide_000413__en-us_topic_0000001674340529_i74921624195211">default_cluster</em><em id="admin_guide_000413__en-us_topic_0000001674340529_i2090502483718">, cktest, test2</em><strong id="admin_guide_000413__en-us_topic_0000001674340529_b894562383719">, rand());</strong></p>
|
|
</p></li><li id="admin_guide_000413__en-us_topic_0000001674340529_li1731923715372"><span>Run the following command to insert data to the table:</span><p><p id="admin_guide_000413__en-us_topic_0000001674340529_p12751135514438"><strong id="admin_guide_000413__en-us_topic_0000001674340529_b1931013114388">insert into </strong><em id="admin_guide_000413__en-us_topic_0000001674340529_i13114116384">cktest.test2</em><strong id="admin_guide_000413__en-us_topic_0000001674340529_b331118117386"> values('2023-08-01',111,111,111);</strong></p>
|
|
<p id="admin_guide_000413__en-us_topic_0000001674340529_p1629119427456"><strong id="admin_guide_000413__en-us_topic_0000001674340529_b327715146385">insert into </strong><em id="admin_guide_000413__en-us_topic_0000001674340529_i22789145384">cktest.test2</em><strong id="admin_guide_000413__en-us_topic_0000001674340529_b1327731419388"> values('2023-08-02',222,111,111);</strong></p>
|
|
</p></li><li id="admin_guide_000413__en-us_topic_0000001674340529_li395254382813"><span>Run the following SQL statement for the created table to check whether the rule takes effect:</span><p><p id="admin_guide_000413__en-us_topic_0000001674340529_p17942122510388"><strong id="admin_guide_000413__en-us_topic_0000001674340529_b1368324333816">alter table </strong><em id="admin_guide_000413__en-us_topic_0000001674340529_i9684184363812">cktest.test2</em><strong id="admin_guide_000413__en-us_topic_0000001674340529_b1365814763810"> on cluster </strong><em id="admin_guide_000413__en-us_topic_0000001674340529_i20283204953817">default_cluster</em><strong id="admin_guide_000413__en-us_topic_0000001674340529_b365874723819"> update CounterID = toUInt32(222) where EventDate='2023-08-01' ;</strong></p>
|
|
<pre class="screen" id="admin_guide_000413__en-us_topic_0000001674340529_screen448253123216">...
|
|
<Warning> SQLDefender: Distributed DDL ALTER UPDATE queries are undesirable.
|
|
...</pre>
|
|
<p id="admin_guide_000413__en-us_topic_0000001674340529_p0321103613102">If the operation set in the rule is <strong id="admin_guide_000413__en-us_topic_0000001674340529_b8415114713418">Intercept</strong>, the statement fails to be executed and the following information is displayed:</p>
|
|
<pre class="screen" id="admin_guide_000413__en-us_topic_0000001674340529_screen155901141194817">...
|
|
DB::Exception: Distributed DDL ALTER TABLE UPDATE queries are undesirable..(QUERY_IS_PROHIBITED)
|
|
...</pre>
|
|
<div class="note" id="admin_guide_000413__en-us_topic_0000001674340529_note155477311118"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="admin_guide_000413__en-us_topic_0000001674340529_p21079524568">For more ClickHouse SQL inspection rules, see <a href="admin_guide_000409.html#admin_guide_000409__en-us_topic_0000001662442869_section19510043143814">MRS SQL Inspection Rules</a>.</p>
|
|
</div></div>
|
|
</p></li></ol>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="admin_guide_000407.html">SQL Inspector</a></div>
|
|
</div>
|
|
</div>
|
|
|