yexinru/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
Files
03e77f3e5b0080047fdcb72efdfd3333dd3bcf92
doc-exports/docs/css/umn/css_01_0071.html
zhengxiu 93d856d5c5 css umn 25.6.0 version 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: zhengxiu <zhengxiu@huawei.com>
Co-committed-by: zhengxiu <zhengxiu@huawei.com>
2025-11-25 11:34:43 +00:00

241 lines
39 KiB
HTML
Raw Blame History

<a name="EN-US_TOPIC_0000001972416129"></a><a name="EN-US_TOPIC_0000001972416129"></a>
<h1 class="topictitle1">Accessing an Elasticsearch Cluster Through the MRS Hive Client</h1>
<div id="body8662426"><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p8753161311559">The Elasticsearch-Hadoop (ES-Hadoop) connector combines the massive data storage and in-depth processing capabilities of Hadoop with the real-time search and analysis capabilities of Elasticsearch. It allows you to quickly get to know big data and work better in the Hadoop ecosystem.</p>
<p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p176905285334">This topic uses the ES-Hadoop of MRS as an example to describe how to connect to a CSS cluster. You can configure any other applications that need to use the Elasticsearch cluster. Ensure the network connection between the client and the Elasticsearch cluster is normal.</p>
<div class="section" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_section3170105416917"><h4 class="sectiontitle">Prerequisites</h4><ul id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_ul115524012184"><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li181191829144220">The CSS cluster is available.</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li578832518434">The client can communicate with the CSS cluster.</li></ul>
<ul id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_ul344818693917"><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li17448136113913">The CSS and MRS clusters are in the same region, AZ, VPC, and subnet.</li></ul>
</div>
<div class="section" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_section1012419118103"><h4 class="sectiontitle">Accessing a Cluster</h4><ol id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_ol1093072045219"><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li1693015206524">Obtain the private network address of the cluster. It is used to access the cluster.<p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_p152184916443"><a name="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li1693015206524"></a><a name="en-us_topic_0000001934179734_li1693015206524"></a>This topic uses a private IP address as an example to describe how to access a cluster. The cluster access address varies with the network configurations used. For details, see <a href="css_01_0190.html#EN-US_TOPIC_0000001975823337__section855085010198">Network Configuration</a>.</p>
<ol type="a" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000001961178833_ol1742819417471"><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000002338102386_li962471018233"><span id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000002338102386_ph1162461016234">Log in to the CSS management console.</span></li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000002338102386_li562441032314">In the navigation pane on the left, choose <span class="uicontrol" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000002338102386_uicontrol206034223985117"><b>Clusters &gt; Elasticsearch</b></span>.</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000002338102386_en-us_topic_0000001961178833_li392216231538">In the cluster list, obtain the target cluster's private IP address from the <strong id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000002338102386_b7217183618175">Private IP Address</strong> column. Generally, the IP address format is <em id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000002338102386_i8377769238175">&lt;host&gt;</em>:<em id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000002338102386_i16969141938175">&lt;port&gt;</em> or <em id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000002338102386_i18383433328175">&lt;host&gt;</em>:<em id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000002338102386_i16522841718175">&lt;port&gt;</em>,<em id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000002338102386_i6783075698175">&lt;host&gt;</em>:<em id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000002338102386_i18005065588175">&lt;port&gt;</em>.<p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000002338102386_en-us_topic_0000001961178833_p18422185125018">If the cluster has only one node, the IP address and port number of this single node are displayed, for example, <span class="parmvalue" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000002338102386_en-us_topic_0000001961178833_parmvalue1916781025515"><b>10.62.179.32:9200</b></span>. If the cluster has multiple nodes and all of them are data nodes, the IP addresses and port numbers of all these nodes are displayed; if some of them are client nodes, only the IP addresses and port numbers of these client nodes are displayed; for example, <span class="parmvalue" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000002338102386_en-us_topic_0000001961178833_parmvalue1425141583119"><b>10.62.179.32:9200,10.62.179.33:9200</b></span>.</p>
</li></ol>
</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li516624022113">Log in to an MRS cluster node.</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li150422172717">Run the cURL command on an MRS cluster node to check the network connectivity. Ensure every node in the MRS cluster can connect to the CSS cluster.<ul id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_ul48942022152318"><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li789411220236">Cluster in non-security mode<pre class="screen" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_screen12262625142419">curl -X GET http://&lt;host&gt;:&lt;port&gt;</pre>
</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li10172522142412">Cluster in security mode + HTTP<pre class="screen" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_screen921113585271">curl -X GET http://&lt;host&gt;:&lt;port&gt; -u &lt;user&gt;:&lt;password&gt;</pre>
</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li1984395519251">Cluster in security mode + HTTPS<pre class="screen" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_screen7365674284">curl -X GET https://&lt;host&gt;:&lt;port&gt; -u &lt;user&gt;:&lt;password&gt; -ik</pre>
</li></ul>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_table7898225115319" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Variables</caption><thead align="left"><tr id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000001961178833_row91731411337"><th align="left" class="cellrowborder" valign="top" width="23.7%" id="mcps1.3.4.2.3.2.2.3.1.1"><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000001961178833_p14171714153311">Variable</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="76.3%" id="mcps1.3.4.2.3.2.2.3.1.2"><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000001961178833_p417131412333">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000001961178833_row192601805113"><td class="cellrowborder" valign="top" width="23.7%" headers="mcps1.3.4.2.3.2.2.3.1.1 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000001961178833_p19261218155110">&lt;host&gt;</p>
</td>
<td class="cellrowborder" valign="top" width="76.3%" headers="mcps1.3.4.2.3.2.2.3.1.2 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000001961178833_p42611819514">IP address of each node in the cluster. If the cluster contains multiple nodes, there will be multiple IP addresses. You can use any of them.</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000001961178833_row73721818175113"><td class="cellrowborder" valign="top" width="23.7%" headers="mcps1.3.4.2.3.2.2.3.1.1 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000001961178833_p133723186514">&lt;port&gt;</p>
</td>
<td class="cellrowborder" valign="top" width="76.3%" headers="mcps1.3.4.2.3.2.2.3.1.2 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000001961178833_p16372318165118">Port number for accessing a cluster node. Generally, the port number is 9200.</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000001961178833_row131711473319"><td class="cellrowborder" valign="top" width="23.7%" headers="mcps1.3.4.2.3.2.2.3.1.1 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000001961178833_p4171147336">&lt;user&gt;</p>
</td>
<td class="cellrowborder" valign="top" width="76.3%" headers="mcps1.3.4.2.3.2.2.3.1.2 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000001961178833_p111791417336">Username for accessing the cluster.</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000001961178833_row161711420338"><td class="cellrowborder" valign="top" width="23.7%" headers="mcps1.3.4.2.3.2.2.3.1.1 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000001961178833_p17171149336">&lt;password&gt;</p>
</td>
<td class="cellrowborder" valign="top" width="76.3%" headers="mcps1.3.4.2.3.2.2.3.1.2 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000001961178833_p11713147339">Password of the user.</p>
<p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000001961178833_p12911206191917">If the password contains special characters, enclose the username and password in single quotation marks, for example, <span class="parmvalue" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001945217682_en-us_topic_0000001961178833_parmvalue139114565204"><b>curl -u "user:password!" "http://&lt;host&gt;:&lt;port&gt;"</b></span>.</p>
</td>
</tr>
</tbody>
</table>
</div>
</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li7348153512813">Download the <a href="https://www.elastic.co/downloads/hadoop" target="_blank" rel="noopener noreferrer">ES-Hadoop lib package</a> and decompress it to obtain the <span class="filepath" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_filepath9226635822"><b>elasticsearch-hadoop-x.x.x.jar</b></span> file. The version must be the same as the CSS cluster version. For example, if the CSS cluster version is 7.6.2, you are advised to download <strong id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_b980527185011">elasticsearch-hadoop-7.6.2.zip</strong>.</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li232618574437">Download the HttpClient dependency package <a href="https://mvnrepository.com/artifact/commons-httpclient/commons-httpclient/3.1" target="_blank" rel="noopener noreferrer">commons-httpclient:commons-httpclient-3.1.jar</a>. In the package name, <strong id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_b1645354885013">3.1</strong> indicates the version number. Select the package of the version you need.</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li3493113165916">Install the MRS client. If the MRS client has been installed, skip this step.</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li1828517308617">Log in to the MRS client. Upload the JAR dependency packages of ES-Hadoop and HttpClient to the MRS client.</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li4861143116217">Create an HDFS directory on the MRS client. Upload the ES-Hadoop lib package and the HttpClient dependency package to the directory.<pre class="screen" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_screen11400805312">hadoop fs -mkdir /tmp/hadoop-es
hadoop fs -put elasticsearch-hadoop-x.x.x.jar /tmp/hadoop-es
hadoop fs -put commons-httpclient-3.1.jar /tmp/hadoop-es</pre>
</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li16374593318">Log in to the Hive client from the MRS client.</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li15356121711711">On the Hive client, add the ES-Hadoop lib package and the HttpClient dependency package. This command is valid only for the current session.<p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p8621743876"><a name="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li15356121711711"></a><a name="en-us_topic_0000001934179734_li15356121711711"></a>Enter <b><span class="cmdname" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_cmdname16958151610911">beeline</span></b> or <b><span class="cmdname" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_cmdname699812133911">hive</span></b> to go to the execution page and run the following commands:</p>
<pre class="screen" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_screen184281457794">add jar hdfs:///tmp/hadoop-es/commons-httpclient-3.1.jar;
add jar hdfs:///tmp/hadoop-es/elasticsearch-hadoop-x.x.x.jar;</pre>
</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li145341213171017">On the Hive client, create a Hive foreign table.<ul id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_ul141971315101112"><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li8197415191112">Cluster in non-security mode<pre class="screen" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_screen73236111114">CREATE EXTERNAL table IF NOT EXISTS <i><span class="varname" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_varname7671177125618">student</span></i>(
id BIGINT,
name STRING,
addr STRING
)
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES(
'es.nodes' = '<i><span class="varname" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_varname1290613542397">xxx.xxx.xxx.xxx:9200</span></i>',
'es.port' = '9200',
'es.net.ssl' = 'false',
'es.nodes.wan.only' = 'false',
'es.nodes.discovery'='false',
'es.input.use.sliced.partitions'='false',
'es.resource' = '<i><span class="varname" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_varname444524710551">student/_doc</span></i>'
);</pre>
</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li8197151551111">Cluster in security mode + HTTP<pre class="screen" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_screen458619149384">CREATE EXTERNAL table IF NOT EXISTS <i><span class="varname" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_varname1571181625619">student</span></i>(
id BIGINT,
name STRING,
addr STRING
)
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES(
'es.nodes' = '<i><span class="varname" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_varname3243122274113">xxx.xxx.xxx.xxx:9200</span></i>',
'es.port' = '9200',
'es.net.ssl' = 'false',
'es.nodes.wan.only' = 'false',
'es.nodes.discovery'='false',
'es.input.use.sliced.partitions'='false',
'es.nodes.client.only'='true',
'es.resource' = '<i><span class="varname" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_varname097412565">student/_doc</span></i>',
'es.net.http.auth.user' = '<i><span class="varname" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_varname109834815416">username</span></i>',
'es.net.http.auth.pass' = '<i><span class="varname" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_varname1378915324116">password</span></i>'
);</pre>
</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li61971115201117">Cluster in security mode + HTTPS<ol type="a" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_ol25051560432"><li id="EN-US_TOPIC_0000001972416129__li762721916281"><a href="#EN-US_TOPIC_0000001972416129__section16306122401412">Obtaining and Uploading a Security Certificate</a>.</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li474074311442">Put the .jks file to the same path of each node in the MRS cluster, for example, <strong id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_b9947194815717">/tmp</strong>. You can run the <strong id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_b57649527577">scp</strong> command to transfer the file. Ensure user <strong id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_b28181137589">omm</strong> has the permission to read the file. You can run the following command to set the permission:<pre class="screen" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_screen20822125843617">chown -R omm truststore.jks</pre>
</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li182297426458">Create a Hive foreign table.<pre class="screen" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_screen16237153134611">CREATE EXTERNAL table IF NOT EXISTS <i><span class="varname" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_varname197790513571">student</span></i>(
id BIGINT,
name STRING,
addr STRING
)
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES(
'es.nodes' = '<i><span class="varname" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_varname18772151820421">https://xxx.xxx.xxx.xxx:9200</span></i>',
'es.port' = '9200',
'es.net.ssl' = 'true',
'es.net.ssl.truststore.location' = '<i><span class="varname" id="EN-US_TOPIC_0000001972416129__varname1540814344617">certFilePath</span></i>',
'es.net.ssl.truststore.pass' = '<i><span class="varname" id="EN-US_TOPIC_0000001972416129__varname7530545164713">certPassword</span></i>',
'es.nodes.wan.only' = 'false',
'es.nodes.discovery'='false',
'es.nodes.client.only'='true',
'es.input.use.sliced.partitions'='false',
'es.resource' = '<i><span class="varname" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_varname1585313332588">student/_doc</span></i>',
'es.net.http.auth.user' = '<i><span class="varname" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_varname17445183410444">username</span></i>',
'es.net.http.auth.pass' = '<i><span class="varname" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_varname9114543164412">password</span></i>'
);</pre>
</li></ol>
</li></ul>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_table3157144413463" frame="border" border="1" rules="all"><caption><b>Table 2 </b>ES-Hadoop parameters</caption><thead align="left"><tr id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_row1115712446465"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.4.2.11.2.2.4.1.1"><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p81571244154619">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="22.28222822282228%" id="mcps1.3.4.2.11.2.2.4.1.2"><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p91571044134618">Default Value</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="44.38443844384438%" id="mcps1.3.4.2.11.2.2.4.1.3"><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p015724434615">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_row18157184484619"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.4.2.11.2.2.4.1.1 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p1115734412466">es.nodes</p>
</td>
<td class="cellrowborder" valign="top" width="22.28222822282228%" headers="mcps1.3.4.2.11.2.2.4.1.2 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p1157164410461">localhost</p>
</td>
<td class="cellrowborder" valign="top" width="44.38443844384438%" headers="mcps1.3.4.2.11.2.2.4.1.3 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p141571443461">Address for accessing the CSS cluster. You can check the private network address in the cluster list.</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_row14157194411467"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.4.2.11.2.2.4.1.1 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p91571344104615">es.port</p>
</td>
<td class="cellrowborder" valign="top" width="22.28222822282228%" headers="mcps1.3.4.2.11.2.2.4.1.2 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p815715447463">9200</p>
</td>
<td class="cellrowborder" valign="top" width="44.38443844384438%" headers="mcps1.3.4.2.11.2.2.4.1.3 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p215784410460">Port number for accessing a cluster. Generally, the port number is 9200.</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_row81571544194610"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.4.2.11.2.2.4.1.1 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p31571844184619">es.nodes.wan.only</p>
</td>
<td class="cellrowborder" valign="top" width="22.28222822282228%" headers="mcps1.3.4.2.11.2.2.4.1.2 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p17157194424617">false</p>
</td>
<td class="cellrowborder" valign="top" width="44.38443844384438%" headers="mcps1.3.4.2.11.2.2.4.1.3 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p1015718445461">Whether to perform node sniffing.</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_row161571944144612"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.4.2.11.2.2.4.1.1 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p161574441466">es.nodes.discovery</p>
</td>
<td class="cellrowborder" valign="top" width="22.28222822282228%" headers="mcps1.3.4.2.11.2.2.4.1.2 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p17157174412463">true</p>
</td>
<td class="cellrowborder" valign="top" width="44.38443844384438%" headers="mcps1.3.4.2.11.2.2.4.1.3 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p5157544164619">Whether to disable node discovery.</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_row13157644194614"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.4.2.11.2.2.4.1.1 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p3157344104617">es.input.use.sliced.partitions</p>
</td>
<td class="cellrowborder" valign="top" width="22.28222822282228%" headers="mcps1.3.4.2.11.2.2.4.1.2 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p715719442466">true</p>
</td>
<td class="cellrowborder" valign="top" width="44.38443844384438%" headers="mcps1.3.4.2.11.2.2.4.1.3 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p91571544174615">Whether to use slices. Its value can be:</p>
<ul id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_ul915711441461"><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li11157134411463"><strong id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_b216136121020">true</strong></li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li16157344104617"><strong id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_b33212407109">false</strong></li></ul>
<div class="note" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_note1464318305576"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p1764313303578">If this parameter is set to <strong id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_b48556467106">true</strong>, the index prefetch time may be significantly prolonged, and may even be much longer than the data query time. You are advised to set this parameter to <strong id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_b11868183881315">false</strong> to improve query efficiency.</p>
</div></div>
</td>
</tr>
<tr id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_row1715724434610"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.4.2.11.2.2.4.1.1 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p21588441467">es.resource</p>
</td>
<td class="cellrowborder" valign="top" width="22.28222822282228%" headers="mcps1.3.4.2.11.2.2.4.1.2 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p1315864454619">NA</p>
</td>
<td class="cellrowborder" valign="top" width="44.38443844384438%" headers="mcps1.3.4.2.11.2.2.4.1.3 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p9158744194614">Specifies the index and type to be read and written.</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_row1158744144612"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.4.2.11.2.2.4.1.1 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p16158444174614">es.net.http.auth.user</p>
</td>
<td class="cellrowborder" valign="top" width="22.28222822282228%" headers="mcps1.3.4.2.11.2.2.4.1.2 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p13158184404617">NA</p>
</td>
<td class="cellrowborder" valign="top" width="44.38443844384438%" headers="mcps1.3.4.2.11.2.2.4.1.3 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p21581044164619">Username for accessing the cluster. Set this parameter only if the security mode is enabled.</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_row1915817447464"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.4.2.11.2.2.4.1.1 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p15158134417461">es.net.http.auth.pass</p>
</td>
<td class="cellrowborder" valign="top" width="22.28222822282228%" headers="mcps1.3.4.2.11.2.2.4.1.2 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p0158444104616">NA</p>
</td>
<td class="cellrowborder" valign="top" width="44.38443844384438%" headers="mcps1.3.4.2.11.2.2.4.1.3 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p111581244124613">Password of the user. Set this parameter only if the security mode is enabled.</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_row21581844134611"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.4.2.11.2.2.4.1.1 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p15158344184616">es.net.ssl</p>
</td>
<td class="cellrowborder" valign="top" width="22.28222822282228%" headers="mcps1.3.4.2.11.2.2.4.1.2 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p11158044114610">false</p>
</td>
<td class="cellrowborder" valign="top" width="44.38443844384438%" headers="mcps1.3.4.2.11.2.2.4.1.3 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p415874444615">Whether to enable SSL. If SSL is enabled, you need to configure the security certificate information.</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_row131581344134614"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.4.2.11.2.2.4.1.1 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p16158204419464">es.net.ssl.truststore.location</p>
</td>
<td class="cellrowborder" valign="top" width="22.28222822282228%" headers="mcps1.3.4.2.11.2.2.4.1.2 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p4158144413466">NA</p>
</td>
<td class="cellrowborder" valign="top" width="44.38443844384438%" headers="mcps1.3.4.2.11.2.2.4.1.3 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p41581444154614">Path of the .jks certificate file, for example, <strong id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_b788715153153">file:///tmp/truststore.jks</strong>.</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_row10639732131916"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.4.2.11.2.2.4.1.1 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p19640193215197">es.nodes.client.only</p>
</td>
<td class="cellrowborder" valign="top" width="22.28222822282228%" headers="mcps1.3.4.2.11.2.2.4.1.2 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p146401532161911">false</p>
</td>
<td class="cellrowborder" valign="top" width="44.38443844384438%" headers="mcps1.3.4.2.11.2.2.4.1.3 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p1658794711387">Check whether the IP address of an independent Client node is configured for <strong id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_b1697515582164">es.nodes</strong> (that is, whether the Client node is enabled during Elasticsearch cluster creation). If yes, change the value to <strong id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_b1526194311192">true</strong>, or an error will be reported, indicating that the data node cannot be found.</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_row3158164411461"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.4.2.11.2.2.4.1.1 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p1015874414618">es.net.ssl.truststore.pass</p>
</td>
<td class="cellrowborder" valign="top" width="22.28222822282228%" headers="mcps1.3.4.2.11.2.2.4.1.2 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p10158144413462">NA</p>
</td>
<td class="cellrowborder" valign="top" width="44.38443844384438%" headers="mcps1.3.4.2.11.2.2.4.1.3 "><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p13158124424613">Password of the .jks certificate file.</p>
</td>
</tr>
</tbody>
</table>
</div>
<p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p10136184034710">For details about ES-Hadoop configuration items, see the <a href="https://www.elastic.co/guide/en/elasticsearch/hadoop/current/configuration.html" target="_blank" rel="noopener noreferrer">official configuration description</a>.</p>
</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li684114616486">On the Hive client, insert data.<pre class="screen" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_screen1092224210493">INSERT INTO TABLE student VALUES (1, "Lucy", "address1"), (2, "Lily", "address2");</pre>
</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li4803124115493">On the Hive client, execute a query.<pre class="screen" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_screen1972242211519">select * from student;</pre>
<p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_p119693544513">The query result is as follows:</p>
<pre class="screen" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_screen1539663185216">+-------------+---------------+---------------+
| student.id | student.name | student.addr |
+-------------+---------------+---------------+
| 1 | Lucy | address1 |
| 2 | Lily | address2 |
+-------------+---------------+---------------+
2 rows selected (0.116 seconds)</pre>
</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li1216263014507">Log in to Kibana.<ol type="a" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000002271391393_ol13172153674611"><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000002271391393_en-us_topic_0000001268594549_li5509181942419"><span id="EN-US_TOPIC_0000001972416129__ph152259618315">Log in to the CSS management console.</span></li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000002271391393_li4250043448">In the navigation pane on the left, choose <span class="uicontrol" id="EN-US_TOPIC_0000001972416129__uicontrol91390158685120"><b>Clusters &gt; Elasticsearch</b></span>.</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000002271391393_en-us_topic_0000001268154521_li6012554340">In the cluster list, find the target cluster, and click <strong id="EN-US_TOPIC_0000001972416129__b214458950685735">Kibana</strong> in the <strong id="EN-US_TOPIC_0000001972416129__b8406233385735">Operation</strong> column to log in to the Kibana console.</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000002271391393_li270075935613">In the left navigation pane, choose <span class="uicontrol" id="EN-US_TOPIC_0000001972416129__uicontrol18610408217"><b>Dev Tools</b></span>.</li></ol>
</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_li927171291011">On the <strong id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_b1180163105810">Dev Tools</strong> page of Kibana, run a query and check the result.<pre class="screen" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_screen3655825556">GET /student/_search</pre>
<div class="fignone" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_fig19376181224113"><span class="figcap"><b>Figure 1 </b>Kibana query result</span><br><span><img id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001934179734_image19642184218170" src="figure/en-us_image_0000001945226538.png"></span></div>
</li></ol>
</div>
<div class="section" id="EN-US_TOPIC_0000001972416129__section16306122401412"><a name="EN-US_TOPIC_0000001972416129__section16306122401412"></a><a name="section16306122401412"></a><h4 class="sectiontitle">Obtaining and Uploading a Security Certificate</h4><p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_p14303249141014">To access a security-mode Elasticsearch cluster that uses HTTPS, a security certificate must be loaded. Perform the following steps to obtain the security certificate and upload it to the client:</p>
<ol id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_ol882703218108"><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_li1182773217101">Obtain the security certificate <strong id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_b53005936784837">CloudSearchService.cer</strong>.<ol type="a" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_ol0827133291012"><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_en-us_topic_0000002338102386_en-us_topic_0000001933318570_li1798555810535"><span id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_en-us_topic_0000002338102386_ph523831318386">Log in to the CSS management console.</span></li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_en-us_topic_0000002338102386_li164461631123811">In the navigation pane on the left, choose <span class="uicontrol" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_en-us_topic_0000002338102386_uicontrol86487267685117"><b>Clusters &gt; Elasticsearch</b></span>.</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_en-us_topic_0000002338102386_li184472316387">In the cluster list, click the name of the target cluster. The cluster information page is displayed.</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_en-us_topic_0000002338102386_en-us_topic_0000001933318570_li146461036135810">Click the <span class="wintitle" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_en-us_topic_0000002338102386_wintitle326154154412"><b>Overview</b></span> tab. In the <span class="wintitle" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_en-us_topic_0000002338102386_wintitle1076156104413"><b>Configuration</b></span> area, click <strong id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_en-us_topic_0000002338102386_b1317531104720">Download Certificate</strong> next to <span class="uicontrol" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_en-us_topic_0000002338102386_uicontrol263216261449"><b>HTTPS Access</b></span>.<div class="fignone" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_en-us_topic_0000002338102386_en-us_topic_0000001223914344_fig1877019549519"><span class="figcap"><b>Figure 2 </b>Downloading a security certificate</span><br><span><img id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_en-us_topic_0000002338102386_en-us_topic_0000001223914344_image2793173420519" src="images/en-us_image_0000002412557593.png"></span></div>
</li></ol>
</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_li1182718329108">Convert the security certificate <strong id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_en-us_topic_0000001961178813_b69670521924">CloudSearchService.cer</strong>. Upload the downloaded security certificate to the client and use keytool to convert the <span class="filepath" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_filepath282793217101"><b>.cer</b></span> certificate into a <span class="filepath" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_filepath1582783210101"><b>.jks</b></span> certificate that can be read by Java.<ul id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_ul2082783281010"><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_li9827133218109">In Linux, run the following command to convert the certificate:<pre class="screen" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_screen182712328102">keytool -import -alias <i><span class="varname" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_varname2827153211109">newname</span></i> -keystore ./truststore.jks -file ./CloudSearchService.cer </pre>
</li><li id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_li16827432171014">In Windows, run the following command to convert the certificate:<pre class="screen" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_screen10827232131020">keytool -import -alias <i><span class="varname" id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_varname1282823281016">newname</span></i> -keystore .\truststore.jks -file .\CloudSearchService.cer</pre>
</li></ul>
<p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_p188281132101015">In the preceding command, <em id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_en-us_topic_0000001961178813_i2126133912318">newname</em> indicates the user-defined certificate name.</p>
<p id="EN-US_TOPIC_0000001972416129__en-us_topic_0000001972375889_p0828193281015">After this command is executed, you will be prompted to set the certificate password and confirm the password. Securely store the password. It will be used for accessing the cluster.</p>
</li></ol>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="css_01_0210.html">Accessing Elasticsearch Clusters</a></div>
</div>
</div>
View Git Blame Copy Permalink