doc-exports/docs/css/umn/css_01_0087.html

<a name="EN-US_TOPIC_0000001992205789"></a><a name="EN-US_TOPIC_0000001992205789"></a>

<h1 class="topictitle1">Using Open Source OpenSearch APIs to Import Data to an OpenSearch Cluster</h1>
<div id="body0000001992205789"><p id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_p6252043134616">With <span id="EN-US_TOPIC_0000001992205789__text7509164315139">CSS</span>, you can use open-source OpenSearch APIs on OpenSearch Dashboards or an ECS server to import data to an OpenSearch cluster. JSON files are supported. For details about OpenSearch APIs, see <a href="https://opensearch.org/docs/latest/api-reference/" target="_blank" rel="noopener noreferrer">REST API reference</a>.</p>
<ul id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_ul195013127811"><li id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_li195011612585"><a href="#EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_section1430231820400">Using an Open-Source OpenSearch API to Import Data on OpenSearch Dashboards</a>: Run POST commands to import data.</li><li id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_li175011122080"><a href="#EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_section239718062912">Using an Open-Source OpenSearch API to Import Data on an ECS Server</a>: Run cURL commands to import data.</li></ul>
<div class="section" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_section1430231820400"><a name="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_section1430231820400"></a><a name="en-us_topic_0000001938377776_en-us_topic_0000001223594412_section1430231820400"></a><h4 class="sectiontitle">Using an Open-Source OpenSearch API to Import Data on OpenSearch Dashboards</h4><div class="p" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_p7201111917100">On OpenSearch Dashboards, you can run POST commands to import single pieces of data using an open-source OpenSearch API.<ol id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_ol2068013371496"><li id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_li1865174111515">Log in to the OpenSearch Dashboards.<ol type="a" id="EN-US_TOPIC_0000001992205789__ol116101159259"><li id="EN-US_TOPIC_0000001992205789__en-us_topic_0000002338102386_li111185453113"><span id="EN-US_TOPIC_0000001992205789__en-us_topic_0000002338102386_ph134648105316">Log in to the CSS management console.</span></li><li id="EN-US_TOPIC_0000001992205789__en-us_topic_0000002338102386_li1711804571114">In the navigation pane on the left, choose <span class="uicontrol" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000002338102386_uicontrol1511834581115"><b>Clusters &gt; OpenSearch</b></span>.</li><li id="EN-US_TOPIC_0000001992205789__en-us_topic_0000002338102386_li201181045151113">In the cluster list, find the target cluster, and click <strong id="EN-US_TOPIC_0000001992205789__en-us_topic_0000002338102386_b8963520674129">Dashboards</strong> in the <strong id="EN-US_TOPIC_0000001992205789__en-us_topic_0000002338102386_b1363282744129">Operation</strong> column to log in to OpenSearch Dashboards.</li><li id="EN-US_TOPIC_0000001992205789__en-us_topic_0000002338102386_li811824512111">In the left navigation pane, choose <strong id="EN-US_TOPIC_0000001992205789__en-us_topic_0000002338102386_b45261424710">Dev Tools</strong>.<p id="EN-US_TOPIC_0000001992205789__en-us_topic_0000002338102386_p539424154519">The left part of the console is the command input box, and the triangle icon in its upper-right corner is the execution button. The right part shows the execution result.</p>
</li></ol>
</li><li id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_li167981434143611">Run the following command on OpenSearch Dashboards to check whether the cluster has indexes:<pre class="screen" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_screen1778816833813">GET _cat/indices?v</pre>
<ul id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_ul585911504127"><li id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_li78591650181211">If indexes are available in the cluster to which you want to import data, you do not need to create an index. Go to <a href="#EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_li599201141619">7</a>.</li><li id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_li78591550181212">If no indexes are available in the cluster, go to the next step to create an index.</li></ul>
</li><li id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_li1718421717426">Run the following command on OpenSearch Dashboards to create an index for storing imported data and create a custom mapping to define the data type.<p id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_p366432292117"><a name="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_li1718421717426"></a><a name="en-us_topic_0000001938377776_en-us_topic_0000001223594412_li1718421717426"></a>For example, run the following command to create index <span class="parmname" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_parmname3407144082617"><b>my_store</b></span>:</p>
<pre class="screen" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_screen1181163818278">PUT /<em id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_i1754713447233">my_store</em>
{
    "settings": {
        "number_of_shards": 1
    },
    "mappings": {
        "properties": {
            "productName": {
                "type": "text"
            },
            "size": {
                "type": "keyword"
            }
        }
    }
}</pre>
</li><li id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_li599201141619"><a name="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_li599201141619"></a><a name="en-us_topic_0000001938377776_en-us_topic_0000001223594412_li599201141619"></a>Run commands on OpenSearch Dashboards to import data. In the example below, only a single piece of data is imported:<pre class="screen" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_screen184976293110">POST /my_store/_bulk  
{"index":{}}  
{"productName":"Latest art shirts for women in 2017 autumn","size":"L"}</pre>
<p id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_p116351525183015">The command output is similar to that shown in <a href="#EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_fig86351225133018">Figure 1</a>. If the value of the <span class="parmname" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_parmname163472516306"><b>errors</b></span> field in the result is <span class="parmvalue" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_parmvalue19635152543018"><b>false</b></span>, the data is successfully imported.</p>
<div class="fignone" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_fig86351225133018"><a name="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_fig86351225133018"></a><a name="en-us_topic_0000001938377776_en-us_topic_0000001223594412_fig86351225133018"></a><span class="figcap"><b>Figure 1 </b>Response message</span><br><span><img id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_image19635132563018" src="figure/en-us_image_0000001956150404.png"></span></div>
</li></ol>
</div>
</div>
<div class="section" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_section239718062912"><a name="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_section239718062912"></a><a name="en-us_topic_0000001938377776_en-us_topic_0000001223594412_section239718062912"></a><h4 class="sectiontitle">Using an Open-Source OpenSearch API to Import Data on an ECS Server</h4><p id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_p202850191256">On an ECS server, you can run cURL commands to use an open-source OpenSearch API to import JSON files.</p>
<p id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_p893852815257">In the example below, a cluster in non-security mode is used to describe how to import data using cURL commands. For the commands to use with a security cluster, see <a href="css_01_0030.html">Accessing an OpenSearch Cluster Using cURL Commands</a>.</p>
<ol id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_ol4678613573"><li id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001268314481_li919584122415">Obtain the private network address of the cluster.<ol type="a" id="EN-US_TOPIC_0000001992205789__ol71721615142619"><li id="EN-US_TOPIC_0000001992205789__en-us_topic_0000002338102386_li1515612820249"><span id="EN-US_TOPIC_0000001992205789__en-us_topic_0000002338102386_ph115718818245">Log in to the CSS management console.</span></li><li id="EN-US_TOPIC_0000001992205789__en-us_topic_0000002338102386_li215708162415">In the navigation pane on the left, choose <span class="uicontrol" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000002338102386_uicontrol722453846"><b>Clusters &gt; OpenSearch</b></span>.</li><li id="EN-US_TOPIC_0000001992205789__en-us_topic_0000002338102386_li815717822419">In the cluster list, obtain the target cluster's private IP address from the <strong id="EN-US_TOPIC_0000001992205789__en-us_topic_0000002338102386_b11512044152320">Private IP Address</strong> column. Generally, the IP address format is <em id="EN-US_TOPIC_0000001992205789__en-us_topic_0000002338102386_i815104492319">&lt;host&gt;</em>:<em id="EN-US_TOPIC_0000001992205789__en-us_topic_0000002338102386_i6151344152317">&lt;port&gt;</em> or <em id="EN-US_TOPIC_0000001992205789__en-us_topic_0000002338102386_i1115144452317">&lt;host&gt;</em>:<em id="EN-US_TOPIC_0000001992205789__en-us_topic_0000002338102386_i14151144412237">&lt;port&gt;</em>,<em id="EN-US_TOPIC_0000001992205789__en-us_topic_0000002338102386_i12151194432312">&lt;host&gt;</em>:<em id="EN-US_TOPIC_0000001992205789__en-us_topic_0000002338102386_i16151444142315">&lt;port&gt;</em>.<p id="EN-US_TOPIC_0000001992205789__en-us_topic_0000002338102386_p615720817246">If the cluster has only one node, the IP address and port number of this single node are displayed, for example, <span class="parmvalue" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000002338102386_parmvalue1950910310013"><b>10.62.179.32:9200</b></span>. If the cluster has multiple nodes and all of them are data nodes, the IP addresses and port numbers of all these nodes are displayed; if some of them are client nodes, only the IP addresses and port numbers of these client nodes are displayed; for example, <span class="parmvalue" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000002338102386_parmvalue250963305"><b>10.62.179.32:9200,10.62.179.33:9200</b></span>.</p>
</li></ol>
</li><li id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_li1681812204612">Log in to the ECS that you are going to use to import data.</li><li id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_li1241183810523">Upload a JSON file to the ECS.<div class="p" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_p796375616261"><a name="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_li1241183810523"></a><a name="en-us_topic_0000001938377776_li1241183810523"></a>For example, save the following data as a JSON file and upload the file to the ECS:<pre class="screen" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_screen6683416102718">{"index": {"_index":"my_store"}}
{"productName":"Autumn new woman blouses 2019","size":"M"}
{"index": {"_index":"my_store"}}
{"productName":"Autumn new woman blouses 2019","size":"L"}</pre>
</div>
</li><li id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_li74710325013">Run the following command in the path where the JSON file is stored in the ECS to import the JSON file to an OpenSearch cluster.<pre class="screen" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_screen14946155711418">curl -X PUT "http://{<em id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_i9475182012386">Private network address and port number of the node</em>}/_bulk" -H 'Content-Type: application/json' --data-binary @test.json</pre>
<p id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_p1126010231663">If communication encryption is enabled for the cluster where you are importing data, you need to send HTTPS requests by add <strong id="EN-US_TOPIC_0000001992205789__b1660645912225">-k</strong> to the cURL commands.</p>
<pre class="screen" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_screen950417302617">curl -X PUT -k "https://{<em id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_i15935126102415">Private network address and port number of the node</em>}/_bulk" -H 'Content-Type: application/json' --data-binary @test.json</pre>
<p id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_p719713266457">Replace <em id="EN-US_TOPIC_0000001992205789__i117880382316"><strong id="EN-US_TOPIC_0000001992205789__b07883372315">{Private network address and port number of the node}</strong></em> with the private network address and port number of a node in the cluster. The value of the <strong id="EN-US_TOPIC_0000001992205789__b799721217172">-X</strong> parameter is a command and that of the <strong id="EN-US_TOPIC_0000001992205789__b1399820129176">-H</strong> parameter is a message header. In the preceding command, <span class="parmvalue" id="EN-US_TOPIC_0000001992205789__parmvalue399861215175"><b>PUT</b></span> is the value of the <strong id="EN-US_TOPIC_0000001992205789__b29981612131715">-X</strong> parameter and <span class="parmvalue" id="EN-US_TOPIC_0000001992205789__parmvalue109981112191715"><b>'Content-Type: application/json' --data-binary @test.json</b></span> is the value of the <strong id="EN-US_TOPIC_0000001992205789__b699801231711">-H</strong> parameter. Do not add <strong id="EN-US_TOPIC_0000001992205789__b8853171415174">-k</strong> between a parameter and its value. <span class="filepath" id="EN-US_TOPIC_0000001992205789__filepath8759415122312"><b>test.json</b></span> indicates the JSON file to be imported.</p>
<div class="note" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_note3364193816247"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_p1736511381240">In the case of the failure of a cluster node, if the cluster contains multiple nodes, you can replace {<em id="EN-US_TOPIC_0000001992205789__i11693162017178">Private network address and port number of the node</em>} with the private network address and port number of any available node in the cluster; if the cluster contains only one node, restore this node and execute the command again.</p>
</div></div>
<p id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_p1441510225910"><strong id="EN-US_TOPIC_0000001992205789__b1031004102315">Example 1:</strong> In this example, assume that you need to import data in the <span class="filepath" id="EN-US_TOPIC_0000001992205789__filepath831034120235"><b>test.json</b></span> file to an OpenSearch cluster, where communication encryption is disabled and the private network address and port number of one node are <span class="parmvalue" id="EN-US_TOPIC_0000001992205789__parmvalue1831154111233"><b>192.168.0.90</b></span> and <span class="parmvalue" id="EN-US_TOPIC_0000001992205789__parmvalue4311124122317"><b>9200</b></span> respectively.</p>
<ol type="a" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_ol4472337122815"><li id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_li1992511339283">Run the following command to create an index named <strong id="EN-US_TOPIC_0000001992205789__b75515512246">my_store</strong>:<pre class="screen" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_screen293919145336">curl -X PUT http://192.168.0.90:9200/my_store -H 'Content-Type: application/json' -d '
{
    "settings": {
        "number_of_shards": 1
    },
    "mappings": {
        "properties": {
            "productName": {
                "type": "text"
            },
            "size": {
                "type": "keyword"
            }
        }
    }
}'</pre>
</li><li id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_li63151641202810">Run the following command to import the <strong id="EN-US_TOPIC_0000001992205789__b1861611147247">test.json</strong> file:<pre class="screen" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_screen12455428121611">curl -X PUT "http://192.168.0.90:9200/_bulk" -H 'Content-Type: application/json' --data-binary @test.json</pre>
<p id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_p1718013477396">In this case, if the following information is displayed, the data is successfully imported:</p>
<pre class="screen" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_screen492214294018">{"took":204,"errors":false,"items":[{"index":{"_index":"my_store","_type":"_doc","_id":"DJQkBIwBbJvUd2769Wi-","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1,"status":201}},{"index":{"_index":"my_store","_type":"_doc","_id":"DZQkBIwBbJvUd2769Wi_","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":1,"_primary_term":1,"status":201}}]}</pre>
</li></ol>
<div class="p" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_p89581835184219"><strong id="EN-US_TOPIC_0000001992205789__b01652024102417">Example 2:</strong> In this example, assume that you need to import data in the <span class="filepath" id="EN-US_TOPIC_0000001992205789__filepath316552417248"><b>test.json</b></span> file to an OpenSearch cluster, where communication encryption has been enabled The private network address and port of one node are <strong id="EN-US_TOPIC_0000001992205789__b13743133932417">192.168.0.90</strong> and <strong id="EN-US_TOPIC_0000001992205789__b2744193992412">9200</strong>, respectively.<ol type="a" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_ol59631635184211"><li id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_li15965163514220">Run the following command to create an index named <strong id="EN-US_TOPIC_0000001992205789__b1027404582412">my_store</strong>:<pre class="screen" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_screen129651350429">curl -X PUT -k https://192.168.0.90:9200/my_store -H 'Content-Type: application/json' -d '
 { 
   "settings": { 
     "number_of_shards": 1 
   }, 
   "mappings": { 
       "properties": { 
         "productName": { 
           "type": "text" 
           }, 
         "size": { 
           "type": "keyword" 
         } 
       }  
   } 
 }'</pre>
</li><li id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_li169761035204215">Run the following command to import the <strong id="EN-US_TOPIC_0000001992205789__b51532963452143">test.json</strong> file:<pre class="screen" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_en-us_topic_0000001223594412_screen1095155018491">curl -X PUT -k "https://192.168.0.90:9200/_bulk" -H 'Content-Type: application/json' --data-binary @test.json</pre>
<p id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_p812654134010">In this case, if the following information is displayed, the data is successfully imported:</p>
<pre class="screen" id="EN-US_TOPIC_0000001992205789__en-us_topic_0000001938377776_screen1455933713459">{"took":204,"errors":false,"items":[{"index":{"_index":"my_store","_type":"_doc","_id":"DJQkBIwBbJvUd2769Wi-","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1,"status":201}},{"index":{"_index":"my_store","_type":"_doc","_id":"DZQkBIwBbJvUd2769Wi_","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":1,"_primary_term":1,"status":201}}]}</pre>
</li></ol>
</div>
</li></ol>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="css_01_0059.html">Importing Data to an OpenSearch Cluster</a></div>
</div>
</div>