forked from docs/doc-exports
zhengxiu
93d856d5c5
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: zhengxiu <zhengxiu@huawei.com> Co-committed-by: zhengxiu <zhengxiu@huawei.com>
54 lines
6.1 KiB
HTML
54 lines
6.1 KiB
HTML
<a name="EN-US_TOPIC_0000001992205785"></a><a name="EN-US_TOPIC_0000001992205785"></a>
|
|
|
|
<h1 class="topictitle1">Different Ways to Ingest Data into an OpenSearch Cluster</h1>
|
|
<div id="body0000001992205785"><div class="section" id="EN-US_TOPIC_0000001992205785__en-us_topic_0000001945265112_section1218985910454"><h4 class="sectiontitle">Introduction</h4><p id="EN-US_TOPIC_0000001992205785__en-us_topic_0000001945265112_en-us_topic_0000001961259049_p5636738181213">OpenSearch clusters support multiple data ingestion methods, as listed in <a href="#EN-US_TOPIC_0000001992205785__en-us_topic_0000001945265112_en-us_topic_0000001961259049_table114913297713">Table 1</a>. Select one that fits your needs the best. Before starting to ingest data, determine whether to enhance the data ingestion performance of OpenSearch clusters first. For details, see <a href="css_01_0090.html">Enhancing the Data Ingestion Performance of OpenSearch Clusters</a>.</p>
|
|
|
|
<div class="tablenoborder"><a name="EN-US_TOPIC_0000001992205785__en-us_topic_0000001945265112_en-us_topic_0000001961259049_table114913297713"></a><a name="en-us_topic_0000001945265112_en-us_topic_0000001961259049_table114913297713"></a><table cellpadding="4" cellspacing="0" summary="" id="EN-US_TOPIC_0000001992205785__en-us_topic_0000001945265112_en-us_topic_0000001961259049_table114913297713" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Different ways to ingest data into an OpenSearch cluster</caption><thead align="left"><tr id="EN-US_TOPIC_0000001992205785__en-us_topic_0000001945265112_en-us_topic_0000001961259049_row914911291275"><th align="left" class="cellrowborder" valign="top" width="24.072407240724072%" id="mcps1.3.1.3.2.5.1.1"><p id="EN-US_TOPIC_0000001992205785__en-us_topic_0000001945265112_en-us_topic_0000001961259049_p10149152919711">Data Ingestion Method</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="28.992899289928992%" id="mcps1.3.1.3.2.5.1.2"><p id="EN-US_TOPIC_0000001992205785__en-us_topic_0000001945265112_en-us_topic_0000001961259049_p41497296713">Scenario</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="12.73127312731273%" id="mcps1.3.1.3.2.5.1.3"><p id="EN-US_TOPIC_0000001992205785__en-us_topic_0000001945265112_p10123334115516">Supported Data Formats</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="34.20342034203421%" id="mcps1.3.1.3.2.5.1.4"><p id="EN-US_TOPIC_0000001992205785__en-us_topic_0000001945265112_en-us_topic_0000001961259049_p181496292716">Details</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="EN-US_TOPIC_0000001992205785__en-us_topic_0000001945265112_en-us_topic_0000001961259049_row214982915714"><td class="cellrowborder" valign="top" width="24.072407240724072%" headers="mcps1.3.1.3.2.5.1.1 "><p id="EN-US_TOPIC_0000001992205785__en-us_topic_0000001945265112_p11792181416384">Open-source Logstash</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="28.992899289928992%" headers="mcps1.3.1.3.2.5.1.2 "><p id="EN-US_TOPIC_0000001992205785__en-us_topic_0000001945265112_p1194151715616">Open-source Logstash offers a server-side, real-time data processing pipeline, which supports data ingestion from multiple sources. It can be used to ingest various types of data, such as logs, monitoring data, and metrics.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="12.73127312731273%" headers="mcps1.3.1.3.2.5.1.3 "><p id="EN-US_TOPIC_0000001992205785__en-us_topic_0000001945265112_p12123334205518">JSON, CSV, and text</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="34.20342034203421%" headers="mcps1.3.1.3.2.5.1.4 "><p id="EN-US_TOPIC_0000001992205785__en-us_topic_0000001945265112_p6138135455019"><a href="css_01_0085.html">Using In-house Built Logstash to Ingest Data into an OpenSearch Cluster</a></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="EN-US_TOPIC_0000001992205785__en-us_topic_0000001945265112_en-us_topic_0000001961259049_row123514502113"><td class="cellrowborder" valign="top" width="24.072407240724072%" headers="mcps1.3.1.3.2.5.1.1 "><p id="EN-US_TOPIC_0000001992205785__en-us_topic_0000001945265112_p1378911419381">Open-source OpenSearch API</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="28.992899289928992%" headers="mcps1.3.1.3.2.5.1.2 "><p id="EN-US_TOPIC_0000001992205785__en-us_topic_0000001945265112_p26761016183811">Open-source Elasticsearch APIs can be used to ingest data. This method is flexible, as you can write your own application code.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="12.73127312731273%" headers="mcps1.3.1.3.2.5.1.3 "><p id="EN-US_TOPIC_0000001992205785__en-us_topic_0000001945265112_p3123934165511">JSON</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="34.20342034203421%" headers="mcps1.3.1.3.2.5.1.4 "><p id="EN-US_TOPIC_0000001992205785__en-us_topic_0000001945265112_p513811544509"><a href="css_01_0087.html">Using Open Source OpenSearch APIs to Import Data to an OpenSearch Cluster</a></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="EN-US_TOPIC_0000001992205785__en-us_topic_0000001945265112_en-us_topic_0000001961259049_row814919291979"><td class="cellrowborder" valign="top" width="24.072407240724072%" headers="mcps1.3.1.3.2.5.1.1 "><p id="EN-US_TOPIC_0000001992205785__en-us_topic_0000001945265112_p1478961411380">Cloud Data Migration (CDM)</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="28.992899289928992%" headers="mcps1.3.1.3.2.5.1.2 "><p id="EN-US_TOPIC_0000001992205785__en-us_topic_0000001945265112_p1678831418380">You can use CDM for batch data migration. For example, if data is stored in OBS or an Oracle database, CDM is recommended.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="12.73127312731273%" headers="mcps1.3.1.3.2.5.1.3 "><p id="EN-US_TOPIC_0000001992205785__en-us_topic_0000001945265112_p81232034205515">JSON</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="34.20342034203421%" headers="mcps1.3.1.3.2.5.1.4 "><p id="EN-US_TOPIC_0000001992205785__en-us_topic_0000001945265112_p3137754165015"><a href="css_01_0089.html">Using CDM to Import Data to an OpenSearch Cluster</a></p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="css_01_0059.html">Importing Data to an OpenSearch Cluster</a></div>
|
|
</div>
|
|
</div>
|
|
|