forked from docs/doc-exports
zhengxiu
93d856d5c5
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: zhengxiu <zhengxiu@huawei.com> Co-committed-by: zhengxiu <zhengxiu@huawei.com>
54 lines
5.2 KiB
HTML
54 lines
5.2 KiB
HTML
<a name="EN-US_TOPIC_0000001945265112"></a><a name="EN-US_TOPIC_0000001945265112"></a>
|
|
|
|
<h1 class="topictitle1">Different Ways to Ingest Data into an Elasticsearch Cluster</h1>
|
|
<div id="body0000001945265112"><div class="section" id="EN-US_TOPIC_0000001945265112__section1218985910454"><h4 class="sectiontitle">Introduction</h4><p id="EN-US_TOPIC_0000001945265112__en-us_topic_0000001961259049_p5636738181213">Elasticsearch clusters support multiple data ingestion methods, as listed in <a href="#EN-US_TOPIC_0000001945265112__en-us_topic_0000001961259049_table114913297713">Table 1</a>. Select one that fits your needs the best. Before starting to ingest data, determine whether to enhance the data ingestion performance of Elasticsearch clusters first. For details, see <a href="css_01_0228.html">Enhancing the Data Ingestion Performance of Elasticsearch Clusters</a>.</p>
|
|
|
|
<div class="tablenoborder"><a name="EN-US_TOPIC_0000001945265112__en-us_topic_0000001961259049_table114913297713"></a><a name="en-us_topic_0000001961259049_table114913297713"></a><table cellpadding="4" cellspacing="0" summary="" id="EN-US_TOPIC_0000001945265112__en-us_topic_0000001961259049_table114913297713" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Different ways to ingest data into an Elasticsearch cluster</caption><thead align="left"><tr id="EN-US_TOPIC_0000001945265112__en-us_topic_0000001961259049_row914911291275"><th align="left" class="cellrowborder" valign="top" width="18%" id="mcps1.3.1.3.2.5.1.1"><p id="EN-US_TOPIC_0000001945265112__en-us_topic_0000001961259049_p10149152919711">Data Ingestion Method</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="30%" id="mcps1.3.1.3.2.5.1.2"><p id="EN-US_TOPIC_0000001945265112__en-us_topic_0000001961259049_p41497296713">Scenario</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="22%" id="mcps1.3.1.3.2.5.1.3"><p id="EN-US_TOPIC_0000001945265112__p10123334115516">Supported Data Formats/Sources</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="30%" id="mcps1.3.1.3.2.5.1.4"><p id="EN-US_TOPIC_0000001945265112__en-us_topic_0000001961259049_p181496292716">Details</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="EN-US_TOPIC_0000001945265112__en-us_topic_0000001961259049_row214982915714"><td class="cellrowborder" valign="top" width="18%" headers="mcps1.3.1.3.2.5.1.1 "><p id="EN-US_TOPIC_0000001945265112__p11792181416384">Open-source Logstash</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.1.3.2.5.1.2 "><p id="EN-US_TOPIC_0000001945265112__p1194151715616">Open-source Logstash offers a server-side, real-time data processing pipeline, which supports data ingestion from multiple sources. It can be used to ingest various types of data, such as logs, monitoring data, and metrics.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="22%" headers="mcps1.3.1.3.2.5.1.3 "><p id="EN-US_TOPIC_0000001945265112__p12123334205518">JSON, CSV, and text</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.1.3.2.5.1.4 "><p id="EN-US_TOPIC_0000001945265112__p6138135455019"><a href="css_01_0048.html">Using In-house Built Logstash to Import Data to Elasticsearch</a></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="EN-US_TOPIC_0000001945265112__en-us_topic_0000001961259049_row123514502113"><td class="cellrowborder" valign="top" width="18%" headers="mcps1.3.1.3.2.5.1.1 "><p id="EN-US_TOPIC_0000001945265112__p1378911419381">Open-source Elasticsearch API</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.1.3.2.5.1.2 "><p id="EN-US_TOPIC_0000001945265112__p26761016183811">Open-source Elasticsearch APIs can be used to ingest data. This method is flexible, as you can write your own application code.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="22%" headers="mcps1.3.1.3.2.5.1.3 "><p id="EN-US_TOPIC_0000001945265112__p3123934165511">JSON</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.1.3.2.5.1.4 "><p id="EN-US_TOPIC_0000001945265112__p513811544509"><a href="css_01_0024.html">Using Open Source Elasticsearch APIs to Import Data to Elasticsearch</a></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="EN-US_TOPIC_0000001945265112__en-us_topic_0000001961259049_row814919291979"><td class="cellrowborder" valign="top" width="18%" headers="mcps1.3.1.3.2.5.1.1 "><p id="EN-US_TOPIC_0000001945265112__p1478961411380">Cloud Data Migration (CDM)</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.1.3.2.5.1.2 "><p id="EN-US_TOPIC_0000001945265112__p1678831418380">You can use CDM for batch data migration. For example, if data is stored in OBS or an Oracle database, CDM is recommended.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="22%" headers="mcps1.3.1.3.2.5.1.3 "><p id="EN-US_TOPIC_0000001945265112__p81232034205515">JSON</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.1.3.2.5.1.4 "><p id="EN-US_TOPIC_0000001945265112__p3137754165015"><a href="css_01_0046.html">Using CDM to Import Data to Elasticsearch</a></p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="css_01_0004.html">Importing Data to an Elasticsearch Cluster</a></div>
|
|
</div>
|
|
</div>
|
|
|