zhengxiu
93d856d5c5
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: zhengxiu <zhengxiu@huawei.com> Co-committed-by: zhengxiu <zhengxiu@huawei.com>
6.1 KiB
Different Ways to Ingest Data into an OpenSearch Cluster
Introduction
OpenSearch clusters support multiple data ingestion methods, as listed in Table 1. Select one that fits your needs the best. Before starting to ingest data, determine whether to enhance the data ingestion performance of OpenSearch clusters first. For details, see Enhancing the Data Ingestion Performance of OpenSearch Clusters.
Data Ingestion Method |
Scenario |
Supported Data Formats |
Details |
|---|---|---|---|
Open-source Logstash |
Open-source Logstash offers a server-side, real-time data processing pipeline, which supports data ingestion from multiple sources. It can be used to ingest various types of data, such as logs, monitoring data, and metrics. |
JSON, CSV, and text |
Using In-house Built Logstash to Ingest Data into an OpenSearch Cluster |
Open-source OpenSearch API |
Open-source Elasticsearch APIs can be used to ingest data. This method is flexible, as you can write your own application code. |
JSON |
Using Open Source OpenSearch APIs to Import Data to an OpenSearch Cluster |
Cloud Data Migration (CDM) |
You can use CDM for batch data migration. For example, if data is stored in OBS or an Oracle database, CDM is recommended. |
JSON |