forked from docs/doc-exports
zhengxiu
2125539080
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: zhengxiu <zhengxiu@huawei.com> Co-committed-by: zhengxiu <zhengxiu@huawei.com>
99 lines
10 KiB
HTML
99 lines
10 KiB
HTML
<a name="css_01_0454"></a><a name="css_01_0454"></a>
|
|
|
|
<h1 class="topictitle1">Different Ways to Import Data to an OpenSearch Cluster</h1>
|
|
<div id="body0000001992205785"><div class="section" id="css_01_0454__css_01_0394_section1218985910454"><h4 class="sectiontitle">Introduction</h4><p id="css_01_0454__css_01_0394_en-us_topic_0000001961259049_p5636738181213">OpenSearch clusters support multiple data import methods, as listed in <a href="#css_01_0454__css_01_0394_en-us_topic_0000001961259049_table114913297713">Table 1</a>. Select one that fits your needs the best. Before starting to import data, determine whether to enhance the data import performance of OpenSearch clusters first. For details, see <a href="css_01_0458.html">Enhancing the Data Import Performance of OpenSearch Clusters</a>.</p>
|
|
|
|
<div class="tablenoborder"><a name="css_01_0454__css_01_0394_en-us_topic_0000001961259049_table114913297713"></a><a name="css_01_0394_en-us_topic_0000001961259049_table114913297713"></a><table cellpadding="4" cellspacing="0" summary="" id="css_01_0454__css_01_0394_en-us_topic_0000001961259049_table114913297713" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Different ways to import data to an OpenSearch cluster</caption><thead align="left"><tr id="css_01_0454__css_01_0394_en-us_topic_0000001961259049_row914911291275"><th align="left" class="cellrowborder" valign="top" width="24.072407240724072%" id="mcps1.3.1.3.2.5.1.1"><p id="css_01_0454__css_01_0394_en-us_topic_0000001961259049_p10149152919711">Data Import Method</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="28.992899289928992%" id="mcps1.3.1.3.2.5.1.2"><p id="css_01_0454__css_01_0394_en-us_topic_0000001961259049_p41497296713">Scenario</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="12.73127312731273%" id="mcps1.3.1.3.2.5.1.3"><p id="css_01_0454__css_01_0394_p10123334115516">Supported Data Formats</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="34.20342034203421%" id="mcps1.3.1.3.2.5.1.4"><p id="css_01_0454__css_01_0394_en-us_topic_0000001961259049_p181496292716">Details</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="css_01_0454__css_01_0394_en-us_topic_0000001961259049_row214982915714"><td class="cellrowborder" valign="top" width="24.072407240724072%" headers="mcps1.3.1.3.2.5.1.1 "><p id="css_01_0454__css_01_0394_p11792181416384">Logstash data processing pipeline</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="28.992899289928992%" headers="mcps1.3.1.3.2.5.1.2 "><p id="css_01_0454__css_01_0394_p1194151715616">Open-source Logstash offers a server-side, real-time data processing pipeline, which supports data ingestion from multiple sources. It can be used to collect various data, such as logs, monitoring data, and metrics.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="12.73127312731273%" headers="mcps1.3.1.3.2.5.1.3 "><p id="css_01_0454__css_01_0394_p12123334205518">JSON, CSV, and text</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="34.20342034203421%" headers="mcps1.3.1.3.2.5.1.4 "><p id="css_01_0454__css_01_0394_p6138135455019"><a href="css_01_0455.html">Using In-house Built Logstash to Import Data to an OpenSearch Cluster</a></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="css_01_0454__css_01_0394_en-us_topic_0000001961259049_row123514502113"><td class="cellrowborder" valign="top" width="24.072407240724072%" headers="mcps1.3.1.3.2.5.1.1 "><p id="css_01_0454__css_01_0394_p1378911419381">Open-source OpenSearch API</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="28.992899289928992%" headers="mcps1.3.1.3.2.5.1.2 "><p id="css_01_0454__css_01_0394_p26761016183811">Open-source Elasticsearch APIs can be used to import data. This method is flexible, as you can write your own application code.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="12.73127312731273%" headers="mcps1.3.1.3.2.5.1.3 "><p id="css_01_0454__css_01_0394_p3123934165511">JSON</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="34.20342034203421%" headers="mcps1.3.1.3.2.5.1.4 "><p id="css_01_0454__css_01_0394_p513811544509"><a href="css_01_0456.html">Using Open Source OpenSearch APIs to Import Data to an OpenSearch Cluster</a></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="css_01_0454__css_01_0394_en-us_topic_0000001961259049_row814919291979"><td class="cellrowborder" valign="top" width="24.072407240724072%" headers="mcps1.3.1.3.2.5.1.1 "><p id="css_01_0454__css_01_0394_p1478961411380">Cloud Data Migration (CDM)</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="28.992899289928992%" headers="mcps1.3.1.3.2.5.1.2 "><p id="css_01_0454__css_01_0394_p1678831418380">You can use CDM for batch data migration. For example, if data is stored in OBS or an Oracle database, CDM is recommended.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="12.73127312731273%" headers="mcps1.3.1.3.2.5.1.3 "><p id="css_01_0454__css_01_0394_p81232034205515">JSON</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="34.20342034203421%" headers="mcps1.3.1.3.2.5.1.4 "><p id="css_01_0454__css_01_0394_p3137754165015"><a href="css_01_0457.html">Using CDM to Import Data to an OpenSearch Cluster</a></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="css_01_0454__css_01_0394_en-us_topic_0000001961259049_row61491729276"><td class="cellrowborder" valign="top" width="24.072407240724072%" headers="mcps1.3.1.3.2.5.1.1 "><p id="css_01_0454__css_01_0394_p578751412380">Data Replication Service (DRS)</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="28.992899289928992%" headers="mcps1.3.1.3.2.5.1.2 "><p id="css_01_0454__css_01_0394_p14859123412563">DRS can be used for online database migration and real-time data synchronization.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="12.73127312731273%" headers="mcps1.3.1.3.2.5.1.3 "><p id="css_01_0454__css_01_0394_p1012317348559">Relational Database Service (RDS)</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="34.20342034203421%" headers="mcps1.3.1.3.2.5.1.4 "><p id="css_01_0454__css_01_0394_p12136165419509"><a href="#css_01_0454__css_01_0394_section1896852904516">Using DRS to Import Data from a Database to OpenSearch</a></p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="css_01_0454__css_01_0394_section1896852904516"><a name="css_01_0454__css_01_0394_section1896852904516"></a><a name="css_01_0394_section1896852904516"></a><h4 class="sectiontitle">Using DRS to Import Data from a Database to OpenSearch</h4><p id="css_01_0454__css_01_0394_p010114705211">DRS is an easy-to-use, stable, and efficient cloud service for online database migration and real-time database synchronization. Real-time data synchronization refers to the real-time replication of data from one source to another while ensuring data consistency.</p>
|
|
<p id="css_01_0454__css_01_0394_p276917415521">DRS can be used to import data from multiple types of relational databases to OpenSearch clusters. For details about the supported software versions for source databases and destination clusters, see <a href="#css_01_0454__css_01_0394_table1099912411111">Table 2</a>.</p>
|
|
|
|
<div class="tablenoborder"><a name="css_01_0454__css_01_0394_table1099912411111"></a><a name="css_01_0394_table1099912411111"></a><table cellpadding="4" cellspacing="0" summary="" id="css_01_0454__css_01_0394_table1099912411111" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Using DRS to import data from a database to an OpenSearch cluster</caption><thead align="left"><tr id="css_01_0454__css_01_0394_row69991141101116"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.4.2.4.1.1"><p id="css_01_0454__css_01_0394_p14999341161110">Scenario</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.4.2.4.1.2"><p id="css_01_0454__css_01_0394_p1899916413113">Source DB</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.4.2.4.1.3"><p id="css_01_0454__css_01_0394_p79991741191119">Destination OpenSearch Cluster</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="css_01_0454__css_01_0394_row7044291113"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.1 "><p id="css_01_0454__css_01_0394_p146841547131217">Importing data from an RDS for MySQL database to a CSS OpenSearch cluster</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.2 "><p id="css_01_0454__css_01_0394_p40042171115">RDS for MySQL 5.5, 5.6, 5.7, or 8.0</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.3 "><p id="css_01_0454__css_01_0394_p2802274138">OpenSearch 1.3.6</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="css_01_0454__css_01_0394_row140042111110"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.1 "><p id="css_01_0454__css_01_0394_p1956072019156">Importing data from a TaurusDB database to a CSS Elasticsearch cluster</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.2 "><p id="css_01_0454__css_01_0394_p403429113">Primary/standby TaurusDB instances</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.3 "><p id="css_01_0454__css_01_0394_p0084251117">OpenSearch 1.3.6</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="css_01_0454__css_01_0394_row12064271119"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.1 "><p id="css_01_0454__css_01_0394_p2015428116">Importing data from an in-house built MySQL database to a CSS OpenSearch cluster</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.2 "><p id="css_01_0454__css_01_0394_p182141424154314">MySQL database 5.5, 5.6, 5.7, or 8.0 created on a local server or ECS</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.3 "><p id="css_01_0454__css_01_0394_p120842101117">OpenSearch 1.3.6</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="css_01_0453.html">Importing Data to an OpenSearch Cluster</a></div>
|
|
</div>
|
|
</div>
|
|
|