Querying task statuses
diff --git a/docs/drs/umn/drs_02_0002.html b/docs/drs/umn/drs_02_0002.html
index 84941417e..5d4411798 100644
--- a/docs/drs/umn/drs_02_0002.html
+++ b/docs/drs/umn/drs_02_0002.html
@@ -3,7 +3,7 @@
Step 1: Create a Migration Task
ProcessA complete real-time migration consists of creating a migration task, tracking task progress, analyzing migration logs, and comparing data consistency. By comparing multiple items and data, you can determine the proper time for service migration to minimize the service downtime.
A complete migration involves the following procedures.
- Figure 1 Migration process
+ Figure 1 Migration process
This section uses the migration from MySQL to RDS MySQL as an example to describe how to configure a migration task over a VPC network on the DRS console.
VPC is suitable for migrations of cloud databases in the same region.
@@ -335,3 +335,10 @@
Step 4: Compare Migration Items
This section describes how to compare migration items to check if there are any differences between source and destination databases. By comparing migration objects, you can determine the proper time for service migration to minimize the service downtime.
- Figure 1 Comparison process
+ Figure 1 Comparison process
Comparison ScenariosYou can compare migration objects with different dimensions:
- Object-level comparison: It helps you compare databases, indexes, tables, views, stored procedures and functions, and sorting rules of tables. You are advised to perform the comparison after a full migration is complete.
- Data-level comparison is classified into row comparison and value comparison.
- Row comparison: It helps you compare the number of rows in the tables to be migrated. This comparison method is recommended because it is fast.
- Value comparison: It helps you check whether data in the migrated table is consistent. The comparison process is relatively slow.
- Account comparison: It compares usernames and permissions of the source and destination databases.
- Periodic comparison: DRS periodically compares the number of rows or objects in the source database table with those in the destination database table and displays the comparison results. To compare rows or objects periodically, enable comparison policy.
@@ -51,3 +51,10 @@
- Yes
You can customize the maximum DR speed.
In addition, you can set the time range based on your service requirements. The traffic rate setting usually includes setting of a rate limiting time period and a traffic rate value. Flow can be controlled all day or during specific time ranges. The default value is All day. A maximum of three time ranges can be set, and they cannot overlap.
The flow rate must be set based on the service scenario and cannot exceed 9,999 MB/s.
-Figure 2 Flow control
+Figure 2 Flow control
- No
The DR speed is not limited and the outbound bandwidth of the source database is maximally used, which causes read consumption on the source database accordingly. For example, if the outbound bandwidth of the source database is 100 MB/s and 80% bandwidth is used, the I/O consumption on the source database is 80 MB/s. NOTE: - Flow control mode takes effect during the initial DR phase only.
- You can also change the flow control mode when the task is in the Configuration state. On the Basic Information tab, In the DR Information area, click Modify next to Flow Control. In the dialog box that is displayed, change the flow control mode. The flow control mode cannot be changed for a task that is in Starting state.
- After the verification is successful, click Generate Processing Rule. The rule is displayed.
- Click Next.
-Processing Columns- On the Process Data page, select Processing Columns.
- In the Object area, select the objects to be processed.
Figure 1 Processing Columns
- - Click Edit to the right of the selected object.
- In the Edit Column dialog box, select the columns to be mapped and enter new column names.
Figure 2 Edit Column
+Processing Columns- On the Process Data page, select Processing Columns.
- In the Object area, select the objects to be processed.
Figure 1 Processing Columns
+ - Click Edit to the right of the selected object.
- In the Edit Column dialog box, select the columns to be mapped and enter new column names.
Figure 2 Edit Column
 - You can query or filter columns or create new column names.
- After the column name is edited, the column name of the destination database is changed to the new name.
- The new column name cannot be the same as the original column name or an existing column name.
- The column name in the synchronized table cannot be modified.
- Only the selected columns can be synchronized.
- MySQL to MySQL and MySQL to TaurusDB Cluster synchronizations do not support column mapping based on the partitioning column of a partitioned table.
- GaussDB partition keys cannot be filtered.
- Click Confirm.
- Click Next.
@@ -23,3 +23,10 @@
From MySQL to MySQL
+Supported Source and Destination Databases
+ Table 1 Supported databasesSource DB
+ |
+Destination DB
+ |
+
|---|
+
+- On-premises MySQL databases
- MySQL databases on an ECS
+ |
+- On-premises MySQL databases
- MySQL databases on an ECS
+ |
+
+
+
+
+
+ Prerequisites- You have logged in to the DRS console.
- For details about the DB types and versions supported by real-time synchronization, see Real-Time Synchronization.
+
+ Suggestions - When a task is being started or in the full synchronization phase, do not perform DDL operations on the source database. Otherwise, the task may be abnormal.
- To keep data consistency before and after the synchronization, ensure that no data is written to the destination database during the synchronization.
+
+ - The success of database synchronization depends on environment and manual operations. To ensure a smooth synchronization, perform a synchronization trial before you start the synchronization to help you detect and resolve problems in advance.
- Start your synchronization task during off-peak hours. A less active database is easier to synchronize successfully. If the data is fairly static, there is less likely to be any severe performance impacts during the synchronization. If you have to synchronize data during peak hours, you can select Yes for Flow Control to adjust the synchronization speed.
- If network bandwidth is not limited, the query rate of the source database increases by about 50 MB/s during full synchronization, and two to four CPUs are occupied.
- To ensure data consistency, tables to be synchronized without a primary key may be locked for 3s.
- The data being synchronized may be locked by other transactions for a long period of time, resulting in read timeout.
- Due to the inherent characteristics of MySQL, in certain scenarios the performance may be negatively affected. For example, if the CPU resources are insufficient and the storage engine is TokuDB, the read speed on tables may be decreased by 10%.
- When DRS concurrently reads data from a database, it will use about 6 to 10 sessions. The impact of the connections on services must be considered.
- If you read a table, especially a large table, during the full migration, the exclusive lock on that table may be blocked.
+
+ - Data-Level Comparison
To obtain accurate comparison results, compare data at a specified time point during off-peak hours. If it is needed, select Start at a specified time for Comparison Time. Due to slight time difference and continuous operations on data, data inconsistency may occur, reducing the reliability and validity of the comparison results.
+ - For many-to-one synchronization tasks that involve the synchronization of the same table, DDL operations cannot be performed on source databases. Otherwise, all synchronization tasks fail.
+
+ PrecautionsBefore creating a synchronization task, read the following notes:
+ - You are advised to create an independent database account for DRS task connection to prevent task failures caused by account modification.
- After changing the account passwords for the source or destination databases, modify the connection information in the DRS task as soon as possible to prevent automatic retry after a task failure. Automatic retry will lock the database accounts.
+
+
+ Table 2 PrecautionsType
+ |
+Restrictions
+ |
+
|---|
+
+Database permissions
+ |
+- The source database user must have the following permissions: SELECT, SHOW VIEW, EVENT, LOCK TABLES, REPLICATION SLAVE, and REPLICATION CLIENT.
- The destination database user must have the following permissions: SELECT, CREATE, DROP, DELETE, INSERT, UPDATE, ALTER, and REFERENCES.
+ |
+
+Synchronization object
+ |
+- Only tables, primary key indexes, unique indexes, common indexes, store procedures, views, and functions can be synchronized.
+- During database mapping, the source database cannot contain stored procedures, views, or functions.
- The databases to be mapped to the destination can only contain tables during synchronization. Otherwise, the synchronization task fails.
If the database table name contains characters other than letters, digits, and underscores (_), or the mapped database table name contains hyphens (-) and number signs (#), the name length cannot exceed 42 characters.
+ - Tables with storage engine different to MyISAM and InnoDB cannot be synchronized.
- For an incremental synchronization task, ensure that the synchronization objects exist in the destination database.
+ |
+
+Source database
+ |
+- During the incremental synchronization, the binlog of the source MySQL database must be enabled and use the row-based format.
- If the storage space is sufficient, store the source database binlog for as long as possible. The recommended retention period is three days.
- If the expire_logs_days value of the source database is set to 0, the synchronization may fail.
- Enable GTID for the source database. If GTID is not enabled for the source database, primary/standby switchover is not supported. DRS tasks will be interrupted and cannot be restored during a switchover.
- During an incremental synchronization, the server_id value of the MySQL source database must be set. If the source database version is MySQL 5.6 or earlier, the server_id value ranges from 2 to 4294967296. If the source database is MySQL 5.7 or later, the server_id value ranges from 1 to 4294967296.
- The source database names cannot contain non-ASCII characters, or the following characters: '<`>/\
- The table and view names in the source database cannot contain non-ASCII characters, or special characters '<>/\
- The source database name or mapped name cannot start with ib_logfile or be ib_buffer_pool, ib_doublewrite, ibdata1 or ibtmp1.
- Tables with storage engine different to MyISAM and InnoDB cannot be synchronized to RDS.
- Database mapping does not support views or stored procedures. If the source database contains views or stored procedures, the synchronization may fail.
+ |
+
+Destination database
+ |
+- Data cannot be synchronized from a newer version database to an older version database.
+- The destination DB instance is running properly. If the destination DB instance type is primary/standby, the replication status must also be normal.
- The destination DB instance must have sufficient storage space.
- If the destination database (excluding MySQL system database) has the same name as the source database, the table structures in the destination database must be consistent with those in the source database.
- The character set of the destination database must be the same as that of the source database.
- The time zone of the destination database must be the same as that of the source database.
- During a synchronization, a large amount of data is written to the destination database. If the value of the max_allowed_packet parameter of the destination database is too small, data cannot be written. You are advised to set the max_allowed_packet parameter to a value greater than 100 MB.
- If the MyISAM tables are included in the synchronization objects, the sql_mode parameter in the destination database cannot contain the no_engine_substitution parameter. Otherwise, the synchronization fails.
- The source database names mapped to the destination database cannot contain the following characters: dots (.), angle brackets (<>), backslash (\), and single quotation marks (')
+ |
+
+Precautions
+ |
+- Objects that have dependencies must be synchronized at the same time to avoid synchronization failure. Common dependencies: tables referenced by views, views referenced by views, views and tables referenced by stored procedures/functions/triggers, and tables referenced by primary and foreign keys
- Only MySQL to MySQL synchronization supports many-to-one synchronization. During table-level many-to-one synchronization, tables without primary keys cannot exist in the source database.
- If the source or destination MySQL database does not support TLS 1.2 or is a self-built database of an earlier version (earlier than 5.6.46 or between 5.7 and 5.7.28), you need to submit an O&M application for testing the SSL connection.
- Resumable upload is supported, but data may be repeatedly inserted into a table that does not have a primary key.
- The destination table can contain more columns than the source table. However, the following failures must be avoided:
- Assume that extra columns on the destination cannot be null or have default values. If newly inserted data records are synchronized from the source to the destination, the extra columns will become null, which does not meet the requirements of the destination and will cause the task to fail.
- Assume that extra columns on the destination must be fixed at a default value and have a unique constraint. If newly inserted data records are synchronized from the source to the destination, the extra columns will contain multiple default values. That does not meet the unique constraint of the destination and will cause the task to fail.
+ - The source database does not support point-in-time recovery (PITR).
- The destination database cannot be restored to a point in time when a full synchronization was being performed.
- Cascade operations cannot be performed on tables with foreign keys. If the foreign key index of a table is a common index, the table structure may fail to be created. You are advised to use a unique index.
- Binlogs cannot be forcibly deleted. Otherwise, the synchronization task fails.
- The source database does not support the reset master or reset master to command, which may cause DRS task failures or data inconsistency.
- Set the expire_log_day parameter to a proper value to ensure that the binlog does not expire before data transfer resumes. This ensures that services can be recovered after interruption.
- After a task is created, the destination database cannot be set to read-only.
- A real-time synchronization task may fail due to the change of the username and password of the source or destination database. If it happens, rectify the information and then retry the synchronization task on the DRS console. Generally, you are advised not to modify the preceding information during synchronization.
- If the source or destination database port is changed during data synchronization, the synchronization task fails. You can rectify the fault as follows:
+
- During real-time synchronization, do not change the IP address.
- DDL operations are not supported during full synchronization.
- Data inconsistency may occur when the MyISAM table is modified during a full synchronization.
- In a full plus incremental synchronization task, the source database cannot contain a table with the same name as a table without a primary key in the destination database.
- For an incremental synchronization task, if the synchronization point is not precise, data inconsistency may occur.
- During table-level incremental synchronization, only DDL operations on tables are allowed.
- During an incremental synchronization of table-level objects, renaming tables is not recommended.
- For an incremental synchronization task, the start point must be in the standard format. You can run the show master status command in the source database to obtain the start point.
- If GTID mode is disabled, the format of the start point is File:Position (for example, mysql-bin.014734:8937).
- After the GTID is enabled, the format of the start point is File:Position. Executed_Gtid_Set (for example, mysql-bin.014734:8937;ea352a82-e9a0-11ea-b650-fa163e1e82ed:1-23377228, 8404bb4b-cc96-11ea-ae7f-fa163ead1b99:1-59684135)
+
+ |
+
+
+
+
+
+ Procedure- On the Data Synchronization Management page, click Create Synchronization Task.
- On the Create Synchronization Instance page, specify the task name, description, and the synchronization instance details, and click Next.
+ Table 3 Task and recipient descriptionParameter
+ |
+Description
+ |
+
|---|
+
+Region
+ |
+The region where the synchronization instance is deployed. You can change the region.
+ |
+
+Project
+ |
+The project corresponds to the current region and can be changed.
+ |
+
+Task Name
+ |
+The task name must start with a letter and consist of 4 to 50 characters. It can contain only letters, digits, hyphens (-), and underscores (_).
+ |
+
+Description
+ |
+The description consists of a maximum of 256 characters and cannot contain special characters !=<>'&"\
+ |
+
+
+
+ Table 4 Synchronization instance settingsParameter
+ |
+Description
+ |
+
|---|
+
+Data Flow
+ |
+Choose Self-built to self-built.
+ |
+
+Source DB Engine
+ |
+Select MySQL.
+ |
+
+Destination DB Engine
+ |
+Select MySQL.
+ |
+
+Network Type
+ |
+The Public network is used as an example. Available options: Public network and VPN or Direct Connect
+ |
+
+Instance Type
+ |
+DRS instance type. The value can be single or primary/standby.
+- primary/standby: This architecture provides HA, improving the reliability of DRS instances. After the primary and standby instances are created, the DRS creates an active subtask and a standby subtask at the same time. If the task on the active node is abnormal, the DRS automatically starts the task on the standby node to continue the synchronization, improving the reliability of the synchronization task.
- Single: The single-node deployment architecture is used, which is cost-effective.
+ |
+
+VPC
+ |
+Select an available VPC.
+ |
+
+Synchronization Instance Subnet
+ |
+Select the subnet where the synchronization instance is located. You can also click View Subnet to go to the network console to view the subnet where the instance resides.
+By default, the DRS instance and the destination DB instance are in the same subnet. You need to select the subnet where the DRS instance resides and ensure that there are available IP addresses. To ensure that the synchronization instance is successfully created, only subnets with DHCP enabled are displayed.
+ |
+
+Security Group
+ |
+Select a security group. You can use security group rules to allow or deny access to the instance.
+ |
+
+Synchronization Mode
+ |
+Select Incremental.
+
+- Incremental
Through log parsing, incremental data generated on the source database is synchronized to the destination database.
+During synchronization, the source database continues to provide services for external systems with zero downtime.
+
+ |
+
+AZ
+ |
+Select the AZ where you want to create the DRS instance. Selecting the one housing the source or destination database can provide better performance.
+If Instance Type is set to primary/standby, you can specify Primary AZ and Standby AZ.
+ |
+
+Specifications
+ |
+DRS instance specifications. Different specifications have different performance upper limits. For details, see Real-Time Synchronization.
+ |
+
+Tags
+ |
+- This setting is optional. Adding tags helps you better identify and manage your tasks. Each task can have up to 20 tags.
- After a task is created, you can view its tag details on the Tags tab. For details, see Tag Management.
+ |
+
+
+
+ If a task fails to be created, DRS retains the task for three days by default. After three days, the task automatically ends.
+
+ - On the Configure Source and Destination Databases page, wait until the synchronization instance is created. Then, specify source and destination database information and click Test Connection for both the source and destination databases to check whether they have been connected to the synchronization instance. After the connection tests are successful, select the check box before the agreement and click Next.
+ Table 5 Source database settingsParameter
+ |
+Description
+ |
+
|---|
+
+IP Address or Domain Name
+ |
+The IP address or domain name of the source database.
+ |
+
+Port
+ |
+The port of the source database. Range: 1 – 65535
+ |
+
+Database Username
+ |
+The username for accessing the source database.
+ |
+
+Database Password
+ |
+The password for the database username.
+ |
+
+SSL Connection
+ |
+SSL encrypts the connections between the source and destination databases. If SSL is enabled, upload the SSL CA root certificate.
+ NOTE: - The maximum size of a single certificate file that can be uploaded is 500 KB.
- If the SSL certificate is not used, your data may be at risk.
+
+ |
+
+
+
+ Table 6 Destination database settingsParameter
+ |
+Description
+ |
+
|---|
+
+IP Address or Domain Name
+ |
+The IP address or domain name of the destination database.
+ |
+
+Port
+ |
+The port of the destination database. Range: 1 - 65535
+ |
+
+Database Username
+ |
+The username for accessing the destination database.
+ |
+
+Database Password
+ |
+The password for the database username.
+ |
+
+SSL Connection
+ |
+SSL encrypts the connections between the source and destination databases. If SSL is enabled, upload the SSL CA root certificate.
+ NOTE: - The maximum size of a single certificate file that can be uploaded is 500 KB.
- If the SSL certificate is not used, your data may be at risk.
+
+ |
+
+
+
+ The username and password of the source and destination databases are encrypted and stored in DRS, and will be cleared after the task is deleted.
+
+ - On the Set Synchronization Task page, select the synchronization policy, objects, and data format, and click Next.
+ Table 7 Synchronization mode and objectParameter
+ |
+Description
+ |
+
|---|
+
+Conflict Policy
+ |
+The conflict policy refers to the conflict handling policy during incremental synchronization. By default, conflicts in the full synchronization phase are ignored. Select any of the following conflict policies:
+- Ignore
The system will skip the conflicting data and continue the subsequent synchronization process.
+ - Report error
The synchronization task will be stopped and fail.
+ - Overwrite
Conflicting data will be overwritten.
+
+ |
+
+Filter DROP DATABASE
+ |
+During real-time synchronization, executing DDL operations on the source database may affect the synchronization performance. To reduce the risk of synchronization failure, DRS allows you to filter out DDL operations. Currently, only the delete operations on databases can be filtered by default.
+- If you select Yes, the database deletion operation performed on the source database is not synchronized during data synchronization.
- If you select No, related operations are synchronized to the destination database during data synchronization.
+ |
+
+Synchronize
+ |
+Supports index synchronization. You can determine whether to synchronize indexes based on the service requirements.
+ |
+
+Start Point
+ |
+The position where the incremental log was created based on the source log. It allows you to perform resumable data transfer.
+For an incremental synchronization task, the start point must be in the standard format. You can run the show master status command in the source database to obtain the start point.
+- If GTID mode is disabled, the format of the start point is File:Position (for example, mysql-bin.014734:8937).
- After the GTID is enabled, the format of the start point is File:Position; Executed_Gtid_Set (for example, mysql-bin.014734:8937;ea352a82-e9a0-11ea-b650-fa163e1e82ed:1-23377228, 8404bb4b-cc96-11ea-ae7f-fa163ead1b99:1-59684135)
+ |
+
+Synchronization Object
+ |
+Select Tables, Databases, or Import object file as required.
+ NOTE: - You can search for table names to quickly select the required database objects.
- If there are changes made to the source databases or objects, click
 in the upper right corner to update the objects to be synchronized.
+ - If an object name contains spaces, the spaces before and after the object name are not displayed. If there are two or more consecutive spaces in the middle of the object name, only one space is displayed.
- The name of the selected synchronization object cannot contain spaces.
+
+ |
+
+
+
+ - On the Process Data page, set the filtering rules for data processing.
- If you do not need to set a processing rule, select No processing and click Next.
- If you need to set data processing rules, configure related filtering rules by referring to Processing Data.
+ - On the Check Task page, check the synchronization task.
- If any check fails, review the cause and rectify the fault. After the fault is rectified, click Check Again.
- If all check items are successful, click Next.
You can proceed to the next step only when all checks are successful. If there are any items that require confirmation, view and confirm the details first before proceeding to the next step.
+
+
+ - On the Confirm Task page, specify Start Time, confirm that the configured information is correct, and click Submit to submit the task.
+ Table 8 Task startup settingsParameter
+ |
+Description
+ |
+
|---|
+
+Started Time
+ |
+Set Start Time to Start upon task creation or Start at a specified time based on site requirements.
+ NOTE: After a synchronization task is started, the performance of the source and destination databases may be affected. You are advised to start a synchronization task during off-peak hours.
+
+ |
+
+
+
+ - After the task is submitted, you can view and manage it on the Data Synchronization Management page.
- You can view the task status. For more information about task status, see Task Statuses.
- You can click
 in the upper-right corner to view the latest task status. - By default, DRS retains a task in the Configuration state for three days. After three days, DRS automatically deletes background resources, but the task status remains unchanged. When you reconfigure the task, DRS applies for resources for the task again.
+
+
+ - Task name
- Description
- Task start time
PrerequisitesYou have logged in to the DRS console.
Procedure- On the Disaster Recovery Management page, click the target DR task in the Task Name/ID column.
- On the Basic Information tab, locate the information to be modified in the Task Information area.
- You can click
 to modify the task name and description.- To submit the change, click
 . - To cancel the change, click
 .
+Procedure- On the Disaster Recovery Management page, click the target DR task in the Task Name/ID column.
- On the Basic Information tab, locate the information to be modified in the Task Information area.
- You can click to modify the task name and description.
- To submit the change, click .
- To cancel the change, click .
Table 1 Real-time DR task informationTask Information
|
@@ -38,3 +38,10 @@
+
+
\ No newline at end of file
diff --git a/docs/drs/umn/drs_04_0006.html b/docs/drs/umn/drs_04_0006.html
index 99848b4c6..41cada8a5 100644
--- a/docs/drs/umn/drs_04_0006.html
+++ b/docs/drs/umn/drs_04_0006.html
@@ -6,7 +6,7 @@
- Task name
- Description
- Task start time
Prerequisites- You have logged in to the DRS console.
- A migration task has been created.
Procedure- On the Online Migration Management page, click the target migration task name in the Task Name/ID column.
- On the Basic Information tab, locate the information to be modified in the Task Information area.
- You can click
 to modify the task name and description.- To submit the change, click
 . - To cancel the change, click
 .
+Procedure- On the Online Migration Management page, click the target migration task name in the Task Name/ID column.
- On the Basic Information tab, locate the information to be modified in the Task Information area.
- You can click to modify the task name and description.
- To submit the change, click .
- To cancel the change, click .
Table 1 Task informationTask Information
|
@@ -38,3 +38,10 @@
+
+
\ No newline at end of file
diff --git a/docs/drs/umn/drs_04_0020.html b/docs/drs/umn/drs_04_0020.html
index 2599f79bd..20f978257 100644
--- a/docs/drs/umn/drs_04_0020.html
+++ b/docs/drs/umn/drs_04_0020.html
@@ -3,9 +3,9 @@
How Do I Ensure that All Services on the Database Are Stopped?
To ensure that all services on the database are stopped, perform the following steps:
- Run the following statement on the source database to check whether active connections exist:
show processlist;
-Figure 1 Checking active connections
+Figure 1 Checking active connections
- Optional: If there are active connections, locate the service processes based on the values in the Host column in the command output and stop the service processes.
- Run the following statement in the source database to check the binlog position. Then, record the two values in the file and position columns as ckpt1:
show master status;
-Figure 2 Viewing the binlog position
+Figure 2 Viewing the binlog position
- Wait for more than 30s. Run the following statement in the source database to check the binlog position again. Then, record the two values in the file and position columns as ckpt2. If ckpt1 and ckpt2 are equal, no data is written to the source database.
show master status;
@@ -16,3 +16,10 @@
- Yes
You can customize the maximum DR speed.
In addition, you can set the time range based on your service requirements. The traffic rate setting usually includes setting of a rate limiting time period and a traffic rate value. Flow can be controlled all day or during specific time ranges. The default value is All day. A maximum of three time ranges can be set, and they cannot overlap.
The flow rate must be set based on the service scenario and cannot exceed 9,999 MB/s.
-Figure 1 Flow control
+Figure 1 Flow control
- No
The DR speed is not limited and the outbound bandwidth of the source database is maximally used, which causes read consumption on the source database accordingly. For example, if the outbound bandwidth of the source database is 100 MB/s and 80% bandwidth is used, the I/O consumption on the source database is 80 MB/s. NOTE: - Flow control mode takes effect during the initial DR phase only.
- You can also change the flow control mode when the task is in the Configuration state. On the Basic Information tab, In the DR Information area, click Modify next to Flow Control. In the dialog box that is displayed, change the flow control mode. The flow control mode cannot be changed for a task that is in Starting state.
- Yes
You can customize the maximum DR speed.
In addition, you can set the time range based on your service requirements. The traffic rate setting usually includes setting of a rate limiting time period and a traffic rate value. Flow can be controlled all day or during specific time ranges. The default value is All day. A maximum of three time ranges can be set, and they cannot overlap.
The flow rate must be set based on the service scenario and cannot exceed 9,999 MB/s.
-Figure 1 Flow control
+Figure 1 Flow control
- No
The DR speed is not limited and the outbound bandwidth of the source database is maximally used, which causes read consumption on the source database accordingly. For example, if the outbound bandwidth of the source database is 100 MB/s and 80% bandwidth is used, the I/O consumption on the source database is 80 MB/s. NOTE: - Flow control mode takes effect during the initial DR phase only.
- You can also change the flow control mode when the task is in the Configuration state. On the Basic Information tab, In the DR Information area, click Modify next to Flow Control. In the dialog box that is displayed, change the flow control mode. The flow control mode cannot be changed for a task that is in Starting state.
From MySQL to Kafka
-Supported Source and Destination Databases
- Table 1 Supported databasesSource DB
- |
-Destination DB
- |
-
|---|
-
-- On-premises MySQL databases
- MySQL databases on an ECS
- |
-
- |
-
-
-
-
-
- Prerequisites- You have logged in to the DRS console.
- For details about the DB types and versions supported by real-time synchronization, see Real-Time Synchronization.
-
- Suggestions- The success of database synchronization depends on environment and manual operations. To ensure a smooth synchronization, perform a synchronization trial before you start the synchronization to help you detect and resolve problems in advance.
- It is recommended that you start a task during off-peak hours to minimize the impact of synchronization on your services.
-
- PrecautionsBefore creating a synchronization task, read the following notes:
- - You are advised to create an independent database account for DRS task connection to prevent task failures caused by account modification.
- After changing the account passwords for the source or destination databases, modify the connection information in the DRS task as soon as possible to prevent automatic retry after a task failure. Automatic retry will lock the database accounts.
-
-
- Table 2 PrecautionsType
- |
-Restrictions
- |
-
|---|
-
-Database permissions
- |
-- The source database user must have the following permissions: SELECT, SHOW VIEW, EVENT, LOCK TABLES, REPLICATION SLAVE, REPLICATION CLIENT, and RELOAD.
- |
-
-Synchronization object
- |
-- The table data can be synchronized.
-- Tables with storage engine different to MyISAM and InnoDB cannot be synchronized.
- |
-
-Source database
- |
-- During the incremental synchronization, the binlog of the source MySQL database must be enabled and use the row-based format.
- If the storage space is sufficient, store the source database binlog for as long as possible. The recommended retention period is three days.
- If the expire_logs_days value of the source database is set to 0, the synchronization may fail.
- Enable GTID for the source database. If GTID is not enabled for the source database, primary/standby switchover is not supported. DRS tasks will be interrupted and cannot be restored during a switchover.
- During an incremental synchronization, the server_id value of the MySQL source database must be set. If the source database version is MySQL 5.6 or earlier, the server_id value ranges from 2 to 4294967296. If the source database is MySQL 5.7, the server_id value ranges from 1 to 4294967296.
- The database and table names in the source database cannot contain non-ASCII characters, or special characters '<`>/\
- |
-
-Destination database
- |
-- The destination database is a Kafka database.
- |
-
-Precautions
- |
-- Objects that have dependencies must be synchronized at the same time to avoid synchronization failure. Common dependencies: tables referenced by views, views referenced by views, views and tables referenced by stored procedures/functions/triggers, and tables referenced by primary and foreign keys
- Cascade operations cannot be performed on tables with foreign keys. If the foreign key index of a table is a common index, the table structure may fail to be created. You are advised to use a unique index.
- Binlogs cannot be forcibly deleted. Otherwise, the synchronization task fails.
- The source database does not support the reset master or reset master to command, which may cause DRS task failures or data inconsistency.
- If the source MySQL database does not support TLS 1.2 or is a self-built database of an earlier version (earlier than 5.6.46 or between 5.7 and 5.7.28), you need to submit an O&M application for testing the SSL connection.
- During the synchronization, do not delete or change the username, password, or permission of the source database, or change the port of the destination database.
- Data inconsistency may occur when the MyISAM table is modified during synchronization.
- During synchronization of table-level objects, renaming tables is not recommended.
- |
-
-
-
-
-
- Procedure- On the Data Synchronization Management page, click Create Synchronization Task.
- On the Create Synchronization Instance page, specify the task name, description, and the synchronization instance details, and click Next.
- Table 3 Task and recipient descriptionParameter
- |
-Description
- |
-
|---|
-
-Region
- |
-The region where the synchronization instance is deployed. You can change the region.
- |
-
-Project
- |
-The project corresponds to the current region and can be changed.
- |
-
-Task Name
- |
-The task name must start with a letter and consist of 4 to 50 characters. It can contain only letters, digits, hyphens (-), and underscores (_).
- |
-
-Description
- |
-The description consists of a maximum of 256 characters and cannot contain special characters !=<>'&"\
- |
-
-
-
- Table 4 Synchronization instance settingsParameter
- |
-Description
- |
-
|---|
-
-Data Flow
- |
-Choose Self-built to self-built.
- |
-
-Source DB Engine
- |
-Select MySQL.
- |
-
-Destination DB Engine
- |
-Select Kafka.
- |
-
-Network Type
- |
-The Public network is used as an example. Available options: VPC, Public network and VPN or Direct Connect
- |
-
-AZ
- |
-Select the AZ where you want to create the DRS instance. Selecting the one housing the source or destination database can provide better performance.
-If Instance Type is set to primary/standby, you can specify Primary AZ and Standby AZ.
- |
-
-VPC
- |
-Select an available VPC.
- |
-
-Synchronization Instance Subnet
- |
-Select the subnet where the synchronization instance is located. You can also click View Subnet to go to the network console to view the subnet where the instance resides.
-By default, the DRS instance and the destination DB instance are in the same subnet. You need to select the subnet where the DRS instance resides and ensure that there are available IP addresses. To ensure that the synchronization instance is successfully created, only subnets with DHCP enabled are displayed.
- |
-
-Security Group
- |
-Select a security group. You can use security group rules to allow or deny access to the instance.
- |
-
-Synchronization Mode
- |
-- Incremental
Through log parsing, incremental data generated on the source database is synchronized to the destination database.
-During synchronization, the source database continues to provide services for external systems with zero downtime.
-
- |
-
-Specifications
- |
-DRS instance specifications. Different specifications have different performance upper limits. For details, see Real-Time Synchronization.
- |
-
-Tags
- |
-- This setting is optional. Adding tags helps you better identify and manage your tasks. Each task can have up to 20 tags.
- After a task is created, you can view its tag details on the Tags tab. For details, see Tag Management.
- |
-
-
-
- If a task fails to be created, DRS retains the task for three days by default. After three days, the task automatically ends.
-
- - On the Configure Source and Destination Databases page, wait until the synchronization instance is created. Then, specify source and destination database information and click Test Connection for both the source and destination databases to check whether they have been connected to the synchronization instance. After the connection tests are successful, select the check box before the agreement and click Next.
- Table 5 Source database settingsParameter
- |
-Description
- |
-
|---|
-
-IP Address or Domain Name
- |
-The IP address or domain name of the source database.
- |
-
-Port
- |
-The port of the source database. Range: 1 – 65535
- |
-
-Database Username
- |
-The username for accessing the source database.
- |
-
-Database Password
- |
-The password for the database username.
- |
-
-SSL Connection
- |
-SSL encrypts the connections between the source and destination databases. If SSL is enabled, upload the SSL CA root certificate.
- NOTE: - The maximum size of a single certificate file that can be uploaded is 500 KB.
- If the SSL certificate is not used, your data may be at risk.
-
- |
-
-
-
- The username and password of the source database are encrypted and stored in DRS and will be cleared after the task is deleted.
-
-
-Table 6 Source database informationParameter
- |
-Description
- |
-
|---|
-
-IP Address or Domain Name
- |
-The IP address or domain name of the destination database.
- |
-
-Security Protocol
- |
-Available options: PLAINTEXT, SSL, SASL_PLAINTEXT, and SASL_SSL. For details, see Kafka Authentication.
- |
-
-
-
- - On the Set Synchronization Task page, select the synchronization policy, objects, and data format, and click Next.
- Table 7 Synchronization ObjectParameter
- |
-Description
- |
-
|---|
-
-Topic Synchronization Policy
- |
-Topic synchronization policy. The options are as follows:
-- Select A specified topic if the data volume of the source database is small.
- Select Auto-generated topics if each table contains a lot of data. Then, the system automatically generates a topic for each table.
- |
-
-Topic
- |
-Select the topic to be synchronized to the destination database. This parameter is available when the topic is set to A specified topic.
- |
-
-Topic Name Format
- |
-Topic name format. This parameter is available when Topic Synchronization Policy is set to Auto-generated topics.
-Only variables database and tablename are supported. The other characters must be constants. Replace $database$ with the database name and $tablename$ with the table name.
-For example, if this parameter is set to $database$-$tablename$ and the database name is db1, and the table name is tab1, the topic name is db1-tab1. If DDL statements are synchronized, $tablename$ is empty and the topic name is db1.
- |
-
-Number of Partitions
- |
-This parameter is available when Topic Synchronization Policy is set to Auto-generated topics.
-The number of partitions of a topic. Each topic can have multiple partitions. More partitions can provide higher throughput but consume more resources. Set the number of partitions based on the actual situation of brokers.
- |
-
-Replication Factor
- |
-This parameter is available when Topic Synchronization Policy is set to Auto-generated topics.
-Number of copies of a topic. Each topic can have multiple copies, and the copies are placed on different brokers in a cluster. The number of copies cannot exceed the number of brokers. Otherwise, the topic fails to be created.
- |
-
-Synchronize Topic To
- |
-The policy for synchronizing topics to the Kafka partitions.
-- If topics are synchronized to different partitions by hash value of the database and table names, the performance on a single table query can be improved.
- If topics are synchronized to partition 0, strong consistency can be obtained but write performance is impacted.
- If topics are synchronized to different partitions by hash value of the primary key, one table corresponds to one topic. This prevents data from being written to the same partition, and consumers can obtain data from different partitions concurrently.
- |
-
-Data Format in Kafka
- |
-Select the data format to be delivered from MySQL to Kafka.
-- JSON: JSON message format, which is easy to interpret but takes up more space.
- JSON-C: A data format that is compatible with multiple batch and stream computing frameworks.
-For details, see Kafka Message Format.
- |
-
-Synchronization Object
- |
-Available options: Tables and Databases.
-
- |
-
-
-
- - On the Process Data page, select the columns to be processed.
- If data processing is not required, click Next.
- If you need to process columns, set processing rules by referring to Processing Data.
- - On the Check Task page, check the synchronization task.
- If any check fails, review the cause and rectify the fault. After the fault is rectified, click Check Again.
- If all check items are successful, click Next.
You can proceed to the next step only when all checks are successful. If there are any items that require confirmation, view and confirm the details first before proceeding to the next step.
-
-
- - On the Confirm Task page, specify Start Time, confirm that the configured information is correct, and click Submit to submit the task.
- Table 8 Task startup settingsParameter
- |
-Description
- |
-
|---|
-
-Started Time
- |
-Set Start Time to Start upon task creation or Start at a specified time based on site requirements.
- NOTE: After a synchronization task is started, the performance of the source and destination databases may be affected. You are advised to start a synchronization task during off-peak hours.
-
- |
-
-
-
- - After the task is submitted, you can view and manage it on the Data Synchronization Management page.
- You can view the task status. For more information about task status, see Task Statuses.
- You can click in the upper-right corner to view the latest task status.
- By default, DRS retains a task in the Configuration state for three days. After three days, DRS automatically deletes background resources, but the task status remains unchanged. When you reconfigure the task, DRS applies for resources for the task again.
-
-
-
diff --git a/docs/drs/umn/drs_04_0458.html b/docs/drs/umn/drs_04_0458.html
index b13b32559..a0039352b 100644
--- a/docs/drs/umn/drs_04_0458.html
+++ b/docs/drs/umn/drs_04_0458.html
@@ -19,7 +19,7 @@ AND SP.name NOT LIKE '##%##'
AND SP.name NOT LIKE 'NT AUTHORITY%'
AND SP.name NOT LIKE 'NT SERVICE%'
AND SP.name NOT IN ('rdsadmin','rdsbackup','rdsuser','rdsmirror','public')
- - Execute the script in 1:
Figure 1 Obtaining the script
+ - Execute the script in 1:
Figure 1 Obtaining the script
- Copy and execute the script obtain in 2 on the destination instance. The created Login account is the same as the original one.
- Map the newly created Login account to the database user permissions that have been migrated to the RDS SQL Server DB instance to ensure permission consistency.
declare @DBName nvarchar(200)
declare @Login_name nvarchar(200)
declare @SQL nvarchar(MAX)
@@ -50,7 +50,7 @@ deallocate DBName_Cursor
Database LinkSQL Server allows you to create database links to interact with databases on external DB instances. Therefore you can query, synchronize, and compare databases of different types or on different DB instances. However, these links cannot be automatically synchronized to the DB instance on cloud so you need to synchronize them manually.
- Connect the local DB instance and cloud DB instance through Microsoft SQL Server Management Studio. Choose and locate the DBLink of the current DB instance.
Figure 2 Viewing database links
- - Select the linked server and press F7. The Object Explore page is displayed. On this page, you can quickly create a script.
Figure 3 Creating the script
+ - Select the linked server and press F7. The Object Explore page is displayed. On this page, you can quickly create a script.
Figure 3 Creating the script
- In the displayed window, view all the scripts for creating DBLinks of the current DB instance. You only need to copy the scripts to the destination DB instance and change the password on @rmtpassword.
USE [master]
GO
@@ -65,7 +65,7 @@ GO
Agent JOBAgent Job is the agent service of Microsoft SQL Server. It helps you quickly create scheduled tasks on DB instances, perform routine O&M, and process data. You need to manually migrate local Job scripts.
- Connect the local DB instance and cloud DB instance through Microsoft SQL Server Management Studio. Choose and locate all the jobs of the current DB instance.
Figure 4 Viewing Jobs
- - Select a job and press F7. All jobs are displayed on the Object Explore page. Select all jobs and create a script in the new window.
Figure 5 Creating a script
+ - Select a job and press F7. All jobs are displayed on the Object Explore page. Select all jobs and create a script in the new window.
Figure 5 Creating a script
- Copy the T-SQL script in the new window to the new DB instance, and then modify the following key items to ensure that the creation is successful.
- Modify the owner account of each job.
Example:
@owner_login_name=N'rdsuser'
- Modify the DB instance name of each job.
Example:
@@ -95,7 +95,7 @@ ALTER DATABASE [tempdb] ADD FILE ( NAME = N'tempdb6', FILENAME = N'D:\RDSDBDATA\
GO
ALTER DATABASE [tempdb] ADD FILE ( NAME = N'tempdb7', FILENAME = N'D:\RDSDBDATA\Temp\tempdb7.ndf' , SIZE = 65536KB , FILEGROWTH = 65536KB )
GO
-Figure 6 Checking temporary files
+Figure 6 Checking temporary files
- Database isolation level: Check whether the database isolation level is enabled on the source DB instance and synchronize the isolation level to the RDS SQL Server DB instance. There are two snapshot isolation parameters:
- Is Read Committed Snapshot On
- Allow Snapshot Isolation
If the database isolation level of the source DB instance is enabled, you can run the following script on the destination database to enable the database isolation level:
USE [DBName]
@@ -105,11 +105,11 @@ GO
ALTER DATABASE [DBName] SET ALLOW_SNAPSHOT_ISOLATION ON
GO
- Max Degree of Parallelism: The maximum degree of parallelism is set to 0 by default on the RDS SQL Server instance. You can also set the value based on the local settings to avoid exceptions in different service scenarios.
In Object Explorer, right-click a local server and select Properties. Click the Advanced node. In the Max Degree of Parallelism box, view the value of the local instance and change the max degree of parallelism value in the parameter group of the destination RDS SQL Server instance to the same.
-Figure 7 Max Degree of Parallelism
-Log in to the RDS console. On the Instance Management page, click the target DB instance name. Choose Parameters, search for the max degree of parallelism parameter, and change its value. Figure 8 max degree of parallelism
+ Figure 7 Max Degree of Parallelism
+ Log in to the RDS console. On the Instance Management page, click the target DB instance name. Choose Parameters, search for the max degree of parallelism parameter, and change its value. Figure 8 max degree of parallelism
- Check whether the database recovery model on the cloud is set to Full. If not, change the mode.
Right-click the database and choose Properties from the shortcut menu. In the displayed page, select Options. Then, verify that Recovery Model is set to Full. Ensure that the database is highly available and the backup policy is executable.
-Figure 9 Checking the database recovery model
+Figure 9 Checking the database recovery model
@@ -118,3 +118,10 @@ GO
+
+
\ No newline at end of file
diff --git a/docs/drs/umn/drs_05_0006.html b/docs/drs/umn/drs_05_0006.html
index 2031a492d..fce1bad20 100644
--- a/docs/drs/umn/drs_05_0006.html
+++ b/docs/drs/umn/drs_05_0006.html
@@ -22,8 +22,6 @@
- Restarting a Synchronization Task
-- Changing a Single-AZ Task to a Dual-AZ Task
-
- Cloning a Synchronization Task
- Stopping a Synchronization Task
diff --git a/docs/drs/umn/drs_05_0018.html b/docs/drs/umn/drs_05_0018.html
index 1df0a0d45..db1505411 100644
--- a/docs/drs/umn/drs_05_0018.html
+++ b/docs/drs/umn/drs_05_0018.html
@@ -2,10 +2,10 @@
Kafka Authentication
PLAINTEXTNo security authentication mode is available. You only need to enter the IP address and port for connection.
- Figure 1 PLAINTEXT
+ Figure 1 PLAINTEXT
SASL_PLAINTEXTThe SASL mechanism is used to connect to Kafka, and you need to configure SASL parameters.
- Figure 2 SASL_PLAINTEXT
+ Figure 2 SASL_PLAINTEXT
Table 1 Parameter settingsParameter
|
@@ -39,7 +39,7 @@
SSLSSL is used to encrypt the connection to Kafka. Related parameters need to be configured.
- Figure 3 SSL
+ Figure 3 SSL
Table 2 Parameter settingsParameter
|
@@ -87,7 +87,7 @@
SASL_SSLIf the SASL and SSL are used, configure SSL and SASL parameters. For details, see SASL_PLAINTEXT and SSL.
- Figure 4 SASL_SSL
+ Figure 4 SASL_SSL
@@ -96,3 +96,10 @@
+
+
\ No newline at end of file
diff --git a/docs/drs/umn/drs_06_0010.html b/docs/drs/umn/drs_06_0010.html
deleted file mode 100644
index 708f4cb4a..000000000
--- a/docs/drs/umn/drs_06_0010.html
+++ /dev/null
@@ -1,15 +0,0 @@
-
-
-Changing a Single-AZ Task to a Dual-AZ Task
-DRS allows you to change a single-AZ task to a dual-AZ task, improving task reliability while remaining the original task.
- Constraints- A dual-AZ task cannot be changed to a single-AZ task.
- Only tasks in the Incremental or Incremental failed state can be changed.
- Only synchronization tasks from GaussDB Centralized to Kafka can be changed.
-
- Procedure- Log in to the management console.
- Click
 in the upper left corner and select a region and project. - Choose Databases > Data Replication Service. The Data Replication Service page is displayed.
- On the Data Synchronization Management page, select the target task and choose More > Change to Primary/Standby in the Operation column.
- On the displayed page, select the standby AZ. If the task is over a public network, you need to specify the EIP of the standby task and click OK.
- After submitting the change, click Back to Task List. On the Data Synchronization Management page, the task status is Changing to primary/standby. After the task is changed, you can see that the parent task contains one primary and one standby subtask. The standby subtask and the parent task are in the Configuration state.
- Click Edit in the Operation column of the parent task. On the displayed page, test the connections to the source and destination databases. If the connections are normal, click Next.
- On the Confirm Task page, click Start Task. If the task is billed on a yearly/monthly basis, click Pay and Start to go to the payment page.
-
- - Task name
- Description
- Task start time
PrerequisitesYou have logged in to the DRS console.
Procedure- On the Data Synchronization Management page, click the target synchronization task name in the Task Name/ID column.
- On the Basic Information tab, locate the information to be modified in the Task Information area.
- You can click
 to modify the task name and description.- To submit the change, click
 . - To cancel the change, click
 .
+Procedure- On the Data Synchronization Management page, click the target synchronization task name in the Task Name/ID column.
- On the Basic Information tab, locate the information to be modified in the Task Information area.
- You can click to modify the task name and description.
- To submit the change, click .
- To cancel the change, click .
Table 1 Task informationTask Information
|
@@ -38,3 +38,10 @@
+
+
\ No newline at end of file
diff --git a/docs/drs/umn/drs_10_0410.html b/docs/drs/umn/drs_10_0410.html
index f7f38c768..d2e8c0c04 100644
--- a/docs/drs/umn/drs_10_0410.html
+++ b/docs/drs/umn/drs_10_0410.html
@@ -40,7 +40,7 @@
Yes
|
-- MySQL->MySQL
- MySQL->PostgreSQL
- MySQL -> GaussDB Distributed
- MySQL->TaurusDB
- MySQL->Kafka
- MySQL->Oracle
- Microsoft SQL Server->MySQL
- Microsoft SQL Server->PostgreSQL
- Microsoft SQL Server -> GaussDB Distributed
- Microsoft SQL Server->TaurusDB
- Microsoft SQL Server->Microsoft SQL Server
- DDM->DDM
- DDM->MySQL
+ | - MySQL->MySQL
- MySQL -> GaussDB Distributed
- MySQL->TaurusDB
- MySQL->Kafka
- DDM->DDM
- DDM->MySQL
|
|---|
read_task_num
@@ -57,7 +57,7 @@
|
Yes
|
-- MySQL->MySQL
- MySQL->PostgreSQL
- MySQL -> GaussDB Distributed
- MySQL->TaurusDB
- MySQL->Kafka
- MySQL->Oracle
- Microsoft SQL Server->MySQL
- Microsoft SQL Server->PostgreSQL
- Microsoft SQL Server -> GaussDB Distributed
- Microsoft SQL Server->TaurusDB
- Microsoft SQL Server->Microsoft SQL Server
- DDM->DDM
- DDM->MySQL
+ | - MySQL->MySQL
- MySQL -> GaussDB Distributed
- MySQL->TaurusDB
- MySQL->Kafka
- DDM->DDM
- DDM->MySQL
|
shard_task_num
@@ -74,7 +74,7 @@
|
Yes
|
-- MySQL->MySQL
- MySQL->PostgreSQL
- MySQL -> GaussDB Distributed
- MySQL->TaurusDB
- MySQL->Kafka
- MySQL->Oracle
- Oracle->MySQL
- Oracle->PostgreSQL
- Oracle -> GaussDB Distributed
- Oracle->TaurusDB
- Oracle->DDM
- Microsoft SQL Server->MySQL
- Microsoft SQL Server->PostgreSQL
- Microsoft SQL Server -> GaussDB Distributed
- Microsoft SQL Server->TaurusDB
- Microsoft SQL Server->Microsoft SQL Server
- DDM->DDM
- DDM->MySQL
+ | - MySQL->MySQL
- MySQL -> GaussDB Distributed
- MySQL->TaurusDB
- MySQL->Kafka
- Oracle->PostgreSQL
- DDM->DDM
- DDM->MySQL
|
increment_writer_num
@@ -91,7 +91,7 @@
|
Yes
|
-- MySQL->MySQL
- MySQL->PostgreSQL
- MySQL -> GaussDB Distributed
- MySQL->TaurusDB
- MySQL->Kafka
- MySQL->Oracle
- DDM->DDM
- DDM->MySQL
+ | - MySQL->MySQL
- MySQL -> GaussDB Distributed
- MySQL->TaurusDB
- MySQL->Kafka
- DDM->DDM
- DDM->MySQL
|
relay_socket_timeout
@@ -106,7 +106,7 @@
|
Yes
|
-- MySQL->MySQL
- MySQL->PostgreSQL
- MySQL -> GaussDB Distributed
- MySQL->TaurusDB
- MySQL->Kafka
- MySQL->Oracle
- DDM->DDM
- DDM->KAFKA
- DDM->MySQL
+ | - MySQL->MySQL
- MySQL -> GaussDB Distributed
- MySQL->TaurusDB
- MySQL->Kafka
- DDM->DDM
- DDM->MySQL
|
datamove_source_socket_timeout
@@ -114,17 +114,16 @@
| Timeout interval for obtaining data from the source database during the full synchronization phase.
|
The value range depends on the DRS task engine.
-- MySQL -> MySQL: 120000-2147483647 ms
- MySQL -> PostgreSQL: 120000-2147483647 ms
- MySQL -> GaussDB Distributed: 120000-2147483647 ms
- MySQL -> TaurusDB: 120000-2147483647 ms
- MySQL -> Kafka: 120000-2147483647 ms
- MySQL -> CSS/ES: 120000-2147483647 ms
- MySQL -> Oracle: 120000-2147483647 ms
- Oracle -> MySQL: 600000-2147483647 ms
- Oracle -> PostgreSQL: 600000-2147483647 ms
- Oracle -> GaussDB Distributed: 600000-2147483647 ms
- Oracle -> DDM: 600000-2147483647 ms
- Oracle -> Kafka: 600000-2147483647 ms
- Microsoft SQL Server -> MySQL: 360000-2147483647 ms
- Microsoft SQL Server -> PostgreSQL: 360000-2147483647 ms
- Microsoft SQL Server -> GaussDB Distributed: 360000-2147483647 ms
- Microsoft SQL Server -> TaurusDB: 360000-2147483647 ms
- Microsoft SQL Server -> Kafka: 360000-2147483647 ms
- Microsoft SQL Server -> Microsoft SQL Server: 360000-2147483647 ms
+- MySQL -> MySQL: 120000-2147483647 ms
- MySQL -> PostgreSQL: 120000-2147483647 ms
- MySQL -> GaussDB Distributed: 120000-2147483647 ms
- MySQL -> TaurusDB: 120000-2147483647 ms
- MySQL -> Kafka: 120000-2147483647 ms
- MySQL -> Oracle: 120000-2147483647 ms
- Oracle -> MySQL: 600000-2147483647 ms
- Oracle -> PostgreSQL: 600000-2147483647 ms
- Oracle -> GaussDB Distributed: 600000-2147483647 ms
- Oracle -> DDM: 600000-2147483647 ms
- Oracle -> Kafka: 600000-2147483647 ms
- Microsoft SQL Server -> MySQL: 360000-2147483647 ms
- Microsoft SQL Server -> PostgreSQL: 360000-2147483647 ms
- Microsoft SQL Server -> GaussDB Distributed: 360000-2147483647 ms
- Microsoft SQL Server -> TaurusDB: 360000-2147483647 ms
- Microsoft SQL Server -> Kafka: 360000-2147483647 ms
- Microsoft SQL Server -> Microsoft SQL Server: 360000-2147483647 ms
|
The value range depends on the DRS task engine.
-- MySQL -> MySQL: 120000 ms
- MySQL -> PostgreSQL: 120000 ms
- MySQL -> GaussDB Distributed: 120000 ms
- MySQL -> TaurusDB: 120000 ms
- MySQL -> Kafka: 120000 ms
- MySQL -> CSS/ES: 120000 ms
- MySQL->Oracle
: 120000 ms
- - Oracle -> MySQL: 600000 ms
- Oracle -> PostgreSQL: 600000 ms
- Oracle -> GaussDB Distributed: 600000 ms
- Oracle -> GaussDB Centralized: 600000 ms
- Oracle -> TaurusDB: 600000 ms
- Oracle -> DDM: 600000 ms
- Oracle -> Kafka: 600000 ms
- Microsoft SQL Server -> MySQL: 360000 ms
- Microsoft SQL Server -> PostgreSQL: 360000 ms
- Microsoft SQL Server -> GaussDB Distributed: 360000 ms
- Microsoft SQL Server -> GaussDB Centralized: 360000 ms
- Microsoft SQL Server -> TaurusDB: 360000 ms
- Microsoft SQL Server -> Kafka: 360000 ms
- Microsoft SQL Server -> Microsoft SQL Server: 360000 ms
+- MySQL -> MySQL: 120000 ms
- MySQL -> GaussDB Distributed: 120000 ms
- MySQL -> TaurusDB: 120000 ms
- MySQL -> Kafka: 120000 ms
- Oracle -> PostgreSQL: 600000 ms
|
int
|
Yes
|
-- MySQL->MySQL
- MySQL->PostgreSQL
- MySQL -> GaussDB Distributed
- MySQL->TaurusDB
- MySQL->Kafka
- MySQL->CSS/ES
- MySQL->Oracle
- Oracle->MySQL
- Oracle->PostgreSQL
- Oracle -> GaussDB Distributed
- Oracle -> GaussDB Centralized
- Oracle->TaurusDB
- Oracle->DDM
- Oracle->Kafka
- Microsoft SQL Server->MySQL
- Microsoft SQL Server->PostgreSQL
- Microsoft SQL Server -> GaussDB Distributed
- Microsoft SQL Server -> GaussDB Centralized
- Microsoft SQL Server->TaurusDB
- Microsoft SQL Server->Kafka
- Microsoft SQL Server->Microsoft SQL Server
- DDM->DDM
- DDM->KAFKA
- DDM->MySQL
+ | - MySQL->MySQL
- MySQL -> GaussDB Distributed
- MySQL->TaurusDB
- MySQL->Kafka
- Oracle->PostgreSQL
- DDM->DDM
- DDM->MySQL
|
shard_length
@@ -139,7 +138,7 @@
|
Yes
|
-- Oracle->MySQL
- Oracle->PostgreSQL
- Oracle -> GaussDB Distributed
- Oracle -> GaussDB Centralized
- Oracle->TaurusDB
- Oracle->DDM
- Microsoft SQL Server->MySQL
- Microsoft SQL Server->PostgreSQL
- Microsoft SQL Server -> GaussDB Distributed
- Microsoft SQL Server -> GaussDB Centralized
- Microsoft SQL Server->TaurusDB
- Microsoft SQL Server->Microsoft SQL Server
- DDM->DDM
- DDM->DWS
- DDM->KAFKA
- DDM->MySQL
+ | - Oracle->PostgreSQL
- DDM->DDM
- DDM->MySQL
|
datamove_fetchsize
@@ -149,18 +148,14 @@
| 10-10000
|
The value range depends on the DRS task engine.
-- Oracle -> MySQL: 1000
- Oracle->PostgreSQL
: 1000
- - Oracle -> GaussDB Distributed: 10000
- Oracle->TaurusDB
: 1000
- - Oracle->DDM
: 1000
- - Oracle->Kafka
: 1000
+
-
|
int
|
Yes
|
-- Oracle->MySQL
- Oracle->PostgreSQL
- Oracle -> GaussDB Distributed
- Oracle->TaurusDB
- Oracle->DDM
+ |
|
datamove_copy_mode
@@ -176,7 +171,7 @@
|
Yes
|
-- Oracle->PostgreSQL
- Oracle -> GaussDB Distributed
- Microsoft SQL Server->PostgreSQL
- Microsoft SQL Server -> GaussDB Distributed
+ |
|
datamove_split_partition
@@ -191,7 +186,7 @@
|
Yes
|
-- Oracle->MySQL
- Oracle->PostgreSQL
- Oracle -> GaussDB Distributed
- Oracle->TaurusDB
- Oracle->DDM
+ |
|
read_log_num
@@ -206,7 +201,7 @@
|
Yes
|
-- Oracle->MySQL
- Oracle->PostgreSQL
- Oracle -> GaussDB Distributed
- Oracle->TaurusDB
- Oracle->DDM
- Oracle->Kafka
+ |
|
logminer_fetchsize
@@ -221,7 +216,7 @@
|
Yes
|
-- Oracle->MySQL
- Oracle->PostgreSQL
- Oracle -> GaussDB Distributed
- Oracle -> GaussDB Centralized
- Oracle->TaurusDB
- Oracle->DDM
- Oracle->Kafka
+ |
|
deal_hot_table
@@ -236,7 +231,7 @@
|
Yes
|
-- Oracle->MySQL
- Oracle->PostgreSQL
- Oracle -> GaussDB Distributed
- Oracle -> GaussDB Centralized
- Oracle->TaurusDB
- Oracle->DDM
- Microsoft SQL Server->MySQL
- Microsoft SQL Server->PostgreSQL
- Microsoft SQL Server -> GaussDB Distributed
- Microsoft SQL Server -> GaussDB Centralized
- Microsoft SQL Server->TaurusDB
- Microsoft SQL Server->Microsoft SQL Server
- DDM->DDM
- DDM->DWS
- DDM->MySQL
+ | - Oracle->PostgreSQL
- DDM->DDM
- DDM->MySQL
|
skip_all_ddl
@@ -251,7 +246,7 @@
|
Yes
|
-- Oracle->MySQL
- Oracle->PostgreSQL
- Oracle -> GaussDB Distributed
- Oracle -> GaussDB Centralized
- Oracle->TaurusDB
- Oracle->DDM
- Oracle->Kafka
- DDM->DDM
- DDM->MySQL
+ | - Oracle->PostgreSQL
- DDM->DDM
- DDM->MySQL
|
replace_invalid_time
@@ -266,7 +261,7 @@
|
Yes
|
-- Oracle->PostgreSQL
- Oracle -> GaussDB Distributed
- Oracle -> GaussDB Centralized
- Microsoft SQL Server->PostgreSQL
- Microsoft SQL Server -> GaussDB Distributed
- Microsoft SQL Server -> GaussDB Centralized
+ |
|
replace_0x00
@@ -281,7 +276,7 @@
|
Yes
|
-- Oracle->PostgreSQL
- Oracle -> GaussDB Distributed
- Microsoft SQL Server->PostgreSQL
- Microsoft SQL Server -> GaussDB Distributed
+ |
|
replace_0xefbfbd
@@ -296,67 +291,9 @@
|
Yes
|
-- Oracle->PostgreSQL
- Oracle -> GaussDB Distributed
- Microsoft SQL Server->PostgreSQL
- Microsoft SQL Server -> GaussDB Distributed
+ |
|
-big_column_value_limit
- |
-Large field filtering threshold.
- |
-0-99 * 1024 * 1024
- |
-0
- |
-int
- |
-Yes
- |
-
- |
-
-big_column_value_replace
- |
-Large field replacement characters.
- |
-A character string without special characters
- |
--
- |
-string
- |
-Yes
- |
-
- |
-
-datamove_replace_invalid_time
- |
-Whether to replace the source time type with the default time when the source time type is invalid after being synchronized to the destination database during a full synchronization. Default timestamp: 1970-01-01 00:00:00; default date: 1970-01-01; default time: 00:00:00. true: the source time type is replaced with the default time; false: the source time type is not replaced with the default time.
- |
-true/false
- |
-false
- |
-boolean
- |
-Yes
- |
- |
-
-increment_replace_invalid_time
- |
-Whether to replace the source time type with the default time when the source time type is invalid after being synchronized to the destination database during an incremental synchronization. Default timestamp: 1970-01-01 00:00:00; default date: 1970-01-01; default time: 00:00:00. true: the source time type is replaced with the default time; false: the source time type is not replaced with the default time.
- |
-true/false
- |
-false
- |
-boolean
- |
-Yes
- |
- |
-
datamove_chunk_size
|
The amount of data transferred to the destination database at a time, in bytes.
diff --git a/docs/drs/umn/drs_10_0412.html b/docs/drs/umn/drs_10_0412.html
index 0cbc87d9b..85cd4d883 100644
--- a/docs/drs/umn/drs_10_0412.html
+++ b/docs/drs/umn/drs_10_0412.html
@@ -2,7 +2,7 @@
Restarting a Synchronization Task
DRS allows you to restart a synchronization task after task parameters are changed in Changing Task Parameters.
- Constraints- Tasks in the Full, Full synchronization failed, Incremental, or Incremental synchronization failed state can be restarted.
- Only the synchronization tasks of the following engines can be restarted:
- MySQL->MySQL
- MySQL->PostgreSQL
- MySQL -> GaussDB Distributed
- MySQL -> GaussDB Centralized
- MySQL->TaurusDB
- MySQL->Kafka
- MySQL->CSS/ES
- MySQL->Oracle
+Constraints- Tasks in the Full, Full synchronization failed, Incremental, or Incremental synchronization failed state can be restarted.
- Only the synchronization tasks of the following engines can be restarted:
- MySQL->MySQL
- MySQL -> GaussDB Distributed
- MySQL->TaurusDB
- MySQL->Kafka
Method 1- In the task list on the Data Synchronization Management page, locate the target task and click Restart in the Operation column.
- In the displayed Restart Task dialog box, click Yes.
diff --git a/docs/drs/umn/drs_11_0009.html b/docs/drs/umn/drs_11_0009.html
index f91503b46..8e123a750 100644
--- a/docs/drs/umn/drs_11_0009.html
+++ b/docs/drs/umn/drs_11_0009.html
@@ -153,29 +153,6 @@
|
MySQL -> PostgreSQL Synchronization
- Table 4 Checking whether the source database user has sufficient permissions Check Item
- |
-Whether the source database user has sufficient permissions
- |
-
|---|
-Description
- |
-Check whether the source database user permissions meet the migration requirements. If the permissions are insufficient, the migration will fail.
- |
-
|---|
-Failure Cause and Handling Suggestion
- |
-Failure cause: The source database user does not have sufficient permissions. During the incremental synchronization, the SELECT permission on INFORMATION_SCHEMA is required.
-Handling suggestions: Grant the source database user the corresponding permissions.
-Run the following command:
-grant select on [dbname];
- |
-
|---|
-
-
-
-
diff --git a/docs/drs/umn/drs_11_0013.html b/docs/drs/umn/drs_11_0013.html
index c7cae5c51..0cc767ec0 100644
--- a/docs/drs/umn/drs_11_0013.html
+++ b/docs/drs/umn/drs_11_0013.html
@@ -31,9 +31,9 @@
Handling suggestion: Modify the character sets.
Run commands to modify the self-created source database.
- Check whether source and destination database character sets are consistent.
show variables like "character_set_server"\G;
-
+
- Modify the character set of the source database server.
set character_set_server='utf8';
-
+
|---|
@@ -52,3 +52,10 @@
+
+
\ No newline at end of file
diff --git a/docs/drs/umn/drs_11_0014.html b/docs/drs/umn/drs_11_0014.html
index 2373fe5b3..7eb625921 100644
--- a/docs/drs/umn/drs_11_0014.html
+++ b/docs/drs/umn/drs_11_0014.html
@@ -31,9 +31,9 @@
Failure cause: The binlog function is disabled on the source database.
Handling suggestion:
- If the source is an on-premises database, perform the following operations to enable binlog.
- Run the following command to check whether binlog is enabled:
show variables like "log_bin"\G;
-
- - If binlog is disabled, add log-bin = mysql-bin followed by [mysqld] in the MySQL configuration file my.cnf or my.ini.
- - Restart the database.
+
+ - If binlog is disabled, add log-bin = mysql-bin followed by [mysqld] in the MySQL configuration file my.cnf or my.ini.
+ - Restart the database.
- If the source is an RDS instance, enable binlog by referring to the RDS official documentation.
|
@@ -49,3 +49,10 @@
+
+
\ No newline at end of file
diff --git a/docs/drs/umn/drs_11_0075.html b/docs/drs/umn/drs_11_0075.html
deleted file mode 100644
index 6900e1bbe..000000000
--- a/docs/drs/umn/drs_11_0075.html
+++ /dev/null
@@ -1,36 +0,0 @@
-
-
-Checking Whether Destination Contains the Configured Database
-MySQL > PostgreSQL
- Table 1 Checking whether destination contains the configured databaseCheck Item
- |
-Whether the destination contains the configured database.
- |
-
|---|
-Description
- |
-Databases and schemas cannot be migrated. You need to manually create databases and schemas on the destination database. Otherwise, the migration will fail.
- |
-
|---|
-Failure Cause and Handling Suggestion
-
- |
-Failure cause: Databases cannot be migrated from MySQL to PostgreSQL.
-Handling suggestion: In the destination database, manually create databases and schemas with the same names as those of the source database.
- |
-
|---|
-Failure cause: The objects to be synchronized already exist in the destination database.
-Handling suggestions: Delete the tables to be synchronized from the destination database or select the tables that do not exist in the destination database for synchronization.
- |
-
-
-
-
-
- DRS real-time synchronization supports multiple topology types. You can plan the topology types as required. For details, see the following content.
To ensure data consistency, do not modify the synchronization objects in the destination database.
- One-to-One Real-Time Synchronization
+ One-to-One Real-Time Synchronization
You can create a one-to-one synchronization task.
- One-to-Many Real-Time Synchronization
+ One-to-Many Real-Time Synchronization
You need to create multiple synchronization tasks to implement one-to-many real-time synchronization. For example, to synchronize data from instance A to instances B, C, and D, you need to create three synchronization tasks.
- Many-to-One Real-Time Synchronization
+ Many-to-One Real-Time Synchronization
You need to create multiple synchronization tasks to implement many-to-one real-time synchronization. For example, to synchronize data from instances B, C, and D to instance A, you need to create three synchronization tasks. Multiple tables can be synchronized to one table.
@@ -20,3 +20,10 @@
+
+
\ No newline at end of file
diff --git a/docs/drs/umn/drs_12_0006.html b/docs/drs/umn/drs_12_0006.html
index bf88df681..cb441d15b 100644
--- a/docs/drs/umn/drs_12_0006.html
+++ b/docs/drs/umn/drs_12_0006.html
@@ -14,8 +14,6 @@
- Checking Whether the Extensions Are Supported
- - Checking Whether Destination Contains the Configured Database
-
- Checking Whether the Destination DB Instance Is Available
diff --git a/docs/drs/umn/drs_16_0001.html b/docs/drs/umn/drs_16_0001.html
index 5a530aaa7..4146e7370 100644
--- a/docs/drs/umn/drs_16_0001.html
+++ b/docs/drs/umn/drs_16_0001.html
@@ -9,7 +9,7 @@
print("waiting...");
sleep(1000);
}
- - If the command output is waiting, the balancer is migrating chunks. In this case, do not disable the balancer. Otherwise, data inconsistency may occur.
Figure 1 Viewing the command output
+- If the command output is waiting, the balancer is migrating chunks. In this case, do not disable the balancer. Otherwise, data inconsistency may occur.
Figure 1 Viewing the command output
- If no command output is displayed, the balancer is not migrating any chunks. In this case, you can disable the balancer:
- Disable the balancer.
Operation Suggestions- To ensure that there is sufficient space during task creation, you are advised to calculate the total data volume of the source database and plan how to allocate the disk space of the destination instance. The remaining disk space must be greater than the total data volume of the source database. For example, if the data volume of source system1 is 1 GB, the data volume of source system2 is 3 GB, and the data volume of source system3 is 6 GB, the remaining disk space of the destination instance must be greater than 10 GB.
- To improve the performance of the destination MySQL database, you are advised to use the Save Change function to configure common parameters (except max_connections). For performance parameters, you need to manually change the parameter values based on the specifications of the destination database.
- When you create a many-to-one synchronization task, the task created later may block the task created earlier. This is because each synchronization task involves index creation. When an index is created, a schema lock may occur on the destination database, which blocks the synchronization of other tables in the schema. As a result, the previously created tasks cannot be synchronized. To avoid this problem, you are advised to set Start Time to Start at a specified time to start a task during off-peak hours.
- For many-to-one synchronization tasks that involve the synchronization of the same table, DDL operations cannot be performed on source databases. Otherwise, all synchronization tasks fail.
Many-to-One Data MigrationData migration aims to migrate the entire database. Multiple databases can be migrated at the instance level. Databases with the same name in the source system cannot be migrated and database name mapping is not supported.
- Figure 1 Many-to-one data migration
+ Figure 1 Many-to-one data migration
Flow ChartWhen creating a task, ensure that the second task is created after the first task has entered the full migration state.
- Figure 2 Flow chart
+ Figure 2 Flow chart
@@ -17,3 +17,10 @@
+
+
\ No newline at end of file
diff --git a/docs/drs/umn/drs_change_history.html b/docs/drs/umn/drs_change_history.html
index d37aea301..1dda20c23 100644
--- a/docs/drs/umn/drs_change_history.html
+++ b/docs/drs/umn/drs_change_history.html
@@ -8,15 +8,30 @@
-2025-02-10
+ | 2025-09-09
|
-This issue is the seventeenth official release, which incorporates the following change:
+ | This is the twentieth official release.
+Added the following content:
+- Supported self-built-to-self-built synchronization from MySQL to MySQL.
+Deleted the following content:
+- Deleted self-built-to-self-built synchronization from MySQL to Kafka.
- Deleted GaussDB Centralized, CSS/ES, MySQL -> Oracle, DDM -> Oracle, DDM -> GaussDB(DWS), and Microsoft SQL Server from task parameters in real-time synchronization parameter management.
- Deleted the section "Changing a Single-AZ Task to a Dual-AZ Task" in "Real-Time Synchronization Task Life Cycle".
- Deleted the section "Checking Whether Destination Contains the Configured Database" in "Destination DB Instance Statuses" of Troubleshooting.
+ |
+
+2025-07-17
+ |
+This issue is the nineteenth official release, which incorporates the following change:
+- Modified the description of DRS synchronization object names.
+ |
+
+2025-02-10
+ |
+This issue is the eighteenth official release, which incorporates the following change:
- Added six types of specifications for DRS synchronization tasks.
|
2024-06-11
|
-This issue is the sixteenth official release, which incorporates the following changes:
+ | This issue is the seventeenth official release, which incorporates the following changes:
- Optimized the content in "Processing Data."
- Optimized the content in "Importing Synchronization Objects."
- Supported real-time synchronization for PostgreSQL 15.
|
@@ -83,7 +98,7 @@
2021-08-30
|
This issue is the sixth official release, which incorporates the following changes:
-- Supported real-time synchronization from MySQL to TaurusDB across AZs.
+- Supported real-time synchronization from MySQL to TaurusDB Cluster across AZs.
|
2021-05-31
diff --git a/docs/drs/umn/en-us_image_0000001918683154.png b/docs/drs/umn/en-us_image_0000001918683154.png
deleted file mode 100644
index fafc744b1..000000000
Binary files a/docs/drs/umn/en-us_image_0000001918683154.png and /dev/null differ
diff --git a/docs/drs/umn/public_sys-resources/imageclose.gif b/docs/drs/umn/public_sys-resources/imageclose.gif
new file mode 100644
index 000000000..3a3344af4
Binary files /dev/null and b/docs/drs/umn/public_sys-resources/imageclose.gif differ
diff --git a/docs/drs/umn/public_sys-resources/imageclosehover.gif b/docs/drs/umn/public_sys-resources/imageclosehover.gif
new file mode 100644
index 000000000..8699d5e36
Binary files /dev/null and b/docs/drs/umn/public_sys-resources/imageclosehover.gif differ
diff --git a/docs/drs/umn/public_sys-resources/imagemax.gif b/docs/drs/umn/public_sys-resources/imagemax.gif
new file mode 100644
index 000000000..99c07dc25
Binary files /dev/null and b/docs/drs/umn/public_sys-resources/imagemax.gif differ
diff --git a/docs/drs/umn/public_sys-resources/imagemaxhover.gif b/docs/drs/umn/public_sys-resources/imagemaxhover.gif
new file mode 100644
index 000000000..d01d77d6e
Binary files /dev/null and b/docs/drs/umn/public_sys-resources/imagemaxhover.gif differ
diff --git a/docs/drs/umn/public_sys-resources/macFFBgHack.png b/docs/drs/umn/public_sys-resources/macFFBgHack.png
new file mode 100644
index 000000000..ec811470c
Binary files /dev/null and b/docs/drs/umn/public_sys-resources/macFFBgHack.png differ
|
|---|
|---|
|---|
|
