forked from docs/doc-exports
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
270 lines
53 KiB
HTML
270 lines
53 KiB
HTML
<a name="mrs_01_0501"></a><a name="mrs_01_0501"></a>
|
|
|
|
<h1 class="topictitle1">Configuring HBase Replication</h1>
|
|
<div id="body1589421623825"><div class="section" id="mrs_01_0501__section55510827154955"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_0501__p83168154955">As a key feature to ensure high availability of the HBase cluster system, HBase cluster replication provides HBase with remote data replication in real time. It provides basic O&M tools, including tools for maintaining and re-establishing active/standby relationships, verifying data, and querying data synchronization progress. To achieve real-time data replication, you can replicate data from the HBase cluster to another one. </p>
|
|
</div>
|
|
<div class="section" id="mrs_01_0501__section748518154955"><h4 class="sectiontitle">Prerequisites</h4><ul id="mrs_01_0501__ul60630031154955"><li id="mrs_01_0501__li8799372154955">The active and standby clusters have been successfully installed and started (the cluster status is <strong id="mrs_01_0501__b19570111213419">Running</strong> on the <strong id="mrs_01_0501__b95705126414">Active Clusters</strong> page), and you have the administrator rights of the clusters.</li></ul>
|
|
</div>
|
|
<ul id="mrs_01_0501__ul12085484154955"><li id="mrs_01_0501__li41660498154955">The network between the active and standby clusters is normal and ports can be used properly.</li><li id="mrs_01_0501__li39400166154955">Cross-cluster mutual trust has been configured. For details, see <a href="https://docs.otc.t-systems.com/usermanual/mrs/mrs_01_0354.html" target="_blank" rel="noopener noreferrer">Configuring Cross-Cluster Mutual Trust Relationships</a>. </li><li id="mrs_01_0501__li27466911181447">If historical data exists in the active cluster and needs to be synchronized to the standby cluster, cross-cluster replication must be configured for the active and standby clusters. For details, see <a href="mrs_01_0502.html">Enabling Cross-Cluster Copy</a>.</li><li id="mrs_01_0501__li37296868154955">Time is consistent between the active and standby clusters and the Network Time Protocol (NTP) service on the active and standby clusters uses the same time source.</li><li id="mrs_01_0501__li127496154955">Mapping relationships between the names of all hosts in the active and standby clusters and service IP addresses have been configured in the <span class="filepath" id="mrs_01_0501__filepath17461135918131"><b>/etc/hosts</b></span> file by appending <strong id="mrs_01_0501__b168494714716">192.***.***.*** host1</strong> to the <strong id="mrs_01_0501__b73931611184710">hosts</strong> file.</li><li id="mrs_01_0501__li1147471154955">The network bandwidth between the active and standby clusters is determined based on service volume, which cannot be less than the possible maximum service volume.</li></ul>
|
|
<div class="section" id="mrs_01_0501__section10327240154955"><h4 class="sectiontitle">Constraints</h4><ul id="mrs_01_0501__ul31200129154955"><li id="mrs_01_0501__li12365706154955">Despite that HBase cluster replication provides the real-time data replication function, the data synchronization progress is determined by several factors, such as the service loads in the active cluster and the health status of processes in the standby cluster. In normal cases, the standby cluster should not take over services. In extreme cases, system maintenance personnel and other decision makers determine whether the standby cluster takes over services according to the current data synchronization indicators.</li></ul>
|
|
</div>
|
|
<ul id="mrs_01_0501__ul44182490154955"><li id="mrs_01_0501__li62098090154955">Currently, the replication function supports only one active cluster and one standby cluster in HBase.</li><li id="mrs_01_0501__li22011899154955">Typically, do not perform operations on data synchronization tables in the standby cluster, such as modifying table properties or deleting tables. If any misoperation on the standby cluster occurs, data synchronization between the active and standby clusters will fail and data of the corresponding table in the standby cluster will be lost.</li><li id="mrs_01_0501__li63889366154955">If the replication function of HBase tables in the active cluster is enabled for data synchronization, after modifying the structure of a table in the active cluster, you need to manually modify the structure of the corresponding table in the standby cluster to ensure table structure consistency.</li></ul>
|
|
<div class="section" id="mrs_01_0501__section38133389154955"><h4 class="sectiontitle">Procedure</h4><p id="mrs_01_0501__p1796791154955"><strong id="mrs_01_0501__b210750102219">Enable the replication function for the active cluster to synchronize data written by Put.</strong></p>
|
|
</div>
|
|
<ol id="mrs_01_0501__ol11322402154955"><li id="mrs_01_0501__li155891430132615"><a name="mrs_01_0501__li155891430132615"></a><a name="li155891430132615"></a><span>Log in to the service page.</span><p><p id="mrs_01_0501__p1528815213240">For versions earlier than MRS 1.9.2: Log in to <a href="https://docs.otc.t-systems.com/usermanual/mrs/mrs_01_0102.html" target="_blank" rel="noopener noreferrer">MRS Manager</a>, and choose <strong id="mrs_01_0501__b25913815159">Services</strong>.</p>
|
|
<p id="mrs_01_0501__p463355062416">For MRS 1.9.2 or later: Click the cluster name on the MRS console and choose <strong id="mrs_01_0501__b046217256434">Components</strong>.</p>
|
|
</p></li><li id="mrs_01_0501__li1966213718714"><span>Go to the <strong id="mrs_01_0501__b17128165751013">All Configurations</strong> page of the HBase service. For details, see <a href="mrs_01_2125.html">Modifying Cluster Service Configuration Parameters</a>.</span><p><div class="note" id="mrs_01_0501__note3349205551313"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_0501__p1832113512387">For clusters of MRS 1.9.2 or later:</p>
|
|
<p id="mrs_01_0501__p158007468574">If the <strong id="mrs_01_0501__b396711645817">Components</strong> tab is unavailable, complete IAM user synchronization first. (On the <strong id="mrs_01_0501__b169672016175814">Dashboard</strong> page, click <strong id="mrs_01_0501__b7967516195814">Synchronize</strong> on the right side of <span class="parmname" id="mrs_01_0501__parmname896718168583"><b>IAM User Sync</b></span> to synchronize IAM users.)</p>
|
|
</div></div>
|
|
</p></li><li id="mrs_01_0501__li66750118154955"><span>Choose <strong id="mrs_01_0501__b04661948202219">RegionServer</strong> > <strong id="mrs_01_0501__b94719483225">Replication</strong> and check whether the value of <strong id="mrs_01_0501__b104710486220">hbase.replication</strong> is <strong id="mrs_01_0501__b204721348172220">true</strong>. If the value is <strong id="mrs_01_0501__b15205218236">false</strong>, set <strong id="mrs_01_0501__b42114252318">hbase.replication</strong> to <strong id="mrs_01_0501__b32112182310">true</strong>.</span><p><div class="note" id="mrs_01_0501__note167051426175"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_0501__p070504201710">In MRS 2.<em id="mrs_01_0501__i115350715123">x</em>, this configuration has been removed. Skip this step.</p>
|
|
</div></div>
|
|
</p></li><li id="mrs_01_0501__li63880153154955"><span>(Optional) Set configuration items listed in <a href="#mrs_01_0501__table6909942154955">Table 1</a>. You can set the parameters based on the description or use the default values.</span><p>
|
|
<div class="tablenoborder"><a name="mrs_01_0501__table6909942154955"></a><a name="table6909942154955"></a><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_0501__table6909942154955" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Optional configuration items</caption><thead align="left"><tr id="mrs_01_0501__row52059452154955"><th align="left" class="cellrowborder" valign="top" width="18.98%" id="mcps1.3.7.4.2.1.2.5.1.1"><p id="mrs_01_0501__p56066123154955">Navigation Path</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="28.01%" id="mcps1.3.7.4.2.1.2.5.1.2"><p id="mrs_01_0501__p45062147154955">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="13.350000000000001%" id="mcps1.3.7.4.2.1.2.5.1.3"><p id="mrs_01_0501__p26155275154955">Default Value</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="39.660000000000004%" id="mcps1.3.7.4.2.1.2.5.1.4"><p id="mrs_01_0501__p38202507154955">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_0501__row7395372154955"><td class="cellrowborder" rowspan="2" valign="top" width="18.98%" headers="mcps1.3.7.4.2.1.2.5.1.1 "><p id="mrs_01_0501__p62154261154955">HMaster > Performance</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="28.01%" headers="mcps1.3.7.4.2.1.2.5.1.2 "><p id="mrs_01_0501__p1330415154955">hbase.master.logcleaner.ttl</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="13.350000000000001%" headers="mcps1.3.7.4.2.1.2.5.1.3 "><p id="mrs_01_0501__p40654818154955">600000</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="39.660000000000004%" headers="mcps1.3.7.4.2.1.2.5.1.4 "><p id="mrs_01_0501__p4705941154955">Time to live (TTL) of HLog files. If the value is set to <strong id="mrs_01_0501__b187151463248">604800000</strong> (unit: millisecond), the retention period of HLog is 7 days.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_0501__row42353470154955"><td class="cellrowborder" valign="top" headers="mcps1.3.7.4.2.1.2.5.1.1 "><p id="mrs_01_0501__p8079028154955">hbase.master.cleaner.interval</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.7.4.2.1.2.5.1.2 "><p id="mrs_01_0501__p50421568154955">60000</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.7.4.2.1.2.5.1.3 "><p id="mrs_01_0501__p57615175154955">Interval for the HMaster to delete historical HLog files. The HLog that exceeds the configured period will be automatically deleted. You are advised to set it to the maximum value to save more HLogs.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_0501__row48774533154955"><td class="cellrowborder" rowspan="5" valign="top" width="18.98%" headers="mcps1.3.7.4.2.1.2.5.1.1 "><p id="mrs_01_0501__p58423123154955">RegionServer > Replication</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="28.01%" headers="mcps1.3.7.4.2.1.2.5.1.2 "><p id="mrs_01_0501__p34652497154955">replication.source.size.capacity</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="13.350000000000001%" headers="mcps1.3.7.4.2.1.2.5.1.3 "><p id="mrs_01_0501__p55388839154955">16777216</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="39.660000000000004%" headers="mcps1.3.7.4.2.1.2.5.1.4 "><p id="mrs_01_0501__p57310974154955">Maximum size of edits, in bytes. If the edit size exceeds the value, HLog edits will be sent to the standby cluster.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_0501__row46036721154955"><td class="cellrowborder" valign="top" headers="mcps1.3.7.4.2.1.2.5.1.1 "><p id="mrs_01_0501__p37986924154955">replication.source.nb.capacity</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.7.4.2.1.2.5.1.2 "><p id="mrs_01_0501__p57041979154955">25000</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.7.4.2.1.2.5.1.3 "><p id="mrs_01_0501__p56997575154955">Maximum number of edits, which is another condition for triggering HLog edits to be sent to the standby cluster. After data in the active cluster is synchronized to the standby cluster, the active cluster reads and sends data in HLog according to this parameter value. This parameter is used together with <strong id="mrs_01_0501__b1584662311272">replication.source.size.capacity</strong>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_0501__row43216127154955"><td class="cellrowborder" valign="top" headers="mcps1.3.7.4.2.1.2.5.1.1 "><p id="mrs_01_0501__p10845360154955">replication.source.maxretriesmultiplier</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.7.4.2.1.2.5.1.2 "><p id="mrs_01_0501__p6058962154955">10</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.7.4.2.1.2.5.1.3 "><p id="mrs_01_0501__p21013900154955">Maximum number of retries when an exception occurs during replication.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_0501__row54907378154955"><td class="cellrowborder" valign="top" headers="mcps1.3.7.4.2.1.2.5.1.1 "><p id="mrs_01_0501__p18312644154955">replication.source.sleepforretries</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.7.4.2.1.2.5.1.2 "><p id="mrs_01_0501__p6929219154955">1000</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.7.4.2.1.2.5.1.3 "><p id="mrs_01_0501__p24395862154955">Retry interval (unit: ms)</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_0501__row18236171154955"><td class="cellrowborder" valign="top" headers="mcps1.3.7.4.2.1.2.5.1.1 "><p id="mrs_01_0501__p734872154955">hbase.regionserver.replication.handler.count</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.7.4.2.1.2.5.1.2 "><p id="mrs_01_0501__p59524653154955">6</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.7.4.2.1.2.5.1.3 "><p id="mrs_01_0501__p56767603154955">Number of replication RPC server instances on RegionServer</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</p></li></ol>
|
|
<p id="mrs_01_0501__p41146384154955"><strong id="mrs_01_0501__b71751026104818">Enable the replication function for the active cluster to synchronize data written by bulkload.</strong></p>
|
|
<ol start="5" id="mrs_01_0501__ol44522785154955"><li id="mrs_01_0501__li65160752154955"><a name="mrs_01_0501__li65160752154955"></a><a name="li65160752154955"></a><span>Determine whether to enable bulkload replication.</span><p><div class="note" id="mrs_01_0501__note46277048984"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_0501__p391601249711">If bulkload import is used and data needs to be synchronized, you need to enable Bulkload replication.</p>
|
|
</div></div>
|
|
<p id="mrs_01_0501__p49575864154955">If yes, go to <a href="#mrs_01_0501__li57688977154955">6</a>.</p>
|
|
<p id="mrs_01_0501__p56222010154955">If no, go to <a href="#mrs_01_0501__li6210082154955">10</a>.</p>
|
|
</p></li><li id="mrs_01_0501__li57688977154955"><a name="mrs_01_0501__li57688977154955"></a><a name="li57688977154955"></a><span>Go to the <strong id="mrs_01_0501__b139596521126">All Configurations</strong> page of the HBase service parameters by referring to <a href="mrs_01_2125.html">Modifying Cluster Service Configuration Parameters</a>.</span></li><li id="mrs_01_0501__li42295589154955"><span>On the HBase configuration interface of the active and standby clusters, search for <strong id="mrs_01_0501__b169641627102815">hbase.replication.cluster.id</strong> and modify it. It specifies the HBase ID of the active and standby clusters. For example, the HBase ID of the active cluster is set to <strong id="mrs_01_0501__b1920923311287">replication1</strong> and the HBase ID of the standby cluster is set to <strong id="mrs_01_0501__b3325455132815">replication2</strong> for connecting the active cluster to the standby cluster. To save data overhead, the parameter value length is not recommended to exceed 30.</span></li><li id="mrs_01_0501__li3244131341713"><a name="mrs_01_0501__li3244131341713"></a><a name="li3244131341713"></a><span>On the HBase configuration interface of the standby cluster, search for <strong id="mrs_01_0501__b14277360338">hbase.replication.conf.dir</strong> and modify it. It specifies the HBase configurations of the active cluster client used by the standby cluster and is used for data replication when the bulkload data replication function is enabled. The parameter value is a path name, for example, <strong id="mrs_01_0501__b6375324151316">/home</strong>.</span><p><div class="note" id="mrs_01_0501__note0771101318917"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="mrs_01_0501__ul183455195611"><li id="mrs_01_0501__li1498465282715">In versions earlier than MRS 3.x, you do not need to set this parameter. Skip <a href="#mrs_01_0501__li3244131341713">8</a>.</li><li id="mrs_01_0501__li1526159185615">When bulkload replication is enabled, you need to manually place the HBase client configuration files (<strong id="mrs_01_0501__b1261104534018">core-site.xml</strong>, <strong id="mrs_01_0501__b6621345114014">hdfs-site.xml</strong>, and <strong id="mrs_01_0501__b166294574012">hbase-site.xml</strong>) in the active cluster on all RegionServer nodes in the standby cluster. The actual path for placing the configuration file is <span class="filepath" id="mrs_01_0501__filepath1535414652413"><b>${hbase.replication.conf.dir}/${hbase.replication.cluster.id}</b></span>. For example, if <strong id="mrs_01_0501__b1396611594195">hbase.replication.conf.dir</strong> of the standby cluster is set to <strong id="mrs_01_0501__b1967195991915">/home</strong> and <strong id="mrs_01_0501__b196712593194">hbase.replication.cluster.id</strong> of the active cluster is set to <strong id="mrs_01_0501__b896755917197">replication1</strong>, the actual path for placing the configuration files in the standby cluster is <strong id="mrs_01_0501__b196715596192">/home/replication1</strong>. You also need to change the corresponding directory and file permissions by running the <strong id="mrs_01_0501__b12950741172619">chown -R omm:wheel /home/replication1</strong> command.</li><li id="mrs_01_0501__li59631495619">You can obtain the client configuration files from the client in the active cluster, for example, the <span class="filepath" id="mrs_01_0501__filepath5748185916261"><b>/opt/client/HBase/hbase/conf</b></span> path. For details about how to update the configuration file, see <a href="https://docs.otc.t-systems.com/usermanual/mrs/mrs_01_0089.html" target="_blank" rel="noopener noreferrer">Updating a Client</a>.</li></ul>
|
|
</div></div>
|
|
</p></li><li id="mrs_01_0501__li49438752154955"><span>On the HBase configuration page of the active cluster, search for and change the value of <strong id="mrs_01_0501__b315633212458">hbase.replication.bulkload.enabled</strong> to <strong id="mrs_01_0501__b3652138164512">true</strong> to enable bulkload replication.</span></li></ol>
|
|
<p id="mrs_01_0501__p45115985154955"><strong id="mrs_01_0501__b205516120911">Restarting the HBase service and install the client</strong></p>
|
|
<ol start="10" id="mrs_01_0501__ol30516170154955"><li id="mrs_01_0501__li6210082154955"><a name="mrs_01_0501__li6210082154955"></a><a name="li6210082154955"></a><span>Save the configurations and restart HBase.</span></li><li id="mrs_01_0501__li11385192216347"><a name="mrs_01_0501__li11385192216347"></a><a name="li11385192216347"></a><span>In the active and standby clusters of MRS 1.9.2 or earlier, choose <strong id="mrs_01_0501__b1410344019461">Cluster</strong> > <strong id="mrs_01_0501__b6103240174616">Dashboard</strong> > <strong id="mrs_01_0501__b610417406465">More</strong> > <strong id="mrs_01_0501__b131042405468">Download Client</strong> of MRS 1.9.2 or later, choose <strong id="mrs_01_0501__b142171950174612">Cluster</strong> > <strong id="mrs_01_0501__b1521711508466">Dashboard</strong> > <strong id="mrs_01_0501__b1921875084615">More</strong> > <strong id="mrs_01_0501__b1621805034614">Download Client</strong>. For details about how to update the client configuration file, see <a href="https://docs.otc.t-systems.com/usermanual/mrs/mrs_01_0089.html" target="_blank" rel="noopener noreferrer">Updating a Client</a>.</span></li></ol>
|
|
<p id="mrs_01_0501__p562774110491"><strong id="mrs_01_0501__b165261018163714">Synchronize table data of the active cluster. (Skip this step if the active cluster has no data.)</strong></p>
|
|
<ol start="12" id="mrs_01_0501__ol2512178117406"><li id="mrs_01_0501__li12641483154955"><a name="mrs_01_0501__li12641483154955"></a><a name="li12641483154955"></a><span>Access the HBase shell of the active cluster as user <strong id="mrs_01_0501__b1346896175110">hbase</strong>.</span><p><ol type="a" id="mrs_01_0501__ol574666129352"><li id="mrs_01_0501__li474374619352">On the active management node where the client has been updated, run the following command to go to the client directory:<p id="mrs_01_0501__aa4134e1860f64d04b8af130e80d60c1f"><a name="mrs_01_0501__li474374619352"></a><a name="li474374619352"></a><strong id="mrs_01_0501__af7849077e90740ccbbe480ab0aa7ec10">cd /opt/client</strong></p>
|
|
</li><li id="mrs_01_0501__li533801889352">Run the following command to configure environment variables:<p class="litext" id="mrs_01_0501__p106596489352"><a name="mrs_01_0501__li533801889352"></a><a name="li533801889352"></a><strong id="mrs_01_0501__a07d0d8687a6d4a55bc7ddd4c880258a4">source bigdata_env</strong></p>
|
|
</li><li id="mrs_01_0501__li533641019352">If Kerberos authentication is enabled for the current cluster, run the following command to authenticate the current user. If Kerberos authentication is disabled for the current cluster, skip this step.<p class="litext" id="mrs_01_0501__p105148679352"><a name="mrs_01_0501__li533641019352"></a><a name="li533641019352"></a><strong id="mrs_01_0501__b1110519393552">kinit hbase</strong></p>
|
|
<div class="note" id="mrs_01_0501__note26966143154529"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_0501__p41368695154529">The system prompts you to enter the password after you run <strong id="mrs_01_0501__b2076272212819">kinit</strong> <strong id="mrs_01_0501__b6762222122810">hbase</strong>. The default password of user <strong id="mrs_01_0501__b176262218285">hbase</strong> is <strong id="mrs_01_0501__b11762162222813">Hbase@123</strong>.</p>
|
|
</div></div>
|
|
</li><li id="mrs_01_0501__li12253839352">Run the following HBase client command:<p id="mrs_01_0501__p110284519352"><a name="mrs_01_0501__li12253839352"></a><a name="li12253839352"></a><strong id="mrs_01_0501__b321471989352">hbase shell</strong></p>
|
|
</li></ol>
|
|
</p></li></ol><ol start="13" id="mrs_01_0501__ol898298710491"><li id="mrs_01_0501__li1256410216595"><span>Check whether historical data exists in the standby cluster. If historical data exists and data in the active and standby clusters must be consistent, delete data from the standby cluster first.</span><p><ol type="a" id="mrs_01_0501__ol24504688174528"><li id="mrs_01_0501__li20650279174444">On the HBase shell of the standby cluster, run the <strong id="mrs_01_0501__b183621811013">list</strong> command to view the existing tables in the standby cluster.</li><li id="mrs_01_0501__li29698483175147">Delete data tables from the standby cluster based on the output list.<p id="mrs_01_0501__p8395123015306"><a name="mrs_01_0501__li29698483175147"></a><a name="li29698483175147"></a><strong id="mrs_01_0501__b830015249304">disable</strong> '<i><span class="varname" id="mrs_01_0501__varname15300122443014">tableName</span></i>'</p>
|
|
<p id="mrs_01_0501__p937512843016"><strong id="mrs_01_0501__b9930101911309">drop</strong> '<i><span class="varname" id="mrs_01_0501__varname79301192302">tableName</span></i>'</p>
|
|
</li></ol>
|
|
</p></li><li id="mrs_01_0501__li66624452174837"><span>After HBase replication is configured and data synchronization is enabled, check whether tables and data exist in the active cluster and whether the historical data needs to be synchronized to the standby cluster.</span><p><p id="mrs_01_0501__p5653335610491">Run the <strong id="mrs_01_0501__b1758125865611">list</strong> command to check the existing tables in the active cluster and run the <strong id="mrs_01_0501__b35865835620">scan</strong> '<i><span class="varname" id="mrs_01_0501__varname1059175845619">tableName</span></i><strong id="mrs_01_0501__b1859125815561">'</strong> command to check whether the tables contain historical data.</p>
|
|
<ul id="mrs_01_0501__ul797440510491"><li id="mrs_01_0501__li466078310491">If tables exist and data needs to be synchronized, go to <a href="#mrs_01_0501__li4226821210491">15</a>.</li><li id="mrs_01_0501__li4197917010491">If no, no further action is required.</li></ul>
|
|
</p></li><li id="mrs_01_0501__li4226821210491"><a name="mrs_01_0501__li4226821210491"></a><a name="li4226821210491"></a><span>The HBase replication configuration does not support automatic synchronization of historical data in tables. You need to back up the historical data of the active cluster and then manually synchronize the historical data to the standby cluster.</span><p><p id="mrs_01_0501__p4486958910491">Manual synchronization refers to the synchronization of a single table that is implemented by Export, distcp, and Import.</p>
|
|
<p id="mrs_01_0501__p117311810491">The process for manually synchronizing data of a single table is as follows:</p>
|
|
<ol type="a" id="mrs_01_0501__ol1055806410491"><li id="mrs_01_0501__li2791371610491">Export table data from the active cluster.<p id="mrs_01_0501__p4989686010491"><a name="mrs_01_0501__li2791371610491"></a><a name="li2791371610491"></a><strong id="mrs_01_0501__b1927641910616">hbase org.apache.hadoop.hbase.mapreduce.Export -Dhbase.mapreduce.include.deleted.rows=true</strong> <em id="mrs_01_0501__i3821143315619">Table name</em> <em id="mrs_01_0501__i103572411268">Directory where the source data is stored</em></p>
|
|
<p id="mrs_01_0501__p180698310491">Example: <strong id="mrs_01_0501__b1626285310491">hbase org.apache.hadoop.hbase.mapreduce.Export -Dhbase.mapreduce.include.deleted.rows=true t1 /user/hbase/t1</strong></p>
|
|
</li><li id="mrs_01_0501__li1214795510491">Copy the data that has been exported to the standby cluster.<p id="mrs_01_0501__p4222273910491"><a name="mrs_01_0501__li1214795510491"></a><a name="li1214795510491"></a><strong id="mrs_01_0501__b511971519714">hadoop distcp</strong> <em id="mrs_01_0501__i165741025779">Directory for storing source data in the active cluster</em> <strong id="mrs_01_0501__b1930891919817">hdfs://</strong><em id="mrs_01_0501__i16568192320810">ActiveNameNodeIP</em>:<strong id="mrs_01_0501__b159528261081">9820/</strong> <em id="mrs_01_0501__i1126984119719">Directory for storing source data in the standby cluster</em></p>
|
|
<p id="mrs_01_0501__p6510999110491"><strong id="mrs_01_0501__b10615343778">ActiveNameNodeIP</strong> indicates the IP address of the active NameNode in the standby cluster.</p>
|
|
<p id="mrs_01_0501__p4911901110491">Example: <strong id="mrs_01_0501__b3941791810491">hadoop distcp /user/hbase/t1 hdfs://192.168.40.2:9820/user/hbase/t1</strong></p>
|
|
<div class="note" id="mrs_01_0501__note14067447129"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_0501__p184064449121">In MRS 1.6.2 and earlier versions, the default port number is 25000. For details, see <a href="https://docs.otc.t-systems.com/usermanual/mrs/mrs_01_0504.html" target="_blank" rel="noopener noreferrer">List of Open Source Component Ports</a>.</p>
|
|
</div></div>
|
|
</li><li id="mrs_01_0501__li1921694510491">Import data to the standby cluster as the HBase table user of the standby cluster.<p id="mrs_01_0501__p3873478110491"><a name="mrs_01_0501__li1921694510491"></a><a name="li1921694510491"></a><strong id="mrs_01_0501__b9280184811111">hbase org.apache.hadoop.hbase.mapreduce.Import</strong> <em id="mrs_01_0501__i711615384123">-Dimport.bulk.output=Directory where the output data is stored in the standby cluster Table name Directory where the source data is stored in the standby cluster</em></p>
|
|
<p id="mrs_01_0501__p5193305110491"><strong id="mrs_01_0501__b1183703811311">hbase org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles</strong> <em id="mrs_01_0501__i1857715503137">Directory where the output data is stored in the standby cluster Table name</em></p>
|
|
<p id="mrs_01_0501__p979513510491">For example, <strong id="mrs_01_0501__b2104735110491">hbase org.apache.hadoop.hbase.mapreduce.Import -Dimport.bulk.output=/user/hbase/output_t1 t1 /user/hbase/t1</strong> and</p>
|
|
<p id="mrs_01_0501__p5520843210491"><strong id="mrs_01_0501__b2711384310491">hbase org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles /user/hbase/output_t1 t1</strong></p>
|
|
</li></ol>
|
|
</p></li></ol>
|
|
<p id="mrs_01_0501__p30856119154955"><strong id="mrs_01_0501__b1390285715526">Add the replication relationship between the active and standby clusters.</strong></p>
|
|
<ol start="16" id="mrs_01_0501__ol16317690154955"><li id="mrs_01_0501__li46664485154955"><a name="mrs_01_0501__li46664485154955"></a><a name="li46664485154955"></a><span>Run the following command on the HBase Shell to create the replication synchronization relationship between the active cluster and the standby cluster:</span><p><p id="mrs_01_0501__p17327185154955"><strong id="mrs_01_0501__b21726945154955">add_peer</strong> '<em id="mrs_01_0501__i17924162141311">Standby cluster ID',</em> <em id="mrs_01_0501__i117717125320">CLUSTER_KEY =></em> '<em id="mrs_01_0501__i87722125319">ZooKeeper address of the standby cluster</em>',<strong id="mrs_01_0501__b15052117154955">{HDFS_CONFS => true}</strong></p>
|
|
<ul id="mrs_01_0501__ul1251332154955"><li id="mrs_01_0501__li11261994154955"><em id="mrs_01_0501__i23852026138">Standby cluster ID</em> indicates an ID for the active cluster to recognize the standby cluster. It is recommended that the ID contain letters and digits.</li><li id="mrs_01_0501__li34249088154955">The ZooKeeper address of the standby cluster includes the service IP address of ZooKeeper, the port for listening to client connections, and the HBase root directory of the standby cluster on ZooKeeper.</li><li id="mrs_01_0501__li39806344154955"><strong id="mrs_01_0501__b149992351221">{HDFS_CONFS => true}</strong> indicates that the default HDFS configuration of the active cluster will be synchronized to the standby cluster. This parameter is used for HBase of the standby cluster to access HDFS of the active cluster. If bulkload replication is disabled, you do not need to use this parameter.<p id="mrs_01_0501__p6451131181115">Suppose the standby cluster ID is replication2 and the ZooKeeper address of the standby cluster is <strong id="mrs_01_0501__b198971625133420">192.168.40.2,192.168.40.3,192.168.40.4:2181:/hbase</strong>.</p>
|
|
<ul id="mrs_01_0501__ul10816121716116"><li id="mrs_01_0501__li1238861419117">For versions later than MRS 1.9.2: Run the <strong id="mrs_01_0501__b331541519472">add_peer</strong> <strong id="mrs_01_0501__b1031591519471">'replication2',<strong id="mrs_01_0501__b1315141518471">CLUSTER_KEY =></strong></strong> <strong id="mrs_01_0501__b1316615144714">'192.168.40.2,192.168.40.3,192.168.40.4:2181:/hbase'</strong><strong id="mrs_01_0501__b53166156474">,CONFIG => { "hbase.regionserver.kerberos.principal" => "<val>", "hbase.master.kerberos.principal" => "<val2>" }</strong> command for a security cluster and the <strong id="mrs_01_0501__b13161015194713">add_peer</strong> <strong id="mrs_01_0501__b143161152475">'replication2',<strong id="mrs_01_0501__b1931691518471">CLUSTER_KEY =></strong></strong> <strong id="mrs_01_0501__b031715156474">'192.168.40.2,192.168.40.3,192.168.40.4:2181:/hbase'</strong> command for a common cluster.<div class="p" id="mrs_01_0501__p1424722410294">The <strong id="mrs_01_0501__b84235270695035">hbase.master.kerberos.principal</strong> and <strong id="mrs_01_0501__b84235270695050">hbase.regionserver.kerberos.principal</strong> parameters are the Kerberos users of HBase in the security cluster. You can search the <strong id="mrs_01_0501__b84235270695148">hbase-site.xml</strong> file on the client for the parameter values. For example, if the client is installed in the <span class="filepath" id="mrs_01_0501__filepath695613621315"><b>/opt/client</b></span> directory of the Master node, you can run the <strong id="mrs_01_0501__b693484521317">grep "kerberos.principal" /opt/client/HBase/hbase/conf/hbase-site.xml -A1</strong> command to obtain the principal of HBase. See the following figure.<div class="fignone" id="mrs_01_0501__fig8470448204618"><span class="figcap"><b>Figure 1 </b>Obtaining the principal of HBase</span><br><span><img id="mrs_01_0501__image104702048154610" src="en-us_image_0000001295770664.png"></span></div>
|
|
</div>
|
|
</li><li id="mrs_01_0501__li1147812371211">For MRS 1.9.2 or earlier: Run the <strong id="mrs_01_0501__b11151850594">add_peer</strong> <strong id="mrs_01_0501__b41513503910">'replication2',<strong id="mrs_01_0501__b115165015915">CLUSTER_KEY =></strong></strong> <strong id="mrs_01_0501__b14151950295">'192.168.40.2,192.168.40.3,192.168.40.4:2181:/hbase'</strong> command.</li></ul>
|
|
<div class="note" id="mrs_01_0501__note19098158182631"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ol type="a" id="mrs_01_0501__ol41918209182742"><li id="mrs_01_0501__li121187281536">Obtain the ZooKeeper service IP address.<p id="mrs_01_0501__p1642717328320"><a name="mrs_01_0501__li121187281536"></a><a name="li121187281536"></a>For versions earlier than MRS 1.9.2: Choose <strong id="mrs_01_0501__b238211473582">Services</strong> > <strong id="mrs_01_0501__b169131549165810">ZooKeeper</strong> > <strong id="mrs_01_0501__b515813534582">Instance</strong> to obtain the service IP address of ZooKeeper.</p>
|
|
<p id="mrs_01_0501__p1082216212210">For MRS 1.9.2 or later: Log in to the MRS console, click the cluster name, and choose <strong id="mrs_01_0501__b1660412811711">Components</strong> > <strong id="mrs_01_0501__b146048281477">ZooKeeper</strong> > <strong id="mrs_01_0501__b106057282716">Instances</strong> to obtain the ZooKeeper service IP address.</p>
|
|
</li><li id="mrs_01_0501__li4660924183020">On the ZooKeeper service parameter configuration page, search for clientPort, which is the port for the client to connect to the server.</li><li id="mrs_01_0501__li42529842104450">Run the <strong id="mrs_01_0501__b1370321622418">list_peers</strong> command to check whether the replication relationship between the active and standby clusters is added. If the following information is displayed, the relationship is successfully added.<pre class="screen" id="mrs_01_0501__screen1790615447351">hbase(main):003:0> list_peers
|
|
PEER_ID CLUSTER_KEY ENDPOINT_CLASSNAME STATE REPLICATE_ALL NAMESPACES TABLE_CFS BANDWIDTH SERIAL
|
|
replication2 192.168.0.13,192.168.0.177,192.168.0.25:2181:/hbase ENABLED true 0 false</pre>
|
|
<div class="p" id="mrs_01_0501__p355612093515">For versions earlier than MRS 1.9.2: If the following information is displayed after you run the <strong id="mrs_01_0501__b16388193218313">list_peers</strong> command, the operation is successful.<pre class="screen" id="mrs_01_0501__screen145561920143511">hbase(main):003:0> list_peers
|
|
PEER_ID CLUSTER_KEY STATE TABLE_CFS
|
|
replication2 192.168.0.13,192.168.0.177,192.168.0.25:2181:/hbase ENABLED</pre>
|
|
</div>
|
|
</li></ol>
|
|
</div></div>
|
|
</li></ul>
|
|
</p></li></ol>
|
|
<p id="mrs_01_0501__p48834300154955"><strong id="mrs_01_0501__b1728317588248">Specify the data writing status for the active and standby clusters.</strong></p>
|
|
<ol start="17" id="mrs_01_0501__ol63264227154955"><li id="mrs_01_0501__li32507134154955"><span>On the HBase shell of the active cluster, run the following command to retain the data writing status:</span><p><p id="mrs_01_0501__p24128754154955"><strong id="mrs_01_0501__b15832201154955">set_clusterState_active</strong></p>
|
|
<p id="mrs_01_0501__p8272082154955">The command is run successfully if the following information is displayed:</p>
|
|
<pre class="screen" id="mrs_01_0501__screen7339880154955">hbase(main):001:0> set_clusterState_active
|
|
=> true</pre>
|
|
</p></li><li id="mrs_01_0501__li66058923154955"><span>On the HBase shell of the standby cluster, run the following command to retain the data read-only status:</span><p><p id="mrs_01_0501__p57659396154955"><strong id="mrs_01_0501__b49172516154955">set_clusterState_standby</strong></p>
|
|
<p id="mrs_01_0501__p39899460154955">The command is run successfully if the following information is displayed:</p>
|
|
<pre class="screen" id="mrs_01_0501__screen23550821154955">hbase(main):001:0> set_clusterState_standby
|
|
=> true</pre>
|
|
</p></li></ol>
|
|
<p id="mrs_01_0501__p10630800154955"><strong id="mrs_01_0501__b41891844265">Enable the HBase replication function to synchronize data.</strong></p>
|
|
<ol start="19" id="mrs_01_0501__ol55788497154955"><li id="mrs_01_0501__li32334433154955"><span>Check whether a namespace exists in the HBase service instance of the standby cluster and the namespace has the same name as the namespace of the HBase table for which the replication function is to be enabled.</span><p><div class="p" id="mrs_01_0501__p20459284183327">On the HBase shell of the standby cluster, run the <strong id="mrs_01_0501__b739912301346">list_namespace</strong> command to query the namespace.<ul id="mrs_01_0501__ul22574447154955"><li id="mrs_01_0501__li1843436154955">If the same namespace exists, go to <a href="#mrs_01_0501__li15192291154955">20</a>.</li><li id="mrs_01_0501__li1860181994448">If the same namespace does not exist, on the HBase shell of the standby cluster, run the following command to create a namespace with the same name and go to <a href="#mrs_01_0501__li15192291154955">20</a>:<p id="mrs_01_0501__p437217494448"><strong id="mrs_01_0501__b927710487366">create_namespace'ns1</strong></p>
|
|
</li></ul>
|
|
</div>
|
|
</p></li><li id="mrs_01_0501__li15192291154955"><a name="mrs_01_0501__li15192291154955"></a><a name="li15192291154955"></a><span>On the HBase shell of the active cluster, run the following command to enable real-time replication for tables in the active cluster. This ensures that modified data in the active cluster can be synchronized to the standby cluster in real time.</span><p><p id="mrs_01_0501__p2512899154955">You can only synchronize data of one HTable at one time.</p>
|
|
<p id="mrs_01_0501__p22616096154955"><strong id="mrs_01_0501__b1846512533917">enable_table_replication '</strong><em id="mrs_01_0501__i87901161374">Table name</em>'</p>
|
|
<div class="note" id="mrs_01_0501__note6509922154955"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="mrs_01_0501__ul58589304154955"><li id="mrs_01_0501__li57541688154955">If the standby cluster does not contain a table with the same name as the table for which real-time synchronization is to be enabled, the table is automatically created.</li><li id="mrs_01_0501__li48113149154955">If a table with the same name as the table for which real-time synchronization is to be enabled exists in the standby cluster, the structures of the two tables must be the same.</li><li id="mrs_01_0501__li30365158154955">If the encryption algorithm SMS4 or AES is configured for '<em id="mrs_01_0501__i68901155192917">Table name</em>', the function for synchronizing data from the active cluster to the standby cluster cannot be enabled for the HBase table.</li><li id="mrs_01_0501__li4850974154955">If the standby cluster is offline or has tables with the same name but different structures, the replication function cannot be enabled.<p id="mrs_01_0501__p6612548394859"><a name="mrs_01_0501__li4850974154955"></a><a name="li4850974154955"></a>If the standby cluster is offline, start it.</p>
|
|
<p id="mrs_01_0501__p3149199491">If the standby cluster has a table with the same name but different structure, modify the table structure to make it as the same as the table structure of the active cluster. On the HBase shell of the standby cluster, run the <i><b><span class="cmdname" style="font-family:Arial" id="mrs_01_0501__cmdname97641001271">alter</span></b></i> command to change the password by referring to the example.</p>
|
|
</li></ul>
|
|
</div></div>
|
|
</p></li><li id="mrs_01_0501__li3638114154955"><a name="mrs_01_0501__li3638114154955"></a><a name="li3638114154955"></a><span>On the HBase shell of the active cluster, run the following command to enable the real-time replication function for the active cluster to synchronize the HBase permission table:</span><p><div class="p" id="mrs_01_0501__p32743030154955"><strong id="mrs_01_0501__b26251818154955">enable_table_replication 'hbase:acl'</strong><div class="note" id="mrs_01_0501__note31022574154955"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_0501__p10767711154955">After the permission of the active HBase source data table is modified, to ensure that the standby cluster can properly read data, modify the role permission for the standby cluster.</p>
|
|
</div></div>
|
|
</div>
|
|
</p></li></ol>
|
|
<p id="mrs_01_0501__p29096400104944"><strong id="mrs_01_0501__b13130142163419">Check the data synchronization status for the active and standby clusters.</strong></p>
|
|
<ol start="22" id="mrs_01_0501__ol36040501154955"><li id="mrs_01_0501__li55929056154955"><span>Run the following command on the HBase client to check the synchronized data of the active and standby clusters. After the replication function is enabled, you can run this command to check whether the newly synchronized data is consistent.</span><p><p id="mrs_01_0501__p33599458154955"><strong id="mrs_01_0501__b889103394111">hbase org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication --starttime</strong><em id="mrs_01_0501__i11892233154116">=Start time</em> <strong id="mrs_01_0501__b1689353384117">--endtime</strong><em id="mrs_01_0501__i13893333184110">=End time</em> <em id="mrs_01_0501__i8894433104114">Column family name ID of the standby cluster Table name</em></p>
|
|
<div class="note" id="mrs_01_0501__note28620386154955"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="mrs_01_0501__ul56256884154955"><li id="mrs_01_0501__li36549914154955">The start time must be earlier than the end time.</li><li id="mrs_01_0501__li60513771154955">The value of <strong id="mrs_01_0501__b441574819420">starttime</strong> and <strong id="mrs_01_0501__b151218511543">endtime</strong> must be in the timestamp format. You need to run <strong id="mrs_01_0501__b4339185614410">date -d "2015-09-30 00:00:00" +%s to</strong> change a common time format to a timestamp format. The command output is a 10-digit number (accurate to second), but HBase identifies a 13-digit number (accurate to millisecond). Therefore, you need to add three zeros (000) to the end of the command output.</li></ul>
|
|
</div></div>
|
|
<p id="mrs_01_0501__p14932188155939"><strong id="mrs_01_0501__b1128454163615">Switch over active and standby clusters.</strong></p>
|
|
<div class="note" id="mrs_01_0501__note4704913416138"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ol type="a" id="mrs_01_0501__ol22926057161135"><li id="mrs_01_0501__li37390325161135">If the standby cluster needs to be switched over to the active cluster, reconfigure the active/standby relationship by referring to <a href="#mrs_01_0501__li155891430132615">1</a> to <a href="#mrs_01_0501__li11385192216347">11</a> and <a href="#mrs_01_0501__li46664485154955">16</a> to <a href="#mrs_01_0501__li3638114154955">21</a>.</li><li id="mrs_01_0501__li6007965161139">Do not perform <a href="#mrs_01_0501__li12641483154955">12</a> to <a href="#mrs_01_0501__li4226821210491">15</a>.</li></ol>
|
|
</div></div>
|
|
</p></li></ol>
|
|
<div class="section" id="mrs_01_0501__section7753034154955"><h4 class="sectiontitle">Related Commands</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_0501__table24016027154955" frame="border" border="1" rules="all"><caption><b>Table 2 </b>HBase replication</caption><thead align="left"><tr id="mrs_01_0501__row21421163154955"><th align="left" class="cellrowborder" valign="top" width="15.661566156615661%" id="mcps1.3.23.2.2.4.1.1"><p id="mrs_01_0501__p57392613154955">Operation</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="41.7941794179418%" id="mcps1.3.23.2.2.4.1.2"><p id="mrs_01_0501__p18290059154955">Command</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="42.544254425442546%" id="mcps1.3.23.2.2.4.1.3"><p id="mrs_01_0501__p5099847154955">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_0501__row10434430154955"><td class="cellrowborder" valign="top" width="15.661566156615661%" headers="mcps1.3.23.2.2.4.1.1 "><p id="mrs_01_0501__p39882522154955">Set up the active/standby relationship.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="41.7941794179418%" headers="mcps1.3.23.2.2.4.1.2 "><p id="mrs_01_0501__p9994133411135"><strong id="mrs_01_0501__b1299417343139">add_peer</strong> <em id="mrs_01_0501__i4996123451316">'Standby cluster ID', 'Standby cluster address'</em></p>
|
|
<p id="mrs_01_0501__p1799914347139">Examples:</p>
|
|
<p id="mrs_01_0501__p1711335101319"><strong id="mrs_01_0501__b111133501310">add_peer '1', 'zk1,zk2,zk3:2181:/hbase'</strong></p>
|
|
<p id="mrs_01_0501__p1561135151311"><strong id="mrs_01_0501__b9843541319">add_peer '1', 'zk1,zk2,zk3:2181:/hbase1'</strong></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="42.544254425442546%" headers="mcps1.3.23.2.2.4.1.3 "><p id="mrs_01_0501__p32979070154955">Set up the relationship between the active cluster and the standby cluster. To enable bulkload replication, run the <strong id="mrs_01_0501__b16928959133912">add_peer </strong><em id="mrs_01_0501__i123731121853">'Standby cluster ID'</em><strong id="mrs_01_0501__b292945914398">,<em id="mrs_01_0501__i1792918596390">CLUSTER_KEY =></em> </strong><em id="mrs_01_0501__i14570141925317">'Standby cluster address'</em> command, configure <strong id="mrs_01_0501__b108905491719">hbase.replication.conf.dir</strong>, and manually copy the HBase client configuration file in the active cluster to all RegionServer nodes in the standby cluster. For details, see <a href="#mrs_01_0501__li65160752154955">5</a> to <a href="#mrs_01_0501__li11385192216347">11</a>.</p>
|
|
<p id="mrs_01_0501__p1640102532413">For MRS 1.9.2 or earlier, to enable bulkload replication, run the following command: <strong id="mrs_01_0501__b17357125272214">add_peer</strong> <em id="mrs_01_0501__i1263114475228">'Standby cluster ID',<em id="mrs_01_0501__i1863274742216">'Standby cluster address'</em></em>,<strong id="mrs_01_0501__b1490918110233">{HDFS_CONF => true}</strong>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_0501__row16700746154955"><td class="cellrowborder" valign="top" width="15.661566156615661%" headers="mcps1.3.23.2.2.4.1.1 "><p id="mrs_01_0501__p10583211154955">Remove the active/standby relationship.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="41.7941794179418%" headers="mcps1.3.23.2.2.4.1.2 "><p id="mrs_01_0501__p51933801154955"><strong id="mrs_01_0501__b64751030154955">remove_peer</strong> <em id="mrs_01_0501__i45888363154955">'Standby cluster ID'</em></p>
|
|
<p id="mrs_01_0501__p10342090154955">Example:</p>
|
|
<p id="mrs_01_0501__p25969946154955"><strong id="mrs_01_0501__b32402925154955">remove_peer '1'</strong></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="42.544254425442546%" headers="mcps1.3.23.2.2.4.1.3 "><p id="mrs_01_0501__p7391306154955">Remove standby cluster information from the active cluster.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_0501__row66521757154955"><td class="cellrowborder" valign="top" width="15.661566156615661%" headers="mcps1.3.23.2.2.4.1.1 "><p id="mrs_01_0501__p19553217154955">Query the active/standby relationship.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="41.7941794179418%" headers="mcps1.3.23.2.2.4.1.2 "><p id="mrs_01_0501__p40306738154955"><strong id="mrs_01_0501__b27216322154955">list_peers</strong></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="42.544254425442546%" headers="mcps1.3.23.2.2.4.1.3 "><p id="mrs_01_0501__p57038480154955">Query standby cluster information (mainly Zookeeper information) in the active cluster.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_0501__row43584280154955"><td class="cellrowborder" valign="top" width="15.661566156615661%" headers="mcps1.3.23.2.2.4.1.1 "><p id="mrs_01_0501__p40665823154955">Enable the real-time user table synchronization function.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="41.7941794179418%" headers="mcps1.3.23.2.2.4.1.2 "><p id="mrs_01_0501__p5597329154955"><strong id="mrs_01_0501__b50375967154955">enable_table_replication</strong> <em id="mrs_01_0501__i50730520154955">'Table name'</em></p>
|
|
<p id="mrs_01_0501__p53921503154955">Example:</p>
|
|
<p id="mrs_01_0501__p15531484154955"><strong id="mrs_01_0501__b5565632154955">enable_table_replication 't1'</strong></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="42.544254425442546%" headers="mcps1.3.23.2.2.4.1.3 "><p id="mrs_01_0501__p48163049154955">Synchronize user tables from the active cluster to the standby cluster.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_0501__row30814265154955"><td class="cellrowborder" valign="top" width="15.661566156615661%" headers="mcps1.3.23.2.2.4.1.1 "><p id="mrs_01_0501__p12927522154955">Disable the real-time user table synchronization function.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="41.7941794179418%" headers="mcps1.3.23.2.2.4.1.2 "><p id="mrs_01_0501__p40496360154955"><strong id="mrs_01_0501__b28922921154955">disable_table_replication</strong> <em id="mrs_01_0501__i58979698154955">'Table name'</em></p>
|
|
<p id="mrs_01_0501__p61055236154955">Example:</p>
|
|
<p id="mrs_01_0501__p12626216154955"><strong id="mrs_01_0501__b46527088154955">disable_table_replication 't1'</strong></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="42.544254425442546%" headers="mcps1.3.23.2.2.4.1.3 "><p id="mrs_01_0501__p10597780154955">Do not synchronize user tables from the active cluster to the standby cluster.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_0501__row28271163154955"><td class="cellrowborder" valign="top" width="15.661566156615661%" headers="mcps1.3.23.2.2.4.1.1 "><p id="mrs_01_0501__p8262887154955">Verify data of the active and standby clusters.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="41.7941794179418%" headers="mcps1.3.23.2.2.4.1.2 "><p id="mrs_01_0501__p50956389154955"><strong id="mrs_01_0501__b158151120244">bin/hbase org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication</strong> <em id="mrs_01_0501__i170918150242">--starttime --endtime Column family name Standby cluster ID Table name</em></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="42.544254425442546%" headers="mcps1.3.23.2.2.4.1.3 "><p id="mrs_01_0501__p17917034154955">Verify whether data of the specified table is the same between the active cluster and the standby cluster.</p>
|
|
<p id="mrs_01_0501__p27035579154955">The description of the parameters in this command is as follows:</p>
|
|
<ul id="mrs_01_0501__ul41993619154955"><li id="mrs_01_0501__li42398251154955">Start time: If start time is not specified, the default value <strong id="mrs_01_0501__b207814382433">0</strong> will be used.</li><li id="mrs_01_0501__li46039940154955">End time: If end time is not specified, the time when the current operation is submitted will be used by default.</li><li id="mrs_01_0501__li11706279154955">Table name: If a table name is not entered, all user tables for which the real-time synchronization function is enabled will be verified by default.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_0501__row38247654154955"><td class="cellrowborder" valign="top" width="15.661566156615661%" headers="mcps1.3.23.2.2.4.1.1 "><p id="mrs_01_0501__p11052256154955">Switch the data writing status. </p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="41.7941794179418%" headers="mcps1.3.23.2.2.4.1.2 "><p id="mrs_01_0501__p22817563154955"><strong id="mrs_01_0501__b4031480154955">set_clusterState_active</strong></p>
|
|
<p id="mrs_01_0501__p36283327154955"><strong id="mrs_01_0501__b58114487154955">set_clusterState_standby</strong></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="42.544254425442546%" headers="mcps1.3.23.2.2.4.1.3 "><p id="mrs_01_0501__p9653002154955">Specifies whether data can be written to the cluster HBase tables.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_0501__row19768159154955"><td class="cellrowborder" valign="top" width="15.661566156615661%" headers="mcps1.3.23.2.2.4.1.1 "><p id="mrs_01_0501__p57717036154955">Add or update the active cluster HDFS configurations saved in the peer cluster.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="41.7941794179418%" headers="mcps1.3.23.2.2.4.1.2 "><p id="mrs_01_0501__p44568321154955"><strong id="mrs_01_0501__b65570575154955">set_replication_hdfs_confs 'PeerId', {'key1' => 'value1', 'key2' => 'value2'}</strong></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="42.544254425442546%" headers="mcps1.3.23.2.2.4.1.3 "><p id="mrs_01_0501__p9616324154955">Enable replication for data including bulkload data. When HDFS parameters are modified in the active cluster, the modification cannot be automatically synchronized to the standby cluster. You need to manually run the command to synchronize the changes. The affected parameters are as follows:</p>
|
|
<ul id="mrs_01_0501__ul19438053154955"><li id="mrs_01_0501__li40724757154955">fs.defaultFS</li><li id="mrs_01_0501__li30978500154955">dfs.client.failover.proxy.provider.hacluster</li><li id="mrs_01_0501__li10371052154955">dfs.client.failover.connection.retries.on.timeouts</li><li id="mrs_01_0501__li26230604154955">dfs.client.failover.connection.retries</li></ul>
|
|
<p id="mrs_01_0501__p34748846154955">For example, if the value of <strong id="mrs_01_0501__b201551428172717">fs.defaultFS</strong> is changed to <strong id="mrs_01_0501__b6867163472716">hdfs://hacluster_sale</strong>, run the <strong id="mrs_01_0501__b44304164154955">set_replication_hdfs_confs '1', {'fs.defaultFS' => 'hdfs://hacluster_sale'}</strong> command to synchronization the HDFS configuration to the standby cluster whose ID is 1.</p>
|
|
<p id="mrs_01_0501__p2172155419011">In versions later than MRS 1.9.2, this command has been removed. If synchronization is required, manually copy the changed client configurations in the active cluster to the standby cluster. For details, see <a href="#mrs_01_0501__li3244131341713">8</a>.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_0500.html">Using HBase</a></div>
|
|
</div>
|
|
</div>
|
|
|