forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
35 lines
7.5 KiB
HTML
35 lines
7.5 KiB
HTML
<a name="mrs_01_0502"></a><a name="mrs_01_0502"></a>
|
|
|
|
<h1 class="topictitle1">Enabling Cross-Cluster Copy</h1>
|
|
<div id="body1589421624075"><div class="section" id="mrs_01_0502__section4982615016933"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_0502__p938634316933">DistCp is used to copy the data stored on HDFS from a cluster to another cluster. DistCp depends on the cross-cluster copy function, which is disabled by default. This function needs to be enabled in both clusters.</p>
|
|
<p id="mrs_01_0502__p6464882916933">This section describes how to enable cross-cluster copy.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_0502__section4496855716933"><h4 class="sectiontitle">Impact on the System</h4><p id="mrs_01_0502__p1857453216933">Yarn needs to be restarted to enable the cross-cluster copy function and cannot be accessed during the restart.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_0502__section3295306016933"><h4 class="sectiontitle">Prerequisites</h4><p id="mrs_01_0502__p5195218816933">The <strong id="mrs_01_0502__b1521625426">hadoop.rpc.protection</strong> parameter of the two HDFS clusters must be set to the same data transmission mode, which can be <strong id="mrs_01_0502__b1677512291125">privacy</strong> (encryption enabled) or <strong id="mrs_01_0502__b0623153411211">authentication</strong> (encryption disabled).</p>
|
|
<div class="note" id="mrs_01_0502__note36461715185452"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_0502__p191517115388">Go to the <strong id="mrs_01_0502__b549132718174">All Configurations</strong> page by referring to <a href="mrs_01_2125.html">Modifying Cluster Service Configuration Parameters</a> and search for <strong id="mrs_01_0502__b134399372176">hadoop.rpc.protection</strong>.</p>
|
|
<p id="mrs_01_0502__p1885710212539">For versions earlier than MRS 3.x, choose <strong id="mrs_01_0502__b10434155193411">Components</strong> > <strong id="mrs_01_0502__b1943511514345">HDFS</strong> > <strong id="mrs_01_0502__b14435205153410">Service Configuration</strong> on the cluster details page. Switch <strong id="mrs_01_0502__b1543516519345">Basic</strong> to <strong id="mrs_01_0502__b143513533419">All</strong>, and search for <strong id="mrs_01_0502__b1643511573414">hadoop.rpc.protection</strong>.</p>
|
|
</div></div>
|
|
</div>
|
|
<div class="section" id="mrs_01_0502__section6491650916933"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_0502__ol2374590516933"><li id="mrs_01_0502__li328938153213"><span>Log in to the service page.</span><p><p id="mrs_01_0502__p72818387329">For versions earlier than MRS 1.9.2: Log in to <a href="https://docs.otc.t-systems.com/usermanual/mrs/mrs_01_0102.html" target="_blank" rel="noopener noreferrer">MRS Manager</a>, and choose <strong id="mrs_01_0502__b9693174431">Services</strong>.</p>
|
|
<p id="mrs_01_0502__p428183883217">For MRS 1.9.2 or later: Click the cluster name on the MRS console and choose <strong id="mrs_01_0502__b142481044311">Components</strong>.</p>
|
|
</p></li><li id="mrs_01_0502__li1966213718714"><span>Go to the <strong id="mrs_01_0502__b2751163393918">All Configurations</strong> page of the Yarn service. For details, see <a href="mrs_01_2125.html">Modifying Cluster Service Configuration Parameters</a>.</span><p><div class="note" id="mrs_01_0502__note3349205551313"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_0502__p158007468574">If the <strong id="mrs_01_0502__b18432028175820">Components</strong> tab is unavailable, complete IAM user synchronization first. (On the <strong id="mrs_01_0502__b19432112835811">Dashboard</strong> page, click <strong id="mrs_01_0502__b1643242815582">Synchronize</strong> on the right side of <span class="parmname" id="mrs_01_0502__parmname194321528135815"><b>IAM User Sync</b></span> to synchronize IAM users.)</p>
|
|
</div></div>
|
|
</p></li><li id="mrs_01_0502__li6378706216933"><span>In the navigation pane, choose <strong id="mrs_01_0502__b1640775534813">Yarn</strong> > <strong id="mrs_01_0502__b1794423334719">Distcp</strong>.</span></li><li id="mrs_01_0502__li3721265416933"><span>Set <strong id="mrs_01_0502__b1569813219450">haclusterX.remotenn1</strong> of <strong id="mrs_01_0502__b068010164452">dfs.namenode.rpc-address</strong> to the service IP address and RPC port number of one NameNode instance of the peer cluster, and set <strong id="mrs_01_0502__b124923454911">haclusterX.remotenn2</strong> to the service IP address and RPC port number of the other NameNode instance of the peer cluster. Enter a value in the <em id="mrs_01_0502__i17809946303">IP address:port</em> format.</span><p><div class="note" id="mrs_01_0502__note60675655181130"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_0502__p690124713013">For MRS 1.9.2 or later, log in to the MRS console, click the cluster name, and choose <strong id="mrs_01_0502__b6816222010">Components</strong> > <strong id="mrs_01_0502__b5973547205">HDFS</strong> > <strong id="mrs_01_0502__b239568172014">Instances</strong> to obtain the service IP address of the NameNode instance.</p>
|
|
<p id="mrs_01_0502__p1831135910351">You can also log in to FusionInsight Manager in MRS 3.x clusters, and choose <strong id="mrs_01_0502__b16571115211">Cluster</strong> > <em id="mrs_01_0502__i964575115238">Name of the desired cluster</em> > <strong id="mrs_01_0502__b1646915518210">Services</strong> > <strong id="mrs_01_0502__b26321353152115">HDFS</strong> > <strong id="mrs_01_0502__b2805951162112">Instance</strong> to obtain the service IP address of the NameNode instance.</p>
|
|
</div></div>
|
|
<p id="mrs_01_0502__p6647843216933"><strong id="mrs_01_0502__b625311530509">dfs.namenode.rpc-address.haclusterX.remotenn1</strong> and <strong id="mrs_01_0502__b158577145115">dfs.namenode.rpc-address.haclusterX.remotenn2</strong> do not distinguish active and standby NameNode instances. The default NameNode RPC port is 9820 and cannot be modified on MRS Manager.</p>
|
|
<p id="mrs_01_0502__p6143498316933">For example, <strong id="mrs_01_0502__b258505053117">10.1.1.1:9820</strong> and <strong id="mrs_01_0502__b16903545315">10.1.1.2:9820</strong>.</p>
|
|
<div class="note" id="mrs_01_0502__note65601137102310"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_0502__p9560103742315">For MRS 1.6.2 or earlier, the default port number is <strong id="mrs_01_0502__b84561446105219">25000</strong>. For details, see <a href="https://docs.otc.t-systems.com/usermanual/mrs/mrs_01_0504.html" target="_blank" rel="noopener noreferrer">List of Open Source Component Ports</a>.</p>
|
|
</div></div>
|
|
</p></li><li id="mrs_01_0502__li2449066216933"><span>Save the configuration. On the <strong id="mrs_01_0502__b169576303174">Dashboard</strong> tab page, and choose <strong id="mrs_01_0502__b169571530131719">More</strong> > <strong id="mrs_01_0502__b16957123012175">Restart Service</strong> to restart the Yarn service.</span><p><p id="mrs_01_0502__p1908936716933"><strong id="mrs_01_0502__b181312576571">Operation succeeded</strong> is displayed. Click <strong id="mrs_01_0502__b177011150586">Finish</strong>. The Yarn service is started successfully.</p>
|
|
</p></li><li id="mrs_01_0502__li3758657716933"><span>Log in to the other cluster and repeat the preceding operations.</span></li></ol>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_0500.html">Using HBase</a></div>
|
|
</div>
|
|
</div>
|
|
|