forked from docs/doc-exports
yangtong
c285e88a17
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: yangtong <yangtong2@huawei.com> Co-committed-by: yangtong <yangtong2@huawei.com>
86 lines
18 KiB
HTML
86 lines
18 KiB
HTML
<a name="admin_guide_000133"></a><a name="admin_guide_000133"></a>
|
|
|
|
<h1 class="topictitle1">Switching the Scheduler</h1>
|
|
<div id="body1529658735915"><div class="section" id="admin_guide_000133__s3d9906c41b9248c890dcc4e7a53811f7"><h4 class="sectiontitle">Scenario</h4><p id="admin_guide_000133__a6dc08aa12cb94c9ab2c31c45578064ef">The newly installed <span id="admin_guide_000133__text05971020143218">MRS</span> cluster uses the Superior scheduler by default. If the cluster is upgraded from an earlier version, you can switch the YARN scheduler from the Capacity scheduler to the Superior scheduler with a few clicks.</p>
|
|
</div>
|
|
<div class="section" id="admin_guide_000133__s5257bb6140f542fe93d84b1b8e667d5e"><h4 class="sectiontitle">Prerequisites</h4><ul id="admin_guide_000133__ul98181317865"><li id="admin_guide_000133__li181711719613">The network connectivity of the cluster is proper and secure, and the YARN service status is normal.</li><li id="admin_guide_000133__li1479241816614">During scheduler switching, tenants cannot be added, deleted, or modified. In addition, services cannot be started or stopped.</li></ul>
|
|
</div>
|
|
<div class="section" id="admin_guide_000133__section5354105215440"><h4 class="sectiontitle">Switching Between the Capacity Scheduler and Superior Scheduler (Available for Clusters of MRS 3.3.0 or Later)</h4><p id="admin_guide_000133__p04281321144419">This function is only available for clusters of MRS 3.3.0 or later.</p>
|
|
<div class="p" id="admin_guide_000133__p19536125163116"><strong id="admin_guide_000133__b1079516093519">Constraints</strong><ul id="admin_guide_000133__ul1343992563117"><li id="admin_guide_000133__li93615540277">This operation is available for only the scenario where a cluster is newly provisioned and the scheduler needs to be switched.</li><li id="admin_guide_000133__li19249173819388">During the scheduler switchover, do not perform any operation on the cluster. Otherwise, the operation may fail due to database modification.</li></ul>
|
|
</div>
|
|
<div class="p" id="admin_guide_000133__p579712189313"><strong id="admin_guide_000133__b98311910143711">Impact on the system</strong><ul id="admin_guide_000133__ul6944162718316"><li id="admin_guide_000133__li12944172711315">Because the ResourceManager is restarted during scheduler switching, submitting jobs to Yarn will fail at that time.</li><li id="admin_guide_000133__li194512276315">After the scheduler is switched, the parameters of the scheduler that takes over the workload are used.</li></ul>
|
|
</div>
|
|
<p id="admin_guide_000133__p14807203119316"><strong id="admin_guide_000133__b1925433793710">Procedure</strong></p>
|
|
<ol id="admin_guide_000133__ol144382057135"><li id="admin_guide_000133__li1243819571336"><span>Log in to MRS Manager. Choose <strong id="admin_guide_000133__b8981742103716">Cluster</strong> > <strong id="admin_guide_000133__b098144283714">Services</strong> > <strong id="admin_guide_000133__b1598114217374">Yarn</strong> and check whether the Yarn service status is normal. If the service is abnormal, restore the service.</span></li><li id="admin_guide_000133__li1689319201845"><span>Log in to the active management node as user <strong id="admin_guide_000133__b20684125019373">omm</strong>.</span></li><li id="admin_guide_000133__li4811192515420"><span>Switch the scheduler.</span><p><ul id="admin_guide_000133__ul10660853125914"><li id="admin_guide_000133__li2661165375915">Run the following command to switch from the Capacity scheduler to the Superior scheduler:<p id="admin_guide_000133__p13169746668"><a name="admin_guide_000133__li2661165375915"></a><a name="li2661165375915"></a><strong id="admin_guide_000133__b171811039103">s</strong><strong id="admin_guide_000133__b997773519018">h ${BIGDATA_HOME}/om-server/om/sbin/cleanSwitchScheduler.sh 1</strong></p>
|
|
<p id="admin_guide_000133__p8572531215">If information similar to the following is displayed, the switch is successful:</p>
|
|
<pre class="screen" id="admin_guide_000133__screen124917202402"><strong id="admin_guide_000133__b748715564018">Will change scheduler type to SUPERIOR</strong>
|
|
Start to delete all tenant resource.
|
|
End to delete all tenant resource.
|
|
Start to delete all resource pool.
|
|
End to delete all resource pool.
|
|
...
|
|
<strong id="admin_guide_000133__b1978713011111">End to switch scheduler by reset.</strong></pre>
|
|
</li><li id="admin_guide_000133__li16611253185910">Run the following command to switch from the Capacity scheduler to the Superior scheduler:<p id="admin_guide_000133__p151111192810"><a name="admin_guide_000133__li16611253185910"></a><a name="li16611253185910"></a><strong id="admin_guide_000133__b102201092218">sh ${BIGDATA_HOME}/om-server/om/sbin/cleanSwitchScheduler.sh 0</strong></p>
|
|
<p id="admin_guide_000133__p9865174620213">If information similar to the following is displayed, the switch is successful:</p>
|
|
<pre class="screen" id="admin_guide_000133__screen6999151793314"><strong id="admin_guide_000133__b55827541111">Will change scheduler type to CAPACITY </strong>
|
|
Start to delete all tenant resource.
|
|
End to delete all tenant resource.
|
|
Start to delete all resource pool.
|
|
End to delete all resource pool.
|
|
...
|
|
<strong id="admin_guide_000133__b17357105715119">End to switch scheduler by reset. </strong></pre>
|
|
</li></ul>
|
|
<div class="note" id="admin_guide_000133__note1262145711405"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="admin_guide_000133__p7622057104017">You can query the scheduler switching logs on the active management node.</p>
|
|
<ul id="admin_guide_000133__ul206223576401"><li id="admin_guide_000133__li10622145710401">${BIGDATA_LOG_HOME}/controller/aos/clean_switch_scheduler.log</li><li id="admin_guide_000133__li156221457194013">${BIGDATA_LOG_HOME}/controller/aos/aos.log</li><li id="admin_guide_000133__li37270423415">${BIGDATA_LOG_HOME}/controller/aos/plugin.log</li></ul>
|
|
</div></div>
|
|
</p></li></ol>
|
|
</div>
|
|
<div class="section" id="admin_guide_000133__section191862915619"><h4 class="sectiontitle">Switching from the Capacity Scheduler to the Superior Scheduler</h4><p id="admin_guide_000133__p1694251184714"><strong id="admin_guide_000133__b49429115475">Impact on the System</strong></p>
|
|
<ul id="admin_guide_000133__ul1942211134711"><li id="admin_guide_000133__li159435116474">Because the ResourceManager is restarted during scheduler switching, submitting jobs to YARN will fail at that time.</li><li id="admin_guide_000133__li14943111184716">During scheduler switching, tasks in a job being executed on YARN will continue, but new tasks cannot be started.</li><li id="admin_guide_000133__li189431811114719">After scheduler switching is complete, jobs executed on YARN may fail, causing service interruptions.</li><li id="admin_guide_000133__li10943161134718">After scheduler switching is complete, parameters of the Superior scheduler are used for tenant management.</li><li id="admin_guide_000133__li39431611204712">After scheduler switching is complete, tenant queues whose capacity is 0 in the Capacity scheduler cannot be allocated resources in the Superior scheduler. As a result, jobs submitted to these tenant queues fail to be executed. Therefore, you are advised not to set the capacity of a tenant queue to 0 in the Capacity scheduler.</li><li id="admin_guide_000133__li59431411144716">After scheduler switching is complete, you cannot add or delete resource pools, YARN node labels, or tenants during the observation period. If such an operation is performed, the scheduler cannot be rolled back to the Capacity scheduler.<div class="note" id="admin_guide_000133__note294341114719"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="admin_guide_000133__p978613288470">The recommended observation period for scheduler switching is one week. If resource pools, YARN node labels, or tenants are added or deleted during this period, the observation period ends immediately.</p>
|
|
</div></div>
|
|
</li><li id="admin_guide_000133__li994331194718">The scheduler rollback may cause the loss of partial or all YARN job information.</li></ul>
|
|
<p id="admin_guide_000133__p17279417477"><strong id="admin_guide_000133__b14611248153817">Procedure</strong></p>
|
|
<ol id="admin_guide_000133__o70f9a39aa18849668523da7fae280d06"><li id="admin_guide_000133__l8cf4320ed5c34bc99301fedccb17c899"><span>Modify YARN service parameters and ensure that the YARN service status is normal.</span><p><ol type="a" id="admin_guide_000133__o03fbad1361724fc8a58fa3d301134c50"><li id="admin_guide_000133__l532be7cc996446a59c67ac1cb6df1bfe">Log in to <span id="admin_guide_000133__text15946118176">MRS</span> Manager as an administrator.</li><li id="admin_guide_000133__li1592163516456">Log in to <span id="admin_guide_000133__text9288815124918">MRS</span> Manager and choose <strong id="admin_guide_000133__b1911218173817">Cluster</strong> > <strong id="admin_guide_000133__b1972392010387">Services</strong> > <strong id="admin_guide_000133__b687019235389">Yarn</strong>. Click <strong id="admin_guide_000133__b175301137103912">Configurations</strong> then <strong id="admin_guide_000133__b1485284703914">All Configurations</strong>, search for <strong id="admin_guide_000133__b1328017553393">yarn.resourcemanager.webapp.pagination.enable</strong>, and check whether the value is <strong id="admin_guide_000133__b1512610511409">true</strong>.<ul id="admin_guide_000133__ul130819475555"><li id="admin_guide_000133__li330811471550">If yes, go to <a href="#admin_guide_000133__l62a2981fbee6466387b51feb634bb77f">1.c</a>.</li><li id="admin_guide_000133__li10309114715557">If no, set the parameter to <strong id="admin_guide_000133__b849022817407">true</strong> and click <strong id="admin_guide_000133__b78004367405">Save</strong> to save the configuration. On the <strong id="admin_guide_000133__b13773949114012">Dashboard</strong> tab page of YARN, choose <strong id="admin_guide_000133__b19440915164118">More</strong> > <strong id="admin_guide_000133__b3469131915419">Restart Service</strong>, verify the identity, and click <strong id="admin_guide_000133__b104251025194114">OK</strong>. After the service is restarted, go to <a href="#admin_guide_000133__l62a2981fbee6466387b51feb634bb77f">1.c</a>.</li></ul>
|
|
</li><li id="admin_guide_000133__l62a2981fbee6466387b51feb634bb77f"><a name="admin_guide_000133__l62a2981fbee6466387b51feb634bb77f"></a><a name="l62a2981fbee6466387b51feb634bb77f"></a>Choose <strong id="admin_guide_000133__b15777131019105">Cluster</strong> > <em id="admin_guide_000133__i59901050184310">Name of the desired cluster</em> > <strong id="admin_guide_000133__b13225193112108">Services</strong>, and check whether the YARN service status is normal.</li></ol>
|
|
</p></li><li id="admin_guide_000133__lcb6a5446114646deb84fc246b5b1c5c4"><span>Log in to the active management node as user <strong id="admin_guide_000133__b800684465112155">omm</strong>.</span></li><li id="admin_guide_000133__l16e6ec556e8e45ddb50be745dcc4d916"><span>Switch the scheduler.</span><p><p id="admin_guide_000133__accaf34bc242848909670c019a45883bf">The following switching modes are available:</p>
|
|
<p id="admin_guide_000133__ae51aae8756c744b6a7dc5410e5675588"><strong id="admin_guide_000133__b165234138135">0</strong>: converts the Capacity scheduler configurations into the Superior scheduler configurations and then switches the Capacity scheduler to the Superior scheduler.</p>
|
|
<p id="admin_guide_000133__a8c043ab915014790bc0b66608b081785"><strong id="admin_guide_000133__b20525450201614">1</strong>: converts the Capacity scheduler configurations into the Superior scheduler configurations only.</p>
|
|
<p id="admin_guide_000133__a247ccb33398b4a0aaf382132d853b9e3"><strong id="admin_guide_000133__b16324196172">2</strong>: switches the Capacity scheduler to the Superior scheduler only.</p>
|
|
<ul id="admin_guide_000133__u0d6dda848c1a4a35ab8610e5ae72eba8"><li id="admin_guide_000133__l913fd66e7fa944549e4cc352dcb2d3bb">Mode <strong id="admin_guide_000133__b1823910333173">0</strong> is recommended if the cluster environment is simple and the number of tenants is less than 20.<p id="admin_guide_000133__ab3b100c2ee9d4d28b27aa71951c00e34">Run the following command:</p>
|
|
<p id="admin_guide_000133__a877a94c8a203469a9c4b1f9c40bc9f81"><strong id="admin_guide_000133__b53901840195612">sh ${BIGDATA_HOME}/om-server/om/sbin/switchScheduler.sh </strong><strong id="admin_guide_000133__b10813289583">-c </strong><em id="admin_guide_000133__i117112815582">Cluster ID</em><strong id="admin_guide_000133__b733991044215"> -m 0</strong></p>
|
|
<div class="note" id="admin_guide_000133__note10298321174210"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="admin_guide_000133__p1730017211425">You can choose <strong id="admin_guide_000133__b58916161810">Cluster</strong>, click the cluster name, and choose <strong id="admin_guide_000133__b181410161186">Cluster Properties</strong> on <span id="admin_guide_000133__text184911917104910">MRS</span> Manager to view the cluster ID.</p>
|
|
</div></div>
|
|
<pre class="screen" id="admin_guide_000133__s5a3bb72e5ad14898b63f9b1cff82d275">Start to convert Capacity scheduler to Superior Scheduler, clusterId=1
|
|
Start to convert Capacity scheduler configurations to Superior. Please wait...
|
|
Convert configurations successfully.
|
|
Start to switch the Yarn scheduler to Superior. Please wait...
|
|
Switch the Yarn scheduler to Superior successfully.</pre>
|
|
</li><li id="admin_guide_000133__l176070bf1f2c429c9569bfc74578e71f">If the cluster environment or tenant information is complex and you need to retain the queue configurations of the Capacity scheduler on the Superior scheduler, it is recommended that you use mode <strong id="admin_guide_000133__b12885463192">1</strong> first to convert the Capacity scheduler configurations, check the converted configurations, and then use mode <strong id="admin_guide_000133__b38391614112018">2</strong> to switch the Capacity scheduler to the Superior scheduler.<ol type="a" id="admin_guide_000133__o760d89751aba4683947787a9da698dda"><li id="admin_guide_000133__lb79f2dfa560449ba860d22dd19878234">Run the following command to convert the Capacity scheduler configurations into the Superior scheduler configurations:<p id="admin_guide_000133__a95fa3acc1bbd49178d911e955c99ea17"><a name="admin_guide_000133__lb79f2dfa560449ba860d22dd19878234"></a><a name="lb79f2dfa560449ba860d22dd19878234"></a><strong id="admin_guide_000133__b21099414152912">sh ${BIGDATA_HOME}/</strong><strong id="admin_guide_000133__b18737113135914">om-server/om/sbin/switchScheduler.sh -c </strong><em id="admin_guide_000133__i9738183115920">Cluster ID</em><strong id="admin_guide_000133__b373703145916"> -m 1</strong></p>
|
|
<pre class="screen" id="admin_guide_000133__s1a363874ed2743978e123d154a1613d4">Start to convert Capacity scheduler to Superior Scheduler, clusterId=1
|
|
Start to convert Capacity scheduler configurations to Superior. Please wait...
|
|
Convert configurations successfully.</pre>
|
|
</li><li id="admin_guide_000133__l4ddfa3c79e5846759e9c35f1b8b41f80">Run the following command to switch the Capacity scheduler to the Superior scheduler:<p id="admin_guide_000133__a78538a50908842dabd67f25652347aaf"><a name="admin_guide_000133__l4ddfa3c79e5846759e9c35f1b8b41f80"></a><a name="l4ddfa3c79e5846759e9c35f1b8b41f80"></a><strong id="admin_guide_000133__b1470272016594">sh ${BIGDATA_HOME}/om-server/om/sbin/switchScheduler.sh -c </strong><em id="admin_guide_000133__i37041120125912">Cluster ID</em><strong id="admin_guide_000133__b2070382045912"> -m 2</strong></p>
|
|
<pre class="screen" id="admin_guide_000133__s44a115d4820e456b9c04aaaea1b7ac98">Start to convert Capacity scheduler to Superior Scheduler, clusterId=1
|
|
Start to switch the Yarn scheduler to Superior. Please wait...
|
|
Switch the Yarn scheduler to Superior successfully.</pre>
|
|
</li></ol>
|
|
</li><li id="admin_guide_000133__lf35352e1821d4891be6e7d5d62be5123">If you do not need to retain the queue configurations of the Capacity scheduler, use mode <strong id="admin_guide_000133__b118671837112214">2</strong>.<ol type="a" id="admin_guide_000133__oaeb0b0bda7624aa79f9a110b87651a3f"><li id="admin_guide_000133__la7785ee622704638b43154dd520a291d">Log in to <span id="admin_guide_000133__text03621819164912">MRS</span> Manager and delete all tenants except the default tenant.</li><li id="admin_guide_000133__l7e705c69f115463f810607be38dba922">On <span id="admin_guide_000133__text1234802115490">MRS</span> Manager, delete all resource pools except the default resource pool.<p id="admin_guide_000133__af5cacb7d28464935b470d3843df3aba3">Run the following command to switch the Capacity scheduler to the Superior scheduler:</p>
|
|
<p id="admin_guide_000133__acde43504ea5146569f47313261493702"><strong id="admin_guide_000133__b9294104614231">sh ${BIGDATA_HOME}/om-server/om/sbin/switchScheduler.sh -c </strong><em id="admin_guide_000133__i1829411460235">Cluster ID</em><strong id="admin_guide_000133__b14295184602318"> -m 2</strong></p>
|
|
<pre class="screen" id="admin_guide_000133__sdac62f5c91c8412e97b4300d0159206b">Start to convert Capacity scheduler to Superior Scheduler, clusterId=1
|
|
Start to switch the Yarn scheduler to Superior. Please wait...
|
|
Switch the Yarn scheduler to Superior successfully.</pre>
|
|
</li></ol>
|
|
</li></ul>
|
|
<div class="note" id="admin_guide_000133__n2bd885e61ee240a29b44c9a02ba3fc38"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="admin_guide_000133__a1a92c89ef2984d35940f680994fdba9b">You can query the scheduler switching logs on the active management node.</p>
|
|
<ul id="admin_guide_000133__ueedd76816b874a2aadd665e84a3b84e0"><li id="admin_guide_000133__l35776819808443928cc81f9ba677e283">${BIGDATA_LOG_HOME}/controller/aos/switch_scheduler.log</li><li id="admin_guide_000133__l8354612882c64dd3a2664f1bbb50e6c8">${BIGDATA_LOG_HOME}/controller/aos/aos.log</li></ul>
|
|
</div></div>
|
|
</p></li></ol>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="admin_guide_000087.html">Tenant Resources</a></div>
|
|
</div>
|
|
</div>
|
|
|