forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
24 lines
4.4 KiB
HTML
24 lines
4.4 KiB
HTML
<a name="mrs_01_1590"></a><a name="mrs_01_1590"></a>
|
|
|
|
<h1 class="topictitle1">Configuring Process Parameters</h1>
|
|
<div id="body1596442815781"><div class="section" id="mrs_01_1590__sacb66973a7e54492b0514dde92fffe4d"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_1590__a71353ef3fa2d451da83856be7592202c">In Flink on Yarn mode, there are JobManagers and TaskManagers. JobManagers and TaskManagers schedule and run tasks.</p>
|
|
<p id="mrs_01_1590__afa6c4cefe2e54084ae02e66caaa7c86a">Therefore, configuring parameters of JobManagers and TaskManagers can optimize the execution performance of a Flink application. Perform the following steps to optimize the Flink cluster performance.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_1590__sbeef4ba7acae43d3b1dadff6928dc9a8"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_1590__o474bf189d16f45c596f753886bc52759"><li id="mrs_01_1590__l80c0060c7801462b88a0b8d764ee2e3e"><span>Configure JobManager memory.</span><p><p id="mrs_01_1590__a246c38768dd04abf96bfc53f75bfa335">JobManagers are responsible for task scheduling and message communications between TaskManagers and ResourceManagers. JobManager memory needs to be increased as the number of tasks and the DOP increases.</p>
|
|
<p id="mrs_01_1590__aba4592187cd0428d9dda744495662c5a">JobManager memory needs to be configured based on the number of tasks.</p>
|
|
<ul id="mrs_01_1590__u47ef4ed206224ab3ac33c60efe2a83b1"><li id="mrs_01_1590__l6120c34682cb4a848d63f2b44dfbe656">When running the <strong id="mrs_01_1590__b62198855783846">yarn-session</strong> command, add the <strong id="mrs_01_1590__b74885188983846">-jm MEM</strong> parameter to configure the memory.</li><li id="mrs_01_1590__l1d479339d3804db9bc11791e625f6a33">When running the <strong id="mrs_01_1590__b10740192383846">yarn-cluster</strong> command, add the <strong id="mrs_01_1590__b130783855183846">-yjm MEM</strong> parameter to configure the memory.</li></ul>
|
|
</p></li><li id="mrs_01_1590__l09258a595e2e4cf6b4a6a574a89c4d77"><span>Configure the number of TaskManagers.</span><p><p id="mrs_01_1590__a1e4d5b3402fb469a88f3f59beefab7b0">Each core of a TaskManager can run a task at the same time. Increasing the number of TaskManagers has the same effect as increasing the DOP. Therefore, you can increase the number of TaskManagers to improve efficiency when there are sufficient resources.</p>
|
|
</p></li><li id="mrs_01_1590__l5e7895768fb94fc59468cc5f1a53b1f4"><span>Configure the number of TaskManager slots.</span><p><p id="mrs_01_1590__a946a8d8df42f4544bc3ccfc203b06750">Multiple cores of a TaskManager can process multiple tasks at the same time. This has the same effect as increasing the DOP. However, the balance between the number of cores and the memory must be maintained, because all cores of a TaskManager share the memory.</p>
|
|
<ul id="mrs_01_1590__u88c819da6f3745e8bbfccb9415f7ac34"><li id="mrs_01_1590__l4ca087abf6ea4231b74825f07dfab9e0">When running the <strong id="mrs_01_1590__b82025366583846">yarn-session</strong> command, add the <strong id="mrs_01_1590__b93673866883846">-s NUM</strong> parameter to configure the number of slots.</li><li id="mrs_01_1590__l4993f5fb8a694963bb765d30e1df71b7">When running the <strong id="mrs_01_1590__b112329050383846">yarn-cluster</strong> command, add the <strong id="mrs_01_1590__b200269059383846">-ys NUM</strong> parameter to configure the number of slots.</li></ul>
|
|
</p></li><li id="mrs_01_1590__laf9cb94e14c94dd6be7603c7452351c9"><span>Configure TaskManager memory.</span><p><p id="mrs_01_1590__a2e5f10f014a54ef7818cb34bc91f0df6">TaskManager memory is used for task execution and communication. A large-size task requires more resources. In this case, you can increase the memory.</p>
|
|
<ul id="mrs_01_1590__ud08fea63bedb45a099a61f830810ea7a"><li id="mrs_01_1590__l8d19aa8054174d04b26f4eab6317e483">When running the <strong id="mrs_01_1590__b172911244153215">yarn-session</strong> command, add the <strong id="mrs_01_1590__b2291184410324">-tm MEM</strong> parameter to configure the memory.</li><li id="mrs_01_1590__leb2c8b2ad50240f882c1840631ff0737">When running the <strong id="mrs_01_1590__b52485844183846">yarn-cluster</strong> command, add the <strong id="mrs_01_1590__b58198574283846">-ytm MEM</strong> parameter to configure the memory.</li></ul>
|
|
</p></li></ol>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1587.html">Optimization DataStream</a></div>
|
|
</div>
|
|
</div>
|
|
|