forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
39 lines
4.7 KiB
HTML
39 lines
4.7 KiB
HTML
<a name="mrs_01_1675"></a><a name="mrs_01_1675"></a>
|
|
|
|
<h1 class="topictitle1">Configuring Reserved Percentage of Disk Usage on DataNodes</h1>
|
|
<div id="body1595904095021"><div class="section" id="mrs_01_1675__sb0e252c3839640208ac6dbede3179749"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_1675__abad97e34a4314bdcb49496ae65909d2b">When the Yarn local directory and DataNode directory are on the same disk, the disk with larger capacity can run more tasks. Therefore, more intermediate data is stored in the Yarn local directory.</p>
|
|
<p id="mrs_01_1675__adb5346b00f2e4d4d88c2b15cd5ae00b3">Currently, you can set <span class="parmname" id="mrs_01_1675__p8e9513b5eeaf404882c7937ae6ba94ef"><b>dfs.datanode.du.reserved</b></span> to configure the absolute value of the reserved disk space on DataNodes. A small value cannot meet the requirements of a disk with large capacity. However, configuring a large value for a disk with same capacity wastes a lot of disk space.</p>
|
|
<p id="mrs_01_1675__af99c926f083648a1a5e1f5ddf747b5a6">To avoid this problem, a new parameter <span class="parmname" id="mrs_01_1675__pa9fe6fdfd204476cae640cad8939743a"><b>dfs.datanode.du.reserved.percentage</b></span> is introduced to configure the reserved percentage of the disk space.</p>
|
|
<div class="note" id="mrs_01_1675__n1370fcb9728545c7bba562f9a8a90aed"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="mrs_01_1675__u43f3a679eabb42d09b84df845f441b56"><li id="mrs_01_1675__le92fa4ce60904cb88b9015234812ea41">If <span class="parmname" id="mrs_01_1675__pfb4fe3fa9ffe4206871448d61926b961"><b>dfs.datanode.du.reserved.percentage</b></span> and <span class="parmname" id="mrs_01_1675__p9b2cd7bcfe3a4a108e3965e43fc8f3f4"><b>dfs.datanode.du.reserved</b></span> are configured at the same time, the larger value of the reserved disk space calculated using the two parameters is used as the reserved space of the data nodes.</li><li id="mrs_01_1675__l5a1a85bb104447feac8bb6a5bf359a79">You are advised to set <span class="parmname" id="mrs_01_1675__pcf5efb880dec48e99ab7fdc619ca3667"><b>dfs.datanode.du.reserved</b></span> or <span class="parmname" id="mrs_01_1675__p7e703d28c24e41839edb9eaf2a6c9a0b"><b>dfs.datanode.du.reserved.percentage</b></span> based on the actual disk space.</li></ul>
|
|
</div></div>
|
|
</div>
|
|
<div class="section" id="mrs_01_1675__s3625eef70e3f4994a3fd7500805c9d64"><h4 class="sectiontitle">Configuration Description</h4><p id="mrs_01_1675__p39519895015">Go to the <strong id="mrs_01_1675__b374415166263">All Configurations</strong> page of HDFS and enter a parameter name in the search box by referring to <a href="mrs_01_2125.html">Modifying Cluster Service Configuration Parameters</a>.</p>
|
|
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1675__t2133f452b3054aabb604e34db82954ab" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="mrs_01_1675__r8d12e90142f441fab2873e4fb293ae24"><th align="left" class="cellrowborder" valign="top" width="33.23%" id="mcps1.3.2.3.2.4.1.1"><p id="mrs_01_1675__en-us_topic_0039590212_p310726115357"><strong id="mrs_01_1675__a531be98653e84f99829a48198dfac598">Parameter</strong></p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="49.94%" id="mcps1.3.2.3.2.4.1.2"><p id="mrs_01_1675__a1dadb94f3d0a4600a5b5db38896f31fe"><strong id="mrs_01_1675__ad48d211a8daf48be8c279fee403e5514">Description</strong></p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="16.830000000000002%" id="mcps1.3.2.3.2.4.1.3"><p id="mrs_01_1675__ab9155cddda684bb9b5d9ac98503c34b3"><strong id="mrs_01_1675__af5bce31f83c5417cb278eb2637dc1d23">Default Value</strong></p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_1675__r92eccb62350c4b6ab3d6cf6d68160e40"><td class="cellrowborder" valign="top" width="33.23%" headers="mcps1.3.2.3.2.4.1.1 "><p id="mrs_01_1675__a92cd9e06fe734e40bd1ef60e97c1dd38">dfs.datanode.du.reserved.percentage</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="49.94%" headers="mcps1.3.2.3.2.4.1.2 "><p id="mrs_01_1675__a9fa16d295c924ea98db550f1fdc8ca22">Indicates the percentage of the reserved disk space on DataNodes. The DataNode permanently reserves the disk space calculated using this percentage.</p>
|
|
<p id="mrs_01_1675__aa713109fe17c42a0946cde2a9eef009d">The value is an integer ranging from 0 to 100.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.830000000000002%" headers="mcps1.3.2.3.2.4.1.3 "><p id="mrs_01_1675__en-us_topic_0039590212_p434079081387">10</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_0790.html">Using HDFS</a></div>
|
|
</div>
|
|
</div>
|
|
|