vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/dws/umn/dws_01_1512.html
Lu, Huayi 95132e24fc DWS UMN 830.201_new version 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Reviewed-by: Rechenburg, Matthias <matthias.rechenburg@t-systems.com>
Co-authored-by: Lu, Huayi <luhuayi@huawei.com>
Co-committed-by: Lu, Huayi <luhuayi@huawei.com>
2024-05-27 11:54:34 +00:00

127 lines
20 KiB
HTML
Raw Blame History

<a name="EN-US_TOPIC_0000001658895346"></a><a name="EN-US_TOPIC_0000001658895346"></a>
<h1 class="topictitle1">Feature Description</h1>
<div id="body8662426"><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p444042710229">GaussDB(DWS) resource pools provide concurrency management, memory management, CPU management, and exception rules.</p>
<div class="section" id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_section17310122571"><h4 class="sectiontitle">Concurrency Management</h4><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p111411320141919">Concurrency represents the maximum number of concurrent queries in a resource pool. Concurrency management can limit the number of concurrent queries to reduce resource contention and improve resource utilization.</p>
<p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p13311944576">The concurrency management rules are as follows:</p>
<ul id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_ul1551655115912"><li id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_li15516125114597">If short query acceleration is enabled, complex queries are under resource pool concurrency control, and simple queries are under short query concurrency control.</li><li id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_li651725119593">If short query acceleration is disabled, complex and simple queries are both under resource pool concurrency control. Short query concurrency control is invalid.</li></ul>
</div>
<div class="section" id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_section6729105320220"><h4 class="sectiontitle">Memory Management</h4><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p2755015556">Each resource pool occupies a certain percentage of memory.</p>
<p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p1844582118315">Memory management aims to prevent out of memory (OOM) in a database, isolate the memory of different resource pools, and to control memory usage. Memory is managed from the following aspects:</p>
<ul id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_ul186961738032"><li id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_li984017357319">Global memory management<p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p0445421135"><a name="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_li984017357319"></a><a name="en-us_topic_0000001372679822_li984017357319"></a>To prevent OOM, set the global memory upper limit (<strong id="EN-US_TOPIC_0000001658895346__b1656210321146">max_process_memory</strong>) to a proper value. Global memory management before a query controls memory usage to prevent OOM management. Global memory management during a query prevents errors during query execution.</p>
<ul id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_ul490493702017"><li id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_li2565335142014">Management before a query<p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p444510211836"><a name="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_li2565335142014"></a><a name="en-us_topic_0000001372679822_li2565335142014"></a>The service checks the estimated memory usage of a query in the slow queue, and compares it with the actual usage. The estimation will be adjusted if it is smaller than the actual usage. Before a query is executed, the service checks whether the available memory is sufficient for the query. If yes, the query can be executed directly. If no, the query needs to be queued and executed after other queries release resources.</p>
</li><li id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_li188541240132019">Management during a query<p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p13862124019207"><a name="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_li188541240132019"></a><a name="en-us_topic_0000001372679822_li188541240132019"></a>During a query, the service checks whether the requested memory exceeds a certain limit. If yes, an error will be reported, and memory occupied by the query will be released.</p>
</li></ul>
</li><li id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_li26524186418">Resource pool memory management<p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p14207037171211"><a name="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_li26524186418"></a><a name="en-us_topic_0000001372679822_li26524186418"></a>Resource pool memory management puts a limit on dedicated quotas. A workload queue can only use the memory allocated to it, and cannot use idle memory in other resource pools.</p>
<p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p17141647111212">The resource pool memory is allocated in percentage. The value range is 0 to 100. The value <strong id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_b214835955211">0</strong> indicates that the resource pool does not perform memory management. The value <strong id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_b162265735418">100</strong> indicates that the resource pool performs memory management and can use all the global memory.</p>
<p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p368419184415">The sum of memory percentages allocated to all resource pools cannot exceed 100. Resource pool memory management is performed only before a query in the slow queue starts. It works in a way similar to the global memory management before a query. Before a query in the slow queue in a resource pool is executed, its memory usage is estimated. If the estimation is greater than the resource pool memory, the query needs to be queued and can be executed only after earlier queries in the pool are complete and resources released.</p>
</li></ul>
</div>
<div class="section" id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_section8129175313310"><h4 class="sectiontitle">CPU Management</h4><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p1010245142910">CPU share and CPU limit can be managed.</p>
<ul id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_ul148371439219"><li id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_li1237694417414">CPU share: If the system is heavily loaded, CPU resources are allocated to resource pools based on the specific CPU shares. If the system not busy, this configuration does not take effect.</li><li id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_li484013917116">CPU limit: It specifies the maximum number of CPU cores used by a resource pool. The resource usage of jobs in the resource pool cannot exceed this limit no matter whether the system is busy or not.</li></ul>
<p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p7841123913118">Choose either of the preceding management methods as needed. In CPU share management, CPUs can be shared and fully utilized, but resource pools are not isolated and may affect the query performance of each other. In CPU limit management, the CPUs of different resource pools are isolated, but this may result in the waste of idle resources.</p>
<div class="note" id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_note12836143920117"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p3836239213">The CPU usage limit is supported only by clusters of version 8.1.3 or later.</p>
</div></div>
</div>
<div class="section" id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_section6688114018815"><h4 class="sectiontitle">Exception Rules</h4><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p94164510278">To avoid query blocking or performance deterioration, you can configure exception rules to let the service automatically identify and handle abnormal queries, preventing slow SQL statements from occupying too many resources for a long time.</p>
<p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p538718597266"></p>
<p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p5591342141711"><span><img id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_image123361743131711" src="figure/en-us_image_0000001759579597.png" title="Click to enlarge" class="imgResize"></span></p>
<p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p2981141415117">The following table describes exception rules.</p>
<div class="tablenoborder"><a name="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_table595493692317"></a><a name="en-us_topic_0000001372679822_table595493692317"></a><table cellpadding="4" cellspacing="0" summary="" id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_table595493692317" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Exception rule parameters</caption><thead align="left"><tr id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_row14955536172317"><th align="left" class="cellrowborder" valign="top" width="20%" id="mcps1.3.5.6.2.5.1.1"><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p1195513613231">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="40%" id="mcps1.3.5.6.2.5.1.2"><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p1595523622313">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="20%" id="mcps1.3.5.6.2.5.1.3"><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p8955103652312">Value Range (0 Means No Limit)</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="20%" id="mcps1.3.5.6.2.5.1.4"><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p1495563619235">Operation</p>
</th>
</tr>
</thead>
<tbody><tr id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_row19955113692312"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.5.6.2.5.1.1 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p860271820245">Blocking Time</p>
</td>
<td class="cellrowborder" valign="top" width="40%" headers="mcps1.3.5.6.2.5.1.2 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p159551736152313">Job blocking time. It refers to the total time spent in global and local concurrent queuing. The unit is second.</p>
<p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p12211103312264">For example, if the blocking time is set to 300s, a job executed by a user in the resource pool will be terminated after being blocked for 300 seconds.</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.5.6.2.5.1.3 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p192451432182819">An integer in the range 1 to 2,147,483,647. The value <strong id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_b7797684279">0</strong> indicates no limit.</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.5.6.2.5.1.4 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p8956436112315"><strong id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_b1965392617718">Terminated</strong> or <strong id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_b98103118710">Not limited</strong></p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_row15611153115475"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.5.6.2.5.1.1 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p156111931114717">Execution Time</p>
</td>
<td class="cellrowborder" valign="top" width="40%" headers="mcps1.3.5.6.2.5.1.2 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p1458131142619">Time that has been spent in executing the job, in seconds.</p>
<p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p1761116313473">For example, if <strong id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_b165712920410639">Time required for execution</strong> is set to 100s, a job executed by a user in the resource pool will be terminated after being executed for more than 100 seconds.</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.5.6.2.5.1.3 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p391183319384">An integer in the range 1 to 2,147,483,647. The value <strong id="EN-US_TOPIC_0000001658895346__b1243261207">0</strong> indicates no limit.</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.5.6.2.5.1.4 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p346112222205"><strong id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_b440493214715">Terminated</strong> or <strong id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_b1340419325710">Not limited</strong></p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_row1961111313473"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.5.6.2.5.1.1 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p471355673819">Total CPU time on all DNs.</p>
</td>
<td class="cellrowborder" valign="top" width="40%" headers="mcps1.3.5.6.2.5.1.2 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p19611143119478">Total CPU time spent in executing a job on all DNs, in seconds.</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.5.6.2.5.1.3 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p1440253133915">An integer in the range 1 to 2,147,483,647. The value <strong id="EN-US_TOPIC_0000001658895346__b95228125">0</strong> indicates no limit.</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.5.6.2.5.1.4 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p10618132442017"><strong id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_b3720341279">Terminated</strong> or <strong id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_b1171334576">Not limited</strong></p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_row46117310476"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.5.6.2.5.1.1 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p28251415123913">Interval for Checking CPU Skew Rate</p>
</td>
<td class="cellrowborder" valign="top" width="40%" headers="mcps1.3.5.6.2.5.1.2 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p156112315474">Interval for checking the CPU skew, in seconds. This parameter must be set together with <strong id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_b25202847010639">Total CPU Time on All DNs</strong>.</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.5.6.2.5.1.3 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p1092815814711">An integer in the range 1 to 2,147,483,647. The value <strong id="EN-US_TOPIC_0000001658895346__b716611406">0</strong> indicates no limit.</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.5.6.2.5.1.4 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p17192849102014"><strong id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_b651035872">Terminated</strong> or <strong id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_b35263519710">Not limited</strong></p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_row8988125711397"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.5.6.2.5.1.1 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p122671642241">Total CPU Time Skew Rate on All DNs</p>
</td>
<td class="cellrowborder" valign="top" width="40%" headers="mcps1.3.5.6.2.5.1.2 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p898918570399">CPU time skew rate of a job executed on DNs. The value depends on the setting of <strong id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_b160554643610639">Interval for Checking CPU Skew Rate</strong>.</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.5.6.2.5.1.3 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p11926153115118">An integer in the range 1 to 100. The value <strong id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_b1413772553019">0</strong> indicates no limit.</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.5.6.2.5.1.4 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p483005013206"><strong id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_b26604351717">Terminated</strong> or <strong id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_b2066012351375">Not limited</strong></p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_row52801184451"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.5.6.2.5.1.1 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p428117844519">Data Spilled to Disk Per DN</p>
</td>
<td class="cellrowborder" valign="top" width="40%" headers="mcps1.3.5.6.2.5.1.2 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p328110814516">Allowed maximum job data spilled to disks on a DN. The unit is MB.</p>
<div class="note" id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_note1661461965019"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p1161415192504">This rule is supported only by clusters of version 8.2.0 or later.</p>
</div></div>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.5.6.2.5.1.3 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p82816819456">An integer in the range 1 to 2,147,483,647. The value <strong id="EN-US_TOPIC_0000001658895346__b909782406">0</strong> indicates no limit.</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.5.6.2.5.1.4 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p1828117884512"><strong id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_b727010213310">Terminated</strong> or <strong id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_b1927082336">Not limited</strong></p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_row1238293519473"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.5.6.2.5.1.1 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p1382153513479">Average CPU Usage Per DN</p>
</td>
<td class="cellrowborder" valign="top" width="40%" headers="mcps1.3.5.6.2.5.1.2 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p938233564714">Average CPU usage of a job on each DN. If <strong id="EN-US_TOPIC_0000001658895346__b453993515410">Interval for Checking CPU Skew Rate</strong> is configured, the interval takes effect for this parameter. If the interval is not configured, the check interval is 30 seconds by default.</p>
<div class="note" id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_note2071552855011"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p464222925110">This rule is supported only by clusters of version 8.2.0 or later.</p>
</div></div>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.5.6.2.5.1.3 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p1538283524715">An integer in the range 1 to 100. The value <strong id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_b1674133215313">0</strong> indicates no limit.</p>
</td>
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.5.6.2.5.1.4 "><p id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_p538233554717"><strong id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_b1180613336311">Terminated</strong> or <strong id="EN-US_TOPIC_0000001658895346__en-us_topic_0000001372679822_b118074338319">Not limited</strong></p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dws_01_0729.html">Resource Pool</a></div>
</div>
</div>
<script language="JavaScript">
<!--
image_size('.imgResize');
var msg_imageMax = "view original image";
var msg_imageClose = "close";
//--></script>
View Git Blame Copy Permalink