forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
18 lines
2.0 KiB
HTML
18 lines
2.0 KiB
HTML
<a name="mrs_01_2037"></a><a name="mrs_01_2037"></a>
|
|
|
|
<h1 class="topictitle1">Why Is "RejectedExecutionException" Displayed When I Exit Spark SQL?</h1>
|
|
<div id="body1595920221845"><div class="section" id="mrs_01_2037__s5166f26043214b29b04bd74605927e3b"><h4 class="sectiontitle">Question</h4><p id="mrs_01_2037__ad23bea19e51c46ccbfbb53000f963e73">After successfully running Spark tasks with large data volume, for example, 2-TB TPCDS test suite, why is the abnormal stack information <span class="uicontrol" id="mrs_01_2037__ud29db7b8b3004467886e5b044f31bd01"><b>"RejectedExecutionException"</b></span> displayed sometimes? The log is as follows:</p>
|
|
<pre class="screen" id="mrs_01_2037__s23f3b2bc95174f9aa72e05a3030b7178">16/07/16 10:19:56 ERROR TransportResponseHandler: Still have 2 requests outstanding when connection from linux-192/10.1.1.5:59250 is closed
|
|
java.util.concurrent.RejectedExecutionException: Task scala.concurrent.impl.CallbackRunnable@5fc1ab rejected from java.util.concurrent.ThreadPoolExecutor@52fa7e19[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 3025]</pre>
|
|
</div>
|
|
<div class="section" id="mrs_01_2037__s30b0de0fe1964d1799c667adff94f1a4"><h4 class="sectiontitle">Answer</h4><p id="mrs_01_2037__a9dbe7bb8b37243708af4d54d350363e7">When Spark SQL is closed, the application and the message channel are closed. If there are unprocessed messages, the connection should be closed to rectify the exception. If the thread pool inside Scala is closed, the abnormal stack information "RejectedExecutionException" is displayed. This abnormal stack information will not be displayed if the thread pool inside Scala is not closed.</p>
|
|
<p id="mrs_01_2037__a84204717fd30409d8cc32744d4e6b2af">The error occurs when the application is successfully run and closed. Therefore, the error will not affect the services.</p>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_2022.html">Spark SQL and DataFrame</a></div>
|
|
</div>
|
|
</div>
|
|
|