vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/mrs/component-operation-guide-lts/mrs_01_24745.html
Yang, Tong 48706b7552 MRS COMP-LTS 320-lts.1 version 
Reviewed-by: Kacur, Michal <michal.kacur@t-systems.com>
Co-authored-by: Yang, Tong <yangtong2@huawei.com>
Co-committed-by: Yang, Tong <yangtong2@huawei.com>
2024-04-12 12:51:10 +00:00

4.6 KiB
Raw Permalink Blame History

Configuring the Drop Partition Command to Support Batch Deletion

This section applies only to MRS 3.2.0 or later.

Scenario

Currently, the Drop Partition command in Spark supports partition deletion using only the equal sign (=). This configuration allows multiple filter criteria to be used to delete partitions in batches, for example, <, <=, >, >=, !>, and !<.

Configuration

Log in to FusionInsight Manager and choose Cluster. Click the name of the desired cluster and choose Services > Spark2x. On the page that is displayed, click the Configurations tab then All Configurations, and search for the following parameters.

Parameter

Description

Default Value

spark.sql.dropPartitionsInBatch.enabled

If this parameter is set to true, the Drop Partition command supports the following filter criteria: <, <=, >, >=, !>, and !<.

true

spark.sql.dropPartitionsInBatch.limit

Indicates the maximum number of partitions that can be batch dropped.

1000