:original_name: mrs_01_1950.html

.. _mrs_01_1950:

Configuring Whether to Display Spark SQL Statements Containing Sensitive Words
==============================================================================

Scenario
--------

SQL statements executed by users may contain sensitive information (such as passwords). Disclosure of such information may incur security risks. You can configure the **spark.sql.redaction.string.regex** parameter to shield SQL keywords that contain sensitive words in logs and on the web UI.

Spark SQL statements consist of two parts:

#. Spark SQL statements in logs:

   -  Driver log: In the JDBCServer service, each time an SQL statement is executed through Beeline, the corresponding SQL statement is added to the Driver log, for example, **Running query'show tables' with 0f8fee16-4291-4854-a7b4-b87a162f7cbb**.
   -  eventLog log: If the eventLog writing function is enabled for each Spark application (**spark.eventLog.enabled** is set to **true**), event logs will be written. SQL statements executed using JDBCServer and Spark SQL will also be added to the eventLog file.

#. Spark SQL statements on the web UI:

   -  SparkUI: When executing SQL statements, you can view the executed SQL statements on the **Jobs** and **Stages** tab pages on the SparkUI.
   -  HistoryServer: HistoryServer reads the eventLog file and displays app information on the page. Therefore, if the eventLog file contains SQL statement records, you can view the corresponding SQL statements on the HistoryServer page.

Configuration
-------------

.. table:: **Table 1** Parameter description

   +----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+
   | Parameter                        | Description                                                                                                                                                                                                                         | Default Value         |
   +==================================+=====================================================================================================================================================================================================================================+=======================+
   | spark.sql.redaction.string.regex | Regular expression that determines which part in a string generated by Spark is sensitive words. If the regular expression matches sensitive words of the string, these sensitive words are replaced with **\*********(redacted)**. | pwd|password          |
   |                                  |                                                                                                                                                                                                                                     |                       |
   |                                  | .. note::                                                                                                                                                                                                                           |                       |
   |                                  |                                                                                                                                                                                                                                     |                       |
   |                                  |    The value must be a regular expression.                                                                                                                                                                                          |                       |
   +----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+

.. note::

   -  If Spark Beeline is used, you need to restart JDBCServer for the configuration to take effect. If spark-sql is used, you need to restart spark-sql for the configuration to take effect.
   -  The preceding parameters are valid only for the SQL statements that are executed after the configuration.
