Yang, Tong 48706b7552 MRS COMP-LTS 320-lts.1 version

Reviewed-by: Kacur, Michal <michal.kacur@t-systems.com>
Co-authored-by: Yang, Tong <yangtong2@huawei.com>
Co-committed-by: Yang, Tong <yangtong2@huawei.com>

2024-04-12 12:51:10 +00:00

2.0 KiB

Raw Permalink Blame History

Enabling Schema Evolution

Schema evolution cannot be disabled once being enabled.

To use spark-beeline, log in to FusionInsight Manager, choose Cluster > Services > Spark2x, and click the Configurations tab then the All Configurations sub-tab.
Search for spark.sql.extensions in the search box and change its value of JDBCServer to org.apache.spark.sql.hive.FISparkSessionExtension,org.apache.spark.sql.hudi.HoodieSparkSessionExtension,org.apache.spark.sql.hive.CarbonInternalExtensions.

For SQL operations, run the following command before running any SQL statements:
```
set hoodie.schema.evolution.enable=true
```
For API calls, specify the following parameter in DataFrame options:
```
hoodie.schema.evolution.enable -> true
```

Parent topic: SparkSQL Schema Evolution and Syntax Description

2.0 KiB Raw Permalink Blame History

Enabling Schema Evolution

2.0 KiB

Raw Permalink Blame History