forked from docs/doc-exports
Yang, Tong
48706b7552
Reviewed-by: Kacur, Michal <michal.kacur@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
13 KiB
13 KiB
Concurrency for Schema Evolution
When creating a table, you need to set hoodie.cleaner.policy.failed.writes to LAZY. Otherwise, rollback will be triggered when concurrent submission operations are performed.
DDL Concurrency
DDL Operation |
add |
rename |
change type |
change comment |
drop |
|---|---|---|---|---|---|
add |
Y |
Y |
Y |
Y |
Y |
rename |
Y |
Y |
Y |
Y |
Y |
change type |
Y |
Y |
Y |
Y |
Y |
change comment |
Y |
Y |
Y |
Y |
Y |
drop |
Y |
Y |
Y |
Y |
N |
When performing DDL operations on the same column concurrently, pay attention to the following:
- Multiple drop operations cannot be concurrently performed on the same column. Otherwise, only the first drop operation can be successfully performed and then exception message "java.lang.UnsupportedOperationException: cannot evolution schema implicitly, the column for which the update operation is performed does not exist." is thrown.
- When drop, rename, change type, and change comment operations are concurrently performed , drop operations must be executed last. Otherwise, only drop and operations before drop can be performed, and exception message "java.lang.UnsupportedOperationException: cannot evolution schema implicitly, the column for which the update operation is performed does not exist." is thrown when operations after drop are performed.
DDL and DML Concurrency
DDL Operation |
insert into |
update |
delete |
set/reset |
|---|---|---|---|---|
add |
Y |
Y |
Y |
Y |
rename |
N |
N |
Y |
N |
change type |
N |
N |
Y |
N |
change comment |
Y |
Y |
Y |
Y |
drop |
N |
N |
Y |
N |
Exception message "cannot evolution schema implicitly, actions such as rename, delete, and type change were found" is thrown when unsupported DDL or DML operations are performed concurrently.
Parent topic: Hudi Schema Evolution