Update #
Now, only Spark SQL supports DELETE & UPDATE, you can take a look to Spark Write.
Example:
DELETE FROM my_table WHERE currency = 'UNKNOWN';
Update append table has two modes:
- COW (Copy on Write): search for the hit files and then rewrite each file to remove the data that needs to be deleted from the files. This operation is costly.
- MOW (Merge on Write): By specifying
'deletion-vectors.enabled' = 'true'
, the Deletion Vectors mode can be enabled. Only marks certain records of the corresponding file for deletion and writes the deletion file, without rewriting the entire file.