//paimon.apache.org/docs/1.4/primary-key-table/merge-engine/ Recent content in Merge Engine on Apache Paimon Hugo -- gohugo.io en-us //paimon.apache.org/docs/1.4/primary-key-table/merge-engine/overview/ Mon, 01 Jan 0001 00:00:00 +0000 //paimon.apache.org/docs/1.4/primary-key-table/merge-engine/overview/ Overview # When Paimon sink receives two or more records with the same primary keys, it will merge them into one record to keep primary keys unique. By specifying the merge-engine table property, users can choose how records are merged together. Always set table.exec.sink.upsert-materialize to NONE in Flink SQL TableConfig, sink upsert-materialize may result in strange behavior. When the input is out of order, we recommend that you use Sequence Field to correct disorder. //paimon.apache.org/docs/1.4/primary-key-table/merge-engine/partial-update/ Mon, 01 Jan 0001 00:00:00 +0000 //paimon.apache.org/docs/1.4/primary-key-table/merge-engine/partial-update/ Partial Update # By specifying 'merge-engine' = 'partial-update', users have the ability to update columns of a record through multiple updates until the record is complete. This is achieved by updating the value fields one by one, using the latest data under the same primary key. However, null values are not overwritten in the process. For example, suppose Paimon receives three records: <1, 23.0, 10, NULL>- <1, NULL, NULL, 'This is a book'> <1, 25. //paimon.apache.org/docs/1.4/primary-key-table/merge-engine/aggregation/ Mon, 01 Jan 0001 00:00:00 +0000 //paimon.apache.org/docs/1.4/primary-key-table/merge-engine/aggregation/ Aggregation # NOTE: Always set table.exec.sink.upsert-materialize to NONE in Flink SQL TableConfig. Sometimes users only care about aggregated results. The aggregation merge engine aggregates each value field with the latest data one by one under the same primary key according to the aggregate function. Each field not part of the primary keys can be given an aggregate function, specified by the fields.<field-name>.aggregate-function table property, otherwise it will use last_non_null_value aggregation as default. //paimon.apache.org/docs/1.4/primary-key-table/merge-engine/first-row/ Mon, 01 Jan 0001 00:00:00 +0000 //paimon.apache.org/docs/1.4/primary-key-table/merge-engine/first-row/ First Row # By specifying 'merge-engine' = 'first-row', users can keep the first row of the same primary key. It differs from the deduplicate merge engine that in the first-row merge engine, it will generate insert only changelog. first-row merge engine only supports none and lookup changelog producer. For streaming queries must be used with the lookup changelog producer. You can not specify sequence.field. Not accept DELETE and UPDATE_BEFORE message.