org.apache.paimon.flink.action.cdc.format.debezium

Class DebeziumJsonRecordParser

java.lang.Object

org.apache.paimon.flink.action.cdc.format.AbstractRecordParser

org.apache.paimon.flink.action.cdc.format.AbstractJsonRecordParser

org.apache.paimon.flink.action.cdc.format.debezium.DebeziumJsonRecordParser

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.functions.FlatMapFunction<CdcSourceRecord,RichCdcMultiplexRecord>, org.apache.flink.api.common.functions.Function

public class DebeziumJsonRecordParser
extends AbstractJsonRecordParser

The DebeziumRecordParser class extends the abstract AbstractJsonRecordParser and is designed to parse records from Debezium's JSON change data capture (CDC) format. Debezium is a CDC solution for MySQL databases that captures row-level changes to database tables and outputs them in JSON format. This parser extracts relevant information from the Debezium-JSON format and converts it into a list of RichCdcMultiplexRecord objects.

The class supports various database operations such as INSERT, UPDATE, DELETE, and READ (snapshot reads), and creates corresponding RichCdcMultiplexRecord objects to represent these changes.

Validation is performed to ensure that the JSON records contain all necessary fields, including the 'before' and 'after' states for UPDATE operations, and the class also supports schema extraction for the Kafka topic. Debezium's specific fields such as 'source', 'op' for operation type, and primary key field names are used to construct the details of each record event.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.paimon.flink.action.cdc.format.AbstractJsonRecordParser

root

Fields inherited from class org.apache.paimon.flink.action.cdc.format.AbstractRecordParser

computedColumns, FIELD_DATABASE, FIELD_TABLE, typeMapping

Constructor Summary

Constructors

Constructor and Description
DebeziumJsonRecordParser(TypeMapping typeMapping, List<ComputedColumn> computedColumns)

Method Summary

Modifier and Type	Method and Description
protected String	dataField()
List<RichCdcMultiplexRecord>	extractRecords()
protected Map<String,String>	extractRowData(org.apache.paimon.shade.jackson2.com.fasterxml.jackson.databind.JsonNode record, RowType.Builder rowTypeBuilder)
protected String	format()
protected String	getDatabaseName()
protected String	getTableName()
protected String	primaryField()
protected void	setRoot(CdcSourceRecord record)

Methods inherited from class org.apache.paimon.flink.action.cdc.format.AbstractJsonRecordParser

checkNotNull, checkNotNull, extractPrimaryKeys, fillDefaultTypes, getAndCheck, getAndCheck, mergeOldRecord, processRecord

Methods inherited from class org.apache.paimon.flink.action.cdc.format.AbstractRecordParser

buildSchema, createRecord, evalComputedColumns, flatMap, isDDL

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DebeziumJsonRecordParser

public DebeziumJsonRecordParser(TypeMapping typeMapping,
                                List<ComputedColumn> computedColumns)

Method Detail

extractRecords

public List<RichCdcMultiplexRecord> extractRecords()

Specified by:

extractRecords in class AbstractRecordParser

setRoot

protected void setRoot(CdcSourceRecord record)

Overrides:

setRoot in class AbstractJsonRecordParser

extractRowData

protected Map<String,String> extractRowData(org.apache.paimon.shade.jackson2.com.fasterxml.jackson.databind.JsonNode record,
                                            RowType.Builder rowTypeBuilder)

Overrides:

extractRowData in class AbstractJsonRecordParser

primaryField

protected String primaryField()

Specified by:

primaryField in class AbstractJsonRecordParser

dataField

protected String dataField()

Specified by:

dataField in class AbstractJsonRecordParser

getTableName

@Nullable
protected String getTableName()

Overrides:

getTableName in class AbstractJsonRecordParser

getDatabaseName

@Nullable
protected String getDatabaseName()

Overrides:

getDatabaseName in class AbstractJsonRecordParser

format

protected String format()

Specified by:

format in class AbstractRecordParser