org.apache.paimon.flink.action.cdc

Class SynchronizationActionBase

java.lang.Object

org.apache.paimon.flink.action.ActionBase

org.apache.paimon.flink.action.cdc.SynchronizationActionBase

All Implemented Interfaces:

Action

Direct Known Subclasses:

SyncDatabaseActionBase, SyncTableActionBase

public abstract class SynchronizationActionBase
extends ActionBase

Base Action for table/database synchronizing job.

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	allowUpperCase
protected org.apache.flink.configuration.Configuration	cdcSourceConfig
protected String	database
protected CdcMetadataConverter[]	metadataConverters
protected SyncJobHandler	syncJobHandler
protected Map<String,String>	tableConfig
protected TypeMapping	typeMapping

Fields inherited from class org.apache.paimon.flink.action.ActionBase

batchTEnv, catalog, catalogName, catalogOptions, env, flinkCatalog

Constructor Summary

Constructors

Constructor and Description
SynchronizationActionBase(String warehouse, String database, Map<String,String> catalogConfig, Map<String,String> cdcSourceConfig, SyncJobHandler syncJobHandler)

Method Summary

Modifier and Type	Method and Description
protected FileStoreTable	alterTableOptions(Identifier identifier, FileStoreTable table)
protected void	beforeBuildingSourceSink()
void	build() Builds the action within the given Flink Stream Execution Environment.
protected abstract EventParser.Factory<RichCdcMultiplexRecord>	buildEventParserFactory()
protected abstract void	buildSink(org.apache.flink.streaming.api.datastream.DataStream<RichCdcMultiplexRecord> input, EventParser.Factory<RichCdcMultiplexRecord> parserFactory)
protected Object	buildSource()
protected CdcTimestampExtractor	createCdcTimestampExtractor()
protected abstract org.apache.flink.api.common.functions.FlatMapFunction<CdcSourceRecord,RichCdcMultiplexRecord>	recordParse()
void	run() The execution method of the action.
Map<String,String>	tableConfig()
protected abstract void	validateCaseSensitivity()
protected void	validateRuntimeExecutionMode()
SynchronizationActionBase	withMetadataColumns(List<String> metadataColumns)
SynchronizationActionBase	withTableConfig(Map<String,String> tableConfig)
SynchronizationActionBase	withTypeMapping(TypeMapping typeMapping)

Methods inherited from class org.apache.paimon.flink.action.ActionBase

catalogConfig, catalogLoader, compatibleCheck, execute, initFlinkCatalog, initFlinkEnv, initPaimonCatalog, toPaimonTypes, withStreamExecutionEnvironment

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

database

protected final String database

cdcSourceConfig

protected final org.apache.flink.configuration.Configuration cdcSourceConfig

syncJobHandler

protected final SyncJobHandler syncJobHandler

allowUpperCase

protected final boolean allowUpperCase

tableConfig

protected Map<String,String> tableConfig

typeMapping

protected TypeMapping typeMapping

metadataConverters

protected CdcMetadataConverter[] metadataConverters

Constructor Detail

SynchronizationActionBase

public SynchronizationActionBase(String warehouse,
                                 String database,
                                 Map<String,String> catalogConfig,
                                 Map<String,String> cdcSourceConfig,
                                 SyncJobHandler syncJobHandler)

Method Detail

withTableConfig

public SynchronizationActionBase withTableConfig(Map<String,String> tableConfig)

withTypeMapping

public SynchronizationActionBase withTypeMapping(TypeMapping typeMapping)

withMetadataColumns

public SynchronizationActionBase withMetadataColumns(List<String> metadataColumns)

tableConfig

@VisibleForTesting
public Map<String,String> tableConfig()

build

public void build()
           throws Exception

Description copied from interface: Action

Builds the action within the given Flink Stream Execution Environment.

This method is responsible for setting up any necessary configurations or resources needed for the action to run. It is called before the `run` method to prepare the environment for execution.

By default, this method is empty and can be overridden by subclasses to provide custom setup logic.

Throws:

Exception - If an error occurs during the build process.

validateCaseSensitivity

protected abstract void validateCaseSensitivity()

beforeBuildingSourceSink

protected void beforeBuildingSourceSink()
                                 throws Exception

Throws:

Exception

buildSource

protected Object buildSource()

createCdcTimestampExtractor

protected CdcTimestampExtractor createCdcTimestampExtractor()

validateRuntimeExecutionMode

protected void validateRuntimeExecutionMode()

recordParse

protected abstract org.apache.flink.api.common.functions.FlatMapFunction<CdcSourceRecord,RichCdcMultiplexRecord> recordParse()

buildEventParserFactory

protected abstract EventParser.Factory<RichCdcMultiplexRecord> buildEventParserFactory()

buildSink

protected abstract void buildSink(org.apache.flink.streaming.api.datastream.DataStream<RichCdcMultiplexRecord> input,
                                  EventParser.Factory<RichCdcMultiplexRecord> parserFactory)

alterTableOptions

protected FileStoreTable alterTableOptions(Identifier identifier,
                                           FileStoreTable table)

run

public void run()
         throws Exception

Description copied from interface: Action

The execution method of the action.

Throws:

Exception