org.apache.paimon.flink.action.cdc.mysql

Class MySqlSyncTableAction

java.lang.Object

org.apache.paimon.flink.action.ActionBase

org.apache.paimon.flink.action.cdc.SynchronizationActionBase

org.apache.paimon.flink.action.cdc.SyncTableActionBase

org.apache.paimon.flink.action.cdc.mysql.MySqlSyncTableAction

All Implemented Interfaces:

Action

public class MySqlSyncTableAction
extends SyncTableActionBase

An Action which synchronize one or multiple MySQL tables into one Paimon table.

You should specify MySQL source table in mySqlConfig. See document of flink-cdc-connectors for detailed keys and values.

If the specified Paimon table does not exist, this action will automatically create the table. Its schema will be derived from all specified MySQL tables. If the Paimon table already exists, its schema will be compared against the schema of all specified MySQL tables.

This action supports a limited number of schema changes. Currently, the framework can not drop columns, so the behaviors of `DROP` will be ignored, `RENAME` will add a new column. Currently supported schema changes includes:

• Adding columns.
• Altering column types. More specifically,
  • altering from a string type (char, varchar, text) to another string type with longer length,
  • altering from a binary type (binary, varbinary, blob) to another binary type with longer length,
  • altering from an integer type (tinyint, smallint, int, bigint) to another integer type with wider range,
  • altering from a floating-point type (float, double) to another floating-point type with wider range,
  are supported.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.paimon.flink.action.cdc.SyncTableActionBase

SyncTableActionBase.SchemaRetrievalException

Field Summary

Fields inherited from class org.apache.paimon.flink.action.cdc.SyncTableActionBase

computedColumnArgs, computedColumns, fileStoreTable, partitionKeys, primaryKeys, table

Fields inherited from class org.apache.paimon.flink.action.cdc.SynchronizationActionBase

allowUpperCase, cdcSourceConfig, database, metadataConverters, syncJobHandler, tableConfig, typeMapping

Fields inherited from class org.apache.paimon.flink.action.ActionBase

batchTEnv, catalog, catalogName, catalogOptions, env, flinkCatalog

Constructor Summary

Constructors

Constructor and Description
MySqlSyncTableAction(String warehouse, String database, String table, Map<String,String> catalogConfig, Map<String,String> mySqlConfig)

Method Summary

Modifier and Type	Method and Description
protected org.apache.flink.cdc.connectors.mysql.source.MySqlSource<CdcSourceRecord>	buildSource()
protected CdcTimestampExtractor	createCdcTimestampExtractor()
protected Schema	retrieveSchema()

Methods inherited from class org.apache.paimon.flink.action.cdc.SyncTableActionBase

beforeBuildingSourceSink, buildEventParserFactory, buildPaimonSchema, buildSink, fileStoreTable, recordParse, validateCaseSensitivity, withComputedColumnArgs, withPartitionKeys, withPartitionKeys, withPrimaryKeys, withPrimaryKeys

Methods inherited from class org.apache.paimon.flink.action.cdc.SynchronizationActionBase

alterTableOptions, build, run, tableConfig, validateRuntimeExecutionMode, withMetadataColumns, withTableConfig, withTypeMapping

Methods inherited from class org.apache.paimon.flink.action.ActionBase

catalogConfig, catalogLoader, compatibleCheck, execute, initFlinkCatalog, initFlinkEnv, initPaimonCatalog, toPaimonTypes, withStreamExecutionEnvironment

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MySqlSyncTableAction

public MySqlSyncTableAction(String warehouse,
                            String database,
                            String table,
                            Map<String,String> catalogConfig,
                            Map<String,String> mySqlConfig)

Method Detail

retrieveSchema

protected Schema retrieveSchema()
                         throws Exception

Specified by:

retrieveSchema in class SyncTableActionBase

Throws:

Exception

buildSource

protected org.apache.flink.cdc.connectors.mysql.source.MySqlSource<CdcSourceRecord> buildSource()

Overrides:

buildSource in class SynchronizationActionBase

createCdcTimestampExtractor

protected CdcTimestampExtractor createCdcTimestampExtractor()

Overrides:

createCdcTimestampExtractor in class SynchronizationActionBase