org.apache.paimon.flink.action.cdc.postgres

Class PostgresSyncTableAction

java.lang.Object

org.apache.paimon.flink.action.ActionBase

org.apache.paimon.flink.action.cdc.SynchronizationActionBase

org.apache.paimon.flink.action.cdc.SyncTableActionBase

org.apache.paimon.flink.action.cdc.postgres.PostgresSyncTableAction

All Implemented Interfaces:

Action

public class PostgresSyncTableAction
extends SyncTableActionBase

An Action which synchronize one or multiple PostgreSQL tables into one Paimon table.

You should specify PostgreSQL source table in postgresConfig. See document of flink-cdc-connectors for detailed keys and values.

If the specified Paimon table does not exist, this action will automatically create the table. Its schema will be derived from all specified PostgreSQL tables. If the Paimon table already exists, its schema will be compared against the schema of all specified PostgreSQL tables.

This action supports a limited number of schema changes. Currently, the framework can not drop columns, so the behaviors of `DROP` will be ignored, `RENAME` will add a new column. Currently supported schema changes includes:

• Adding columns.
• Altering column types. More specifically,
  • altering from a string type (char, varchar, text) to another string type with longer length,
  • altering from a binary type (binary, varbinary) to another binary type with longer length,
  • altering from an integer type (smallint, int, bigint) to another integer type with wider range,
  • altering from a floating-point type (float, double) to another floating-point type with wider range,
  are supported.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.paimon.flink.action.cdc.SyncTableActionBase

SyncTableActionBase.SchemaRetrievalException

Field Summary

Fields inherited from class org.apache.paimon.flink.action.cdc.SyncTableActionBase

computedColumnArgs, computedColumns, fileStoreTable, partitionKeys, primaryKeys, table

Fields inherited from class org.apache.paimon.flink.action.cdc.SynchronizationActionBase

allowUpperCase, cdcSourceConfig, database, metadataConverters, syncJobHandler, tableConfig, typeMapping

Fields inherited from class org.apache.paimon.flink.action.ActionBase

batchTEnv, catalog, catalogName, catalogOptions, env, flinkCatalog

Constructor Summary

Constructors

Constructor and Description
PostgresSyncTableAction(String warehouse, String database, String table, Map<String,String> catalogConfig, Map<String,String> postgresConfig)

Method Summary

Modifier and Type	Method and Description
protected org.apache.flink.cdc.connectors.base.source.jdbc.JdbcIncrementalSource<CdcSourceRecord>	buildSource()
protected Schema	retrieveSchema()

Methods inherited from class org.apache.paimon.flink.action.cdc.SyncTableActionBase

beforeBuildingSourceSink, buildEventParserFactory, buildPaimonSchema, buildSink, fileStoreTable, recordParse, validateCaseSensitivity, withComputedColumnArgs, withPartitionKeys, withPartitionKeys, withPrimaryKeys, withPrimaryKeys

Methods inherited from class org.apache.paimon.flink.action.cdc.SynchronizationActionBase

alterTableOptions, build, createCdcTimestampExtractor, run, tableConfig, validateRuntimeExecutionMode, withMetadataColumns, withTableConfig, withTypeMapping

Methods inherited from class org.apache.paimon.flink.action.ActionBase

catalogConfig, catalogLoader, compatibleCheck, execute, initFlinkCatalog, initFlinkEnv, initPaimonCatalog, toPaimonTypes, withStreamExecutionEnvironment

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PostgresSyncTableAction

public PostgresSyncTableAction(String warehouse,
                               String database,
                               String table,
                               Map<String,String> catalogConfig,
                               Map<String,String> postgresConfig)

Method Detail

retrieveSchema

protected Schema retrieveSchema()
                         throws Exception

Specified by:

retrieveSchema in class SyncTableActionBase

Throws:

Exception

buildSource

protected org.apache.flink.cdc.connectors.base.source.jdbc.JdbcIncrementalSource<CdcSourceRecord> buildSource()

Overrides:

buildSource in class SynchronizationActionBase