org.apache.paimon.flink.kafka

Class KafkaLogSourceProvider

java.lang.Object

org.apache.paimon.flink.kafka.KafkaLogSourceProvider

All Implemented Interfaces:

Serializable, LogSourceProvider

public class KafkaLogSourceProvider
extends Object
implements LogSourceProvider

A Kafka LogSourceProvider.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description

KafkaLogSourceProvider(String topic, Properties properties, org.apache.flink.table.types.DataType physicalType, int[] primaryKey, org.apache.flink.api.common.serialization.DeserializationSchema<org.apache.flink.table.data.RowData> primaryKeyDeserializer, org.apache.flink.api.common.serialization.DeserializationSchema<org.apache.flink.table.data.RowData> valueDeserializer, int[][] projectFields, CoreOptions.LogConsistency consistency, CoreOptions.StartupMode scanMode, Long timestampMills)

Method Summary

Modifier and Type	Method and Description
org.apache.flink.connector.kafka.source.KafkaSource<org.apache.flink.table.data.RowData>	createSource(Map<Integer,Long> bucketOffsets) Creates a Source instance.
void	preCreateSource() Do pre-operations before log Source creation if you need, like system properties setting before job submitting, for the log Source inside of HybridSource will be created during job running.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

KafkaLogSourceProvider

public KafkaLogSourceProvider(String topic,
                              Properties properties,
                              org.apache.flink.table.types.DataType physicalType,
                              int[] primaryKey,
                              @Nullable
                              org.apache.flink.api.common.serialization.DeserializationSchema<org.apache.flink.table.data.RowData> primaryKeyDeserializer,
                              org.apache.flink.api.common.serialization.DeserializationSchema<org.apache.flink.table.data.RowData> valueDeserializer,
                              @Nullable
                              int[][] projectFields,
                              CoreOptions.LogConsistency consistency,
                              CoreOptions.StartupMode scanMode,
                              @Nullable
                              Long timestampMills)

Method Detail

preCreateSource

public void preCreateSource()

Description copied from interface: LogSourceProvider

Do pre-operations before log Source creation if you need, like system properties setting before job submitting, for the log Source inside of HybridSource will be created during job running.

Specified by:

preCreateSource in interface LogSourceProvider

createSource

public org.apache.flink.connector.kafka.source.KafkaSource<org.apache.flink.table.data.RowData> createSource(@Nullable
                                                                                                             Map<Integer,Long> bucketOffsets)

Description copied from interface: LogSourceProvider

Creates a Source instance.

Specified by:

createSource in interface LogSourceProvider

Parameters:

bucketOffsets - optional, configure if you need to specify the startup offset.