org.apache.paimon.flink.source

Class DataTableSource

java.lang.Object

org.apache.paimon.flink.source.FlinkTableSource

org.apache.paimon.flink.source.DataTableSource

public class DataTableSource
extends FlinkTableSource

Table source to create StaticFileStoreSource or ContinuousFileStoreSource under batch mode or change-tracking is disabled. For streaming mode with change-tracking enabled and FULL scan mode, it will create a HybridSource of LogHybridSourceFactory.FlinkHybridFirstSource and kafka log source created by LogSourceProvider.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.paimon.flink.source.FlinkTableSource

FlinkTableSource.SplitStatistics

Field Summary

Fields inherited from class org.apache.paimon.flink.source.FlinkTableSource

FLINK_INFER_SCAN_PARALLELISM, limit, predicate, projectFields, splitStatistics, table

Constructor Summary

Constructors

Constructor and Description
DataTableSource(org.apache.flink.table.catalog.ObjectIdentifier tableIdentifier, Table table, boolean streaming, org.apache.flink.table.factories.DynamicTableFactory.Context context, LogStoreTableFactory logStoreTableFactory)
DataTableSource(org.apache.flink.table.catalog.ObjectIdentifier tableIdentifier, Table table, boolean streaming, org.apache.flink.table.factories.DynamicTableFactory.Context context, LogStoreTableFactory logStoreTableFactory, Predicate predicate, int[][] projectFields, Long limit, org.apache.flink.api.common.eventtime.WatermarkStrategy<org.apache.flink.table.data.RowData> watermarkStrategy, List<String> dynamicPartitionFilteringFields)

Method Summary

Modifier and Type	Method and Description
void	applyDynamicFiltering(List<String> candidateFilterFields)
String	asSummaryString()
DataTableSource	copy()
org.apache.flink.table.connector.ChangelogMode	getChangelogMode()
org.apache.flink.table.connector.source.LookupTableSource.LookupRuntimeProvider	getLookupRuntimeProvider(org.apache.flink.table.connector.source.LookupTableSource.LookupContext context)
org.apache.flink.table.connector.source.ScanTableSource.ScanRuntimeProvider	getScanRuntimeProvider(org.apache.flink.table.connector.source.ScanTableSource.ScanContext scanContext)
boolean	isStreaming()
List<String>	listAcceptedFilterFields()
void	pushWatermark(org.apache.flink.api.common.eventtime.WatermarkStrategy<org.apache.flink.table.data.RowData> watermarkStrategy)
org.apache.flink.table.plan.stats.TableStats	reportStatistics()

Methods inherited from class org.apache.paimon.flink.source.FlinkTableSource

getTable, inferSourceParallelism, pushFilters, pushLimit, pushProjection, scanSplitsForInference

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DataTableSource

public DataTableSource(org.apache.flink.table.catalog.ObjectIdentifier tableIdentifier,
                       Table table,
                       boolean streaming,
                       org.apache.flink.table.factories.DynamicTableFactory.Context context,
                       @Nullable
                       LogStoreTableFactory logStoreTableFactory)

DataTableSource

public DataTableSource(org.apache.flink.table.catalog.ObjectIdentifier tableIdentifier,
                       Table table,
                       boolean streaming,
                       org.apache.flink.table.factories.DynamicTableFactory.Context context,
                       @Nullable
                       LogStoreTableFactory logStoreTableFactory,
                       @Nullable
                       Predicate predicate,
                       @Nullable
                       int[][] projectFields,
                       @Nullable
                       Long limit,
                       @Nullable
                       org.apache.flink.api.common.eventtime.WatermarkStrategy<org.apache.flink.table.data.RowData> watermarkStrategy,
                       @Nullable
                       List<String> dynamicPartitionFilteringFields)

Method Detail

getChangelogMode

public org.apache.flink.table.connector.ChangelogMode getChangelogMode()

Specified by:

getChangelogMode in class FlinkTableSource

getScanRuntimeProvider

public org.apache.flink.table.connector.source.ScanTableSource.ScanRuntimeProvider getScanRuntimeProvider(org.apache.flink.table.connector.source.ScanTableSource.ScanContext scanContext)

Specified by:

getScanRuntimeProvider in class FlinkTableSource

copy

public DataTableSource copy()

Specified by:

copy in class FlinkTableSource

pushWatermark

public void pushWatermark(org.apache.flink.api.common.eventtime.WatermarkStrategy<org.apache.flink.table.data.RowData> watermarkStrategy)

Specified by:

pushWatermark in class FlinkTableSource

getLookupRuntimeProvider

public org.apache.flink.table.connector.source.LookupTableSource.LookupRuntimeProvider getLookupRuntimeProvider(org.apache.flink.table.connector.source.LookupTableSource.LookupContext context)

Specified by:

getLookupRuntimeProvider in class FlinkTableSource

reportStatistics

public org.apache.flink.table.plan.stats.TableStats reportStatistics()

Specified by:

reportStatistics in class FlinkTableSource

asSummaryString

public String asSummaryString()

Specified by:

asSummaryString in class FlinkTableSource

listAcceptedFilterFields

public List<String> listAcceptedFilterFields()

Specified by:

listAcceptedFilterFields in class FlinkTableSource

applyDynamicFiltering

public void applyDynamicFiltering(List<String> candidateFilterFields)

Specified by:

applyDynamicFiltering in class FlinkTableSource

isStreaming

public boolean isStreaming()

Specified by:

isStreaming in class FlinkTableSource