org.apache.paimon.flink.source

Class BaseDataTableSource

java.lang.Object

org.apache.paimon.flink.source.FlinkTableSource

org.apache.paimon.flink.source.BaseDataTableSource

All Implemented Interfaces:

org.apache.flink.table.connector.source.abilities.SupportsAggregatePushDown, org.apache.flink.table.connector.source.abilities.SupportsFilterPushDown, org.apache.flink.table.connector.source.abilities.SupportsLimitPushDown, org.apache.flink.table.connector.source.abilities.SupportsProjectionPushDown, org.apache.flink.table.connector.source.abilities.SupportsWatermarkPushDown, org.apache.flink.table.connector.source.DynamicTableSource, org.apache.flink.table.connector.source.LookupTableSource, org.apache.flink.table.connector.source.ScanTableSource

Direct Known Subclasses:

DataTableSource

public abstract class BaseDataTableSource
extends FlinkTableSource
implements org.apache.flink.table.connector.source.LookupTableSource, org.apache.flink.table.connector.source.abilities.SupportsWatermarkPushDown, org.apache.flink.table.connector.source.abilities.SupportsAggregatePushDown

Table source to create StaticFileStoreSource or ContinuousFileStoreSource under batch mode or streaming mode.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.paimon.flink.source.FlinkTableSource

FlinkTableSource.SplitStatistics

Nested classes/interfaces inherited from interface org.apache.flink.table.connector.source.LookupTableSource

org.apache.flink.table.connector.source.LookupTableSource.LookupContext, org.apache.flink.table.connector.source.LookupTableSource.LookupRuntimeProvider

Nested classes/interfaces inherited from interface org.apache.flink.table.connector.source.DynamicTableSource

org.apache.flink.table.connector.source.DynamicTableSource.Context, org.apache.flink.table.connector.source.DynamicTableSource.DataStructureConverter

Nested classes/interfaces inherited from interface org.apache.flink.table.connector.source.ScanTableSource

org.apache.flink.table.connector.source.ScanTableSource.ScanContext, org.apache.flink.table.connector.source.ScanTableSource.ScanRuntimeProvider

Nested classes/interfaces inherited from interface org.apache.flink.table.connector.source.abilities.SupportsFilterPushDown

org.apache.flink.table.connector.source.abilities.SupportsFilterPushDown.Result

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.flink.table.factories.DynamicTableFactory.Context	context
protected Long	countPushed
protected LogStoreTableFactory	logStoreTableFactory
protected boolean	streaming
protected org.apache.flink.table.catalog.ObjectIdentifier	tableIdentifier
protected org.apache.flink.api.common.eventtime.WatermarkStrategy<org.apache.flink.table.data.RowData>	watermarkStrategy

Fields inherited from class org.apache.paimon.flink.source.FlinkTableSource

FLINK_INFER_SCAN_PARALLELISM, limit, predicate, projectFields, splitStatistics, table

Constructor Summary

Constructors

Constructor and Description

BaseDataTableSource(org.apache.flink.table.catalog.ObjectIdentifier tableIdentifier, Table table, boolean streaming, org.apache.flink.table.factories.DynamicTableFactory.Context context, LogStoreTableFactory logStoreTableFactory, Predicate predicate, int[][] projectFields, Long limit, org.apache.flink.api.common.eventtime.WatermarkStrategy<org.apache.flink.table.data.RowData> watermarkStrategy, Long countPushed)

Method Summary

Modifier and Type	Method and Description
boolean	applyAggregates(List<int[]> groupingSets, List<org.apache.flink.table.expressions.AggregateExpression> aggregateExpressions, org.apache.flink.table.types.DataType producedDataType)
void	applyWatermark(org.apache.flink.api.common.eventtime.WatermarkStrategy<org.apache.flink.table.data.RowData> watermarkStrategy)
String	asSummaryString()
protected abstract List<String>	dynamicPartitionFilteringFields()
org.apache.flink.table.connector.ChangelogMode	getChangelogMode()
protected FileStoreLookupFunction	getFileStoreLookupFunction(org.apache.flink.table.connector.source.LookupTableSource.LookupContext context, Table table, int[] projection, int[] joinKey)
org.apache.flink.table.connector.source.LookupTableSource.LookupRuntimeProvider	getLookupRuntimeProvider(org.apache.flink.table.connector.source.LookupTableSource.LookupContext context)
org.apache.flink.table.connector.source.ScanTableSource.ScanRuntimeProvider	getScanRuntimeProvider(org.apache.flink.table.connector.source.ScanTableSource.ScanContext scanContext)
boolean	isStreaming()

Methods inherited from class org.apache.paimon.flink.source.FlinkTableSource

applyFilters, applyLimit, applyProjection, getTable, inferSourceParallelism, scanSplitsForInference, supportsNestedProjection

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.table.connector.source.DynamicTableSource

copy

Methods inherited from interface org.apache.flink.table.connector.source.abilities.SupportsProjectionPushDown

applyProjection

Field Detail

tableIdentifier

protected final org.apache.flink.table.catalog.ObjectIdentifier tableIdentifier

streaming

protected final boolean streaming

context

protected final org.apache.flink.table.factories.DynamicTableFactory.Context context

logStoreTableFactory

@Nullable
protected final LogStoreTableFactory logStoreTableFactory

watermarkStrategy

@Nullable
protected org.apache.flink.api.common.eventtime.WatermarkStrategy<org.apache.flink.table.data.RowData> watermarkStrategy

countPushed

@Nullable
protected Long countPushed

Constructor Detail

BaseDataTableSource

public BaseDataTableSource(org.apache.flink.table.catalog.ObjectIdentifier tableIdentifier,
                           Table table,
                           boolean streaming,
                           org.apache.flink.table.factories.DynamicTableFactory.Context context,
                           @Nullable
                           LogStoreTableFactory logStoreTableFactory,
                           @Nullable
                           Predicate predicate,
                           @Nullable
                           int[][] projectFields,
                           @Nullable
                           Long limit,
                           @Nullable
                           org.apache.flink.api.common.eventtime.WatermarkStrategy<org.apache.flink.table.data.RowData> watermarkStrategy,
                           @Nullable
                           Long countPushed)

Method Detail

getChangelogMode

public org.apache.flink.table.connector.ChangelogMode getChangelogMode()

Specified by:

getChangelogMode in interface org.apache.flink.table.connector.source.ScanTableSource

getScanRuntimeProvider

public org.apache.flink.table.connector.source.ScanTableSource.ScanRuntimeProvider getScanRuntimeProvider(org.apache.flink.table.connector.source.ScanTableSource.ScanContext scanContext)

Specified by:

getScanRuntimeProvider in interface org.apache.flink.table.connector.source.ScanTableSource

dynamicPartitionFilteringFields

protected abstract List<String> dynamicPartitionFilteringFields()

applyWatermark

public void applyWatermark(org.apache.flink.api.common.eventtime.WatermarkStrategy<org.apache.flink.table.data.RowData> watermarkStrategy)

Specified by:

applyWatermark in interface org.apache.flink.table.connector.source.abilities.SupportsWatermarkPushDown

getLookupRuntimeProvider

public org.apache.flink.table.connector.source.LookupTableSource.LookupRuntimeProvider getLookupRuntimeProvider(org.apache.flink.table.connector.source.LookupTableSource.LookupContext context)

Specified by:

getLookupRuntimeProvider in interface org.apache.flink.table.connector.source.LookupTableSource

getFileStoreLookupFunction

protected FileStoreLookupFunction getFileStoreLookupFunction(org.apache.flink.table.connector.source.LookupTableSource.LookupContext context,
                                                             Table table,
                                                             int[] projection,
                                                             int[] joinKey)

applyAggregates

public boolean applyAggregates(List<int[]> groupingSets,
                               List<org.apache.flink.table.expressions.AggregateExpression> aggregateExpressions,
                               org.apache.flink.table.types.DataType producedDataType)

Specified by:

applyAggregates in interface org.apache.flink.table.connector.source.abilities.SupportsAggregatePushDown

asSummaryString

public String asSummaryString()

Specified by:

asSummaryString in interface org.apache.flink.table.connector.source.DynamicTableSource

isStreaming

public boolean isStreaming()

Specified by:

isStreaming in class FlinkTableSource