org.apache.paimon.flink.source.align

Class AlignedSourceReader

java.lang.Object

org.apache.flink.connector.base.source.reader.SourceReaderBase<E,T,SplitT,SplitStateT>

org.apache.flink.connector.base.source.reader.SingleThreadMultiplexSourceReaderBase<org.apache.flink.connector.file.src.reader.BulkFormat.RecordIterator<org.apache.flink.table.data.RowData>,org.apache.flink.table.data.RowData,FileStoreSourceSplit,FileStoreSourceSplitState>

org.apache.paimon.flink.source.FileStoreSourceReader

org.apache.paimon.flink.source.align.AlignedSourceReader

All Implemented Interfaces:

AutoCloseable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.api.connector.source.ExternallyInducedSourceReader<org.apache.flink.table.data.RowData,FileStoreSourceSplit>, org.apache.flink.api.connector.source.SourceReader<org.apache.flink.table.data.RowData,FileStoreSourceSplit>

public class AlignedSourceReader
extends FileStoreSourceReader
implements org.apache.flink.api.connector.source.ExternallyInducedSourceReader<org.apache.flink.table.data.RowData,FileStoreSourceSplit>

The difference with FileStoreSourceReader is that only after the allocated splits are fully consumed, checkpoints will be made and the next batch of splits will be requested.

Field Summary

Fields inherited from class org.apache.flink.connector.base.source.reader.SourceReaderBase

config, context, options, recordEmitter, splitFetcherManager

Constructor Summary

Constructors

Constructor and Description

AlignedSourceReader(org.apache.flink.api.connector.source.SourceReaderContext readerContext, TableRead tableRead, FileStoreSourceReaderMetrics metrics, IOManager ioManager, Long limit, org.apache.flink.connector.base.source.reader.synchronization.FutureCompletingBlockingQueue<org.apache.flink.connector.base.source.reader.RecordsWithSplitIds<org.apache.flink.connector.file.src.reader.BulkFormat.RecordIterator<org.apache.flink.table.data.RowData>>> elementsQueue)

Method Summary

Modifier and Type	Method and Description
void	handleSourceEvents(org.apache.flink.api.connector.source.SourceEvent sourceEvent)
protected void	onSplitFinished(Map<String,FileStoreSourceSplitState> finishedSplitIds)
Optional<Long>	shouldTriggerCheckpoint()

Methods inherited from class org.apache.paimon.flink.source.FileStoreSourceReader

close, initializedState, start, toSplitType

Methods inherited from class org.apache.flink.connector.base.source.reader.SourceReaderBase

addSplits, getNumberOfCurrentlyAssignedSplits, isAvailable, notifyNoMoreSplits, pauseOrResumeSplits, pollNext, snapshotState

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.api.connector.source.SourceReader

addSplits, isAvailable, notifyCheckpointComplete, notifyNoMoreSplits, pauseOrResumeSplits, pollNext, snapshotState, start

Methods inherited from interface java.lang.AutoCloseable

close

Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointAborted

Constructor Detail

AlignedSourceReader

public AlignedSourceReader(org.apache.flink.api.connector.source.SourceReaderContext readerContext,
                           TableRead tableRead,
                           FileStoreSourceReaderMetrics metrics,
                           IOManager ioManager,
                           @Nullable
                           Long limit,
                           org.apache.flink.connector.base.source.reader.synchronization.FutureCompletingBlockingQueue<org.apache.flink.connector.base.source.reader.RecordsWithSplitIds<org.apache.flink.connector.file.src.reader.BulkFormat.RecordIterator<org.apache.flink.table.data.RowData>>> elementsQueue)

Method Detail

handleSourceEvents

public void handleSourceEvents(org.apache.flink.api.connector.source.SourceEvent sourceEvent)

Specified by:

handleSourceEvents in interface org.apache.flink.api.connector.source.SourceReader<org.apache.flink.table.data.RowData,FileStoreSourceSplit>

Overrides:

handleSourceEvents in class org.apache.flink.connector.base.source.reader.SourceReaderBase<org.apache.flink.connector.file.src.reader.BulkFormat.RecordIterator<org.apache.flink.table.data.RowData>,org.apache.flink.table.data.RowData,FileStoreSourceSplit,FileStoreSourceSplitState>

onSplitFinished

protected void onSplitFinished(Map<String,FileStoreSourceSplitState> finishedSplitIds)

Overrides:

onSplitFinished in class FileStoreSourceReader

shouldTriggerCheckpoint

public Optional<Long> shouldTriggerCheckpoint()

Specified by:

shouldTriggerCheckpoint in interface org.apache.flink.api.connector.source.ExternallyInducedSourceReader<org.apache.flink.table.data.RowData,FileStoreSourceSplit>