org.apache.paimon.flink.source.assigners

Class DynamicPartitionPruningAssigner

java.lang.Object

org.apache.paimon.flink.source.assigners.DynamicPartitionPruningAssigner

All Implemented Interfaces:

SplitAssigner

public class DynamicPartitionPruningAssigner
extends Object
implements SplitAssigner

Assigner to perform dynamic partition pruning by given DynamicFilteringData.

Constructor Summary

Constructors

Constructor and Description
DynamicPartitionPruningAssigner(SplitAssigner innerAssigner, Projection partitionRowProjection, org.apache.flink.table.connector.source.DynamicFilteringData dynamicFilteringData)

Method Summary

Modifier and Type	Method and Description
void	addSplit(int suggestedTask, FileStoreSourceSplit splits) Add one split of a specified subtask to the assigner.
void	addSplitsBack(int subtask, List<FileStoreSourceSplit> splits) Adds a set of splits to this assigner.
static SplitAssigner	createDynamicPartitionPruningAssignerIfNeeded(int subtaskId, SplitAssigner oriAssigner, Projection partitionRowProjection, org.apache.flink.api.connector.source.SourceEvent sourceEvent, org.slf4j.Logger logger)
List<FileStoreSourceSplit>	getNext(int subtask, String hostname) Gets the next split.
Optional<Long>	getNextSnapshotId(int subtask) Gets the snapshot id of the next split.
int	numberOfRemainingSplits() Gets the current number of remaining splits.
Collection<FileStoreSourceSplit>	remainingSplits() Gets the remaining splits that this assigner has pending.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DynamicPartitionPruningAssigner

public DynamicPartitionPruningAssigner(SplitAssigner innerAssigner,
                                       Projection partitionRowProjection,
                                       org.apache.flink.table.connector.source.DynamicFilteringData dynamicFilteringData)

Method Detail

getNext

public List<FileStoreSourceSplit> getNext(int subtask,
                                          @Nullable
                                          String hostname)

Description copied from interface: SplitAssigner

Gets the next split.

When this method returns an empty Optional, then the set of splits is assumed to be done and the source will finish once the readers finished their current splits.

Specified by:

getNext in interface SplitAssigner

addSplit

public void addSplit(int suggestedTask,
                     FileStoreSourceSplit splits)

Description copied from interface: SplitAssigner

Add one split of a specified subtask to the assigner.

Specified by:

addSplit in interface SplitAssigner

addSplitsBack

public void addSplitsBack(int subtask,
                          List<FileStoreSourceSplit> splits)

Description copied from interface: SplitAssigner

Adds a set of splits to this assigner. This happens for example when some split processing failed and the splits need to be re-added, or when new splits got discovered.

Specified by:

addSplitsBack in interface SplitAssigner

remainingSplits

public Collection<FileStoreSourceSplit> remainingSplits()

Description copied from interface: SplitAssigner

Gets the remaining splits that this assigner has pending.

Specified by:

remainingSplits in interface SplitAssigner

createDynamicPartitionPruningAssignerIfNeeded

public static SplitAssigner createDynamicPartitionPruningAssignerIfNeeded(int subtaskId,
                                                                          SplitAssigner oriAssigner,
                                                                          Projection partitionRowProjection,
                                                                          org.apache.flink.api.connector.source.SourceEvent sourceEvent,
                                                                          org.slf4j.Logger logger)

getNextSnapshotId

public Optional<Long> getNextSnapshotId(int subtask)

Description copied from interface: SplitAssigner

Gets the snapshot id of the next split.

Specified by:

getNextSnapshotId in interface SplitAssigner

numberOfRemainingSplits

public int numberOfRemainingSplits()

Description copied from interface: SplitAssigner

Gets the current number of remaining splits. This method should be guaranteed to be thread-safe.

Specified by:

numberOfRemainingSplits in interface SplitAssigner