org.apache.paimon.flink.source.assigners

Class FIFOSplitAssigner

java.lang.Object

org.apache.paimon.flink.source.assigners.FIFOSplitAssigner

All Implemented Interfaces:

SplitAssigner

public class FIFOSplitAssigner
extends Object
implements SplitAssigner

Splits are assigned preemptively in the order requested by the task. Only one split is assigned to the task at a time. This is unaware of subtask, it does not care about the relationship between splits and subtask, so only one queue is created and splits are fetched in order.

Constructor Summary

Constructors

Constructor and Description
FIFOSplitAssigner(Collection<FileStoreSourceSplit> splits)

Method Summary

Modifier and Type	Method and Description
void	addSplit(int suggestedTask, FileStoreSourceSplit split) Add one split of a specified subtask to the assigner.
void	addSplitsBack(int subtask, List<FileStoreSourceSplit> splits) Adds a set of splits to this assigner.
List<FileStoreSourceSplit>	getNext(int subtask, String hostname) Gets the next split.
Optional<Long>	getNextSnapshotId(int subtask) Gets the snapshot id of the next split.
int	numberOfRemainingSplits() Gets the current number of remaining splits.
Collection<FileStoreSourceSplit>	remainingSplits() Gets the remaining splits that this assigner has pending.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FIFOSplitAssigner

public FIFOSplitAssigner(Collection<FileStoreSourceSplit> splits)

Method Detail

getNext

public List<FileStoreSourceSplit> getNext(int subtask,
                                          @Nullable
                                          String hostname)

Description copied from interface: SplitAssigner

Gets the next split.

When this method returns an empty Optional, then the set of splits is assumed to be done and the source will finish once the readers finished their current splits.

Specified by:

getNext in interface SplitAssigner

addSplit

public void addSplit(int suggestedTask,
                     FileStoreSourceSplit split)

Description copied from interface: SplitAssigner

Add one split of a specified subtask to the assigner.

Specified by:

addSplit in interface SplitAssigner

addSplitsBack

public void addSplitsBack(int subtask,
                          List<FileStoreSourceSplit> splits)

Description copied from interface: SplitAssigner

Adds a set of splits to this assigner. This happens for example when some split processing failed and the splits need to be re-added, or when new splits got discovered.

Specified by:

addSplitsBack in interface SplitAssigner

remainingSplits

public Collection<FileStoreSourceSplit> remainingSplits()

Description copied from interface: SplitAssigner

Gets the remaining splits that this assigner has pending.

Specified by:

remainingSplits in interface SplitAssigner

getNextSnapshotId

public Optional<Long> getNextSnapshotId(int subtask)

Description copied from interface: SplitAssigner

Gets the snapshot id of the next split.

Specified by:

getNextSnapshotId in interface SplitAssigner

numberOfRemainingSplits

public int numberOfRemainingSplits()

Description copied from interface: SplitAssigner

Gets the current number of remaining splits. This method should be guaranteed to be thread-safe.

Specified by:

numberOfRemainingSplits in interface SplitAssigner