org.apache.paimon.table.source

Class DataTableStreamScan

java.lang.Object

org.apache.paimon.table.source.AbstractDataTableScan

org.apache.paimon.table.source.DataTableStreamScan

All Implemented Interfaces:

DataTableScan, InnerTableScan, StreamDataTableScan, StreamTableScan, TableScan, Restorable<Long>

Direct Known Subclasses:

LookupDataTableScan

public class DataTableStreamScan
extends AbstractDataTableScan
implements StreamDataTableScan

StreamTableScan implementation for streaming planning.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.paimon.table.source.TableScan

TableScan.Plan

Field Summary

Fields inherited from class org.apache.paimon.table.source.AbstractDataTableScan

snapshotReader

Constructor Summary

Constructors

Constructor and Description
DataTableStreamScan(CoreOptions options, SnapshotReader snapshotReader, SnapshotManager snapshotManager, boolean supportStreamingReadOverwrite, DefaultValueAssigner defaultValueAssigner)

Method Summary

Modifier and Type	Method and Description
Long	checkpoint() Checkpoint to return next snapshot id.
protected BoundedChecker	createBoundedChecker()
protected FollowUpScanner	createFollowUpScanner()
protected SnapshotReader.Plan	handleOverwriteSnapshot(Snapshot snapshot)
List<PartitionEntry>	listPartitionEntries()
void	notifyCheckpointComplete(Long nextSnapshot) Notifies the checkpoint complete with next snapshot id.
TableScan.Plan	plan() Plan splits, throws EndOfScanException if the scan is ended.
void	restore(Long nextSnapshotId) Restore from checkpoint next snapshot id.
void	restore(Long nextSnapshotId, boolean scanAllSnapshot) Restore from checkpoint next snapshot id with scan kind.
StartingContext	startingContext()
Long	watermark() Current watermark for consumed snapshot.
DataTableStreamScan	withFilter(Predicate predicate)
DataTableScan	withShard(int indexOfThisSubtask, int numberOfParallelSubtasks) Specify the shard to be read, and allocate sharded files to read records.

Methods inherited from class org.apache.paimon.table.source.AbstractDataTableScan

createStartingScanner, dropStats, options, withBucket, withBucketFilter, withLevelFilter, withMetricsRegistry, withPartitionFilter, withPartitionFilter

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.paimon.table.source.InnerTableScan

dropStats, withBucketFilter, withLevelFilter, withLimit, withMetricsRegistry, withPartitionFilter, withPartitionFilter

Methods inherited from interface org.apache.paimon.table.source.TableScan

listPartitions

Constructor Detail

DataTableStreamScan

public DataTableStreamScan(CoreOptions options,
                           SnapshotReader snapshotReader,
                           SnapshotManager snapshotManager,
                           boolean supportStreamingReadOverwrite,
                           DefaultValueAssigner defaultValueAssigner)

Method Detail

withFilter

public DataTableStreamScan withFilter(Predicate predicate)

Specified by:

withFilter in interface InnerTableScan

startingContext

public StartingContext startingContext()

Specified by:

startingContext in interface StreamDataTableScan

plan

public TableScan.Plan plan()

Description copied from interface: TableScan

Plan splits, throws EndOfScanException if the scan is ended.

Specified by:

plan in interface TableScan

listPartitionEntries

public List<PartitionEntry> listPartitionEntries()

Specified by:

listPartitionEntries in interface TableScan

handleOverwriteSnapshot

@Nullable
protected SnapshotReader.Plan handleOverwriteSnapshot(Snapshot snapshot)

createFollowUpScanner

protected FollowUpScanner createFollowUpScanner()

createBoundedChecker

protected BoundedChecker createBoundedChecker()

checkpoint

@Nullable
public Long checkpoint()

Description copied from interface: StreamTableScan

Checkpoint to return next snapshot id.

Specified by:

checkpoint in interface StreamTableScan

Specified by:

checkpoint in interface Restorable<Long>

watermark

@Nullable
public Long watermark()

Description copied from interface: StreamTableScan

Current watermark for consumed snapshot.

Specified by:

watermark in interface StreamTableScan

restore

public void restore(@Nullable
                    Long nextSnapshotId)

Description copied from interface: StreamTableScan

Restore from checkpoint next snapshot id.

Specified by:

restore in interface StreamTableScan

Specified by:

restore in interface Restorable<Long>

restore

public void restore(@Nullable
                    Long nextSnapshotId,
                    boolean scanAllSnapshot)

Description copied from interface: StreamDataTableScan

Restore from checkpoint next snapshot id with scan kind.

Specified by:

restore in interface StreamDataTableScan

notifyCheckpointComplete

public void notifyCheckpointComplete(@Nullable
                                     Long nextSnapshot)

Description copied from interface: StreamTableScan

Notifies the checkpoint complete with next snapshot id.

Specified by:

notifyCheckpointComplete in interface StreamTableScan

withShard

public DataTableScan withShard(int indexOfThisSubtask,
                               int numberOfParallelSubtasks)

Description copied from interface: DataTableScan

Specify the shard to be read, and allocate sharded files to read records.

Specified by:

withShard in interface DataTableScan