org.apache.paimon.operation

Class MergeFileSplitRead

java.lang.Object

org.apache.paimon.operation.MergeFileSplitRead

All Implemented Interfaces:

SplitRead<KeyValue>

public class MergeFileSplitRead
extends Object
implements SplitRead<KeyValue>

An implementation for KeyValueFileStore, this class handle LSM merging and changelog row kind things, it will force reading fields such as sequence and row_kind.

See Also:

If in batch mode and reading raw files, it is recommended to use this read.

Constructor Summary

Constructors

Constructor and Description
MergeFileSplitRead(CoreOptions options, TableSchema schema, RowType keyType, RowType valueType, Comparator<InternalRow> keyComparator, MergeFunctionFactory<KeyValue> mfFactory, KeyValueFileReaderFactory.Builder readerFactoryBuilder)

Method Summary

Modifier and Type	Method and Description
RecordReader<KeyValue>	createMergeReader(BinaryRow partition, int bucket, List<DataFileMeta> files, List<DeletionFile> deletionFiles, boolean keepDelete)
RecordReader<KeyValue>	createNoMergeReader(BinaryRow partition, int bucket, List<DataFileMeta> files, List<DeletionFile> deletionFiles, boolean onlyFilterKey)
RecordReader<KeyValue>	createReader(DataSplit split) Create a RecordReader from split.
UserDefinedSeqComparator	createUdsComparator()
MergeFileSplitRead	forceKeepDelete()
Comparator<InternalRow>	keyComparator()
MergeSorter	mergeSorter()
MergeFileSplitRead	withFilter(Predicate predicate)
MergeFileSplitRead	withIOManager(IOManager ioManager)
MergeFileSplitRead	withKeyProjection(int[][] projectedFields)
MergeFileSplitRead	withProjection(int[][] projectedFields)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.paimon.operation.SplitRead

convert

Constructor Detail

MergeFileSplitRead

public MergeFileSplitRead(CoreOptions options,
                          TableSchema schema,
                          RowType keyType,
                          RowType valueType,
                          Comparator<InternalRow> keyComparator,
                          MergeFunctionFactory<KeyValue> mfFactory,
                          KeyValueFileReaderFactory.Builder readerFactoryBuilder)

Method Detail

withKeyProjection

public MergeFileSplitRead withKeyProjection(@Nullable
                                            int[][] projectedFields)

keyComparator

public Comparator<InternalRow> keyComparator()

mergeSorter

public MergeSorter mergeSorter()

withProjection

public MergeFileSplitRead withProjection(@Nullable
                                         int[][] projectedFields)

Specified by:

withProjection in interface SplitRead<KeyValue>

withIOManager

public MergeFileSplitRead withIOManager(IOManager ioManager)

Specified by:

withIOManager in interface SplitRead<KeyValue>

forceKeepDelete

public MergeFileSplitRead forceKeepDelete()

Specified by:

forceKeepDelete in interface SplitRead<KeyValue>

withFilter

public MergeFileSplitRead withFilter(Predicate predicate)

Specified by:

withFilter in interface SplitRead<KeyValue>

createReader

public RecordReader<KeyValue> createReader(DataSplit split)
                                    throws IOException

Description copied from interface: SplitRead

Create a RecordReader from split.

Specified by:

createReader in interface SplitRead<KeyValue>

Throws:

IOException

createMergeReader

public RecordReader<KeyValue> createMergeReader(BinaryRow partition,
                                                int bucket,
                                                List<DataFileMeta> files,
                                                @Nullable
                                                List<DeletionFile> deletionFiles,
                                                boolean keepDelete)
                                         throws IOException

Throws:

IOException

createNoMergeReader

public RecordReader<KeyValue> createNoMergeReader(BinaryRow partition,
                                                  int bucket,
                                                  List<DataFileMeta> files,
                                                  @Nullable
                                                  List<DeletionFile> deletionFiles,
                                                  boolean onlyFilterKey)
                                           throws IOException

Throws:

IOException

createUdsComparator

@Nullable
public UserDefinedSeqComparator createUdsComparator()