org.apache.paimon.mergetree.compact

Class MergeTreeCompactRewriter

java.lang.Object

org.apache.paimon.mergetree.compact.AbstractCompactRewriter

org.apache.paimon.mergetree.compact.MergeTreeCompactRewriter

All Implemented Interfaces:

Closeable, AutoCloseable, CompactRewriter

Direct Known Subclasses:

ChangelogMergeTreeRewriter

public class MergeTreeCompactRewriter
extends AbstractCompactRewriter

Default CompactRewriter for merge trees.

Field Summary

Fields

Modifier and Type	Field and Description
protected Comparator<InternalRow>	keyComparator
protected MergeSorter	mergeSorter
protected MergeFunctionFactory<KeyValue>	mfFactory
protected FileReaderFactory<KeyValue>	readerFactory
protected FieldsComparator	userDefinedSeqComparator
protected KeyValueFileWriterFactory	writerFactory

Constructor Summary

Constructors

Constructor and Description
MergeTreeCompactRewriter(FileReaderFactory<KeyValue> readerFactory, KeyValueFileWriterFactory writerFactory, Comparator<InternalRow> keyComparator, FieldsComparator userDefinedSeqComparator, MergeFunctionFactory<KeyValue> mfFactory, MergeSorter mergeSorter)

Method Summary

Modifier and Type	Method and Description
protected void	notifyRewriteCompactBefore(List<DataFileMeta> files)
protected <T> RecordReader<T>	readerForMergeTree(List<List<SortedRun>> sections, MergeFunctionWrapper<T> mergeFunctionWrapper)
CompactResult	rewrite(int outputLevel, boolean dropDelete, List<List<SortedRun>> sections) Rewrite sections to new level.
protected CompactResult	rewriteCompaction(int outputLevel, boolean dropDelete, List<List<SortedRun>> sections)

Methods inherited from class org.apache.paimon.mergetree.compact.AbstractCompactRewriter

close, extractFilesFromSections, upgrade

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

readerFactory

protected final FileReaderFactory<KeyValue> readerFactory

writerFactory

protected final KeyValueFileWriterFactory writerFactory

keyComparator

protected final Comparator<InternalRow> keyComparator

userDefinedSeqComparator

@Nullable
protected final FieldsComparator userDefinedSeqComparator

mfFactory

protected final MergeFunctionFactory<KeyValue> mfFactory

mergeSorter

protected final MergeSorter mergeSorter

Constructor Detail

MergeTreeCompactRewriter

public MergeTreeCompactRewriter(FileReaderFactory<KeyValue> readerFactory,
                                KeyValueFileWriterFactory writerFactory,
                                Comparator<InternalRow> keyComparator,
                                @Nullable
                                FieldsComparator userDefinedSeqComparator,
                                MergeFunctionFactory<KeyValue> mfFactory,
                                MergeSorter mergeSorter)

Method Detail

rewrite

public CompactResult rewrite(int outputLevel,
                             boolean dropDelete,
                             List<List<SortedRun>> sections)
                      throws Exception

Description copied from interface: CompactRewriter

Rewrite sections to new level.

Parameters:

outputLevel - new level

dropDelete - whether to drop the deletion, see MergeTreeCompactManager.triggerCompaction(boolean)

sections - list of sections (section is a list of SortedRuns, and key intervals between sections do not overlap)

Returns:

compaction result

Throws:

Exception - exception

rewriteCompaction

protected CompactResult rewriteCompaction(int outputLevel,
                                          boolean dropDelete,
                                          List<List<SortedRun>> sections)
                                   throws Exception

Throws:

Exception

readerForMergeTree

protected <T> RecordReader<T> readerForMergeTree(List<List<SortedRun>> sections,
                                                 MergeFunctionWrapper<T> mergeFunctionWrapper)
                                          throws IOException

Throws:

IOException

notifyRewriteCompactBefore

protected void notifyRewriteCompactBefore(List<DataFileMeta> files)