public abstract class ChangelogMergeTreeRewriter extends MergeTreeCompactRewriter
MergeTreeCompactRewriter
which produces changelog files while performing compaction.Modifier and Type | Class and Description |
---|---|
protected static class |
ChangelogMergeTreeRewriter.UpgradeStrategy
Strategy for upgrade.
|
Modifier and Type | Field and Description |
---|---|
protected int |
maxLevel |
protected CoreOptions.MergeEngine |
mergeEngine |
keyComparator, mergeSorter, mfFactory, readerFactory, userDefinedSeqComparator, writerFactory
Constructor and Description |
---|
ChangelogMergeTreeRewriter(int maxLevel,
CoreOptions.MergeEngine mergeEngine,
FileReaderFactory<KeyValue> readerFactory,
KeyValueFileWriterFactory writerFactory,
Comparator<InternalRow> keyComparator,
FieldsComparator userDefinedSeqComparator,
MergeFunctionFactory<KeyValue> mfFactory,
MergeSorter mergeSorter,
boolean produceChangelog,
boolean forceDropDelete) |
Modifier and Type | Method and Description |
---|---|
protected abstract MergeFunctionWrapper<ChangelogResult> |
createMergeWrapper(int outputLevel) |
CompactResult |
rewrite(int outputLevel,
boolean dropDelete,
List<List<SortedRun>> sections)
Rewrite sections to new level.
|
protected abstract boolean |
rewriteChangelog(int outputLevel,
boolean dropDelete,
List<List<SortedRun>> sections) |
protected boolean |
rewriteLookupChangelog(int outputLevel,
List<List<SortedRun>> sections) |
CompactResult |
upgrade(int outputLevel,
DataFileMeta file)
Upgrade file to new level, usually file data is not rewritten, only the metadata is updated.
|
protected abstract ChangelogMergeTreeRewriter.UpgradeStrategy |
upgradeStrategy(int outputLevel,
DataFileMeta file) |
notifyRewriteCompactBefore, readerForMergeTree, rewriteCompaction
close, extractFilesFromSections
protected final int maxLevel
protected final CoreOptions.MergeEngine mergeEngine
public ChangelogMergeTreeRewriter(int maxLevel, CoreOptions.MergeEngine mergeEngine, FileReaderFactory<KeyValue> readerFactory, KeyValueFileWriterFactory writerFactory, Comparator<InternalRow> keyComparator, @Nullable FieldsComparator userDefinedSeqComparator, MergeFunctionFactory<KeyValue> mfFactory, MergeSorter mergeSorter, boolean produceChangelog, boolean forceDropDelete)
protected abstract boolean rewriteChangelog(int outputLevel, boolean dropDelete, List<List<SortedRun>> sections)
protected abstract ChangelogMergeTreeRewriter.UpgradeStrategy upgradeStrategy(int outputLevel, DataFileMeta file)
protected abstract MergeFunctionWrapper<ChangelogResult> createMergeWrapper(int outputLevel)
protected boolean rewriteLookupChangelog(int outputLevel, List<List<SortedRun>> sections)
public CompactResult rewrite(int outputLevel, boolean dropDelete, List<List<SortedRun>> sections) throws Exception
CompactRewriter
rewrite
in interface CompactRewriter
rewrite
in class MergeTreeCompactRewriter
outputLevel
- new leveldropDelete
- whether to drop the deletion, see MergeTreeCompactManager.triggerCompaction(boolean)
sections
- list of sections (section is a list of SortedRun
s, and key intervals
between sections do not overlap)Exception
- exceptionpublic CompactResult upgrade(int outputLevel, DataFileMeta file) throws Exception
CompactRewriter
ChangelogMergeTreeRewriter
upgrade
in interface CompactRewriter
upgrade
in class AbstractCompactRewriter
outputLevel
- new levelfile
- file to be updatedException
- exceptionCopyright © 2023–2024 The Apache Software Foundation. All rights reserved.