org.apache.paimon.append

Class BucketedAppendCompactManager

java.lang.Object

org.apache.paimon.compact.CompactFutureManager

org.apache.paimon.append.BucketedAppendCompactManager

All Implemented Interfaces:

Closeable, AutoCloseable, CompactManager

public class BucketedAppendCompactManager
extends CompactFutureManager

Compact manager for AppendOnlyFileStore.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BucketedAppendCompactManager.AutoCompactTask A CompactTask impl for append-only table auto-compaction.
static interface	BucketedAppendCompactManager.CompactRewriter Compact rewriter for append-only table.
static class	BucketedAppendCompactManager.FullCompactTask A CompactTask impl for full compaction of append-only table.

Field Summary

Fields inherited from class org.apache.paimon.compact.CompactFutureManager

taskFuture

Constructor Summary

Constructors

Constructor and Description
BucketedAppendCompactManager(ExecutorService executor, List<DataFileMeta> restored, DeletionVectorsMaintainer dvMaintainer, int minFileNum, int maxFileNum, long targetFileSize, BucketedAppendCompactManager.CompactRewriter rewriter, CompactionMetrics.Reporter metricsReporter)

Method Summary

Modifier and Type	Method and Description
void	addNewFile(DataFileMeta file) Add a new file.
List<DataFileMeta>	allFiles()
void	close()
static Comparator<DataFileMeta>	fileComparator(boolean ignoreOverlap) New files may be created during the compaction process, then the results of the compaction may be put after the new files, and this order will be disrupted.
Optional<CompactResult>	getCompactionResult(boolean blocking) Finish current task, and update result files to toCompact.
boolean	shouldWaitForLatestCompaction() Should wait compaction finish.
boolean	shouldWaitForPreparingCheckpoint()
void	triggerCompaction(boolean fullCompaction) Trigger a new compaction task.

Methods inherited from class org.apache.paimon.compact.CompactFutureManager

cancelCompaction, innerGetCompactionResult, isCompacting, obtainCompactResult

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BucketedAppendCompactManager

public BucketedAppendCompactManager(ExecutorService executor,
                                    List<DataFileMeta> restored,
                                    @Nullable
                                    DeletionVectorsMaintainer dvMaintainer,
                                    int minFileNum,
                                    int maxFileNum,
                                    long targetFileSize,
                                    BucketedAppendCompactManager.CompactRewriter rewriter,
                                    @Nullable
                                    CompactionMetrics.Reporter metricsReporter)

Method Detail

triggerCompaction

public void triggerCompaction(boolean fullCompaction)

Description copied from interface: CompactManager

Trigger a new compaction task.

Parameters:

fullCompaction - if caller needs a guaranteed full compaction

shouldWaitForLatestCompaction

public boolean shouldWaitForLatestCompaction()

Description copied from interface: CompactManager

Should wait compaction finish.

shouldWaitForPreparingCheckpoint

public boolean shouldWaitForPreparingCheckpoint()

addNewFile

public void addNewFile(DataFileMeta file)

Description copied from interface: CompactManager

Add a new file.

allFiles

public List<DataFileMeta> allFiles()

getCompactionResult

public Optional<CompactResult> getCompactionResult(boolean blocking)
                                            throws ExecutionException,
                                                   InterruptedException

Finish current task, and update result files to toCompact.

Throws:

ExecutionException

InterruptedException

close

public void close()
           throws IOException

Throws:

IOException

fileComparator

public static Comparator<DataFileMeta> fileComparator(boolean ignoreOverlap)

New files may be created during the compaction process, then the results of the compaction may be put after the new files, and this order will be disrupted. We need to ensure this order, so we force the order by sequence.