public class UnawareAppendTableCompactionCoordinator extends Object
Note: UnawareAppendTableCompactionCoordinator
scan files in snapshot, read APPEND and
COMPACT snapshot then load those new files. It will try it best to generate compaction task for
the restored files scanned in snapshot, but to reduce memory usage, it won't remain single file
for a long time. After ten times scan, single file with one partition will be ignored and removed
from memory, which means, it will not participate in compaction again until restart the
compaction job.
When a third task delete file in latest snapshot(including batch delete/update and overwrite), the file in coordinator will still remain and participate in compaction task. When this happens, compaction job will fail in commit stage, and fail-over to rescan the restored files in latest snapshot.
Modifier and Type | Field and Description |
---|---|
protected static int |
COMPACT_AGE |
protected static int |
REMOVE_AGE |
Constructor and Description |
---|
UnawareAppendTableCompactionCoordinator(FileStoreTable table) |
UnawareAppendTableCompactionCoordinator(FileStoreTable table,
boolean isStreaming) |
UnawareAppendTableCompactionCoordinator(FileStoreTable table,
boolean isStreaming,
Predicate filter) |
protected static final int REMOVE_AGE
protected static final int COMPACT_AGE
public UnawareAppendTableCompactionCoordinator(FileStoreTable table)
public UnawareAppendTableCompactionCoordinator(FileStoreTable table, boolean isStreaming)
public UnawareAppendTableCompactionCoordinator(FileStoreTable table, boolean isStreaming, @Nullable Predicate filter)
public List<UnawareAppendCompactionTask> run()
Copyright © 2023–2024 The Apache Software Foundation. All rights reserved.