org.apache.paimon.operation

Class LocalOrphanFilesClean

java.lang.Object

org.apache.paimon.operation.OrphanFilesClean

org.apache.paimon.operation.LocalOrphanFilesClean

All Implemented Interfaces:

Serializable

public class LocalOrphanFilesClean
extends OrphanFilesClean

Local OrphanFilesClean, it will use thread pool to execute deletion.

Note that, this class will be used when the orphan clean mode is local, else orphan clean will use distributed one. See `FlinkOrphanFilesClean` and `SparkOrphanFilesClean`.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.paimon.operation.OrphanFilesClean

OrphanFilesClean.ReaderWithIOException<T>

Field Summary

Fields inherited from class org.apache.paimon.operation.OrphanFilesClean

fileCleaner, fileIO, location, LOG, olderThanMillis, partitionKeysNum, READ_FILE_RETRY_INTERVAL, READ_FILE_RETRY_NUM, table

Constructor Summary

Constructors

Constructor and Description
LocalOrphanFilesClean(FileStoreTable table)
LocalOrphanFilesClean(FileStoreTable table, long olderThanMillis)
LocalOrphanFilesClean(FileStoreTable table, long olderThanMillis, SerializableConsumer<Path> fileCleaner)

Method Summary

Modifier and Type	Method and Description
List<Path>	clean()
static List<LocalOrphanFilesClean>	createOrphanFilesCleans(Catalog catalog, String databaseName, String tableName, long olderThanMillis, SerializableConsumer<Path> fileCleaner, Integer parallelism)
static long	executeDatabaseOrphanFiles(Catalog catalog, String databaseName, String tableName, long olderThanMillis, SerializableConsumer<Path> fileCleaner, Integer parallelism)

Methods inherited from class org.apache.paimon.operation.OrphanFilesClean

cleanSnapshotDir, collectWithoutDataFile, collectWithoutDataFileWithManifestFlag, createFileCleaner, listPaimonFileDirs, oldEnough, olderThanMillis, retryReadingFiles, safelyGetAllSnapshots, tryBestListingDirs, validBranches

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LocalOrphanFilesClean

public LocalOrphanFilesClean(FileStoreTable table)

LocalOrphanFilesClean

public LocalOrphanFilesClean(FileStoreTable table,
                             long olderThanMillis)

LocalOrphanFilesClean

public LocalOrphanFilesClean(FileStoreTable table,
                             long olderThanMillis,
                             SerializableConsumer<Path> fileCleaner)

Method Detail

clean

public List<Path> clean()
                 throws IOException,
                        ExecutionException,
                        InterruptedException

Throws:

IOException

ExecutionException

InterruptedException

createOrphanFilesCleans

public static List<LocalOrphanFilesClean> createOrphanFilesCleans(Catalog catalog,
                                                                  String databaseName,
                                                                  @Nullable
                                                                  String tableName,
                                                                  long olderThanMillis,
                                                                  SerializableConsumer<Path> fileCleaner,
                                                                  @Nullable
                                                                  Integer parallelism)
                                                           throws Catalog.DatabaseNotExistException,
                                                                  Catalog.TableNotExistException

Throws:

Catalog.DatabaseNotExistException

Catalog.TableNotExistException

executeDatabaseOrphanFiles

public static long executeDatabaseOrphanFiles(Catalog catalog,
                                              String databaseName,
                                              @Nullable
                                              String tableName,
                                              long olderThanMillis,
                                              SerializableConsumer<Path> fileCleaner,
                                              @Nullable
                                              Integer parallelism)
                                       throws Catalog.DatabaseNotExistException,
                                              Catalog.TableNotExistException

Throws:

Catalog.DatabaseNotExistException

Catalog.TableNotExistException