org.apache.paimon.flink.orphan

Class FlinkOrphanFilesClean

java.lang.Object

org.apache.paimon.operation.OrphanFilesClean

org.apache.paimon.flink.orphan.FlinkOrphanFilesClean

All Implemented Interfaces:

Serializable

public class FlinkOrphanFilesClean
extends OrphanFilesClean

Flink OrphanFilesClean, it will submit a job for a table.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.paimon.operation.OrphanFilesClean

OrphanFilesClean.ReaderWithIOException<T>

Field Summary

Fields

Modifier and Type	Field and Description
protected Integer	parallelism

Fields inherited from class org.apache.paimon.operation.OrphanFilesClean

fileCleaner, fileIO, location, LOG, olderThanMillis, partitionKeysNum, READ_FILE_RETRY_INTERVAL, READ_FILE_RETRY_NUM, table

Constructor Summary

Constructors

Constructor and Description
FlinkOrphanFilesClean(FileStoreTable table, long olderThanMillis, SerializableConsumer<Path> fileCleaner, Integer parallelism)

Method Summary

Modifier and Type	Method and Description
org.apache.flink.streaming.api.datastream.DataStream<CleanOrphanFilesResult>	doOrphanClean(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment env)
static CleanOrphanFilesResult	executeDatabaseOrphanFiles(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment env, Catalog catalog, long olderThanMillis, SerializableConsumer<Path> fileCleaner, Integer parallelism, String databaseName, String tableName)

Methods inherited from class org.apache.paimon.operation.OrphanFilesClean

cleanSnapshotDir, collectWithoutDataFile, collectWithoutDataFileWithManifestFlag, createFileCleaner, listPaimonFileDirs, oldEnough, olderThanMillis, retryReadingFiles, safelyGetAllSnapshots, tryBestListingDirs, validBranches

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

parallelism

@Nullable
protected final Integer parallelism

Constructor Detail

FlinkOrphanFilesClean

public FlinkOrphanFilesClean(FileStoreTable table,
                             long olderThanMillis,
                             SerializableConsumer<Path> fileCleaner,
                             @Nullable
                             Integer parallelism)

Method Detail

doOrphanClean

@Nullable
public org.apache.flink.streaming.api.datastream.DataStream<CleanOrphanFilesResult> doOrphanClean(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment env)

executeDatabaseOrphanFiles

public static CleanOrphanFilesResult executeDatabaseOrphanFiles(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment env,
                                                                Catalog catalog,
                                                                long olderThanMillis,
                                                                SerializableConsumer<Path> fileCleaner,
                                                                @Nullable
                                                                Integer parallelism,
                                                                String databaseName,
                                                                @Nullable
                                                                String tableName)
                                                         throws Catalog.DatabaseNotExistException,
                                                                Catalog.TableNotExistException

Throws:

Catalog.DatabaseNotExistException

Catalog.TableNotExistException