org.apache.paimon.flink.clone

Class CloneFilesUtil

java.lang.Object

org.apache.paimon.flink.clone.CloneFilesUtil

public class CloneFilesUtil
extends Object

Util class for get used files' paths of a table's latest snapshot.

Constructor Summary

Constructors

Constructor and Description
CloneFilesUtil()

Method Summary

Modifier and Type	Method and Description
static List<Pair<Path,Path>>	getDataUsedFilesForSnapshot(FileStoreTable table, long snapshotId) Retrieves a map of data file types to their corresponding list of file paths for a given snapshot.
static FileIO	getFileIO(Map<String,FileIO> fileIOs, String identifier, Catalog catalog)
static List<Path>	getManifestUsedFilesForSnapshot(FileStoreTable table, long snapshotId) Retrieves a map of manifest file types to their corresponding list of file paths for a given snapshot.
static Path	getPath(Map<String,Path> locations, String identifier, Catalog catalog)
static Path	getPathExcludeTableRoot(Path absolutePath, Path sourceTableRoot)
static List<Path>	getSchemaUsedFilesForSnapshot(FileStoreTable table, long snapshotId) Retrieves a map of schema file types to their corresponding list of file paths for a given snapshot。 The schema file types include: Snapshot, Schema, ManifestList, StatisticFile and IndexFile .
static Path	pathOfTable(Table table)
static <T> T	retryReadingFiles(SupplierWithIOException<T> reader)
static List<CloneFileInfo>	toCloneFileInfos(List<Pair<Path,Path>> fileList, String sourceIdentifier, String targetIdentifier)
static List<CloneFileInfo>	toCloneFileInfos(List<Path> fileList, Path sourceTableRoot, String sourceIdentifier, String targetIdentifier)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CloneFilesUtil

public CloneFilesUtil()

Method Detail

getSchemaUsedFilesForSnapshot

public static List<Path> getSchemaUsedFilesForSnapshot(FileStoreTable table,
                                                       long snapshotId)
                                                throws IOException

Retrieves a map of schema file types to their corresponding list of file paths for a given snapshot。 The schema file types include: Snapshot, Schema, ManifestList, StatisticFile and IndexFile .

Parameters:

table - The FileStoreTable object representing the table.

snapshotId - The ID of the snapshot to retrieve files for.

Returns:

A map where the key is the FileType and the value is a list of file paths.

Throws:

FileNotFoundException - If the snapshot file is not found.

IOException

getDataUsedFilesForSnapshot

public static List<Pair<Path,Path>> getDataUsedFilesForSnapshot(FileStoreTable table,
                                                                long snapshotId)
                                                         throws FileNotFoundException

Retrieves a map of data file types to their corresponding list of file paths for a given snapshot. The data file types include: DataFile and ChangelogFile.

Parameters:

table - The FileStoreTable object representing the table.

snapshotId - The ID of the snapshot to retrieve files for.

Returns:

A map where the key is the FileType and the value is a list of file paths. the pair is the data file's absolute path and data file's relative path.

Throws:

FileNotFoundException - If the snapshot file is not found.

getManifestUsedFilesForSnapshot

public static List<Path> getManifestUsedFilesForSnapshot(FileStoreTable table,
                                                         long snapshotId)
                                                  throws IOException

Retrieves a map of manifest file types to their corresponding list of file paths for a given snapshot. The manifest file types include: ManifestFile.

Parameters:

table - The FileStoreTable object representing the table.

snapshotId - The ID of the snapshot to retrieve files for.

Returns:

A map where the key is the FileType and the value is a list of file paths.

Throws:

FileNotFoundException - If the snapshot file is not found.

IOException

retryReadingFiles

@Nullable
public static <T> T retryReadingFiles(SupplierWithIOException<T> reader)
                                         throws IOException

Throws:

IOException

toCloneFileInfos

public static List<CloneFileInfo> toCloneFileInfos(List<Path> fileList,
                                                   Path sourceTableRoot,
                                                   String sourceIdentifier,
                                                   String targetIdentifier)

toCloneFileInfos

public static List<CloneFileInfo> toCloneFileInfos(List<Pair<Path,Path>> fileList,
                                                   String sourceIdentifier,
                                                   String targetIdentifier)

getPathExcludeTableRoot

public static Path getPathExcludeTableRoot(Path absolutePath,
                                           Path sourceTableRoot)

getFileIO

public static FileIO getFileIO(Map<String,FileIO> fileIOs,
                               String identifier,
                               Catalog catalog)

getPath

public static Path getPath(Map<String,Path> locations,
                           String identifier,
                           Catalog catalog)

pathOfTable

public static Path pathOfTable(Table table)