org.apache.paimon.fs

Interface FileIO

All Superinterfaces:

Serializable

All Known Implementing Classes:

FlinkFileIO, HadoopCompliantFileIO, HadoopCompliantFileIO, HadoopFileIO, LocalFileIO, OSSFileIO, PluginFileIO, S3FileIO

@Public
 @ThreadSafe
public interface FileIO
extends Serializable

File IO to read and write file.

Since:

0.4.0

Field Summary

Fields

Modifier and Type	Field and Description
static org.slf4j.Logger	LOG

Method Summary

Modifier and Type	Method and Description
default void	checkOrMkdirs(Path path)
void	configure(CatalogContext context) Configure by CatalogContext.
default void	copyFile(Path sourcePath, Path targetPath, boolean overwrite) Copy content of one file into another.
default void	copyFiles(Path sourceDirectory, Path targetDirectory, boolean overwrite) Copy all files in sourceDirectory to directory targetDirectory.
boolean	delete(Path path, boolean recursive) Delete a file.
default void	deleteDirectoryQuietly(Path directory)
default void	deleteFilesQuietly(List<Path> files)
default void	deleteQuietly(Path file)
static Map<String,FileIOLoader>	discoverLoaders() Discovers all FileIOLoader by service loader.
boolean	exists(Path path) Check if exists.
static FileIO	get(Path path, CatalogContext config) Returns a reference to the FileIO instance for accessing the file system identified by the given path.
default long	getFileSize(Path path)
FileStatus	getFileStatus(Path path) Return a file status object that represents the path.
default boolean	isDir(Path path)
boolean	isObjectStore()
default FileStatus[]	listDirectories(Path path) List the statuses of the directories in the given path if the path is a directory.
FileStatus[]	listStatus(Path path) List the statuses of the files/directories in the given path if the path is a directory.
boolean	mkdirs(Path path) Make the given file and all non-existent parents into directories.
SeekableInputStream	newInputStream(Path path) Opens an SeekableInputStream at the indicated Path.
PositionOutputStream	newOutputStream(Path path, boolean overwrite) Opens an PositionOutputStream at the indicated Path.
default void	overwriteFileUtf8(Path path, String content) Overwrite file by content atomically, different FileIOs have different atomic implementations.
default String	readFileUtf8(Path path) Read file to UTF_8 decoding.
default Optional<String>	readOverwrittenFileUtf8(Path path) Read file from overwriteFileUtf8(org.apache.paimon.fs.Path, java.lang.String) file.
boolean	rename(Path src, Path dst) Renames the file/directory src to dst.
default boolean	tryToWriteAtomic(Path path, String content) Write content to one file atomically, initially writes to temp hidden file and only renames to the target file once temp file is closed.
default void	writeFile(Path path, String content, boolean overwrite)

Field Detail

LOG

static final org.slf4j.Logger LOG

Method Detail

isObjectStore

boolean isObjectStore()

configure

void configure(CatalogContext context)

Configure by CatalogContext.

newInputStream

SeekableInputStream newInputStream(Path path)
                            throws IOException

Opens an SeekableInputStream at the indicated Path.

Parameters:

path - the file to open

Throws:

IOException

newOutputStream

PositionOutputStream newOutputStream(Path path,
                                     boolean overwrite)
                              throws IOException

Opens an PositionOutputStream at the indicated Path.

Parameters:

path - the file name to open

overwrite - if a file with this name already exists, then if true, the file will be overwritten, and if false an error will be thrown.

Throws:

IOException - Thrown, if the stream could not be opened because of an I/O, or because a file already exists at that path and the write mode indicates to not overwrite the file.

getFileStatus

FileStatus getFileStatus(Path path)
                  throws IOException

Return a file status object that represents the path.

Parameters:

path - The path we want information from

Returns:

a FileStatus object

Throws:

FileNotFoundException - when the path does not exist; IOException see specific implementation

IOException

listStatus

FileStatus[] listStatus(Path path)
                 throws IOException

List the statuses of the files/directories in the given path if the path is a directory.

Parameters:

path - given path

Returns:

the statuses of the files/directories in the given path

Throws:

IOException

listDirectories

default FileStatus[] listDirectories(Path path)
                              throws IOException

List the statuses of the directories in the given path if the path is a directory.

FileIO implementation may have optimization for list directories.

Parameters:

path - given path

Returns:

the statuses of the directories in the given path

Throws:

IOException

exists

boolean exists(Path path)
        throws IOException

Check if exists.

Parameters:

path - source file

Throws:

IOException

delete

boolean delete(Path path,
               boolean recursive)
        throws IOException

Delete a file.

Parameters:

path - the path to delete

recursive - if path is a directory and set to true, the directory is deleted else throws an exception. In case of a file the recursive can be set to either true or false

Returns:

true if delete is successful, false otherwise

Throws:

IOException

mkdirs

boolean mkdirs(Path path)
        throws IOException

Make the given file and all non-existent parents into directories. Has the semantics of Unix 'mkdir -p'. Existence of the directory hierarchy is not an error.

Parameters:

path - the directory/directories to be created

Returns:

true if at least one new directory has been created, false otherwise

Throws:

IOException - thrown if an I/O error occurs while creating the directory

rename

boolean rename(Path src,
               Path dst)
        throws IOException

Renames the file/directory src to dst.

Parameters:

src - the file/directory to rename

dst - the new name of the file/directory

Returns:

true if the renaming was successful, false otherwise

Throws:

IOException

deleteQuietly

default void deleteQuietly(Path file)

deleteFilesQuietly

default void deleteFilesQuietly(List<Path> files)

deleteDirectoryQuietly

default void deleteDirectoryQuietly(Path directory)

getFileSize

default long getFileSize(Path path)
                  throws IOException

Throws:

IOException

isDir

default boolean isDir(Path path)
               throws IOException

Throws:

IOException

checkOrMkdirs

default void checkOrMkdirs(Path path)
                    throws IOException

Throws:

IOException

readFileUtf8

default String readFileUtf8(Path path)
                     throws IOException

Read file to UTF_8 decoding.

Throws:

IOException

tryToWriteAtomic

default boolean tryToWriteAtomic(Path path,
                                 String content)
                          throws IOException

Write content to one file atomically, initially writes to temp hidden file and only renames to the target file once temp file is closed.

Returns:

false if target file exists

Throws:

IOException

writeFile

default void writeFile(Path path,
                       String content,
                       boolean overwrite)
                throws IOException

Throws:

IOException

overwriteFileUtf8

default void overwriteFileUtf8(Path path,
                               String content)
                        throws IOException

Overwrite file by content atomically, different FileIOs have different atomic implementations.

Throws:

IOException

copyFile

default void copyFile(Path sourcePath,
                      Path targetPath,
                      boolean overwrite)
               throws IOException

Copy content of one file into another.

Throws:

IOException - Thrown, if the stream could not be opened because of an I/O, or because target file already exists at that path and the write mode indicates to not overwrite the file.

copyFiles

default void copyFiles(Path sourceDirectory,
                       Path targetDirectory,
                       boolean overwrite)
                throws IOException

Copy all files in sourceDirectory to directory targetDirectory.

Throws:

IOException

readOverwrittenFileUtf8

default Optional<String> readOverwrittenFileUtf8(Path path)
                                          throws IOException

Read file from overwriteFileUtf8(org.apache.paimon.fs.Path, java.lang.String) file.

Throws:

IOException

get

static FileIO get(Path path,
                  CatalogContext config)
           throws IOException

Returns a reference to the FileIO instance for accessing the file system identified by the given path.

Throws:

IOException

discoverLoaders

static Map<String,FileIOLoader> discoverLoaders()

Discovers all FileIOLoader by service loader.