org.apache.paimon.catalog

Interface Catalog

All Superinterfaces:

AutoCloseable

All Known Implementing Classes:

AbstractCatalog, CachingCatalog, DelegateCatalog, FileSystemCatalog, HiveCatalog, JdbcCatalog, PrivilegedCatalog

@Public
public interface Catalog
extends AutoCloseable

This interface is responsible for reading and writing metadata such as database/table from a paimon catalog.

Since:

0.4.0

See Also:

CatalogFactory

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	Catalog.ColumnAlreadyExistException Exception for trying to alter a column that already exists.
static class	Catalog.ColumnNotExistException Exception for trying to operate on a column that doesn't exist.
static class	Catalog.DatabaseAlreadyExistException Exception for trying to create a database that already exists.
static class	Catalog.DatabaseNotEmptyException Exception for trying to drop on a database that is not empty.
static class	Catalog.DatabaseNotExistException Exception for trying to operate on a database that doesn't exist.
static interface	Catalog.Loader Loader of Catalog.
static class	Catalog.PartitionNotExistException Exception for trying to operate on a partition that doesn't exist.
static class	Catalog.ProcessSystemDatabaseException Exception for trying to operate on a system database.
static class	Catalog.TableAlreadyExistException Exception for trying to create a table that already exists.
static class	Catalog.TableNotExistException Exception for trying to operate on a table that doesn't exist.

Field Summary

Fields

Modifier and Type	Field and Description
static String	COMMENT_PROP
static String	DB_LOCATION_PROP
static String	DB_SUFFIX
static String	DEFAULT_DATABASE
static String	SYSTEM_BRANCH_PREFIX
static String	SYSTEM_DATABASE_NAME
static String	SYSTEM_TABLE_SPLITTER
static String	TABLE_DEFAULT_OPTION_PREFIX

Method Summary

Modifier and Type	Method and Description
boolean	allowUpperCase() Return a boolean that indicates whether this catalog allow upper case.
void	alterTable(Identifier identifier, List<SchemaChange> changes, boolean ignoreIfNotExists) Modify an existing table from SchemaChanges.
default void	alterTable(Identifier identifier, SchemaChange change, boolean ignoreIfNotExists) Modify an existing table from a SchemaChange.
default void	createDatabase(String name, boolean ignoreIfExists) Create a database, see createDatabase(String name, boolean ignoreIfExists, Map properties).
void	createDatabase(String name, boolean ignoreIfExists, Map<String,String> properties) Create a database with properties.
void	createTable(Identifier identifier, Schema schema, boolean ignoreIfExists) Create a new table.
default boolean	databaseExists(String databaseName) Check if a database exists in this catalog.
void	dropDatabase(String name, boolean ignoreIfNotExists, boolean cascade) Drop a database.
void	dropPartition(Identifier identifier, Map<String,String> partitions) Drop the partition of the specify table.
void	dropTable(Identifier identifier, boolean ignoreIfNotExists) Drop a table.
FileIO	fileIO()
Table	getTable(Identifier identifier) Return a Table identified by the given Identifier.
Path	getTableLocation(Identifier identifier) Get the table location in this catalog.
default void	invalidateTable(Identifier identifier) Invalidate cached table metadata for an identifier.
List<String>	listDatabases() Get the names of all databases in this catalog.
List<String>	listTables(String databaseName) Get names of all tables under this database.
Map<String,String>	loadDatabaseProperties(String name) Load database properties.
default Optional<CatalogLockContext>	lockContext() Get lock context for lock factory to create a lock.
Optional<CatalogLockFactory>	lockFactory() Get lock factory from catalog.
default Optional<MetastoreClient.Factory>	metastoreClientFactory(Identifier identifier) Get metastore client factory for the table specified by identifier.
Map<String,String>	options() Catalog options.
void	renameTable(Identifier fromTable, Identifier toTable, boolean ignoreIfNotExists) Rename a table.
default void	repairCatalog()
default void	repairDatabase(String databaseName)
default void	repairTable(Identifier identifier)
static Map<String,String>	tableDefaultOptions(Map<String,String> options)
default boolean	tableExists(Identifier identifier) Check if a table exists in this catalog.
static void	validateCaseInsensitive(boolean caseSensitive, String type, List<String> names) Validate database, table and field names must be lowercase when not case-sensitive.
static void	validateCaseInsensitive(boolean caseSensitive, String type, String... names) Validate database, table and field names must be lowercase when not case-sensitive.
String	warehouse() Warehouse root path containing all database directories in this catalog.

Methods inherited from interface java.lang.AutoCloseable

close

Field Detail

DEFAULT_DATABASE

static final String DEFAULT_DATABASE

See Also:

Constant Field Values

SYSTEM_TABLE_SPLITTER

static final String SYSTEM_TABLE_SPLITTER

See Also:

Constant Field Values

SYSTEM_DATABASE_NAME

static final String SYSTEM_DATABASE_NAME

See Also:

Constant Field Values

SYSTEM_BRANCH_PREFIX

static final String SYSTEM_BRANCH_PREFIX

See Also:

Constant Field Values

COMMENT_PROP

static final String COMMENT_PROP

See Also:

Constant Field Values

TABLE_DEFAULT_OPTION_PREFIX

static final String TABLE_DEFAULT_OPTION_PREFIX

See Also:

Constant Field Values

DB_LOCATION_PROP

static final String DB_LOCATION_PROP

See Also:

Constant Field Values

DB_SUFFIX

static final String DB_SUFFIX

See Also:

Constant Field Values

Method Detail

warehouse

String warehouse()

Warehouse root path containing all database directories in this catalog.

options

Map<String,String> options()

Catalog options.

fileIO

FileIO fileIO()

lockFactory

Optional<CatalogLockFactory> lockFactory()

Get lock factory from catalog. Lock is used to support multiple concurrent writes on the object store.

lockContext

default Optional<CatalogLockContext> lockContext()

Get lock context for lock factory to create a lock.

metastoreClientFactory

default Optional<MetastoreClient.Factory> metastoreClientFactory(Identifier identifier)

Get metastore client factory for the table specified by identifier.

listDatabases

List<String> listDatabases()

Get the names of all databases in this catalog.

Returns:

a list of the names of all databases

databaseExists

default boolean databaseExists(String databaseName)

Check if a database exists in this catalog.

Parameters:

databaseName - Name of the database

Returns:

true if the given database exists in the catalog false otherwise

createDatabase

default void createDatabase(String name,
                            boolean ignoreIfExists)
                     throws Catalog.DatabaseAlreadyExistException

Create a database, see createDatabase(String name, boolean ignoreIfExists, Map properties).

Throws:

Catalog.DatabaseAlreadyExistException

createDatabase

void createDatabase(String name,
                    boolean ignoreIfExists,
                    Map<String,String> properties)
             throws Catalog.DatabaseAlreadyExistException

Create a database with properties.

Parameters:

name - Name of the database to be created

ignoreIfExists - Flag to specify behavior when a database with the given name already exists: if set to false, throw a DatabaseAlreadyExistException, if set to true, do nothing.

properties - properties to be associated with the database

Throws:

Catalog.DatabaseAlreadyExistException - if the given database already exists and ignoreIfExists is false

loadDatabaseProperties

Map<String,String> loadDatabaseProperties(String name)
                                   throws Catalog.DatabaseNotExistException

Load database properties.

Parameters:

name - Database name

Returns:

The requested database's properties

Throws:

Catalog.DatabaseNotExistException - if the requested database does not exist

dropDatabase

void dropDatabase(String name,
                  boolean ignoreIfNotExists,
                  boolean cascade)
           throws Catalog.DatabaseNotExistException,
                  Catalog.DatabaseNotEmptyException

Drop a database.

Parameters:

name - Name of the database to be dropped.

ignoreIfNotExists - Flag to specify behavior when the database does not exist: if set to false, throw an exception, if set to true, do nothing.

cascade - Flag to specify behavior when the database contains table or function: if set to true, delete all tables and functions in the database and then delete the database, if set to false, throw an exception.

Throws:

Catalog.DatabaseNotEmptyException - if the given database is not empty and isRestrict is true

Catalog.DatabaseNotExistException

getTable

Table getTable(Identifier identifier)
        throws Catalog.TableNotExistException

Return a Table identified by the given Identifier.

System tables can be got by '$' splitter.

Parameters:

identifier - Path of the table

Returns:

The requested table

Throws:

Catalog.TableNotExistException - if the target does not exist

getTableLocation

Path getTableLocation(Identifier identifier)

Get the table location in this catalog. If the table exists, return the location of the table; If the table does not exist, construct the location for table.

Returns:

the table location

listTables

List<String> listTables(String databaseName)
                 throws Catalog.DatabaseNotExistException

Get names of all tables under this database. An empty list is returned if none exists.

NOTE: System tables will not be listed.

Returns:

a list of the names of all tables in this database

Throws:

Catalog.DatabaseNotExistException - if the database does not exist

tableExists

default boolean tableExists(Identifier identifier)

Check if a table exists in this catalog.

Parameters:

identifier - Path of the table

Returns:

true if the given table exists in the catalog false otherwise

dropTable

void dropTable(Identifier identifier,
               boolean ignoreIfNotExists)
        throws Catalog.TableNotExistException

Drop a table.

NOTE: System tables can not be dropped.

Parameters:

identifier - Path of the table to be dropped

ignoreIfNotExists - Flag to specify behavior when the table does not exist: if set to false, throw an exception, if set to true, do nothing.

Throws:

Catalog.TableNotExistException - if the table does not exist

createTable

void createTable(Identifier identifier,
                 Schema schema,
                 boolean ignoreIfExists)
          throws Catalog.TableAlreadyExistException,
                 Catalog.DatabaseNotExistException

Create a new table.

NOTE: System tables can not be created.

Parameters:

identifier - path of the table to be created

schema - the table definition

ignoreIfExists - flag to specify behavior when a table already exists at the given path: if set to false, it throws a TableAlreadyExistException, if set to true, do nothing.

Throws:

Catalog.TableAlreadyExistException - if table already exists and ignoreIfExists is false

Catalog.DatabaseNotExistException - if the database in identifier doesn't exist

renameTable

void renameTable(Identifier fromTable,
                 Identifier toTable,
                 boolean ignoreIfNotExists)
          throws Catalog.TableNotExistException,
                 Catalog.TableAlreadyExistException

Rename a table.

NOTE: If you use object storage, such as S3 or OSS, please use this syntax carefully, because the renaming of object storage is not atomic, and only partial files may be moved in case of failure.

NOTE: System tables can not be renamed.

Parameters:

fromTable - the name of the table which need to rename

toTable - the new table

ignoreIfNotExists - Flag to specify behavior when the table does not exist: if set to false, throw an exception, if set to true, do nothing.

Throws:

Catalog.TableNotExistException - if the fromTable does not exist

Catalog.TableAlreadyExistException - if the toTable already exists

alterTable

void alterTable(Identifier identifier,
                List<SchemaChange> changes,
                boolean ignoreIfNotExists)
         throws Catalog.TableNotExistException,
                Catalog.ColumnAlreadyExistException,
                Catalog.ColumnNotExistException

Modify an existing table from SchemaChanges.

NOTE: System tables can not be altered.

Parameters:

identifier - path of the table to be modified

changes - the schema changes

ignoreIfNotExists - flag to specify behavior when the table does not exist: if set to false, throw an exception, if set to true, do nothing.

Throws:

Catalog.TableNotExistException - if the table does not exist

Catalog.ColumnAlreadyExistException

Catalog.ColumnNotExistException

invalidateTable

default void invalidateTable(Identifier identifier)

Invalidate cached table metadata for an identifier.

If the table is already loaded or cached, drop cached data. If the table does not exist or is not cached, do nothing. Calling this method should not query remote services.

Parameters:

identifier - a table identifier

dropPartition

void dropPartition(Identifier identifier,
                   Map<String,String> partitions)
            throws Catalog.TableNotExistException,
                   Catalog.PartitionNotExistException

Drop the partition of the specify table.

Parameters:

identifier - path of the table to drop partition

partitions - the partition to be deleted

Throws:

Catalog.TableNotExistException - if the table does not exist

Catalog.PartitionNotExistException - if the partition does not exist

alterTable

default void alterTable(Identifier identifier,
                        SchemaChange change,
                        boolean ignoreIfNotExists)
                 throws Catalog.TableNotExistException,
                        Catalog.ColumnAlreadyExistException,
                        Catalog.ColumnNotExistException

Modify an existing table from a SchemaChange.

NOTE: System tables can not be altered.

Parameters:

identifier - path of the table to be modified

change - the schema change

ignoreIfNotExists - flag to specify behavior when the table does not exist: if set to false, throw an exception, if set to true, do nothing.

Throws:

Catalog.TableNotExistException - if the table does not exist

Catalog.ColumnAlreadyExistException

Catalog.ColumnNotExistException

allowUpperCase

boolean allowUpperCase()

Return a boolean that indicates whether this catalog allow upper case.

repairCatalog

default void repairCatalog()

repairDatabase

default void repairDatabase(String databaseName)

repairTable

default void repairTable(Identifier identifier)
                  throws Catalog.TableNotExistException

Throws:

Catalog.TableNotExistException

tableDefaultOptions

static Map<String,String> tableDefaultOptions(Map<String,String> options)

validateCaseInsensitive

static void validateCaseInsensitive(boolean caseSensitive,
                                    String type,
                                    String... names)

Validate database, table and field names must be lowercase when not case-sensitive.

validateCaseInsensitive

static void validateCaseInsensitive(boolean caseSensitive,
                                    String type,
                                    List<String> names)

Validate database, table and field names must be lowercase when not case-sensitive.