org.apache.paimon.catalog

Interface Catalog

All Superinterfaces:

AutoCloseable

All Known Implementing Classes:

AbstractCatalog, CachingCatalog, DelegateCatalog, FileSystemCatalog, HiveCatalog, JdbcCatalog, PrivilegedCatalog

@Public
public interface Catalog
extends AutoCloseable

This interface is responsible for reading and writing metadata such as database/table from a paimon catalog.

Since:

0.4.0

See Also:

CatalogFactory

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	Catalog.ColumnAlreadyExistException Exception for trying to alter a column that already exists.
static class	Catalog.ColumnNotExistException Exception for trying to operate on a column that doesn't exist.
static class	Catalog.DatabaseAlreadyExistException Exception for trying to create a database that already exists.
static class	Catalog.DatabaseNotEmptyException Exception for trying to drop on a database that is not empty.
static class	Catalog.DatabaseNotExistException Exception for trying to operate on a database that doesn't exist.
static interface	Catalog.Loader Loader of Catalog.
static class	Catalog.PartitionNotExistException Exception for trying to operate on a partition that doesn't exist.
static class	Catalog.ProcessSystemDatabaseException Exception for trying to operate on a system database.
static class	Catalog.TableAlreadyExistException Exception for trying to create a table that already exists.
static class	Catalog.TableNotExistException Exception for trying to operate on a table that doesn't exist.
static class	Catalog.ViewAlreadyExistException Exception for trying to create a view that already exists.
static class	Catalog.ViewNotExistException Exception for trying to operate on a view that doesn't exist.

Field Summary

Fields

Modifier and Type	Field and Description
static String	COMMENT_PROP
static String	DB_LOCATION_PROP
static String	DB_SUFFIX
static String	DEFAULT_DATABASE
static String	HIVE_LAST_UPDATE_TIME_PROP
static String	LAST_UPDATE_TIME_PROP
static String	NUM_FILES_PROP
static String	NUM_ROWS_PROP
static String	OWNER_PROP
static String	SYSTEM_BRANCH_PREFIX
static String	SYSTEM_DATABASE_NAME
static String	SYSTEM_TABLE_SPLITTER
static String	TABLE_DEFAULT_OPTION_PREFIX
static String	TOTAL_SIZE_PROP

Method Summary

Modifier and Type	Method and Description
boolean	allowUpperCase() Return a boolean that indicates whether this catalog allow upper case.
void	alterTable(Identifier identifier, List<SchemaChange> changes, boolean ignoreIfNotExists) Modify an existing table from SchemaChanges.
default void	alterTable(Identifier identifier, SchemaChange change, boolean ignoreIfNotExists) Modify an existing table from a SchemaChange.
default void	createDatabase(String name, boolean ignoreIfExists) Create a database, see createDatabase(String name, boolean ignoreIfExists, Map properties).
void	createDatabase(String name, boolean ignoreIfExists, Map<String,String> properties) Create a database with properties.
void	createPartition(Identifier identifier, Map<String,String> partitionSpec) Create the partition of the specify table.
void	createTable(Identifier identifier, Schema schema, boolean ignoreIfExists) Create a new table.
default void	createView(Identifier identifier, View view, boolean ignoreIfExists) Create a new view.
void	dropDatabase(String name, boolean ignoreIfNotExists, boolean cascade) Drop a database.
void	dropPartition(Identifier identifier, Map<String,String> partitions) Drop the partition of the specify table.
void	dropTable(Identifier identifier, boolean ignoreIfNotExists) Drop a table.
default void	dropView(Identifier identifier, boolean ignoreIfNotExists) Drop a view.
FileIO	fileIO()
Database	getDatabase(String name) Return a Database identified by the given name.
Table	getTable(Identifier identifier) Return a Table identified by the given Identifier.
Path	getTableLocation(Identifier identifier) Get the table location in this catalog.
default View	getView(Identifier identifier) Return a View identified by the given Identifier.
default void	invalidateTable(Identifier identifier) Invalidate cached table metadata for an identifier.
List<String>	listDatabases() Get the names of all databases in this catalog.
List<PartitionEntry>	listPartitions(Identifier identifier) Get PartitionEntry of all partitions of the table.
List<String>	listTables(String databaseName) Get names of all tables under this database.
default List<String>	listViews(String databaseName) Get names of all views under this database.
Map<String,String>	options() Catalog options.
void	renameTable(Identifier fromTable, Identifier toTable, boolean ignoreIfNotExists) Rename a table.
default void	renameView(Identifier fromView, Identifier toView, boolean ignoreIfNotExists) Rename a view.
default void	repairCatalog()
default void	repairDatabase(String databaseName)
default void	repairTable(Identifier identifier)
static Map<String,String>	tableDefaultOptions(Map<String,String> options)
static void	validateCaseInsensitive(boolean caseSensitive, String type, List<String> names) Validate database, table and field names must be lowercase when not case-sensitive.
static void	validateCaseInsensitive(boolean caseSensitive, String type, String... names) Validate database, table and field names must be lowercase when not case-sensitive.
String	warehouse() Warehouse root path containing all database directories in this catalog.

Methods inherited from interface java.lang.AutoCloseable

close

Field Detail

DEFAULT_DATABASE

static final String DEFAULT_DATABASE

See Also:

Constant Field Values

SYSTEM_TABLE_SPLITTER

static final String SYSTEM_TABLE_SPLITTER

See Also:

Constant Field Values

SYSTEM_DATABASE_NAME

static final String SYSTEM_DATABASE_NAME

See Also:

Constant Field Values

SYSTEM_BRANCH_PREFIX

static final String SYSTEM_BRANCH_PREFIX

See Also:

Constant Field Values

TABLE_DEFAULT_OPTION_PREFIX

static final String TABLE_DEFAULT_OPTION_PREFIX

See Also:

Constant Field Values

DB_SUFFIX

static final String DB_SUFFIX

See Also:

Constant Field Values

COMMENT_PROP

static final String COMMENT_PROP

See Also:

Constant Field Values

OWNER_PROP

static final String OWNER_PROP

See Also:

Constant Field Values

DB_LOCATION_PROP

static final String DB_LOCATION_PROP

See Also:

Constant Field Values

NUM_ROWS_PROP

static final String NUM_ROWS_PROP

See Also:

Constant Field Values

NUM_FILES_PROP

static final String NUM_FILES_PROP

See Also:

Constant Field Values

TOTAL_SIZE_PROP

static final String TOTAL_SIZE_PROP

See Also:

Constant Field Values

LAST_UPDATE_TIME_PROP

static final String LAST_UPDATE_TIME_PROP

See Also:

Constant Field Values

HIVE_LAST_UPDATE_TIME_PROP

static final String HIVE_LAST_UPDATE_TIME_PROP

See Also:

Constant Field Values

Method Detail

warehouse

String warehouse()

Warehouse root path containing all database directories in this catalog.

options

Map<String,String> options()

Catalog options.

fileIO

FileIO fileIO()

listDatabases

List<String> listDatabases()

Get the names of all databases in this catalog.

Returns:

a list of the names of all databases

createDatabase

default void createDatabase(String name,
                            boolean ignoreIfExists)
                     throws Catalog.DatabaseAlreadyExistException

Create a database, see createDatabase(String name, boolean ignoreIfExists, Map properties).

Throws:

Catalog.DatabaseAlreadyExistException

createDatabase

void createDatabase(String name,
                    boolean ignoreIfExists,
                    Map<String,String> properties)
             throws Catalog.DatabaseAlreadyExistException

Create a database with properties.

Parameters:

name - Name of the database to be created

ignoreIfExists - Flag to specify behavior when a database with the given name already exists: if set to false, throw a DatabaseAlreadyExistException, if set to true, do nothing.

properties - properties to be associated with the database

Throws:

Catalog.DatabaseAlreadyExistException - if the given database already exists and ignoreIfExists is false

getDatabase

Database getDatabase(String name)
              throws Catalog.DatabaseNotExistException

Return a Database identified by the given name.

Parameters:

name - Database name

Returns:

The requested Database

Throws:

Catalog.DatabaseNotExistException - if the requested database does not exist

dropDatabase

void dropDatabase(String name,
                  boolean ignoreIfNotExists,
                  boolean cascade)
           throws Catalog.DatabaseNotExistException,
                  Catalog.DatabaseNotEmptyException

Drop a database.

Parameters:

name - Name of the database to be dropped.

ignoreIfNotExists - Flag to specify behavior when the database does not exist: if set to false, throw an exception, if set to true, do nothing.

cascade - Flag to specify behavior when the database contains table or function: if set to true, delete all tables and functions in the database and then delete the database, if set to false, throw an exception.

Throws:

Catalog.DatabaseNotEmptyException - if the given database is not empty and isRestrict is true

Catalog.DatabaseNotExistException

getTable

Table getTable(Identifier identifier)
        throws Catalog.TableNotExistException

Return a Table identified by the given Identifier.

System tables can be got by '$' splitter.

Parameters:

identifier - Path of the table

Returns:

The requested table

Throws:

Catalog.TableNotExistException - if the target does not exist

getTableLocation

Path getTableLocation(Identifier identifier)

Get the table location in this catalog. If the table exists, return the location of the table; If the table does not exist, construct the location for table.

Returns:

the table location

listTables

List<String> listTables(String databaseName)
                 throws Catalog.DatabaseNotExistException

Get names of all tables under this database. An empty list is returned if none exists.

NOTE: System tables will not be listed.

Returns:

a list of the names of all tables in this database

Throws:

Catalog.DatabaseNotExistException - if the database does not exist

dropTable

void dropTable(Identifier identifier,
               boolean ignoreIfNotExists)
        throws Catalog.TableNotExistException

Drop a table.

NOTE: System tables can not be dropped.

Parameters:

identifier - Path of the table to be dropped

ignoreIfNotExists - Flag to specify behavior when the table does not exist: if set to false, throw an exception, if set to true, do nothing.

Throws:

Catalog.TableNotExistException - if the table does not exist

createTable

void createTable(Identifier identifier,
                 Schema schema,
                 boolean ignoreIfExists)
          throws Catalog.TableAlreadyExistException,
                 Catalog.DatabaseNotExistException

Create a new table.

NOTE: System tables can not be created.

Parameters:

identifier - path of the table to be created

schema - the table definition

ignoreIfExists - flag to specify behavior when a table already exists at the given path: if set to false, it throws a TableAlreadyExistException, if set to true, do nothing.

Throws:

Catalog.TableAlreadyExistException - if table already exists and ignoreIfExists is false

Catalog.DatabaseNotExistException - if the database in identifier doesn't exist

renameTable

void renameTable(Identifier fromTable,
                 Identifier toTable,
                 boolean ignoreIfNotExists)
          throws Catalog.TableNotExistException,
                 Catalog.TableAlreadyExistException

Rename a table.

NOTE: If you use object storage, such as S3 or OSS, please use this syntax carefully, because the renaming of object storage is not atomic, and only partial files may be moved in case of failure.

NOTE: System tables can not be renamed.

Parameters:

fromTable - the name of the table which need to rename

toTable - the new table

ignoreIfNotExists - Flag to specify behavior when the table does not exist: if set to false, throw an exception, if set to true, do nothing.

Throws:

Catalog.TableNotExistException - if the fromTable does not exist

Catalog.TableAlreadyExistException - if the toTable already exists

alterTable

void alterTable(Identifier identifier,
                List<SchemaChange> changes,
                boolean ignoreIfNotExists)
         throws Catalog.TableNotExistException,
                Catalog.ColumnAlreadyExistException,
                Catalog.ColumnNotExistException

Modify an existing table from SchemaChanges.

NOTE: System tables can not be altered.

Parameters:

identifier - path of the table to be modified

changes - the schema changes

ignoreIfNotExists - flag to specify behavior when the table does not exist: if set to false, throw an exception, if set to true, do nothing.

Throws:

Catalog.TableNotExistException - if the table does not exist

Catalog.ColumnAlreadyExistException

Catalog.ColumnNotExistException

invalidateTable

default void invalidateTable(Identifier identifier)

Invalidate cached table metadata for an identifier.

If the table is already loaded or cached, drop cached data. If the table does not exist or is not cached, do nothing. Calling this method should not query remote services.

Parameters:

identifier - a table identifier

createPartition

void createPartition(Identifier identifier,
                     Map<String,String> partitionSpec)
              throws Catalog.TableNotExistException

Create the partition of the specify table.

Only catalog with metastore can support this method, and only table with 'metastore.partitioned-table' can support this method.

Parameters:

identifier - path of the table to drop partition

partitionSpec - the partition to be created

Throws:

Catalog.TableNotExistException - if the table does not exist

dropPartition

void dropPartition(Identifier identifier,
                   Map<String,String> partitions)
            throws Catalog.TableNotExistException,
                   Catalog.PartitionNotExistException

Drop the partition of the specify table.

Parameters:

identifier - path of the table to drop partition

partitions - the partition to be deleted

Throws:

Catalog.TableNotExistException - if the table does not exist

Catalog.PartitionNotExistException - if the partition does not exist

listPartitions

List<PartitionEntry> listPartitions(Identifier identifier)
                             throws Catalog.TableNotExistException

Get PartitionEntry of all partitions of the table.

Parameters:

identifier - path of the table to list partitions

Throws:

Catalog.TableNotExistException - if the table does not exist

alterTable

default void alterTable(Identifier identifier,
                        SchemaChange change,
                        boolean ignoreIfNotExists)
                 throws Catalog.TableNotExistException,
                        Catalog.ColumnAlreadyExistException,
                        Catalog.ColumnNotExistException

Modify an existing table from a SchemaChange.

NOTE: System tables can not be altered.

Parameters:

identifier - path of the table to be modified

change - the schema change

ignoreIfNotExists - flag to specify behavior when the table does not exist: if set to false, throw an exception, if set to true, do nothing.

Throws:

Catalog.TableNotExistException - if the table does not exist

Catalog.ColumnAlreadyExistException

Catalog.ColumnNotExistException

getView

default View getView(Identifier identifier)
              throws Catalog.ViewNotExistException

Return a View identified by the given Identifier.

Parameters:

identifier - Path of the view

Returns:

The requested view

Throws:

Catalog.ViewNotExistException - if the target does not exist

dropView

default void dropView(Identifier identifier,
                      boolean ignoreIfNotExists)
               throws Catalog.ViewNotExistException

Drop a view.

Parameters:

identifier - Path of the view to be dropped

ignoreIfNotExists - Flag to specify behavior when the view does not exist: if set to false, throw an exception, if set to true, do nothing.

Throws:

Catalog.ViewNotExistException - if the view does not exist

createView

default void createView(Identifier identifier,
                        View view,
                        boolean ignoreIfExists)
                 throws Catalog.ViewAlreadyExistException,
                        Catalog.DatabaseNotExistException

Create a new view.

Parameters:

identifier - path of the view to be created

view - the view definition

ignoreIfExists - flag to specify behavior when a view already exists at the given path: if set to false, it throws a ViewAlreadyExistException, if set to true, do nothing.

Throws:

Catalog.ViewAlreadyExistException - if view already exists and ignoreIfExists is false

Catalog.DatabaseNotExistException - if the database in identifier doesn't exist

listViews

default List<String> listViews(String databaseName)
                        throws Catalog.DatabaseNotExistException

Get names of all views under this database. An empty list is returned if none exists.

Returns:

a list of the names of all views in this database

Throws:

Catalog.DatabaseNotExistException - if the database does not exist

renameView

default void renameView(Identifier fromView,
                        Identifier toView,
                        boolean ignoreIfNotExists)
                 throws Catalog.ViewNotExistException,
                        Catalog.ViewAlreadyExistException

Rename a view.

Parameters:

fromView - identifier of the view to rename

toView - new view identifier

Throws:

Catalog.ViewNotExistException - if the fromView does not exist

Catalog.ViewAlreadyExistException - if the toView already exists

allowUpperCase

boolean allowUpperCase()

Return a boolean that indicates whether this catalog allow upper case.

repairCatalog

default void repairCatalog()

repairDatabase

default void repairDatabase(String databaseName)

repairTable

default void repairTable(Identifier identifier)
                  throws Catalog.TableNotExistException

Throws:

Catalog.TableNotExistException

tableDefaultOptions

static Map<String,String> tableDefaultOptions(Map<String,String> options)

validateCaseInsensitive

static void validateCaseInsensitive(boolean caseSensitive,
                                    String type,
                                    String... names)

Validate database, table and field names must be lowercase when not case-sensitive.

validateCaseInsensitive

static void validateCaseInsensitive(boolean caseSensitive,
                                    String type,
                                    List<String> names)

Validate database, table and field names must be lowercase when not case-sensitive.