org.apache.paimon.catalog

Class CachingCatalog

java.lang.Object

org.apache.paimon.catalog.DelegateCatalog

org.apache.paimon.catalog.CachingCatalog

All Implemented Interfaces:

AutoCloseable, Catalog

public class CachingCatalog
extends DelegateCatalog

A Catalog to cache databases and tables and manifests.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.paimon.catalog.Catalog

Catalog.ColumnAlreadyExistException, Catalog.ColumnNotExistException, Catalog.DatabaseAlreadyExistException, Catalog.DatabaseNotEmptyException, Catalog.DatabaseNotExistException, Catalog.Loader, Catalog.PartitionNotExistException, Catalog.ProcessSystemDatabaseException, Catalog.TableAlreadyExistException, Catalog.TableNotExistException

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.paimon.shade.caffeine2.com.github.benmanes.caffeine.cache.Cache<String,Map<String,String>>	databaseCache
protected SegmentsCache<Path>	manifestCache
protected org.apache.paimon.shade.caffeine2.com.github.benmanes.caffeine.cache.Cache<Identifier,Table>	tableCache

Fields inherited from class org.apache.paimon.catalog.DelegateCatalog

wrapped

Fields inherited from interface org.apache.paimon.catalog.Catalog

COMMENT_PROP, DB_LOCATION_PROP, DB_SUFFIX, DEFAULT_DATABASE, SYSTEM_BRANCH_PREFIX, SYSTEM_DATABASE_NAME, SYSTEM_TABLE_SPLITTER, TABLE_DEFAULT_OPTION_PREFIX

Constructor Summary

Constructors

Constructor and Description
CachingCatalog(Catalog wrapped)
CachingCatalog(Catalog wrapped, java.time.Duration expirationInterval, MemorySize manifestMaxMemory, long manifestCacheThreshold)
CachingCatalog(Catalog wrapped, java.time.Duration expirationInterval, MemorySize manifestMaxMemory, long manifestCacheThreshold, org.apache.paimon.shade.caffeine2.com.github.benmanes.caffeine.cache.Ticker ticker)

Method Summary

Modifier and Type	Method and Description
void	alterTable(Identifier identifier, List<SchemaChange> changes, boolean ignoreIfNotExists) Modify an existing table from SchemaChanges.
void	dropDatabase(String name, boolean ignoreIfNotExists, boolean cascade) Drop a database.
void	dropTable(Identifier identifier, boolean ignoreIfNotExists) Drop a table.
Table	getTable(Identifier identifier) Return a Table identified by the given Identifier.
void	invalidateTable(Identifier identifier) Invalidate cached table metadata for an identifier.
Map<String,String>	loadDatabaseProperties(String databaseName) Load database properties.
void	renameTable(Identifier fromTable, Identifier toTable, boolean ignoreIfNotExists) Rename a table.
static Catalog	tryToCreate(Catalog catalog, Options options)

Methods inherited from class org.apache.paimon.catalog.DelegateCatalog

allowUpperCase, close, createDatabase, createTable, dropPartition, fileIO, getTableLocation, listDatabases, listTables, lockContext, lockFactory, metastoreClientFactory, options, repairCatalog, repairDatabase, repairTable, warehouse, wrapped

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.paimon.catalog.Catalog

alterTable, createDatabase, databaseExists, tableDefaultOptions, tableExists, validateCaseInsensitive, validateCaseInsensitive

Field Detail

databaseCache

protected final org.apache.paimon.shade.caffeine2.com.github.benmanes.caffeine.cache.Cache<String,Map<String,String>> databaseCache

tableCache

protected final org.apache.paimon.shade.caffeine2.com.github.benmanes.caffeine.cache.Cache<Identifier,Table> tableCache

manifestCache

@Nullable
protected final SegmentsCache<Path> manifestCache

Constructor Detail

CachingCatalog

public CachingCatalog(Catalog wrapped)

CachingCatalog

public CachingCatalog(Catalog wrapped,
                      java.time.Duration expirationInterval,
                      MemorySize manifestMaxMemory,
                      long manifestCacheThreshold)

CachingCatalog

public CachingCatalog(Catalog wrapped,
                      java.time.Duration expirationInterval,
                      MemorySize manifestMaxMemory,
                      long manifestCacheThreshold,
                      org.apache.paimon.shade.caffeine2.com.github.benmanes.caffeine.cache.Ticker ticker)

Method Detail

tryToCreate

public static Catalog tryToCreate(Catalog catalog,
                                  Options options)

loadDatabaseProperties

public Map<String,String> loadDatabaseProperties(String databaseName)
                                          throws Catalog.DatabaseNotExistException

Description copied from interface: Catalog

Load database properties.

Specified by:

loadDatabaseProperties in interface Catalog

Overrides:

loadDatabaseProperties in class DelegateCatalog

Parameters:

databaseName - Database name

Returns:

The requested database's properties

Throws:

Catalog.DatabaseNotExistException - if the requested database does not exist

dropDatabase

public void dropDatabase(String name,
                         boolean ignoreIfNotExists,
                         boolean cascade)
                  throws Catalog.DatabaseNotExistException,
                         Catalog.DatabaseNotEmptyException

Description copied from interface: Catalog

Drop a database.

Specified by:

dropDatabase in interface Catalog

Overrides:

dropDatabase in class DelegateCatalog

Parameters:

name - Name of the database to be dropped.

ignoreIfNotExists - Flag to specify behavior when the database does not exist: if set to false, throw an exception, if set to true, do nothing.

cascade - Flag to specify behavior when the database contains table or function: if set to true, delete all tables and functions in the database and then delete the database, if set to false, throw an exception.

Throws:

Catalog.DatabaseNotEmptyException - if the given database is not empty and isRestrict is true

Catalog.DatabaseNotExistException

dropTable

public void dropTable(Identifier identifier,
                      boolean ignoreIfNotExists)
               throws Catalog.TableNotExistException

Description copied from interface: Catalog

Drop a table.

NOTE: System tables can not be dropped.

Specified by:

dropTable in interface Catalog

Overrides:

dropTable in class DelegateCatalog

Parameters:

identifier - Path of the table to be dropped

ignoreIfNotExists - Flag to specify behavior when the table does not exist: if set to false, throw an exception, if set to true, do nothing.

Throws:

Catalog.TableNotExistException - if the table does not exist

renameTable

public void renameTable(Identifier fromTable,
                        Identifier toTable,
                        boolean ignoreIfNotExists)
                 throws Catalog.TableNotExistException,
                        Catalog.TableAlreadyExistException

Description copied from interface: Catalog

Rename a table.

NOTE: If you use object storage, such as S3 or OSS, please use this syntax carefully, because the renaming of object storage is not atomic, and only partial files may be moved in case of failure.

NOTE: System tables can not be renamed.

Specified by:

renameTable in interface Catalog

Overrides:

renameTable in class DelegateCatalog

Parameters:

fromTable - the name of the table which need to rename

toTable - the new table

ignoreIfNotExists - Flag to specify behavior when the table does not exist: if set to false, throw an exception, if set to true, do nothing.

Throws:

Catalog.TableNotExistException - if the fromTable does not exist

Catalog.TableAlreadyExistException - if the toTable already exists

alterTable

public void alterTable(Identifier identifier,
                       List<SchemaChange> changes,
                       boolean ignoreIfNotExists)
                throws Catalog.TableNotExistException,
                       Catalog.ColumnAlreadyExistException,
                       Catalog.ColumnNotExistException

Description copied from interface: Catalog

Modify an existing table from SchemaChanges.

NOTE: System tables can not be altered.

Specified by:

alterTable in interface Catalog

Overrides:

alterTable in class DelegateCatalog

Parameters:

identifier - path of the table to be modified

changes - the schema changes

ignoreIfNotExists - flag to specify behavior when the table does not exist: if set to false, throw an exception, if set to true, do nothing.

Throws:

Catalog.TableNotExistException - if the table does not exist

Catalog.ColumnAlreadyExistException

Catalog.ColumnNotExistException

getTable

public Table getTable(Identifier identifier)
               throws Catalog.TableNotExistException

Description copied from interface: Catalog

Return a Table identified by the given Identifier.

System tables can be got by '$' splitter.

Specified by:

getTable in interface Catalog

Overrides:

getTable in class DelegateCatalog

Parameters:

identifier - Path of the table

Returns:

The requested table

Throws:

Catalog.TableNotExistException - if the target does not exist

invalidateTable

public void invalidateTable(Identifier identifier)

Description copied from interface: Catalog

Invalidate cached table metadata for an identifier.

If the table is already loaded or cached, drop cached data. If the table does not exist or is not cached, do nothing. Calling this method should not query remote services.

Parameters:

identifier - a table identifier