org.apache.paimon.catalog

Class CachingCatalog

java.lang.Object

org.apache.paimon.catalog.DelegateCatalog

org.apache.paimon.catalog.CachingCatalog

All Implemented Interfaces:

AutoCloseable, Catalog

public class CachingCatalog
extends DelegateCatalog

A Catalog to cache databases and tables and manifests.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	CachingCatalog.CacheSizes Cache sizes of a caching catalog.

Nested classes/interfaces inherited from interface org.apache.paimon.catalog.Catalog

Catalog.BranchAlreadyExistException, Catalog.BranchNotExistException, Catalog.ColumnAlreadyExistException, Catalog.ColumnNotExistException, Catalog.DatabaseAlreadyExistException, Catalog.DatabaseNoPermissionException, Catalog.DatabaseNotEmptyException, Catalog.DatabaseNotExistException, Catalog.DialectAlreadyExistException, Catalog.DialectNotExistException, Catalog.ProcessSystemDatabaseException, Catalog.TableAlreadyExistException, Catalog.TableNoPermissionException, Catalog.TableNotExistException, Catalog.TagNotExistException, Catalog.ViewAlreadyExistException, Catalog.ViewNotExistException

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.paimon.shade.caffeine2.com.github.benmanes.caffeine.cache.Cache<String,Database>	databaseCache
protected SegmentsCache<Path>	manifestCache
protected org.apache.paimon.shade.caffeine2.com.github.benmanes.caffeine.cache.Cache<Identifier,List<Partition>>	partitionCache
protected org.apache.paimon.shade.caffeine2.com.github.benmanes.caffeine.cache.Cache<Identifier,Table>	tableCache

Fields inherited from class org.apache.paimon.catalog.DelegateCatalog

wrapped

Fields inherited from interface org.apache.paimon.catalog.Catalog

COMMENT_PROP, DB_LOCATION_PROP, DB_SUFFIX, DEFAULT_DATABASE, LAST_UPDATE_TIME_PROP, NUM_FILES_PROP, NUM_ROWS_PROP, OWNER_PROP, SYSTEM_BRANCH_PREFIX, SYSTEM_DATABASE_NAME, SYSTEM_TABLE_SPLITTER, TABLE_DEFAULT_OPTION_PREFIX, TOTAL_SIZE_PROP

Constructor Summary

Constructors

Constructor and Description
CachingCatalog(Catalog wrapped, Options options)

Method Summary

Modifier and Type	Method and Description
void	alterDatabase(String name, List<PropertyChange> changes, boolean ignoreIfNotExists) Alter a database.
void	alterPartitions(Identifier identifier, List<PartitionStatistics> partitions) Alter partitions of the specify table.
void	alterTable(Identifier identifier, List<SchemaChange> changes, boolean ignoreIfNotExists) Modify an existing table from SchemaChanges.
CatalogLoader	catalogLoader() Serializable loader to create catalog.
void	dropDatabase(String name, boolean ignoreIfNotExists, boolean cascade) Drop a database.
void	dropPartitions(Identifier identifier, List<Map<String,String>> partitions) Drop partitions of the specify table.
void	dropTable(Identifier identifier, boolean ignoreIfNotExists) Drop a table.
CachingCatalog.CacheSizes	estimatedCacheSizes() Cache sizes for compute engine.
Database	getDatabase(String databaseName) Return a Database identified by the given name.
Table	getTable(Identifier identifier) Return a Table identified by the given Identifier.
void	invalidateTable(Identifier identifier) Invalidate cached table metadata for an identifier.
List<Partition>	listPartitions(Identifier identifier) Get Partition of all partitions of the table.
void	refreshPartitions(Identifier identifier) Partition cache will affect the latency of table, so refresh method is provided for compute engine.
void	renameTable(Identifier fromTable, Identifier toTable, boolean ignoreIfNotExists) Rename a table.
static Catalog	tryToCreate(Catalog catalog, Options options)

Methods inherited from class org.apache.paimon.catalog.DelegateCatalog

alterView, caseSensitive, close, commitSnapshot, createBranch, createDatabase, createPartitions, createTable, createView, dropBranch, dropView, fastForward, getView, listBranches, listDatabases, listDatabasesPaged, listPartitionsPaged, listTableDetailsPaged, listTables, listTablesPaged, listViewDetailsPaged, listViews, listViewsPaged, loadSnapshot, markDonePartitions, options, renameView, repairCatalog, repairDatabase, repairTable, rollbackTo, rootCatalog, supportsListObjectsPaged, supportsVersionManagement, wrapped

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.paimon.catalog.Catalog

alterTable, createDatabase

Field Detail

databaseCache

protected org.apache.paimon.shade.caffeine2.com.github.benmanes.caffeine.cache.Cache<String,Database> databaseCache

tableCache

protected org.apache.paimon.shade.caffeine2.com.github.benmanes.caffeine.cache.Cache<Identifier,Table> tableCache

manifestCache

@Nullable
protected final SegmentsCache<Path> manifestCache

partitionCache

@Nullable
protected org.apache.paimon.shade.caffeine2.com.github.benmanes.caffeine.cache.Cache<Identifier,List<Partition>> partitionCache

Constructor Detail

CachingCatalog

public CachingCatalog(Catalog wrapped,
                      Options options)

Method Detail

tryToCreate

public static Catalog tryToCreate(Catalog catalog,
                                  Options options)

catalogLoader

public CatalogLoader catalogLoader()

Description copied from interface: Catalog

Serializable loader to create catalog.

getDatabase

public Database getDatabase(String databaseName)
                     throws Catalog.DatabaseNotExistException

Description copied from interface: Catalog

Return a Database identified by the given name.

Specified by:

getDatabase in interface Catalog

Overrides:

getDatabase in class DelegateCatalog

Parameters:

databaseName - Database name

Returns:

The requested Database

Throws:

Catalog.DatabaseNotExistException - if the requested database does not exist

dropDatabase

public void dropDatabase(String name,
                         boolean ignoreIfNotExists,
                         boolean cascade)
                  throws Catalog.DatabaseNotExistException,
                         Catalog.DatabaseNotEmptyException

Description copied from interface: Catalog

Drop a database.

Specified by:

dropDatabase in interface Catalog

Overrides:

dropDatabase in class DelegateCatalog

Parameters:

name - Name of the database to be dropped.

ignoreIfNotExists - Flag to specify behavior when the database does not exist: if set to false, throw an exception, if set to true, do nothing.

cascade - Flag to specify behavior when the database contains table or function: if set to true, delete all tables and functions in the database and then delete the database, if set to false, throw an exception.

Throws:

Catalog.DatabaseNotEmptyException - if the given database is not empty and isRestrict is true

Catalog.DatabaseNotExistException

alterDatabase

public void alterDatabase(String name,
                          List<PropertyChange> changes,
                          boolean ignoreIfNotExists)
                   throws Catalog.DatabaseNotExistException

Description copied from interface: Catalog

Alter a database.

Specified by:

alterDatabase in interface Catalog

Overrides:

alterDatabase in class DelegateCatalog

Parameters:

name - Name of the database to alter.

changes - the property changes

ignoreIfNotExists - Flag to specify behavior when the database does not exist: if set to false, throw an exception, if set to true, do nothing.

Throws:

Catalog.DatabaseNotExistException - if the given database is not exist and ignoreIfNotExists is false

dropTable

public void dropTable(Identifier identifier,
                      boolean ignoreIfNotExists)
               throws Catalog.TableNotExistException

Description copied from interface: Catalog

Drop a table.

NOTE: System tables can not be dropped.

Specified by:

dropTable in interface Catalog

Overrides:

dropTable in class DelegateCatalog

Parameters:

identifier - Path of the table to be dropped

ignoreIfNotExists - Flag to specify behavior when the table does not exist: if set to false, throw an exception, if set to true, do nothing.

Throws:

Catalog.TableNotExistException - if the table does not exist

renameTable

public void renameTable(Identifier fromTable,
                        Identifier toTable,
                        boolean ignoreIfNotExists)
                 throws Catalog.TableNotExistException,
                        Catalog.TableAlreadyExistException

Description copied from interface: Catalog

Rename a table.

NOTE: If you use object storage, such as S3 or OSS, please use this syntax carefully, because the renaming of object storage is not atomic, and only partial files may be moved in case of failure.

NOTE: System tables can not be renamed.

Specified by:

renameTable in interface Catalog

Overrides:

renameTable in class DelegateCatalog

Parameters:

fromTable - the name of the table which need to rename

toTable - the new table

ignoreIfNotExists - Flag to specify behavior when the table does not exist: if set to false, throw an exception, if set to true, do nothing.

Throws:

Catalog.TableNotExistException - if the fromTable does not exist

Catalog.TableAlreadyExistException - if the toTable already exists

alterTable

public void alterTable(Identifier identifier,
                       List<SchemaChange> changes,
                       boolean ignoreIfNotExists)
                throws Catalog.TableNotExistException,
                       Catalog.ColumnAlreadyExistException,
                       Catalog.ColumnNotExistException

Description copied from interface: Catalog

Modify an existing table from SchemaChanges.

NOTE: System tables can not be altered.

Specified by:

alterTable in interface Catalog

Overrides:

alterTable in class DelegateCatalog

Parameters:

identifier - path of the table to be modified

changes - the schema changes

ignoreIfNotExists - flag to specify behavior when the table does not exist: if set to false, throw an exception, if set to true, do nothing.

Throws:

Catalog.TableNotExistException - if the table does not exist

Catalog.ColumnAlreadyExistException

Catalog.ColumnNotExistException

getTable

public Table getTable(Identifier identifier)
               throws Catalog.TableNotExistException

Description copied from interface: Catalog

Return a Table identified by the given Identifier.

System tables can be got by '$' splitter.

Specified by:

getTable in interface Catalog

Overrides:

getTable in class DelegateCatalog

Parameters:

identifier - Path of the table

Returns:

The requested table

Throws:

Catalog.TableNotExistException - if the target does not exist

listPartitions

public List<Partition> listPartitions(Identifier identifier)
                               throws Catalog.TableNotExistException

Description copied from interface: Catalog

Get Partition of all partitions of the table.

Specified by:

listPartitions in interface Catalog

Overrides:

listPartitions in class DelegateCatalog

Parameters:

identifier - path of the table to list partitions

Throws:

Catalog.TableNotExistException - if the table does not exist

dropPartitions

public void dropPartitions(Identifier identifier,
                           List<Map<String,String>> partitions)
                    throws Catalog.TableNotExistException

Description copied from interface: Catalog

Drop partitions of the specify table. Ignore non-existent partitions.

Specified by:

dropPartitions in interface Catalog

Overrides:

dropPartitions in class DelegateCatalog

Parameters:

identifier - path of the table to drop partitions

partitions - partitions to be deleted

Throws:

Catalog.TableNotExistException - if the table does not exist

alterPartitions

public void alterPartitions(Identifier identifier,
                            List<PartitionStatistics> partitions)
                     throws Catalog.TableNotExistException

Description copied from interface: Catalog

Alter partitions of the specify table. For non-existent partitions, partitions will be created directly.

Specified by:

alterPartitions in interface Catalog

Overrides:

alterPartitions in class DelegateCatalog

Parameters:

identifier - path of the table to alter partitions

partitions - partitions to be altered

Throws:

Catalog.TableNotExistException - if the table does not exist

invalidateTable

public void invalidateTable(Identifier identifier)

Description copied from interface: Catalog

Invalidate cached table metadata for an identifier.

If the table is already loaded or cached, drop cached data. If the table does not exist or is not cached, do nothing. Calling this method should not query remote services.

Parameters:

identifier - a table identifier

refreshPartitions

public void refreshPartitions(Identifier identifier)
                       throws Catalog.TableNotExistException

Partition cache will affect the latency of table, so refresh method is provided for compute engine.

Throws:

Catalog.TableNotExistException

estimatedCacheSizes

public CachingCatalog.CacheSizes estimatedCacheSizes()

Cache sizes for compute engine. This method can let the outside know the specific usage of cache.