public class RESTCatalog extends Object implements Catalog, SupportsSnapshots, SupportsBranches
SupportsBranches.BranchAlreadyExistException, SupportsBranches.BranchNotExistException, SupportsBranches.TagNotExistException
Catalog.ColumnAlreadyExistException, Catalog.ColumnNotExistException, Catalog.DatabaseAlreadyExistException, Catalog.DatabaseNoPermissionException, Catalog.DatabaseNotEmptyException, Catalog.DatabaseNotExistException, Catalog.ProcessSystemDatabaseException, Catalog.TableAlreadyExistException, Catalog.TableNoPermissionException, Catalog.TableNotExistException, Catalog.ViewAlreadyExistException, Catalog.ViewNotExistException
Modifier and Type | Field and Description |
---|---|
static String |
HEADER_PREFIX |
static String |
MAX_RESULTS |
static String |
PAGE_TOKEN |
static String |
QUERY_PARAMETER_WAREHOUSE_KEY |
COMMENT_PROP, DB_LOCATION_PROP, DB_SUFFIX, DEFAULT_DATABASE, LAST_UPDATE_TIME_PROP, NUM_FILES_PROP, NUM_ROWS_PROP, OWNER_PROP, SYSTEM_BRANCH_PREFIX, SYSTEM_DATABASE_NAME, SYSTEM_TABLE_SPLITTER, TABLE_DEFAULT_OPTION_PREFIX, TOTAL_SIZE_PROP
Constructor and Description |
---|
RESTCatalog(CatalogContext context) |
RESTCatalog(CatalogContext context,
boolean configRequired) |
Modifier and Type | Method and Description |
---|---|
void |
alterDatabase(String name,
List<PropertyChange> changes,
boolean ignoreIfNotExists)
Alter a database.
|
void |
alterPartitions(Identifier identifier,
List<Partition> partitions)
Alter partitions of the specify table.
|
void |
alterTable(Identifier identifier,
List<SchemaChange> changes,
boolean ignoreIfNotExists)
Modify an existing table from
SchemaChange s. |
boolean |
caseSensitive()
Return a boolean that indicates whether this catalog is case-sensitive.
|
RESTCatalogLoader |
catalogLoader()
Serializable loader to create catalog.
|
void |
close() |
boolean |
commitSnapshot(Identifier identifier,
Snapshot snapshot,
List<Partition> statistics)
Commit the
Snapshot for table identified by the given Identifier . |
void |
createBranch(Identifier identifier,
String branch,
String fromTag)
Create a new branch for this table.
|
void |
createDatabase(String name,
boolean ignoreIfExists,
Map<String,String> properties)
Create a database with properties.
|
void |
createPartitions(Identifier identifier,
List<Map<String,String>> partitions)
Create partitions of the specify table.
|
void |
createTable(Identifier identifier,
Schema schema,
boolean ignoreIfExists)
Create a new table.
|
void |
createView(Identifier identifier,
View view,
boolean ignoreIfExists)
Create a new view.
|
void |
dropBranch(Identifier identifier,
String branch)
Drop the branch for this table.
|
void |
dropDatabase(String name,
boolean ignoreIfNotExists,
boolean cascade)
Drop a database.
|
void |
dropPartitions(Identifier identifier,
List<Map<String,String>> partitions)
Drop partitions of the specify table.
|
void |
dropTable(Identifier identifier,
boolean ignoreIfNotExists)
Drop a table.
|
void |
dropView(Identifier identifier,
boolean ignoreIfNotExists)
Drop a view.
|
void |
fastForward(Identifier identifier,
String branch)
Fast-forward a branch to main branch.
|
Database |
getDatabase(String name)
Return a
Database identified by the given name. |
Table |
getTable(Identifier identifier)
Return a
Table identified by the given Identifier . |
View |
getView(Identifier identifier)
Return a
View identified by the given Identifier . |
List<String> |
listBranches(Identifier identifier)
List all branches of the table.
|
List<String> |
listDatabases()
Get the names of all databases in this catalog.
|
PagedList<String> |
listDatabasesPaged(Integer maxResults,
String pageToken)
Get paged list names of all databases in this catalog.
|
protected <T> List<T> |
listDataFromPageApi(java.util.function.Function<Map<String,String>,PagedResponse<T>> pageApi) |
List<Partition> |
listPartitions(Identifier identifier)
Get Partition of all partitions of the table.
|
PagedList<Partition> |
listPartitionsPaged(Identifier identifier,
Integer maxResults,
String pageToken)
Get paged partitioned list of the table.
|
PagedList<Table> |
listTableDetailsPaged(String db,
Integer maxResults,
String pageToken)
Get paged list of table details under this database.
|
List<String> |
listTables(String databaseName)
Get names of all tables under this database.
|
PagedList<String> |
listTablesPaged(String databaseName,
Integer maxResults,
String pageToken)
Get paged list names of tables under this database.
|
PagedList<View> |
listViewDetailsPaged(String db,
Integer maxResults,
String pageToken)
Get paged list view details under this database.
|
List<String> |
listViews(String databaseName)
Get names of all views under this database.
|
PagedList<String> |
listViewsPaged(String databaseName,
Integer maxResults,
String pageToken)
Get paged list names of views under this database.
|
Optional<TableSnapshot> |
loadSnapshot(Identifier identifier)
Return the snapshot of table identified by the given
Identifier . |
protected GetTableTokenResponse |
loadTableToken(Identifier identifier) |
void |
markDonePartitions(Identifier identifier,
List<Map<String,String>> partitions)
Mark partitions done of the specify table.
|
Map<String,String> |
options()
Catalog options for re-creating this catalog.
|
void |
renameTable(Identifier fromTable,
Identifier toTable,
boolean ignoreIfNotExists)
Rename a table.
|
void |
renameView(Identifier fromView,
Identifier toView,
boolean ignoreIfNotExists)
Rename a view.
|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
alterTable, createDatabase, invalidateTable, repairCatalog, repairDatabase, repairTable
public static final String HEADER_PREFIX
public static final String MAX_RESULTS
public static final String PAGE_TOKEN
public static final String QUERY_PARAMETER_WAREHOUSE_KEY
public RESTCatalog(CatalogContext context)
public RESTCatalog(CatalogContext context, boolean configRequired)
public Map<String,String> options()
Catalog
public RESTCatalogLoader catalogLoader()
Catalog
catalogLoader
in interface Catalog
public List<String> listDatabases()
Catalog
listDatabases
in interface Catalog
public PagedList<String> listDatabasesPaged(Integer maxResults, String pageToken)
Catalog
NOTE: Currently only RestCatalog will return pagedList data, other catalog will return all databases names like listDatabases.
listDatabasesPaged
in interface Catalog
maxResults
- Optional parameter indicating the maximum number of results to include in
the result. If maxResults is not specified or set to 0, will return the default number of
max results.pageToken
- Optional parameter indicating the next page token allows list to be start
from a specific point.public void createDatabase(String name, boolean ignoreIfExists, Map<String,String> properties) throws Catalog.DatabaseAlreadyExistException
Catalog
createDatabase
in interface Catalog
name
- Name of the database to be createdignoreIfExists
- Flag to specify behavior when a database with the given name already
exists: if set to false, throw a DatabaseAlreadyExistException, if set to true, do
nothing.properties
- properties to be associated with the databaseCatalog.DatabaseAlreadyExistException
- if the given database already exists and ignoreIfExists
is falsepublic Database getDatabase(String name) throws Catalog.DatabaseNotExistException
Catalog
Database
identified by the given name.getDatabase
in interface Catalog
name
- Database nameDatabase
Catalog.DatabaseNotExistException
- if the requested database does not existpublic void dropDatabase(String name, boolean ignoreIfNotExists, boolean cascade) throws Catalog.DatabaseNotExistException, Catalog.DatabaseNotEmptyException
Catalog
dropDatabase
in interface Catalog
name
- Name of the database to be dropped.ignoreIfNotExists
- Flag to specify behavior when the database does not exist: if set to
false, throw an exception, if set to true, do nothing.cascade
- Flag to specify behavior when the database contains table or function: if set
to true, delete all tables and functions in the database and then delete the database, if
set to false, throw an exception.Catalog.DatabaseNotEmptyException
- if the given database is not empty and isRestrict is trueCatalog.DatabaseNotExistException
public void alterDatabase(String name, List<PropertyChange> changes, boolean ignoreIfNotExists) throws Catalog.DatabaseNotExistException
Catalog
alterDatabase
in interface Catalog
name
- Name of the database to alter.changes
- the property changesignoreIfNotExists
- Flag to specify behavior when the database does not exist: if set to
false, throw an exception, if set to true, do nothing.Catalog.DatabaseNotExistException
- if the given database is not exist and ignoreIfNotExists is
falsepublic List<String> listTables(String databaseName) throws Catalog.DatabaseNotExistException
Catalog
NOTE: System tables will not be listed.
listTables
in interface Catalog
Catalog.DatabaseNotExistException
- if the database does not existpublic PagedList<String> listTablesPaged(String databaseName, @Nullable Integer maxResults, @Nullable String pageToken) throws Catalog.DatabaseNotExistException
Catalog
NOTE: Currently only RestCatalog will return pagedList data, other catalog will return all tables names like listTables.
NOTE: System tables will not be listed.
listTablesPaged
in interface Catalog
databaseName
- Name of the database to list tables.maxResults
- Optional parameter indicating the maximum number of results to include in
the result. If maxResults is not specified or set to 0, will return the default number of
max results.pageToken
- Optional parameter indicating the next page token allows list to be start
from a specific point.Catalog.DatabaseNotExistException
- if the database does not existpublic PagedList<Table> listTableDetailsPaged(String db, @Nullable Integer maxResults, @Nullable String pageToken) throws Catalog.DatabaseNotExistException
Catalog
NOTE: Currently only RestCatalog will return pagedList data, other catalog will return all table details, please use this method carefully for other catalog.
NOTE: System tables will not be listed.
listTableDetailsPaged
in interface Catalog
db
- Name of the database to list table details.maxResults
- Optional parameter indicating the maximum number of results to include in
the result. If maxResults is not specified or set to 0, will return the default number of
max results.pageToken
- Optional parameter indicating the next page token allows list to be start
from a specific point.Catalog.DatabaseNotExistException
- if the database does not existpublic Table getTable(Identifier identifier) throws Catalog.TableNotExistException
Catalog
getTable
in interface Catalog
identifier
- Path of the tableCatalog.TableNotExistException
- if the target does not existpublic Optional<TableSnapshot> loadSnapshot(Identifier identifier) throws Catalog.TableNotExistException
SupportsSnapshots
Identifier
.loadSnapshot
in interface SupportsSnapshots
identifier
- Path of the tableCatalog.TableNotExistException
- if the target does not existpublic boolean commitSnapshot(Identifier identifier, Snapshot snapshot, List<Partition> statistics) throws Catalog.TableNotExistException
SupportsSnapshots
Snapshot
for table identified by the given Identifier
.commitSnapshot
in interface SupportsSnapshots
identifier
- Path of the tablesnapshot
- Snapshot to be committedstatistics
- statistics information of this changeCatalog.TableNotExistException
- if the target does not existpublic void createTable(Identifier identifier, Schema schema, boolean ignoreIfExists) throws Catalog.TableAlreadyExistException, Catalog.DatabaseNotExistException
Catalog
NOTE: System tables can not be created.
createTable
in interface Catalog
identifier
- path of the table to be createdschema
- the table definitionignoreIfExists
- flag to specify behavior when a table already exists at the given path:
if set to false, it throws a TableAlreadyExistException, if set to true, do nothing.Catalog.TableAlreadyExistException
- if table already exists and ignoreIfExists is falseCatalog.DatabaseNotExistException
- if the database in identifier doesn't existpublic void renameTable(Identifier fromTable, Identifier toTable, boolean ignoreIfNotExists) throws Catalog.TableNotExistException, Catalog.TableAlreadyExistException
Catalog
NOTE: If you use object storage, such as S3 or OSS, please use this syntax carefully, because the renaming of object storage is not atomic, and only partial files may be moved in case of failure.
NOTE: System tables can not be renamed.
renameTable
in interface Catalog
fromTable
- the name of the table which need to renametoTable
- the new tableignoreIfNotExists
- Flag to specify behavior when the table does not exist: if set to
false, throw an exception, if set to true, do nothing.Catalog.TableNotExistException
- if the fromTable does not existCatalog.TableAlreadyExistException
- if the toTable already existspublic void alterTable(Identifier identifier, List<SchemaChange> changes, boolean ignoreIfNotExists) throws Catalog.TableNotExistException, Catalog.ColumnAlreadyExistException, Catalog.ColumnNotExistException
Catalog
SchemaChange
s.
NOTE: System tables can not be altered.
alterTable
in interface Catalog
identifier
- path of the table to be modifiedchanges
- the schema changesignoreIfNotExists
- flag to specify behavior when the table does not exist: if set to
false, throw an exception, if set to true, do nothing.Catalog.TableNotExistException
- if the table does not existCatalog.ColumnAlreadyExistException
Catalog.ColumnNotExistException
public void dropTable(Identifier identifier, boolean ignoreIfNotExists) throws Catalog.TableNotExistException
Catalog
NOTE: System tables can not be dropped.
dropTable
in interface Catalog
identifier
- Path of the table to be droppedignoreIfNotExists
- Flag to specify behavior when the table does not exist: if set to
false, throw an exception, if set to true, do nothing.Catalog.TableNotExistException
- if the table does not existpublic void createPartitions(Identifier identifier, List<Map<String,String>> partitions) throws Catalog.TableNotExistException
Catalog
createPartitions
in interface Catalog
identifier
- path of the table to create partitionspartitions
- partitions to be createdCatalog.TableNotExistException
- if the table does not existpublic void dropPartitions(Identifier identifier, List<Map<String,String>> partitions) throws Catalog.TableNotExistException
Catalog
dropPartitions
in interface Catalog
identifier
- path of the table to drop partitionspartitions
- partitions to be deletedCatalog.TableNotExistException
- if the table does not existpublic void alterPartitions(Identifier identifier, List<Partition> partitions) throws Catalog.TableNotExistException
Catalog
alterPartitions
in interface Catalog
identifier
- path of the table to alter partitionspartitions
- partitions to be alteredCatalog.TableNotExistException
- if the table does not existpublic void markDonePartitions(Identifier identifier, List<Map<String,String>> partitions) throws Catalog.TableNotExistException
Catalog
markDonePartitions
in interface Catalog
identifier
- path of the table to mark done partitionspartitions
- partitions to be marked doneCatalog.TableNotExistException
- if the table does not existpublic List<Partition> listPartitions(Identifier identifier) throws Catalog.TableNotExistException
Catalog
listPartitions
in interface Catalog
identifier
- path of the table to list partitionsCatalog.TableNotExistException
- if the table does not existpublic void createBranch(Identifier identifier, String branch, @Nullable String fromTag) throws Catalog.TableNotExistException, SupportsBranches.BranchAlreadyExistException, SupportsBranches.TagNotExistException
SupportsBranches
#fromTag
, a branch will be created from the tag and the
data files will be inherited from it.createBranch
in interface SupportsBranches
identifier
- path of the table, cannot be system or branch name.branch
- the branch namefromTag
- from the tagSupportsBranches.BranchAlreadyExistException
- if the branch already existsSupportsBranches.TagNotExistException
- if the tag doesn't existCatalog.TableNotExistException
public void dropBranch(Identifier identifier, String branch) throws SupportsBranches.BranchNotExistException
SupportsBranches
dropBranch
in interface SupportsBranches
identifier
- path of the table, cannot be system or branch name.branch
- the branch nameSupportsBranches.BranchNotExistException
- if the branch doesn't existpublic void fastForward(Identifier identifier, String branch) throws SupportsBranches.BranchNotExistException
SupportsBranches
fastForward
in interface SupportsBranches
identifier
- path of the table, cannot be system or branch name.branch
- the branch nameSupportsBranches.BranchNotExistException
- if the branch doesn't existpublic List<String> listBranches(Identifier identifier) throws Catalog.TableNotExistException
SupportsBranches
listBranches
in interface SupportsBranches
identifier
- path of the table, cannot be system or branch name.Catalog.TableNotExistException
public PagedList<Partition> listPartitionsPaged(Identifier identifier, @Nullable Integer maxResults, @Nullable String pageToken) throws Catalog.TableNotExistException
Catalog
NOTE: Currently only RestCatalog will return pagedList data, other catalog will return all partitions.
listPartitionsPaged
in interface Catalog
identifier
- path of the table to list partitionsmaxResults
- Optional parameter indicating the maximum number of results to include in
the result. If maxResults is not specified or set to 0, will return the default number of
max results.pageToken
- Optional parameter indicating the next page token allows list to be start
from a specific point.Catalog.TableNotExistException
- if the table does not existpublic View getView(Identifier identifier) throws Catalog.ViewNotExistException
Catalog
View
identified by the given Identifier
.getView
in interface Catalog
identifier
- Path of the viewCatalog.ViewNotExistException
- if the target does not existpublic void dropView(Identifier identifier, boolean ignoreIfNotExists) throws Catalog.ViewNotExistException
Catalog
dropView
in interface Catalog
identifier
- Path of the view to be droppedignoreIfNotExists
- Flag to specify behavior when the view does not exist: if set to
false, throw an exception, if set to true, do nothing.Catalog.ViewNotExistException
- if the view does not existpublic void createView(Identifier identifier, View view, boolean ignoreIfExists) throws Catalog.ViewAlreadyExistException, Catalog.DatabaseNotExistException
Catalog
createView
in interface Catalog
identifier
- path of the view to be createdview
- the view definitionignoreIfExists
- flag to specify behavior when a view already exists at the given path:
if set to false, it throws a ViewAlreadyExistException, if set to true, do nothing.Catalog.ViewAlreadyExistException
- if view already exists and ignoreIfExists is falseCatalog.DatabaseNotExistException
- if the database in identifier doesn't existpublic List<String> listViews(String databaseName) throws Catalog.DatabaseNotExistException
Catalog
listViews
in interface Catalog
Catalog.DatabaseNotExistException
- if the database does not existpublic PagedList<String> listViewsPaged(String databaseName, @Nullable Integer maxResults, @Nullable String pageToken) throws Catalog.DatabaseNotExistException
Catalog
NOTE: Currently only RestCatalog will return pagedList data, other catalog will return all view names, like listViews.
listViewsPaged
in interface Catalog
databaseName
- Name of the database to list views.maxResults
- Optional parameter indicating the maximum number of results to include in
the result. If maxResults is not specified or set to 0, will return the default number of
max results.pageToken
- Optional parameter indicating the next page token allows list to be start
from a specific point.Catalog.DatabaseNotExistException
- if the database does not existpublic PagedList<View> listViewDetailsPaged(String db, @Nullable Integer maxResults, @Nullable String pageToken) throws Catalog.DatabaseNotExistException
Catalog
NOTE: Currently only RestCatalog will return pagedList data, other catalog which supports view will return all view details, please use this method carefully for other catalog which supports view.
listViewDetailsPaged
in interface Catalog
db
- Name of the database to list views.maxResults
- Optional parameter indicating the maximum number of results to include in
the result. If maxResults is not specified or set to 0, will return the default number of
max results.pageToken
- Optional parameter indicating the next page token allows list to be start
from a specific point.Catalog.DatabaseNotExistException
- if the database does not existpublic void renameView(Identifier fromView, Identifier toView, boolean ignoreIfNotExists) throws Catalog.ViewNotExistException, Catalog.ViewAlreadyExistException
Catalog
renameView
in interface Catalog
fromView
- identifier of the view to renametoView
- new view identifierCatalog.ViewNotExistException
- if the fromView does not existCatalog.ViewAlreadyExistException
- if the toView already existspublic boolean caseSensitive()
Catalog
caseSensitive
in interface Catalog
public void close() throws Exception
close
in interface AutoCloseable
Exception
protected GetTableTokenResponse loadTableToken(Identifier identifier) throws Catalog.TableNotExistException
Catalog.TableNotExistException
protected <T> List<T> listDataFromPageApi(java.util.function.Function<Map<String,String>,PagedResponse<T>> pageApi)
Copyright © 2023–2025 The Apache Software Foundation. All rights reserved.