org.apache.paimon.spark.procedure

Class MarkPartitionDoneProcedure

java.lang.Object

org.apache.paimon.spark.procedure.MarkPartitionDoneProcedure

All Implemented Interfaces:

Procedure

public class MarkPartitionDoneProcedure
extends Object

Partition mark done procedure. Usage:

  CALL sys.mark_partition_done('tableId', 'partition1;partition2')
 

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	BaseProcedure.Builder<T extends org.apache.paimon.spark.procedure.BaseProcedure>

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	MarkPartitionDoneProcedure(org.apache.spark.sql.connector.catalog.TableCatalog tableCatalog)

Method Summary

Modifier and Type	Method and Description
static ProcedureBuilder	builder()
org.apache.spark.sql.catalyst.InternalRow[]	call(org.apache.spark.sql.catalyst.InternalRow args) Executes the given stored procedure.
protected org.apache.spark.sql.execution.datasources.v2.DataSourceV2Relation	createRelation(org.apache.spark.sql.connector.catalog.Identifier ident)
String	description() Returns the description of stored procedure.
protected org.apache.paimon.spark.SparkTable	loadSparkTable(org.apache.spark.sql.connector.catalog.Identifier ident)
static void	markDone(List<String> partitions, List<PartitionMarkDoneAction> actions)
protected <T> T	modifyPaimonTable(org.apache.spark.sql.connector.catalog.Identifier ident, java.util.function.Function<Table,T> func)
protected org.apache.spark.sql.catalyst.InternalRow	newInternalRow(Object... values)
org.apache.spark.sql.types.StructType	outputType() Returns the type of rows produced by stored procedure.
ProcedureParameter[]	parameters() Returns the input parameters of stored procedure.
protected void	refreshSparkCache(org.apache.spark.sql.connector.catalog.Identifier ident, org.apache.spark.sql.connector.catalog.Table table)
protected org.apache.spark.sql.SparkSession	spark()
protected org.apache.spark.sql.connector.catalog.TableCatalog	tableCatalog()
protected SparkUtils.CatalogAndIdentifier	toCatalogAndIdentifier(String identifierAsString, String argName, org.apache.spark.sql.connector.catalog.CatalogPlugin catalog)
protected org.apache.spark.sql.connector.catalog.Identifier	toIdentifier(String identifierAsString, String argName)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MarkPartitionDoneProcedure

protected MarkPartitionDoneProcedure(org.apache.spark.sql.connector.catalog.TableCatalog tableCatalog)

Method Detail

parameters

public ProcedureParameter[] parameters()

Description copied from interface: Procedure

Returns the input parameters of stored procedure.

outputType

public org.apache.spark.sql.types.StructType outputType()

Description copied from interface: Procedure

Returns the type of rows produced by stored procedure.

call

public org.apache.spark.sql.catalyst.InternalRow[] call(org.apache.spark.sql.catalyst.InternalRow args)

Description copied from interface: Procedure

Executes the given stored procedure.

Parameters:

args - Input arguments.

Returns:

The result of executing stored procedure with the given arguments.

markDone

public static void markDone(List<String> partitions,
                            List<PartitionMarkDoneAction> actions)

builder

public static ProcedureBuilder builder()

description

public String description()

Description copied from interface: Procedure

Returns the description of stored procedure.

toIdentifier

protected org.apache.spark.sql.connector.catalog.Identifier toIdentifier(String identifierAsString,
                                                                         String argName)

toCatalogAndIdentifier

protected SparkUtils.CatalogAndIdentifier toCatalogAndIdentifier(String identifierAsString,
                                                                 String argName,
                                                                 org.apache.spark.sql.connector.catalog.CatalogPlugin catalog)

modifyPaimonTable

protected <T> T modifyPaimonTable(org.apache.spark.sql.connector.catalog.Identifier ident,
                                  java.util.function.Function<Table,T> func)

loadSparkTable

protected org.apache.paimon.spark.SparkTable loadSparkTable(org.apache.spark.sql.connector.catalog.Identifier ident)

createRelation

protected org.apache.spark.sql.execution.datasources.v2.DataSourceV2Relation createRelation(org.apache.spark.sql.connector.catalog.Identifier ident)

refreshSparkCache

protected void refreshSparkCache(org.apache.spark.sql.connector.catalog.Identifier ident,
                                 org.apache.spark.sql.connector.catalog.Table table)

newInternalRow

protected org.apache.spark.sql.catalyst.InternalRow newInternalRow(Object... values)

spark

protected org.apache.spark.sql.SparkSession spark()

tableCatalog

protected org.apache.spark.sql.connector.catalog.TableCatalog tableCatalog()