public class SparkUtils extends Object
Modifier and Type | Class and Description |
---|---|
static class |
SparkUtils.CatalogAndIdentifier
This mimics a class inside of Spark which is private inside of LookupCatalog.
|
Constructor and Description |
---|
SparkUtils() |
Modifier and Type | Method and Description |
---|---|
static <C,T> Pair<C,T> |
catalogAndIdentifier(List<String> nameParts,
java.util.function.Function<String,C> catalogProvider,
java.util.function.BiFunction<String[],String,T> identifierProvider,
C currentCatalog,
String[] currentNamespace)
A modified version of Spark's LookupCatalog.CatalogAndIdentifier.unapply Attempts to find the
catalog and identifier a multipart identifier represents.
|
static SparkUtils.CatalogAndIdentifier |
catalogAndIdentifier(org.apache.spark.sql.SparkSession spark,
List<String> nameParts,
org.apache.spark.sql.connector.catalog.CatalogPlugin defaultCatalog)
A modified version of Spark's LookupCatalog.CatalogAndIdentifier.unapply Attempts to find the
catalog and identifier a multipart identifier represents.
|
static SparkUtils.CatalogAndIdentifier |
catalogAndIdentifier(org.apache.spark.sql.SparkSession spark,
String name,
org.apache.spark.sql.connector.catalog.CatalogPlugin defaultCatalog) |
static SparkUtils.CatalogAndIdentifier |
catalogAndIdentifier(String description,
org.apache.spark.sql.SparkSession spark,
String name,
org.apache.spark.sql.connector.catalog.CatalogPlugin defaultCatalog) |
static IOManager |
createIOManager() |
public static IOManager createIOManager()
public static <C,T> Pair<C,T> catalogAndIdentifier(List<String> nameParts, java.util.function.Function<String,C> catalogProvider, java.util.function.BiFunction<String[],String,T> identifierProvider, C currentCatalog, String[] currentNamespace)
nameParts
- Multipart identifier representing a tablepublic static SparkUtils.CatalogAndIdentifier catalogAndIdentifier(org.apache.spark.sql.SparkSession spark, List<String> nameParts, org.apache.spark.sql.connector.catalog.CatalogPlugin defaultCatalog)
spark
- Spark session to use for resolutionnameParts
- Multipart identifier representing a tabledefaultCatalog
- Catalog to use if none is specifiedpublic static SparkUtils.CatalogAndIdentifier catalogAndIdentifier(org.apache.spark.sql.SparkSession spark, String name, org.apache.spark.sql.connector.catalog.CatalogPlugin defaultCatalog) throws org.apache.spark.sql.catalyst.parser.ParseException
org.apache.spark.sql.catalyst.parser.ParseException
public static SparkUtils.CatalogAndIdentifier catalogAndIdentifier(String description, org.apache.spark.sql.SparkSession spark, String name, org.apache.spark.sql.connector.catalog.CatalogPlugin defaultCatalog)
Copyright © 2023–2024 The Apache Software Foundation. All rights reserved.