org.apache.paimon.flink.sink

Class FlinkSinkBuilder

java.lang.Object

org.apache.paimon.flink.sink.FlinkSinkBuilder

Direct Known Subclasses:

LogFlinkSinkBuilder, SortCompactSinkBuilder

@Public
public class FlinkSinkBuilder
extends Object

DataStream API for building Flink Sink.

Since:

0.8

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	compactSink
protected LogSinkFunction	logSinkFunction
protected Map<String,String>	overwritePartition
protected Integer	parallelism
protected FileStoreTable	table

Constructor Summary

Constructors

Constructor and Description
FlinkSinkBuilder(Table table)

Method Summary

Modifier and Type	Method and Description
org.apache.flink.streaming.api.datastream.DataStreamSink<?>	build() Build DataStreamSink.
protected org.apache.flink.streaming.api.datastream.DataStreamSink<?>	buildDynamicBucketSink(org.apache.flink.streaming.api.datastream.DataStream<InternalRow> input, boolean globalIndex)
protected org.apache.flink.streaming.api.datastream.DataStreamSink<?>	buildForFixedBucket(org.apache.flink.streaming.api.datastream.DataStream<InternalRow> input)
FlinkSinkBuilder	clusteringIfPossible(String clusteringColumns, String clusteringStrategy, boolean sortInCluster, int sampleFactor) Clustering the input data if possible.
FlinkSinkBuilder	forRow(org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.types.Row> input, org.apache.flink.table.types.DataType rowDataType) From DataStream with Row, need to provide a DataType for builder to convert those Rows to a RowData DataStream.
FlinkSinkBuilder	forRowData(org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.table.data.RowData> input) From DataStream with RowData.
protected org.apache.flink.streaming.api.datastream.DataStream<InternalRow>	mapToInternalRow(org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.table.data.RowData> input, RowType rowType)
FlinkSinkBuilder	overwrite() INSERT OVERWRITE.
FlinkSinkBuilder	overwrite(Map<String,String> overwritePartition) INSERT OVERWRITE PARTITION (...).
FlinkSinkBuilder	parallelism(int parallelism) Set sink parallelism.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

table

protected final FileStoreTable table

overwritePartition

@Nullable
protected Map<String,String> overwritePartition

parallelism

@Nullable
protected Integer parallelism

compactSink

protected boolean compactSink

logSinkFunction

@Nullable
protected LogSinkFunction logSinkFunction

Constructor Detail

FlinkSinkBuilder

public FlinkSinkBuilder(Table table)

Method Detail

forRow

public FlinkSinkBuilder forRow(org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.types.Row> input,
                               org.apache.flink.table.types.DataType rowDataType)

From DataStream with Row, need to provide a DataType for builder to convert those Rows to a RowData DataStream.

forRowData

public FlinkSinkBuilder forRowData(org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.table.data.RowData> input)

From DataStream with RowData.

overwrite

public FlinkSinkBuilder overwrite()

INSERT OVERWRITE.

overwrite

public FlinkSinkBuilder overwrite(Map<String,String> overwritePartition)

INSERT OVERWRITE PARTITION (...).

parallelism

public FlinkSinkBuilder parallelism(int parallelism)

Set sink parallelism.

clusteringIfPossible

public FlinkSinkBuilder clusteringIfPossible(String clusteringColumns,
                                             String clusteringStrategy,
                                             boolean sortInCluster,
                                             int sampleFactor)

Clustering the input data if possible.

build

public org.apache.flink.streaming.api.datastream.DataStreamSink<?> build()

Build DataStreamSink.

mapToInternalRow

protected org.apache.flink.streaming.api.datastream.DataStream<InternalRow> mapToInternalRow(org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.table.data.RowData> input,
                                                                                             RowType rowType)

buildDynamicBucketSink

protected org.apache.flink.streaming.api.datastream.DataStreamSink<?> buildDynamicBucketSink(org.apache.flink.streaming.api.datastream.DataStream<InternalRow> input,
                                                                                             boolean globalIndex)

buildForFixedBucket

protected org.apache.flink.streaming.api.datastream.DataStreamSink<?> buildForFixedBucket(org.apache.flink.streaming.api.datastream.DataStream<InternalRow> input)