T
- type of record to write.public interface RecordWriter<T>
RecordWriter
is responsible for writing data and handling in-progress files used to
write yet un-staged data. The incremental files ready to commit is returned to the system by the
prepareCommit(boolean)
.Modifier and Type | Method and Description |
---|---|
void |
addNewFiles(List<DataFileMeta> files)
Add files to the internal
CompactManager . |
void |
close()
Close this writer, the call will delete newly generated but not committed files.
|
void |
compact(boolean fullCompaction)
Compact files related to the writer.
|
Collection<DataFileMeta> |
dataFiles()
Get all data files maintained by this writer.
|
boolean |
isCompacting()
Check if a compaction is in progress, or if a compaction result remains to be fetched.
|
long |
maxSequenceNumber()
Get max sequence number of records written by this writer.
|
CommitIncrement |
prepareCommit(boolean waitCompaction)
Prepare for a commit.
|
void |
sync()
Sync the writer.
|
void |
withInsertOnly(boolean insertOnly)
This method is called when the insert only status of the records changes.
|
void |
write(T record)
Add a key-value element to the writer.
|
void write(T record) throws Exception
Exception
void compact(boolean fullCompaction) throws Exception
fullCompaction
- whether to trigger full compaction or just normal compactionException
void addNewFiles(List<DataFileMeta> files)
CompactManager
.files
- files to addCollection<DataFileMeta> dataFiles()
long maxSequenceNumber()
CommitIncrement prepareCommit(boolean waitCompaction) throws Exception
waitCompaction
- if this method need to wait for current compaction to completeException
boolean isCompacting()
void sync() throws Exception
Exception
void withInsertOnly(boolean insertOnly)
insertOnly
- If true, all the following records would be of RowKind.INSERT
, and no two records would have the same primary
key.Copyright © 2023–2024 The Apache Software Foundation. All rights reserved.