org.apache.paimon.hash

Class BytesMap<K,V>

java.lang.Object

org.apache.paimon.hash.BytesMap<K,V>

Type Parameters:

K - type of the map key.

V - type of the map value.

Direct Known Subclasses:

BytesHashMap

public abstract class BytesMap<K,V>
extends Object

Base class for BytesHashMap.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BytesMap.LookupInfo<K,V> Result fetched when looking up a key.

Field Summary

Fields

Modifier and Type	Field and Description
static int	BUCKET_SIZE
protected static int	BUCKET_SIZE_BITS
protected List<MemorySegment>	bucketSegments
protected static int	ELEMENT_POINT_LENGTH
protected static int	END_OF_LIST
protected int	growthThreshold The map will be expanded once the number of elements exceeds this threshold.
protected static long	INIT_BUCKET_MEMORY_IN_BYTES
protected int	lastBucketPosition
protected static double	LOAD_FACTOR
protected int	log2NumBuckets
protected MemorySegmentPool	memoryPool
protected int	numBucketsMask
protected int	numBucketsMask2
protected int	numBucketsPerSegment
protected int	numBucketsPerSegmentBits
protected int	numBucketsPerSegmentMask
protected int	numElements
protected long	numSpillFiles
static int	RECORD_EXTRA_LENGTH
protected org.apache.paimon.hash.BytesMap.RecordArea<K,V>	recordArea The segments where the actual data is stored.
protected int	reservedNumBuffers
protected K	reusedKey Used as a reused object when lookup and iteration.
protected V	reusedValue Used as a reused object when retrieve the map's value by key and iteration.
protected int	segmentSize
protected long	spillInBytes
protected static int	STEP_INCREMENT

Constructor Summary

Constructors

Constructor and Description
BytesMap(MemorySegmentPool memoryPool, PagedTypeSerializer<K> keySerializer)

Method Summary

Modifier and Type	Method and Description
protected int	calcSecondHashCode(int firstHashCode)
void	free()
int	getNumElements()
abstract long	getNumKeys() Returns the number of keys in this map.
long	getNumSpillFiles()
long	getSpillInBytes()
protected void	growAndRehash()
protected void	initBucketSegments(int numBucketSegments)
BytesMap.LookupInfo<K,V>	lookup(K key)
void	reset() reset the map's record and bucket area's memory segments for reusing.
protected void	resetBucketSegments(List<MemorySegment> resetBucketSegs)
protected void	returnSegments(List<MemorySegment> segments)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

BUCKET_SIZE

public static final int BUCKET_SIZE

See Also:

Constant Field Values

END_OF_LIST

protected static final int END_OF_LIST

See Also:

Constant Field Values

STEP_INCREMENT

protected static final int STEP_INCREMENT

See Also:

Constant Field Values

ELEMENT_POINT_LENGTH

protected static final int ELEMENT_POINT_LENGTH

See Also:

Constant Field Values

RECORD_EXTRA_LENGTH

public static final int RECORD_EXTRA_LENGTH

See Also:

Constant Field Values

BUCKET_SIZE_BITS

protected static final int BUCKET_SIZE_BITS

See Also:

Constant Field Values

numBucketsPerSegment

protected final int numBucketsPerSegment

numBucketsPerSegmentBits

protected final int numBucketsPerSegmentBits

numBucketsPerSegmentMask

protected final int numBucketsPerSegmentMask

lastBucketPosition

protected final int lastBucketPosition

segmentSize

protected final int segmentSize

memoryPool

protected final MemorySegmentPool memoryPool

bucketSegments

protected List<MemorySegment> bucketSegments

reservedNumBuffers

protected final int reservedNumBuffers

numElements

protected int numElements

numBucketsMask

protected int numBucketsMask

log2NumBuckets

protected int log2NumBuckets

numBucketsMask2

protected int numBucketsMask2

LOAD_FACTOR

protected static final double LOAD_FACTOR

See Also:

Constant Field Values

INIT_BUCKET_MEMORY_IN_BYTES

protected static final long INIT_BUCKET_MEMORY_IN_BYTES

See Also:

Constant Field Values

growthThreshold

protected int growthThreshold

The map will be expanded once the number of elements exceeds this threshold.

recordArea

protected org.apache.paimon.hash.BytesMap.RecordArea<K,V> recordArea

The segments where the actual data is stored.

reusedKey

protected K reusedKey

Used as a reused object when lookup and iteration.

reusedValue

protected V reusedValue

Used as a reused object when retrieve the map's value by key and iteration.

numSpillFiles

protected long numSpillFiles

spillInBytes

protected long spillInBytes

Constructor Detail

BytesMap

public BytesMap(MemorySegmentPool memoryPool,
                PagedTypeSerializer<K> keySerializer)

Method Detail

getNumKeys

public abstract long getNumKeys()

Returns the number of keys in this map.

initBucketSegments

protected void initBucketSegments(int numBucketSegments)

resetBucketSegments

protected void resetBucketSegments(List<MemorySegment> resetBucketSegs)

getNumSpillFiles

public long getNumSpillFiles()

getSpillInBytes

public long getSpillInBytes()

getNumElements

public int getNumElements()

free

public void free()

reset

public void reset()

reset the map's record and bucket area's memory segments for reusing.

lookup

public BytesMap.LookupInfo<K,V> lookup(K key)

Parameters:

key - by which looking up the value in the hash map. Only support the key in the BinaryRowData form who has only one MemorySegment.

Returns:

BytesMap.LookupInfo

growAndRehash

protected void growAndRehash()
                      throws EOFException

Throws:

EOFException - if the map can't allocate much more memory.

returnSegments

protected void returnSegments(List<MemorySegment> segments)

calcSecondHashCode

protected int calcSecondHashCode(int firstHashCode)