org.apache.paimon.utils

Class ThreadPoolUtils

java.lang.Object

org.apache.paimon.utils.ThreadPoolUtils

public class ThreadPoolUtils
extends Object

Utils for thread pool.

Constructor Summary

Constructors

Constructor and Description
ThreadPoolUtils()

Method Summary

Modifier and Type	Method and Description
static ThreadPoolExecutor	createCachedThreadPool(int threadNum, String namePrefix) Create a thread pool with max thread number.
static <U,T> Iterator<T>	randomlyExecute(ExecutorService executor, java.util.function.Function<U,List<T>> processor, Collection<U> input)
static <U> void	randomlyOnlyExecute(ExecutorService executor, java.util.function.Consumer<U> processor, Collection<U> input)
static <T,U> Iterable<T>	sequentialBatchedExecute(ThreadPoolExecutor executor, java.util.function.Function<U,List<T>> processor, List<U> input, Integer queueSize) This method aims to parallel process tasks with memory control and sequentially.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ThreadPoolUtils

public ThreadPoolUtils()

Method Detail

createCachedThreadPool

public static ThreadPoolExecutor createCachedThreadPool(int threadNum,
                                                        String namePrefix)

Create a thread pool with max thread number. Inactive threads will automatically exit.

The Executors.newCachedThreadPool() cannot limit max thread number. Non-core threads must be used with SynchronousQueue, but synchronous queue will be blocked when there is max thread number.

sequentialBatchedExecute

public static <T,U> Iterable<T> sequentialBatchedExecute(ThreadPoolExecutor executor,
                                                         java.util.function.Function<U,List<T>> processor,
                                                         List<U> input,
                                                         @Nullable
                                                         Integer queueSize)

This method aims to parallel process tasks with memory control and sequentially.

randomlyOnlyExecute

public static <U> void randomlyOnlyExecute(ExecutorService executor,
                                           java.util.function.Consumer<U> processor,
                                           Collection<U> input)

randomlyExecute

public static <U,T> Iterator<T> randomlyExecute(ExecutorService executor,
                                                java.util.function.Function<U,List<T>> processor,
                                                Collection<U> input)