flink 运行模式 批处理与流处理模式
所属分类 flink
浏览量 980
https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/datastream_execution_mode.html
Execution Mode (Batch/Streaming)
By enabling BATCH execution, we allow Flink to apply additional optimizations that we can only do when we know that our input is bounded.
输入有界,启用BATCH执行,允许Flink应用额外的优化
The BATCH execution mode can only be used for Jobs/Flink Programs that are bounded.
STREAMING execution mode,can be used for both bounded and unbounded jobs.
execution.runtime-mode
STREAMING: The classic DataStream execution mode (default)
BATCH: Batch-style execution on the DataStream API
AUTOMATIC: Let the system decide based on the boundedness of the sources
bin/flink run -Dexecution.runtime-mode=BATCH examples/streaming/WordCount.jar
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.setRuntimeMode(RuntimeExecutionMode.BATCH);
Task scheduling and network shuffles work differently for BATCH and STREAMING execution mode.
Mostly due to the fact that we know our input data is bounded in BATCH execution mode,
which allows Flink to use more efficient data structures and algorithms.
BATCH作业可以在更少的资源上执行(就TaskManagers的可用插槽而言),因为系统可以一个接一个顺序执行任务
In BATCH mode, the configured state backend is ignored.
上一篇
下一篇
flink datastream batch mode wordcount实例
flink内存管理机制
为何kafka要去掉zookeeper依赖
BigDecimal 使用注意点
小巧的Java编译器 Janino
Flink 知识点汇总