首页  

flink批处理wordcount实例2     所属分类 flink 浏览量 84
依赖

org.apache.flink:flink-java:1.9.0
org.apache.flink:flink-streaming-java_2.12:1.9.0



import org.apache.flink.api.common.functions.FlatMapFunction;
import org.apache.flink.api.java.DataSet;
import org.apache.flink.api.java.ExecutionEnvironment;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.util.Collector;

public class WordCount2 {

    public static void main(String[] args) throws Exception {
        
        final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
         
        DataSet<String> text = env.fromElements(
            "hello java","hello tiger","hello tiger","cat cat cat","dog"
        );
 
        DataSet<Tuple2<String, Integer>> wordCounts = text
            .flatMap(new WordSplit())
            .groupBy(0)
            .sum(1);
 
        wordCounts.print();
    }
    
    private static class WordSplit implements FlatMapFunction<String, Tuple2<String, Integer>> {     
        @Override
        public void flatMap(String value, Collector<Tuple2<String, Integer>> out) {     
            String[] words = value.toLowerCase().split(" "); 
            for (String word : words) {
                word = word.trim();
                if (word.length() > 0) {
                    out.collect(new Tuple2<String, Integer>(word, 1));
                }   
            }
        }
    }
}



wordCounts.print(); 控制台直接输出 (tiger,2) (java,1) (dog,1) (cat,3) (hello,3)
完整代码 https://gitee.com/dyyx/demos/blob/master/flinkdemo/src/main/java/dyyx/batch/WordCount2.java
flink流处理 WordCount 实例 flink批处理 WordCount 实例

上一篇     下一篇
Flink理论基础

flink流处理WordCount 实例

flink批处理wordcount实例

flink介绍

Flink编程模型

java正则表达式实例之简单分词