首页  

Flink1.18.1 本地单机部署 及 flinkCDC3.0 测试     所属分类 flink 浏览量 373
https://nightlies.apache.org/flink/flink-docs-release-1.8/

https://nightlies.apache.org/flink/flink-cdc-docs-release-3.0/docs/get-started/quickstart/mysql-to-doris/

阿里云开发者社区 实时计算 Flink
https://developer.aliyun.com/group/sc/



jdk8 环境准备

下载安装flink1.18.1
https://archive.apache.org/dist/flink/flink-1.18.1/
flink-1.18.1-bin-scala_2.12.tgz   

解压 
tar xvf flink-1.18.1-bin-scala_2.12.tgz   
cd flink-1.18.0/bin 

/Users/dugang/work/test/flink-1.18.1/bin


./start-cluster.sh
Starting cluster.
Starting standalonesession daemon on host master.
Starting taskexecutor daemon on host master.

http://127.0.0.1:8081


conf/flink-conf.yaml
#rest.port: 8081





https://nightlies.apache.org/flink/flink-cdc-docs-release-3.0/zh/docs/get-started/quickstart/mysql-to-doris/ https://www.apache.org/dyn/closer.lua/flink/flink-1.18.1/flink-1.18.1-bin-scala_2.12.tgz https://dlcdn.apache.org/flink/flink-1.18.1/flink-1.18.1-bin-scala_2.12.tgz 这个下载快
下载 flinkcdc3.0.0的 mysql和 doris 连接器放到 lib目录下 /Users/dugang/work/test/soft/flink/flink-cdc-3.0.0/lib/ flink-cdc-pipeline-connector-mysql-3.0.0.jar flink-cdc-pipeline-connector-doris-3.0.0.jar flinkcdc3.0 测试 cd /Users/dugang/work/test/soft/flink/flink-cdc-3.0.0 ./bin/flink-cdc.sh --flink-home /Users/dugang/work/test/flink-1.18.1 conf/flink-cdc-mysql-doris-001.yaml Pipeline has been submitted to cluster. Job ID: 895e24f476ec88c695dbb6859c6c845b Job Description: mysql-doris-001
flink-cdc-mysql-doris-001.yaml source: type: mysql hostname: 192.168.1.10 port: 6454 username: user password: user123 tables: demo.t1, demo.t2 # server-id: 5400-5404 server-time-zone: Asia/Shanghai sink: type: doris fenodes: 192.168.10.20:8030 username: root password: root123 table.create.properties.light_schema_change: true table.create.properties.replication_num: 3 pipeline: name: mysql-doris-001 parallelism: 1 route: - source-table: demo.t1 sink-table: demo.t1 - source-table: demo.t2 sink-table: demo.t2

上一篇     下一篇
杭州市内游玩交通

Kube-Install 一键离线安装多K8S集群

flink checkpoint 检查点

Skywalking 术语

linux安装切换多个版本jdk

doris 简介