首页  

flink-CDC-3.0 mysql to doris 数据同步任务 经常报错 stream load error: [LABEL_ALREADY_EXISTS]     所属分类 flink 浏览量 433
具体版本信息
flink1.18.0
Doris 2.0.2
flink-cdc-3.0.0
flink-cdc-pipeline-connector-mysql-3.0.0.jar
flink-cdc-pipeline-connector-doris-3.0.0.jar

org.apache.doris.flink.exception.DorisBatchLoadException: stream load error: [LABEL_ALREADY_EXISTS]

Caused by: org.apache.doris.flink.exception.DorisBatchLoadException: stream load error: [LABEL_ALREADY_EXISTS]TStatus: errCode = 2, detailMessage = Label [_0_xxxxxxxx_064ad128-ed6c-44b8-99b4-6e715fb637af] has already been used, relate to txn [272201], see more in null


异常堆栈


org.apache.doris.flink.exception.DorisBatchLoadException: org.apache.doris.flink.exception.DorisBatchLoadException: stream load error:
at org.apache.doris.flink.sink.batch.DorisBatchStreamLoad.checkFlushException(DorisBatchStreamLoad.java:172)
at org.apache.doris.flink.sink.batch.DorisBatchStreamLoad.writeRecord(DorisBatchStreamLoad.java:125)
at org.apache.doris.flink.sink.batch.DorisBatchWriter.write(DorisBatchWriter.java:106)
at org.apache.flink.streaming.runtime.operators.sink.SinkWriterOperator.processElement(SinkWriterOperator.java:161)
at com.ververica.cdc.runtime.operators.sink.DataSinkWriterOperator.processElement(DataSinkWriterOperator.java:154)
at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:75)
at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:50)
at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:29)
at org.apache.flink.streaming.api.operators.StreamMap.processElement(StreamMap.java:38)
at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:237)
at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:146)
at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:110)
at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)
at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:562)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231)
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:858)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:807)
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:953)
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:932)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:746)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562)
at java.lang.Thread.run(Thread.java:750)
Caused by: org.apache.doris.flink.exception.DorisBatchLoadException: stream load error:
at org.apache.doris.flink.sink.batch.DorisBatchStreamLoad$LoadAsyncExecutor.load(DorisBatchStreamLoad.java:257)
at org.apache.doris.flink.sink.batch.DorisBatchStreamLoad$LoadAsyncExecutor.run(DorisBatchStreamLoad.java:208)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: org.apache.doris.flink.exception.DorisBatchLoadException: stream load error: [LABEL_ALREADY_EXISTS]TStatus: errCode = 2, detailMessage = Label [_0_xxxxxxxx_064ad128-ed6c-44b8-99b4-6e715fb637af] has already been used, relate to txn [272201], see more in null
at org.apache.doris.flink.sink.batch.DorisBatchStreamLoad$LoadAsyncExecutor.load(DorisBatchStreamLoad.java:249)
... 4 more




https://nightlies.apache.org/flink/flink-cdc-docs-release-3.0/zh/docs/connectors/doris/ 4. errCode = 2, detailMessage = Label [label_0_1] has already been used, relate to txn [19650] Exactly-Once场景下,Flink Job重启时必须从最新的Checkpoint/Savepoint启动,否则会报如上错误。 不要求Exactly-Once时,也可通过关闭2PC提交(sink.enable-2pc=false) 或 更换不同的sink.label-prefix解决 sink.properties. optional (none) String StreamLoad的参数。 For example: sink.properties.strict_mode: true. StreamLoad的Properties 属性 https://doris.apache.org/zh-CN/docs/dev/sql-manual/sql-reference/Data-Manipulation-Statements/Load/STREAM-LOAD/ 设置 sink.enable-2pc=false sink: type: doris fenodes: 192.168.10.21:8030 username: hello password: hello table.create.properties.light_schema_change: true table.create.properties.replication_num: 3 enable-2pc: false
grafana上配置 检查点失败次数 job重启次数监控 flink_jobmanager_job_numberOfFailedCheckpoints and on(job,job_id) (increase(flink_jobmanager_job_runningTime[5m]) >0) flink_jobmanager_job_fullRestarts and on(job,job_id) (increase(flink_jobmanager_job_runningTime[5m]) >0)
flink-CDC-3.0 mysql to doris 数据同步任务 经常报错 stream load error: [LABEL_ALREADY_EXISTS] ,达到重启最大次数后退出 https://developer.aliyun.com/ask/611236

上一篇     下一篇
PromQL内置函数

kafka 与 rabbitMQ 比较

常用组件告警规则

promQL ON 使用

杭州西山游步道爬山路线汇总

监控原则