首页  

flinkcdc 同步报错问题排查记录     所属分类 flink 浏览量 24
flink1.18 
flinkcdc3.2.0 


flink job 详情页 
Exceptions 
  Root Exception
  Exception History


org.apache.doris.flink.exception.DorisBatchLoadException



org.apache.doris.flink.exception.DorisBatchLoadException: org.apache.doris.flink.exception.DorisBatchLoadException: stream load error: 
	at org.apache.doris.flink.sink.batch.DorisBatchStreamLoad.checkFlushException(DorisBatchStreamLoad.java:209)
	at org.apache.doris.flink.sink.batch.DorisBatchStreamLoad.writeRecord(DorisBatchStreamLoad.java:152)
	at org.apache.doris.flink.sink.batch.DorisBatchWriter.writeOneDorisRecord(DorisBatchWriter.java:148)
	at org.apache.doris.flink.sink.batch.DorisBatchWriter.write(DorisBatchWriter.java:114)
	at org.apache.flink.streaming.runtime.operators.sink.SinkWriterOperator.processElement(SinkWriterOperator.java:161)
	at org.apache.flink.cdc.runtime.operators.sink.DataSinkWriterOperator.processElement(DataSinkWriterOperator.java:177)
	at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:75)
	at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:50)
	at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:29)
	at org.apache.flink.streaming.api.operators.StreamMap.processElement(StreamMap.java:38)
	at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:237)
	at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:146)
	at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:110)
	at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:562)
	at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:858)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:807)
	at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:953)
	at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:932)
	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:746)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562)
	at java.lang.Thread.run(Thread.java:750)
Caused by: org.apache.doris.flink.exception.DorisBatchLoadException: stream load error: 
	at org.apache.doris.flink.sink.batch.DorisBatchStreamLoad$LoadAsyncExecutor.load(DorisBatchStreamLoad.java:303)
	at org.apache.doris.flink.sink.batch.DorisBatchStreamLoad$LoadAsyncExecutor.run(DorisBatchStreamLoad.java:245)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	... 1 more
Caused by: org.apache.doris.flink.exception.DorisBatchLoadException: stream load error: [INTERNAL_ERROR]too many filtered rows

	0#  std::_Function_handler<void (doris::RuntimeState*, doris::Status*), doris::StreamLoadExecutor::execute_plan_fragment(std::shared_ptr<doris::StreamLoadContext>)::$_0>::_M_invoke(std::_Any_data const&, doris::RuntimeState*&&, doris::Status*&&) at /root/src/doris-2.0/be/src/common/status.h:354
	1#  doris::FragmentMgr::_exec_actual(std::shared_ptr<doris::FragmentExecState>, std::function<void (doris::RuntimeState*, doris::Status*)> const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:360
	2#  std::_Function_handler<void (), doris::FragmentMgr::exec_plan_fragment(doris::TExecPlanFragmentParams const&, std::function<void (doris::RuntimeState*, doris::Status*)> const&)::$_0>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:701
	3#  doris::ThreadPool::dispatch_thread() at /root/src/doris-2.0/be/src/util/threadpool.cpp:0
	4#  doris::Thread::supervise_thread(void*) at /var/local/ldb-toolchain/bin/../usr/include/pthread.h:562
	5#  ?
	6#  __clone
, see more in http://192.168.1.10:8040/api/_load_error_log?file=__shard_761/error_log_insert_stmt_56439e544b137ec5-ca395daad8fd6595_56439e544b137ec5_ca395daad8fd6595
	at org.apache.doris.flink.sink.batch.DorisBatchStreamLoad$LoadAsyncExecutor.load(DorisBatchStreamLoad.java:292)
	... 4 more




查看详细日志信息 curl http://192.168.1.10:8040/api/_load_error_log?file=__shard_761/error_log_insert_stmt_56439e544b137ec5-ca395daad8fd6595_56439e544b137ec5_ca395daad8fd6595 Reason: column_name[bank_name], the length of input is too long than schema. first 32 bytes of input str: [5E0119EA67D35241851F11EDF4B1B630] schema length: 192; actual length: 224; . src line []; 查看字段名 为 bank_name 的 字段信息 select * from information_schema.COLUMNS where column_name='bank_name' 修改字段长度 ALTER TABLE db001.table001 MODIFY COLUMN bank_name varchar(500) NULL COMMENT "bank_name"

上一篇     下一篇
阳明心学的核心:心即理、事上练、致良知

《强势》笔记

《大衰退年代》笔记

Flink1.18.1 本地单机部署 及 flinkCDC3.2.1 测试

A股33年(1990-2023)

《繁花》经典台词