【flink】Chunk splitting has encountered exception

这篇具有很好参考价值的文章主要介绍了【flink】Chunk splitting has encountered exception。希望对大家有所帮助。如果存在错误或未考虑完全的地方,请大家不吝赐教,您也可以点击"举报违法"按钮提交疑问。

执行任务报错: Chunk splitting has encountered exception

错误信息截图:

【flink】Chunk splitting has encountered exception,flink,大数据

 【flink】Chunk splitting has encountered exception,flink,大数据

完整的错误信息:

16:30:43,911 ERROR org.apache.flink.runtime.source.coordinator.SourceCoordinator [SourceCoordinator-Source: CDC Sourceorg.jobslink.flink.UserCDCJob]  - Uncaught exception in the SplitEnumerator for Source Source: CDC Sourceorg.flink.UserCDCJob while handling operator event RequestSplitEvent (host='') from subtask 0. Triggering job failover.
org.apache.flink.util.FlinkRuntimeException: Chunk splitting has encountered exception
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.checkSplitterErrors(MySqlSnapshotSplitAssigner.java:522)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:278)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:298)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlHybridSplitAssigner.getNext(MySqlHybridSplitAssigner.java:123)
	at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.assignSplits(MySqlSourceEnumerator.java:201)
	at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.handleSplitRequest(MySqlSourceEnumerator.java:117)
	at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$handleEventFromOperator$2(SourceCoordinator.java:230)
	at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$runInEventLoop$9(SourceCoordinator.java:406)
	at org.apache.flink.util.ThrowableCatchingRunnable.run(ThrowableCatchingRunnable.java:40)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Error when splitting chunks for jobslink_user.jobslink_account
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.splitTable(MySqlSnapshotSplitAssigner.java:246)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.splitChunksForRemainingTables(MySqlSnapshotSplitAssigner.java:505)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	... 3 more
Caused by: java.lang.RuntimeException: Fail to analyze table in chunk splitter.
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter.analyzeTable(MySqlChunkSplitter.java:154)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter.splitChunks(MySqlChunkSplitter.java:117)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.splitTable(MySqlSnapshotSplitAssigner.java:244)
	... 7 more
Caused by: org.apache.flink.table.api.ValidationException: Incremental snapshot for tables requires primary key, but table user.account doesn't have primary key.
	at com.ververica.cdc.connectors.mysql.source.utils.ChunkUtils.getChunkKeyColumn(ChunkUtils.java:55)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter.analyzeTable(MySqlChunkSplitter.java:148)
	... 9 more
16:30:43,929 INFO  org.apache.flink.runtime.jobmaster.JobMaster                 [flink-akka.actor.default-dispatcher-9]  - Trying to recover from a global failure.
org.apache.flink.util.FlinkException: Global failure triggered by OperatorCoordinator for 'Source: CDC Sourceorg.jobslink.flink.UserCDCJob -> Filter -> Flat Map -> Filter -> (Sink: Print to Std. Out, Sink: sink jobslink_account)' (operator cbc357ccb763df2852fee8c4fc7d55f2).
	at org.apache.flink.runtime.operators.coordination.OperatorCoordinatorHolder$LazyInitializedCoordinatorContext.failJob(OperatorCoordinatorHolder.java:556)
	at org.apache.flink.runtime.operators.coordination.RecreateOnResetOperatorCoordinator$QuiesceableContext.failJob(RecreateOnResetOperatorCoordinator.java:231)
	at org.apache.flink.runtime.source.coordinator.SourceCoordinatorContext.failJob(SourceCoordinatorContext.java:316)
	at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$runInEventLoop$9(SourceCoordinator.java:420)
	at org.apache.flink.util.ThrowableCatchingRunnable.run(ThrowableCatchingRunnable.java:40)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.flink.util.FlinkRuntimeException: Chunk splitting has encountered exception
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.checkSplitterErrors(MySqlSnapshotSplitAssigner.java:522)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:278)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:298)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlHybridSplitAssigner.getNext(MySqlHybridSplitAssigner.java:123)
	at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.assignSplits(MySqlSourceEnumerator.java:201)
	at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.handleSplitRequest(MySqlSourceEnumerator.java:117)
	at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$handleEventFromOperator$2(SourceCoordinator.java:230)
	at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$runInEventLoop$9(SourceCoordinator.java:406)
	... 9 more
Caused by: java.lang.IllegalStateException: Error when splitting chunks for jobslink_user.jobslink_account
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.splitTable(MySqlSnapshotSplitAssigner.java:246)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.splitChunksForRemainingTables(MySqlSnapshotSplitAssigner.java:505)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	... 3 more
Caused by: java.lang.RuntimeException: Fail to analyze table in chunk splitter.
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter.analyzeTable(MySqlChunkSplitter.java:154)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter.splitChunks(MySqlChunkSplitter.java:117)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.splitTable(MySqlSnapshotSplitAssigner.java:244)
	... 7 more
Caused by: org.apache.flink.table.api.ValidationException: Incremental snapshot for tables requires primary key, but table user.account doesn't have primary key.
	at com.ververica.cdc.connectors.mysql.source.utils.ChunkUtils.getChunkKeyColumn(ChunkUtils.java:55)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter.analyzeTable(MySqlChunkSplitter.java:148)
	... 9 more
16:30:43,933 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph       [flink-akka.actor.default-dispatcher-9]  - Job org.flink.UserCDCJob (35780b6f0490a368fc256b228e99f7cb) switched from state RUNNING to RESTARTING.
16:30:43,936 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph       [flink-akka.actor.default-dispatcher-9]  - Source: CDC Sourceorg.flink.UserCDCJob -> Filter -> Flat Map -> Filter -> (Sink: Print to Std. Out, Sink: sink account) (1/1) (45028a2ced14e3bbe2d7cf3349defba0) switched from RUNNING to CANCELING.

 Incremental snapshot for tables requires primary key, but table user.account doesn't have primary key.

表的增量快照需要主键,但表user.account没有主键。

user库account表增加主键

【flink】Chunk splitting has encountered exception,flink,大数据

添加上主键以后就可以正常执行了!!! 文章来源地址https://www.toymoban.com/news/detail-632415.html

到了这里,关于【flink】Chunk splitting has encountered exception的文章就介绍完了。如果您还想了解更多内容,请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章,希望大家以后多多支持TOY模板网!

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处: 如若内容造成侵权/违法违规/事实不符,请点击违法举报进行投诉反馈,一经查实,立即删除!

领支付宝红包 赞助服务器费用

相关文章

  • 【Python】成功解决AttributeError: ‘list‘ object has no attribute ‘split‘

    【Python】成功解决AttributeError: ‘list‘ object has no attribute ‘split‘ 🌈 个人主页:高斯小哥 🔥 高质量专栏:Matplotlib之旅:零基础精通数据可视化、Python基础【高质量合集】、PyTorch零基础入门教程👈 希望得到您的订阅和支持~ 💡 创作高质量博文(平均质量分92+),分享更多关

    2024年04月14日
    浏览(51)
  • 打包运行时抛出chunk name [mini-css-extract-plugin] Conflicting order. Following module has been added错误!

    今天在执行npm run build 时,控制台出现一大片警告错误,虽然不影响项目打包,但是看着碍眼,报错如下: 首先我们看到这个错误是由,[mini-css-extract-plugin]抛出的,那么这个插件有什么作用?这个插件主要是将css抽离出来到一个文件里面,它的特点是不会重复编译css文件 所

    2024年02月12日
    浏览(30)
  • 完美解决AttributeError: ‘NoneType‘ object has no attribute ‘split‘ 报错处理

    已解决AttributeError: ‘NoneType’ object has no attribute ‘split’ 报错处理 AttributeError: ‘NoneType‘ object has no attribute ‘split‘ 这个错误通常出现在使用 NoneType 对象调用 split() 方法时。NoneType 是 Python 中表示空值的类型。 下滑查看解决方法 产生该错误的原因可能是你没有正确地初始

    2024年02月12日
    浏览(50)
  • QMimeDatabase: Error loading internal MIME data An error has been encountered at line 1 of <internal

    QMimeDatabase: Error loading internal MIME data An error has been encountered at line 1 of : Premature end of document.: Traceback (most recent call last): File “D:anacondalibsite-packageslibscanvas.py”, line 530, in paintEvent p.drawLine(self.prev_point.x(), 0, self.prev_point.x(), self.pixmap.height()) TypeError: arguments did not match any overlo

    2024年02月06日
    浏览(44)
  • C++ has a different exception specifier

    这个报错: 正常来说就是你声明的函数定义和真正函数实现的时候,参数或者函数类型不一致,仔细检查,把参数复制成一样的即可,但是我的确定不存在不一致的问题, 继承了基类,实现了对应的虚函数,最后的问题是出在throw()和noexcept两个上,二者的目标都是限

    2024年02月14日
    浏览(30)
  • Selenium Exception AttributeError: “‘Service‘ object has no attribute ‘process‘“

    服务端:win10 professional 22H2; python环境:python3.9.16+centos 7+selenium 4.8.0+firefox 68.10.0esr+geckodriver v0.32.1; 将被测程序部署在win10物理机上,在centos 7中运行pycharm。起初运行时,日志提示服务异常退出。搜索后得知大概率是因为驱动路径不对,或者驱动的版本不对。我这里的问题是驱

    2024年02月12日
    浏览(29)
  • ElasticSearch创建索引报错:mapper_parsing_exception Root mapping definition has unsupported parameters

    ElasticSearch版本号:5.6.14,这个错误和ES版本有一定的关系,还是先交代下版本号,免得有的读者根据我的方法操作后无效 错误翻译: mapper_parsing_exception :映射解析异常 Root mapping definition has unsupported parameters :根映射定义包含不受支持的参数 错误映射语句1: 错误映射语句

    2024年02月11日
    浏览(52)
  • VMware Workstation 不可恢复错误: (vcpu-1) Exception 0xc0000005 (access violation) has occurred终极解决方案

    VMware Workstation 不可恢复错误: (vcpu-0) Exception 0xc0000005 (access violation) has occurred. 尝试了网上一切能找到的方法,都无法解决,实际上还是问题根源没找到。 你们可以先用前面几种解决方案试试,如果能够解决就好,说明你的问题就是其中一种。 但是最苦恼的是不知道什么原因

    2024年02月03日
    浏览(54)
  • Error: Could not create the Java Virtual Machine. Error: A fatal exception has occurred……

    本人使用 idea 创建 web 工程后,运行 tomcat 服务器时出现报错: 并且 tomcat 出现错误信息: 本人尝试更换 JDK 版本,但并没有什么变化。最后发现是忘了修改 Project Structure... 中的 SDK 。 由于需要修改配置的地方较多,很容易漏掉一小步,下面整理了比较全面的需要修改的地方

    2024年02月07日
    浏览(60)
  • Error: Could not create the Java Virtual Machine.Error: A fatal exception has occurred. Progra

    本人学化学的小白,自学JAVA。本来电脑第一次本来装JDK17的时候,应该是装成功了,后来eclipse出了点问题,我以为是JDK17的问题,就卸载重装, 结果就悲剧了。 每次重装都有问题,win命令行输入java -v都显示下面错误。 在csdn里搜到好多方法 这个原因是因为在安装JDK的时候在

    2023年04月09日
    浏览(47)

觉得文章有用就打赏一下文章作者

支付宝扫一扫打赏

博客赞助

微信扫一扫打赏

请作者喝杯咖啡吧~博客赞助

支付宝扫一扫领取红包,优惠每天领

二维码1

领取红包

二维码2

领红包