flink mysql cdc调试问题记录-Toy模板网

这篇具有很好参考价值的文章主要介绍了flink mysql cdc调试问题记录。希望对大家有所帮助。如果存在错误或未考虑完全的地方，请大家不吝赐教，您也可以点击"举报违法"按钮提交疑问。

最近需要用到flink cdc作为数据流处理框架，在demo运行中发现一些问题，特此记录问题和解决过程。

无法读取table

Caused by: java.lang.IllegalArgumentException: Can't find any matched tables, please check your configured database-name: [localdb] and table-name: [flink_cdc_message]

at com.ververica.cdc.connectors.mysql.debezium.DebeziumUtils.discoverCapturedTables(DebeziumUtils.java:167)

at com.ververica.cdc.connectors.mysql.source.MySqlSource.createEnumerator(MySqlSource.java:179)

... 43 more

出现上述异常时，需要检查如下情况：

是否已经对用户设置了SELECT, REPLICATION SLAVE, REPLICATION CLIENT这三个权限
Java streaming设置tableList时需要使用[database].[table]格式，显式指定table名称

guava版本异常

org.apache.flink.util.FlinkException: Global failure triggered by OperatorCoordinator for 'Source: CDCJob -> Sink: Unnamed' (operator cbc357ccb763df2852fee8c4fc7d55f2).
   at org.apache.flink.runtime.operators.coordination.OperatorCoordinatorHolder$LazyInitializedCoordinatorContext.failJob(OperatorCoordinatorHolder.java:545)
   at org.apache.flink.runtime.operators.coordination.RecreateOnResetOperatorCoordinator$QuiesceableContext.failJob(RecreateOnResetOperatorCoordinator.java:223)
   at org.apache.flink.runtime.source.coordinator.SourceCoordinatorContext.failJob(SourceCoordinatorContext.java:285)
   at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$runInEventLoop$8(SourceCoordinator.java:358)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoClassDefFoundError: org/apache/flink/shaded/guava18/com/google/common/util/concurrent/ThreadFactoryBuilder
   at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.startAsynchronouslySplit(MySqlSnapshotSplitAssigner.java:196)
   at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.open(MySqlSnapshotSplitAssigner.java:164)
   at com.ververica.cdc.connectors.mysql.source.assigners.MySqlHybridSplitAssigner.open(MySqlHybridSplitAssigner.java:95)
   at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.start(MySqlSourceEnumerator.java:92)
   at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$start$0(SourceCoordinator.java:141)
   at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$runInEventLoop$8(SourceCoordinator.java:344)
   ... 3 more
Caused by: java.lang.ClassNotFoundException: org.apache.flink.shaded.guava18.com.google.common.util.concurrent.ThreadFactoryBuilder
   at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
   at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
   at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
   ... 9 more

出现问题的原因可以参考：https://github.com/ververica/flink-cdc-connectors/pull/947

原因在于sql和非sql connector实现中对于shaded guava的处理不同，使用

<!-- https://mvnrepository.com/artifact/com.ververica/flink-sql-connector-mysql-cdc -->
<dependency>
    <groupId>com.ververica</groupId>
    <artifactId>flink-sql-connector-mysql-cdc</artifactId>
    <version>2.2.0</version>
</dependency>

代替

<!-- https://mvnrepository.com/artifact/com.ververica/flink-connector-mysql-cdc -->
<dependency>
    <groupId>com.ververica</groupId>
    <artifactId>flink-connector-mysql-cdc</artifactId>
    <version>2.2.0</version>
    <scope>provided</scope>
</dependency>

即可解决问题。

发现一篇很好的文章，涉及的问题都有提到：

Flink CDC for Postgres: Lessons Learned - sap1ens blog

这是Debezium相关的文章集合

Resources on the Web 文章来源地址https://www.toymoban.com/news/detail-415968.html

到了这里，关于flink mysql cdc调试问题记录的文章就介绍完了。如果您还想了解更多内容，请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章，希望大家以后多多支持TOY模板网！