目录
背景:
现象:
问题定位:
问题原因:
解决:
背景:
生产环境一个活动给某个用户发送积分失败,核心业务接口使用Redisson分布式锁
同事答复:redis主从切换导致的问题。
个人表示怀疑,所以想定位下真实原因。
redisson 3.17.3
sentinel模式:master slave1 slave2
<dependency>
<groupId>org.redisson</groupId>
<artifactId>redisson-spring-boot-starter</artifactId>
<version>3.17.3</version>
</dependency>
RLock rLock = redisson.getLock("xxxxxx");
rLock.lock(15, TimeUnit.SECONDS);
现象:
rLock.lock(15, TimeUnit.SECONDS);
没走到下边的业务代码就报错
16:52:58.382 2023-11-14 16:52:57 [http-nio-8080-exec-142] ERROR c.c.p.configure.aspect.ApiAspect 60 - Failed to invoke interface [Map cn.xx.xx.api.customer.xx.yyy(String)]
16:52:58.382 org.redisson.client.RedisException: Unexpected exception while processing command
16:52:58.382 at org.redisson.command.CommandAsyncService.convertException(CommandAsyncService.java:276) ~[redisson-3.17.3.jar!/:3.17.3]
16:52:58.382 at org.redisson.command.CommandAsyncService.get(CommandAsyncService.java:115) ~[redisson-3.17.3.jar!/:3.17.3]
16:52:58.382 at org.redisson.RedissonObject.get(RedissonObject.java:82) ~[redisson-3.17.3.jar!/:3.17.3]
16:52:58.382 at org.redisson.RedissonLock.tryAcquire(RedissonLock.java:144) ~[redisson-3.17.3.jar!/:3.17.3]
16:52:58.486 Caused by: java.lang.IllegalStateException: Only 1 of 2 slaves were synced
16:52:58.486 at org.redisson.RedissonBaseLock.lambda$evalWriteAsync$0(RedissonBaseLock.java:226) ~[redisson-3.17.3.jar!/:3.17.3]
16:52:58.486 at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:822) ~[na:1.8.0_201]
16:52:58.486 at java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:797) ~[na:1.8.0_201]
16:52:58.487 at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[na:1.8.0_201]
16:52:58.487 at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) ~[na:1.8.0_201]
16:52:58.487 at org.redisson.command.CommandBatchService.lambda$executeAsync$7(CommandBatchService.java:322) ~[redisson-3.17.3.jar!/:3.17.3]
16:52:58.487 at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) ~[na:1.8.0_201]
关键错误日志
c.c.p.configure.aspect.ApiAspect 60 - Failed to invoke interface [Map cn.xx.xx.api.customer.xx.yyy(String)]
org.redisson.client.RedisException: Unexpected exception while processing command
16:52:58.486 Caused by: java.lang.IllegalStateException: Only 1 of 2 slaves were synced
问题定位:
找运维同学,看监控,没看到异常,反馈其他应用没问题。从自己代码定位问题。
带着问题看源码
org.redisson.command.CommandBatchService#executeAsync
org.redisson.RedissonBaseLock#createCommandBatchService
问题原因:
分布式锁,默认值是等1秒全部slave 同步成功master再继续,如果slave没全部成功就报错(本例中一台slave成功一台没在1s内成功)。
经过和运维确认,那个时间点redis在进行数据持久化,持久化策略RDB,运维侧看,不能避免1秒内全部完成所有节点。
解决:
升级客户端版本,3.21以上,据说3.21.* 以上,只要一个slave成功即可(懒得贴代码了)。
。文章来源:https://www.toymoban.com/news/detail-766714.html
if (getServiceManager().getCfg().isCheckLockSyncedSlaves()&& res.getSyncedSlaves() == 0 && availableSlaves > 0) {
throw new CompletionException(
new IllegalStateException("None of slaves were synced"));
}
文章来源地址https://www.toymoban.com/news/detail-766714.html
到了这里,关于redisson Unexpected exception while processing command Only 1 of 2 slaves were synced的文章就介绍完了。如果您还想了解更多内容,请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章,希望大家以后多多支持TOY模板网!