RedisClient支持Sentinel与Cluster踩坑

RedisClient是一款纯java开发的开源客户端,原版本:https://github.com/caoxinyu/RedisClient,作者目前已经基本不再维护,最近想要使用一下,结果发现已经开始各种异常。应该是很久没更新的缘故。由于我们公司使用的哨兵模式,而且查看客户端的jedis版本确实有些古老并且发现使用的是单机版的Jedis,难怪会出现异常。例如:ERR unknown command 'AUTH’
肿么办?看了下介绍代码是开源的并且是纯java开发,要不自己改一改?好吧,开始我们的趟坑之旅
本文修改后的RedisClient版本:https://github.com/GallantKong/RedisClient

升级为Sentinel客户端可行性确认

  1. 比较生猛的直接找到JedisCommand将其中的Jedis实例创建改为从Sentinel连接池中获取
  2. 哈哈,果然一切都变得顺畅了,连接正常了。但是在我点击某个db时发现会卡死。。。于是准备放弃点击关闭客户端的按钮发现客户端恢复了,不再卡在那里不动了,而且db下的key等信息全部刷新正常了。。。

客户端卡死问题分析

卡死时与正常时的堆栈比对一哈,当然要感谢一波IBM大婶们提供的开源工具(IBM Thread and Monitor Dump Analyzer for Java),很好用,可以直接定位到唯一的不同点就在main线程内。下面我们看下main线程的堆栈
RedisClient支持Sentinel与Cluster踩坑

卡死时main线程的堆栈先搞一波

//仅截取了堆栈异常的地方,下面的堆栈是客户端一直卡死时导出的
"main" #1 prio=5 os_prio=0 tid=0x0000000002be3800 nid=0x9680 runnable [0x0000000002a0e000]
   java.lang.Thread.State: RUNNABLE
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:171)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at java.net.SocketInputStream.read(SocketInputStream.java:127)
	at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:195)
	at redis.clients.util.RedisInputStream.readByte(RedisInputStream.java:40)
	at redis.clients.jedis.Protocol.process(Protocol.java:132)
	at redis.clients.jedis.Protocol.read(Protocol.java:196)
	at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:288)
	at redis.clients.jedis.Connection.getIntegerReply(Connection.java:213)
	at redis.clients.jedis.Jedis.ttl(Jedis.java:292)
	at com.cxy.redisclient.integration.JedisCommand.isPersist(JedisCommand.java:85)
	at com.cxy.redisclient.integration.key.ListContainerKeys.command(ListContainerKeys.java:74)
	at com.cxy.redisclient.integration.JedisCommand.runCommand(JedisCommand.java:45)
	at com.cxy.redisclient.integration.JedisCommand.execute(JedisCommand.java:30)
	at com.cxy.redisclient.service.NodeService.listContainerKeys(NodeService.java:96)
	at com.cxy.redisclient.presentation.RedisClient.tableItemOrderSelected(RedisClient.java:2651)
	at com.cxy.redisclient.presentation.RedisClient.dbContainerTreeItemSelected(RedisClient.java:2557)
	at com.cxy.redisclient.presentation.RedisClient.treeItemSelected(RedisClient.java:2503)
	at com.cxy.redisclient.presentation.RedisClient.selectTreeItem(RedisClient.java:3274)
	at com.cxy.redisclient.presentation.RedisClient.access$2000(RedisClient.java:95)
	at com.cxy.redisclient.presentation.RedisClient$20.widgetSelected(RedisClient.java:616)
	at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
	at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
	at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
	at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
	at com.cxy.redisclient.presentation.RedisClient.open(RedisClient.java:212)
	at com.cxy.redisclient.presentation.RedisClient.main(RedisClient.java:194)

正常时main线程堆栈当然也要搞一波哈哈

//我擦,一看就很正常,哈哈哈
"main" #1 prio=5 os_prio=0 tid=0x0000000002be3800 nid=0x9680 runnable [0x0000000002a0f000]
   java.lang.Thread.State: RUNNABLE
	at org.eclipse.swt.internal.win32.OS.WaitMessage(Native Method)
	at org.eclipse.swt.widgets.Display.sleep(Unknown Source)
	at com.cxy.redisclient.presentation.RedisClient.open(RedisClient.java:213)
	at com.cxy.redisclient.presentation.RedisClient.main(RedisClient.java:194)
  1. 可以看到线程执行在ListContainerKeys命令中判断key是否是持久类型这个动作。并没有阻塞,于是我们断点查看一下,35W+的key需要封装为DataNode类型缓存你在本地keys。。。并且这个动作是同步执行的,所以给用户的感觉就是客户端卡死了,什么都不可以操作。。。

断点查看此处处理真得是巨慢,所以异步一下?如果异步了疯狂点会不会吃满线程池?当然会那么怎么办?断点续跑?如果keys超级多会不会吃爆本地内存?当然会。。。
看来我们需要解决的问题还是有一些的。。。

集群模式

集群模式不支持select db命令

看上去没什么,但是搜索了一下,调用此命令的地方是真滴多,一个个修改?好绝望,当然可以使用代理模式啊。ok动态代理搞起来,结果竟然抛出了连接被拒绝的异常。。。如果不适用代理就不会抛出该异常,是什么原因导致的呢?先贴下异常堆栈

redis.clients.jedis.exceptions.JedisConnectionException: java.net.ConnectException: Connection refused: connect
	at redis.clients.jedis.Connection.connect(Connection.java:155)
	at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:83)
	at redis.clients.jedis.Connection.sendCommand(Connection.java:107)
	at redis.clients.jedis.BinaryClient.info(BinaryClient.java:841)
	at redis.clients.jedis.BinaryJedis.info(BinaryJedis.java:2665)
	at redis.clients.jedis.Jedis$$EnhancerByCGLIB$$b30bc1af.CGLIB$info$250(<generated>)
	at redis.clients.jedis.Jedis$$EnhancerByCGLIB$$b30bc1af$$FastClassByCGLIB$$6427e67d.invoke(<generated>)
	at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
	at com.cxy.redisclient.integration.JedisProxy.intercept(JedisProxy.java:34)
	at redis.clients.jedis.Jedis$$EnhancerByCGLIB$$b30bc1af.info(<generated>)
	at com.cxy.redisclient.integration.JedisCommand.getRedisVersion(JedisCommand.java:94)
	at com.cxy.redisclient.integration.JedisCommand.runCommand(JedisCommand.java:42)
	at com.cxy.redisclient.integration.JedisCommand.execute(JedisCommand.java:32)
	at com.cxy.redisclient.service.ServerService.listDBs(ServerService.java:109)
	at com.cxy.redisclient.presentation.RedisClient.serverTreeItemSelected(RedisClient.java:2760)
	at com.cxy.redisclient.presentation.RedisClient.treeItemSelected(RedisClient.java:2499)
	at com.cxy.redisclient.presentation.RedisClient.selectTreeItem(RedisClient.java:3274)
	at com.cxy.redisclient.presentation.RedisClient.access$2000(RedisClient.java:95)
	at com.cxy.redisclient.presentation.RedisClient$20.widgetSelected(RedisClient.java:616)
	at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
	at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
	at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
	at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
	at com.cxy.redisclient.presentation.RedisClient.open(RedisClient.java:212)
	at com.cxy.redisclient.presentation.RedisClient.main(RedisClient.java:194)
Caused by: java.net.ConnectException: Connection refused: connect
	at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
	at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:589)
	at redis.clients.jedis.Connection.connect(Connection.java:149)
	... 25 more

出现该异常的代理实现代码

public class JedisIProxy implements MethodInterceptor {
    public Object getInstance(Object target) {
        this.target = target;
        Enhancer enhancer = new Enhancer();
        enhancer.setSuperclass(this.target.getClass());
        enhancer.setCallback(this);
        return enhancer.create();
    }
    @Override
    public Object intercept(Object obj, Method method, Object[] args, MethodProxy proxy)
            throws Throwable {
        return proxy.invokeSuper(obj, args);
    }
}

改用另一个接口实现代理则是正常,不会出现上面的connect被拒绝的异常,实现代码如下

public class JedisProxy implements InvocationHandler {
    private Object target;
    @Override
    public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
        // 集群模式不支持select命令
        if (server != null && server.isJedisClusterType() && SELECT.equals(method.getName())) {
            return null;
        }
        return method.invoke(target, args);
    }
    public Object getInstance(Object target) {
        this.target = target;
        Enhancer enhancer = new Enhancer();
        enhancer.setSuperclass(this.target.getClass());
        enhancer.setCallback(this);
        return enhancer.create();
    }
}

cglib字节码代码

配置路径cglib会在生成字节码时同时保存至文件:System.setProperty(“cglib.debugLocation”,“D:/tmp/cglib”);
可以看到总共生成了3个代理类;其中BinaryJedis是Jedis的父类,可以看出来Jedis代理生成的同时,父类也生成了相应的动态代理字节码RedisClient支持Sentinel与Cluster踩坑
我们跟着异常堆栈看到三个文件的执行顺序

  1. JedisEnhancerByCGLIBEnhancerByCGLIBEnhancerByCGLIBb30bc1af.info
  2. 内部类调用:JedisEnhancerByCGLIBEnhancerByCGLIBEnhancerByCGLIBb30bc1afFastClassByCGLIBFastClassByCGLIBFastClassByCGLIB6427e67d.invoke
  3. 代理方法调用:JedisEnhancerByCGLIBEnhancerByCGLIBEnhancerByCGLIBb30bc1af.CGLIB$info$250
  4. 还有一个是BinaryJedis的代理类实现:BinaryJedisFastClassByCGLIBFastClassByCGLIBFastClassByCGLIB47dad0be(暂不关注)

Jedis代理类代码JedisEnhancerByCGLIBEnhancerByCGLIBEnhancerByCGLIB19cf8dd3

字节码代码梳理

Jedis代理info

public final String info() {
    //动态代理实现的MethodInterceptor接口对象的实例,当前案例中即JedisProxy
    MethodInterceptor var10000 = this.CGLIB$CALLBACK_0;
    if (this.CGLIB$CALLBACK_0 == null) {
        CGLIB$BIND_CALLBACKS(this);
        var10000 = this.CGLIB$CALLBACK_0;
    }
    //存在callback回调则回调intercept拦截方法
    //this即字节码生成的jedis动态代理类
    //CGLIB$info$249$Method:源码中可以看到是原始方法
    //CGLIB$emptyArgs:原始方法的入参列表
    //CGLIB$info$249$Proxy:根据原始方法cglib生成的代理方法
    return var10000 != null ? (String)var10000.intercept(this, CGLIB$info$249$Method, CGLIB$emptyArgs, CGLIB$info$249$Proxy) : super.info();
}
//代理类实例属性
CGLIB$info$249$Method = var10000[22];
CGLIB$info$249$Proxy = MethodProxy.create(var1, var0, "()Ljava/lang/String;", "info", "CGLIB$info$249");

MethodInterceptor的实现中就是我们自己的代码,即直接调用代理方法MethodProxy.invokeSuper。可以看到是直接调用的fastclass代理类的invoke方法

public Object invokeSuper(Object obj, Object[] args) throws Throwable {
    try {
        init();
        FastClassInfo fci = fastClassInfo;
        return fci.f2.invoke(fci.i2, obj, args);
    } catch (InvocationTargetException e) {
        throw e.getTargetException();
    }
}

调用fastclass代理类JedisEnhancerByCGLIBEnhancerByCGLIBEnhancerByCGLIB19cf8dd3FastClassByCGLIBFastClassByCGLIBFastClassByCGLIB16a06cad.invoke方法,堆栈中可以看到调用Jedis代理类的info方法

//可以看到fastclass其实就是一个根据指令index调用Enhance增强的代理类的适配器。
//将所有的调用根据index路由到相应的方法
public Object invoke(int var1, Object var2, Object[] var3) throws InvocationTargetException {
    //转换为cglib生成的Jedis代理类型:Jedis$$EnhancerByCGLIB$$19cf8dd3
    19cf8dd3 var10000 = (19cf8dd3)var2;
    int var10001 = var1;
    try {
        //根据指令码调用对应的代理类方法,例如:shutdown,connect,info等等
        switch(var10001) {
        case 0:
            return var10000.shutdown();
        case 1:
            return var10000.get((String)var3[0]);
        case 2:
            return var10000.get((byte[])var3[0]);
        case 3:
            return var10000.type((String)var3[0]);
        case 4:
            return var10000.type((byte[])var3[0]);
		...
}
//Jedis代理类的info方法
final String CGLIB$info$250(String var1) {
    return super.info(var1);
}

Jedis代理类的info方法直接调用父类也就是Jedis的info方法,Jedis的info方法继承了BinaryJedis,也就是最终调用BinaryJedis的info方法,我们前面所看到的BinaryJedis的fastclass(BinaryJedisFastClassByCGLIBFastClassByCGLIBFastClassByCGLIB47dad0be)生成了就不会不使用,我们通过debug验证我们的推测
我们发现当前的父类已经不是Jedis的原始父类,因为我们的Jedis连接host、port均是指定的配置的,当前却均变成了localhost等等一看就是兜底的配置,使用这些配置连接不超时才怪!!!-_-。。。
RedisClient支持Sentinel与Cluster踩坑

是BinaryJedis的fastclass生成的父类有问题吗?

其实不是动态代理生成的实例有问题,而是我个人对接口的使用及理解错误导致的。实现MethodInterceptor接口并通过Enhance创建增强类本来就是通过指定的构造器类型创建实例,并为指定的target目标类的每个方法生成intercept拦截,而我们此处使用的是无参构造器创建enhancer.create(),当然走的都是默认的兜底配置,连接固然会失败
最终我不得不选择InvocationHandler接口来实现代理,因为我们的jedis实例是有各种工厂类提供,如果我自己再重新根据参数创建的话有两个坏处

  1. 重复构建,本来已经获得了jedis实例,何必再重新构建
  2. jedis均通过各种工厂创建,自己创建会破坏很多原则,比如cluster模式下的jedis是由工厂shuffle之后创建的

SWT

Composite先后实例化顺序决定了按钮的位置顺序

上一篇:使用JavaFaker生成测试数据


下一篇:Python伪数据生成工具 Faker 使用文档