RedisClient是一款纯java开发的开源客户端,原版本:https://github.com/caoxinyu/RedisClient,作者目前已经基本不再维护,最近想要使用一下,结果发现已经开始各种异常。应该是很久没更新的缘故。由于我们公司使用的哨兵模式,而且查看客户端的jedis版本确实有些古老并且发现使用的是单机版的Jedis,难怪会出现异常。例如:ERR unknown command 'AUTH’
肿么办?看了下介绍代码是开源的并且是纯java开发,要不自己改一改?好吧,开始我们的趟坑之旅
本文修改后的RedisClient版本:https://github.com/GallantKong/RedisClient
升级为Sentinel客户端可行性确认
- 比较生猛的直接找到JedisCommand将其中的Jedis实例创建改为从Sentinel连接池中获取
- 哈哈,果然一切都变得顺畅了,连接正常了。但是在我点击某个db时发现会卡死。。。于是准备放弃点击关闭客户端的按钮发现客户端恢复了,不再卡在那里不动了,而且db下的key等信息全部刷新正常了。。。
客户端卡死问题分析
卡死时与正常时的堆栈比对一哈,当然要感谢一波IBM大婶们提供的开源工具(IBM Thread and Monitor Dump Analyzer for Java),很好用,可以直接定位到唯一的不同点就在main线程内。下面我们看下main线程的堆栈
卡死时main线程的堆栈先搞一波
//仅截取了堆栈异常的地方,下面的堆栈是客户端一直卡死时导出的
"main" #1 prio=5 os_prio=0 tid=0x0000000002be3800 nid=0x9680 runnable [0x0000000002a0e000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.net.SocketInputStream.read(SocketInputStream.java:127)
at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:195)
at redis.clients.util.RedisInputStream.readByte(RedisInputStream.java:40)
at redis.clients.jedis.Protocol.process(Protocol.java:132)
at redis.clients.jedis.Protocol.read(Protocol.java:196)
at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:288)
at redis.clients.jedis.Connection.getIntegerReply(Connection.java:213)
at redis.clients.jedis.Jedis.ttl(Jedis.java:292)
at com.cxy.redisclient.integration.JedisCommand.isPersist(JedisCommand.java:85)
at com.cxy.redisclient.integration.key.ListContainerKeys.command(ListContainerKeys.java:74)
at com.cxy.redisclient.integration.JedisCommand.runCommand(JedisCommand.java:45)
at com.cxy.redisclient.integration.JedisCommand.execute(JedisCommand.java:30)
at com.cxy.redisclient.service.NodeService.listContainerKeys(NodeService.java:96)
at com.cxy.redisclient.presentation.RedisClient.tableItemOrderSelected(RedisClient.java:2651)
at com.cxy.redisclient.presentation.RedisClient.dbContainerTreeItemSelected(RedisClient.java:2557)
at com.cxy.redisclient.presentation.RedisClient.treeItemSelected(RedisClient.java:2503)
at com.cxy.redisclient.presentation.RedisClient.selectTreeItem(RedisClient.java:3274)
at com.cxy.redisclient.presentation.RedisClient.access$2000(RedisClient.java:95)
at com.cxy.redisclient.presentation.RedisClient$20.widgetSelected(RedisClient.java:616)
at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at com.cxy.redisclient.presentation.RedisClient.open(RedisClient.java:212)
at com.cxy.redisclient.presentation.RedisClient.main(RedisClient.java:194)
正常时main线程堆栈当然也要搞一波哈哈
//我擦,一看就很正常,哈哈哈
"main" #1 prio=5 os_prio=0 tid=0x0000000002be3800 nid=0x9680 runnable [0x0000000002a0f000]
java.lang.Thread.State: RUNNABLE
at org.eclipse.swt.internal.win32.OS.WaitMessage(Native Method)
at org.eclipse.swt.widgets.Display.sleep(Unknown Source)
at com.cxy.redisclient.presentation.RedisClient.open(RedisClient.java:213)
at com.cxy.redisclient.presentation.RedisClient.main(RedisClient.java:194)
- 可以看到线程执行在ListContainerKeys命令中判断key是否是持久类型这个动作。并没有阻塞,于是我们断点查看一下,35W+的key需要封装为DataNode类型缓存你在本地keys。。。并且这个动作是同步执行的,所以给用户的感觉就是客户端卡死了,什么都不可以操作。。。
断点查看此处处理真得是巨慢,所以异步一下?如果异步了疯狂点会不会吃满线程池?当然会那么怎么办?断点续跑?如果keys超级多会不会吃爆本地内存?当然会。。。
看来我们需要解决的问题还是有一些的。。。
集群模式
集群模式不支持select db命令
看上去没什么,但是搜索了一下,调用此命令的地方是真滴多,一个个修改?好绝望,当然可以使用代理模式啊。ok动态代理搞起来,结果竟然抛出了连接被拒绝的异常。。。如果不适用代理就不会抛出该异常,是什么原因导致的呢?先贴下异常堆栈
redis.clients.jedis.exceptions.JedisConnectionException: java.net.ConnectException: Connection refused: connect
at redis.clients.jedis.Connection.connect(Connection.java:155)
at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:83)
at redis.clients.jedis.Connection.sendCommand(Connection.java:107)
at redis.clients.jedis.BinaryClient.info(BinaryClient.java:841)
at redis.clients.jedis.BinaryJedis.info(BinaryJedis.java:2665)
at redis.clients.jedis.Jedis$$EnhancerByCGLIB$$b30bc1af.CGLIB$info$250(<generated>)
at redis.clients.jedis.Jedis$$EnhancerByCGLIB$$b30bc1af$$FastClassByCGLIB$$6427e67d.invoke(<generated>)
at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
at com.cxy.redisclient.integration.JedisProxy.intercept(JedisProxy.java:34)
at redis.clients.jedis.Jedis$$EnhancerByCGLIB$$b30bc1af.info(<generated>)
at com.cxy.redisclient.integration.JedisCommand.getRedisVersion(JedisCommand.java:94)
at com.cxy.redisclient.integration.JedisCommand.runCommand(JedisCommand.java:42)
at com.cxy.redisclient.integration.JedisCommand.execute(JedisCommand.java:32)
at com.cxy.redisclient.service.ServerService.listDBs(ServerService.java:109)
at com.cxy.redisclient.presentation.RedisClient.serverTreeItemSelected(RedisClient.java:2760)
at com.cxy.redisclient.presentation.RedisClient.treeItemSelected(RedisClient.java:2499)
at com.cxy.redisclient.presentation.RedisClient.selectTreeItem(RedisClient.java:3274)
at com.cxy.redisclient.presentation.RedisClient.access$2000(RedisClient.java:95)
at com.cxy.redisclient.presentation.RedisClient$20.widgetSelected(RedisClient.java:616)
at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at com.cxy.redisclient.presentation.RedisClient.open(RedisClient.java:212)
at com.cxy.redisclient.presentation.RedisClient.main(RedisClient.java:194)
Caused by: java.net.ConnectException: Connection refused: connect
at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at redis.clients.jedis.Connection.connect(Connection.java:149)
... 25 more
出现该异常的代理实现代码
public class JedisIProxy implements MethodInterceptor {
public Object getInstance(Object target) {
this.target = target;
Enhancer enhancer = new Enhancer();
enhancer.setSuperclass(this.target.getClass());
enhancer.setCallback(this);
return enhancer.create();
}
@Override
public Object intercept(Object obj, Method method, Object[] args, MethodProxy proxy)
throws Throwable {
return proxy.invokeSuper(obj, args);
}
}
改用另一个接口实现代理则是正常,不会出现上面的connect被拒绝的异常,实现代码如下
public class JedisProxy implements InvocationHandler {
private Object target;
@Override
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
// 集群模式不支持select命令
if (server != null && server.isJedisClusterType() && SELECT.equals(method.getName())) {
return null;
}
return method.invoke(target, args);
}
public Object getInstance(Object target) {
this.target = target;
Enhancer enhancer = new Enhancer();
enhancer.setSuperclass(this.target.getClass());
enhancer.setCallback(this);
return enhancer.create();
}
}
cglib字节码代码
配置路径cglib会在生成字节码时同时保存至文件:System.setProperty(“cglib.debugLocation”,“D:/tmp/cglib”);
可以看到总共生成了3个代理类;其中BinaryJedis是Jedis的父类,可以看出来Jedis代理生成的同时,父类也生成了相应的动态代理字节码
我们跟着异常堆栈看到三个文件的执行顺序
- JedisEnhancerByCGLIBb30bc1af.info
- 内部类调用:JedisEnhancerByCGLIBb30bc1afFastClassByCGLIB6427e67d.invoke
- 代理方法调用:JedisEnhancerByCGLIBb30bc1af.CGLIB$info$250
- 还有一个是BinaryJedis的代理类实现:BinaryJedisFastClassByCGLIB47dad0be(暂不关注)
Jedis代理类代码JedisEnhancerByCGLIB19cf8dd3
字节码代码梳理
Jedis代理info
public final String info() {
//动态代理实现的MethodInterceptor接口对象的实例,当前案例中即JedisProxy
MethodInterceptor var10000 = this.CGLIB$CALLBACK_0;
if (this.CGLIB$CALLBACK_0 == null) {
CGLIB$BIND_CALLBACKS(this);
var10000 = this.CGLIB$CALLBACK_0;
}
//存在callback回调则回调intercept拦截方法
//this即字节码生成的jedis动态代理类
//CGLIB$info$249$Method:源码中可以看到是原始方法
//CGLIB$emptyArgs:原始方法的入参列表
//CGLIB$info$249$Proxy:根据原始方法cglib生成的代理方法
return var10000 != null ? (String)var10000.intercept(this, CGLIB$info$249$Method, CGLIB$emptyArgs, CGLIB$info$249$Proxy) : super.info();
}
//代理类实例属性
CGLIB$info$249$Method = var10000[22];
CGLIB$info$249$Proxy = MethodProxy.create(var1, var0, "()Ljava/lang/String;", "info", "CGLIB$info$249");
MethodInterceptor的实现中就是我们自己的代码,即直接调用代理方法MethodProxy.invokeSuper。可以看到是直接调用的fastclass代理类的invoke方法
public Object invokeSuper(Object obj, Object[] args) throws Throwable {
try {
init();
FastClassInfo fci = fastClassInfo;
return fci.f2.invoke(fci.i2, obj, args);
} catch (InvocationTargetException e) {
throw e.getTargetException();
}
}
调用fastclass代理类JedisEnhancerByCGLIB19cf8dd3FastClassByCGLIB16a06cad.invoke方法,堆栈中可以看到调用Jedis代理类的info方法
//可以看到fastclass其实就是一个根据指令index调用Enhance增强的代理类的适配器。
//将所有的调用根据index路由到相应的方法
public Object invoke(int var1, Object var2, Object[] var3) throws InvocationTargetException {
//转换为cglib生成的Jedis代理类型:Jedis$$EnhancerByCGLIB$$19cf8dd3
19cf8dd3 var10000 = (19cf8dd3)var2;
int var10001 = var1;
try {
//根据指令码调用对应的代理类方法,例如:shutdown,connect,info等等
switch(var10001) {
case 0:
return var10000.shutdown();
case 1:
return var10000.get((String)var3[0]);
case 2:
return var10000.get((byte[])var3[0]);
case 3:
return var10000.type((String)var3[0]);
case 4:
return var10000.type((byte[])var3[0]);
...
}
//Jedis代理类的info方法
final String CGLIB$info$250(String var1) {
return super.info(var1);
}
Jedis代理类的info方法直接调用父类也就是Jedis的info方法,Jedis的info方法继承了BinaryJedis,也就是最终调用BinaryJedis的info方法,我们前面所看到的BinaryJedis的fastclass(BinaryJedisFastClassByCGLIB47dad0be)生成了就不会不使用,我们通过debug验证我们的推测
我们发现当前的父类已经不是Jedis的原始父类,因为我们的Jedis连接host、port均是指定的配置的,当前却均变成了localhost等等一看就是兜底的配置,使用这些配置连接不超时才怪!!!-_-。。。
是BinaryJedis的fastclass生成的父类有问题吗?
其实不是动态代理生成的实例有问题,而是我个人对接口的使用及理解错误导致的。实现MethodInterceptor接口并通过Enhance创建增强类本来就是通过指定的构造器类型创建实例,并为指定的target目标类的每个方法生成intercept拦截,而我们此处使用的是无参构造器创建enhancer.create(),当然走的都是默认的兜底配置,连接固然会失败
最终我不得不选择InvocationHandler接口来实现代理,因为我们的jedis实例是有各种工厂类提供,如果我自己再重新根据参数创建的话有两个坏处
SWT
Composite先后实例化顺序决定了按钮的位置顺序