Node.js~ioredis处理耗时请求时连接数瀑增

回到目录

关于redis连接数过高的解释

对于node.js开发环境里,使用传统的redis或者使用ioredis都是不错的选择,而在处理大数据请求程中,偶尔出现了连接池( redis服务端的最大可用连接数,默认为1万)不够用的情况,一般的提示如下:

It was not possible to connect to the redis server(s); to create a disconnected multiplexer, disable AbortOnConnectFail

在redis-cli上输入info命令也可以进行查看

aaarticlea/png;base64,iVBORw0KGgoAAAANSUhEUgAAAT4AAABmCAIAAADd8GpJAAALrklEQVR4nO2ce4hc1R3HrzP7mH3Na/fO4+7OvfPcndfuzOzO5haaShrBqphQxVaxJVVZMARNI8YYRDCSaJbEEEIkvhpakVYsRLExJam0jm2sof4RIWJspCpF2hKDiFBs8k/6R3CZzL3nzLl3zt2Zs/P9/CFnfvec3/mec893z2TuuUqXL1++fPnyB++fuvLfD94/tWPHjjVr1kgAgE4G1gVASGBdAIQE1gVASGBdAIQE1gVASGBdAIQE1gVASGBdAIQE1gVASGBdAIQE1gVASGBdAIQE1gVASGBdAIQE1gVASCjWrdVqtVqtvtCUWh30YMPVlscBQJfB0br11YxNHLUuzA+6DpJ1a2ZQ8pjutJQKHMG+DboRXruuVW9L5C/SpnHSHxHK3xfGPzoACEkbrWsar//IUiZlptQHYCXQmdY1bphWrcsoGwBRafu/dW34kLEJy1UARKXTfmFm2WlJTRjrA7ASWObnukZH0eMNSYzlhiZNxQCwQsBpKgCEBNYFQEhgXQCEBNYFQEhgXQCEBNZdZlyS5HO7x/v6sh5PdWhojde7LhD4YSCwQZbvluWFUOjecHhTOLw5EnkgGn1IUbYryqPj44+Oj29XlIcU5YFodHMksikcvjccXgiF7pblDbJ8SyBwcyCwxuudGxrKejzjfX0+t9vV7pECZ4F1OeKSpERvrz48vD4QWAiFHhkfPxCPv5zJ/CmfP1Mq/aNS+XrVqou6fqFa/Wx29kyp9G6x+GYu9+rk5KKqHk6lnk8mDyUSTycSB+LxfZq2R1WfjMV2xmKPTUw8NjGxMxZ7Mhbbo6r7NO1APP50IvFMIvF8Mnk4lVqMxV6bmnozlztVLJ4plT6bnb1QrV7U9a9XrTpXKp0plf6Yz7+cyRyIxx8ZH18IhdYHAvrwcLyv75p2zxiwz4q3buuPdk2fDw+7XOXBwVuDwXM7dvzz/vtP5HIfVyoXdf332ezJQuHI5OShROLxiYlN4fBtweD3RkayHo/c0zPgWo69cEnqgMs11tOT9Xiu9XpvCwY3hcOPT0w8k0gcmZw8WSi8kc1e0vVz5fLxXO5QIrFVUW4NBsuDg0PXwNEiIJZ17TmQy6mMk0ePvv/ss9sV5beTkx9XKn8tFN6bnn4lk/l027YPFxfXer3x/n7byTkeGrH6pyrR33+dz7cQCu1W1Vcymfemp08WCufK5VcymYcV5XqfT+7p4aUN8EQg69p2oO2GYz09d8nyfk37tFL579q15zds2BmL3RIMtuJSLvJI/uR1gCzR339rMLgrFjuWzf5rbu6TSmW/pv1Mlkdh486h6UFIxgOGtatpGm8xP+kSKQ/LUl6q+c6rr25XlJOFwr/n5l5IJm/w+Sb6+vjqb2VcEvXgp8TpW0Y9sb6+G32+X6RS/5mb+0uh8LCiKPBw26G/OXSlTieUjR9t5zHlSh2XJD2laV/cfvsn27frw8Okjihxp8dF189e3zbfGR7eFYv9uVDYo6oOdQGYYLFuPXxtWY+xDqXrhiA9D+PSP/naaxd1/eeRCHvvxrgNPYyZm+onde0QD0Sjl3Q9jO23XbTXuvT8LJes6iRRq9W+Wb16mPwLMKNgG3ro8iijtjRRTuBzuy9Uq8vQETBBxC/MLeYx5aNduz7auZNSn5SE4kMnxlUfIQ2KZby8uC8S2Uz4ngKche/PVOxlq/kbLrHUrw+yrOZvrr327WPH6PmNw6HHG5KYzoPVcdGx18oeQZfr/Nyc070AcwR6OMQFo9+WVnmkp+fiqlUPRqPt1igGDynK/3RddrvbLaRb6TbrNmVRVd/O53er6ndHRtqtpRNZPTKyqKrv5PNPxmLt1tLdwLpGIm73VkWp5fNfVKu/TKVu9vv5nsEQjkR//zqf71ep1IVq9a18/sFoNITNtu3AuhT8bvdPx8aeUtWPK5Xz1erxXG63qv5odDTt8bRbmrNkPJ4fj44uquqJXO58tXquXN6rqj8ZG/PBsZ0DrMtIwO2+zufbqii/yWTOlsunisXTMzNHJif3atrGcPh6ny/t8Qh3bN8lSRmP5wc+38ZweK+mHZmcPD0z826x+GGp9Ot0+sFodK3XG4BdOxNY1x4DklQcGFgfCGyJRg/G48ey2bPl8iVdP5HNnioWX5+aej6ZfCIW2xyJ3DE6+n2vtzgwEO3tHXG7l83e10jSiNsd6e0tDgys9XrvGB3dHIk8EYu9kEy+PjV1qlg8nstd1PUPS6U3stmD8fiWaHR9IFAcGPDgzSEhgHX5MtHbOzc0dJPff7csb1OUfZr2Ujr9h1zu9MzM30ulL+fnL+n6V/Pzn8/Oni2X/zY9/VY+/7upqT2q+mI6fTiVei6ZPJRIHIzH92vaXk3brar17+vuVtW9mrZf0w7G44cSieeSycOp1Ivp9F5NO5rNvpXPvzc9fbZc/nx29qv5+Uu6/uX8/NlS6fTMzIlc7qV0ep+mbVOUu2T5Jr9/bmhoAgehhMaGdelPDpfniSK7ng5kyOUK9/amPZ7K4ODqkZEb/f51fv+dY2MbZPkeWV4IhTaGw/dFIlui0a1X/18ytirKlmj0vkhkYzi8EArdI8sbZPnOsbF1fv8Nfv/qkZHy4GDa4wn39g458GJwu+678TEekKQWdl1Hb6GNDB11X51Wwis/x3m2NP+WatYIJ1u6He7WbR17mTvnpjqthFf+ds0zJYNxa63/iI33KmwchLyCMU6qbxqvXQ0pzn6fWuxXunplmK4YG/lZ9FPykMrGJg3KKW1t6zQmoeinj6tpE2N3NqSucKy+frAEaRJJU89YpmSm02K/vMpW9XPMb9rE6XkmxbnPj3E43Y7Vl/6WsHQL6zFWc2JJsfTFXjbqp+i0tzQZy4z9LpWdnmdKnOP8mA6n21ke69KrObGkrFqCpczSr1X9NjRYmlKn59lGnNJv7VtIEXuaVyad/IXZ0n3i1S9lrTihn2N+0ya8dJr2Qonb67dh8ulpu50W39c13iF6vCGJsdzQhGUIHPs1jbPMA0mSVf2kJCRJpDykyix5WHQap5EeJ6Wy0S9Lk65AlNNUxvXB90Ya1xyvzPU5ndMvcZK9DDoBH0Sx7jIg9EoVWjywA6wLgJDAugAICawLgJDAugAICawLgJCI/r5u059VRfnRtUN02vuZ2lIrPHbiwwp4X7cDrWujR0d1cpxPSn2Whg114F77rID3dTvt9ouuh2N949Za/xEbb0usgPd1WVKZ1pfIK8k0bmNcVvUzDkoyrHt62baepvmNH01TUfSzSAImdPLrB4zQpbaoZxn0W9JJuuT0fNLHvgR7cmNaYI0V8L4uJRU9OXvZdHUyzoCNUdAHZXrJ6fnklb9hJu3NFZCkFfG+LiUVPbnVctPe7em3pJN0yen5tJe/9i2kiL25ApLU2V+YGe8rXSopQllVVjW3qJ+ehPGS0/PJqMc0FaWypSkCVyH6+7oNa8vYBWkUpv2axlnmwTQDi36STsr8kAZFam5JT0OGpvkpk8A4XsZWoBFRTlMZ17e9G0+yBKnMC176ecnjqAe0B1GsyxHSShViBQshEiwHXWhdAFYCsC4AQgLrAiAksC4AQgLrAiAk9CMZpj9jcvmR03YGjl1TRmc7OUfw2AbQaHokg9Sw9fVkOwOXpeyodbnLg3tBI7Aud2xkNm6t9R+x8QITGA9CGteNMUiqTI+TyvQ8LOuYXrlF/SQxNQMsekiVm44CdC+Muy5pQZt+ZClLhtVpO48pTeuTknAclyU9pPrGKQJAkpbXug3U6jAN2shJksdeoXU9jJlZ9C81sdQWdAVtty5j/fZa15IeetemV+tdahqBdUEjnfCFWWKwcSvWpbiF3rDFcbHrr30LiwwAJInhVXvSqqrHGK/vwjRu2pYxj2TFvXQ9pH6b6iGJNx0dXQ+LfsYmoIsQ/TSV0W9Y6KArEN26AHQpsC4AQgLrAiAksC4AQgLrAiAksC4AQgLrAiAksC4AQgLrAiAksC4AQgLrAiAksC4AIvJ/uW6JWA6QdgIAAAAASUVORK5CYII=" alt="" />

redis-server.conf里配置了它默认的最大连接数

maxclients 10000

产生它的原因有几个:

  1. 单个请求使用结果后,没有释放,client.end()没有使用,这主要是redis组件
  2. 而使用了ioredis组件后,需要redis会自动释放,但时机也是http请求结束之后才执行,所以对于长时间没有响应的请求,也会出现占用redis线程的问题,解决方法手动使用redis.quit()即可
  3. 单个请求时间过长,导师redis连接一直被一个请求占用,而在请求数过多时,这种现象就会引用连接池不够用
  4. 多线程环境下(非node.js),使用了实例模块,而没有使用单例模式,因为很多redis驱动是支持多路复用的

大叔建议的作法:

减少单次请求的响应时间,建议把redis从一个大请求中拿出来,因为纯redis还是很快的

正确使用redis组件,用完就关了

正确理解多线程与socket连接,要知道socket连接直接影响你的服务器CPU性能

ioredis代码实例

ioredis是个好东西,它完全支持了redis的cluster,sentinal等新技术

new Redis()       // Connect to 127.0.0.1:6379
new Redis(6380) // 127.0.0.1:6380
new Redis(6379, '192.168.1.1') // 192.168.1.1:6379
new Redis('/tmp/redis.sock')
new Redis({
port: 6379, // Redis port
host: '127.0.0.1', // Redis host
family: 4, // 4 (IPv4) or 6 (IPv6)
password: 'auth',
db: 0
})

同时支持标准的字符连接串

// Connect to 127.0.0.1:6380, db 4, using password "authpassword":
new Redis('redis://:authpassword@127.0.0.1:6380/4')

支持发布与订阅,它会存储在进程里,它不会被持久化,所有会有消息丢失的情况

var Redis = require('ioredis');
var redis = new Redis();
var pub = new Redis();
redis.subscribe('news', 'music', function (err, count) {
// Now we are subscribed to both the 'news' and 'music' channels.
// `count` represents the number of channels we are currently subscribed to. pub.publish('news', 'Hello world!');
pub.publish('music', 'Hello again!');
});

好了,下次我们有时间去讲讲ioredis的具体操作!

感谢各位的阅读!

回到目录

上一篇:nginx加载字体文件的时候报错


下一篇:ArcGIS客户端API中加载大量数据的几种解决办法