我正在使用pyzmq从经销商向ROUTER发送20000条消息.

当我在每条消息之间暂停0.0001秒时,它们都会到达,但如果我通过暂停每条消息0.00001将它们发送速度提高10倍,则只有大约一半的消息到达.

是什么导致了这个问题？

解决方法:

是什么导致了这个问题？

ZMQ IO线程的默认设置 – 负责操作模式.

如果你投入时间并深入了解优秀的ZMQ概念和架构,我敢于称之为问题.

从早期版本的ZMQ库开始,就有一些重要的参数可以帮助中心杰作(IO线程)保持稳定和可扩展的理由,从而为您提供这个强大的框架.

零共享/零复制/(几乎)零延迟是不能实现零成本的格言.

ZMQ.Context实例具有相当丰富的内部参数化,可以通过API方法进行修改.

让我引用一个奇妙而宝贵的资料–Pieter HINTJENS的书,Code Connected,第1卷.

(绝对值得花时间和逐步完成PDF副本.C语言代码片段不会伤害任何人的pythonic心态,因为关键信息在Pieter已经制作成300个令人兴奋的页面的文本和故事中).

High-Water Marks

When you can send messages rapidly from process to process, you soon discover that memory is a precious resource, and one that can be trivially filled up. A few seconds of delay somewhere in a process can turn into a backlog that blows up a server unless you understand the problem and take precautions.

…

ØMQ uses the concept of HWM (high-water mark) to define the capacity of its internal pipes. Each connection out of a socket or into a socket has its own pipe, and HWM for sending, and/or receiving, depending on the socket type. Some sockets (PUB, PUSH) only have send buffers. Some (SUB, PULL, REQ, REP) only have receive buffers. Some (DEALER, ROUTER, PAIR) have both send and receive buffers.

In ØMQ v2.x, the HWM was infinite by default. This was easy but also typically fatal for high-volume publishers. In ØMQ v3.x, it’s set to 1,000 by default, which is more sensible. If you’re still using ØMQ v2.x, you should always set a HWM on your sockets, be it 1,000 to match ØMQ v3.x or another figure that takes into account your message sizes and expected subscriber performance.

When your socket reaches its HWM, it will either block or drop data depending on the socket type. PUB and ROUTER sockets will drop data if they reach their HWM, while other socket types will block. Over the inproc transport, the sender and receiver share the same buffers, so the real HWM is the sum of the HWM set by both sides.

Lastly, the HWM-s are not exact; while you may get up to 1,000 messages by default, the real buffer size may be much lower (as little as half), due to the way libzmq implements its queues.