Message Queue属于比较重量级的应用，在规范化的企业流程中用的比较多。如果企业有很多部门，部门都有自己的系统，那么不同的系统之间的集成通信，Message Queue是很好的选择。MQ一般都做为企业级IT应用的中间件存在，有很多企业是作为标准IT基础结构存在的。在市面上常见的MQ中间件有IBM websphere message queue service，Oracle Advanced Queuing，Microsoft Message Queue(MSMQ)，Apache ActiveMQ等
如果使用WebService的话，就要写很多的WebService的代码，去建立这些WebServcie，然后暴露出这些接口，相互之间调用，很费事。但是如果使用Message Queue的话，只要把这个中间件的服务器搭建起来，只要在需要的时候加入不同的Queue Manager就可以了，然后就可以访问了，就可以作为不同系统之间的桥梁了。
长耗时的报表，这个在程序中经常遇见，处理海量数据时，可能生成一个报表需要5分中或是更长的时间，客户不能在线实时等待，报表处理比较耗费资源，不能同时处理很多请求，甚至同时只允许处理一个，这时就可以使用MQ。客户端将报表请求和一些必要的报表条件放到Queue中，报表由另一个服务一个一个的处理，处理好后再给用户发一个消息（MSN消息，或mail等）用户再在浏览器或其他报表浏览器中查看报表。
在线商店，在客户下订单的过程后，系统只需做减库存、记录收货人信息和必要的日志，其他的必须配送处理、交易统计等其他处理可以不同时完成，这时就可以将后续处理消息放入Queue中，让另一台（组）服务器去处理，这样可以加快下订单的过程，提高客户的体验；
WebService通常是实时性要求较高，Client端向Server端发出请求后，这是一个短连接，一个Http请求，这个请求发出后，Client端就会一直等到获取到这个结果。但是使用MQ的话，因为有了中间的这一块区域，当请求发出后，Client端可以继续去干别的事情。等到一段时间以后再去中间件的存储区域上查看一下有结果了么，有了结果就取出来，没有的话就再等会再看。
MessageQueue在于容错处理保持数据一致性上有很大的优势。下边的例子很好的说明了这一点。
Introduction
In this article, I will introduce a new and independent Open Source Message Queue system that is entirely built in C# and .NET framework 3.5. DotNetMQ is a message broker that has several features including guaranteed delivering, routing, load balancing, server graphs... so on. I will start by explaining messaging concepts and the need for message brokers. Then I will examine what DotNetMQ is and how to use it.
What Is Messaging?
Messaging is a way of asynchronous communication of applications running on same or different machines with reliable delivery. Programs communicate by sending packets of data called messages to each other [1].
A message may be a string, a byte array, an object... etc. Typically, a sender (producer) program creates a message and pushes it to a message queue and a receiver (consumer) program gets the message from the queue and processes it. The sender and receiver programs dona€?t have to be running at the same time, since messaging is an asynchronous process. This is called loosely coupled communication.
On the other hand, a Web Service method call (Remote Method Invocation) is a type of tightly coupled andsynchronous communication (both applications have to be running and available during the whole communication; if the Web Service is offline or an error occurs during the method call, the client application gets an exception).
In the figure above, two applications communicate over a message queue in a loosely coupled manner. If the receiver consumes messages slower than the sender produces it, the message count on the queue will increase. Also, the receiver may be offline while the sender is sending messages. In this situation, the receiver gets the messages from the queue when it becomes online (when it starts and joins the queue).
Message Queues are typically provided by Message Brokers. A Message Broker is a standalone application (service) that other applications connect to and send/receive messages. A Message Broker is responsible to store messages until a receiver receives them. A Message Broker can route messages across machines to deliver a message to the destination application and can try delivering the message until the receiver correctly handles it. A Message Broker is sometimes called a Message Oriented Middleware (MOM) or simply Message Queue (MQ).
What is DotNetMQ?
DotNetMQ is an open source Message Broker that has several features:
- Persistent or non-persistent messaging.
- Guaranteed delivery of persistent messages even in a system crash.
- Automatic and manual routing of messages in a custom machine graph.
- Supports multiple databases (MS SQL Server, MySQL, SQLite, and memory-based storage for now).
- Supports dona€?t store, direct send style messaging.
- Supports Request/Reply style messaging.
- Easy to use client library to communicate with the DotNetMQ Message Broker.
- Built-in framework to easily construct RMI services upon message queues.
- Supports delivering messages to ASP.NET Web Services.
- GUI-based management and monitoring tool.
- Easy to install, manage, and use.
- Written entirely in C# (using .NET Framework 3.5).
I preferred to name DotNetMQ as MDS (Message Delivery System) when first creating it. Because it is designed not just to be a message queue, but also as a system that delivers messages directly to applications and an environment that provides a framework to build application services. I called it DotNetMQ since it is entirely developed using .NET and the DotNetMQ name is more memorable. So, ita€?s original name (and internal project name) is MDS and the applications have many classes with the prefix MDS.
The Need for a Message Broker
First, I will demonstrate a simple situation where a message broker is needed.
In my experiences in business life, I‘ve observed really bad and uncommon asynchronous enterprise application integration solutions. Usually there is an application that runs on a server and performs some tasks and produces data, and then sends the result data to another application on another server. The second application performs other tasks on the data or evaluates the result (the servers are on the same network or connected over the internet). Also, the message data must be persistent. Even if the remote application is not working or the network is not available, themessage must be delivered on the first chance.
Leta€?s look at the design in the figure below.
Application - 1 and Application - 2 are executable applications (or Windows services) and Sender Service is a Windows service. Application - 1 performs some task, produces data, and calls a Remote Web Service method onServer - B to transmit data. This Web Service inserts data into a database table. Application - 2 periodically checks the table for new incoming data rows and processes them (and deletes them from the table or marks them as processed to not process the same data again).
If an error occurs during the Web Service call or while processing data in the Web Service, data must not be lost and must be sent later. However, Application - 1 has other tasks to do, so it can not try to send data again and again. It simply inserts data into a database table. Another Windows service (or a thread in Application - 1, if the application always runs) checks this table periodically and tries to send data to the Web Service until data is successfully sent.
This scenario is really reliable (messages are guaranteed to be delivered) but is not an efficient way of communicating between two applications. This solution has some very critical problems:
- It takes a long time to develop (to code).
- Individual coding for all message types (or remote method calls). For a new Web Service method call, you must change all the services, applications, and database tables.
- Almost same software and structures must be developed (or copied and modified) for every similar service.
- Testing and maintenance of too many services/applications/databases after coding.
- Some applications and services periodically check the database even if there is no new message (if the database is not well indexed and optimized, this may consume serious system resources).
Message Brokers do all this job and takes all the responsibility to deliver messages to the remote application in the most efficient way. The same application integration using DotNetMQ is shown in the figure below.
DotNetMQ is a standalone Windows service that runs on both Server - A and Server - B. Thus, you just need to write code to communicate with DotNetMQ. Using the DotNetMQ Client Library, it is very easy and fast to connect and send/receive messages to/from the DotNetMQ service. Application - 1 prepares the message, sets the destination, and passes the message to the DotNetMQ Broker. DotNetMQ brokers will deliver the message toApplication - 2 in the most efficient and fastest way.