消息队列各种比较

hh3755 2016-01-06

展开全文

http://blog.csdn.net/archimedes_zht/article/details/7401210

最近的工作需要用到MessageQueue来做“任务分发”，自己写了一个简单的，但是感觉不够满意。主要还是感觉消息队列持久化要做的好很难，还有各种异常情况等等，短时间开发一个，挑战很大，还是找一个开源的实现吧。

先是找到了RabbitMQ，是erlang写的，而我们的系统是用Go写的，目前它还没有Go的绑定，部署系统的时候也要装erlang虚拟机。考察了之后决定放弃。

然后是最近大名鼎鼎的ZeroMQ，第一次听说是在云风大哥的博客上（http://blog./2011/02/zeromq_message_patterns.html），我后来在还没有用它写代码之前就分析过它的源码了（http://blog.csdn.net/archimedes_zht/article/details/7342937，http://www./docs:omq-2-1-class-diagram）。:D

分析代码的时候就知道它没有分发任务、执行任务时候的各种确认，任务的持久化以防止任务丢失。不过ZeroMQ可以把内存中放不下的消息先写到磁盘上去，然后再读进内存（见swap_t类）。这次再大致看了下它的文档，可以确认它的名字虽然是MQ，但是它提倡的是“结构化网络编程”，它建立了非常灵活的编写网络程序的基础设施，可以用在各种场合，不过这些场合都需要你自己利用它提供的基础设施来定制（比如我现在急切需要的各种任务分发相关的特性）。目前知道的是它可以创建一个queue的device。不过这个目前和我的“任务分发”需求相差很远。以后有空再研究这个精彩的、雄心勃勃的项目吧。

然后是Gearman（http:///），它是专门做任务分发的。任务可以分为前台任务（同步任务）和后台任务（异步任务），而且提供了任务的持久化机制（配置一个数据库就行了，支持MySQL，Sqlite3等），非常符合我的胃口。:) 而且 mikespook大侠已经提供好了Gearman协议的Golang实现（http://www./2011/05/go-%E8%AF%AD%E8%A8%80%E7%9A%84-gearman-api/）代码也实现的非常漂亮，学习了。而且我听说有Gearman这个好东东也是看了mikespook大侠的Go语言介绍才知道的（http://ecug./svn/trunk/ecug-con/2011/mikespook/）。

我个人觉得Gearman的文档最值得看的就是它的协议交互了（http:///index.php?id=protocol ），另外就是它的C接口使用（http:///docs/dev/ ），分为Client和Worker两部分。

不过在下载安装Gearman的时候，发现竟然要先装Boost，唉。。。
另外如果Gearman仍然是C开发的就好了，为什么要转向C++呢。。。搞不懂，不明白。。

最后以查找资料的时候，国外牛人对各种“MQ”的总结结束。

RabbitMQ、ZeroMq和ActiveMQ的比较：

RabbitMQ is one of the leading implementation of the AMQP protocol (along with Apache Qpid). Therefore, it implements a broker architecture, meaning that messages are queued on a central node before being sent to clients. This approach makes RabbitMQ very easy to use and deploy, because advanced scenarios like routing, load balancing or persistent message queuing are supported in just a few lines of code. However, it also makes it less scalable and “slower” because the central node adds latency and message envelopes are quite big.

ZeroMq is a very lightweight messaging system specially designed for high throughput/low latency scenarios like the one you can find in the financial world. Zmq supports many advanced messaging scenarios but contrary to RabbitMQ, you’ll have to implement most of them yourself by combining various pieces of the framework (e.g : sockets and devices). Zmq is very flexible but you’ll have to study the 80 pages or so of the guide (which I recommend reading for anybody writing distributed system, even if you don’t use Zmq) before being able to do anything more complicated that sending messages between 2 peers.

ActiveMQ is in the middle ground. Like Zmq, it can be deployed with both broker and P2P topologies. Like RabbitMQ, it’s easier to implement advanced scenarios but usually at the cost of raw performance. It’s the Swiss army knife of messaging :-).

Finally, all 3 products:
(1)have client apis for the most common languages (C++, Java, .Net, Python, Php, Ruby, …)
(2)have strong documentation
(3)are actively supported

Gearman与RabbitMQ的比较：

I would say that Gearman is better for queuing "jobs" and RabbitMQ is better for queuing "data". Of course, they are both really the same thing, but the way it works out for me is that if you are trying to "fan out" work to be done, and the workers can work independently, Gearman is the better way to do it. But if you are trying to feed data from a lot of sources down into fewer data consumers, RabbitMQ is the better solution.

The history of RabbitMQ, as something that allowed Twitter to take bursty loads of messages, and feed them into crusty old SMS gateways that could keep only one connection open, were rate limited, and didnt have retries, is illustrative of the kind of problems that RabbitMQ is good at solving.

一篇文章：

常见的开源消息系统——网站公告、通知、消息的实现方式

消息系统的作用：异步处理、削减峰值、减少组件之间的耦合。

选择消息系统根据业务需要需要考虑以下几个方面：