Rabbitmq queue sharding

Posted by kolchanov on Stack Overflow See other posts from Stack Overflow or by kolchanov
Published on 2012-09-11T20:31:07Z Indexed on 2012/09/12 15:38 UTC
Read the original article Hit count: 321

Filed under:

rabbitmq

|

sharding

I have to implement this scenario:
An external application publish message to rabbitmq. This message has a client_id property. We can place this id to routing key or message header or some other property.
I have to implement sharding in a exchange routng logic - the message should be delivered to specific queue based on the client_id range.

Is it possible to implement in a standard exchanges?
If not what exchange should I take as the base?
How to dynamicly change client_id ranges?

© Stack Overflow or respective owner

Related posts about rabbitmq

Unable to start rabbitmq-server on Ubuntu 12.04

as seen on Server Fault - Search for 'Server Fault'
I try to install rabbitmq-server on ubuntu-server 12.04 but failed. Then I add the apt source list following the guide in http://www.rabbitmq.com/install-debian.html But reinstall still have the same error as following: $ sudo aptitude install rabbitmq-server ... Setting up rabbitmq-server (2.8… >>> More
Veewee, Vagrant, Puppet, Erlang and RabbitMQ

as seen on Server Fault - Search for 'Server Fault'
I am kinda stuck with a problem I am trying to wrap my head around for days now. Here is what I am doing: By using Veewee, I am creating a VirtualBox image and then I create a Vagrant box from it. See here, here Finally I run puppet from Vagrant to install RabbitMQ, see here. Veewee, Vagrant… >>> More
RabbitMQ and persistence (blocking writes?)

as seen on Server Fault - Search for 'Server Fault'
I want to create a RabbitMQ server on a virtual machine (VMware) to be used in production. It will contain persistent queues. I'm wondering if it is a bad idea to store the server on a NAS that's accessed over NFS. Basically my questions are: Will RabbitMQ's writes be blocking? Will the entire… >>> More
php amqp error while running make

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I'm trying to install php amqp according to this answer http://stackoverflow.com/a/9997263/2271028 but at the make command i see following: /bin/bash /var/www/rabbitmq-c/amqp-1.4.0/libtool --mode=compile cc -I. -I/var/www/rabbitmq-c/amqp-1.4.0 -DPHP_ATOM_INC -I/var/www/rabbitmq-c/amqp-1.4.0/include… >>> More
Does RabbitMq do round-robin from the exchange to the queues

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, I am currently evaluating message queue systems and RabbitMq seems like a good candidate, so I'm digging a little more into it. To give a little context I'm looking to have something like one exchange load balancing the message publishing to multiple queues. I don't want to replicate the messages… >>> More

Related posts about sharding

Custom Lucene Sharding with Hibernate Search

as seen on Stack Overflow - Search for 'Stack Overflow'
Has anyone experience with custom Lucene sharding / paritioning using Hibernate Search? The documentation of Hibernate Search says the following about Lucene Sharding : In some cases, it is necessary to split (shard) the indexing data of a given entity type into several Lucene indexes.… >>> More
PASS Conference 2011 Topic: Multitenant Design and Sharding with SQL Azure

as seen on Geeks with Blogs - Search for 'Geeks with Blogs'
I am really happy to announce that I have been accepted as a speaker at the 2011 PASS Conference in Seattle. The topic? It will be about SQL Azure scalability using shards, and the Data Federation feature of SQL Azure. I will also talk extensively about the community open-source sharding library Enzo… >>> More
SQL Sharding and SQL Azure…

as seen on Geeks with Blogs - Search for 'Geeks with Blogs'
Herve Roggero has just published a paper that outlines patterns for scaling using SQL Azure and the Blue Syntax (he and Scott Klein’s company) sharding api. You can find the paper at: http://www.bluesyntax.net/files/EnzoFramework.pdf Herve and Scott have also just released an Apress book Pro SQL… >>> More
Webscale is all about sharding and its coming to SQL Azure

as seen on SQL Blogcasts - Search for 'SQL Blogcasts'
There are many that joke about developers always talking about webscale and needing to shard to be able to scale. In reality many systems, if not most, don’t need to be able to scale to numerous nodes because todays processing is so powerful. However in the cloud world where you don’t have 1 big box… >>> More
High-concurrency counters without sharding

as seen on Stack Overflow - Search for 'Stack Overflow'
This question concerns two implementations of counters which are intended to scale without sharding (with a tradeoff that they might under-count in some situations): http://appengine-cookbook.appspot.com/recipe/high-concurrency-counters-without-sharding/ (the code in the comments) http://blog.notdot… >>> More