Using x-queue-type quorum with RabbitMQ

1

Hello all Namekans™!

I’m trying to use Nameko in one of our production evironments and one of the most important things in our case is full HA (in context of RabbitMQ). RabbitMQ have a new way of mirroring queues, the quorum queues.
From documentation:

To declare a quorum queue set the x-queue-type queue argument to quorum (the default is classic). This argument must be provided by a client at queue declaration time; it cannot be set or changed using a policy. This is because policy definition or applicable policy can be changed dynamically but queue type cannot. It must be specified at the time of declaration.

Declaring a queue with an x-queue-type argument set to quorum will declare a quorum queue with up to five replicas (default replication factor), one per each cluster node.

Is it possible to implement this kind of queue with Nameko? I’m using Rabbit cluster in Kubernetes, so adding automation to expanding quorum for queues with rabbitmq-queues add_member is highly unwanted. Of course I can implement some watcher, event publisher via API and automate this, but client side declaration is much more sexy.

2

Hey @PigNatovsky,

To do this I think you’d have to extend the RPC extensions. The queues are established in the RpcConsumer:

and ReplyListener:

These classes are components of the RPC extensions. The @rpc entrypoint creates its RpcConsumer here:

And the client creates its ReplyListener here:

You’ll want to subclass these to have them use your modified versions. The standalone client in nameko.standalone.rpc will need similar modifications.

It’s probably not a huge amount of work. If you get it working it’d be cool to have quorum queue support in the main library :slight_smile:

Note that all the references above are to the 3.x pre-release version of Nameko, in which the AMQP extensions were pretty significantly refactored.

2 Likes