The pattern you’re describing is called Surveyor-Responder. It isn’t available out of the box, but could be implemented without too much effort.
Nameko RPC is an abstraction on top of normal AMQP publishing and consuming. You can reimplement Nameko RPC or Surveyor-Responder on top of the lower level
Consumer extensions in
nameko.messaging (or the utility
In AMQP, you publish events to an exchange with a routing key. The exchange duplicates that message into every queue bound to that exchange with a matching routing key. Consumers consume messages from queues. If multiple consumers are consuming from the same queue the messages are distributed between them.
Nameko RPC arranges the AMQP entities in the following way:
- There is a queue for RPC requests for each service (named simply
- The RPC request queue is bound to the RPC exchange (
nameko-rpc) with a wildcard routing key (
- The RPC entrypoint creates a consumer for the RPC request queue from every service instance. This means that RPC requests are distributed amongst all the connected services.
- The RPC client publishes requests to the RPC exchange with the routing key
<service-name>.<method-name>. It is delivered by the exchange into the matching service queue.
- The request message is consumed by one of the service instances, and the entrypoint uses the routing key to determine which method to execute.
- The RPC client also creates a consumer and a queue for to receiving the RPC responses. This queue is bound to the RPC exchange with a unique routing key, which is specified as the
reply-to header in the RPC requests.
- When the service method has executed and a response is available, the RPC entrypoint publishes it back to the RPC exchange with the unique
reply-to as its routing key.
- The RPC client consumes the response from the reply queue.
You can verify all this by creating a tiny service with an RPC client and entrypoint and poking around in the RabbitMQ management interface. Example service:
from nameko.rpc import rpc, RpcProxy
name = "service"
service_rpc = RpcProxy("service")
Nameko Events use a different arrangement of queues, exchanges and routing keys. This is briefly documented in nameko/events.py. Again, set up a small example and poke around in the RabbitMQ interface and the arrangement should be clear.
For your usecase, I would emit something like a
BROADCAST event for the request, and then set up a separate
Consumer to receive the replies.
If you study the RPC and Events implementations you should get the idea. It’s been refactored and clarified a lot in the v3.0.0-rc branch, so I’d recommend looking at that.
If you end up building a robust Surveyor-Responder implementation it would be a good candidate for a standalone library.