Hi all,
A few of us who are based in London recently got together in person. It was
fairly unstructured and we talked about lots of things, but I made notes of
the topics we discussed in depth.
In the interest of transparency and the wider community, here’s a summary
of our conversations:
* AMQP internals:
We talked generally about the AMQP spec (and RabbitMQ’s extensions) and the
limitations of the nameko implementation. It became clear that our current
implementation could be more sympathetic to how AMQP is designed:
The biggest problem the QueueConsumer, which is used to share a single
Consumer between all AMQP-based entrypoints. It would be better if each
entrypoint had its own Consumer, so that QoS and other settings can be
applied per-entrypoint or per-entrypoint-type. At the moment, for example,
it’s not possible to have a QoS of 10 for RPC methods, and a different QoS
for event handlers, without using two QueueConsumers (which is a clunky
workaround). This refactor could probably be made without breaking
backwards compatibility.
We also talked about possibly changing exchange names, routing keys and
queue names used by the current AMQP implementations. At present they are
fairly inconsistent, and some improvements could be made by updating them.
For example, using an RPC exchange per service would make detecting
“UnknownService” errors easier. Any changes to routing keys, queues or
exchanges would be backwards incompatible, though.
We also talked about replacing kombu with a more flexible and powerful
library. Kombu is a general-purpose messaging library that supports
RabbitMQ as one of many possible backends, none of which we make use of.
Kombu also doesn’t support some desirable AMQP features, such as connection
heartbeats. An alternative AMQP library, such as the official one (pika),
would give us more control.
* Config:
We decided to accept the recent community proposal of a yaml loader that
could also read environment variables
(https://github.com/onefinestay/nameko/issues/223). We’ll give the proposer
a few weeks to raise a PR and otherwise implement it ourselves (and give
attribution).
We decided to remove the ``—-broker`` option to nameko run and provide an
alternative way to specify configuration options at run time (without a
configuration file). We reaffirmed a previous suggestion that extensions
should declare their own configuration options, including registering an
option for the command-line runner if it’s useful to be able to provide it
there.
* Documentation:
We agreed the most important next step would be a reference application.
This should be something that people can clone, tweak and run as easily as
possible (e.g. include dockerfiles so no other dependencies are required).
It should make particular effort to show how to use the nameko test
utilities, because these are powerful but easiest to understand through
example.
A fledgling example has been started in
https://github.com/nameko/nameko-examples. It’s going an ordering system
for the fictional "Air Ship LTD”.
Services:
Orders:
- RPC CRUD for Orders/OrderDetails
- RDBMS Data Store
- Events: order_created
- External dependencies: nameko-sqlalchemy and alembic migrations
Products:
- RPC CRUD for Products
- Document Data Store with Redis to demonstrate creation of dependency.
- Events: handle "order_created" event and decrease stock
Gateway:
- HTTP API - Create Order, List Orders, Show order details
- Async RPC calls to get orders and product details
- Dependencies: config dependency to prepend domain name to returned
product images. Showing how to use access config in service code.
https://hub.docker.com/u/nameko/dashboard has been created for the
dockerfiles.
* Project website:
We spec’d out a draft for this. I need to tidy up the language slightly and
then will put it in a gist or somewhere for people to see. We’re also
searching for a designer to make it look pretty. It’ll be hosted at
http://nameko.github.io and replace the basic page there.
* PRs:
We went over all the outstanding (and old!) PRs and discussed next and/or
final steps. Some have been cleaned up already, others are waiting for an
agreed next step.
