Async IO Support

Async IO Support

I’m no fan of Async IO, but it’s increasingly becoming unavoidable day to day. However I am a fan of Nameko, and I think it provides great abstractions to various RPC interfaces.

I know this has obviously been floated in the past, and people have been a little unsure how to integrate Async IO without fundamentally changing the design of Nameko. I’ve seen some other libraries that for a long time have been ‘sync only’ like SQLAlchemy use some Greenlet magic to integrate with the Async IO event loop.

Here’s a little example that was used as the basis for integrating Async IO into SQLAlchemy.

1 Like

If you’re interested in this, I’d be happy to put the time in to build a POC like this:

from nameko.rpc import rpc

class GreetingService:
    name = "greeting_service"

    @rpc
    async def hello(self, name):
        return "Hello, {}!".format(name)

Hi @sonthonaxrk,

This is cool, thanks for writing. The Python concurrency landscape has changed a lot since Nameko was conceived, and, frankly, being based on Eventlet is problematic. I would love to see more compatibility with the new status quo.

I’m hopeful that some concurrency primitives could be packaged up in a way that supports multiple frameworks, like Eventlet/Gevent, Asyncio or even plain threads.

Nameko is reasonably clear about its concurrency requirements:

  • Workers are single-threaded and should not need to consider concurrency
  • Entrypoints should expect to spawn multiple worker threads concurrently
  • Each worker thread will call get_dependency on each DependencyProvider

I’d love to see a POC.