Hey,
I’ve been looking to integrate a microservice that is capable of doing thousands of HTTP requests in a few minutes. When doing this with a monolithic application I am simply using asyncio with aiohttp, but with Nameko and its own asynchronous loop I am unsure if this would be feasible. Any tips would be helpful.
Thank you!
By “fast HTTP” I guess you mean highly concurrent HTTP, as in making or servicing lots of requests at the same time. Nameko can certainly do that.
Do you want the Nameko service to be the client or the server?
As a server – you will get good performance out of Nameko’s HTTP decorator as long as you increase the default worker pool size. Set the max_workers config key to something large.
As a client – Eventlet has excellent support for making concurrent web requests. See https://eventlet.net/doc/examples.html#web-crawler for a simple example. Again you’ll want to set the pool size to something suitable.
I don’t know about the performance characteristics of aiohttp vs Nameko. A bare aiohttp implementation is probably faster in terms of the CPU cycles it spends processing a single request, but perhaps not enough to matter, depending on your use case.
Awesome! Yes, I meant multiple concurrent http requests from the microservice (scraper)– I should’ve been more detailed. I will take a look into the Eventlets doc and come back here if I have any further questions. I appreciate the swift reply.