Service that dynamically spawns and kills managed threads?

1

Any suggestions on creating a nameko service that allows for an RPC call to dynamically spawn, and then later, kill, a managed thread?

So far, I’ve been able to spawn managed threads, but I’m looking for a way to stop them.

2

Managed threads will run until they exit; there is no particular API to stop them again, but it can be done…

The easiest way would probably be to communicate to the thread that it should terminate. The nameko-grpc library does something like this for the threads it uses to manage each connection. See nameko-grpc/connection.py at master · nameko/nameko-grpc · GitHub

3

Using an event makes a lot of sense. What I ended up implementing is a mapping (dict) in the Dependency Injector that gets handed off to each Worker that spawns the new thread (self.container.spawn_managed_thread), where each thread is keyed by a unique identifier associated external to the Service. When it comes time to stop it, the thread is keyed again in the Worker and the .kill() method is invoked. I will look at converting this to an event-driven model. Do you see any benefits either way? Thanks!

4

Your approach seems fine, but advantages of using events to do the fan-out are:

  1. You’re using Nameko’s high-level API rather than using pseudo-internals (.kill() on the managed thread is an eventlet API)
  2. If you have multiple service instances, events will allow them all to participate, whereas threads are constrained to just the instance that spawns them.