# Orchestrating a set of services

**URL:** <https://discourse.nameko.io/t/orchestrating-a-set-of-services/236>\
**Category:** googlegroup\
**Created:** [February 6, 2018, 9:59am UTC](https://discourse.nameko.io/t/orchestrating-a-set-of-services/236 "2018-02-06T09:59:31Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Chris\_Platts](https://avatars.discourse-cdn.com/v4/letter/c/ec9cab/32.png) [@Chris\_Platts](https://discourse.nameko.io/u/Chris_Platts)\
**Post date:** [February 6, 2018, 9:59am UTC](https://discourse.nameko.io/t/orchestrating-a-set-of-services/236/1 "2018-02-06T09:59:31Z")

</div>

Hi all,

Having attained the status of 'reasonably competent, unlikely to blow  
things up' with nameko, I'm thinking about how to reimplement some of my  
company's systems in a nice microservice-y way.

We have a bunch of actions, file transfers, zip/unzip operations, data  
imports, data processing, etc. These actions are chained together in  
specific ways for each of our customers.

Now, if these operations were implemented as nameko services, I'd need some  
way of storing the 'plan' for each of our customers and executing that plan  
as needed.

What I'm thinking of is this:

A \*Control\* service which owns a database containing the client-specific  
plan. The service is told to execute the plan for client 'A'. It consults  
the plan and triggers the first service action in the plan. The service  
(and all others) will end with a call back to the Control service to  
indicate success/failure/etc. The Control service then kicks off the next  
service in the plan. Lather, rinse, repeat until the plan is complete.

This appears to be a decent enough solution. It would give the ability to  
resume a plan from a failed step, plus there could be methods in the  
Control service to do things like finding out how far a plan has progressed.

So, a) does this sound sane?, and b) has anyone done something similar with  
success?

Cheers,  
Chris

---

<div class="post-metadata">

**Author:** ![mattbennett](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.nameko.io/mattbennett/32/13_2.png) [@mattbennett](https://discourse.nameko.io/u/mattbennett)\
**Post date:** [February 12, 2018, 9:48am UTC](https://discourse.nameko.io/t/orchestrating-a-set-of-services/236/2 "2018-02-12T09:48:40Z")

</div>

Hi Chris,

Sorry for not replying to this sooner.

Quick answers:

a) Yes  
b) Kind of. We distribute work but don't have a workflow that waits for  
results before moving on to the next step.

My recommendation would be to use asynchronous messaging for the  
communication between the Controller and the downstream services, i.e. have  
the downstream services call back when they're done, rather than using RPC  
and waiting for a reply in the controller.

Our task distribution is built using the nameko.messaging.Consumer and  
Publisher, and we used custom Nameko extensions to wrap them and add  
syntactic sugar. The API looks like:

class Worker:  
&nbsp;&nbsp;&nbsp;&nbsp;name = "worker"

&nbsp;&nbsp;&nbsp;&nbsp;@task  
&nbsp;&nbsp;&nbsp;&nbsp;def some\_task(self, \*args):  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;return "result"

class Controller:  
&nbsp;&nbsp;&nbsp;&nbsp;name = "controller"  
&nbsp;&nbsp;&nbsp;&nbsp;  
&nbsp;&nbsp;&nbsp;&nbsp;schedule\_task = TaskScheduler()

&nbsp;&nbsp;&nbsp;&nbsp;@entrypoint  
&nbsp;&nbsp;&nbsp;&nbsp;def initiate\_work(self):  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;self.schedule\_task("some\_task", "arg1", "arg2")

You could expand this concept and have the task results routed back to a  
different entrypoint on the controller:

class Worker:  
&nbsp;&nbsp;&nbsp;&nbsp;name = "worker"

&nbsp;&nbsp;&nbsp;&nbsp;@task  
&nbsp;&nbsp;&nbsp;&nbsp;def some\_task(self, \*args):  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;return "result"

class Controller:  
&nbsp;&nbsp;&nbsp;&nbsp;name = "controller"  
&nbsp;&nbsp;&nbsp;&nbsp;  
&nbsp;&nbsp;&nbsp;&nbsp;schedule\_task = TaskScheduler()

&nbsp;&nbsp;&nbsp;&nbsp;@entrypoint  
&nbsp;&nbsp;&nbsp;&nbsp;def initiate\_work(self):  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;task\_id = self.schedule\_task("some\_task", "arg1", "arg2")  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;...

&nbsp;&nbsp;&nbsp;&nbsp;@task\_result  
&nbsp;&nbsp;&nbsp;&nbsp;def handle\_result(self, task\_id, result):  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;...

> **···**
>
> On Tuesday, February 6, 2018 at 9:59:31 AM UTC, Chris Platts wrote:
> 
> > Hi all,
> > 
> > Having attained the status of 'reasonably competent, unlikely to blow  
> > things up' with nameko, I'm thinking about how to reimplement some of my  
> > company's systems in a nice microservice-y way.
> > 
> > We have a bunch of actions, file transfers, zip/unzip operations, data  
> > imports, data processing, etc. These actions are chained together in  
> > specific ways for each of our customers.
> > 
> > Now, if these operations were implemented as nameko services, I'd need  
> > some way of storing the 'plan' for each of our customers and executing that  
> > plan as needed.
> > 
> > What I'm thinking of is this:
> > 
> > A \*Control\* service which owns a database containing the client-specific  
> > plan. The service is told to execute the plan for client 'A'. It consults  
> > the plan and triggers the first service action in the plan. The service  
> > (and all others) will end with a call back to the Control service to  
> > indicate success/failure/etc. The Control service then kicks off the next  
> > service in the plan. Lather, rinse, repeat until the plan is complete.
> > 
> > This appears to be a decent enough solution. It would give the ability to  
> > resume a plan from a failed step, plus there could be methods in the  
> > Control service to do things like finding out how far a plan has progressed.
> > 
> > So, a) does this sound sane?, and b) has anyone done something similar  
> > with success?
> > 
> > Cheers,  
> > Chris

---

<div class="post-metadata">

**Author:** ![Chris\_Platts](https://avatars.discourse-cdn.com/v4/letter/c/ec9cab/32.png) [@Chris\_Platts](https://discourse.nameko.io/u/Chris_Platts)\
**Post date:** [February 13, 2018, 1:35pm UTC](https://discourse.nameko.io/t/orchestrating-a-set-of-services/236/3 "2018-02-13T13:35:52Z")

</div>

Thanks Matt -- that all makes sense, particularly using events rather than  
RPC!