# Service talking to blocking process via stdin/out

**URL:** <https://discourse.nameko.io/t/service-talking-to-blocking-process-via-stdin-out/240>\
**Category:** googlegroup\
**Created:** [February 21, 2018, 2:14pm UTC](https://discourse.nameko.io/t/service-talking-to-blocking-process-via-stdin-out/240 "2018-02-21T14:14:57Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Chris\_Platts](https://avatars.discourse-cdn.com/v4/letter/c/ec9cab/32.png) [@Chris\_Platts](https://discourse.nameko.io/u/Chris_Platts)\
**Post date:** [February 21, 2018, 2:14pm UTC](https://discourse.nameko.io/t/service-talking-to-blocking-process-via-stdin-out/240/1 "2018-02-21T14:14:57Z")

</div>

Here's an interesting one...

One of my services will need to talk to a process which it launches. It  
sends commands via stdin and reads data back via stdout.

In the current implementation, it simply blocks reading lines from stdout.  
However, the commands can sometimes take many minutes to execute.

This service would be called via RPC by other services -- and the RPC  
should not return until the process returns complete data from stdout.

What would be the best way of doing this, whilst not interrupting nameko's  
heartbeats?

Thanks,  
Chris

---

<div class="post-metadata">

**Author:** ![Chris\_Platts](https://avatars.discourse-cdn.com/v4/letter/c/ec9cab/32.png) [@Chris\_Platts](https://discourse.nameko.io/u/Chris_Platts)\
**Post date:** [February 21, 2018, 3:38pm UTC](https://discourse.nameko.io/t/service-talking-to-blocking-process-via-stdin-out/240/2 "2018-02-21T15:38:37Z")

</div>

...thinking about it, the calls to the process which take a long time are known in advance, so...

Would this be fixed by also having a pub/sub arrangement for the long-running calls? The calling service would trigger the process command via a an RPC call\_async, and subscribe to an “I’m done!” event from the process service, along with some metadata to describe which call finished?

---

<div class="post-metadata">

**Author:** ![Chris\_Platts](https://avatars.discourse-cdn.com/v4/letter/c/ec9cab/32.png) [@Chris\_Platts](https://discourse.nameko.io/u/Chris_Platts)\
**Post date:** [February 21, 2018, 4:03pm UTC](https://discourse.nameko.io/t/service-talking-to-blocking-process-via-stdin-out/240/3 "2018-02-21T16:03:18Z")

</div>

...or would the blocking nature of the code still cause timeouts even when  
being called via rpc's call\_async? (sorry for the spam!)

---

<div class="post-metadata">

**Author:** ![mattbennett](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.nameko.io/mattbennett/32/13_2.png) [@mattbennett](https://discourse.nameko.io/u/mattbennett)\
**Post date:** [February 21, 2018, 10:53pm UTC](https://discourse.nameko.io/t/service-talking-to-blocking-process-via-stdin-out/240/4 "2018-02-21T22:53:19Z")

</div>

You can wait on the subprocess inside a greenthread. That won't block the  
heartbeats.

If you're happy to block a worker until your process exits, this solution  
would work fine. A nicer but more complex approach would be to detach the  
new process and provide a callback somehow. How about using the @timer  
entrypoint and subprocess.poll() to check the status of the process, and  
replying with an "I'm done" type message when it exits?

> **···**
>
> On Wednesday, February 21, 2018 at 4:03:18 PM UTC, Chris Platts wrote:
> 
> > ...or would the blocking nature of the code still cause timeouts even when  
> > being called via rpc's call\_async? (sorry for the spam!)

---

<div class="post-metadata">

**Author:** ![Chris\_Platts](https://avatars.discourse-cdn.com/v4/letter/c/ec9cab/32.png) [@Chris\_Platts](https://discourse.nameko.io/u/Chris_Platts)\
**Post date:** [February 22, 2018, 8:10pm UTC](https://discourse.nameko.io/t/service-talking-to-blocking-process-via-stdin-out/240/5 "2018-02-22T20:10:39Z")

</div>

Thanks, Matt -- calling the stdout-read loop via eventlet.tpool.execute()  
did the job.

Blocking the worker is actually the desired behaviour. This service is  
purely an interface to the process, and we only want $max\_workers of these  
processes running at once.

(as an aside, this is turning into one of those 'proof-of-concept' projects  
that's rapidly become 'just-build-the-thing'... but it is all coming  
together! Thanks again for your frequent help!)

Chris