Sharing Methods

Github Issues
1

From @geoffjukes on Wed Aug 23 2017 21:40:05 GMT+0000 (UTC)

Hi,

I feel like all I do is ask questions - Sorry! I do try to read the docs, but I come from an Ops background, so some obvious things are a mystery to me - so I figure I should just ask :slight_smile:

Is it considered bad practice to call a method defined in an RPC service, directly?
How ‘normal’ is a class that defines an RPC task?

For example, what if all the RPC methods have the same args - and they all do the same things, like split out a filename, an extension, etc? Normally, I’d put that in an __init__.

A contrived example, would be a File Information method that can called via RPC when the file exists in shared storage, but is called ‘directly’ for a file in local storage. Each of the File information calls (such as MD5) can also be called via RPC (when the caller requires only that piece of information)

import os
import hashlib
import tempfile
import shutil
from nameko import rpc

class FileInfoService:
    name = 'file_info'

    @rpc
    def summary(self, filepath):
        return  {'filepath': filepath,
                'filename': os.path.basename(filepath),
                'location': os.path.dirname(filepath),
                'size': self.size(filepath),
                'md5': self.md5(filepath)}

    @rpc
    def size(self, filepath):
        return os.path.getsize(filepath)

    @rpc
    def md5(self, filepath):
        # Calculates the MD5 of a file at `filepath`
        f = open(filepath, 'rb')
        m = hashlib.md5()
        while True:
            data = f.read(10240)
            if len(data) == 0:
                break
            m.update(data)
        return m.hexdigest()


class FileProcess:
    name = 'file_process'

    def __init__(self, filepath):
        self.filepath = filepath
        self.filename = os.path.basename(filepath)


    @rpc
    def process(self, filepath):
        with tempfile.TemporaryDirectory(dir='/dev/shm') as tmpdir:
            tmp = os.path.join(tmpdir, self.filename)
            shutil.copy(self.filepath, tmp)
            return self.do_stuff()

    def do_stuff(self):
        # Do stuff to the file that changes it
        # The file only exists in local Temp storage, so I can't make an RPC call for the MD5
        return FileInfoService().summary(self.filepath)

Many thanks,

Geoff

Copied from original issue: https://github.com/nameko/nameko/issues/467

2

From @mattbennett on Fri Aug 25 2017 15:40:23 GMT+0000 (UTC)

Hi Geoff,

Asking questions is good :slight_smile: Although we do have a mailing list that is more appropriate for these kind of things than a Github issue.

I will try to answer these questions here though.

Is it considered bad practice to call a method defined in an RPC service, directly?
How ‘normal’ is a class that defines an RPC task?

One of Nameko’s principles is that entrypoint decorators do not mutate the underlying methods, so it’s not bad practice at all. When one service method calls another, there’s literally no difference between the target method being entrypoint-decorated or not.

In many ways a service that defines an RPC task is just a normal class with one of the methods decorated with @rpc. In some ways it’s not though :wink: One of the biggest is that you don’t control construction, so in your example you’d have no opportunity to pass filepath parameter to FileProcess.

Another is that workers are stateless. Every worker gets a new instance of the service class, so you can’t write something to self in one worker and expect it to be visible to another worker. This is best illustrated with an example (of what not to do):

# this won't work

class Service:
    name = 'serv'
    
    state = "empty"

    @rpc
    def intro(self, name):
         self.state = name

    @rpc
    def greet(self):
        return "Hello {}".format(self.state)

$ nameko shell
>>> n.rpc.serv.intro("geoff")
>>> n.rpc.serv.greet()
.... "Hello empty"

If you need to store state between workers, you should use a DependencyProvider.

I hope that helps!