File transfer under Nameko

Hi Guys,

So I now I have Nameko microservices running on a Raspberry Pi and it all works beautifully. One service controls and Arduino and the other controls a Nikon camera with gphoto2.

The camera produces image files and these are downloaded by one of the services onto the Raspberry pi using one of the gphoto2 options. But now the files are on the Raspberry Pi and the main calling programme is running on a desktop. Is it possible to use Nameko to transfer the files from the Raspberry Pi to the main machine,or do I need to use another protocol like ftp or ssh?

Thanks

Steve

Hi Steve,

You can use simple HTTP based file upload. Nameko HTTP entrypoint is build on top of werkzeug library. Request object has files property you can use to handle incoming file payload.

Cheers,
Jakub

Hi Jakub,

Thanks for your reply. Sadly, I don’t really understand it.

Could you post some example code, please?

Tx

Steve

Hi Steve,

No problem. Please find the example here https://gist.github.com/kooba/49d173e2aeaf93b262d254fb64e157aa

Excellent, thank you, I’ll look through it. Do you think it is faster than SCP, which is what I ended up using?

Regards,

Steve.

Hi Steve,

Not sure what the speed implications would be here, that would require some testing. If scp works for your case then there is nothing wrong with it. Going through Nameko service would only make sense if you need some additional operations to be performed or events triggered before or after that file upload on the receiver side.

Hi Jakub,

Thanks for your support. I’ll probably stick with SCP then, unless I find a reason to move.

Steve.

hello, do you have an example to download file using nameko services? Thank you.

@xolosno Simply use Python Requests library to download a file. There are tons of examples online how to do it. First result after google search is: https://www.codementor.io/aviaryan/downloading-files-from-urls-in-python-77q3bs0un

With nameko, we use the strategy to use external web service, in our cases it is either AWS S3 or alternative S3 implementation minio.

In general, I try to avoid sending larger amount of data via messaging, I prefer to store the file somewhere and notify others about it (via small event message). In case of AWS S3, one can configure S3 notification which sends notification to AWS SNS, from there to (arbitrary number of) SQS and from there, nameko SQS entrypoint consumes the events and fires events within it’s environment. It takes under a second from the moment a file is published to the moment nameko event about new object creation.

With minio, it may work very similarly, they also provide notification system and it uses (almost) the same messaging format of messages.