Debugging nameko services with Visual Studio Code

googlegroup
#1

Hi,

After lots of manual 'nameko run' runs, a lot of Ctrl+C'ing and an
unpleasant amount of debug-by-print()'ing, I'm trying to get a decent
configuration sorted in my IDE (Visual Studio Code, with Python extension
installed.)

In case you've not tried vscode (it's quite good!), it allows configuration
of launch profiles. For Python code, this wants:

a) The path to python.exe
b) The 'program' -- i.e., what's passed to python.exe on the command-line
c) Program arguments
d) Other misc. optional stuff (e.g. a launchTask to execute before
debugging starts, arguments for the program, etc.)

...plus, it can do stuff like attach to a running Python

I'm trying to simulate `nameko run --config=config.yml service`, but with
vscode attached to the process to hit breakpoints, allow watches, etc.

Has anyone had any luck with this (or indeed, with a different IDE? The
principles would be the same...)

I tried using vscode's ability to attach to processes, but the debugger
crashes when connecting to nameko's --backdoor-port.

Or am I over-thinking this, and it's best just to add a __main()__ to the
service's module and spin up a container/runner/etc. from there?

#2

Your launch.json should just have "program" set to the path to nameko
(`which nameko`), otherwise that should just work? Then you can hit debug
in vscode.

So something like an entry in vscode's launch.json:

{
  (other standard python configs)
  "program": "/path/to/nameko",
  "args": [
     "run",
     "<blargh>"
   ]
}

···

On Tuesday, February 13, 2018 at 2:06:44 PM UTC, Chris Platts wrote:

Hi,

After lots of manual 'nameko run' runs, a lot of Ctrl+C'ing and an
unpleasant amount of debug-by-print()'ing, I'm trying to get a decent
configuration sorted in my IDE (Visual Studio Code, with Python extension
installed.)

In case you've not tried vscode (it's quite good!), it allows
configuration of launch profiles. For Python code, this wants:

a) The path to python.exe
b) The 'program' -- i.e., what's passed to python.exe on the command-line
c) Program arguments
d) Other misc. optional stuff (e.g. a launchTask to execute before
debugging starts, arguments for the program, etc.)

...plus, it can do stuff like attach to a running Python

I'm trying to simulate `nameko run --config=config.yml service`, but with
vscode attached to the process to hit breakpoints, allow watches, etc.

Has anyone had any luck with this (or indeed, with a different IDE? The
principles would be the same...)

I tried using vscode's ability to attach to processes, but the debugger
crashes when connecting to nameko's --backdoor-port.

Or am I over-thinking this, and it's best just to add a __main()__ to the
service's module and spin up a container/runner/etc. from there?

#3

Thanks, Tuan -- I'll give that another try!

I was pretty sure I'd tried the above, but hit a problem with vscode
complaining about something being binary when it wasn't expected. Perhaps
this is to do with debugging on Windows, where the 'nameko` command is a
binary executable rather than a script shebang'd to point to python.exe.

(I'm trying to hit that perfect vscode configuration where I can hit F5 on
a service.py and it'll run under the nameko shell on Windows, Linux and
macOS).

#4

Chris, do you also run Nameko on Windows? I've never actually tried it
before... does it work?

···

On Sunday, February 18, 2018 at 6:07:33 PM UTC, Chris Platts wrote:

Thanks, Tuan -- I'll give that another try!

I was pretty sure I'd tried the above, but hit a problem with vscode
complaining about something being binary when it wasn't expected. Perhaps
this is to do with debugging on Windows, where the 'nameko` command is a
binary executable rather than a script shebang'd to point to python.exe.

(I'm trying to hit that perfect vscode configuration where I can hit F5 on
a service.py and it'll run under the nameko shell on Windows, Linux and
macOS).

#5

Hi Matt,

Nameko works perfectly everywhere I’ve run it!

Our deployments are to Windows Server 2012, but my dev work is done in Docker on my Windows work desktop (makes it easier to get a Rabbit/Graylog/Elasticsearch/Mongo stack set up).

At home, I pull our git repo to my macOS machine for dev work. So I use Nameko on all three major platforms without issue.

(I do have to do a lot of venv fiddling due to moving between platforms, but at least my requirements.txt file is always up-to-date!)

#6

Quick update -- it does seem that vscode's struggling with nameko.exe being
binary on Windows:

The relevant excerpt from my launch.json reads:

"pythonPath": "${workspaceFolder}\\env\\Scripts\\python.exe",
"program": "${workspaceFolder}\\env\\Scripts\\nameko.exe",
    "args": [
        "run",
        "service",
        "--config",
        "config.yml"
    ],
"cwd": "${workspaceFolder}"

I'm going to ssh into my home machine and see what the nameko script
contains on a Unix-y system and see if I can replicate that in the
launch.json config.

#7

Chris,

I'm curious how you 'run' the services on WIndows.

···

On Sunday, February 18, 2018 at 2:15:19 PM UTC-8, Chris Platts wrote:

Hi Matt,

Nameko works perfectly everywhere I’ve run it!

Our deployments are to Windows Server 2012, but my dev work is done in
Docker on my Windows work desktop (makes it easier to get a
Rabbit/Graylog/Elasticsearch/Mongo stack set up).

At home, I pull our git repo to my macOS machine for dev work. So I use
Nameko on all three major platforms without issue.

(I do have to do a lot of venv fiddling due to moving between platforms,
but at least my requirements.txt file is always up-to-date!)

#8

Ok, so I ended up creating a 'debug.py' file in the directory containing my
service projects, containing:

import sys
from nameko.cli.main import main
main()

Then, in the launch.json for each of my projects, I have:

"pythonPath": "${workspaceFolder}\\env\\Scripts\\python.exe",
"program": "${workspaceFolder}\\..\\debug.py",
"args": [
    "run",
    "service",
    "--config",
    "config.yml"
],
"cwd": "${workspaceFolder}"

This seems to work fine -- environment-variable substitution when
processing config.yml works as expected.

#9

Hi Geoff,

We've not quite settled on a definitive method.

I'm running Docker on my development machine, so I'm just generating docker
images for my own machine.

On our test app server, I'm experimenting with Docker, but that might not
go anywhere. Our servers are Windows Server 2012, so do not have native
container technology. So Docker on there runs a thin Linux VM to hold the
containers. The machine itself is a VM, so we're entering nested
virtualisation territory and aren't sure if we want to go down that path.

So, without Docker, we've got a couple of methods, but are yet to decide
which to go with.

The first, more straightforward option is simply to install Python on the
target servers. We would then deploy the service code, create a virtualenv,
run *pip install -r requirements.txt* to install nameko and other
libraries, then create a Windows Service using *NSSM*[1] that calls *nameko.exe
run service*

The other is to use *pyinstaller*[2] to package everything up as single
.exe file which would also be executed via *NSSM* as a Windows service.
This is made slightly tricky by the fact that the *nameko* command is an
exe rather than a shebang'd python script on Unix. I think the answer
there is to make a 'start.py' module that executes the bundled nameko.exe
with the proper arguments. That 'start.py' would then be used as
pyinstaller's entrypoint (so would be executed when the package exe is run).

Ideally, we'd go the Docker route. Failing that, our Ops guys would
probably prefer the pyinstaller method, being more self-contained. I'll
let you know how it goes :slight_smile:

[1] NSSM - the Non-Sucking Service Manager: https://nssm.cc
[2] PyInstaller - http://www.pyinstaller.org

Cheers,
Chris

#10

Love Nameko alot. Used it in my previous company to create 4 setups of microservices for orchestration ML training and inference

Join a new company and I am advocating this as well. As part of training and onboarding new engineers to this light weight framework, I have ‘refactor’ nameko-example so that it support Dev Environment using VSCode. It supports local development with ability to easy debug, run unit test and do performance test. Docker are used only for backing service for Rabbit, Reddis, Postgres. The rest of the services are run locally for easy debugging and development which I think the the example is lacking or not easily accessible for starter.

Hope this is useful and not sure if the original engineer will let me merge back to master :wink:

1 Like