Question about threading.RLock


#1

Hi,

I'm using a package that makes use of `threading.RLock` internally
(PyFilesystem2).

The lock is used to prevent multiple threads from accessing an opened
filesystem, which makes sense. But I don't want it to do that for some
specific operations (I'm already ensuring operations are atomic).

So I am wondering if/how Nameko reacts to or is affected by
`threading.RLock` locks. Because if it isn't (my hope!) then I can
parallelize my operations by leveraging Nameko, rather than hacking
PyFilesystem2 :slight_smile:

Thanks in advance for your help :slight_smile:

Geoff


#2

I'm pretty sure Eventlet will patch the lock, so it will apply inside
Nameko.

···

On Monday, April 2, 2018 at 5:18:28 PM UTC+1, juko...@gmail.com wrote:

Hi,

I'm using a package that makes use of `threading.RLock` internally
(PyFilesystem2).

The lock is used to prevent multiple threads from accessing an opened
filesystem, which makes sense. But I don't want it to do that for some
specific operations (I'm already ensuring operations are atomic).

So I am wondering if/how Nameko reacts to or is affected by
`threading.RLock` locks. Because if it isn't (my hope!) then I can
parallelize my operations by leveraging Nameko, rather than hacking
PyFilesystem2 :slight_smile:

Thanks in advance for your help :slight_smile:

Geoff


#3

Hi Matt,

Any thoughts about the level at which it might apply? i.e. I expect it to
apply within a worker context, but not between workers.

Geoff

···

On Tuesday, April 3, 2018 at 1:02:46 AM UTC-7, Matt Yule-Bennett wrote:

I'm pretty sure Eventlet will patch the lock, so it will apply inside
Nameko.

On Monday, April 2, 2018 at 5:18:28 PM UTC+1, juko...@gmail.com wrote:

Hi,

I'm using a package that makes use of `threading.RLock` internally
(PyFilesystem2).

The lock is used to prevent multiple threads from accessing an opened
filesystem, which makes sense. But I don't want it to do that for some
specific operations (I'm already ensuring operations are atomic).

So I am wondering if/how Nameko reacts to or is affected by
`threading.RLock` locks. Because if it isn't (my hope!) then I can
parallelize my operations by leveraging Nameko, rather than hacking
PyFilesystem2 :slight_smile:

Thanks in advance for your help :slight_smile:

Geoff


#4

I would certainly test this rather than making assumptions, but I imagine
it will behave in the way the docs describe, applied to the eventlet
threads.

Each worker is its own thread and should be able to use the "re-entrant"
properties of the lock (i.e. a single worker can acquire it multiple
times), but while locked no other thread, worker or otherwise, will be able
to acquire it.

···

On Tuesday, 3 April 2018 20:21:21 UTC+1, juko...@gmail.com wrote:

Hi Matt,

Any thoughts about the level at which it might apply? i.e. I expect it to
apply within a worker context, but not between workers.

Geoff

On Tuesday, April 3, 2018 at 1:02:46 AM UTC-7, Matt Yule-Bennett wrote:

I'm pretty sure Eventlet will patch the lock, so it will apply inside
Nameko.

On Monday, April 2, 2018 at 5:18:28 PM UTC+1, juko...@gmail.com wrote:

Hi,

I'm using a package that makes use of `threading.RLock` internally
(PyFilesystem2).

The lock is used to prevent multiple threads from accessing an opened
filesystem, which makes sense. But I don't want it to do that for some
specific operations (I'm already ensuring operations are atomic).

So I am wondering if/how Nameko reacts to or is affected by
`threading.RLock` locks. Because if it isn't (my hope!) then I can
parallelize my operations by leveraging Nameko, rather than hacking
PyFilesystem2 :slight_smile:

Thanks in advance for your help :slight_smile:

Geoff