# Too many a\_inodes from Entrypoint

**URL:** <https://discourse.nameko.io/t/too-many-a-inodes-from-entrypoint/374>\
**Category:** Uncategorized\
**Created:** [November 15, 2018, 9:58pm UTC](https://discourse.nameko.io/t/too-many-a-inodes-from-entrypoint/374 "2018-11-15T21:58:07Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![afischbein](https://avatars.discourse-cdn.com/v4/letter/a/e47774/32.png) [@afischbein](https://discourse.nameko.io/u/afischbein)\
**Post date:** [November 15, 2018, 9:58pm UTC](https://discourse.nameko.io/t/too-many-a-inodes-from-entrypoint/374/1 "2018-11-15T21:58:07Z")

</div>

Short version - an Entrypoint I wrote is creating an a\_inode fd on each event trigger, eventually resulting in “OSError: [Errno 24] Too many open files”

My Entrypoint subscribes to data from an external vendor and provides a callback method. When new data arrives, this callback is triggered. Skipping the boring unpacking of the external vendor’s data, here is what i do:

```
    def handle_message(self, vendor_data):

        d = unpack_vendor_data(vendor_data)

        event = Event()
        handle_result = partial(self.handle_result, event)
        args = (d)
        kwargs = {}

        self.container.spawn_worker(self, args, kwargs, handle_result=handle_result)

        result = event.wait()
        return result

    def handle_result(self, event, worker_ctx, result, exc_info):
        event.send(result, exc_info)
        return result, exc_info

```

My service method is trivial:

```
class VendorHandler:

    name = "vendorhandler"
    dispatch = EventDispatcher()

    @vendor_event
    def publish_vendor_msg(self, msg):
        self.dispatch("my_topic", msg)

```

Monitoring shows each time an event is triggered, a new a\_inode item is created and left open:

```
lsof -p 5318

....
nameko 5318 afischbein 158u a_inode 0,11 0 8047 [eventpoll]
nameko 5318 afischbein 159u a_inode 0,11 0 8047 [eventpoll]
nameko 5318 afischbein 160u a_inode 0,11 0 8047 [eventpoll]

```

Eventually the file limit is reached and the process exits with:

```
Exception in event handler:
Traceback (most recent call last):
  File "/home/devadmin/.conda/envs/_bpipepub/lib/python3.6/site-packages/eventlet/hubs/ __init__.py", line 111, in get_hub
AttributeError: '_thread._local' object has no attribute 'hub'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/devadmin/.conda/envs/_bpipepub/lib/python3.6/site-packages/blpapi/session.py", line 100, in __dispatchEvent
  File "/home/devadmin/.conda/envs/_bpipepub/lib/python3.6/site-packages/bpipepub/entrypoint.py", line 275, in processEvent
  File "/home/devadmin/.conda/envs/_bpipepub/lib/python3.6/site-packages/bpipepub/entrypoint.py", line 219, in handle_message
  File "/home/devadmin/.conda/envs/_bpipepub/lib/python3.6/site-packages/nameko-2.11.0-py3.6.egg/nameko/containers.py", line 345, in spawn_worker
  File "/home/devadmin/.conda/envs/_bpipepub/lib/python3.6/site-packages/eventlet/greenpool.py", line 78, in spawn
  File "/home/devadmin/.conda/envs/_bpipepub/lib/python3.6/site-packages/eventlet/greenthread.py", line 50, in spawn
  File "/home/devadmin/.conda/envs/_bpipepub/lib/python3.6/site-packages/eventlet/hubs/ __init__.py", line 117, in get_hub
  File "/home/devadmin/.conda/envs/_bpipepub/lib/python3.6/site-packages/eventlet/hubs/epolls.py", line 38, in __init__
OSError: [Errno 24] Too many open files

```

My module environment:

```
amqp 2.3.2 py_1 conda-forge
asn1crypto 0.24.0 py36_0
backports 1.0 py36_1
backports.os 0.1.1 py36_0
blas 1.0 mkl
blpapi 3.9.2 py36_blpapicpp3.8.18.1_1 conda-forge
ca-certificates 2018.03.07 0
certifi 2018.10.15 py36_0
cffi 1.11.5 py36he75722e_1
chardet 3.0.4 py36_1
cryptography 2.3.1 py36h1ba5d50_2
eventlet 0.23.0 py36_1000 conda-forge
greenlet 0.4.15 py36h7b6447c_0
idna 2.7 py36_0
importlib_metadata 0.6 py36_0
intel-openmp 2019.0 118
kombu 4.2.1 py_0 conda-forge
libedit 3.1.20170329 h6b74fdf_2
libffi 3.2.1 hd88cf55_4
libgcc-ng 8.2.0 hdf63c60_1
libgfortran-ng 7.3.0 hdf63c60_0
libstdcxx-ng 8.2.0 hdf63c60_1
mkl 2019.0 118
mkl_fft 1.0.6 py36h7dd41cf_0
mkl_random 1.0.1 py36h4414c95_1
mock 2.0.0 py36_0
mysqlclient 1.3.12 ubuntu1604 http://uslinuxprod01/repo/python/main
nameko 2.11.0 py36_2 http://uslinuxprod01/repo/python/main
ncurses 6.1 hf484d3e_0
numpy 1.15.4 py36h1d66e8a_0
numpy-base 1.15.4 py36h81de0dd_0
openssl 1.1.1 h7b6447c_0
pandas 0.20.3 py36h6022372_2
path.py 11.5.0 py36_0
pbr 5.1.0 py36_0
pip 18.1 py36_0
pycparser 2.19 py36_0
pyopenssl 18.0.0 py36_0
pysocks 1.6.8 py36_0
python 3.6.7 h0371630_0
python-dateutil 2.7.5 py36_0
pytz 2018.7 py36_0
pyyaml 3.13 py36h14c3975_0
readline 7.0 h7b6447c_5
requests 2.20.0 py36_0
setuptools 40.5.0 py36_0
simplejson 3.16.0 py36h14c3975_0
six 1.11.0 py36_1
sqlalchemy 1.2.13 py36h7b6447c_0
sqlite 3.25.2 h7b6447c_0
tk 8.6.8 hbc83047_0
ujson 1.35 py36h14c3975_0
urllib3 1.23 py36_0
vine 1.1.4 py_1 conda-forge
werkzeug 0.14.1 py36_0
wheel 0.32.2 py36_0
wrapt 1.10.11 py36h14c3975_2
xz 5.2.4 h14c3975_4
yaml 0.1.7 had09818_2
zlib 1.2.11 ha838bed_2

```

Linux version:

```
~$ cat /proc/version
Linux version 4.4.0-96-generic (buildd@lgw01-10) (gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.4) ) #119-Ubuntu SMP Tue Sep 12 14:59:54 UTC 2017

```

Any support you can provide would be most appreciated!

---

<div class="post-metadata">

**Author:** ![mattbennett](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.nameko.io/mattbennett/32/13_2.png) [@mattbennett](https://discourse.nameko.io/u/mattbennett)\
**Post date:** [November 15, 2018, 10:41pm UTC](https://discourse.nameko.io/t/too-many-a-inodes-from-entrypoint/374/2 "2018-11-15T22:41:49Z")

</div>

I think it’s the rest of your Entrypoint that is the interesting bit. There’s nothing shown in your snippet here that relates to files… Where does the `vendor_data` passed to `handle_message` come from?

But also, how many concurrent workers are you running? When the service crashes, have previous workers definitely terminated or could they still be running?

---

<div class="post-metadata">

**Author:** ![afischbein](https://avatars.discourse-cdn.com/v4/letter/a/e47774/32.png) [@afischbein](https://discourse.nameko.io/u/afischbein)\
**Post date:** [November 19, 2018, 2:09pm UTC](https://discourse.nameko.io/t/too-many-a-inodes-from-entrypoint/374/3 "2018-11-19T14:09:04Z")

</div>

Hi Matt - thanks for the quick response.  
Short version - no problem in Nameko, looks like interaction between vendor library and Eventlet - creating new threads from within the vendor library is what appears to cause the file leak. Switching to synchronous/single-threaded mode in the vendor library resolved the issue, so feel free to delete this post if not relevant to Nameko users.

Thanks again.