Identifying a memory leak


#1

From @raybotha on Sun Apr 29 2018 17:30:51 GMT+0000 (UTC)

I’m using the nameko backdoor to try and find the cause of a memory leak, although my service is quite simple and only uses the nameko-sqlalchemy dependency, and rpc.

However the memory use balloons up to around 6GB after handling a few thousand requests, but running pympler does not yield any results:

>>> from pympler import muppy
>>> allob = muppy.get_objects()
>>> from pympler import summary
>>> sum = summary.summarize(allob)
>>> summary.print_(sum)
                                          types |   # objects |   total size
=============================================== | =========== | ============
                                    <class 'str |       43963 |      5.76 MB
                                   <class 'dict |        7918 |      3.52 MB
                                   <class 'code |       15101 |      2.08 MB
                                   <class 'type |        2055 |      2.07 MB
                                    <class 'set |        1299 |    502.16 KB
                                  <class 'tuple |        6963 |    472.64 KB
                                   <class 'list |        3412 |    368.48 KB
                                <class 'weakref |        3720 |    290.62 KB
                            <class 'array.array |           2 |    260.79 KB
  <class 'sqlalchemy.sql.visitors.VisitableType |         248 |    256.65 KB
                            function (__init__) |        1145 |    152.07 KB
                            <class 'abc.ABCMeta |         136 |    135.81 KB
                      <class 'getset_descriptor |        1856 |    130.50 KB
                                    <class 'int |        4373 |    122.17 KB
                     <class 'wrapper_descriptor |        1466 |    114.53 KB

Is there a part of the nameko service memory that pympler can’t access?

Copied from original issue: https://github.com/nameko/nameko/issues/538


#2

From @raybotha on Sun Apr 29 2018 17:32:01 GMT+0000 (UTC)

Versions:

nameko==2.8.4
nameko-sqlalchemy==1.1.0