Microservice memory profiling

• cryptography

  5 5,914 9.6 Python

  cryptography is a package designed to expose cryptographic primitives and recipes to Python developers.

  

  first, I did see a correlation between an endpoint being heavily hit in a given time window, and an increase of memory usage that didn't went down afterwards. The endpoint didn't do much so I went through every instruction - is a global variable appended indefinitely ? Is a cache decorator growing without a limit set ? Do I use a 3rd party that has a known issue ? Turns out, it was using cryptography, so I looked up known issues. Saw an issue about a leak when using load_pem_x509_certificate https://github.com/pyca/cryptography/issues/4833 - which I used ! I could fortunately just upgrade the library

• memray

  7 11,478 8.8 Python

  The endgame Python memory profiler

  

  second time was nastier. I used https://github.com/bloomberg/memray to try to spot it - that's the tool you should try out. You load your service through memray, and it will get you some stats that you can export as a flamegraph. I can't really afford to make it run on production so I ran it in a docker image and repeatedly ran the scenario I thought was responsible. Didn't find anything. I know what I did wrong: I assumed one particular codepath was the problem. If would have find the issue if I had a really complete scenario that covers broadly every possible endpoint and condition. Can't blame memray, that tool is really promising.

• InfluxDB

  www.influxdata.com

  sponsored

  Collect and Analyze Billions of Data Points in Real Time. Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.

  

Suggest a related project