Our great sponsors
-
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
What do you do when your Python program is using too much memory? How do you find the spots in your code with memory allocation, especially in large chunks? It turns out that there is not usually an easy answer to these question, but a number of tools exist that can help you figure out where your code is allocating memory. In this article, I’m going to focus on one of them, memory_profiler.
The memory_profiler tool is similar in spirit (and inspired by) the line_profiler tool , which I’ve written about as well. Whereas line_profiler tells you how much time is spent on each line, memory_profiler tells you how much memory is allocated (or freed) by each line. This allows you to see the real impact of each line of code and get a sense where memory usage. While the tool is quite helpful, there’s a few things to know about it to use it effectively. I’ll cover some details in this article.
It uses the psutil library (or can use tracemalloc or posix) to access process information in a cross platform way, so it works on Windows, Mac, and Linux.