How to check the memory size of a dictionary in Python

Hello, this is Ninomiya of LIFULL CO., LTD. Left behind last year's Advent Calendar is a regret, so I will post it though it is a small story.

During an implementation, I needed to get a rough estimate of the memory usage of a variable. As introduced in this article, sys.getsizeof can display the size of a variable in memory.

import sys

print(sys.getsizeof({"key": "a"}))
# => 248

#The same value appears even though the font size is obviously large
print(sys.getsizeof({"key": "a" * 10000}))
# => 248

Because the substance of a Python dictionary is "a hash table that stores a reference to an object", it doesn't take into account the memory size of the strings inside. I think this article will be helpful for this story.

(I could only find good articles in Japanese for Ruby. However, the story around here is common to both Ruby and Python.)

To consider the size of the objects inside, refer to the article linked here in the Official Document. Must be implemented.

For an example of recursively using getsizeof () to determine the size of a container and its contents, use Recursive sizeof recipe [ please refer to.

However, in my case, it is a dictionary containing only character strings (serializable to json), and I just wanted to know the approximate value, so [StackOverflow here]( Following the writing of usage-of-dictionary-in-python), I ended up with the result of json.dumps.

>>> first = 'abc'*1000
>>> second = 'def'*1000
>>> my_dictionary = {'first': first, 'second': second}
>>> getsizeof(first)
>>> getsizeof(second)
>>> getsizeof(my_dictionary)
>>> getsizeof(json.dumps(my_dictionary))
>>> size = getsizeof(my_dictionary)
>>> size += sum(map(getsizeof, my_dictionary.values())) + sum(map(getsizeof, my_dictionary.keys()))
>>> size

Reference article

Recommended Posts

How to check the memory size of a dictionary in Python
How to check the memory size of a variable in Python
I made a program to check the size of a file in Python
[python] How to check if the Key exists in the dictionary
How to check in Python if one of the elements of a list is in another list
How to determine the existence of a selenium element in Python
How to check if the contents of the dictionary are the same in Python by hash value
How to get the number of digits in Python
How to write a list / dictionary type of Python3
How to pass the execution result of a shell command in a list in Python
How to mention a user group in slack notification, how to check the id of the user group
How to get a list of files in the same directory with python
How to use the __call__ method in a Python class
How to develop in a virtual environment of Python [Memo]
How to get the last (last) value in a list in Python
How to get a list of built-in exceptions in python
How to identify the element with the smallest number of characters in a Python list?
Check the behavior of destructor in Python
How to check the version of Django
How to get a stacktrace in python
How to check opencv version in python
How to know the internal structure of an object in Python
[TensorFlow 2] How to check the contents of Tensor in graph mode
How to find the memory address of a Pandas dataframe value
How to get the vertex coordinates of a feature in ArcPy
Get the value of a specific key up to the specified index in the dictionary list in Python
How to quickly count the frequency of appearance of characters from a character string in Python?
How to pass the execution result of a shell command in a list in Python (non-blocking version)
How to calculate the volatility of a brand
How to use the C library in Python
How to clear tuples in a list (Python)
How to embed a variable in a python string
Get the caller of a function in Python
How to create a JSON file in Python
Make a copy of the list in Python
Summary of how to use MNIST in Python
How to notify a Discord channel in Python
How to get dictionary type elements of Python 2.7
How to get the files in the [Python] folder
Output in the form of a python array
[Python] How to draw a histogram in Matplotlib
[Python] How to specify the window display position and size of matplotlib
[Python] How to put any number of standard inputs in a list
Check the in-memory bytes of a floating point number float in Python
Test & Debug Tips: Create a file of the specified size in Python
How to format a list of dictionaries (or instances) well in Python
What is the fastest way to create a reverse dictionary in python?
How to sort by specifying a column in the Python Numpy array.
Check if the string is a number in python
Easy way to check the source of Python modules
How to retrieve the nth largest value in Python
How to convert / restore a string with [] in python
How to get the variable name itself in python
Create a dictionary in Python
[Circuit x Python] How to find the transfer function of a circuit using Lcapy
How to know the current directory in Python in Blender
[Python] How to expand variables in a character string
How to count the number of elements in Django and output to a template
[python] How to sort by the Nth Mth element of a multidimensional array
Get the size (number of elements) of UnionFind in Python
A memorandum of how to execute the! Sudo magic command in Jupyter Notebook