[PYTHON] Roughly estimate the total memory usage of an object

Environment: Python 3.5.0, 64bit

The memory usage of an object can be checked with sys.getsizeof, but it does not count up to the referenced object.

>>> import sys
>>> sys.getsizeof([1, 2 ,3])
88
>>> sys.getsizeof([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
88

If you want to know the approximate total memory usage including the elements in the container, it seems that you need to trace all the referenced objects and integrate the getsizeof value.

reference: http://code.activestate.com/recipes/577504/

total_size.py

import sys

from itertools import chain
from collections import deque


def total_size(obj, verbose=False):
    seen = set()

    def sizeof(o):
        if id(o) in seen:
            return 0
        seen.add(id(o))
        s = sys.getsizeof(o, default=0)
        if verbose:
            print(s, type(o), repr(o))
        if isinstance(o, (tuple, list, set, frozenset, deque)):
            s += sum(map(sizeof, iter(o)))
        elif isinstance(o, dict):
            s += sum(map(sizeof, chain.from_iterable(o.items())))
        elif "__dict__" in dir(o):  #There must be a better way
            s += sum(map(sizeof, chain.from_iterable(o.__dict__.items())))
        return s

    return sizeof(obj)

There seems to be another way to use pickle.