[PYTHON] Calculate the memory sharing rate of Linux processes

A program that calculates the memory sharing rate by Copy on Write of a Linux process.

With this entry

Seen in the above entry

Perl program using the Linux :: Smaps module introduced with reference to I wrote in Python. I think it is easy to use because it works without installing an external module. We have confirmed that it works with Python 2.6 series and above.

#!/usr/bin/env python
import sys

def read_smaps(pidlist):
        mem = lambda t, f: int(f[1]) if f[0] == '%s:' % t else 0.0
        for pid in pidlist:
            filename = "/proc/%s/smaps" % pid
            with open(filename) as f:
                rss = 0.0
                shared = 0.0
                for line in f:
                    fields = line.split()
                    rss += mem('Rss', fields)
                    shared += mem('Shared_Clean', fields)
                    shared += mem('Shared_Dirty', fields)
                print("%s\t%d\t%d (%.2f%%)\t%d" %
                      (pid, rss, shared, shared/rss*100, rss - shared))
    except IOError as e:
if __name__ == '__main__':
    if len(sys.argv) < 2:
        print("usage: %s [pids]" % __file__)

$ sudo ./read_smaps.py
usage: ./read_smap.py [pids]

##Specify the list of PIDs with the pgrep command and use the memory of the program.(Sharing rate)To calculate
$ sudo ./read_smaps.py `pgrep train_model.py`
23241   330396  202516 (61.29%) 127880
23246   414720  202476 (48.82%) 212244
23247   414836  202480 (48.81%) 212356
23248   414196  202440 (48.88%) 211756

RSS represents the memory usage of the process, SHARED represents the shared memory usage (sharing rate), and NONE_SHARED represents the unshared memory usage. The above example examines the memory sharing rate of a program that performs regression learning and cross-validation in a multi-process written in Python (using scikit-learn), but it can be said that it shares about 50% of the memory. You can read it.

It's a simple program that just reads the contents of the / proc / {PID} / smaps file, so it seems that there are many people writing in AWK on the Web. I often write my work, data science, and elemental technology in Python, so I wrote it in Python, which I'm used to. Recently, I've been secretly studying the Go language, and I've been writing abandoned code in Go even at work.

Recommended Posts

Calculate the memory sharing rate of Linux processes
Calculate the number of changes
Check the memory status of the server with the Linux free command
Check the memory protection of linux kerne with the code for ARM
[Understanding in 3 minutes] The beginning of Linux
100 Language Processing Knock-93 (using pandas): Calculate the accuracy rate of analogy tasks
A quick overview of the Linux kernel
[Java] [Linux] Investigating how the implementation of Java child processes on Linux is realized
[python] Checking the memory consumption of variables
Understand the "temporary" part of UNIX / Linux
[Linux] Learn the basics of shell commands
Reading comprehension of "The Go Memory Model"
The origin of Manjaro Linux is "Mount Kilimanjaro"
Calculate the total number of combinations with python
[2020July] Check the UDID of the iPad on Linux
Calculate the probability of outliers on a boxplot
Linux (command memory)
See here for the amount of free memory of the free command
Calculate volume from the two-dimensional structure of a compound
The story of replacing Nvidia GTX 1650 with Linux Mint 20.1.
The story of building the fastest Linux environment in the world
Open Chrome version of LINE from the command line [Linux]
[Python] Calculate the average value of the pixel value RGB of the object
The story of sharing the pyenv environment with multiple users
[C language] [Linux] Get the value of environment variable
Calculate the regression coefficient of simple regression analysis with python
Steps to calculate the likelihood of a normal distribution
Roughly estimate the total memory usage of an object
Check the type and version of your Linux distribution
Calculate the product of matrices with a character expression?
Understand the attributes of Linux files (ls -l command)
Announcing the availability of Java 11 LTS on Amazon Linux 2