TL;DR Even when dealing with various layers such as OS and middleware, the read / write speed of the disk always exists as a factor that can be a bottleneck. Introducing a simple method for measuring disk throughput when you suspect that a disk is the bottleneck.
--Disk throughput can be easily measured by using the dd command and pv command.
As an example, I think it is useful to investigate by the method introduced when the compression process is slow for some reason when performing the file compression process.
You can measure Read throughput by using the dd command to set the disk to be measured to ʻif and specifying / dev / null for ʻof. By the way, note that it is a dd command, so it is important to be aware that it will be a sequential access.
--Example of measuring Read throughput of / dev / sda1
dd if=/dev/sda1 of=/dev/null bs=16k status=progress
The output will be as follows.
584122368 bytes (584 MB) copied, 4.61756 s, 127 MB/s
The method introduced above results in near pure disc speed. However, as in the usage scene example, for example, when the compression process is slow, you may want to distinguish whether it is pure disk speed or the compression process is the bottleneck and slow. In such a case, the pv command, which can measure the processing speed of each command, is effective.
The pv command is a command that measures the amount of data that crosses a pipe when performing pipe processing. For example, if you have the following example, you can check the amount of data passed from the tar command to the lz4 command, that is, how fast the tar command is being processed. By doing this, you can find out whether the tar command is the bottleneck or the lz4 command is the bottleneck when the execution speed of the following command is slow for some reason.
tar -b 256 -cvf - /data/ | lz4 -c
It is possible to measure by inserting the pv command between the pipes you want to measure as shown below.
--Example of measuring the speed of the tar command
tar -b 256 -cvf - /data/ | pv | lz4 -c > /dev/null
--Example of measuring the speed of the lz4 command
tar -b 256 -cvf - /data/mysql/ | lz4 -c | pv > /dev/null
The output will be as follows.
8.66GiB 0:00:04 [2.19GiB/s] [ <=> ]
I think there are many ways to measure the performance of a disk, but this time I introduced the above method as an easy way to check. Since there are various types of disk access such as Read`` Write, Random IO Sequential IO, and speed index ʻIOPS`` Throughput, "What is this time?" If you are conscious of "Is it a process that causes an access pattern?", I think that it will be easier to find a bottleneck.
Recommended Posts