[LINUX] I tried to control the network bandwidth and delay with the tc command
■ Purpose
The tc command (Traffic Control) can control the Latency and Throughput of the Network. The delay and bandwidth may change depending on the distance of the computer, and the time for data transfer may change. So, let's check if the delay and bandwidth can be controlled with the tc command.
■ Composition
Install two Compute instances on Oracle Cloud and try to control the network performance between them with TC commands.
■ Check the current environment
Check Latency, Throughput, TPS to Client-> Server
● Delay (RTT) confirmation
Ping to check RTT This time, confirm that the average is 0.166ms
[opc@client-inst01 ~]$ sudo ping 10.0.0.2 -c 5
PING 10.0.0.2 (10.0.0.2) 56(84) bytes of data.
64 bytes from 10.0.0.2: icmp_seq=1 ttl=64 time=0.178 ms
64 bytes from 10.0.0.2: icmp_seq=2 ttl=64 time=0.182 ms
64 bytes from 10.0.0.2: icmp_seq=3 ttl=64 time=0.135 ms
64 bytes from 10.0.0.2: icmp_seq=4 ttl=64 time=0.177 ms
64 bytes from 10.0.0.2: icmp_seq=5 ttl=64 time=0.160 ms
--- 10.0.0.2 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4110ms
rtt min/avg/max/mdev = 0.135/0.166/0.182/0.020 ms
● Bandwidth check
Run iperf3 to check throughput This time, we confirmed that it was about 2 Gbits / sec.
[opc@client-inst01 ~]$ sudo iperf3 -c 10.0.0.2
Connecting to host 10.0.0.2, port 5201
[ 4] local 10.0.0.5 port 33750 connected to 10.0.0.2 port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 377 MBytes 3.17 Gbits/sec 1165 2.06 MBytes
[ 4] 1.00-2.00 sec 254 MBytes 2.13 Gbits/sec 2338 323 KBytes
[ 4] 2.00-3.00 sec 231 MBytes 1.94 Gbits/sec 1480 1.31 MBytes
[ 4] 3.00-4.00 sec 244 MBytes 2.04 Gbits/sec 2127 61.2 KBytes
[ 4] 4.00-5.00 sec 229 MBytes 1.92 Gbits/sec 953 166 KBytes
[ 4] 5.00-6.00 sec 239 MBytes 2.00 Gbits/sec 2055 629 KBytes
[ 4] 6.00-7.00 sec 205 MBytes 1.72 Gbits/sec 1336 1.77 MBytes
[ 4] 7.00-8.00 sec 274 MBytes 2.30 Gbits/sec 2216 96.1 KBytes
[ 4] 8.00-9.00 sec 238 MBytes 1.99 Gbits/sec 1198 891 KBytes
[ 4] 9.00-10.00 sec 232 MBytes 1.95 Gbits/sec 1486 638 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-10.00 sec 2.46 GBytes 2.12 Gbits/sec 16354 sender
[ 4] 0.00-10.00 sec 2.46 GBytes 2.11 Gbits/sec receiver
iperf Done.
■ Delay and bandwidth control with tc command
Since the packet is sent from the client side, try controlling the delay and bandwidth on the client computer.
① Check the Network Interface to be set
Network Interfece name confirmation to connect to Server with Client Confirm that the Interface name is ens3 this time
[root@client-inst01 ~]# ifconfig
ens3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 9000
inet 10.0.0.5 netmask 255.255.255.0 broadcast 10.0.0.255
ether 02:00:23:17:3a txqueuelen 1000 (Ethernet)
RX packets 454466 bytes 165785003 (158.1 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 503382 bytes 277089862 (264.2 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
(2) 100ms delay control tc command execution
This time, I will try to control it to 100ms 100ms is about the distance from Japan to the United States
[opc@client-inst01 ~]$ sudo tc qdisc add dev ens3 root handle 1:0 netem delay 100ms
③ 50Mbps bandwidth control tc command execution
This time, I will try to control it to 50 Mbps At that time, try using the following values
・ Rate = 50mbit ・ CONFIG_HZ = 1000 ・ Burst = rate / CONFIG_HZ = 50kb ・ Limit = burst x 10 = 500kb
- Check the CONFIG_HZ value
[opc@client-inst01 ~]$ sudo cat /boot/config-`uname -r` | grep CONFIG_HZ=
CONFIG_HZ=1000
・ 50Mbps bandwidth control tc command execution
[opc@client-inst01 ~]$ sudo tc qdisc add dev ens3 parent 1:1 handle 10:0 tbf rate 50mbit burst 50kb limit 500kb
④ Check tc command settings
[opc@client-inst01 ~]$ sudo tc -s qdisc show dev ens3
qdisc netem 1: root refcnt 17 limit 1000 delay 100.0ms
Sent 288635914 bytes 38521 pkt (dropped 0, overlimits 0 requeues 14)
backlog 102b 1p requeues 14
qdisc tbf 10: parent 1:1 rate 50Mbit burst 25Kb lat 36.9ms
Sent 35582 bytes 73 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
⑤ Delay (RTT) confirmation
Ping to check RTT Confirm that it can be controlled to 100 ms on average
[opc@client-inst01 ~]$ ping 10.0.0.2 -c 5
PING 10.0.0.101 (10.0.0.101) 56(84) bytes of data.
64 bytes from 10.0.0.101: icmp_seq=1 ttl=64 time=100 ms
64 bytes from 10.0.0.101: icmp_seq=2 ttl=64 time=100 ms
64 bytes from 10.0.0.101: icmp_seq=3 ttl=64 time=100 ms
64 bytes from 10.0.0.101: icmp_seq=4 ttl=64 time=100 ms
64 bytes from 10.0.0.101: icmp_seq=5 ttl=64 time=100 ms
--- 10.0.0.101 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4004ms
rtt min/avg/max/mdev = 100.160/100.192/100.218/0.283 ms
⑥ Bandwidth check
Run iperf3 to check RTT Confirm that control is possible to 50Mbits / sec
[opc@client-inst01 ~]$ sudo iperf3 -c 10.0.0.2
Connecting to host 10.0.0.2, port 5201
[ 4] local 10.0.0.5 port 44108 connected to 10.0.0.2 port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 6.18 MBytes 51.8 Mbits/sec 44 743 KBytes
[ 4] 1.00-2.00 sec 6.25 MBytes 52.4 Mbits/sec 5 638 KBytes
[ 4] 2.00-3.00 sec 5.00 MBytes 41.9 Mbits/sec 0 725 KBytes
[ 4] 3.00-4.00 sec 6.25 MBytes 52.4 Mbits/sec 0 751 KBytes
[ 4] 4.00-5.00 sec 6.25 MBytes 52.4 Mbits/sec 0 751 KBytes
[ 4] 5.00-6.00 sec 6.25 MBytes 52.4 Mbits/sec 0 778 KBytes
[ 4] 6.00-7.00 sec 5.00 MBytes 41.9 Mbits/sec 0 813 KBytes
[ 4] 7.00-8.00 sec 6.25 MBytes 52.4 Mbits/sec 0 848 KBytes
[ 4] 8.00-9.00 sec 6.25 MBytes 52.4 Mbits/sec 3 664 KBytes
[ 4] 9.00-10.00 sec 6.25 MBytes 52.4 Mbits/sec 0 786 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-10.00 sec 59.9 MBytes 50.3 Mbits/sec 52 sender
[ 4] 0.00-10.00 sec 56.9 MBytes 47.7 Mbits/sec receiver
iperf Done.
--- 10.0.0.2 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4004ms
rtt min/avg/max/mdev = 100.160/100.192/100.218/0.283 ms
⑦ Cancel tc command setting
[opc@tokyo-inst01 ~]$ sudo tc qdisc del dev ens3 root
Recommended Posts