Linux Policy Base Routing
When building EC2 on AWS, packets may not reach EC2 with multiple NICs in the following situations, so this is a memorandum. As a configuration, the EC2 instance has multiple NICs, each belonging to a different subnet.
(Quoted from the figure) http://blog.serverworks.co.jp/tech/2018/02/16/rhel-pbr-setting/
The routing table (main) has the following situation. The default GW is a router on the subnet (10.0.1.0/24).
[[email protected] etc]# ip route show table main 10.0.1.0/24 dev eth0 proto kernel scope link src 10.0.1.5 10.0.2.0/24 dev eth1 proto kernel scope link src 10.0.2.5 default via 10.0.1.1 dev eth0
In this situation, I tried to ping the 10.0.0.2.x NIC (eth1) from the NW belonging to 10.0.1.0/24, but there is no return packet. As a result of investigating why, PING has arrived at the EC2, but when returning the return packet, it is sending a packet from the default GW of the EC2, and as a result it does not reach the source. It was a thing.
For the same communication, communication in which the receiving NIC and the returning NIC are different is called asymmetric routing. In Linux, from some version, the following kernel parameters are set to drop packets if the NIC that communicates back is different from the received NIC. This setting is enabled at the packet destination, and only the return communication is filtered.
Return route filter
net.ipv4.conf.all.rp_filter = 1 net.ipv4.conf.default.rp_filter = 1
There is a way to disable the above, but it is a security setting for the entire OS to prevent IP spoofing. In order to avoid this, there is also a function called Policy Base Routing, so implement it.
Policy Base Routing
The standard routing table routes packets to a specific router or default gateway by looking at the destination address. On the other hand, Policy Base Routing (PBR) is a function that determines the routing by looking at the source IP address and TCP / UDP port. Also, prioritize PBR as a rule over normal routing.
Therefore, in this case, it can be concluded that the PBR should be set so that packets with a source IP address of 10.0.2.0/24 as a reference will be skipped to 10.0.2.1. (Although it is the source IP address mentioned here, it refers to the part of the return packet of PING. As you can see by tcpdump, the return packet is 10.0.2.x ==> 10.0.1.x.)
--Since this is the current rule, set the priority to a smaller value than "32766: from all lookup main". The lower the priority, the higher the priority rule
[[email protected] etc]# ip rule show 0: from all lookup local 32766: from all lookup main 32767: from all lookup default
--Create route table rules for PBR
ip route flush table 2000 ip route add table 2000 to 10.0.2.0/24 dev eth1 ip route add table 2000 to default via 10.0.2.1 dev eth1 ip rule add from 10.0.2.0/24 table 2000 priority 200
cat << EOF > /etc/sysconfig/network-scripts/pbr02-eth1 from 10.0.2.0/24 table 2000 priority 200 EOF cat << EOF > /etc/sysconfig/network-scripts/route-eth1 10.0.2.0/24 dev eth1 table pbr02 default via 10.0.2.1 dev eth1 table 2000 EOF
--Setting confirmation system
[[email protected] network-scripts]# ip rule show 0: from all lookup local 200: from 10.0.2.0/24 lookup pbr02 32766: from all lookup main 32767: from all lookup default [[email protected] network-scripts]# ip route show table 2000 10.0.2.0/24 dev eth1 scope link default via 10.0.2.1 dev eth1