[PYTHON] Created a tool to output a sequence diagram from a packet capture file of multiple nodes

Created a tool to output a sequence diagram from a packet capture file of multiple nodes

1. Overview

I created a tool called packetseq that outputs a packet sequence diagram in PNG format from a file that was simultaneously captured by Wireshark or tcpdump on multiple nodes on the network. For the language, I used Python and Seqdiag, a library that creates sequence diagrams.

• The code has been uploaded on GitHub (2016/10/6)
• Published on PyPI so that it can be installed and command executed with pip etc. (2016/10/10)

1-1. Background of tool creation

In order to investigate the network delay of the customer environment at the company, the packets collected at each communication node were matched. Since the packet is in pcap format, I browsed it with wireshark, and at first I opened multiple Wiresharks and matched them with my eyes, but it was impossible. Wireshark has a flow diagram, a tool that displays a specific connection, but I gave up because it cannot be used to match packets collected at multiple locations. Since there is no help for it, I decided to create a packet sequence on Excel. I narrowed down the capture file to a specific port communication from the client, exported the packet, converted it to CSV format, and created a sequence diagram by hand on Excel. It was useful for the team to discuss, but it took too long and I didn't want to repeat the same thing again, so I created the tool.

Environment at that time (reference)

Http Client --- Internet --- LLB --- FW --- SLB --- Http Server

1-2. What you need to run the tool

• Python 2.7
• Seqdiag
• Packet capture file (csv format)
• Must be in the following column format

"No.","Time","Source","Destination","Protocol","Length","Info"

• The time display format of the Time column must be the "date and time" format of wireshark's time display format.

 Date and time format (1973-06-14 01: 02: 03.123456)

1-3. Referenced sites

• Sequence diagram generation tool seqdiag

2. Flow

2-1. Environment construction

1. Install Python 2.7

$ curl -L -O https://www.python.org/ftp/python/2.7.12/Python-2.7.12.tgz
$ tar zxvf Python-2.7.12.tgz
$ cd Python-2.7.12
$ ./configure
$ make && make altinstall

2. Install pip

$ curl -kL https://raw.github.com/pypa/pip/master/contrib/get-pip.py | python

3. Modified to use yum Fixed yum command interpreter to run existing Python (Version 2.6)

$ vi /usr/bin/yum
 Example) Corrected as follows
!/usr/bin/python2.6

4. Preparation of the created tool --When using as a command

$ pip install packetseq

--When using only the program Save to the environment where packetseq.py is executed

2-2. Usage procedure (overview)

1. Packet capture at the communication path location
2. Export only the communication of a specific port from the capture file
3. Save as csv file
4. Send the csv file on the tool execution environment created this time
5. Run the tool
6. Enter the name of the opposite device between communication from standard input
7. Sequence diagram created

2-3. Usage procedure (explanation)

This time I will explain how to use it based on the following environment

 Client (Windows 7) --- Proxy (Squid) --- Server (nifty.com, etc.)

1. Packet capture at the communication path location Wireshark was used on Client (Windows 7) and tcpdump was used on Proxy (Squid). Example) Packet capture on the client

 Run wireshark and save when communication is complete
 File name is arbitrary, save as client.pcap

Example) Packet capture on a proxy server

$ tcpdump -i eth1 -w proxy.pcap
 File name is arbitrary, save as proxy.pcap

2. Export only the communication of a specific port from the capture file Open the file you captured earlier with wireshark and narrow it down to a specific port Specifically, check the port number when sending the SYN flag of 3way handshake. Filter on wireshark

 tcp.port == port number

3. Save as csv file Save in csv format with wireshark Example) Packet capture on the client

 File name is arbitrary, save as client.csv

Example) Packet capture on a proxy server

 File name is arbitrary, save as proxy.csv

4. Send the csv file on the tool execution environment created this time
5. Run the tool Since there are two collection points this time, the files captured earlier are passed as arguments in order from the client side and executed. --When installing from pip etc. and using from commands

$ packetseq client.csv proxy.csv

--When executing from saved packetseq.py

$ python packetseq.py client.csv proxy.csv 

6. Enter the name of the opposite device between communication from standard input You will be prompted with standard input to enter the name twice, source-ip and destination-ip for each file The example below is between a client and a proxy, so I chose "Client" and "Proxy".

\########################################
file_name:client.csv
\########################################
src ip:192.168.1.3 -> src name: ???
 input src name> Client (★ standard input)
dst ip:192.168.1.62 -> dst name: ???
 input dst name> Proxy (★ standard input)

Next, input is required between proxy servers, so I chose "Proxy" and "Sever".

\########################################
file_name:proxy.csv
\########################################
src ip:192.168.1.3 -> src name: ???
 input src name> Proxy (★ standard input)
dst ip:192.168.1.62 -> dst name: ???
 input dst name> Server (★ standard input)

7. Sequence diagram created Files out.png and out.diag are created out.png is a PNG format sequence diagram Seqdiag format file from which out.diag is the source of image generation

$ file out.*
out.diag: ASCII text, with very long lines
out.png:  PNG image data, 1856 x 31706, 8-bit/color RGBA, non-interlaced

From the left, it is a sequence diagram of "Client", "Proxy", "Web".
With this, you can see the sequence of packets at multiple locations.

3. About tools

3-1. Source code

See packetseq.py

3-2. Process flow

1. Pass the csv file as an argument
2. Detect between communications from the SYN flag for each file 3.2 Pass the name of the opposite node of 2 as standard input
3. Replace the IP address with the name specified in 3.
4. Output to Seqdiag format
5. Pass it to Seqdiag and output the image

3-3. Optional

3-4. Color specification

Css color keyword can be specified Example) Specification on the source code

Color settings are specified in self.color_dict of the \ __ init__ function

 #Dictionary of color information
        self.color_dict = {
            'SYN': "blue", 'SYN ACK': "red",
            'ACK': "green",
            'FIN': "navy", 'FIN ACK': "maroon",
            'RST': "purple", 'RST ACK': "fuchsia",
            'URG': "olive", 'PSH': "orange",
            'Other': "gray",
        }

3-5. Output of packet information

1. "summary" tcp flag and send data only
2. "info" info Full data display
3. "default" Remove sport and dport

4. Challenges

• I want to make it easier to see the time and see where the delay is occurring
• I want to make it easier to see the correspondence of packets and associate the transmitted data with the corresponding ACK.
• Allow hard-coded parts such as color specification to be specified from the config file

5. Attention

• Since it is sorted by time, the correct sequence diagram will not be obtained if the time is off (although it is natural).
• Depending on the font (?) In the sequence diagram creation environment, the character display of packet information may be slightly corrupted (under investigation).