[PYTHON] I tried using PyEZ and JSNAPy. Part 4: Automate ISP setup with PyEZ and JSNAPy

I tried using PyEZ and JSNAPy. This is the final episode of. This time, I will introduce the contents of automating the eBGP Peering setting work that is common in ISP backbone work using PyEZ and JSNAPy.

I tried using PyEZ and JSNAPy. Part 1: Overview I tried using PyEZ and JSNAPy. Part 2: I tried using PyEZ I tried using PyEZ and JSNAPy. Part 3: I tried using JSNAPy I tried using PyEZ and JSNAPy. Part 4: Automating ISP setting work with PyEZ and JSNAPy (Imakoko)

What to do in the final episode

Let's use what we learned in Part 1 to Part 3 to automate the BGP Peering task, which is one of the tasks that frequently occur in the ISP backbone.

BGP Peering is the work of connecting eBGP to connect the routers of the organization that owns the AS number. By inputting the eBGP settings while the routers are directly connected to each other in a specific data center, route information is exchanged and traffic is exchanged.

The flow of BGP Peering work is as follows.

1. Negotiations between organizations / Arrangement of premises wiring
2. Cable wiring
3. Interface settings
4. Confirmation of communication
5. Input of eBGP settings
6. Confirmation of route public relations
7. Check traffic transition status

This time, we will automate the logical setting work steps 3 to 6. Then, instead of suddenly fully automating the entire process, we automated the setting / confirmation work up to just before the commit, and proceeded on the premise of semi-automation by leaving the commit implementation to the judgment of the worker.

Automation system configuration

This is a system configuration image.

Here, the story is to input the interface settings and eBGP settings to the JUNOS router "Firefly 1" that has not been set up via the PyEZ library and JSNAPy library.

In this system, the worker creates the following "scenario file" as an alternative to the work procedure manual and router config that the worker has traditionally created. By loading this scenario file into the scenario execution program, the router setting and network health check process can proceed according to the procedure intended by the operator. In this system, the worker can work only by creating this scenario file without creating the conventional procedure manual and router config in advance, so the aim is to reduce the burden on the worker and the reviewer.

Scenario file

purpus:  |
This work is done by ABC Co., Ltd.(AS65002)And the router operated by
The purpose is to build a private peer at base A.
Route exchange is received one route at a time/Will be sent and depends on the work
It is assumed that the impact of the network will be minor.
operator: Taiji Tsuchiya
operation_date: 20161115
hosts:
  management_ipaddress: 192.168.34.16
  hostname: firefly1
  model: firefly-perimeter
  username: user1
  password: password1
scenario:
  - test_hostname
  - test_model
  - test_interface:
      interface_name: ge-0/0/2
      interface_status: up
  - set_add_interface:
      interface_name: ge-0/0/2
      interface_address_ipv4: 192.168.35.1
      interface_subnet_ipv4: 30
      interface_description: AS65002_peer
  - test_interface:
      interface_name: ge-0/0/2
      interface_status: up
  - set_add_bgp_neighbor:
      interface_name: ge-0/0/2
      neighbor_asnum: 65002
      neighbor_address_ipv4: 192.168.35.2
      neighbor_description: AS65002_peer
  - test_bgp_neighbor:
      neighbor_address_ipv4: 192.168.35.2
      neighbor_status: Established
  - set_add_bgp_policy_external:
      external_policy_name: AS65002_export
      advertised_route_address_ipv4: 10.10.10.0
      advertised_route_subnet_ipv4: 24
      interface_name: ge-0/0/2
      neighbor_address_ipv4: 192.168.35.2
  - sleep_10sec
  - test_bgp_received_route:
      neighbor_address_ipv4: 192.168.35.2
      received_route_address_ipv4: 10.10.30.0
      received_route_subnet_ipv4: 24
  - test_bgp_advertised_route:
      neighbor_address_ipv4: 192.168.35.2
      advertised_route_address_ipv4: 10.10.10.0
      advertised_route_subnet_ipv4: 24

Looking at the above scenario file, you may find it difficult to read at first glance because it is full of English words. The scenario file consists of a combination of the "procedure name" required when working on the router and the "parameters" required for it. .. The procedure starting with "set _..." indicates the router setting work, and the procedure starting with "test _..." indicates the router confirmation work.

From those who usually make work procedure manuals and router configs and those who are inspected, if you read the scenario file firmly, you can imagine what kind of procedure you are trying to realize. And I think you can get the impression that it is much easier than preparing a router config.

As a design concept of this system, "It is better to simplify the information to the utmost limit like a scenario file rather than a work procedure manual full of ordinary Japanese, for the points that the creator / reviewer should check originally. We are implementing such a system with the idea that it may be possible to concentrate our nerves on the inspection (as there are no unnecessary confirmation items).

At this point, I talked about "the worker does not make a router config in this system", but you may have wondered "how do you make an input config?". In this system, the router config holds the following template files for multiple patterns in the system in advance, and automatically generates the router config by referring to the template file corresponding to each procedure to be called. Is implemented.

Template file_For BGP neighbor configuration

protocols {
    bgp {
        family inet {
            unicast;
        }
        group {{ interface_name }} {
            type external;
            neighbor {{ neighbor_address_ipv4 }} {
                description {{ neighbor_description }};
                peer-as {{ neighbor_asnum }};
            }
        }
    }
}

The router config file is automatically generated by matching the parameters (YAML format) extracted from the scenario file to the above template file (Jinja2 format). By extracting the parameters from the scenario file and generating the router config for each procedure, the operator can save the trouble of the conventional router config pre-creation work.

Template parameters(Extracted from scenario file)

- set_add_bgp_neighbor:
      interface_name: ge-0/0/2
      neighbor_asnum: 65002
      neighbor_address_ipv4: 192.168.35.2
      neighbor_description: AS65002_peer

Automatically generated router configuration file

protocols {
    bgp {
        family inet {
            unicast;
        }
        group ge-0/0/2 {
            type external;
            neighbor 192.168.35.2 {
                description AS65002_peer;
                peer-as 65002;
            }
        }
    }
}

By extracting and generating the router setting procedure and router confirmation procedure from the scenario file in the above flow, you can proceed with the work in the order intended by the worker. ..

Program execution result

The final automation program is published on GitHub. https://github.com/taijiji/scenarioJUNOS

If you execute the program with firefly1 (interface setting not implemented, eBGP setting not implemented) and firely2 (interface setting completed, eBGP setting completed), the program will operate as follows. The left screen shows the automation system, the upper right shows the firefly1 router, and the lower right shows the firefly2 router.

Green characters indicate "the part where normality is confirmed", red characters indicate "the part where abnormality is confirmed", and yellow characters indicate "the part where the operator wants to make a decision".

The execution result of the program is displayed as shown below.

↓ I dare to put out NG here for demonstration. Since it takes several tens of seconds to establish a BGP session, it will not be established unless you sleep and wait.

It may be annoying because many characters are displayed, but during the actual work, you only need to concentrate on the "yellow characters" and "red characters" that may cause abnormalities, and the green characters are concentrated. There is no need to confirm. This will greatly reduce the burden on the worker. Workers can then work with all their nerves focused on detecting dangerous abnormalities that they should be most focused on.

Summary

This time, we were able to build an automation system based on a scenario file that simplifies the conventional work procedure manual. By using PyEZ and JSNAPy, I was able to build this system in about two weeks of actual work without getting stuck in a big wall. (About half of that was the time to load the document.)

PyEZ is a very collaborative library, and the documentation is well maintained, so there was almost no clogging. (If I dare to give it, I just traversed without knowing that the port used for NETCONF over SSH is TCP830.)

JSNAPy was very helpful in creating test tools. Being able to implement this system with zero regular expressions helped me to implement it quickly. However, because the tool itself is new, the documentation is not yet in place, and I've been addicted to it several times. In particular, it was difficult to understand the means for specifying the xpath and obtaining the test results, and I often felt that "I want to get such test results, but how can I write the test file?" Actually, there is no way to do it, so I often wrote down the test conditions I could think of, executed them, and adopted the one that worked well while looking at the results. I imagine that as the number of test file samples increases in the future, the number of addictions around here will decrease. There are a few things we couldn't do to create a test file in less time. For example, how to get the result when test_ping or test_traceroute fails, Some complicated confirmation items such as "conditional statement to confirm that other than the target route is not received / transmitted (double negation)" and "how to confirm that BGP is not working" have not been implemented yet. I did.

Lastly, regarding the demo program I made this time, although it was not implemented sufficiently due to lack of time, I was able to convey the ideal form of network work for the next generation with a concrete image through the demo. I'm glad.

The issues for introducing this system are "whether operators and reviewers will accept this YAML format scenario file" and "whether the same mechanism can be deployed to routers of other companies as a multi-vendor compatible tool". There are still some issues left. In that area, I think it would be good if we could gradually brush up the system according to the needs of the operation site while actually introducing the system.