[PYTHON] I tried using PyEZ and JSNAPy. Part 1: Overview

This is the first day article of NetOpsCoding AdventCalendar2016. There is still space, so if anyone can write an article, please join us! NetOpsCoding Advent Calendar 2016

It doesn't have to be difficult. It can be an emotional article full of passion. It doesn't matter if it's a simple and small tip. (Rather small tips are appreciated). If anyone is frustrated with network operation, I would be very happy as a person in NetOps Coding if you could take the first step of automation as one of the triggers.

Overview

We were invited by Juniper to announce it on December 1st at an event called "Juniper Cloud Builder Community 2016".

As for the content of the announcement, Juniper is released as open software PyEZ and JSNAPy ) Is used to verify whether it can be used at the actual network operation site of the ISP.

The materials used for the presentation are here. Possibility of next-generation network operation created by JSNAPy and PyEZ

Since the announcement took 25 minutes, I could hardly introduce the details of the tools and sample code, so I will introduce the details in this blog.

To conclude first, PyEZ and JSNAPy are software for automating router configuration work and are very powerful tools. Software can replace almost the same procedure as manual work.

When I wrote out the samples and know-how, the amount was too large, so I am writing the blog in the following four parts.

-I tried using PyEZ and JSNAPy. Part 1: Overview (Imakoko) -I tried using PyEZ and JSNAPy. Part 2: I tried using PyEZ -I tried using PyEZ and JSNAPy. Part 3: I tried using JSNAPy -I tried using PyEZ and JSNAPy. Part 4: Automating ISP setting work with PyEZ and JSNAPy

Finally, I made the following demo program. The right side of the screen is the target device Firefly (the upper right firefly1 is the setting target), and the left side of the screen is the automation tool. Introduced in "Part 4: Automating ISP setup work with PyEZ and JSNAPy".

PyEZ overview

PyEZ is a Python library for reconfiguring JUNOS routers. In addition to loading and committing configs, it has exactly the same functions as the manual router configuration procedure.

• commit
• rollback
• load merge --Read the candidate config after leaving the Active config (uncommitted)
• load overload --After erasing the Active config, load the candidate config (uncommitted)
• commit check --Check if the candidate config can be committed
• diff (show | compare) --Check the difference between Active config and candidate config.

When automating router settings, there are many needs such as "I want to perform automatic settings with exactly the same procedure as manual settings", so I am very grateful that "I can do exactly the same as manual settings without degrading". .. (There are unexpectedly many automation tools that have restrictions on available functions and setting methods.)

JSNAPy Overview

JSNAPy is a tool that takes and manages the status of the JUNOS router as a snapshot and determines whether it matches the predefined conditions. Simply put, JSNAPy makes it easy to implement test tools. "Returns" Passed "when interface xe-0 / 0/0 is up" "Failed" when the route information of state 2 increases more than expected for state 1. You can say "pass") "return".

Being able to write tests is very powerful in automating router configuration. It goes without saying that it is important to confirm that "the state after setting is operating normally" in the router setting work. Since the router has a large service impact range, it is hard to see if "the setting failed and the network disappeared".

However, automating this normalization process is actually a very, very painstaking task. "For all router commands input by the router CLI, define all patterns of output results that can be returned by the router, extract the target character string with a regular expression, and determine whether it matches the predefined conditions. That's all there is to say, "The number of patterns that can be output at the time of an error is huge and cannot be defined." "The output result differs slightly depending on the model / OS version." "The program is a pile of regular expressions." There are many barriers such as "It will become impossible for anyone other than the implementer to maintain it", and there is no choice but to overcome it with human-sea tactical operations. If you are interested in the hardships around that, please refer to it as it was introduced on @ stereocat's blog. (Just looking at it makes my stomach hurt) This and that of CLI-based NW automation bad know-how

By using JSNAPy, you can eliminate the trouble of implementing test tools, so I am very grateful. It's hard to tell if all of the tasks I mentioned earlier will be easier, but I'm sure most of them will be easier.

The lack of a router version of the test tool Serverspec in terms of servers is one of the major factors that make it difficult to automate network operations today. So I'm really looking forward to the emergence of test tools like this JSNAPy.

Concept of multi-vendor

Before I introduce PyEZ and JSNAPy in detail, I would like to talk a little about automation.

Some organizations operating networks may criticize that "automating with Juniper's dedicated tools will not be half as effective if it cannot be operated by other companies' routers." That's exactly right. The tools introduced above will only work with JUNOS routers. Of course, there are many network devices other than Juniper in my organization. Of course, the above Juniper tools cannot support all devices.

However, in automating network operations, it is a very difficult task at this point to suddenly aim to support all vendors. That's because there isn't much open software available at this time for all multi-vendor products. (At the moment, it seems that third-party APIs like Ansible and NAPALM are aiming for that position, but you have to be careful because each device is apt to meet the functional requirements you want to automate.)

I think that everyone has their own ideas about how to realize multi-vendor support, so I will not discuss it here, but as my personal idea at the moment, I will proceed with examination and verification according to the following flow. I think that it will move forward well.

1. First, focus on one product and consider a mechanism for automating operation operations-> implementation-> verify and establish an operation policy. --Select one target product based on the frequency of work and the ease of automation. ――At this point, use the APIs and libraries provided by the vendor to make it easier. ――While implementing automation in one product, we will proceed with verification while always imagining multi-vendor support, whether it can be applied to other companies' products, and if so, how the system can be designed to migrate.
2. After implementing 1., if you are confident in the work automation policy, consider whether the same mechanism can be applied to products of other vendors-> Implementation-> Verify.
3. If the existing method does not work in 2., try to implement the alternative version by imitating the mechanism used in 1. If it is difficult to implement by one person, find a collaborator inside or outside the company

As mentioned above, I imagine that the implementation of multi-vendor tools will progress if we can start with the parts that are easy to automate and finally create a flow close to open source software development. In fact, there is also a tool called NAPALM as a multi-vendor library that is being implemented with such ideas.

NAPALM has been introduced in past blogs, so please take a look if you are interested. -I touched the router control API library NAPALM --I plan to write a sequel in the second half of NetOps Coding Advent Calendar 2016.

Based on this idea, in my case

1. First, build a mechanism for trying and verifying automation with Juniper products, which have abundant network automation mechanism and documents. (This announcement / blog is targeted here)
2. Once the automation method for Juniper products has been refined, we are looking for ways to expand / apply the mechanism to other vendor products. --Consider a plan to use the resource management module and instruction execution module as they are, and replace only the router control module with a module that can support products from other vendors (NAPALM is the most promising module at the moment?). --There is no test tool equivalent to JSNAPy that can be used with other vendor products, so we are considering how to implement it.
3. When the prospect of production introduction to network operation is established, deepen discussions inside and outside the company by disclosing ideas and verification results in the external community, and further refine the ideas and implementation methods.

With this idea, we are advancing the policy of automated development.

Although the introduction has become long, I will introduce the specific usage of PyEZ and JSNAPy from the next time.

I tried using PyEZ and JSNAPy. Part 2: I tried using PyEZ