[PYTHON] Try using the PeeringDB 2.0 API

Overview

This is the article on the 9th day of NetOpsCoding AdventCalender.

This time, I will create a program to get AS information using the API published in PeeringDB 2.0.

What is Peering?

The interconnection of networks of organizations with AS number is called Peer or Peering. Specifically, the protocol BGP is run on the routers of both organizations, and the IP addresses and AS numbers of each organization Peering is realized by setting.

There are several ways to do Peering, but I won't go into details here. If you are interested in the details, A lot of information is published, so please refer to the following materials. http://www.slideshare.net/taijitsuchiya5/wakamonog6-tsuchiya-public

What is PeeringDB

Let's say you want to connect to the network of Organization A. At that time, "What is the AS number of organization A?" "Which DC base has the connection router?" "Which IX operator is used?" "Which IP address is used?" "Where is the contact? You need to know information such as "ka".

PeeringDB is the most commonly used information in the world when considering Peering and wanting to find out information about the partner organization. PeeringDB is a Web system developed and operated by a non-profit organization, and organizations around the world that own AS post information on PeeringDB. (However, since the posting of information is left to the organization that owns the AS, the information may be out of date or not posted in the first place.)

Peering DB 2.0 was released as a beta version in 2015. The UI and functions have been greatly improved from the previous PeeringDB, and information can be obtained using the RESTful API. http://docs.peeringdb.com/

Try using PeeringDB

Anyone can see PeeringDB for free. For example, you can check the AS information of BIGLOBE as follows. http://as2518.peeringdb.com/

By checking the PeeringDB, you can find information such as "company information", "AS number", "rough traffic flow rate", "Peering policy (Peering judgment criteria)", and "Peering base".

Almost the same content is published in PeeringDB 2.0. https://beta.peeringdb.com/net/2229

Try using the PeeringDB 2.0 API

The specifications of PeeringDB2.0 API are published on the following page. https://beta.peeringdb.com/apidocs/

For example, if you want to get BIGLOBE (AS2518) information all at once, access as follows.

# Linux/To get it from a Mac terminal
% curl https://beta.peeringdb.com/api/asn/2518

{
  "meta": {}, 
  "data": [
    {
      "id": 2229, 
      "org_id": 2934, 
      "org": {
        "id": 2934, 
        "name": "BIGLOBE Inc.", 
        "website": "", 
        "notes": "", 
        "net_set": [
          2229
        ], 
     
      (Partially omitted)

      "name": "BIGLOBE Inc.", 
      "aka": "FullRoute", 
      "website": "http://www.biglobe.co.jp/en/", 
      "asn": 2518, 
      "looking_glass": "http://lg.fullroute.net/lg/", 
      "route_server": "", 
      "irr_as_set": "AS-MESH", 
      "info_type": "NSP", 
      "info_prefixes4": 750, 
      "info_prefixes6": 750, 
      "info_traffic": "200-300 Gbps", 
      "info_ratio": "Balanced", 
      "info_scope": "Asia Pacific", 
      "info_unicast": true, 
      "info_multicast": false, 
      "info_ipv6": true, 
      "notes": "We are connected to BBIX Singapore but not Hong Kong. We will only peer with Singapore switch participants.\r\n\r\nYou can ping and trace to ping.mesh.ad.jp(both IPv4 and IPv6).\r\n\r\nIRR Records should be viewed with source set to RADB or JPIRR. There are colliding records with the same AS-SET name, but from a different AS on RIPE. \r\n\r\nSee here for Japanese version of the peering policy http://www.biglobe.co.jp/pdf/PeeringPolicy2013_jp_03.pdf",
      "policy_url": "http://www.biglobe.co.jp/en/peering_policy.pdf",
      "policy_general": "Selective",
      "policy_locations": "Preferred",
      "policy_ratio": false,
      "policy_contracts": "Not Required",
      "netfac_set": [
        {
          "id": 17472,
          "fac_id": 19,
          "avail_sonet": false,
          "avail_ethernet": true,
          "avail_atm": false,
          "local_asn": 2518,
          "created": null,
          "updated": "2015-12-05T04:29:57Z",
          "status": ""
        },
        {
          "id": 9426,
          "fac_id": 142,
          "avail_sonet": false,
          "avail_ethernet": true,
          "avail_atm": false,
          "local_asn": 2518,
          "created": null,
          "updated": "2015-12-05T04:25:36Z",
          "status": ""
        },
        (Omitted below)

It should be noted here that the item "netfac_set" contains DC information and IX information, but since the data of "netfac_set" refers to another table as a foreign key, only the ID is included. not. If you want to know more about IX information, you need to create another URL and execute it.

For example, if you want to see more information specified by "fac_id": 19 in "netfac_set", you need to execute an HTTP request with the following URL.

% curl https://beta.peeringdb.com/api/fac/19                                                                                                                                                 (git)-[master]
{
  "meta": {},
  "data": [
    {
      "id": 19,
      "org_id": 618,
      "org_name": "CoreSite",
      "org": {
        "id": 618,
        "name": "CoreSite",
        "website": "",
        "notes": "",
        "net_set": [],
        "fac_set": [
          19,
          20,
          286,
          363,
          440,
          584,
          668,
          669,
          670,
          671,
          1229,
          1619,
          1671
        ],
        "ix_set": [],
        "address1": "",
        "address2": "",
        "city": "",
        "country": "",
        "state": "",
        "zipcode": "",
        "created": "2015-10-12T08:14:45Z",
        "updated": "2015-12-05T09:07:24Z",
        "status": "0"
      },
      "name": "CoreSite - LA1 - One Wilshire",
      "website": "http://www.coresite.com",
      "clli": "LSANCA",
      "rencode": "",
      "npanxx": "213-489",
      "notes": "",
      "created": null,
      "updated": "2015-12-05T09:05:59Z",
      "status": "0",
      "address1": "624 S. Grand Ave.",
      "address2": "Suite 110",
      "city": "Los Angeles",
      "country": "US",
      "state": "CA",
      "zipcode": "90017"
    }
  ]
}%

In the PeeringDB API, information tables such as AS information (asn), DC information (fac), and IX information (ix) are stored separately as published at https://beta.peeringdb.com/apidocs/. And is referenced as a foreign key.

In this way, if you want to get the information you want to know with API, you need to get the foreign key and execute the HTTP request multiple times while changing the URL.

Program to acquire information with PeeringDB2.0 API

Create a program to get information using API.

Here, I made a Python program that outputs the information of all IXs connected to the company information by giving the AS number to the argument and executing it as shown below.

% python get_peeringdb.py <Target AS number>

The program is also available on github. https://github.com/netops-coding/sample_peeringdb

get_peeringdb.py


#! /usr/bin/env python
# -*- coding: utf-8 -*-

from urllib2 import urlopen, HTTPError
from json import loads
import sys

def get_peeringdb(pdp_type, pdp_id):
    pdb_url = 'https://beta.peeringdb.com/api/%s/%s'% (pdp_type, pdp_id)

    try :
        result_json = urlopen(pdb_url).read()
        result_dict = loads(result_json)
    except HTTPError, err:
        if err.code == 404:
            return None
    return result_dict


if __name__ == '__main__' :
    as_num = sys.argv[1]

    # get company and AS information
    as_info_dict = get_peeringdb('asn', as_num)
    company_name = as_info_dict["data"][0]["org"]["name"]
    company_website = as_info_dict["data"][0]["website"]

    print as_num
    print company_name
    print company_website

    print '='*30

    # get IX information
    for netixlan_set in as_info_dict["data"][0]["netixlan_set"]:
        ixlan_id = netixlan_set["ixlan_id"]
        ipaddr4 = netixlan_set["ipaddr4"]
        ipaddr6 = netixlan_set["ipaddr6"]

        ixlan_info_dict = get_peeringdb('ixlan', ixlan_id)
        ix_name =  ixlan_info_dict["data"][0]["ix"]["org"]["name"]

        print ix_name
        print ipaddr4
        print ipaddr6
        print '='*30

Execute it with the following command. Here, BIGLOBE (AS2518) is specified.

% python get_peeringdb.py 2518

                                                                                                                                                         (git)-[master]
2518
BIGLOBE Inc.
http://www.biglobe.co.jp/en/
==============================
Equinix Osaka
None
None
==============================
Equinix San Jose
206.223.116.156
2001:504:0:1::2518:1
==============================
Equinix Singapore
202.79.197.180
2001:de8:4::2518:1
==============================
HKIX
123.255.90.157
2001:7fa:0:1::ca28:a09d
==============================
Equinix Tokyo
203.190.230.56
2001:de8:5::2518:2
==============================
Equinix Tokyo
203.190.230.83
2001:de8:5::2518:3
==============================
SGIX
103.16.102.69
2001:de8:12:100::69
==============================
CoreSite - Any2 California
206.72.210.154
2001:504:13::210:154
==============================
BBIX Hong Kong / Singapore
103.231.152.25
2001:df5:b800:bb00::2518:1
==============================
JPIX
210.171.224.181
2001:de8:8::2518:2
==============================
BBIX Tokyo
218.100.6.47
2001:de8:c::2518:1
==============================
BBIX Tokyo
218.100.6.73
2001:de8:c::2518:2
==============================
JPNAP Tokyo
210.173.176.14
2001:7fa:7:1::2518:2
==============================
JPNAP Tokyo
210.173.176.124
2001:7fa:7:1::2518:3
==============================

As described above, we were able to create a program that acquires a list of IX information to which the organization is connected simply by specifying the AS number.

Published Python package

Looking at the program created this time, the data structure of PeeringDB 2.0 such as "as_info_dict [" data "] [0] [" org "] [" name "]" is used as it is, and a third party programs it. It is very difficult to understand when you look at it, and when developing additional functions, it is necessary to proceed with development while examining the data structure, which tends to be complicated.

By using the Python package that has already been released, you can develop software without being aware of these data structures.

There were two Python packages I found: To conclude, the second package is a powerful way to facilitate software development, but there are problems and it is still completely difficult to use. We are looking forward to the development of the second repair and additional functions.

Finally

This time, I made software to get the information published in PeeringDB via API. It is very convenient to obtain from the public information of PeeringDB when creating a database of company information and network information that is Peering with your own network. Please use it if you have a chance.

Recommended Posts

Try using the PeeringDB 2.0 API
Try using the Twitter API
Try using the Twitter API
Try using the Wunderlist API in Python
Try using the Kraken API in Python
Try using the DropBox Core API in Python
Try using Janus gateway's Admin API
Try using Pleasant's API (python / FastAPI)
Try using Python argparse's action API
Try using the Python Cmd module
I tried using the checkio API
For the time being, try using the docomo chat dialogue API
Try using Tkinter
Try using the web application framework Flask
Try using the $ 6 discount LiDAR (Camsense X1)
Try using the HL band in order
Try using cookiecutter
Try using PDFMiner
Try the Microsoft Cognitive Services Face API
Tweet using the Twitter API in Python
Try using geopandas
Try using Dropbox API v2 with Go
Try using Selenium
Try using scipy
Try cluster analysis using the K-means method
Create an application using the Spotify API
Try using pandas.DataFrame
Try using django-swiftbrowser
Try using matplotlib
Try hitting the YouTube API in Python
Try using tf.metrics
Play with puns using the COTOHA API
Try using PyODE
Try hitting the Spotify API in Django.
Record custom events using the Shotgun API
I tried using the BigQuery Storage API
I checked the library for using the Gracenote API
Hit the Web API using requests Example: Flickr
Python: Try using the UI on Pythonista 3 on iPad
Try using the Chinese morphological analysis engine jieba
I tried using the Google Cloud Vision API
Try using the Python web framework Tornado Part 1
Try using LINE Notify for the time being
Try using the collections module (ChainMap) of python3
Try using kabu station API of kabu.com Securities
Try using the Python web framework Tornado Part 2
Try accessing the YQL API directly from Python 3
Try using ChatWork API and Qiita API in Python
Try using virtualenv (virtualenvwrapper)
[Azure] Try using Azure Functions
Try using virtualenv now
Try using W & B
Try using Django templates.html
[Kaggle] Try using LGBM
Try using Python's feedparser.
Try using Python's Tkinter
Try NNabla's C ++ API
Try using Tweepy [Python2.7]
Try using Pytorch's collate_fn
Try drawing a social graph using Twitter API v2
Initial settings when using the foursquare API in python