[PYTHON] Get data using Ministry of Internal Affairs and Communications API

Get data using Ministry of Internal Affairs and Communications API

I took the data in Python to create the panel data for econometrics. (It is a memorandum)

Preparation

Python runtime environment Acquisition of Ministry of Internal Affairs and Communications API You can get it by registering as a user from here.

code

def_TakeData


import requests,urllib
import pandas as pd
import numpy as np
import json
def get_json(base_url,params):
    params_str=urllib.parse.urlencode(params)
    url=base_url+params_str
    json=requests.get(url).json()
    return json

def take_data(dataid):
    appID="Enter the appID"
    base_url="http://api.e-stat.go.jp/rest/2.1/app/json/getStatsData?"
    params={
        "appId":appID,
        "lang":"J",
        "statsDataId":dataid,
        "metaGetFlg":"Y",
        "cntGetFlg":"N",
        "sectionHeaderFlg":"1"
    }

    data=get_json(base_url,params)
    return data

take


data=take_data(XXXXX)

json format data is converted url is http://api.e-stat.go.jp/rest/2.1/app/getStatsData?appId=1lang=J&statsDataId=(Enter the data ID here) & metaGetFlg = Y & cntGetFlg = N & sectionHeaderFlg = 1 The above format. You can easily refer to the API request URL from the Ministry of Internal Affairs and Communications website.

json->DataFrame

Example


df=pd.DataFrame(data['GET_STATS_DATA']['STATISTICAL_DATA']['DATA_INF']['VALUE'])
#Convert json data to pandas DataFrame(Example)

Store the desired data in the DataFrame.

-`data['GET_STATS_DATA']['STATISTICAL_DATA']['CLASS_INF']['CLASS_OBJ']`

You can see what data is in json with the above code.

Now you have a DataFrame that is easy to operate

Finally

It was easier to use than the Twitter API, so I'll do my best to make more details.

I am a beginner in Python, so please point out any mistakes.

Recommended Posts

Get data using Ministry of Internal Affairs and Communications API
Get Amazon data using Keep API # 1 Get data
Get a list of GA accounts, properties, and views as vertical data using API
Get Youtube data in Python using Youtube Data API
Awareness of using Aurora Severless Data API
[Python] Get all comments using Youtube Data API
How to get article data using Qiita API
Get LEAD data using Marketo's REST API in Python
[Python] Get insight data using Google My Business API
Get comments and subscribers with the YouTube Data API
I tried using the API of the salmon data project
Get all songs of Arashi's song information using Spotify API and verify the index
Get rid of dirty data with Python and regular expressions
[Python] I tried collecting data using the API of wikipedia
Get product name and lowest price using Amazon Product Advertising API
Get the weather using the API and let the Raspberry Pi speak!
How to get followers and followers from python using the Mastodon API
Collect product information and process data using Rakuten product search API [Python]
Get and set the value of the dropdown menu using Python and Selenium
[Python] I tried to get various information using YouTube Data API!
[Ruby on Rails] Display and pinning of GoolgeMAP using Google API
Benefits and examples of using RabbitMq
Data acquisition using python googlemap api
Data acquisition memo using Backlog API
Recommendation of data analysis using MessagePack
Get data from Twitter using Tweepy
[MS Azure] Slack notification of competition information using Azure Functions and Kaggle API
[Windows] Cause and solution of NotImplementedError when using asyncio.create_subprocess_shell () with Fast API
Visualization of latitude / longitude coordinate data (assuming meteorological data) using cartopy and matplotlib
Try to get the road surface condition using big data of road surface management
Development and deployment of REST API in Python using Falcon Web Framework
Get news from three major mobile companies using Django and the News API
Graph display of AIX and Linux nmon data without using MS Excel
Get the number of articles accessed and likes with Qiita API + Python
Data cleansing of open data of the occurrence situation of the Ministry of Health, Labor and Welfare
Get and estimate the shape of the head using Dlib and OpenCV with python
I tried using Twitter api and Line api
Buy and sell cryptocurrencies using Zaif API
Example of using class variables and class methods
Get the column list & data list of CASTable
I tried using YOUTUBE Data API V3
Get mail using Gmail API in Java
Python data structure and internal implementation ~ List ~
Get the minutes of the Diet via API
Get Google Fit API data in Python
Separation of design and data in matplotlib
Collect data using scrapy and populate mongoDB
Creating Google Spreadsheet using Python / Google Data API
FX data collection using OANDA REST API
Image analysis was easy using the data and API provided by Microsoft COCO.
Create an easy-to-read pdf of laws and government ordinances using the law api
Get data via salesforce API (Bulk API) in Python and load it into BigQuery