[PYTHON] Have pandas read the zip file on the web

Download zip locally-> unzip-> load it into pandas A memo for wanting to drop it in the dataframe while it is still in memory

There was such a url on the web https://www.stats.govt.nz/assets/Uploads/Electronic-card-transactions/Electronic-card-transactions-February-2020/Download-data/electronic-card-transactions-february-2020-csv.zip

import requests
import zipfile
import io
import pandas as pd

# url
url = "https://www.stats.govt.nz/assets/Uploads/Electronic-card-transactions/Electronic-card-transactions-February-2020/Download-data/electronic-card-transactions-february-2020-csv.zip"

#Get
res = requests.get(url)
#Defrost
z = zipfile.ZipFile(io.BytesIO(res.content))
#Verification
z.namelist() 
#Output result
# ['electronic-card-transactions-Feb-2020-csv-tables.csv']

#Read csv in binary
with z.open(z.namelist()[0], 'r') as myfile:
    binaryCSV = myfile.read()

# DataFrame
df = pd.read_csv(io.BytesIO(binaryCSV), encoding='utf-8') #Encoding for Japanese='sjis'

#Verification
df.head()

image.png

Recommended Posts

Have pandas read the zip file on the web
Read CSV file: pandas
Unzip the internet zip file
Have python read the command output
Let's read the RINEX file with Python ①
Have Voiceroid read the Twitter TL aloud
Read the file line by line in Python
Download files on the web with Python
Read the file line by line in Python
Read the file by specifying the character code.
Read the file with python and delete the line breaks [Notes on reading the file]
Read file
[Python] Read the specified line in the file
Publish the current directory on the web server
Get the size of the image file on the web (Python3, no additional library required)
Execution environment on the Web by "Project Jupyter"
[Python] How to read excel file with pandas
Note that the Pandas loc specifications have changed
Let's simulate the Izhikevich neuron model on the web!
The .pyd file is the DLL itself on Windows
Read pandas data
Read the csv file with jupyter notebook and write the graph on top of it
Until the web application is released on Sakura VPS
Template of python script to read the contents of the file
Read the csv file and display it in the browser
Read the xml file by referring to the Python tutorial
Execute the command on the web server and display the result
Unzip a ZIP file larger than 4GB on Linux.