Download files on the web with Python

Download the file on the web server

I wrote a useful program in python when I wanted to download automatically. It's very easy to do with urllib.

#!/usr/bin/env python
#-*- coding:utf-8 -*-

import urllib.request
import sys

def download():

	url = sys.argv[1]
	title = sys.argv[2]

if __name__ == "__main__":

Execution method

python [url] [File title]

When executed, the file will be created in the same directory as the file. (If you enter the absolute path in the file title, it will be created on that path.)

sys.argv can be used by importing the sys module. The file name of the program is entered in sys.argv [0], and the arguments entered when executing in the terminal are assigned in order from sys.argv [1]. If you want to download many files automatically, find the regularity of the url, or scrape using a module like Beautifulsoup to extract the url and use the for statement. You can do this by running download () multiple times.

Regularity means that if there is an image URL such as "https://hogehoge/service/10-1.png ", 3 of 3-1.png is a category and 1 is a serial number. It can happen. If you look at multiple samples and find regularity, you can download them all at once. (Since it puts a burden on the server side, it is better to devise such as putting time.sleep (0.1) in the script)


urllib sys Beautifulsoup

Recommended Posts

Download files on the web with Python
Save images on the web to Drive with Python (Colab)
Visualize your pocket money files with the Python web framework Dash
Vienna with Python + Flask web app on Jenkins
Introduction to Python with Atom (on the way)
Download and import files with Splunk external python
Extract the table of image files with OneDrive & Python
Download the file in Python
[Python] How to specify the download location with youtube-dl
Sorting image files with Python (2)
Sort huge files with python
Make a breakpoint on the c layer with python
[Python] How to save images on the Web at once with Beautiful Soup
Web API with Python + Falcon
Sorting image files with Python
Integrate PDF files with Python
Reading .txt files with Python
Call the API with python3.
Web application with Python + Flask ② ③
Web scraping beginner with python
Download csv file with python
Streamline web search with python
Web application with Python + Flask ④ is not reflected on the web server in Python3.
Access the host SQL Server with python27 / pyodbc on the container
[Part.2] Crawling with Python! Click the web page to move!
[CleanArchitecture with Python] Part2: Frameworks & Drivers Layer: Introducing the Web
Extract the xz file with python
Recursively unzip zip files with python
Web scraping with Python ① (Scraping prior knowledge)
Implemented file download with Python + Bottle
Manipulating EAGLE .brd files with Python
Getting Started with Python Web Applications
Web scraping with Python First step
I tried web scraping with python.
Monitor Python web apps with Prometheus
Get the weather with Python requests
Get web screen capture with python
Get the weather with Python requests 2
Download Google Drive files in Python
Decrypt files encrypted with OpenSSL with Python 3
Find the Levenshtein Distance with python
Hit the Etherpad-lite API with Python
Install the Python plugin with Netbeans 8.0.2
Install the data files with
Download the file deployed with
Hit the web API in Python
Handle Excel CSV files with Python
I liked the tweet with python. ..
Memorize the Python commentary on YouTube.
Read files in parallel with Python
Notes on using rstrip with python.
Master the type with Python [Python 3.9 compatible]
Getting started with Python 3.8 on Windows
Sakura Use Python on the Internet
[Memo] Tweet on twitter with python
Life game with Python [I made it] (on the terminal & Tkinter)
How is the progress? Let's get on with the boom ?? in Python
Control the motor with a motor driver using python on Raspberry Pi 3!
The first artificial intelligence. Challenge web output with python. ~ Flask introduction
Get started with the Python framework Django on Mac OS X