How to scrape image data from flickr with python

How to use Flickr

Get the key from the Flickr API

First, go to the Flickr page and click ** Request an API Key **. If you haven't created a Flickr account here, create one and log in. 0.png

The left side is for free (non-commercial law) and the right side is for paid (commercial law), so select ** APPLY FOR A NON-COMMERICIAL KEY ** surrounded by a red line. 1.png

Then, the application registration screen will be displayed. Enter the name and intended use for the application, and click ** SUBMIT **. 2.png

After registration, you can get the key and Secret, so make a note of it in a memo pad. 3.png

Create a program to get images from the Flickr API

Install the Flickr module from the following command.

pip install flickrapi

Before executing the program, create a folder to save the acquired images.

download.py


from flickrapi import FlickrAPI
from urllib.request import urlretrieve
from pprint import pprint
import os,time,sys


key = "Obtained key"
secret = "Acquired Secret"
wait_time = 1

#Get the first value of the command line argument. In the following cases[cat]Get
# python download.py cat 
animalname = sys.argv[1]
savedir = "./"+animalname


# format:Data to receive(Receive with json)
flickr = FlickrAPI(key, secret, format='parsed-json')

"""
text :Search keyword
per_page :Number of data you want to acquire
media :Type of data to search
sort :Sequence of data
safe_seach :Whether to display UI content
extras :The value of the option you want to get(url_q Image address information)
"""
result  = flickr.photos.search(
    text = animalname,
    per_page = 400,
    media = 'photos',
    sort = 'relevance',
    safe_seach = 1,
    extras = 'url_q, licence'
)

#result
photos = result['photos']
pprint(photos)

If the above program is executed and it succeeds, the following display will be displayed.

python


           {'farm': 5,
            'height_q': 150,
            'id': '4375810205',
            'isfamily': 0,
            'isfriend': 0,
            'ispublic': 1,
            'owner': '[email protected]',
            'secret': '8d0a7d24a1',
            'server': '4008',
            'title': 'cat',
            'url_q': 'https://live.staticflickr.com/4008/4375810205_8d0a7d24a1_q.jpg',
            'width_q': 150},
           {'farm': 1,
            'height_q': 150,
            'id': '27083352608',
            'isfamily': 0,
            'isfriend': 0,
            'ispublic': 1,
            'owner': '[email protected]',
            'secret': 'd1cd159107',
            'server': '811',
            'title': 'cat',
            'url_q': 'https://live.staticflickr.com/811/27083352608_d1cd159107_q.jpg',
            'width_q': 150}],

Get the URL registered in'url_q'and get the image data using urllib.

Add a program to acquire image data to the previous program.

python


for i,photo in enumerate(photos['photo']):
    url_q = photo['url_q']
    filepath = savedir + '/' + photo['id'] + '.jpg'

   #If there are duplicate files, skip them.
    if os.path.exists(filepath):continue
   #Download image data
    urlretrieve(url_q, filepath)
   #Wait 1 second to avoid overloading the server
    time.sleep(wait_time)

After executing the program with "cat" as an argument, the image of the cat could be obtained from Flickr.

4.png

Recommended Posts

How to scrape image data from flickr with python
How to crop an image with Python + OpenCV
[Python] How to read data from CIFAR-10 and CIFAR-100
How to scrape horse racing data with BeautifulSoup
[Python] How to FFT mp3 data
Python: How to use async with
Create folders from '01' to '12' with python
How to deal with imbalanced data
How to deal with imbalanced data
How to get started with Python
How to Data Augmentation with PyTorch
How to use FTP with Python
How to calculate date with python
How to access wikipedia from python
[Ev3dev] How to display bmp image on LCD with python
How to scrape at speed per second with Python Selenium
[Introduction to Python] How to get data with the listdir function
Receive textual data from mysql with python
[Note] Get data from PostgreSQL with Python
HTML email with image to send with python
How to update Google Sheets from Python
Introduction to Python Image Inflating Image inflating with ImageDataGenerator
Convert Excel data to JSON with python
Try to extract a character string from an image with Python3
How to work with BigQuery in Python
How to avoid duplication of data when inputting from Python to SQLite.
Introduction to Data Analysis with Python P17-P26 [ch02 1.usa.gov data from bit.ly]
Convert FX 1-minute data to 5-minute data with Python
How to scrape stock prices of individual stocks from the Nikkei newspaper website with Python
How to do portmanteau test with python
Image acquisition from camera with Python + OpenCV
How to connect to Cloud Firestore from Google Cloud Functions with python code
How to display python Japanese with lolipop
How to access RDS from Lambda (python)
[Python / Ruby] Understanding with code How to get data from online and write it to CSV
How to adjust image contrast in Python
How to scrape websites created with SPA
How to use "deque" for Python data
How to enter Japanese with Python curses
[Python] How to deal with module errors
How to read problem data with paiza
How to install python3 with docker centos
Sample to convert image to Wavelet with Python
[Data science basics] I tried saving from csv to mysql with python
How to crop the lower right part of the image with Python OpenCV
How to send a visualization image of data created in Python to Typetalk
How to deal with OAuth2 error when using Google APIs from Python
Data integration from Python app on Linux to Amazon Redshift with ODBC
Try to image the elevation data of the Geographical Survey Institute with Python
Data integration from Python app on Windows to Amazon Redshift with ODBC
Copy data from Amazon S3 to Google Cloud Storage with Python (boto)
How to access data with object ['key'] for your own Python class
How to upload with Heroku, Flask, Python, Git (4)
How to create sample CSV data with hypothesis
How to read a CSV file with Python 2/3
[REAPER] How to play with Reascript in Python
How to open a web browser from python
How to do multi-core parallel processing with python
Convert PDF to image (JPEG / PNG) with Python
Strategy on how to monetize with Python Java
Study from Python Hour7: How to use classes