Easy way to scrape with python using Google Colab

Purpose

Learn the basics of scraping

Usage

――If you want to try scraping --If you want to experience beautiful soup

Things to prepare

Code design

  1. Access the site and get the data
  2. Shape
  3. Print to print

References

--Udemy course https://www.udemy.com/course/pythondjango-a/learn/lecture/8650684

Source code

test_scraping.py


from bs4 import BeautifulSoup
import requests
res = requests.get('https://www.ikuji-kaji-yaruman.work/')
soup = BeautifulSoup(res.text, 'html.parser')
for h2 in soup.find_all('h2'):
  print(h2.text)

important point

--Please do not repeatedly attack my site in the source code lol

Impressions

--Colab is very convenient because you don't need to build a python environment!

Recommended Posts

Easy way to scrape with python using Google Colab
Easy way to customize Python import
Easy way to analyze AGN time fluctuations with google Colab with RXTE satellite PCA detector
Easy way to round off to the nearest whole number with python3
Try using Python with Google Cloud Functions
Easy way to use Wikipedia in Python
The strongest way to use MeCab and CaboCha with Google Colab
Upload images to Google Drive with Python
How to deal with OAuth2 error when using Google APIs from Python
[Python] Easy introduction to machine learning with python (SVM)
Easy way to use Python 2.7 on Cent OS 6
The easiest way to synthesize speech with python
Easy way to plot HR diagram using astroquery
The easiest way to use OpenCV with python
Introduction to Python with Atom (on the way)
Upload files to Google Drive with Lambda (Python)
An easy way to call Java from Python
Easy way to check the source of Python modules
How to scrape image data from flickr with python
[Python] Easy Google Translate app using Eel and Googletrans
An easy way to create an import module with jupyter
How to analyze with Google Colaboratory using Kaggle API
I tried to access Google Spread Sheets using Python
A real way for people using python 3.8.0-2 from windows to work with multibyte characters
An easy way to pad the number with zeros depending on the number of digits [Python]
Connect to BigQuery with Python
[S3] CRUD with S3 using Python [Python]
[Co-occurrence analysis] Easy co-occurrence analysis with Python! [Python]
Using Quaternion with Python ~ numpy-quaternion ~
Post to slack with Python 3
Study Python with Google Colaboratory
About learning with google colab
Easy folder synchronization with Python
Post to Twitter using Python
Easy to make with syntax
Start to Selenium using python
[Python] Another way to import
Switch python to 2.7 with alternatives
Write to csv with Python
Easy Python compilation with NUITKA-Utilities
Easy way to rename files
Easy HTTP server with Python
Access Google Drive with Python
Using OpenCV with Python @Mac
Send using Python with Gmail
Easy to use Nifty Cloud API with botocore and python
How to extract any appointment in Google Calendar with Python
Code snippets often used when using BigQuery with Google Colab
Probably the easiest way to create a pdf with Python3
Post a message to Google Hangouts Chat with a thread (Python)
[Python] An easy way to visualize energy data interactively [plotly.express]
[Introduction] Artificial satellite data analysis using Python (Google Colab environment)
How to scrape at speed per second with Python Selenium
Things to keep in mind when using Python with AtCoder
Save images on the web to Drive with Python (Colab)
Things to keep in mind when using cgi with python.
How to update FC2 blog etc. using XMLRPC with python
PyArmor ~ Easy way to encrypt and deliver python source code ~
Try to implement linear regression using Pytorch with Google Colaboratory
Play with YouTube Data API v3 using Google API Python Client
Complement python with emacs using company-jedi