Introduction

This time, I tried to create scraping with python. I wanted to write it neatly to csv, but due to lack of technology, I just wrote it for the time being, so I want to make it easier to see as shown in the table. I created it in a fumbling state, so it works, but I would appreciate it if you could comment if there is something that should be changed.

environment

windows 10 python 3.8.0 Library urllib.request # Required to access the URL. csv #csv Required when working with files. BeautifulSoup #A module specializing in scraping required for parsing HTML and XML documents. datetime # Required to get the date.

I thought it would be easy to understand at a glance when it was picked up, so I added it this time.

code

`scraping.py`


import urllib.request
import csv
from bs4 import BeautifulSoup
import datetime

#URL of the site
url = "URL of the site you want to scrape"
#Open URL
html = urllib.request.urlopen("URL of the site you want to scrape")
#Open with Beautiful Soup
soup = BeautifulSoup(html, 'html.parser')

#Extraction of elements
site_title = soup.find_all("Elements you want to extract")
print(site_title)


#Include the date when creating the csv file
csv_date = datetime.datetime.today().strftime("%Y%m%d")

#file name
csv_file_name = "freedom" + csv_date + ".csv"

#open csv
csv_file = open(csv_file_name, "w")

#Write to csv
write_csv = csv.writer(csv_file, lineterminator='\n')
write_csv.writerows(site_title)

#close csv
f.close()

[PYTHON] web scraping (prototype)

Introduction

environment

code

scraping.py

`scraping.py`