[GO] Scraping tabelog with python and outputting to CSV

Introduction I read "Business Studies without Sense" by Yoshinori Fukushima of Gunosy.

It was written in an easy-to-understand manner about the importance of data analysis and machine learning. In addition, it describes how and how much effort Gunosy's targeting and user optimization is done.

I was away from programming for about a month on my internship, but I decided to study with the goal of creating a gourmet curation site.

Short-term goal: Create a simple curation site Ultimate goal: A curation site that goes beyond Gurunavi. Recommand function so that you can choose a store without spending time

Goal of this article: Scraping tabelog and outputting store name and URL by SCV

I will spell it as a memo and diary

What I did * Get HTML from URL * Extract specific information from HTML * How to extract by specifying tags and classes * Extract the elements you want using regular expressions

Code

first.py



import csv
import requests
from bs4 import BeautifulSoup
import re

urlName = "https://tabelog.com/osaka/A2701/A270103/rstLst/?SrtT=rt&Srt=D&sort_mode=1"
dataHTML = requests.get(urlName)

soup = BeautifulSoup(dataHTML.content, "html.parser")

elems = soup.select('a.list-rst__rst-name-target.cpy-rst-name')
#data = elems.text

with open(r'C:\Users\daisuke\Desktop\python\first.csv', 'w') as f:
    for i in elems:
        url = re.search(r'"http.*/"', str(i))
        url = re.sub(r'"', '',  str(url.group()))

        name = re.search(r'target="_blank">.*</a>', str(i))
        name = re.sub(r'target="_blank">', '',  str(name.group()))
        name = re.sub(r'</a>', '',  name)

        print(url)
        print(name)

        writer = csv.writer(f)
        writer.writerow([name,url])

What I want to do in the future I was able to scrape in about an hour. I wrote the code on the chord, but was scraping so easy?
  • Hello world with Django
  • Django stores scraping results in DB
  • Get information from DB-Display
  • Organize information obtained from each gourmet media to create a ranking of popular restaurants
  • For that purpose, machine learning is involved

Recommended Posts

Scraping tabelog with python and outputting to CSV
[Part1] Scraping with Python → Organize to csv!
Write to csv with Python
Scraping with Node, Ruby and Python
[Python] Write to csv file with Python
Scraping with Python, Selenium and Chromedriver
Output to csv file with Python
Scraping with Python and Beautiful Soup
Reading and writing CSV with Python
Scraping with Python
Scraping with Python
Fractal to make and play with Python
Read Python csv and export to txt
Practice web scraping with Python and Selenium
Easy web scraping with Python and Ruby
How to import CSV and TSV files into SQLite with Python
Precautions when inputting from CSV with Python and outputting to json to make it an exe
Scraping with Python (preparation)
How to read a CSV file with Python 2/3
Try scraping with Python.
MessagePack-Try to link Java and Python with RPC
Read JSON with Python and output as CSV
Scraping with Python + PhantomJS
Write CSV data to AWS-S3 with AWS-Lambda + Python
Scraping with Selenium [Python]
Read CSV file with Python and convert it to DataFrame as it is
Csv tinkering with python
Scraping with Python + PyQuery
Scraping RSS with Python
Example of reading and writing CSV with Python
WEB scraping with python and try to make a word cloud from reviews
Procedure to load MNIST with python and output to png
I want to handle optimization with python and cplex
I tried to touch the CSV file with Python
Try to operate DB with Python and visualize with d3
I was addicted to scraping with Selenium (+ Python) in 2020
How to convert JSON file to CSV file with Python Pandas
[Python] A memo to write CSV vertically with Pandas
[Python-pptx] Output PowerPoint font information to csv with python
Something to enjoy with Prim Pro (X-Play) and Python
Programming with Python and Tkinter
Connect to BigQuery with Python
Encryption and decryption with Python
Read csv with python pandas
Web scraping with python + JupyterLab
Scraping with selenium in Python
Python and hardware-Using RS232C with Python-
Scraping with Selenium + Python Part 1
Scraping with chromedriver in python
Festive scraping with Python, scrapy
Connect to Wikipedia with Python
Post to slack with Python 3
Scraping with Selenium in Python
Scraping with Tor in Python
Scraping weather forecast with python
Switch python to 2.7 with alternatives
Scraping with Selenium + Python Part 2
python with pyenv and venv
I tried scraping with python
Web scraping beginner with python
Download csv file with python