python selenium chromedriver beautifulsoup

Sample code 1 (simply display)


import time
from selenium import webdriver
import chromedriver_binary

driver = webdriver.Chrome()

driver.get('https://xxx')
time.sleep(2)    #2 seconds weight

driver.close()
driver.quit()

Sample code 2 (headless mode: do not display & parse with beautiful soup)


from bs4 import BeautifulSoup

import time
from selenium import webdriver
import chromedriver_binary
from selenium.webdriver.chrome.options import Options

option = Options()
option.add_argument('--headless')
driver = webdriver.Chrome(options=option)

driver.get('https://xxx')
time.sleep(2)    #2 seconds weight

#When parsing
soup = BeautifulSoup(driver.page_source,'html.parser')


driver.close()
driver.quit()

Recommended Posts

python selenium chromedriver beautifulsoup
Selenium, Phantomjs & BeautifulSoup4
[Python / Selenium] XPath
python chromedriver automatic update
ScreenShot with Selenium (Python)
Scraping with Selenium [Python]
Python web scraping selenium
Studying Footprint Automation for Matching Apps (Python, Selenium, BeautifulSoup,)
Scraping with selenium in Python
Scraping with Selenium + Python Part 1
Python: Working with Firefox with selenium
Python Selenium Dynamic download wait
Start to Selenium using python
Selenium WebDriver + Firefox49 (provisional) (Python)
Web scraping using Selenium (Python)
Scraping with Selenium + Python Part 2
python beautifulsoup requests glob find_all
[Python + Selenium] Tips for scraping
Challenge Python3 and Selenium Webdriver
[Python scraping] I tried google search top10 using Beautifulsoup & selenium
Python
Scraping with Selenium in Python (Basic)
selenium case study summary python pyCharm
selenium
ElasticSearch + Kibana + Selenium + Python for SEO
Write selenium test code in python
Selenium and python to open google
Get html from element with Python selenium
WebUI test with Python2.6 + Selenium 2.44.0 --profile setting
Reboot the router using Python, Selenium, PhantomJS
[Python] Automatically operate the browser with Selenium
Practice web scraping with Python and Selenium
Overwrite download file for python selenium Chrome
Python + Selenium + Headless Chromium with aws lambda