[PYTHON] [EC2] How to run selenium webdriver

[EC2] How to run selenium webdriver

Unlike local, you need to run chrome in a headless browser.

What is a headless browser?

-Process on the back side without launching a visible browser. ・ In the case of chrome, it is called headless chrome.

Description for EC2

python


from selenium import webdriver

#Import Options class (for headless settings)
from selenium.webdriver.chrome.options import Options

#Create an instance of Options (stored in the variable options)
options = Options()

#Set headless to True
options.headless = True

#Start webdriver
driver = webdriver.Chrome(options=options)

#Open the specified URL
driver.get("URL")

#Describe the process to be executed below
processing

The headless browser can be launched and the process can be executed.

Check the contents

Options class

Selenium.webdriver.chrome.options.Options └ Options class location └ Options class in options module in chrome module in webdriver module in selenium module

From selenium.webdriver.chrome.options import Options └ Import Options class └ Call after import is possible with Options

Reference link ・ Explanation of Options-Official Site About importing classes

It is interesting because you can see the contents of the method on the above official website.

.headless method

options.headless = True -Set the value of the argument "headless" to True. -The headless method itself returns whether headless is set as an argument.

Other headless setting methods

You can also set headless using the ** add_argument method **.

options.add_argument('--headless') └ Set "'--headless'" with the argument.

webdriver.Chrome(options=options) -A function that starts the webdriver. -Set startup options with arguments. -Default options = none is set for the created instance. (I'm not confident, please let me know if anyone is familiar with it)


## Supplement ### ① Extract and display the text on the upper left from the google top page.

python


from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.headless = True

driver = webdriver.Chrome(options=options)

driver.get("https://www.google.co.jp/")

element_text = driver.find_element_by_id("hptl").text
print(element_text)

"About Google Store" is displayed.

image.png

### (2) Description when starting locally ``` from selenium import webdriver driver = webdriver.Chrome('chromedriver.exe')

driver.get('url')


 ** ▼ Difference from headless **
 -There is no headless setting.
 -The argument when starting webdriver is chromedriver.exe
 -"Chromedriver.exe" must be in the same hierarchy.
 └ In case of another layer, specify the path as an argument.



<br>
 There is a difference from the execution method in the local environment, but you can use it without problems as long as you make the initial settings.


Recommended Posts

[EC2] How to run selenium webdriver
How to debug selenium
How to run Notepad ++ Python
How to run matplotlib on heroku
How to run TensorFlow 1.0 code in 2.0
How to make Selenium as light as possible
How to run Cython on OSX Memo
How to run a Maya Python script
[EC2] How to take a screen capture of your smartphone with selenium
How to start Python (Flask) when EC2 starts
[2020 version] How to install Python3 on AWS EC2
How to run some script regularly in Django
How to run CNN in 1 system notation in Tensorflow 2
How to run MeCab on Ubuntu 18.04 LTS Python
How to deal with SessionNotCreatedException when using Selenium
How to run Leap Motion in non-Apple Python
[EC2] How to deal with errors that selenium cannot execute (No module named selenium)
How to use xml.etree.ElementTree
[Python] How to run Jupyter-notebook + pandas + multiprocessing (Pool) [pandas] Memo
How to use Python-shell
How to use tf.data
How to use virtualenv
Scraping 2 How to scrape
How to run the Ansible module added in Ansible Tower
How to use Seaboan
How to use image-match
How to use shogun
How to run AutoGluon in Google Colab GPU environment
How to install Python
How to read PyPI
How to install pip
How to use Virtualenv
How to use numpy.vectorize
How to update easy_install
How to install archlinux
How to run python in virtual space (for MacOS)
How to use pytest_report_header
How to run tests in bulk with Python unittest
How to restart gunicorn
How to virtual host
How to use partial
How to use Bio.Phylo
How to read JSON
How to use SymPy
How to use x-means
How to use WikiExtractor.py
How to update Spyder
How to use IPython
How to install BayesOpt
How to use virtualenv
How to run Self bot on Discord.py [Easy vandalism! ]
How to use Matplotlib
How to run setUp only once in python unittest
How to download files from Selenium in Python in Chrome
How to use iptables
How to use numpy
How to use TokyoTechFes2015
How to use venv
How to use dictionary {}
How to use Pyenv
How to grow dotfiles