I summarized Selenium when scraping with Python. Scraping is pretty fun. I have used a wide range of other uses, such as shopping automation and data collection.
Windows10 Python Selenium Dedicated browser driver you want to use (eg Chrome Driver)
If you haven't built a Python environment yet, click here. How to build an environment for Python is posted. (Windows version)
If you have already built a Python environment, please install Selenium.
pip install selenium
You will need a driver that matches the browser you are using. (Example: Chrome Driver)
Open your browser using Selenium.
from selenium import webdriver
driver = webdriver.Chrome() #()Driver in_describe path
driver.get(URL)
driver.close()
driver.quit()
First, open the browser on the first line Specify the URL you want to open in the second line The third line closes the page Line 4 closes the browser
driver_path is ok without description when the driver is in the same directory.
Next is the acquisition of the elements required to operate the page. I often use these two.
id
get_id = driver.find_element_by_id ("write id in this ")
It is recommended because the id will surely be obtained.
xpath
get_xpath = driver.find_element_by_xpath ("write xpath in this ")
Since it often changes when the screen changes, I rewrite the xpath each time.
And if you can't get it because it's not on HTML, but you want that value, it's recommended to get it from js.
price = driver.execute_script ("describe js processing")
By writing as above, you can get the value that is not on the screen but is processed on the js side.
There are times when js doesn't have it either, so I have no choice but to give up at that time: sweat:
When you want to click a button on a form, you can click it with click ().
driver.find_elemnt_by_id ('write id in this'). Click ()
You can click the button element in the above process.
Often, there are times when an error occurs because the element cannot be removed. In that case, use sleep () and wait for the screen to appear.
There are many other ways to use selenium, but I wrote this article to let you know what you can do. If you want to know more details, please see articles written by other people and increase your knowledge. And let's automate everything!
https://kurozumi.github.io/selenium-python/installation.html https://qiita.com/nadechin/items/0a34e2182132cc1a821b
Recommended Posts