Operate your browser using the Selenium Web Driver Python bindings

This article is the 11th day article of Selenium / Appium Advent Calendar 2014. Yesterday's article was @ totutiteta's Robot drives on the web: Robot Framework & Selenium2Library.

Today I've summarized how to work with Firefox using Selenium's Python binding library selenium. I also made a library to operate using selenium.

Try working with Firefox with Selenium's Python binding library selenium

At the time of writing this article, the latest version of selenium is 2.44.0. (Probably the following code will not work with versions less than 2.35)

Start-up

First, try starting the browser. This time, we are targeting Firefox.

Create profile

from selenium.webdriver import FirefoxProfile

default_profile = {
    'security.warn_entering_secure': False,
    'security.warn_entering_secure.show_once': True,
    'security.warn_entering_weak': False,
    'security.warn_entering_weak._show_once': True,
    'security.warn_leaving_secure': False,
    'security.warn_leaving_secure.show_once': True,
    'security.warn_leaving_weak': False,
    'security.warn_leaving_weak._show_once': True,
    'security.warn_submit_insecure': False,
    'security.warn_viewing_mixed': False,
    'security.warn_viewing_mixed.show_once': True,
    }
profile = FirefoxProfile()
for name, value in default_profile.items():
    profile.set_preference(name, value)

Creating Proxy settings

The Proxy object is set with the ftp_proxy, ssl_proxy, http_proxy attributes. None seems to be good when not in use.

from selenium.webdriver import Proxy
proxy = Proxy()
proxy.ftp_proxy = proxy.ssl_proxy = proxy.http_proxy = None

Firefox launch

from selenium.webdriver import Firefox
browser = Firefox(firefox_profile=profile, proxy=proxy)
browser.implicitly_wait = 10  #Set the waiting time until page is loaded
browser.delete_allcookies()  #Erase all cookies

operation

Now, let's operate the launched browser.

Try to transition the page by specifying the URL

url = 'http://qiita.com/'
browser.get(url)

Get element by specifying id attribute

tag = browser.find_element_by_id('elemet-id')

Get a list of elements by specifying the class attribute

tags = browser.find_elements_by_class_name('class-name')

Get a list of elements by specifying a CSS selector

Below you will get a list of input tags.

tags = browser.find_elements_by_css_selector('input')

Click an element

tag.click()

Submit with the submit button on the form

form = browser.find_elements_by_tag_name('form')[0]
for tag in form.find_elements_by_tag_name('input'):
    type_ = tag.get_attribute('type')
    if type_ == 'submit':
        entry.submit()

Enter characters in the text box

textboxes = [tag for tag in browser.find_elements_by_tag_name('input') if tag.get_attribute('type') == 'text']
textbox = textboxes[0]

textbox.send_keys('CHARACTOR')

You can erase the characters by sending the BackSpace key code

from selenium.webdriver.common.keys import Keys
textbox.send_keys(Keys.BACK_SPACE)  #Delete one character
textbox.send_keys(Keys.BACK_SPACE * 10)  #Delete 10 characters

Maximize Window

browser.maximize_window()

Check if the element is visible

if tag.is_displayed():
    print(u'Is displayed')
else:
    print(u'Not displayed')

Execute javascript

Below, the screen is scrolled down.

browser.execute_script('window.scrollTo(0, 1000)')

I made a library to operate using selenium

You can do various operations, but in order to actually use the above, you will often want to make various transitions. In some cases, you may want to keep what you are doing from what page you are currently on, the state of the page, or the state of the previous page. (I actually encountered such a situation)

So, I made a library that can be used like a framework.

pywad

pywad is a library that allows you to describe startup and control in a standard way.

Installation

Since Selenium is used, Selenium must be installed.

$ pip install pywad

How to use

pywad has a Runner class and a Part class. The Part class is a class for describing the control of small control units such as one page, and operates as a part of the Runner class.

Part class

The Part class has an is_target () method and a run () method. is_target () is a method that determines whether it is a controlled object, and if this method returns True, the run () method will be executed. When actually using it, it will be used by overriding is_target () and run (). Both methods are passed a browser object and an object to hold the status, so use it to define the process.

from pywad.part import Part
from pywad.decorator import url_match

class GoogleTop(Part):
    def _is_search_button(self, text):
        for word in self.search_words:
            if word in text:
                return True

    @url_match('www\.google\.')
    def is_target(self, browser, status):
        return True

    def run(self, browser, status):
        entries = browser.find_elements_by_css_selector('input')
        for entry in entries:
            if entry.get_attribute('type') == 'text':
                entry.send_keys('test\n\n')

Runner class

The Runner class manages data that needs to be spanned between browsers and parts. By registering the Part class in the Runner class, the target Part is searched for and executed.

from pywad.runner import Runner
def main():
    url = 'http://www.google.com'
    runner = Runner()
    runner.append(GoogleTop())
    runner.run(url)

You can write it like this.

#! /usr/bin/env python
# -*- coding: utf-8 -*-
from pywad.runner import Runner
from pywad.part import Part
from pywad.decorator import url_match

class GoogleTop(Part):
    def _is_search_button(self, text):
        for word in self.search_words:
            if word in text:
                return True

    @url_match('www\.google\.')
    def is_target(self, browser, status):
        return True

    def run(self, browser, status):
        entries = browser.find_elements_by_css_selector('input')
        for entry in entries:
            if entry.get_attribute('type') == 'text':
                entry.send_keys('test\n\n')

def main():
    url = 'http://www.google.com'
    runner = Runner()
    runner.append(GoogleTop())
    runner.run(url)

if __name__ == '__main__':
    main()

Please use it if you like

tomorrow?

Who will be tomorrow? No one has registered .oO (I want someone to write)

Tomorrow is ***. I can't say that, so I'll put a link to the Advent calendar. http://qiita.com/advent-calendar/2014/selenium

Then ヾ (・ д ・) Byё ヾ (・ д ・) Byё

Recommended Posts

Operate your browser using the Selenium Web Driver Python bindings
[Python] Automatically operate the browser with Selenium
Web scraping using Selenium (Python)
Reboot the router using Python, Selenium, PhantomJS
I tried web scraping using python and selenium
Try using the Python web framework Tornado Part 1
Operate the schedule app using python from iphone
Try using the Python web framework Tornado Part 2
Operate Redmine using Python Redmine
Python web scraping selenium
HTTP server and HTTP client using Socket (+ web browser) --Python3
Try using the Python web framework Django (2) --Look at setting.py
Start to Selenium using python
Turn your Android Smart Phone into a Web Server using python.
Control the motor with a motor driver using python on Raspberry Pi 3!
Operate Maya from an external Python interpreter using the rpyc module
Get and set the value of the dropdown menu using Python and Selenium
Operate Firefox with Selenium from python and save the screen capture
Visualize your pocket money files with the Python web framework Dash
Python in the browser: Brython's recommendation
Hit the web API in Python
Extract the targz file using python
Try using the Python Cmd module
Get a capture of the entire web page in Selenium Python VBA
The LXC Web Panel that can operate LXC with a browser was wonderful
When Selenium tells me that the Chrome driver version is different (Python)
[EV3 x Python] Stream the camera image to your PC using mjpg-streamer.
[Python] Warning came out when I raised from Selenium 3 to 4 [Web Driver]
Try using the Python web framework Django (1)-From installation to server startup
I tried using "Streamlit" which can do the Web only with Python
[Python] How to set the (client) window size inside the browser with Selenium
Try using the web application framework Flask
Try using the Kraken API in Python
Behind the flyer: Using Docker with Python
Port forwarding your web server using iptables
[Beginner] Python web scraping using Google Colaboratory
Tweet using the Twitter API in Python
Working with OpenStack using the Python SDK
Download files on the web with Python
Try to operate Excel using Python (Xlwings)
Automatically manipulate web pages using selenium webdriver
Practice web scraping with Python and Selenium
python setup.py test the code using multiprocess
Environment construction procedure to operate chrome without installing python on Windows (using selenium)
Try out the touch of data-driven testing with Selenium Python Bindings and py.test