I tried to make a periodical process with CentOS7, Selenium, Python and Chrome

before I made a periodical execution process with Selenium and Python I did, but it stopped working before I knew it ... So, even if I build the environment again with the latest firefox and selenium, it doesn't work. (Need geckodriver? Implicitly_wait doesn't work? Centos6 doesn't work? Can't run in parallel? I've followed various steps, but I forgot, so I'll omit the details.)

So, I switched to CentOS 7 and Chrome and tried to build the environment.

environment

Build

Selenium

selenium installation


yum install python-pip
pip install selenium

Xvfb

Xvfb installation


yum install xorg-x11-server-Xvfb

Xvfb service creation


vim /usr/lib/systemd/system/Xvfb.service

/usr/lib/systemd/system/Xvfb.service


[Unit]
Description=Virtual Framebuffer X server for X Version 11

[Service]
Type=simple
EnvironmentFile=-/etc/sysconfig/Xvfb
ExecStart=/usr/bin/Xvfb $OPTION
ExecReload=/bin/kill -HUP ${MAINPID}

[Install]
WantedBy=multi-user.target

Create Xvfb environment variable configuration file


vim /etc/sysconfig/Xvfb

/etc/sysconfig/Xvfb


# Xvfb Enviroment File
OPTION=":1 -screen 0 1366x768x24"

Start Xvfb service


systemctl enable Xvfb
systemctl start Xvfb

Chrome

Create Chrome repository file


vim /etc/yum.repos.d/google-chrome.repo

bash:/etc/yum.repos.d/google-chrome.repo


[google-chrome]
name=google-chrome
baseurl=http://dl.google.com/linux/chrome/rpm/stable/$basearch
enabled=1
gpgcheck=1
gpgkey=https://dl-ssl.google.com/linux/linux_signing_key.pub

google-chrome-stable installation


yum install google-chrome-stable

For Japanese display

Japanese font installation


yum install ipa-pgothic-fonts.noarch

ChromeDriver - WebDriver for Chrome : Downloads Download the appropriate driver from. This time version 2.29.

Chrome Driver installation


wget http://chromedriver.storage.googleapis.com/2.29/chromedriver_linux64.zip
unzip chromedriver_linux64.zip
mv chromedriver /usr/local/bin
rm chromedriver_linux64.zip

I tried to incorporate the above construction contents into the configuration management tool "itamae". If you like, please.  => https://github.com/kotanbo/itamae-selenium-centos7 (The pattern of CentOS6 is described in some (Xvfb), but I gave up on building with CentOS6 ...)

Implementation

ChromeDriver - WebDriver for Chrome : Getting started The following is a sample with reference to.

python sample


# -*- coding: utf-8 -*-
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import Select
from selenium.common.exceptions import NoSuchElementException
from selenium.common.exceptions import NoAlertPresentException
import os, time, datetime, re, util, sys
import selenium.webdriver.chrome.service as service

class Sample():
	def __init__(self):
		self.aservice = service.Service('/usr/local/bin/chromedriver')
		self.aservice.start()
		capabilities = {'chrome.binary': '/usr/bin/google-chrome-stable', "chromeOptions": {"args": ['--no-sandbox']}}
		self.driver = webdriver.Remote(self.aservice.service_url, capabilities)
		self.driver.implicitly_wait(3)
		self.base_url = "http://example.jp/"

	def __del__(self):
		self.driver.close()

	def sample(self):
		driver = self.driver

		try:
			driver.get(self.base_url + "/home")
			driver.find_element_by_xpath("//a[@id='LoginButton']/img").click()
		except:
		finally:

sample = Sample()
sample.sample()

cron sample


0 9 * * * export DISPLAY=localhost:1.0; python /root/selenium/sample.py
0 9 * * * export DISPLAY=localhost:1.0; python /root/selenium/sample.py

0 4 * * * pkill -KILL -f chromedriver

point

reference

Start Xvfb as a daemon with systemd Install Chrome on CentOS 7

Recommended Posts

I tried to make a periodical process with CentOS7, Selenium, Python and Chrome
I tried to make a periodical process with Selenium and Python
I tried to make GUI tic-tac-toe with Python and Tkinter
[2nd] I tried to make a certain authenticator-like tool with python
[3rd] I tried to make a certain authenticator-like tool with python
I tried to make a 2channel post notification application with Python
I tried to make a todo application using bottle with python
[4th] I tried to make a certain authenticator-like tool with python
[1st] I tried to make a certain authenticator-like tool with python
I want to make a game with Python
Python: I tried to make a flat / flat_map just right with a generator
I tried to make a traffic light-like with Raspberry Pi 4 (Python edition)
I tried to draw a route map with Python
I tried to automatically generate a password with Python3
I tried to make a simple mail sending application with tkinter of Python
[Patent analysis] I tried to make a patent map with Python without spending money
[ES Lab] I tried to develop a WEB application with Python and Flask ②
I tried to easily create a fully automatic attendance system with Selenium + Python
I tried to make a simple image recognition API with Fast API and Tensorflow
I tried to make various "dummy data" with Python faker
I tried to make a stopwatch using tkinter in python
I tried to make a real-time sound source separation mock with Python machine learning
I tried to automate the article update of Livedoor blog with Python and selenium.
Rubyist tried to make a simple API with Python + bottle + MySQL
[1 hour challenge] I tried to make a fortune-telling site that is too suitable with Python
A memorandum when I tried to get it automatically with selenium
I tried to make a regular expression of "amount" using Python
[Python] I tried to implement stable sorting, so make a note
I tried using Selenium with Headless chrome
I tried to make a regular expression of "time" using Python
[Python] A memo that I tried to get started with asyncio
Fractal to make and play with Python
How to make a surveillance camera (Security Camera) with Opencv and Python
A memo with Python2.7 and Python3 on CentOS
I tried to create a list of prime numbers with python
I tried to make a regular expression of "date" using Python
I made a server with Python socket and ssl and tried to access it from a browser
I tried a functional language with Python
I tried to make a generator that generates a C # container class from CSV with Python
[Introduction] I want to make a Mastodon Bot with Python! 【Beginners】
I tried to create Bulls and Cows with a shell program
I also tried to imitate the function monad and State monad with a generator in Python
I tried to easily detect facial landmarks with python and dlib
I tried to make a Web API
I tried to make a strange quote for Jojo with LSTM
I want to make a voice changer using Python and SPTK with reference to a famous site
I tried to make an image similarity function with Python + OpenCV
I tried to make a mechanism of exclusive control with Go
I tried to find out the difference between A + = B and A = A + B in Python, so make a note
I tried to communicate with a remote server by Socket communication with Python.
I made a tool to automatically browse multiple sites with Selenium (Python)
I tried to create a program to convert hexadecimal numbers to decimal numbers with python
I tried to make a calculator with Tkinter so I will write it
I tried to make "Sakurai-san" a LINE BOT with API Gateway + Lambda
[AWS] [GCP] I tried to make cloud services easy to use with Python
I tried to discriminate a 6-digit number with a number discrimination application made with python
[Outlook] I tried to automatically create a daily report email with Python
I tried to build a Mac Python development environment with pythonz + direnv
I tried to create a sample to access Salesforce using Python and Bottle
[Zaif] I tried to make it easy to trade virtual currencies with Python
I tried to make a url shortening service serverless with AWS CDK