[PYTHON] What happens to the access log when the website is automatically accessed by selenium-webdriver

Purpose

Actually, I was secretly automating a part of the internal work, but when I looked into the server log, I thought that it would be possible to automate it, so I tried to verify it.

Verification environment

• Use PaizaCloud as a web server
• Use Selenium-webdriver and headless-chrome as an automatic access method
• Use Python as a program to automate

What i did

Introducing Selenium and Headless Chrome

There are many articles about this, so I will omit it.

Set up a web server

Go to PaizaCloud and log in. You can log in with your Google account, etc., and some plans can be used free of charge. This time I am using this free plan.

On the server creation screen, select an appropriate server name and application to install. This time I have Apache installed.

After creating the server, click the browser button on the left to launch the internal browser.

Of course, you can copy and paste the URL and use another browser. For the time being, I was able to confirm that the server was up.

Check the location of the server log

The Apache server log is located at /var/log/apache2/access.log by default. Click the console button on the left to launch the console.

Hit the following command to check the access log.

sudo tail /var/log/apache2/access.log

This time, there was no access log yet.

Write code in python

main.py

#Module import
from selenium import webdriver

#Launch Chrome
options = webdriver.ChromeOptions()
driver = webdriver.Chrome(options=options)

#Make a request to the web server
driver.get('https://bowtin-testserver.paiza-user.cloud/')

When you run the above program, Google Chrome should automatically launch and access the web page.

Check the access log

Check the access log on the server side again.

It says Chrome, and at this point I don't know if it's manual or automatic.

Try changing the code so that the Chrome screen does not start

Add some code to get Chrome to start in the background. With this, Chrome is not displayed on the screen, so it feels like it is automated.

main.py

#Module import
from selenium import webdriver

#Launch Chrome
options = webdriver.ChromeOptions()
options.add_argument('--headless') #add to
options.add_argument('--disable-gpu') #add to
driver = webdriver.Chrome(options=options)

#Make a request to the web server
driver.get('https://bowtin-testserver.paiza-user.cloud/')

After modifying the code, run it again.

Check the server log again

This time, I was able to confirm that the browser type is Headless Chrome.

Conclusion

• If you know it, you can understand it.