[PYTHON] [Selenium] I want to display the browser by hitting the driver on the host OS from WSL

Introduction

WSL is very convenient in that you can use Linux commands while using Windows, but when it comes to Selenium, there is a drawback that it is difficult to check the behavior of the browser because the Ubuntu GUI cannot be used.

You can check the screen display at that point by launching the virtual display and taking a screenshot [^ 1], but it seems like there is nothing like seeing it actually moving.

As a result of various trials and errors, I succeeded in displaying the browser by hitting the driver on the host OS from WSL2 by using Selenium Grid, so I will share it.

environment

Host OS: Windows 10 Home 20H2 WSL: Ubuntu 18.04.5 on WSL2 Python: 3.7.5

Previous research

[Operating a web browser on the Windows side using Selenium from Python on WSL (Ubuntu16.04.4 LTS) – Laboradian](https://laboradian.com/operate-browser-on-win-by-python- of-wsl-via-selenium /) ⇒Since it was an article more than two years ago, I could not reproduce it in my environment. I get the error message Can not connect to the Service. It may be possible if the environment is prepared.

I wanted to operate the Chrome window on the Windows side from Selenium Webdriver or Puppeteer started on WSL2, but I can't-- Corredor ⇒Although it has not been solved yet, it may be helpful for those who are developing in languages other than Python because there are abundant references to previous research.

Method 1: Selenium3 (reliability: medium)

① Windows: Install Java (11 or higher).

See below for the procedure. If it is Java 10 or less, an error may occur. Installation method memo of OpenJDK11 --Qiita

② Windows: Download Selenium Server (Grid).

Go to the here page and download 3.141.59. The location of the downloaded file is not specified.

③ Windows: Download the browser driver.

Chrome Driver from here, geckodriver (Firefox) from here. Please download the driver with the same version ** as the browser installed on Windows **. Extract the downloaded zip and other files, and place the executable file (chromedriver.exe or geckodriver.exe) in the same folder ** as the jar file downloaded in ** ②.

④ Windows: Start Selenium Server.

In Selenium Grid 3, we need to set up two processes, Hub and Node, so we will launch two command prompts. First, start Hub by executing the following command at one command prompt.

java -jar [Storage location of jar file]\selenium-server-standalone-3.141.59.jar -role hub

If the following output is output and the process does not end, there is no problem.

13:11:40.423 INFO [GridLauncherV3.parse] - Selenium server version: 3.141.59, revision: e82be7d358
13:11:40.483 INFO [GridLauncherV3.lambda$buildLaunchers$5] - Launching Selenium Grid hub on port 4444
2020-11-23 13:11:40.848:INFO::main: Logging initialized @636ms to org.seleniumhq.jetty9.util.log.StdErrLog
13:11:41.814 INFO [Hub.start] - Selenium Grid hub is up and running
13:11:41.816 INFO [Hub.start] - Nodes should register to http://192.168.21.1:4444/grid/register/
13:11:41.816 INFO [Hub.start] - Clients should connect to http://192.168.21.1:4444/wd/hub

Then run the following command at the other command prompt to start Node. For [Hub URL], assign the URL that follows Clients should connect to in the above output.

java -jar [Storage location of jar file]\selenium-server-standalone-3.141.59.jar -role node -hub [Hub URL]:4444

If the following output is output and the process does not end, there is no problem.

13:11:59.609 INFO [GridLauncherV3.parse] - Selenium server version: 3.141.59, revision: e82be7d358
13:11:59.688 INFO [GridLauncherV3.lambda$buildLaunchers$7] - Launching a Selenium Grid node on port 31688
2020-11-23 13:12:00.999:INFO::main: Logging initialized @1598ms to org.seleniumhq.jetty9.util.log.StdErrLog
13:12:01.177 INFO [WebDriverServlet.<init>] - Initialising WebDriverServlet
13:12:01.231 INFO [SeleniumServer.boot] - Selenium Server is up and running on port 31688
13:12:01.231 INFO [GridLauncherV3.lambda$buildLaunchers$7] - Selenium Grid node is up and ready to register to the hub
13:12:01.654 INFO [SelfRegisteringRemote$1.run] - Starting auto registration thread. Will try to register every 5000 ms.
13:12:01.883 INFO [SelfRegisteringRemote.registerToHub] - Registering the node to the hub: http://192.168.21.1:4444/grid/register
13:12:02.317 INFO [SelfRegisteringRemote.registerToHub] - The node is registered to the hub and ready to use

⑤ WSL: Install Selenium 3.141.0.

pip install selenium==3.141.0

⑥ WSL: Create sample code.

sample.py

from selenium import webdriver

chrome_options = webdriver.ChromeOptions()
driver = webdriver.Remote(
    command_executor="http://192.168.21.1:4444/wd/hub",
    options=chrome_options
    )
driver.get("https://www.google.com/search?q=qiita")

For command_executor, specify the URL that followsClients should connect to. If you change ChromeOptions () to FirefoxOptions (), you will use geckodriver.

⑦ WSL: Execute the sample code.

python sample.py

If successful, it will look like this:

Method 2: Selenium4 (reliability: high)

To tell the truth, the above method with Selenium 3 did not work well from the beginning, and I found a loophole to succeed by playing with the Selenium source, but I thought it would be a good idea to introduce it, so I put it in storage once. I was there. After that, I discovered how to implement it in Selenium 4, and when I wrote this article, I verified it again with Selenium 3 and it worked. There is a good opinion that "then I should introduce the one with higher certainty first", but since Selenium 4 is still a pre-release, I thought that it would be better if the stable version of Selenium 3 works well. It depends on the priority. Excuse me for the long introduction.

① Windows: Install Java (11 or higher).

Omitted because it is the same as Selenium3. If it is not Java 11 or higher, an error will definitely occur.

② Windows: Download Selenium Server (Grid) 4.

Go to the here page and download from the link labeled Latest Selenium 4 Alpha version.

③ Windows: Download the browser driver.

Same as Selenium3. The difference is that the executable does not have to be in the same location as the jar file.

④ Windows: Add the driver path to the system environment variable PATH.

This procedure is required only for Selenium 4. Open the setting screen by selecting [Start] → [Settings] and enter "environment variables" in the search box. Two candidates, "Edit environment variables" and "Edit system environment variables", will appear. Select "Edit Variable".

Then the following screen will open, so press "Environment Variables". When the "Environment Variables" screen opens, select the Path row in the" System Variables "table and press" Edit ". When the "Edit Environment Variable Name" screen opens, click "New" and enter the path of the folder where the browser driver is stored. When the input is completed, confirm with the Enter key and press "OK". Click "OK" on the "Environment Variables" screen and "System Properties" screen, close the "Settings" screen, and you're done.

⑤ Windows: Start Selenium Server.

Selenium Grid 4 has a mode called standalone, which allows you to launch all components from one command prompt.

Start a command prompt and execute the following command. Please note that the file name depends on the downloaded version.

java -jar [Storage location of jar file]\selenium-server-4.0.0-alpha-7.jar standalone

If the last line of the command output is as follows and the process does not end, there is no problem.

17:08:44.674 INFO [Standalone.execute] - Started Selenium standalone 4.0.0-alpha-7 (revision Unknown): http://172.26.240.1:4444

⑥ WSL: Install Selenium4.

pip install selenium==4.0.0a7

If you have Selenium installed, you can upgrade with the following command.

pip install -U --pre selenium

If the Python version is 3.8 or higher, Selenium == 4.0.0a7 installation will fail. In that case, you will either have to lower the Python version or install Selenium == 4.0.0a6. ** Please note that if you lower the version of Selenium, you also need to lower the version of Selenium Server **.

⑦ WSL: Create sample code.

Same as Selenium3. The URL set in command_executor may be different between Selenium 3 and 4, so change it if necessary.

⑧ WSL: Execute the sample code.

python sample.py

If successful, it will look like this:

Afterword

I'm not familiar with Java at all, so when I ran the jar file and got an error, I was so confused that I almost cried, but as a result I was relieved that it worked. I think the points were ** Java version ** and ** system environment variable PATH **. We look forward to helping you.

Thank you for watching until the end.

References

Remote WebDriver client :: Documentation for Selenium Grid :: Documentation for Selenium

[^ 1]: The following page was helpful for this method. Python: Take a screenshot of the entire web page with Selenium + Headless Chrome-CUBE SUGAR CONTAINER