Summary of tools for operating Windows GUI with Python

There are famous places such as Selenium on the Web, but it seems that there is no de facto tool for operating the GUI of Windows yet.

In reality, there are many options, so I will summarize them here.

A tool for manipulating the Windows GUI with Python


ahk · PyPI

AutoHotkey wrapper for Python.

Below is a sample from the above PyPI.

from ahk import AHK
from ahk.window import Window

ahk = AHK()

win = ahk.active_window  # Get the active window
win = ahk.win_get(title='Untitled - Notepad')  # by title
win = list(  # list of all windows
win = Window(ahk, ahk_id='0xabc123')  # by ahk_id
win = Window.from_mouse_position(ahk)  # the window under the mouse cursor
win = Window.from_pid('20366')  # by process ID

You can operate by specifying the target with the window title or id.


PyAutoIt · PyPI

This is a wrapper for AutoIt for Python.

You can operate it in a similar way to AutoHotKey. (* AutoHotKey itself is a tool originally separated from AutoIt)


autoit · PyPI

To be honest, I'm not sure if it's related to AutoIt, but it seems that I'm doing my best to implement it on my own.


RaiMan's SikuliX

I think SikuliX is rather famous. It comes with an IDE and you can write code by Sikuli alone, but there is also a pattern that you can load and use as a Python module.


moses-palmer/pynput: Sends virtual input commands

It is a tool that can automate mouse operation and keyboard operation, and it seems that it is not possible to acquire Elements on the screen.


asweigart/pyautogui: A cross-platform GUI automation Python module for human beings. Used to programmatically control the mouse & keyboard.

Mouse operation and keyboard operation can be automated here as well, but multi-platform = Windows, Mac, Linux can be operated.


Pywinauto - pywinauto

Compared to other modules of the same type, it has a lively impression due to the large number of commits.

The official sample is below.

from pywinauto import Desktop, Application

Application().start('explorer.exe "C:\\Program Files"')

# connect to another process spawned by explorer.exe
# Note: make sure the script is running as Administrator!
app = Application(backend="uia").connect(path="explorer.exe", title="Program Files")

common_files = app.ProgramFiles.ItemsView.get_item('Common Files')

# this dialog is open in another process (Desktop object doesn't rely on any process id)
Properties = Desktop(backend='uia').Common_Files_Properties
Properties.wait_not('visible') # make sure the dialog is closed

The parts such as ProgramFiles and ItemsView are highly readable and feel good.


microsoft/WinAppDriver: Windows Application Driver

It is a tool made by Microsoft that can operate the GUI of Windows like Selenium.

Even if you look at the sample code, you can see the "Selenium-likeness". It is the operation of the Windows calculator.

def test_combination(self):
        self.driver.find_element_by_name("Multiply by").click()
        self.driver.find_element_by_name("Divide by").click()

However, the operation in the Japanese environment is not so good, and it is difficult to solve it. ..


Maybe it's not good enough to see Pat ...? I thought, but I listed it just in case

Recommended Posts

Summary of tools for operating Windows GUI with Python
[For beginners] Summary of standard input in Python (with explanation)
Summary of various for statements in Python
Summary of useful techniques for Python Scrapy
GUI automation with Python x Windows App Driver
Django with Python Tools 2.2 for Visual Studio (PTVS 2.2)
Image Processing with Python Environment Setup for Windows
[Python] Summary of S3 file operations with boto3
Summary of frequently used Python arrays (for myself)
Python starting with Windows 7
Refactoring tools for Python
Summary of Python arguments
Install Python (for Windows)
Summary of tools needed to analyze data in Python
Summary of pre-processing practices for Python beginners (Pandas dataframe)
U ^ 2-Net for detecting objects of interest (Windows10, Python3.6)
PIL with Python on Windows 8 (for Google App Engine)
Creating BINGO "Web Tools" with Python (Table of Contents)
Summary of the basic flow of machine learning with Python
[Windows] [Python] Camera calibration of fisheye lens with OpenCV
Precautions when operating with string for TmeStampType of PySpark
Python with VS Code (Windows 10)
Run python with PyCharm (Windows)
Summary of python file operations
Summary of Python3 list operations
Python 3.6 installation procedure [for Windows]
[GUI with Python] PyQt5-Layout management-
[GUI with Python] PyQt5 -Preparation-
Time synchronization (Windows) with Python
Install dlib for Python (Windows)
[GUI with Python] PyQt5 -Paint-
[Python] GUI for inserting TeX format into PowerPoint with 2 clicks [PowerPoint]
Simulation of late damages for child support delinquency with python
Summary of Hash (Dictionary) operation support for Ruby and Python
Build Python3 for Windows 10 on ARM with Visual Studio 2019 (x86) on Windows 10 on ARM
Basic summary of data manipulation with Python Pandas-First half: Data creation & manipulation
Turn an array of strings with a for statement (Python3)
Create a Python execution environment for Windows with VScode + Remote WSL
Face detection with YOLO Face (Windows10, Python3.6)
A brief summary of Graphviz in python (explained only for mac)
[GUI with Python] PyQt5 -Widget II-
Let's make a GUI with python.
Building a Windows 7 environment for getting started with machine learning with Python
Be careful of LANG for UnicodeEncodeError when printing Japanese with Python 3
A brief summary of Python collections
Getting Started with Python for PHPer-Classes
[GUI with Python] PyQt5-The first step-
Getting Started with Python Basics of Python
I made a lot of files for RDP connection with Python
The story of making a standard driver for db with python.
[GUI with Python] PyQt5-Drag and drop-
Life game with Python! (Conway's Game of Life)
Build mlpy with python3.3 (64bit) (windows 64bit)
10 functions of "language with battery" python
Installation of Python, SciPy, matplotlib (Windows)
[Let's play with Python] Aiming for automatic sentence generation ~ Completion of automatic sentence generation ~
Use Windows 10 speech synthesis with Python
4th night of loop with for
Implementation of Dijkstra's algorithm with python
[GUI with Python] PyQt5 -Custom Widget-
Introductory table of contents for python3