[PYTHON] Trial and error to speed up Android screen captures

Purpose

--I want to capture the screen of an Android device at high speed. ――To perform an automatic test of sound game, which is one genre of smartphone games these days.

problem

--Conventional method: Use screencap command from adb command --With the full HD screen, it has taken a long time from capture to transfer. --If you are not good at it, it takes about 2 seconds to pull the image from the capture. ~~ You can't test the sound game with this. ~~

trial and error

Android Screen Monitor --Although it was produced by adakoda, it was abandoned because development stopped four years ago and the screen size was so large that it was initially expected that the capture could not catch up.

Using HDMI Capture Device

-After seeing Announcement by Cygames of JaSST'17, I came up with the idea of acquiring images by HDMI Capture via MHL. However, ~~ I can't write in C # ~~ The HDMI Capture Device itself is expensive, and I'm worried about connecting multiple devices, so I abandoned this method.

solution

--CyberAgent has previously announced Smartphone Test Farm (OpenSTF) as a technology called minicap. It seems that I can use this somehow. ―― ~~ I can't write JavaScript, so ~~ Eventually, I will consider implementing it in Python in order to consider cooperation with OpenCV.

Well.

-I made it. --For the part that communicates with the minicap, the one created by the Chinese person was used as it is. - http://www.jianshu.com/p/f8b8123cd062

Implementation

-stream.py is the part that gets the data from the minicap process and becomes the data analysis part. --For the explanation of the data, Honke is the best, so I will omit it. --After acquiring the image data, the sample project minicap.py is processing the acquired image data. Is converted to OpenCV format.

Impressions

――It's nice that the screen capture is faster, but the operation of tapping and flicking the screen remains slow via adb, so I have to improve that.

reference

Recommended Posts

Trial and error to speed up Android screen captures
Trial and error to speed up heat map generation
[Python] Hit Keras from TensorFlow and TensorFlow from c ++ to speed up execution
Numba to speed up as Python
Project Euler 4 Attempt to speed up
How to speed up Python calculations
Trial and error to improve cgo memory profiling by go beginners
[DRF] Snippet to speed up PrimaryKeyRelatedField
How to speed up instantiation of BeautifulSoup
pix2 pix tensorflow2 Record of trial and error
How to speed up scikit-learn like conda Numpy
[Python] Do your best to speed up SQLAlchemy
A story of trial and error trying to create a dynamic user group in Slack