[PYTHON] GUI programming with kivy ~ Part 3 Video and seek bar ~

Introduction

In Previous article, I wrote a lot about the progress bar. In this article, I will try to make something like a video player by myself. (No sound ...) Originally, Video Player provided by kivy to visualize the results of video analysis (object detection, etc.) /api-kivy.uix.videoplayer.html), but I tried to make a customizable player. As the title says, I had a hard time implementing the seek bar for the video player, so I'm writing this article for information sharing.

Environment

As mentioned at the beginning, the purpose is to visualize the video analysis results, so install the library opencv for video (image) analysis.

My operating environment is as follows.

OSX 10.14.6 Python 3.7.2

If you are using pip, you can install it with just the following command.

pip install python-opencv

What is a seek bar

Quoted from wiki I will.

The seek bar is one of the functions provided in music / video playback software, etc., and is a function that displays the playback location of data. You can visually grasp how far you are playing music or movie from the beginning to the end by the position of the "slider", you can move the slider directly with the mouse, and you can start playing from any place. There are advantages such as.

Seek bar is a kind of slider. So what is a slider? [wiki](https://ja.wikipedia.org/wiki/%E3%82%A6%E3%82%A3%E3%82%B8%E3%82%A7%E3%83%83%E3%83 Quoted from% 88_ (GUI) #% E9% 81% B8% E6% 8A% 9E).

Slider — Similar to a scrollbar, but a widget used to set some value, not for scrolling.

kivy has a Slider widget, so you can make it a seek bar.

How to use kivy.uix.slider

I feel like this. シークバー.gif

The source is as follows.


from kivy.app import App
from kivy.lang import Builder
from kivy.uix.boxlayout import BoxLayout
from kivy.clock import Clock

Builder.load_string('''
<MySlider>
    orientation: 'vertical'
    Label : 
        text: "{}".format(slider.value)
        font_size: slider.value
        
    Slider: 
        id: slider
        step: 1
        min: 200
        max: 500
        
    Button: 
        text: "press"
        on_press: root.move_slider_start()
        
''')

class MySlider(BoxLayout):
    def __init__(self, **kwargs):
        super(MySlider, self).__init__(**kwargs)

    def move_slider_start(self):
        Clock.schedule_interval(self.move_slider, 1 / 60)

    def move_slider(self, dt):
        slider = self.ids['slider']
        if slider.max > slider.value:
            slider.value += 1
        else:
            return False
        
class sliderTest(App):

    def build(self):
        return MySlider()

sliderTest().run()

The source is that the value of the slider is linked to the text and font of the label at the top of the screen, and the label changes when the slider is moved. I also added a process to continuously update the slider value with the clock when the button below is pressed (I will do something similar with the video player later).

In the kv language, the Label text is updated using the Slider id.

    Label : 
        text: "{}".format(slider.value)
        font_size: slider.value

    Slider: 
        id: slider #id. It can be called in the kv language or Python.
        step: 1 #The minimum value when moving the slider. If you do not set it, you will get a messed up float value
        min: 200 #Minimum value of slider
        max: 500 #Maximum value of slider

In addition, in the process of automatically moving the slider, select the widget from "ids" where the id assigned to the widget is stored as an associative array and change the parameters as shown below. As in the previous article, it cannot be operated with the for statement, so I changed the slider value with Clock and updated the screen drawing.


    def move_slider(self, dt):
        slider = self.ids['slider'] #here!
        if slider.max > slider.value:
            slider.value += 1
        else:
            return False

What you are trying to make

The image I am trying to make this time is as shown in the figure below.

動画プレイヤー.png

It's like a video player with only the minimum features. All you have to do is load the video, play it, specify an arbitrary playback location, and check the current number of frames.

The video is like handling with opencv.

Source

VideoApp.py


from kivy.app import App
from kivy.lang import Builder
from kivy.uix.boxlayout import BoxLayout
from kivy.graphics.texture import Texture
from kivy.properties import ObjectProperty
from kivy.clock import Clock

from kivy.uix.floatlayout import FloatLayout
from kivy.uix.popup import Popup

import cv2

Builder.load_file('VideoApp.kv')

#Video selection pop-up
class LoadDialog(FloatLayout):
    load = ObjectProperty(None)
    cancel = ObjectProperty(None)

class MyVideoPlayer(BoxLayout):
    image_texture = ObjectProperty(None)
    image_capture = ObjectProperty(None)

    def __init__(self, **kwargs):
        super(MyVideoPlayer, self).__init__(**kwargs)
        self.flagPlay = False #Is the video playing?
        self.now_frame = 0 #Variable to check the playback frame of the video for seek bar
        self.image_index = [] #Array to store opencv images for seek bar

    #Pop-up to load video
    def fileSelect(self):
        content = LoadDialog(load = self.load, cancel = self.dismiss_popup)
        self._popup = Popup( title="File Select", content=content, size_hint=(0.9,0.9))
        self._popup.open()

    #Loading video files
    def load (self, path, filename):
        txtFName = self.ids['txtFName']
        txtFName.text = filename[0]
        self.image_capture = cv2.VideoCapture(txtFName.text)
        self.sliderSetting()
        self.dismiss_popup()

    #Close pop-up
    def dismiss_popup(self):
        self._popup.dismiss()

    #Seek bar settings
    def sliderSetting(self):
        count = self.image_capture.get(cv2.CAP_PROP_FRAME_COUNT)
        self.ids["timeSlider"].max = count

        #Load the video once and save all the frames in an array
        while True:
            ret, frame = self.image_capture.read()
            if ret:
                self.image_index.append(frame)

            else:
                self.image_capture.set(cv2.CAP_PROP_POS_FRAMES, 0)
                break

    #Video playback
    def play(self):
        self.flagPlay = not self.flagPlay
        if self.flagPlay == True:
            self.image_capture.set(cv2.CAP_PROP_POS_FRAMES, self.now_frame)
            Clock.schedule_interval(self.update, 1.0 / self.image_capture.get(cv2.CAP_PROP_FPS))
        else:
            Clock.unschedule(self.update)

    #Video playback clock processing
    def update(self, dt):
        ret, frame = self.image_capture.read()
        #When the next frame can be read
        if ret:
            self.update_image(frame)
            time = self.image_capture.get(cv2.CAP_PROP_POS_FRAMES)
            self.ids["timeSlider"].value = time
            self.now_frame = int(time)

    #Seek bar
    def siderTouchMove(self):
        Clock.schedule_interval(self.sliderUpdate, 0)

    #Screen drawing process when the seek bar is moved
    def sliderUpdate(self, dt):
        #When the seek bar value and the playback frame value are different
        if self.now_frame != int(self.ids["timeSlider"].value):
            frame = self.image_index[self.now_frame-1]
            self.update_image(frame)
            self.now_frame = int(self.ids["timeSlider"].value)

    def update_image(self, frame):
        ##############################
        #Write the image processing source here! !!
        ##############################
        
        #flip upside down
        buf = cv2.flip(frame, 0)
        image_texture = Texture.create(size=(frame.shape[1], frame.shape[0]), colorfmt='bgr')
        image_texture.blit_buffer(buf.tostring(), colorfmt='bgr', bufferfmt='ubyte')
        video = self.ids['video']
        video.texture = image_texture

class TestVideo(App):

    def build(self):
        return MyVideoPlayer()

TestVideo().run()

kv file

VideoApp.kv


<MyVideoPlayer>:
    orientation: 'vertical'
    padding: 0
    spacing: 1

    BoxLayout:
        orientation: 'horizontal'
        padding: 0
        spacing: 1
        size_hint: (1.0, 0.1)

        TextInput:
            id: txtFName
            text: ''
            multiline: False

        Button:
            text: 'file load'
            on_press: root.fileSelect()

    BoxLayout:
        orientation: 'horizontal'
        padding: 0
        spacing: 1

        Image:
            id: video

    BoxLayout:
        orientation: 'horizontal'
        padding: 0
        spacing: 1
        size_hint: (1.0, 0.1)
        Slider:
            id: timeSlider
            value: 0.0
            max: 0.0
            min: 0.0
            step: 1
            on_touch_move: root.siderTouchMove()

    BoxLayout:
        orientation: 'horizontal'
        padding: 0
        spacing: 1
        size_hint: (1.0, 0.1)

        ToggleButton:
            size_hint: (0.2, 1)
            text: 'Play'
            on_press: root.play()

        Label:
            size_hint: (0.2, 1)
            text: str(timeSlider.value) + "/" + str(timeSlider.max)

<LoadDialog>:
    BoxLayout:
        size: root.size
        pos: root.pos
        orientation: 'vertical'
        FileChooserListView:
            id: filechooser
            path: "./"

        BoxLayout:
            size_hint_y : None
            height : 30
            Button:
                text: 'Cancel'
                on_release: root.cancel()

            Button:
                text: 'Load'
                on_release: root.load(filechooser.path, filechooser.selection)

When you run it, you can play it like this, or you can move the seek bar. The video was borrowed from the here site. Player.gif

A little commentary

The video itself will be played at the same dose as opencv's Video Cupture. I'm sure no one has used it, but it's like reading a video frame frame by frame with a while statement and displaying it (like this. / python-opencv-videocapture-file-camera /)). Again, if you use for or while, it will freeze, so use Clock.

    #Video playback clock processing
    def update(self, dt):
        ret, frame = self.image_capture.read()
        #When the next frame can be read
        if ret:
            self.update_image(frame) #Processing to copy to the screen
            time = self.image_capture.get(cv2.CAP_PROP_POS_FRAMES) #Get the number of frames for the seek bar
            self.ids["timeSlider"].value = time #Substitute the number of playback frames for the seek bar value
            self.now_frame = int(time) #For when moving the seek bar

Also, when displaying an image on kivy, use a class called Texture with `` `blit_buffer```. Handle the image as a buffer. At this time, the image data of opencv will be upside down, so add a process to invert it. By doing this, you can implement the video playback function in the same way as normal openCV video playback. Image processing is not performed this time, but if you add processing such as opencv here, you can easily visualize the processing results.

    def update_image(self, frame):
        ##############################
        #Write the image processing source here! !!
        ##############################
        
        #flip upside down
        buf = cv2.flip(frame, 0)
        image_texture = Texture.create(size=(frame.shape[1], frame.shape[0]), colorfmt='bgr')
        image_texture.blit_buffer(buf.tostring(), colorfmt='bgr', bufferfmt='ubyte')
        video = self.ids['video']
        video.texture = image_texture

In the video seek bar, initially from the set function (function that specifies the playback frame) of the VideoCupter class of opencv , I was trying to implement a seek bar by applying the Slider value to the frame number of the video. However, when I implemented it, the operation became extremely heavy. Cause, the set function seems to be a very heavy process, so I searched for another implementation method.

As a result, it was possible to easily implement by storing the image data of opencv as it is in the array and associating the value of Slider with the subscript of the array of images. Therefore, when reading a video, a process is provided to play the video once and assign it to the array.

Since the VideoCupture that has been read once is used in the program, you have to specify `self.image_capture.set (cv2.CAP_PROP_POS_FRAMES, 0)` to return the playback position of the video to the initial state. , You will not be able to play the video.

    #Seek bar settings
    def sliderSetting(self):
        count = self.image_capture.get(cv2.CAP_PROP_FRAME_COUNT) #Get the number of frames in a video
        self.ids["timeSlider"].max = count #Substitute the number of frames in the video for the maximum value of the slider

        #Load the video once and save all frames in an array
        while True:
            ret, frame = self.image_capture.read()
            if ret:
                self.image_index.append(frame)

            else:
                #After reading to the last frame, return to the first frame
                self.image_capture.set(cv2.CAP_PROP_POS_FRAMES, 0)
                break

References

Thank you for all the help you have given me.

How to display images with Opencv and Pillow with Python Kivy Handling of Texture, etc.

Getting Started with Python, Part 7: Select a File and Play Video-Akirachin's Technical Memo Thank you very much for the video.

Recommended Posts

GUI programming with kivy ~ Part 3 Video and seek bar ~
GUI programming with kivy ~ Part 4 Various buttons ~
GUI programming using kivy ~ Part 2 Progress bar ~
GUI programming with kivy ~ Part 5 Creating buttons with images ~
GUI programming using kivy ~ Part 6 Various layouts ~
Programming with Python and Tkinter
[GUI with Python] PyQt5-Drag and drop-
FM modulation and demodulation with Python Part 3
Execute Google Translate and DeepL Translate with GUI
Learn math and English through programming (Part 1)
Learn math and English by programming (Part 2)
GUI creation with Pyside Part 2 <Use of class>