Introduction

This article is from the here Advent calendar.

I started riding a motorcycle in June. I just got a license, so I want to drive safely.

I often hear news about road rage these days, so I installed a dashcam for motorcycles for self-defense.

Mitsuba Sankowa MITSUBA Motorcycle Drive Recorder EDR-21 Front and Rear 2 Cameras EDR-21

With two front and rear cameras, it has a super wide angle of 162 °, is strong in dark places, and is a nice guy that also supports a 256GB SD card. I think it was about 23000 yen at the time of purchase. If you buy and install it at a motorcycle shop, it will cost about 40,000 to 50,000 yen including wages, so I bought it from Amazon and installed it myself.

By the way, you can watch the video shot with this dashcam via your smartphone, or you can connect the SD card to your PC and watch it directly, but the specifications are such that the file is saved in small pieces every 30 seconds. It is quite difficult to handle. In addition, about 0.5 seconds before and after between files are saved in duplicate, and if you try to combine them into one file, it will not be clean unless you cut this duplicate 0.5 seconds, which will be very annoying.

This is the image.

If you try to connect it neatly in consideration of the overlap, you have to do something like this, and it takes time to manually edit this every 30 seconds.

I don't do anything by combining the videos into one, but I couldn't find a good way at this point, so I decided to try it with python for studying as well.

Overview

Since the video of the dash cam also has audio, it will be an image that cuts both video and audio for 0.5 seconds and connects them. To do it with python, it seems difficult to edit video and audio at the same time, so it seems necessary to edit them separately.

The software and libraries used are here.

--Separation / integration of video and audio, video encoding: ffmpeg (executed from Python with subprocess.run) --Cut & combine 0.5 seconds of video: opencv --Cut and combine 0.5 seconds of audio: pydub

Implementation

Preparation

Let's see how the dashcam video file is created.

It seems that the shooting start date and time and the 30-second video start date and time are stored in the file name for each front/rear camera. (N/E ... Ignore it for the time being. It's okay.)

Implementation image

Programmatically, we will group by camera x shooting start date and time, create chunks in order of movie start date and time, and process each chunk as if the following processing is performed.

Separate 30-second video into video and audio
Cut the video and audio butt 0.5 seconds
Connect
When everything is connected, attach the video and audio and re-encode as a video

Implementation

`mitsuba.py`


# -*- coding: utf-8 -*-
import os
import shutil
import cv2
import glob
import subprocess
from pydub import AudioSegment
from collections import defaultdict
from tqdm import tqdm, trange
from multiprocessing import Pool,Process, Queue, TimeoutError
from queue import Empty

DUP_FRAME = 14

# multi processing
WORKERS = 4
TIMEOUT = 10

def comb_movie(movie_files, out_path, num):
    #Skip if created
    if os.path.exists(os.path.join("out",out_path)):
        return

    #The format is mp4
    fourcc = cv2.VideoWriter_fourcc('m', 'p', '4', 'v')

    #Acquisition of video information
    movie = cv2.VideoCapture(movie_files[0])
    fps = movie.get(cv2.CAP_PROP_FPS)
    height = movie.get(cv2.CAP_PROP_FRAME_HEIGHT)
    width = movie.get(cv2.CAP_PROP_FRAME_WIDTH)

    #Open the output file
    out = cv2.VideoWriter(f"tmp/video_{num:02}.mp4", int(fourcc), fps,
                        (int(width), int(height)))

    audio_merged = None
    for movies in movie_files:
        #Read video file, argument is video file path
        movie = cv2.VideoCapture(movies)
        count = movie.get(cv2.CAP_PROP_FRAME_COUNT)
        frames = []
        if movie.isOpened() == False:  #Check if the video file can be read normally
            continue

        for _ in range(int(count)):
            ret, tmp_f = movie.read()  # read():Read the captured image data for one frame
            if ret:
                frames.append(tmp_f)

        #Write the read frame
        for frame in frames[:-DUP_FRAME]:
            out.write(frame)

        command = f"ffmpeg -y -i {movies} -vn -loglevel quiet tmp/audio_{num:02}.wav"
        subprocess.run(command, shell=True)

        audio_tmp = AudioSegment.from_file(f"tmp/audio_{num:02}.wav", format="wav")
        audio_tmp = audio_tmp[:int((count-DUP_FRAME)/fps*1000)]

        if audio_merged is None:
            audio_merged = audio_tmp
        else:
            audio_merged += audio_tmp

    #Combined audio export
    audio_merged.export(f"tmp/audio_merged_{num:02}.wav", format="wav")
    out.release()

    #Video and audio combination
    vf = ""  #Video filter is your choice Example) For slightly soft, saturated, and noise-removed"-vf smartblur=lr=1:ls=1:lt=0:cr=-0.9:cs=-2:ct=-31,eq=brightness=-0.06:saturation=1.4,hqdn3d,pp=ac"
    #If it supports a high-speed encoder, you can use it as you like. Example) h264_videotoolbox, libx264, h264_nvenc
    cv = f"-c:v libx264"
    #The bit rate is fixed according to the resolution.
    if height == 1080: # FHD
        bv = f"-b:v 11m"
    elif height == 720: # HD
        bv = f"-b:v 6m"
    else: # VGA
        bv = f"-b:v 3m"

    loglevel = "-loglevel quiet"
    command = f"ffmpeg -y -i tmp/video_{num:02}.mp4 -i tmp/audio_merged_{num:02}.wav {cv} {bv} {vf} -c:a aac {loglevel} out/{out_path}"
    subprocess.run(command, shell=True)


def wrapper(args):
    comb_movie(*args)

if __name__ == '__main__':
    os.makedirs("./tmp", exist_ok=True)
    os.makedirs("./out", exist_ok=True)

    #Collect videos in the directory by front / rear camera and shooting start time
    files_dict = defaultdict(list)
    for f in glob.glob("./in/*.MP4"):
        files_dict["_".join(f.split("/")[-1].split("_")[:2])].append(f)

    data = []
    for i, (key_name, files_list) in enumerate(files_dict.items()):
        data.append((sorted(files_list), key_name+".mp4", i))

    p = Pool(WORKERS)
    with tqdm(total=len(data)) as t:
        for _ in p.imap_unordered(wrapper, data):
            t.update(1)
    #tmp deleted
    shutil.rmtree('./tmp/')

Supplement

The source code is also available on Github.

Sample video uploaded to YouTube

――Put the live video of the dashcam in the in folder.

--When executed, the progress bar will be displayed. (I really want to display it in a little more detail)

--The video file combined in the out folder is created.

DUP_FRAME = 14 No change is required as the number of overlapping frames. (Video is 28fps: 28 frames/s x 0.5s = 14 frames)
WORKERS = 4 You can specify the number of parallel executions with the option to perform parallel processing and speed up. Please change it according to your environment.

--Video filter is your choice The original video has a slightly sharpness (because it is a dashcam), so you can try softening it and set various filters. Check out the ffmpeg filter for more details.

--If it supports a high-speed encoder, you can use it as you like. --libx264: Can be used in almost any environment --h264_videotoolbox: for mac. Should be several times faster than libx264. --h264_nvenc: For linux (can windows be used?) It should be several times faster than libx264.

bit rate I like it, but since the original video is not so beautiful, increasing the bit rate may not be very effective.

in conclusion

I bought GoPro8, but it's pretty clean. It seems better to use the dashcam as a dashcam and take in-vehicle videos with GoPro.

Use python to successfully combine video files (every 30 seconds, with 0.5 second overlap) taken with the Mitsuba Sankowa motorcycle dashcam EDR-21.

Introduction

Overview

Implementation

Preparation

Implementation image

Implementation

mitsuba.py

Supplement

in conclusion

`mitsuba.py`