[PYTHON] Image processing with MyHDL

What is MyHDL

http://www.myhdl.org/ A library for modeling hardware in Python, developed in. This time, I tried modeling image processing using MyHDL. In terms of layers, I felt that I was in the same position as SystemC.

All sources are up here. https://github.com/natsutan/computervision/tree/master/LOCV/chap5

Description of image processing

I tried to add a simple blur to a certain area of the input image. The algorithm uses the average value of 5x5 around the pixel of interest as that pixel.

Description in OpenCV

I will write here quickly using OpenCV.

def run_opencv():
    src = cv2.imread('../../image/twittan/twittan.jpg')
    dst = src.copy()

    roi_x = 100
    roi_y = 100
    roi_w = 150
    roi_h = 200

    dst[roi_y:roi_y + roi_h, roi_x:roi_x + roi_w] = cv2.blur(src[roi_y:roi_y + roi_h, roi_x:roi_x + roi_w], (5, 5),
                                                             (-1, -1))
    print("image size:width = %d, height = %d" % (dst.shape[1], dst.shape[0]))

    cv2.imwrite('twi_blur_cv.jpg', dst)

Input image

twi.jpg

Output image

There is a slight blur on the face. twi_blur_cv.jpg

Description in MyHDL

file organization

This time I'm using 5 files. --smooth.py This is the entire TOP including OpenCV processing. Run Sim with python smoothy.py. --myhdl_top.py This is the TOP of MyHDL environment. Includes Clk and Reset. --smooth_hdl.py RTL is listed. --mem.py MyHDL simulation memory model. --Reg_driver.py Set the register.

Test bench side

It is a description on the test bench side.

CLK, Reset, etc.

Omitted because it is almost the same as the sample

Register settings

Since the positive logic reset comes only 1clk, we wait for it to set the register. After that, the start register is set to 1 by 1clk to start the process, and the process ends after waiting for the end register to become 1. yield clk.posedge is equivalent to wait posedge (clk) ;.

reg_driver.py


# -*- coding: utf-8 -*-
__author__ = 'natu'
from myhdl import *

def reg_driver_top(
        clk, reset,
        reg_start, reg_end,
        reg_width, reg_height,
        reg_roi_x, reg_roi_y, reg_roi_h, reg_roi_w
        ):

    @instance
    def regDriver():
        while reset == 0:
            yield clk.posedge
        while reset == 1:
            yield clk.posedge

        reg_width.next = 358
        reg_height.next = 557
        reg_roi_x.next = 100
        reg_roi_y.next = 100
        reg_roi_h.next = 200
        reg_roi_w.next = 150
        yield clk.posedge

        reg_start.next = 1
        yield clk.posedge
        reg_start.next = 0
        yield clk.posedge

        while reg_end == 0:
            yield clk.posedge

        print("end == 1")
        yield clk.posedge

    return regDriver

memory

This was where the power of Python could be demonstrated. You can use OpenCV imread to open a jpeg file directly and use it for Sim. You don't have to dump Hex and use readmemh. The output can also be directly converted to an image with OpenCV.

The read side updates read_r, read_g, and read_b when radr changes in the combinational circuit, and the write side writes to the memory when wen is 1 in clk synchronization.

mem.py


# -*- coding: utf-8 -*-
__author__ = 'natu'
from myhdl import *
import numpy
import cv2

dst = None

def mem_top(
        clk, reset,
        read_r, read_g, read_b, radr,
        write_r, write_g, write_b, wadr, wen):
    global dst

    src = cv2.imread('../../image/twittan/twittan.jpg')
    dst = numpy.zeros(src.shape)

    @always_comb
    def mem_read():
        x, y = adr_dec(radr)
        read_r.next = clop_8bit(src[y][x][0])
        read_g.next = clop_8bit(src[y][x][1])
        read_b.next = clop_8bit(src[y][x][2])


    @instance
    def mem_write():
        while True:
            if wen == 1:
                x, y = adr_dec(wadr)
                dst[y][x][0] = write_r
                dst[y][x][1] = write_g
                dst[y][x][2] = write_b
            yield clk.posedge

    return mem_read, mem_write

def write_image():
    cv2.imwrite('twi_blur_rtl.jpg', dst)

def adr_dec(adr):
    width = dst.shape[1]
    x = int(adr) % width
    y = int(adr) / width
    return x, y

def clop_8bit(x):
    if x >= 255:
        return 255

    return int(x)

RTL side

Since the state machine was in the sample, I put it in for the time being, but it is not a very important movement. It is convenient to be able to use for to process x and y.

With this double loop, the periphery 5x5 of the pixel of interest is acquired.

      for ry in range(-2,3):
          for rx in range(-2,3):

The values are added to sum_r, sum_g, and sum_b, and the average value is calculated at // 25. It would be very convenient if you could synthesize it with just this.

Below, all sources

smooth_hdl.py


# -*- coding: utf-8 -*-
__author__ = 'natu'
from myhdl import *

t_State = enum('IDLE', 'RUNNING')

def smoother_top(
        clk, reset,
        rin, gin, bin, radr,
        rout, gout, bout, wadr, wen,
        reg_start, reg_end,
        reg_width, reg_height,
        reg_roi_x, reg_roi_y, reg_roi_h, reg_roi_w
    ):

    state = Signal(t_State.IDLE)

    @instance
    def main_proc():
        while 1:
            if state == t_State.RUNNING:
                for y in range(reg_height):
                    print("y = %d" % y)
                    for x in range(reg_width):
                        if reg_roi_x <= x and x < reg_roi_x + reg_roi_w and reg_roi_y <= y and y < reg_roi_y + reg_roi_h:
                            # ROI
                            sum_r = 0
                            sum_g = 0
                            sum_b = 0
                            for ry in range(-2,3):
                                for rx in range(-2,3):
                                    radr.next = adr(x + rx, y + ry)
                                    yield  clk.posedge
                                    sum_r = sum_r + rin
                                    sum_g = sum_g + gin
                                    sum_b = sum_b + bin
                                    yield  clk.posedge
                            wadr.next = adr(x, y)
                            rout.next = sum_r // 25
                            gout.next = sum_g // 25
                            bout.next = sum_b // 25
                            wen.next = 1
                            yield  clk.posedge
                            wen.next = 0
                        else:
                            radr.next = adr(x, y)
                            yield  clk.posedge
                            wadr.next = adr(x, y)
                            rout.next = rin
                            gout.next = gin
                            bout.next = bin
                            wen.next = 1
                            yield  clk.posedge
                            wen.next = 0
                reg_end.next = 1
                yield  clk.posedge

            yield  clk.posedge


    def adr(x, y):
        return y * reg_width + x

    @always_seq(clk.posedge, reset=reset)
    def fsm():
        if state == t_State.IDLE:
            if reg_start == 1:
                state.next = t_State.RUNNING
        elif state == t_State.RUNNING:
            if reg_end == 1:
                state.next = t_State.IDLE
        else:
            raise ValueError("Undefined state")
            
    return fsm, main_proc

Processing result

Output image by MyHDL

It is almost the same as the result of OpenCV.

twi_blur_rtl.jpg

vcd output

Since vcd can also be output, you can check the waveform with gtkwave. It's working as expected. gtkwave.png

Conversion to Verilog

Conversion method

Conversion to Verilog is also one shot using the toVerilog function. While 1: is OK, while True is useless, and there were some subtle barriers, but I was able to Verilog with this description.

    toVerilog(smoother_top,
        clk, reset,
        rin, gin, bin, radr,
        rout, gout, bout, wadr, wen,
        reg_start, reg_end,
        reg_width, reg_height,
        reg_roi_x, reg_roi_y, reg_roi_h, reg_roi_w
    )

Conversion result

That's exciting.

// File: smoother_top.v
// Generated by MyHDL 0.9.dev0
// Date: Tue May 19 14:30:05 2015


`timescale 1ns/10ps

module smoother_top (
    clk,
    reset,
    rin,
    gin,
    bin,
    radr,
    rout,
    gout,
    bout,
    wadr,
    wen,
    reg_start,
    reg_end,
    reg_width,
    reg_height,
    reg_roi_x,
    reg_roi_y,
    reg_roi_h,
    reg_roi_w
);


input clk;
input reset;
input [7:0] rin;
input [7:0] gin;
input [7:0] bin;
output [19:0] radr;
reg [19:0] radr;
output [7:0] rout;
reg [7:0] rout;
output [7:0] gout;
reg [7:0] gout;
output [7:0] bout;
reg [7:0] bout;
output [19:0] wadr;
reg [19:0] wadr;
output wen;
reg wen;
input reg_start;
output reg_end;
reg reg_end;
input [9:0] reg_width;
input [9:0] reg_height;
input [9:0] reg_roi_x;
input [9:0] reg_roi_y;
input [9:0] reg_roi_h;
input [9:0] reg_roi_w;

reg [0:0] state;




function integer MYHDL13_adr;
    input x;
    integer x;
    input y;
    integer y;
begin: MYHDL17_RETURN
    MYHDL13_adr = ((y * $signed({1'b0, reg_width})) + x);
    disable MYHDL17_RETURN;
end
endfunction

function integer MYHDL14_adr;
    input x;
    integer x;
    input y;
    integer y;
begin: MYHDL18_RETURN
    MYHDL14_adr = ((y * $signed({1'b0, reg_width})) + x);
    disable MYHDL18_RETURN;
end
endfunction

function integer MYHDL15_adr;
    input x;
    integer x;
    input y;
    integer y;
begin: MYHDL19_RETURN
    MYHDL15_adr = ((y * $signed({1'b0, reg_width})) + x);
    disable MYHDL19_RETURN;
end
endfunction

function integer MYHDL16_adr;
    input x;
    integer x;
    input y;
    integer y;
begin: MYHDL20_RETURN
    MYHDL16_adr = ((y * $signed({1'b0, reg_width})) + x);
    disable MYHDL20_RETURN;
end
endfunction


always @(posedge clk, posedge reset) begin: SMOOTHER_TOP_FSM
    if (reset == 1) begin
        state <= 1'b0;
    end
    else begin
        case (state)
            1'b0: begin
                if ((reg_start == 1)) begin
                    state <= 1'b1;
                end
            end
            1'b1: begin
                if ((reg_end == 1)) begin
                    state <= 1'b0;
                end
            end
            default: begin
                $finish;
            end
        endcase
    end
end


initial begin: SMOOTHER_TOP_MAIN_PROC
    integer sum_b;
    integer rx;
    integer ry;
    integer sum_g;
    integer y;
    integer x;
    integer sum_r;
    while (1) begin
        if ((state == 1'b1)) begin
            for (y=0; y<reg_height; y=y+1) begin
                $write("y = ");
                $write("%0d", y);
                $write("\n");
                for (x=0; x<reg_width; x=x+1) begin
                    if ((($signed({1'b0, reg_roi_x}) <= x) && (x < (reg_roi_x + reg_roi_w)) && ($signed({1'b0, reg_roi_y}) <= y) && (y < (reg_roi_y + reg_roi_h)))) begin
                        sum_r = 0;
                        sum_g = 0;
                        sum_b = 0;
                        for (ry=(-2); ry<3; ry=ry+1) begin
                            for (rx=(-2); rx<3; rx=rx+1) begin
                                radr <= MYHDL13_adr((x + rx), (y + ry));
                                @(posedge clk);
                                sum_r = (sum_r + rin);
                                sum_g = (sum_g + gin);
                                sum_b = (sum_b + bin);
                                @(posedge clk);
                            end
                        end
                        wadr <= MYHDL14_adr(x, y);
                        rout <= (sum_r / 25);
                        gout <= (sum_g / 25);
                        bout <= (sum_b / 25);
                        wen <= 1;
                        @(posedge clk);
                        wen <= 0;
                    end
                    else begin
                        radr <= MYHDL15_adr(x, y);
                        @(posedge clk);
                        wadr <= MYHDL16_adr(x, y);
                        rout <= rin;
                        gout <= gin;
                        bout <= bin;
                        wen <= 1;
                        @(posedge clk);
                        wen <= 0;
                    end
                end
            end
            reg_end <= 1;
            @(posedge clk);
        end
        @(posedge clk);
    end
end

endmodule

initial begin: SMOOTHER_TOP_MAIN_PROC、、、

(Tsu д⊂) Gojigoshi → (; ゚ Д ゚)…! ??

People People People People People People > initial begin: <  ̄Y^Y^Y^Y^Y^Y^ ̄

I can't synthesize! No good> <

Impressions

Good place

--PyCharm is convenient. It tells you unused signals, batch replaces variables that understand the scope, etc. However, in the default state, there are too many warnings and it is difficult to use. Oh, there are a lot of line breaks, there aren't many, and I'm annoyed by the space here, so it's a bit painful. --The Python library is useful.

I wanted to try assertions and coverage, but I was so overwhelmed that I couldn't synthesize the circuit I made, so I couldn't go that far.

Where it didn't work

――It is very difficult to write a moving circuit. It is not intuitive to return a function that describes the behavior of the circuit. Difficult to debug. --The error is difficult to understand. Perhaps I'm looking directly at the Python syntax tree, the error message is long and I feel hopeless like C ++. If you mistakenly set sum_r = 0 to sum_r == 0, you will get a 67-line error message. --The description that can be synthesized is very limited. Even in this circuit, the whole is an infinite loop, and wait is all at the rise of clk, so I thought that this would be replaced with a compositable Verilog, but this description was useless.

    @always_seq(clk.posedge, reset=reset)
    def fsm():
        if state == t_State.IDLE:
            if reg_start == 1:
                state.next = t_State.RUNNING
        elif state == t_State.RUNNING:
            if reg_end == 1:
                state.next = t_State.IDLE
        else:
            raise ValueError("Undefined state")

The description that can be synthesized is this pattern, and you have to write a generator that does not have yield in the middle. In other words, it is the same as Verilog-HDL. It's just a modeling language.

Recommended Posts

Image processing with MyHDL
Image processing with Python
Image Processing with PIL
Image processing with Python (Part 2)
Image processing with PIL (Pillow)
Image processing with Python (Part 1)
Image processing with Python (Part 3)
[Python] Image processing with scikit-image
Real-time image processing basics with opencv
Image processing with Python 100 knocks # 3 Binarization
Image processing with Python 100 knocks # 2 Grayscale
[Image processing] Posterization
python image processing
Image processing 100 knocks ①
Basics of binarized image processing with Python
Image processing with Python 100 knock # 10 median filter
Create an image processing viewer with PySimpleGUI
Image processing with Python 100 knocks # 8 Max pooling
Image processing with Python & OpenCV [Tone Curve]
Image processing with Python 100 knock # 12 motion filter
Drawing with Matrix-Reinventor of Python Image Processing-
Easy image processing in Python with Pillow
Image processing with Python 100 knocks # 7 Average pooling
Light image processing with Python x OpenCV
Image processing with Lambda + OpenCV (gray image creation)
Image processing with Python 100 knocks # 9 Gaussian filter
XavierNX accelerates OpenCV image processing with GPU (CUDA)
Processing datasets with pandas (1)
Processing datasets with pandas (2)
Flat Field image processing
Image recognition with keras
Image processing from scratch with python (5) Fourier transform
First Python image processing
Read digital image processing
Image processing from scratch with python (4) Contour extraction
Image Processing with Python Environment Setup for Windows
Parallel processing with multiprocessing
I tried simple image processing with Google Colaboratory.
Notes on HDR and RAW image processing with Python
Image download with Flickr API
100 Language Processing with Python Knock 2015
opencv-python Introduction to image processing
Read image coordinates with Python-matplotlib
Parallel processing with local functions
"Apple processing" with OpenCV3 + Python3
Image editing with python OpenCV
Acoustic signal processing with Python (2)
Digital image processing (spatial filtering)
100 image processing knocks !! (011 --020) Early game
Acoustic signal processing with Python
Image upload & customization with django-ckeditor
Sorting image files with Python (2)
Sorting image files with Python (3)
100 image processing knocks !! (001 --010) Carefully and carefully
Easy image classification with TensorFlow
Parallel processing with Parallel of scikit-learn
Create Image Viewer with Tkinter
Tweet with image in Python
Sorting image files with Python
Image processing by python (Pillow)
Image Processing Collection in Python