[PYTHON] Image processing with MyHDL
What is MyHDL
http://www.myhdl.org/ A library for modeling hardware in Python, developed in. This time, I tried modeling image processing using MyHDL. In terms of layers, I felt that I was in the same position as SystemC.
All sources are up here. https://github.com/natsutan/computervision/tree/master/LOCV/chap5
Description of image processing
I tried to add a simple blur to a certain area of the input image. The algorithm uses the average value of 5x5 around the pixel of interest as that pixel.
Description in OpenCV
I will write here quickly using OpenCV.
def run_opencv():
src = cv2.imread('../../image/twittan/twittan.jpg')
dst = src.copy()
roi_x = 100
roi_y = 100
roi_w = 150
roi_h = 200
dst[roi_y:roi_y + roi_h, roi_x:roi_x + roi_w] = cv2.blur(src[roi_y:roi_y + roi_h, roi_x:roi_x + roi_w], (5, 5),
(-1, -1))
print("image size:width = %d, height = %d" % (dst.shape[1], dst.shape[0]))
cv2.imwrite('twi_blur_cv.jpg', dst)
Input image
Output image
There is a slight blur on the face.
Description in MyHDL
file organization
This time I'm using 5 files. --smooth.py This is the entire TOP including OpenCV processing. Run Sim with python smoothy.py. --myhdl_top.py This is the TOP of MyHDL environment. Includes Clk and Reset. --smooth_hdl.py RTL is listed. --mem.py MyHDL simulation memory model. --Reg_driver.py Set the register.
Test bench side
It is a description on the test bench side.
CLK, Reset, etc.
Omitted because it is almost the same as the sample
Register settings
Since the positive logic reset comes only 1clk, we wait for it to set the register. After that, the start register is set to 1 by 1clk to start the process, and the process ends after waiting for the end register to become 1. yield clk.posedge is equivalent to wait posedge (clk) ;.
reg_driver.py
# -*- coding: utf-8 -*-
__author__ = 'natu'
from myhdl import *
def reg_driver_top(
clk, reset,
reg_start, reg_end,
reg_width, reg_height,
reg_roi_x, reg_roi_y, reg_roi_h, reg_roi_w
):
@instance
def regDriver():
while reset == 0:
yield clk.posedge
while reset == 1:
yield clk.posedge
reg_width.next = 358
reg_height.next = 557
reg_roi_x.next = 100
reg_roi_y.next = 100
reg_roi_h.next = 200
reg_roi_w.next = 150
yield clk.posedge
reg_start.next = 1
yield clk.posedge
reg_start.next = 0
yield clk.posedge
while reg_end == 0:
yield clk.posedge
print("end == 1")
yield clk.posedge
return regDriver
memory
This was where the power of Python could be demonstrated. You can use OpenCV imread to open a jpeg file directly and use it for Sim. You don't have to dump Hex and use readmemh. The output can also be directly converted to an image with OpenCV.
The read side updates read_r, read_g, and read_b when radr changes in the combinational circuit, and the write side writes to the memory when wen is 1 in clk synchronization.
mem.py
# -*- coding: utf-8 -*-
__author__ = 'natu'
from myhdl import *
import numpy
import cv2
dst = None
def mem_top(
clk, reset,
read_r, read_g, read_b, radr,
write_r, write_g, write_b, wadr, wen):
global dst
src = cv2.imread('../../image/twittan/twittan.jpg')
dst = numpy.zeros(src.shape)
@always_comb
def mem_read():
x, y = adr_dec(radr)
read_r.next = clop_8bit(src[y][x][0])
read_g.next = clop_8bit(src[y][x][1])
read_b.next = clop_8bit(src[y][x][2])
@instance
def mem_write():
while True:
if wen == 1:
x, y = adr_dec(wadr)
dst[y][x][0] = write_r
dst[y][x][1] = write_g
dst[y][x][2] = write_b
yield clk.posedge
return mem_read, mem_write
def write_image():
cv2.imwrite('twi_blur_rtl.jpg', dst)
def adr_dec(adr):
width = dst.shape[1]
x = int(adr) % width
y = int(adr) / width
return x, y
def clop_8bit(x):
if x >= 255:
return 255
return int(x)
RTL side
Since the state machine was in the sample, I put it in for the time being, but it is not a very important movement. It is convenient to be able to use for to process x and y.
With this double loop, the periphery 5x5 of the pixel of interest is acquired.
for ry in range(-2,3):
for rx in range(-2,3):
The values are added to sum_r, sum_g, and sum_b, and the average value is calculated at // 25. It would be very convenient if you could synthesize it with just this.
Below, all sources
smooth_hdl.py
# -*- coding: utf-8 -*-
__author__ = 'natu'
from myhdl import *
t_State = enum('IDLE', 'RUNNING')
def smoother_top(
clk, reset,
rin, gin, bin, radr,
rout, gout, bout, wadr, wen,
reg_start, reg_end,
reg_width, reg_height,
reg_roi_x, reg_roi_y, reg_roi_h, reg_roi_w
):
state = Signal(t_State.IDLE)
@instance
def main_proc():
while 1:
if state == t_State.RUNNING:
for y in range(reg_height):
print("y = %d" % y)
for x in range(reg_width):
if reg_roi_x <= x and x < reg_roi_x + reg_roi_w and reg_roi_y <= y and y < reg_roi_y + reg_roi_h:
# ROI
sum_r = 0
sum_g = 0
sum_b = 0
for ry in range(-2,3):
for rx in range(-2,3):
radr.next = adr(x + rx, y + ry)
yield clk.posedge
sum_r = sum_r + rin
sum_g = sum_g + gin
sum_b = sum_b + bin
yield clk.posedge
wadr.next = adr(x, y)
rout.next = sum_r // 25
gout.next = sum_g // 25
bout.next = sum_b // 25
wen.next = 1
yield clk.posedge
wen.next = 0
else:
radr.next = adr(x, y)
yield clk.posedge
wadr.next = adr(x, y)
rout.next = rin
gout.next = gin
bout.next = bin
wen.next = 1
yield clk.posedge
wen.next = 0
reg_end.next = 1
yield clk.posedge
yield clk.posedge
def adr(x, y):
return y * reg_width + x
@always_seq(clk.posedge, reset=reset)
def fsm():
if state == t_State.IDLE:
if reg_start == 1:
state.next = t_State.RUNNING
elif state == t_State.RUNNING:
if reg_end == 1:
state.next = t_State.IDLE
else:
raise ValueError("Undefined state")
return fsm, main_proc
Processing result
Output image by MyHDL
It is almost the same as the result of OpenCV.
vcd output
Since vcd can also be output, you can check the waveform with gtkwave. It's working as expected.
Conversion to Verilog
Conversion method
Conversion to Verilog is also one shot using the toVerilog function. While 1: is OK, while True is useless, and there were some subtle barriers, but I was able to Verilog with this description.
toVerilog(smoother_top,
clk, reset,
rin, gin, bin, radr,
rout, gout, bout, wadr, wen,
reg_start, reg_end,
reg_width, reg_height,
reg_roi_x, reg_roi_y, reg_roi_h, reg_roi_w
)
Conversion result
That's exciting.
// File: smoother_top.v
// Generated by MyHDL 0.9.dev0
// Date: Tue May 19 14:30:05 2015
`timescale 1ns/10ps
module smoother_top (
clk,
reset,
rin,
gin,
bin,
radr,
rout,
gout,
bout,
wadr,
wen,
reg_start,
reg_end,
reg_width,
reg_height,
reg_roi_x,
reg_roi_y,
reg_roi_h,
reg_roi_w
);
input clk;
input reset;
input [7:0] rin;
input [7:0] gin;
input [7:0] bin;
output [19:0] radr;
reg [19:0] radr;
output [7:0] rout;
reg [7:0] rout;
output [7:0] gout;
reg [7:0] gout;
output [7:0] bout;
reg [7:0] bout;
output [19:0] wadr;
reg [19:0] wadr;
output wen;
reg wen;
input reg_start;
output reg_end;
reg reg_end;
input [9:0] reg_width;
input [9:0] reg_height;
input [9:0] reg_roi_x;
input [9:0] reg_roi_y;
input [9:0] reg_roi_h;
input [9:0] reg_roi_w;
reg [0:0] state;
function integer MYHDL13_adr;
input x;
integer x;
input y;
integer y;
begin: MYHDL17_RETURN
MYHDL13_adr = ((y * $signed({1'b0, reg_width})) + x);
disable MYHDL17_RETURN;
end
endfunction
function integer MYHDL14_adr;
input x;
integer x;
input y;
integer y;
begin: MYHDL18_RETURN
MYHDL14_adr = ((y * $signed({1'b0, reg_width})) + x);
disable MYHDL18_RETURN;
end
endfunction
function integer MYHDL15_adr;
input x;
integer x;
input y;
integer y;
begin: MYHDL19_RETURN
MYHDL15_adr = ((y * $signed({1'b0, reg_width})) + x);
disable MYHDL19_RETURN;
end
endfunction
function integer MYHDL16_adr;
input x;
integer x;
input y;
integer y;
begin: MYHDL20_RETURN
MYHDL16_adr = ((y * $signed({1'b0, reg_width})) + x);
disable MYHDL20_RETURN;
end
endfunction
always @(posedge clk, posedge reset) begin: SMOOTHER_TOP_FSM
if (reset == 1) begin
state <= 1'b0;
end
else begin
case (state)
1'b0: begin
if ((reg_start == 1)) begin
state <= 1'b1;
end
end
1'b1: begin
if ((reg_end == 1)) begin
state <= 1'b0;
end
end
default: begin
$finish;
end
endcase
end
end
initial begin: SMOOTHER_TOP_MAIN_PROC
integer sum_b;
integer rx;
integer ry;
integer sum_g;
integer y;
integer x;
integer sum_r;
while (1) begin
if ((state == 1'b1)) begin
for (y=0; y<reg_height; y=y+1) begin
$write("y = ");
$write("%0d", y);
$write("\n");
for (x=0; x<reg_width; x=x+1) begin
if ((($signed({1'b0, reg_roi_x}) <= x) && (x < (reg_roi_x + reg_roi_w)) && ($signed({1'b0, reg_roi_y}) <= y) && (y < (reg_roi_y + reg_roi_h)))) begin
sum_r = 0;
sum_g = 0;
sum_b = 0;
for (ry=(-2); ry<3; ry=ry+1) begin
for (rx=(-2); rx<3; rx=rx+1) begin
radr <= MYHDL13_adr((x + rx), (y + ry));
@(posedge clk);
sum_r = (sum_r + rin);
sum_g = (sum_g + gin);
sum_b = (sum_b + bin);
@(posedge clk);
end
end
wadr <= MYHDL14_adr(x, y);
rout <= (sum_r / 25);
gout <= (sum_g / 25);
bout <= (sum_b / 25);
wen <= 1;
@(posedge clk);
wen <= 0;
end
else begin
radr <= MYHDL15_adr(x, y);
@(posedge clk);
wadr <= MYHDL16_adr(x, y);
rout <= rin;
gout <= gin;
bout <= bin;
wen <= 1;
@(posedge clk);
wen <= 0;
end
end
end
reg_end <= 1;
@(posedge clk);
end
@(posedge clk);
end
end
endmodule
initial begin: SMOOTHER_TOP_MAIN_PROC、、、
(Tsu д⊂) Gojigoshi → (； ﾟ Д ﾟ)…! ??
People People People People People People ＞ initial begin: ＜ ￣Y^Y^Y^Y^Y^Y^￣
I can't synthesize! No good> <
Impressions
Good place
--PyCharm is convenient. It tells you unused signals, batch replaces variables that understand the scope, etc. However, in the default state, there are too many warnings and it is difficult to use. Oh, there are a lot of line breaks, there aren't many, and I'm annoyed by the space here, so it's a bit painful. --The Python library is useful.
I wanted to try assertions and coverage, but I was so overwhelmed that I couldn't synthesize the circuit I made, so I couldn't go that far.
Where it didn't work
――It is very difficult to write a moving circuit. It is not intuitive to return a function that describes the behavior of the circuit. Difficult to debug. --The error is difficult to understand. Perhaps I'm looking directly at the Python syntax tree, the error message is long and I feel hopeless like C ++. If you mistakenly set sum_r = 0 to sum_r == 0, you will get a 67-line error message. --The description that can be synthesized is very limited. Even in this circuit, the whole is an infinite loop, and wait is all at the rise of clk, so I thought that this would be replaced with a compositable Verilog, but this description was useless.
@always_seq(clk.posedge, reset=reset)
def fsm():
if state == t_State.IDLE:
if reg_start == 1:
state.next = t_State.RUNNING
elif state == t_State.RUNNING:
if reg_end == 1:
state.next = t_State.IDLE
else:
raise ValueError("Undefined state")
The description that can be synthesized is this pattern, and you have to write a generator that does not have yield in the middle. In other words, it is the same as Verilog-HDL. It's just a modeling language.
Recommended Posts