[PYTHON] Other applications of dictionary learning

Overview

Implemented some of the other applications in Chapter 15 of Sparse Modeling.

• Separation of line art and texture
• Inpainting
• Impulse noise removal

notebook ch15-02.ipynb ch15-04.ipynb ch15-05.ipynb

result

_ K-SVD Separation of line art and texture by local morphological component analysis (MCA) using a dictionary _

_ Interpolate missing pixels with local inpainting with redundant DCT dictionary_

The numbers are [Peak signal to noise ratio (PSNR)](https://ja.wikipedia.org/wiki/Peak signal to noise ratio)

_ K-Interpolate missing pixels with local inpainting with SVD dictionary_

• At 25% and 50% defect rates, PSNR was improved compared to inpainting with a redundant DCT dictionary.

Removal of impulse noise

• Estimated mask is the threshold value of the difference between the noisy image and the image processed by the median filter.
• PSNR was improved by removing noise from only the pixels detected by the estimation mask.
• PSNR improved a little when the median filter was replaced with local inpainting using a redundant DCT dictionary.

Method

Local MCA with K-SVD dictionary

White noise with a standard deviation of 10 was added to Barbara as a test image.

I learned a dictionary with K-SVD. The patch size is $ 8 \ times 8 $. The number of atoms was set to 256. The stop conditions for orthogonal matching pursuit (OMP) were set to the number of atoms $ k_ {0} = 4 $ and the tolerance $ \ epsilon = 8 ^ {2} \ times 20 ^ {2} $. The number of repetitions of K-SVD was set to 25.

The total number of patches extracted from the image was 255,025. Due to lack of memory, the number of patches used for one iteration of K-SVD was reduced to 1/10. The patches used for K-SVD were randomly extracted every iteration.

K-SVD dictionary

The activity of atom was calculated as follows. $ Activity(\mathbf{a}) = \Sigma_{i=2}^{n} \Sigma_{j=1}^{n} |a[i, j] - a[i-1, j]| + \Sigma_{i=1}^{n} \Sigma_{j=2}^{n} |a[i, j] - a[i, j-1]| $The maximum value of activity was standardized to 1. Atoms with high activity are textures, and atoms with low activity are line art atoms.

Atom activity (0 for black, 1 for white) Atoms with activity less than 0.3 were line art, and the others were textures. Dark gray is a line art atom, and light gray is a texture atom.

The line art image can be obtained from the line art atom and the coefficient of its sparse expression. The image of the texture can be obtained from the atom of the texture and the coefficient of its sparse expression.

Image impainting and impulse noise removal

Added white noise with a standard deviation of 20 to Peppers. 25%, 50%, and 75% pixels were randomly cleared to 0 and used as a test image. The position of the missing pixel was known.

Local inpainting with redundant DCT dictionary

The patch size is $ 8 \ times 8 $. The number of atoms was set to 256. Modified OMP to mask missing pixels and find sparse representation. The OMP stop conditions are $ k_ {0} = 4 $ and $ \ epsilon = (8 ^ {2} -p) \ times 20 ^ {2} \ times 1.1 $. $ p $ is the number of missing pixels in the patch. Images can be restored from the desired sparse representation.

Local inpainting with K-SVD dictionary

The K-SVD was modified as follows.

2. The OMP was modified to mask the missing pixels and a sparse representation was sought. The OMP stop condition was the same as for local inpainting with a redundant DCT dictionary.
3. After interpolating the missing pixels using the above sparse representation, the atom and sparse representations were updated with K-SVD.
4. Repeated 2 and 3.

The initial value of K-SVD was a redundant DCT dictionary. The number of repetitions was 15. 12,800 randomly selected patches were used for one iteration of K-SVD.

K-SVD dictionary obtained from images with 25% loss

K-SVD dictionary obtained from images with 50% loss

K-SVD dictionary obtained from images with 75% missing

• In the case of 75% deficiency, learning did not progress much from the redundant DCT dictionary. In the first iteration, the atom and sparse representations are updated from the patch interpolated by OMP noise removal by the redundant DCT, so if the defect is 75%, it seems that learning did not proceed from there.

Impulse noise removal

The pixel value of 10% of the pixels was randomly increased by +50 or -50 to obtain a test image. The pixel value was clipped to a maximum of 255 and a minimum of 0. The difference from inpainting is that the position of the missing pixel is not known.

Impulse noise was removed with a median filter. The median filter kernel size is $ 3 \ times 3 $. Pixels in which the absolute value of the difference between the noisy image and the image denoised by the median filter was greater than 27 were defined as pixels contaminated with impulse noise.

PSNR was improved by applying a median filter only to the pixels detected by the estimation mask. The same was true for OMP denoising with a redundant DCT dictionary. The patch size for OMP noise removal by the redundant DCT dictionary is $ 8 \ times 8 $, the number of atoms is 256, and the OMP stop condition is $ k_ {0} = 256 $, $ \ epsilon = (8 ^ {2} -p) \ times 5 ^ {2} \ times 1.1 $.

Unimplemented application

• Global MCA with wavelet transform
• Higher resolution by dictionary learning (super resolution)

Summary

• Local MCA with K-SVD dictionary worked quite well. I would like to investigate the curvelet conversion because it seems to be applicable in practice.
• Inpainting seems to be able to improve the accuracy by involving K-SVD when the defect rate is low.
• Is a median filter sufficient to remove impulse noise?
• High resolution by dictionary learning could not be implemented ... It may be better to do it by deep learning, but since it is black (box), gray dictionary learning is also attractive.

reference