EE 569 Digital Image Processing: Homework #1 solution

Description

Problem 1: Image Demosaicing and Histogram Manipulation (50%)
(a) Bilinear Demosaicing (10%)
To capture color images, digital camera sensors are usually arranged in form of a color filter array (CFA),
called the Bayer array, as shown in Figure 1. Since each sensor at a pixel location only captures one of the
three primary colors (R, G, B), the other two colors have to be re-constructed based on their neighbor
pixel values to obtain the full color. Demosaicing is the process of translating this Bayer array of primary
colors into a color image that contains the R, G, B values at each pixel.
Figure 1: (a) Single CCD sensor covered by a CFA and (b) Bayer pattern [1].
Implement the simplest demosaicing method based on bilinear interpolation. Exemplary demosaicing
results are given in Figure 2. With this method, the missing color value at each pixel is approximated by
bilinear interpolation using the average of its two or four adjacent pixels of the same color. To give an
example, the missing blue and green values at pixel 𝑅3,4 are estimated as:
As for pixel 𝐺3,3, the blue and red values are calculated as:
EE 569 Digital Image Processing: Homework #1
Professor C.-C. Jay Kuo Page 2 of 8

(a) (b)
Figure 2: (a) The original image and (b) the demosaiced image by bilinear interpolation.
Figure 3: The dog image with the CFA sensor input.
(1) Apply the bilinear demosaicing to the Dog image in Figure 3 and show your results.
(2) Compare your demosaiced image with the ground truth color image Dog_ori. Do you observe any
artifacts? If yes, explain the cause of the artifacts and provide your ideas to improve the
demosaicing performance.
(b) Malvar-He-Cutler (MHC) Demosaicing (20%)
Malvar et al. [2] proposed an improved linear interpolation demosaicing algorithm. It yields a higher
quality demosaicing result by adding a 2nd-order cross-channel correction term to the basic bilinear
demosaicing result. Both the bilinear and the MHC demosaicing results of the Fruit_Shop image are
shown in Figure 4.
EE 569 Digital Image Processing: Homework #1
Professor C.-C. Jay Kuo Page 3 of 8

Figure 4: Demosacing results of Fruit_Shop image: the CFA input (left), the bilinear
demosaicing result (middle) and the MHC demosaicing result (right).
The MHC algorithm is stated below.
To estimate a green component at a red pixel location, we have
where 𝐺̂𝑏𝑙 is the bilinear interpolation result and the 2
nd term is a correction term. For the 2
nd term, 𝛼 is a
weight factor, and Δ𝑅 is the discrete 5-point Laplacian of the red channel:
To estimate a red component at a green pixel location, we have
where 𝛥𝐺 is a discrete 9-point Laplacian of the green channel.
To estimate a red component at a blue pixel location,
where 𝛥𝐵 is a discrete 9-point Laplacian of the blue channel. The weights control how much correction
is applied, and their default values are:
The above formulas can be generalized to missing color components at each sensor location.
Consequently, the MHC demosaicing can be implemented by convolution with a set of linear filters. There
are eight different filters for interpolating the different color components at different locations as
illustrated in Figure 5 [2].
EE 569 Digital Image Processing: Homework #1
Professor C.-C. Jay Kuo Page 4 of 8
Figure 5: Filter coefficients.
(1) Implement the MHC linear demosaicing algorithm and apply it to the Dog image in Figure 3.
Show your results.
(2) Compare the MHC and the bilinear demosaicing results and explain the performance differences
between these two algorithms in your own words.
(c) Histogram Manipulation (20%)
Implement two histogram equalization techniques:
• Method A: the transfer-function-based histogram equalization method,
• Method B: the cumulative-probability-based histogram equalization method
to enhance the contrast of the Toy image in Figure 6 below.
(1) Plot the histograms of the red, green and blue channels of the original image. The figure should have
the intensity value as the x-axis and the number of pixels as the y-axis.
(2) Apply Method A to the original image and show the enhanced image. Plot the transfer function for
each channel.
(3) Apply Method B to the original image and show the enhanced image. Plot the cumulative histogram
for each channel.
(4) Discuss your observations on these two enhancement results. Do you have any idea to improve the
current result?
EE 569 Digital Image Processing: Homework #1
Professor C.-C. Jay Kuo Page 5 of 8
Note that MATLAB users CANNOT use functions from the Image Processing Toolbox except displaying
function like imshow().
Figure 6: Toy image
Problem 2: Image Denoising (50 %)
In this problem, you will implement a set of denoising algorithms to improve image quality. You can use
the PSNR (peak-signal-to-noise-ratio) quality metric to assess the performance of your denoising
algorithm. The PSNR value for R, G, B channels can be, respectively, calculated as follows:
Remove noise in the image in Figure 7(b), compare it with the original image in Figure 7(a), and answer
the following questions:
(a) Basic denoising methods (10%)
(1) What is the type of embedded noise in Figure 7(b)?
(2) Apply a linear filter of size N by N to the noisy image. Compare the performance of two choices
of the filter parameters – the uniform weight function and the Gaussian weight function.
EE 569 Digital Image Processing: Homework #1
Professor C.-C. Jay Kuo Page 6 of 8

Figure 7: The original and noisy corn images.
(b) Bilateral Filtering (10%)
In most low-pass linear filters, we often see degradation of edges. However, using some nonlinear filters,
we can preserve the edges. Bilateral filters are one such kind of filters. A discrete bilateral filter is given
by:
where (𝑘, 𝑙) is the neighboring pixel location within the window centered around (𝑖,𝑗), 𝐼 is the image with
noise, 𝑌 is the filtered image. 𝜎𝑐 and 𝜎𝑠 are the spread parameters.
(1) Implement the bilateral denoising filter and apply it to the noisy image.
(2) Explain the roles of 𝜎𝑐 and 𝜎𝑠
. Discuss the change in filter’s performance with respect to the values
of 𝜎𝑐 and 𝜎𝑠
.
(3) Does this filter perform better than linear filters you implemented in Problem 3(a)? Justify your
answer in words.
(c) Non-Local Means (NLM) Filtering (10%)
The non-local mean filter utilizes the pixel value from a larger region rather the mean of a local window
centered around the target pixel. A discrete non-local mean filter with Gaussian weighting function is as
follows:
where 𝐼, 𝑌 are the noisy and filtered images respectively, 𝑁𝑥,𝑦 is the window centered around location
(𝑥, 𝑦), and ℎ is the filtering parameter, 𝑁
′ ≤ 𝑁 and 𝑀′ < 𝑀 denote the window size of your choice.
The Gaussian weighted Euclidian distance between window 𝐼(𝑁𝑖,𝑗)and 𝐼(𝑁𝑘,𝑙) is defined as:
‖𝐼(𝑁𝑖,𝑗) − 𝐼(𝑁𝑘,𝑙)‖
2,𝑎
2
= ∑ 𝐺𝑎
(𝑛1, 𝑛2
)(𝐼(𝑖 − 𝑛1,𝑗 − 𝑛2
) − 𝐼(𝑘 − 𝑛1, 𝑙 − 𝑛2
))
𝑛1,𝑛2∈ℵ
2
EE 569 Digital Image Processing: Homework #1
Professor C.-C. Jay Kuo Page 7 of 8
where ℵ denotes the local neighborhood centered at the origin, 𝑛1
, 𝑛2 ∈ ℵ denotes the relative position in the
neighborhood window. 𝑎 > 0 is the standard deviation of the Gaussian kernel.
(1) Implement the NLM filter and apply it to the noisy image. Try several filter parameters and
discuss their effect on filtering process. Clearly state your final choice of parameters in your
report.
(2) Compare the performance of NLM with filters used in Problem 3(a) and Problem 3(b).
(d) Block matching and 3-D (BM3D) transform filter (10%)
In this part, you will get familiar with another state-of-the-art denoising algorithm proposed in [3].
(1) Please explain the BM3D algorithm in your own words.
(2) Implement the BM3D filter (Write your own code or use any available online source code but
include the source in your reference) to denoise the noisy image House in Figure 7. Show the final
noise-removed image, and compare the PSNR value and visual quality with those in (a).
Note: It is recommended that you use the code provided by the authors on their website [4]. Their code is
written in MATLAB.
(e) Mixed noises in color image (10%)
Figure 8 (b) is a noisy color image corrupted with mixed types of noises. Please identify noise types in the
image and answer the following questions:
(1) What types of noises are there?
(2) Should you perform filtering on individual channels separately for both noise types?
(3) What filters would you like use to remove mixed noise? Can you cascade these filters in any order?
Justify your answer.

(a) (b)
Figure 8: (a) the original pepper image (b) the pepper image with mixed noises.
EE 569 Digital Image Processing: Homework #1
Professor C.-C. Jay Kuo Page 8 of 8
Appendix:
Problem 1: Image Demosaicing and Histogram Manipulation
Dog.raw 600×532 8-bit gray
Dog_ori.raw 600×532 24-bit color(RGB)
Toy.raw 560×400 24-bit color(RGB)
Problem 2: Image Denoising
Corn_gray.raw 320×320 8-bit gray
Corn_noisy.raw 320×320 8-bit gray
Reference Images
All images in this homework are from Google images [5] or the USC-SIPI image database [6].
References
[1] M. E. Celebi et al. (eds.), Color Image and Video Enhancement.
[2] Malvar, Henrique S., Li-wei He, and Ross Cutler. “High-quality linear interpolation for demosaicing
of Bayer-patterned color images.” 2004 IEEE International Conference on Acoustics, Speech, and Signal
Processing. Vol. 3. IEEE, 2004.
[3] Dabov, Kostadin, et al. “Image denoising with block-matching and 3D filtering.” Image Processing:
Algorithms and Systems, Neural Networks, and Machine Learning. Vol. 6064. International Society for
Optics and Photonics, 2006.
[4] [Online] Available: https://www.cs.tut.fi/~foi/GCF-BM3D/
[5] [Online] https://images.google.com/
[6] [Online] https://sipi.usc.edu/database/