Description
Problem 1: Image Demosaicing and Histogram Manipulation (50%)
(a) Bilinear Demosaicing (10%)
To capture color images, digital camera sensors are usually arranged in form of a color filter array
(CFA), called the Bayer array, as shown in Figure 1. Since each sensor at a pixel location only captures
one of the three primary colors (R, G, B), the other two colors have to be re-constructed based on their
neighbor pixel values to obtain the full color. Demosaicing is the process of translating this Bayer array
of primary colors into a color image that contains the R, G, B values at each pixel.
Figure 1: (a) Single CCD sensor covered by a CFA and (b) Bayer pattern [1].
Implement the simplest demosaicing method based on bilinear interpolation. Exemplary demosaicing
results are given in Figure 2. With this method, the missing color value at each pixel is approximated by
bilinear interpolation using the average of its two or four adjacent pixels of the same color. To give an
example, the missing blue and green values at pixel R3,4 are estimated as:
EE 569 Digital Image Processing: Homework #1
Professor C.-C. Jay Kuo Page 2 of 10
As for pixel G3,3, the blue and red values are calculated as:
(1) Apply the bilinear demosaicing to the cat image in Figure 3 and show your results.
(2) Do you observe any artifacts? If yes, explain the cause of the artifacts and provide your ideas to
improve the demosaicing performance.
Figure 2: (a) The original lighthouse image and (b) the demosaiced lighthouse image
by bilinear interpolation.
Figure 3: The cat image with the CFA sensor input.
EE 569 Digital Image Processing: Homework #1
Professor C.-C. Jay Kuo Page 3 of 10
(b) Malvar-He-Cutler (MHC) Demosaicing (20%)
Malvar et al. [1] proposed an improved linear interpolation demosaicing algorithm. It yields a higher
quality demosaicing result by adding a 2
nd
-order cross-channel correction term to the basic bilinear
demosaicing result. Both the bilinear and the MHC demosaicing results of the Fruit_Shop image are
shown in Figure 4.
Figure 4: Demosacing results of Fruit_Shop image: the CFA input (left), the bilinear demosaicing
result (middle) and the MHC demosaicing result (right)
The MHC algorithm is stated below.
To estimate a green component at a red pixel location, we have
G
ˆ
(i, j) = G
ˆ bl(i, j)+aDR (i, j)
(1)
where the 1st term at the right-hand-side (RHS) is the bilinear interpolation result given in (1) and the
2
nd term is a correction term. For the 2
nd term, alpha is a weight factor, and is the discrete 5-point
Laplacian of the red channel:
(2)
To estimate a red component at a green pixel location, we have
R
ˆ
(i, j) = R
ˆ bl(i, j)+ bDG (i, j)
(3)
where ΔG is a discrete 9-point Laplacian of the green channel.
To estimate a red component at a blue pixel location,
(4)
where ΔB is a discrete 9-point Laplacian of the blue channel. The weights control how much
correction is applied, and their default values are:
a =
1
2
,b =
5
8
,g =
3
4
(5)
R
DR (i, j) = R(i, j) –
1
4
(R(i – 2, j) + R(i + 2, j) + R(i, j – 2) + R(i, j + 2))
R
ˆ
(i, j) = R
ˆ
bl
(i, j)+gDB (i, j)
a,b,g
EE 569 Digital Image Processing: Homework #1
Professor C.-C. Jay Kuo Page 4 of 10
The above formulas can be generalized to missing color components at each sensor location.
Consequently, the MHC demosaicing can be implemented by convolution with a set of linear filters.
There are eight different filters for interpolating the different color components at different locations as
illustrated in Figure 5 [1].
Figure 5: Filter coefficients
(1) Implement the MHC linear demosaicing algorithm and apply it to the Cat image in Figure 3.
Show your results.
(2) Compare the MHC and the bilinear demosaicing results and explain the performance differences
between these two algorithms in your own words.
EE 569 Digital Image Processing: Homework #1
Professor C.-C. Jay Kuo Page 5 of 10
(c) Histogram Manipulation (20%)
Implement two histogram equalization techniques:
Method A: the transfer-function-based histogram equalization method
Method B: the cumulative-probability-based histogram equalization method
Enhance the contrast of images rose_dark and rose_bright in Figure 6 using both methods. Describe the
procedure and show final results with the following steps.
(1) Plot the histograms of all images (2 input and 4 output images). The figure should have the
intensity value as the x-axis and the number of pixels as the y-axis.
(2) Apply Method A to rose_dark and rose_bright and show the enhanced image. Plot the transfer
function for each testing image.
(3) Apply Method B to rose_dark and rose_bright and show the enhanced image. Plot the
cumulative histogram for each testing image.
(4) Discuss your observations on these two enhancement results. Do you have any idea to improve
the current result?
(5) Apply your implemented Method A and Method B to rose_mix in Figure 4 and show the result.
Can you get similar result as in previous part? If not, how to modify your implementation?
Please justify your answer with discussion.
Note that MATLAB users CANNOT use functions from the Image Processing Toolbox except
displaying function like imshow().
(a) rose_dark (b) rose_bright (c) rose_mix
Figure 6: Three rose images with different gray-scale histograms.
EE 569 Digital Image Processing: Homework #1
Professor C.-C. Jay Kuo Page 6 of 10
Problem 2: Image Denoising (50%)
(a) Gray-level image (20%)
Remove noise in the image in Figure 7(b), compare it with the original image in Figure 7(a), and answer
the following questions:
(a) (b)
Figure 7: (a) the original pepper image and (b) the pepper image with added noise.
(1) What is the type of embedded noise in Figure 7(b)?
(2) Apply a linear filter of size N by N to the noisy image. Compare the performance of two choices
of the filter parameters – the uniform weight function and the Gaussian weight function and
study the effect of the window size (i.e. parameter N). Plot the peak-to-signal-noise (PSNR)
value as a function of N for both weighting functions.
Note: The PSNR value between images I and Y can be calculated as:
PSNR(dB) =
(1)
where
MSE =
(2)
and where I is the noise-free image of size N M, Y is the filtered image of size N M, Max
for 8-bit image is 255.
(3) In most low-pass linear filters, we often see degradation of edges. However, using some nonlinear filters, we can preserve the edges. Bilateral filters are one such kind of filters. A discrete
bilateral filter is given by:
(3)
(
where is the neighboring pixel location within the window centered around , I is the
image with noise, Y is the filtered image. and are the spread parameters. Implement the
bilateral denoising filter and apply it to the noisy image.
EE 569 Digital Image Processing: Homework #1
Professor C.-C. Jay Kuo Page 7 of 10
(4) The non-local mean filter utilizes the pixel value from a larger region rather the mean of a local
window centered around the target pixel. A discrete non-local mean filter with Gaussian
weighting function is as follows:
(5)
(6)
where I, Y are the noisy and filtered images respectively, is the window centered around
location (x, y), and is the filtering parameter,
and denote the window size of
your choice.
The Gaussian weighted Euclidian distance between window and is defined as:
(7)
and
(8)
where denotes the local neighborhood centered at the origin, denotes the relative
position in the neighborhood window. a > 0 is the standard deviation of the Gaussian kernel.
Implement the non-local mean filter and apply to the noisy image.
(a) (b)
Figure 8: (a) the original rose image (b) the rose image with mixed noise.
(b) Color image (20%)
Figure 8 (b) is a noisy color image, where each channel is corrupted by both impulse and uniform
noises. Try your best to remove noise and compare the result with the original image as shown in Figure
EE 569 Digital Image Processing: Homework #1
Professor C.-C. Jay Kuo Page 8 of 10
8(a). Discuss your denoising strategy and answer the following questions:
(1) Should you perform filtering on individual channels separately for both noise types?
(2) What filters would you like use to remove mixed noise?
(3) Can you cascade these filters in any order? Justify your answer.
(4) It could be difficult to remove uniform noise satisfactorily. Filters may blur the object’s edge
when smoothing noise. A successful design of a uniform noise filter depends on its ability to
distinguish the pattern between the noise and the edges. Please suggest an alternative to low pass
filter with Gaussian weight function and discuss why such solution can potentially work better.
(You are welcome to implement the filter, but it is not required)
(c) Shot noise (10%)
Electronic camera image sensor, especially the CMOS sensor typically has noise in the dark parts of the
captured image. Such noise is called shot noise, which is caused by statistical quantum (the photon)
fluctuations.
(a) (b)
Figure 9: (a) the original pepper image and (b) the pepper image with shot noise.
Figure 9(b) shows a typical image with shot noise. Unlike uniform noise, shot noise is Poisson
distributed. A common solution is firstly applying the Anscombe root transformation on each pixel z:
to the input image, resulting an image with additive Gaussian noise of unit variance. Then, a
conventional denoising filter for Gaussian noise is used. The denoised image is finally inverse
transformed. The detailed denoising steps can be found in [3].
(1) Implement the denoising method in [3] and apply it to Figure 9(b), where you can remove the
additive Gaussian noise using two methods.
Use a Gaussian low pass filter
Use the MATLAB code for block-matching and 3-D (BM3D) transform from [4].
(2) Show the final noise-removed pepper images and compare the PSNR values and visual quality of
the resulting images.
EE 569 Digital Image Processing: Homework #1
Professor C.-C. Jay Kuo Page 9 of 10
EE 569 Digital Image Processing: Homework #1
Professor C.-C. Jay Kuo Page 10 of 10
Appendix:
Problem 1: Image Demosaicing and Histogram Manipulation
cat_ori.raw 390x300x3 24-bit color(RGB)
cat.raw 390×300 8-bit gray
rose_ori.raw 400×400 8-bit gray
rose_dark.raw 400×400 8-bit gray
rose_bright.raw 400×400 8-bit gray
rose_mix.raw 400×400 8-bit gray
Problem 2: Noise Removal
pepper.raw 256×256 8-bit gray
pepper_noise.raw 256×256 8-bit gray
rose_color.raw 256x256x3 8-bit color(RGB)
rose_color_noise.raw 256x256x3 8-bit color(RGB)
pepper_dark.raw 256×256 8-bit gray
pepper_dark_noise.raw 256×256 8-bit gray
Reference Images
All images in this homework are from the USC-SIPI image database [5] or Google images [6].
References
[1] M. E. Celebi et al. (eds.), Color Image and Video Enhancement
[2] Malvar, Henrique S., Li-wei He, and Ross Cutler. “High-quality linear interpolation for demosaicing
of Bayer-patterned color images.” Acoustics, Speech, and Signal Processing, 2004. Proceedings.
(ICASSP’04). IEEE International Conference on. Vol. 3. IEEE, 2004.
[3] Makitalo, Markku, and Alessandro Foi. “Optimal inversion of the Anscombe transformation in lowcount Poisson image denoising.” IEEE transactions on Image Processing 20.1 (2011): 99-109.
[4] [Online]. Available: https://www.cs.tut.fi/~foi/GCF-BM3D/
[5] [Online] https://sipi.usc.edu/database/
[6] [Online] https://images.google.com/
