## Description

1. Consider the following motion models: translation, rigid (translation + rotation), rigid and equal scaling in

X,Y directions, rigid and unequal scaling in X,Y directions, affine, non-rigid. Consider each of the following

applications separately.

In each case, identify what is the optimal motion model and justify. Do not needlessly

pick a more complex motion model if it is not needed. For example, in some cases, a rotation is enough, in

which case please do not choose affine as it has additional degrees of freedom. [15 points]

(a) Consider that you have scanned a document twice with the same scanner, when the document was

potentially in slightly different positions. You now want to align these two images. Which motion

model is needed here? Assume there is no stretching or bending of the paper.

(b) In the earlier example, consider that the two images were respectively acquired from two different

scanners with different resolutions. Assume that for both scanners, the X and Y resolutions were the

same. Which motion model is needed here? Assume there is no stretching or bending of the paper.

(c) Consider a document with words written on both sides with ink. When you scan such a document from

one side, some portions from the other side are visible. This is called ‘ink bleeding’. To remove bleeding

artifacts, you need to acquire images of both sides of the document and first align them. Which motion

model is needed here? Assume there is no stretching or bending of the paper.

2. You are viewing the following graph from a research paper. Unfortunately, from the graph, the (x, y) values

at only a few points can be observed. You need to obtain the (x, y) values at many other points. Hence you

can do the following: you extract the image from the paper, and open it through MATLAB which provides a

function called impixelinfo.

This function gives you the (x, y) coordinates of any spatial location pointed by

your mouse. However, the coordinate system of the graph and that of MATLAB will be different. Describe

a procedure to convert from MATLAB’s coordinate system to the coordinate system of the graph. This will

help you obtain the (x, y) coordinates in the coordinate system of the graph.

Support your answer with

suitable equations. There is no need to write any code for this. [15 points]

3. Consider two images I and J whose intensity values (in each location) are randomly drawn from the known

probability mass functions (PMFs) pI (i) and pJ (j) respectively. Derive an expression for the PMF of the

image I + J. The expression resembles which operation that we are currently studying in class ? (will be

studying very soon) [15+5=20 points]

4. Read in the images T1.jpg and T2.jpg from the homework folder using the MATLAB function imread and

cast them as a double array. Let us call these images as J1 and J2. These are magnetic resonance images

of a portion of the human brain, acquired with different settings of the MRI machine.

They both represent

the same anatomical structures and are perfectly aligned (i.e. any pixel at location (x, y) in both images

represents the exact same physical entity). We are going to perform a simulation experiment for image

alignment in a setting where the image intensities of physically corresponding pixels are different.

To this

end, do as follows:

(a) Write a piece of MATLAB code to rotate the second image by θ = 28.5 degrees anti-clockwise. You

can use the imrotate function in MATLAB to implement the rotation using any interpolation method.

Note that the rotation is performed implicitly about the centroid of the image. While doing so, assign

a value of 0 to unoccupied pixels. Let us denote the rotated version of J2 as J3.

(b) Our job will now be to align J3 with J1 keeping J1 fixed. To this end, we will do a brute-force search

over θ ranging from -45 to +45 degrees in steps of 1 degree. For each θ, apply the rotation to J3 to

create an intermediate image J4, and compute the following measures of dependence between J1 and

J4:

the normalized cross-correlation (NCC)

the joint entropy (JE)

a measure of dependence called quadratic mutual information (QMI) defined as P

i1

P

i2

(pI1I2

(i1, i2)−

pI1

(i1)pI2

(i2))2

, where pI1I2

(i1, i2) represents the normalized joint histogram (i.e., joint pmf) of I1

and I2 (‘normalized’ means that the entries sum up to one).

Here, the random variables I1, I2

denote the pixel intensities from the two images respectively. For computing the joint histogram,

use a bin-width of 10 in both I1 and I2. For computing the marginal histograms pI1

and pI2

, you

need to integrate the joint histogram along one of the two directions respectively. You should write

your own joint histogram routine in MATLAB – do not use any inbuilt functions for it.

(c) Plot separate graphs of the values of NCC, JE, QMI versus θ and include them in the report PDF.

(d) Determine the optimal rotation between J3 and J1 using each of these three measures. What do you

observe from the plots with regard to estimating the rotation? Explain in the report PDF.

(e) For the optimal rotation using JE, plot the joint histogram between J1 and J4 using the imagesc

function in MATLAB along with colorbar. Include it in the report PDF.

(f) We have studied NCC and JE in class. What is the intuition regarding QMI? Explain in the report PDF. (Hint: When would random variables I1 and I2 be considered statistically independent?)

[2+10+2+3+3+5=25 points]

5. Read in the images ‘goi1.jpg’ and ‘goi2.jpg’ from the homework folder using the MATLAB imread function

and cast them as double. These are images of the Gateway of India acquired from two different viewpoints.

As such, no motion model we have studied in class is really adequate for representing the motion between

these images, but it turns out that an affine model is a reasonably good approximation, and you will see

this.

We will estimate the affine transformation between these two images in the following manner:

(a) Display both images using imshow(im1) and imshow(im2) in MATLAB. Use the ginput function of

MATLAB to manually select (via an easy graphical user interface) and store n = 12 pairs of physically

corresponding salient feature points from both the images. For this, you can do the following:

for i=1:12, figure(1); imshow(im1/255); [x1(i), y1(i)] = ginput(1);

figure(2); imshow(im2/255); [x2(i), y2(i)] = ginput(1);

Tips: Avoid selecting points which are visible in only one image. Try to select them as accurately as possible, but our procedure is robust to small sub-pixel errors. Make sure x1(i),y1(i) and x2(i),y2(i)

are actually physically corresponding points. Salient feature points are typically points that represent

corners of various structures.

(b) Write MATLAB code to determine the affine transformation which converts the first image (‘goi1’) into

the second one (‘goi2’).

(c) Using nearest neighbor interpolation that you should implement yourself, warp the first image with the

affine transformation matrix determined in the previous step, so that it is now better aligned with the

second image. You are not allowed to use any implementation for this already available in MATLAB.

Display all three images side by side in the report PDF.

(d) Repeat the previous step with bilinear interpolation that you should implement yourself. You are not

allowed to use any implementation for this already available in MATLAB. Display all three images side

by side in the report PDF.

(e) In the first step, suppose that the n points you chose in the first image happened to be collinear. Explain

(in the report PDF) the effect on the estimation of the affine transformation matrix. [5+5+5+5+5=25

points]