## Description

Use the Dynamic Mode Decomposition method on the video clips ski drop.mov and monte carlo.mov containing a foreground and background object and separate the video stream to both the foreground video and a

background.

The DMD spectrum of frequencies can be used to subtract background modes. Specifically, assume that ωp,

where p ∈ {1, 2, . . . , `}, satisfies kωpk ≈ 0, and that kωjk ∀ j 6= p is bounded away from zero. Thus,

XDMD = bpϕp

e

ωpt

| {z }

Background Video

+

X

j6=p

bjϕj

e

ωj t

| {z }

Foreground Video

(1)

Assuming that X ∈ R

n×m, then a proper DMD reconstruction should also produce XDMD ∈ R

n×m. However, each

term of the DMD reconstruction is complex: bjϕj

exp (ωjt) ∈ C

n×m ∀j, though they sum to a real-valued matrix.

This poses a problem when separating the DMD terms into approximate low-rank and sparse reconstructions

because real-valued outputs are desired and knowing how to handle the complex elements can make a significant

difference in the accuracy of the results. Consider calculating the DMD’s approximate low-rank reconstruction

according to

XLow-Rank

DMD = bpϕp

e

ωpt

.

Since it should be true that

X = XLow-Rank

DMD + X

Sparse

DMD ,

then the DMD’s approximate sparse reconstruction,

X

Sparse

DMD =

X

j6=p

bjϕj

e

ωj t

,

can be calculated with real-valued elements only as follows. . .

X

Sparse

DMD = X −

XLow-Rank

DMD

,

where | · | yields the modulus of each element within the matrix. However, this may result in X

Sparse

DMD having

negative values in some of its elements, which would not make sense in terms of having negative pixel intensities.

These residual negative values can be put into a n×m matrix R and then be added back into XLow-Rank

DMD as follows:

XLow-Rank

DMD ← R +

XLow-Rank

DMD

X

Sparse

DMD ← X

Sparse

DMD − R

This way the magnitudes of the complex values from the DMD reconstruction are accounted for, while maintaining

the important constraints that

X = XLow-Rank

DMD + X

Sparse

DMD ,

so that none of the pixel intensities are below zero, and ensuring that the approximate low-rank and sparse DMD

reconstructions are real-valued. This method seems to work well empirically.

1