Fun with Filters and Frequencies!

Part 1.1 - The Finite Difference Operator

One important feature of images to be able to process is edges. These can often be more useful than looking at the values of pixels for determining what is in an image. One method for this is the finite difference operators Dx and Dy.

When convolved with an image, these operators will find the vertical and horizontal edges (respectfully). These edge images can be used to compute edge gradients, the magnitude of which represent overall lines in an image.

Part 1.2 - Derivative of Gaussian (DoG) Filter

One issue with the previous step is that the results are quite noisey and details are lost when thresholding is applied. Therefore we apply a gaussian blur to the image and repeat the same steps.

      kernel_size = 11 (sigma = 1.8)
          

Derivative of Gaussian

Apply a gaussian blur to an image can be an expensive process. We can significantly speed the process up by applying the blur to the finite difference operators, and then convolving those with the base image. This is possible due to the commutative property of convolutions, which is due to convolutions being linear operations.

      kernel_size = 11 (sigma = 1.8)
          

We observe that these results are the same as above.

Part 2.1 - Image Sharpening

Another applications of gaussian blurring is being able to extract high frequencies. This is done using the unsharp mask filter, which is the unit impulse subtracted by the gaussian filter.

This filter can be applied to an image and then added back onto the original image with some gain.

For each of the sharpened images, you can hover over them to view the original image for an easy comparison of the effects.

Taj Mahal

      kernel_size = 11 (sigma = 1.8)
      sharpen_alpha = 1
          

California Spring

Here is the same algorithm run on a few pictures I took. The first is of Albert Bierstadt's California Spring painted in 1875.

      kernel_size = 31 (sigma = 5.17)
      sharpen_alpha = 2
          

Lisbon

The second image is Lisbon, Portugal taken from Castelo de São Jorge.

      kernel_size = 21 (sigma = 3.5)
      sharpen_alpha = 2
          

Part 2.2 - Hybrid Images

These types of images are created by taking one image's low frequencies and combining them with a different image's high frequencies. This allows two different images to be viewed depending on the viewing distance. The reason we see two images is due to a feature of our perception where signals at certain frequences aren't as visible at different distances.

Each image is gaussian blurred at a specific sigma value (either sigma1 for low frequency images, or signma2 for high frequency images). This blurred image is subtracted from the original image only in the high frequency case. The sigmas for each example are displayed below.

Derek and Nutmeg

These are the example images provided with the assignment.

      kernel_size1 = 60 (sigma1 = 10)
      kernel_size2 = 84 (sigma2 = 14)
          

Keanu Reeves, Young and Old (Failure)

I consider it a failure due to the ribbon on his suit in the older picture. It's difficult to find any range that gets rid of it. This is probably due to the fact that there is essentially nothing in that part of the image on the young picture.

      kernel_size1 = 18 (sigma1 = 3)
      kernel_size2 = 18 (sigma2 = 3)
          

Hank Schrader Meme

This is a pretty convenient meme image because it's already aligned and shows two extreme expressions.

      kernel_size1 = 18 (sigma1 = 3)
      kernel_size2 = 24 (sigma2 = 4)
          

Looking at the FFT for the Hank Schrader Meme example (in this case done in grayscale so that the FFT is easier) we can see features in the FFT get merged into the single image.

Part 2.3 - Gaussian and Laplacian Stacks

Gaussian and Laplacian stacks are useful image filtering devices. Gaussian stacks are created by applying a gaussian filter to an image over and over, and saving out a copy each time. Laplacian stacks are created using the Gaussian Stack by subtracting adjacent layers to find the difference between each. This lets us extract a "band" of frequencies from the image based on the gaussian kernel size.

The laplacian images are shown using a normalization function that remaps the lowest and highest values in the frequency band in a range from [0, 1].

      kernel_size = 31 (sigma = 5.17)
      N = 5 (stack size)
          

Apple Gaussian

Apple Laplacian

Orange Gaussian

Orange Laplacian

Part 2.4 - Multiresolution Blending

Multiresolution blending is a technique for combining two images. It is done by breaking the images down into their respective laplacian stacks, effectively separating various frequency bands of each image. Then each image is blended together at these different bands using a different alpha mask, which has also been blurred for each band (gaussian stack). When combined there are no visible edges or borders on the final image.

The Oraple

Here is an example of the algorithm working on the Oraple example.

      kernel_size = 31 (sigma = 5.17)
      N = 5 (stack size)
          

The Footmelon

Can we also blend fruits with non-fruits? Science says "Why not?"

      kernel_size = 31 (sigma = 5.17)
      N = 5 (stack size)
          

The Hummingfish

If birds were real, then this wouldn't be possible... This method was done using a separate image as a mask, instead of splitting the image into two halves.

      kernel_size = 31 (sigma = 5.17)
      N = 5 (stack size)
          

Blending Details (Footmelon)

Here is an example of what each layer of the laplacian stacks look like for the footmelon example.

Layer 0

Layer 1

Layer 2

Layer 3

Layer 4
(Low Pass Gaussian)

Combined