diff --git a/index.html b/index.html index 9937b6a..b9cee5e 100644 --- a/index.html +++ b/index.html @@ -52,7 +52,7 @@

The Fractal Mosaic algorithm is a rotation, scale, and translation invariant photomosaic algorithm. While traditional photomosaics create a larger image out of smaller square images on a rectangular grid, Fractal Mosaics creates a larger image out of rectangular images that can be placed at arbitrary locations, can be scaled to any size, and can be rotated by any angle.

-

This is accomplished with a combination of image search (i.e. image retrieval) algorithms and an image registration algorithm developed with NASA funding by Siavash Zokai and George Wolberg at the City College of New York (pdf).

+

This is accomplished with a combination of image search (i.e. image retrieval) algorithms and an image registration algorithm developed with NASA funding by Siavash Zokai and George Wolberg at the City College of New York (pdf).

Alt text

Figure 1: Fractal Mosaic created with graffiti image set (best viewed large)

@@ -105,7 +105,7 @@

After an initial survey of the image registration literature, a frequency domain image registration algorithm was evaluated; specifically, Fourier-Mellin-based image registration, as implemented in this Matlab code. This would have been much more computationally efficient than the spatial domain algorithm eventually chosen, as rotation, scale, and translation are efficiently recovered in one step. However, this technique proved not robust enough for the photomosaic application. This is likely because it relies more heavily on the assumption made by all image registration algorithms: that the two images being matched are of the same object/scene. In photomosaics however, we are matching parts of the target image to random library images. The assumption that the library images are rotated, scaled, and translated versions of the target image is weak.

-

An image registration algorithm that operates in the spatial domain was tried; specifically, the “Log-polar registration” algorithm found in the paper “Robust Image Registration Using Log-Polar Transform” (pdf). This algorithm proved very robust. And most importantly, it met the main aesthetic criteria: for any two images presented to the algorithm, it aligned them the way a human would. To test this claim out for yourself, check out the video below showing the algorithm at work matching library images to a target image.

+

An image registration algorithm that operates in the spatial domain was tried; specifically, the “Log-polar registration” algorithm found in the paper “Robust Image Registration Using Log-Polar Transform” (pdf). This algorithm proved very robust. And most importantly, it met the main aesthetic criteria: for any two images presented to the algorithm, it aligned them the way a human would. To test this claim out for yourself, check out the video below showing the algorithm at work matching library images to a target image.

The downside to Log-polar registration is increased computation. Because Log-polar registration only recovers rotation and scale, to recover translation, it must be applied to every location (pixel) in the target image. The location resulting in the lowest rms error is then chosen as the estimate. This creates a lot of extra computation. To mitigate this, the authors of the algorithm use a coarse-to-fine multi-resolution framework, whereby estimates computed in lower resolution images are used as initial guesses in higher resolution images. The following section reviews Log-polar registration in Fractal Mosaics.

diff --git a/log_polar_registration_framework.html b/log_polar_registration_framework.html index 9a29aa4..f6cc718 100644 --- a/log_polar_registration_framework.html +++ b/log_polar_registration_framework.html @@ -49,7 +49,7 @@

Image Registration Algorithm Steps

-

The following page details how the multi-resolution framework from the paper “Robust Image Registration Using Log-Polar Transform” (pdf) was adapted to the photomosaic problem. It is assumed that you've already read the main Fractal Mosaics "paper" which gives an overview of the algorithm.

+

The following page details how the multi-resolution framework from the paper “Robust Image Registration Using Log-Polar Transform” (pdf) was adapted to the photomosaic problem. It is assumed that you've already read the main Fractal Mosaics "paper" which gives an overview of the algorithm.

Essentially, we apply Log-polar registration on low resolution images, searching for matches, and only drill down to higher resolution images when the library image being evaluated produces a low enough rms error.

  1. Create low resolution versions of the target image
    2. @@ -108,7 +108,7 @@

    Levels

    -

    The steps described in the first two sections will produce a mosaic. However, the scale range of Log-polar registration will restrict the size of images in the mosaic. In the paper outlining Log-polar registration (pdf), the authors claim up to 10X scale recovery. In this author’s experiments, scale was only recovered reliably up to 2X. This is due to the fact that we’re not registering images of the same objects/scene, but rather random library images. If we were doing traditional image registration, this would be equivalent to matching images with extreme warping/distortion. Therefore, using the image resolutions presented thus far, the smallest image in the mosaic would be the size of a target tile (1/3 of the mosaic size), scaled down by 1/2, or 1/6th the size of the target image. This means the “resolution” of the overall mosaic is low. We can only have relatively large matches (tiles), some of which may not match the target image very well. To create a “higher resolution” mosaic, one where smaller library images fill in target image details, we introduce higher “levels”.

    +

    The steps described in the first two sections will produce a mosaic. However, the scale range of Log-polar registration will restrict the size of images in the mosaic. In the paper outlining Log-polar registration (pdf), the authors claim up to 10X scale recovery. In this author’s experiments, scale was only recovered reliably up to 2X. This is due to the fact that we’re not registering images of the same objects/scene, but rather random library images. If we were doing traditional image registration, this would be equivalent to matching images with extreme warping/distortion. Therefore, using the image resolutions presented thus far, the smallest image in the mosaic would be the size of a target tile (1/3 of the mosaic size), scaled down by 1/2, or 1/6th the size of the target image. This means the “resolution” of the overall mosaic is low. We can only have relatively large matches (tiles), some of which may not match the target image very well. To create a “higher resolution” mosaic, one where smaller library images fill in target image details, we introduce higher “levels”.

    Essentially, at each level we repeat the same algorithm steps, but start with a higher resolution “lowest resolution” target. At level one (L1), we use the 8x8 “lowest resolution” target image, as illustrated in Figure 3. For level 2 (L2), we start with a 16x16 lowest resolution target image, as illustrated in Figure 5. This does not change the resolutions used in Log-polar registration. The 16x16 lowest resolution image is still treated as if it were created from 8x8 tiles. There are just more 8x8 tiles to evaluate now, and those tiles are smaller relative to the actual size of the overall mosaic. Again, we create higher resolution versions of the target image from the lowest resolution image (8x8, 16x16....down to 128x128). At higher levels, the reference target images are simply larger.