[UNDER CONSTRUCTION]
Abstract
The Fractal Mosaic algorithm is a rotation, scale, and translation invariant photomosaic algorithm. While traditional photomosaics create a larger image out of smaller square images on a rectangular grid, Fractal Mosaics creates a larger image out of rectangular images that can be placed at arbitrary locations, can be scaled to any size, and can be rotated by any angle.
@@ -66,7 +65,7 @@As can be seen, using large tiles (left), finer details are lost. Using small tiles (right), we capture target image details, but can’t see individual image detail. Using medium size tiles (middle), we get a compromise between the two. Fractal Mosaics aims to create mosaics that capture both target image details and individual image details, in an aesthetically interesting way.
-Many people have, of course, created photomosaic variants with similar aims. Too many to list here. Some of my favorite photomosaic pioneers that I’m aware of: Tsevis, Village9991, and StellaMe. If I’m unaware of your work, contact me!
+Many people have, of course, created photomosaic variants with similar aims. Too many to list here. Some of my favorite photomosaic pioneers that I’m aware of: Tsevis, Village9991, and StellaMe. If I’m unaware of your work, contact me!
By allowing for arbitrary placement, rotation, and scaling of the “library images” (images making up the mosaic), there are an many more possible matches to evaluate. This allows for larger, often more aesthetically interesting matches than possible with traditional photomosaics. However, searching such a large solution space on a personal computer presents a challenging problem. This “paper” presents my solution to this problem (two years worth of unpaid work...). It explains the algorithm in depth for those wishing to hack the Matlab code. It should also aid those using the code to create mosaics, as understanding how the algorithm works will help tweak parameters that shape mosaic output.
@@ -171,16 +170,16 @@Recovering Translation
-As mentioned previously, Log-polar registration does not recover translation. It assumes the two images presented to it line up, except for rotation and scale differences. To recover translation, the algorithm must be applied to every location (pixel). The translation resulting in the best match (i.e. lowest rms error) is then chosen as the translation estimate. This makes log-polar registration fairly computationally expensive compared to other registration techniques.
+As mentioned previously, Log-polar registration does not recover translation. It assumes the two images presented to it line up, except for rotation and scale differences. To recover translation, the algorithm must be applied to every location (pixel). The translation resulting in the best match (i.e. lowest rms error) is then chosen as the translation estimate.
-To reduce computation, the authors of Log-polar registration implement a coarse-to-fine multi-resolution framework, whereby estimates computed in lower resolution images are used as initial guesses in higher resolution images.
-The video below shows the algorithm at work evaluating library images. As can be seen, most library images aren't evaluated beyond the lowest resolution (8x8). This is because the rms error of the rotated and scaled image after image registration is not below an empirically determined threshold. For details on error thresholds and other implementation details, see the Image Registration Algorithm Steps page.
+To reduce computation, the authors of Log-polar registration implement a multi-resolution framework, whereby estimates computed in lower resolution images are used as initial guesses in higher resolution images. At each resolution, rotation and scale estimates are computed for every location. The location(s) that result in an rms error below a threshold are sent to the next higher resolution for refining.
+The video below shows the algorithm at work evaluating library images. As can be seen, most library images aren't evaluated beyond the lowest resolution (8x8). This is because most library images, even when rotated and scaled to best match the target image, aren't good enough matches to evaluate at higher resolutions. For details on error thresholds and other implementation details, see the Image Registration Algorithm Steps page.
Image Search (Retrieval) Filtering
-An image search (i.e. retrieval) algorithm is used to filter out statistically unlikely matches before the computationally intensive image registration step (i.e., don’t waste cycles trying to match a white part of the target image with an image of the night sky). Specifically, Fractal Mosaics performs Content-Based Image Retrieval (CBIR, similar to Google’s search by image feature. The “search image” (the part of the target image we’re looking to match), is compared to every library image using a similarity metric. Then only the most similar library images are evaluated using image registration.
+An image search (i.e. retrieval) algorithm is used to filter out statistically unlikely matches before the computationally intensive image registration step (i.e., don’t waste cycles trying to match a white part of the target image with an image of the night sky). Specifically, Fractal Mosaics performs Content-Based Image Retrieval (CBIR), similar to Google’s search by image feature. The “search image” (the part of the target image we’re looking to match), is compared to every library image using a similarity metric. Then only the most similar library images are evaluated using image registration.
Numerous similarity metrics were evaluated, including histograms, standard deviation, Hu moments, General Fourier Descriptors (GFD), and Fourier Mellin (FM) shape descriptors. The metrics which performed best were histograms, GFDs, and standard deviation. These metrics were combined to create a custom similarity metric that slightly outperformed any individual similarity metric. This metric is currently only used in black and white mode, as I ran out of time to extend it to color. For color, a relatively simple yet effective similarity metric (histogram distance in the HSV color space) is used. Implementation details for black and white and color retrieval are given in the sections below.
Black and White
diff --git a/log_polar_registration_framework.html b/log_polar_registration_framework.html index 8d0cf27..777526c 100644 --- a/log_polar_registration_framework.html +++ b/log_polar_registration_framework.html @@ -48,9 +48,7 @@[UNDER CONSTRUCTION]
- -Image Registration Algorithm Steps
+Image Registration Algorithm Steps
The following page details how the multi-resolution framework from the paper “Robust Image Registration Using Log-Polar Transform” (pdf) was adapted to the photomosaic problem. It is assumed that you've already read the main Fractal Mosaics "paper" which gives an overview of the algorithm.
Essentially, we apply Log-polar registration on low resolution images, searching for matches, and only drill down to higher resolution images when the library image being evaluated produces a low enough rms error.