Project Overview
This project focuses on reconstructing color images from the digitized glass plate negatives of Sergei Mikhailovich Prokudin-Gorskii, a Russian photographer who pioneered color photography in the early 1900s. Prokudin-Gorskii captured thousands of color photographs by taking three separate exposures of each scene using red, green, and blue filters, recording them onto glass plates.
The challenge lies in automatically aligning these three color channel images to produce a single, artifact-free RGB color image. For this project, I explored grid searching to find that optimal alignment, and using an image pyramid to speed things up for larger images.
Algorithm
The algorithm starts with preprocessing. I first convert the images to float for memory efficiency, then separate the images into their respective color channels. With each channel sepearted, I then demean each image, crop out the border (roughly 5% of each side), then find the neccesary displacements along each axis to align the blue channel to the green channel, and I do the same for aligning the red channel to the green channel. The green channel serves as the reference channel for alignment.
Single-Scale Alignment
For low resolution images, it is sufficient to search over a small window of possible displacements (e.g. [-15, 15] pixels), and compute a similarity metric for each candidate displacement value. The displacement values that perform best with that given metric are chosen, and I align the input channel to the reference channel with those displacement values. Once the blue channel and red channel are sufficiently aligned relative to the reference channel, all three channels can be stacked to form the final color image! We have successfully brought color to the Russian Empire!
Image Matching Metrics
Picking which metric to use is an important design choice that will effect the performance of the algorithm, especially for tricker images like that of the Emir of Bukhara. I first experimented with using simply just the L2 norm (Euclidean Distance) to measure the similarity between the channel and the reference channel(Note that I1 and I2 are two images being compared).
While this metric is simple and computationally efficient, for certain images it is not robust enough to accurately align the channels. This can be seen below with a somewhat hazy image of this monastery.
To remedy this, and introduce more robustness to the alignment algorithm I used the Normalized Cross-Correlation (NCC) metric. Using this, and finding the alignments where NCC is maximized provided significantly better results.
Multiscale Implementation and Image Pyramid
For large images, exhaustive search becomes prohibitively expensive. To account for large resolution images, I implemented an image pyramid approach that processes images at multiple scales, starting from the coarsest (smallest) scale and refining the alignment estimate as it progresses down the pyramid to finer images. In particular, each image size is doubled from one layer of the pyramid to the next. This means that each step down the pyramid, we are much closer to our optimal displacements, allowing us to search over a smaller window, improving the performance of the algorithm dramatically. For this project specifically, I chose to half the search space each time I traverse a layer down the pyramid. This allowed for accurate, but computationally efficient alignment for large image sizes
Results
Below are the results of my color reconstruction algorithm on various Prokudin-Gorskii images. Each image shows the final aligned color reconstruction along with the calculated displacement vectors for both blue-to-green and red-to-green alignments.
Red → Green: (1, 7) pixels
Red → Green: (1, 6) pixels
Red → Green: (18, 56) pixels
Red → Green: (-8, 34) pixels
Red → Green: (-2, 64) pixels
Red → Green: (6, 48) pixels
Red → Green: (14, 38) pixels
Red → Green: (-6, 78) pixels
Red → Green: (-12, 52) pixels
Red → Green: (4, 96) pixels
Red → Green: (8, 98) pixels
Red → Green: (-18, 46) pixels
Red → Green: (-2, 58) pixels
Red → Green: (1, 4) pixels
Personal Images
I also tested my algorithm on additional images from the Prokudin-Gorskii collection to demonstrate its robustness across different types of scenes and photographic conditions. The owls are my favorite!
Red → Green: (34, 52) pixels
Red → Green: (20, 82) pixels
Red → Green: (16, 78) pixels
Conclusion
By using modern image processing techniques it is possible with relative ease to bring beautiful color to Sergei Prokudin-Gorskii's original collection. Further improvements can be made by experimenting with different metrics as well as exploring possibly better representation of pixel features, but using very simple tools and techniques, we are able to create beautiful color images of the Russian Empire.