COMPSCI 180 Project 1: Images of the Russian Empire: Colorizing the Prokudin-Gorskii Photo Collection

Kaitlyn Chen

Background:

In the early 1900’s, a Russian photographer named Sergei Mikhailovich Prokudin-Gorskii traveled across the Russian Empire to capture thousands of colored photographs. He did so by taking 3 versions of each photograph (using red, green, and blue filters) on a glass plate. In this project I digitalized the 3 RGB glass plate negatives to produce a single RGB colored photograph.

Overview:

The input images throughout this project are .jpg and .tif files, where the 3 color chanels are stacked on top of each other as shown on the right. We can align the images overtop of each other to create a single RGB image. However this alignment can be tricky, so the goal of this project is to write an algorithm to align these three negatives accurately and quickly. Two algorithms are explored in this project.

Naive Alignment Algorithm:

In order to measure the accuracy of our alignments, we need a matching metric. 2 different matching metrics were considered:

• L2 Norm :

  $$\sqrt{\sum{( \text{image1} - \text{image2} )^2}}$$

• Normalized Cross Correlation:

  $$\frac{\mathbf{image1} \cdot \mathbf{image2}}{\|\mathbf{image1}\| \|\mathbf{image2}\|}$$

This single scale implementation worked best on low resolution images such as the .jpg images. This algorithm simply did an exhaustive search over all horizontal and vertical offsets in a range of -15 to 15, and chose the offset with the best matching metric score. This metric score was only computed on the internal pixels of each negative since the boarders of the images tamper with the true image, so only 80% of the image was used when calculating L2 scores and 90% of the image was used when calculating NCC scores. I noticed that the NCC matching metric was more robust to the boarders so using 80% versus 90% of the image did not change the results. The two generated similar results, however the NCC matching metric was quite slower compared to L2.

Recursive Image Pyramid Alignment Algorithm:

The naive exhaustive search algorithm proved to only work well on low resolution images like .jpg images, and not on high resolution images like .tif images. This multi-scale algorithm recursively scales down the image by half until it reaches a certain size (I used under 500 pixels), at which it computes and returns the best shifting offset using the exhaustive search method (iterating through range of -15 to 15). Now as the recursion unfolds to the larger versions of the image, the offset is doubled and a local search around +2/-2 of this halved offset for the best offset for this larger image is returned. This allows our algorithm to be much faster than the single scale one. Again, L2 matching metric was used here as it proved to be more efficient.

Results:

Note: **Images attached are compressed version due to limitability of upload on this website**

More Results Gallery:

Note: **Images attached are compressed version due to limitability of upload on this website**

Conclusions:

If I had more time I would have liked to implement auto cropping to give the images a finalized look, such that the colored edges around the image are eliminated. Also would have implemented edge detection to better align the images, particularly needed to emir.tif and monastery.jpg. We can see that image pyramiding overall works quite well on most images however improvements would be necessary for others.