Reflection suppression

Taking a picture from a glossy medium can be quite a challenge. This post presents how we can do it using classical computer vision and several images with different perspectives. This example is an adaptation of a work performed for a client on 16-bit RAW NDA images. In this example, we will simplify the problem using grayscale 8-bit images.

Figure.0 - Base images

Affine registration

Rigid registration can correct for translation, rotation, shearing, and scaling. In the first step, we performed a rigid registration using SIFT feature detector and matching the best features between images. This classical technique works well with detailed, well-defined images, as in this example. Another option is to use ECC-based registration for challenging data with less robust features. By merging all the images (Figure 1), we see that this method corrects the large deformation but that there are slight misalignments in some parts of the picture. These misalignments are because the image on the paper is not flat, and the camera tilt is slightly changed. Therefore, local deformations that cannot be corrected using affine warping are still present.

Figure.1 - Feature-based registration

Non-rigid registration

To correct local deformations, we use the so-called optical flow. We find that a dense optical flow using the DeepFlow algorithm yield the best result for this type of image. This process can be pretty intensive for large images. Sparse optical flow with previously tracked features can be an option to speed up the process.

Figure.2 - Optical flow based registration

Reflection removal

Finally, we remove reflections on the glossy surface by taking the minimum projection of the finely warped images. In the case of a color image, we can convert the images to the HSV color space and compare pixel intensities on the V channel to find which pixels to select to reconstruct the final image. We then select the four corners of the image, perform a simple perspective warp, and do unsharp masking and a contrast enhancement to obtain a clean flat mat appearance.

Figure.3 - Final rendering