Iris art is a niche in photography that consists in taking a close-up picture of the human eye to produce an outstanding view of the iris, enhancing colors and texture unique to each person. The artist generally produces an outstanding image following either a multi-images or single-image approach.
The muti-images approach starts with the photographer taking several pictures of the eye with subtle changes in lighting and eyelid position. The artist combines these images to remove eyelid occlusions, glares, and eyelash reflections and recover the entire iris with high fidelity.
With the single-image approach, the photographer takes only one image. Then the artist will use inpainting technics to recover the iris.
These two methods produce outstanding results, and each has its own challenges. The first challenge is to get precise iris segmentation to single out the iris (colored part) and get rides of the sclera (white part). To produce an esthetic image, the artist has to round the iris and center the pupil (black part). He also reduces the pupil to keep the same ratio between the iris and pupil size for all the pictures, facilitating subsequent iris composition. Finally, he must remove lighting glare, eyelid occlusions, and eyelids reflections to recover the entire iris.
The multi-images approach will solve the last challenge by using image combinations. For that, each image has to be precisely adjusted according to the others; the eye being spherical, a non-rigid registration has to be performed to get a pixel accuracy alignment. The precise color tones and brightness must be matched between images to produce a coherent result. Then the defects must be detected, and images must be smartly combined to remove them. The non-rigid registration is technically complicated to implement, working with high-resolution images in a reasonable amount of time. Automatically finding an algorithm to combine images to eliminate all the defects (bright and dark reflections) is tricky.
The single-image approach solves this challenge using image inpainting, basically copying other parts of the iris and seamless-pasting them to remove defects and recover missing parts of the iris. It is easier and faster than the multi-image approach, at the expense of iris fidelity.
After I had worked with several art galleries to automatize the analysis process, both for multi and single-image, using classical computer vision, I developed a new approach using deep learning to speed up and simplify the iris segmentation and rounding as well as the pupil reduction and centering.
DeepIris is a suite of tools that automatically detect and segment the iris and pupil with great accuracy using deep learning. The algorithm can work from a wide variety of images, from whole-face pictures to eye close-up at any image resolution and for any ratio size of the iris vs image size.
The iris is extracted from the image and rounded using image morphing or cropped, fitting the best circle to gain time, as desired.
The pupil is then automatically detected, reduced to a predefined size, and centered with the iris using a smooth transformation, producing a standardized and esthetic result.
DeepIris will be available for professionals either as a standalone application or to be integrated into an existing pipeline. Fine tailoring can be performed for each gallery using transfer learning to match the artist’s need even more closer.
For the moment, DeepIris only takes care of the image pre-processing. The defect removal needs to be performed by the artist. We are already in the process of implementing a defect detector, still using deep learning.