Nvidia researchers have developed a new AI technology that can reconstruct holes or intentionally removed content from images.
The technique, called “image inpainting,” is similar to the “content-aware autofill” feature that Adobe added to Photoshop CS5 in the late 2000s, a technology that “magically” reconstructed a deleted portion of an image based on the photo’s background.
The content-aware fill feature was one of the primary features that allowed Photoshop to reign supreme over all other image editors all these years.
Nvidia develops algorithm inspired by Photoshop feature
Now, Nvidia says that its image inpainting technique “could be implemented in photo editing software” and perform the same actions that Photoshop content-aware autofill can perform.
“Previous deep learning approaches have focused on rectangular regions located around the center of the image, and often rely on expensive post-processing,” the Nvidia research team said. “Our model can robustly handle holes of any shape, size location, or distance from the image borders.”
But the Nvidia researchers are not the first to have developed such a technology. In late 2017, a team of Yandex and Oxford researchers developed a similar algorithm named Deep Image Prior. Nvidia claims its algorithm is better.
Algorithm can handle holes & lines of different shapes & sizes
Nvidia researchers say this is because they trained their algorithm using 55,116 masks of random holes and lines of arbitrary shapes and sizes. These masks were used to delete content from original images. The algorithm was then showed the cutout image and its original form to teach it how to reconstruct the missing pixels.
Researchers verified their algorithm’s accuracy with another 25,000 masks, but without showing the algorithm the original image, forcing it to reconstruct the missing image pieces on its own.
The research team released an impressive video of their new algorithm in action, showing a human operator actively deleting portions from photos and the algorithm filling in missing pixels based on nearby content, and seamlessly blending the new content into the bigger picture.
More on this technology can be found in a research paper named “Image Inpainting for Irregular Holes Using Partial Convolutions.”
This is not the first AI technology Nvidia researchers have worked on. In November last year, researchers created a tool that generates random human faces.
