U.S. researchers show that AI can faithfully display infrared images in color. This would allow people to see in color in the dark.
Some night vision systems scan dark environments with infrared light that is invisible to the human eye. The system then outputs these images in monochrome as a green video image. With the help of AI software, such infrared light images could be displayed in full color in the future.
AI system learns colors
The basic thesis of the U.S. research team at the University of California Irvine is that every pigment and dye on an object reflects infrared wavelengths in addition to visible wavelengths. An AI system could learn this overlap and translate images from one color mode to another.
For their study, the team first printed out about 200 images of human faces in color (CMYK), which they photographed successively at different wavelengths in the IR spectrum and in the visible range (see cover image). In doing so, they recorded the intensity of the reflection.
Using 140 of these photos, the researchers then trained a convolutional neural network (CNN) to detect differences and similarities between the infrared light images and the conventionally visible images.
In the case of UNet-GAN, the neural network was to predict the visible reflectance values based on the IR reflectance values - initially completely randomly. Controlled by a discriminator that compared the prediction with a reference image, the network went through the prediction process until the predicted value matched the reference image.
Through this training process, the AI system learned to assign appropriate colors from the visible range to pure infrared light images, and thus the ability to color IR images realistically. The UNet GA network performed better in a human evaluation at high zoom than the conventionally supervised trained UNet, which, however, evaluated better zoomed out
Use for night vision and in research
According to the research team, this method is suitable for applications such as color night vision in the private and military sectors or for biological studies in which visible light causes unwanted reactions.
The color night vision system could also help with medical procedures, such as eye operations in which the retina is exposed to intense light for long periods of time, which can be harmful to the eye. The eye surgeon with the color night vision system could then operate in the dark, but view the object of the operation as if it were in visible light.
However, the researchers caution that while their system works well with human faces, it still needs to be trained and tested for other objects that reflect light differently. In addition, video conversion currently only works at three frames per second. This is too slow for many application scenarios, especially in a medical context.
