A camera app connected to DALL-E 2 generates additional AI images of a real object in near real-time.
DALL-E 2 is best known for its ability to generate images that correspond to text input. However, the system can also generate variants of real-world subjects by, for example, showing an object from different perspectives or changing the content of an image. It is also capable of generating photorealistic motifs.
Photographing parallel universes
Photographer Nicholas Sherlock has taken advantage of DALL-E's ability and developed an unusual camera app: it sends a picture taken with the camera directly to DALL-E 2 via mobile Internet. The AI system then generates different versions of this picture and plays it back to the camera.
The photographer describes this new photographic process as a way of "peering into parallel universes". DALL-E 2 "dreams up new versions of them, revealing a world-that-might-have-been."
For simple subjects, the AI-based image variation works surprisingly well. Stones, trees, or plants, for example, are rendered realistically from different perspectives and in different lighting situations. It is sometimes difficult to distinguish between the original and the AI image.
However, in complex scenes like landscape shots, the AI images become more inaccurate and exhibit geometric errors and inappropriate blurring in the image composition. The transitions between image elements become messier.
Since DALL-E 2 is known to have problems with text, it replaces captions with fantasy language. The AI also replaces company logos or mascots with alternative versions. The result is quite entertaining.
Anyone looking for logos in the future could simply photograph suitable motifs on the street and have an image AI make similar suggestions.
Quantum Mirror App violates OpenAI guidelines
The DALL-E-2 app is available for Android-based Sony cameras (e.g. A7, A7R, A7 II, A7R II, but not the A7 III series or newer - list of possible compatible cameras) for free on Github. But beware: according to Sherlock, his app violates OpenAI's guidelines by being rated as web-scraping software. Violating the guidelines will result in an account suspension by OpenAI.
Sherlock's Quantum Mirror app is available on Github. On his project page, the photographer shows numerous other sample images. Sherlock plans to update his app as soon as OpenAI has released an API that complies with the new terms of use.
