The most surreal images: from the camera obscura to DALL-E type neural networks – The Conversation

Maria does not exist. The image on the right corresponds to a person that does not exist and that anyone can generate on this website. They are ghosts of a type of generative artificial intelligence responsible for other creations such as the famous deepfakes. In the 21st century we are witnessing the birth of a new type of image that comes from the deepest layers of artificial intelligence (AI). From the mental representations that the first humans formed around the word and fire to the automated ones, there has been a long technological path full of incredible moments. Capturing eclipses in a dark box The sky has always been a fascinating setting, giving rise to both mythological and scientific interpretations. Centuries ago, some scholars thought that instead of projecting the imagination towards the sky, perhaps it would be more interesting to project light into a chamber and study it. This is how the first images of the sky began to be taken. This was one of the uses of the camera obscura that allowed, through the entry of light through a small hole, to visualize the external image, for example, a solar eclipse, projected on one of its walls. This is the first representation of a camera obscura during the observation of an eclipse in Louvain (Belgium) in January 1544, made by the astronomer Reinerus Gemma-Frisius. Reinerus Gemma-Frisius The first ghosts in the 19th century Capturing phenomena from our reality is fine, but creating our own images and projecting them was going to be a memorable show. The magic lantern was a type of projector originally made up of a candle, a mirror and a cylinder with a lens to concentrate the light through which they traveled from bucolic landscapes to ghosts and demons, giving rise to a genre known as phantasmagoria. The use of smoke and other effects such as moving the projector itself allowed 19th century audiences to be terrified. Illustration from the book Recreational, Scientific and Anecdotal Memoirs (1831) by the physicist and aeronaut EG Robertson representing a phantasmagoria session offered in Paris in 1797. Etienne Gaspard Robertson Electrical impulses: the first image on TV In the second decade of the 20th century, a grayscale portrait of 32 lines of resolution and 5 images per second would go down in history as the first television image. Experimentation and knowledge based on electricity and electromagnetism allowed the gradual arrival of a new generation of images that could reach every home in the world at the same time. The challenge was to capture the scene and, for the first time, transmit it from one place to another as was already the case with sound on the radio. Before transmitting, the scene had to be explored in an orderly manner, and changes in light converted into variations in electrical current. This electromechanical imaging principle can be seen in this video. A baby was the first digital image The baby in history’s first digital image is Russell Kirsch’s son Walden, created by scanning an analog photograph in 1957. Russell A. Kirsch / National Institute of Standards and Technology / Wikimedia Commons In the middle of the 20th century, the first digital image arrived, captured by a scanner. Then there was not even internet open to all audiences. This device made it possible to capture the intensity variations of a photograph and register them in a much more precise way, by encoding the pixels in individual cells. The image converted into a numerical matrix is ​​ready to be encoded, mixed, compressed, recorded and studied using the tools provided by the digital revolution. Hallucinating with artificial intelligence If we can recognize a face it is because deep down all faces look alike or have common elements. There are positions in a photograph of a face where certain pixel values ​​– such as those defining the lips, nose, etc. – are more likely than others. La Gioconda with Deep Dream effect. Pjfinlay / Wikimedia commos Modeling a face from training with thousands of faces is one of the milestones of artificial intelligence machine learning. In addition, the interesting thing is to manipulate that modeled representation, enhancing some characteristics over others. The result is aberrations such as the digital hallucinations of the Google Deep Dream algorithm or the faces of people that do not exist (like the one we showed at the beginning of the article), the result of sampling new images from the generic face model. Now yes, the resulting images are true ghosts. The disturbing resurrection of Salvador Dalí Hallucination is a way of demonstrating that we can force and distort the latent representations of a model at our convenience. Sort of like a supersurgeon being able to remove a donor’s face and transfer it to a recipient. Deepfake has many applications that can be both creative and unethical. One of the applications that has been explored for a few years is that of recreating famous or historical characters. The folkloric Lola Flores in an advertisement was the first popular “resurrection”. Even more disturbing is the one carried out by the Salvador Dalí Museum in the city of Saint Petersburg (Florida). From archive material and interviews, they recreated the facial expressions and voice of the famous surrealist painter. How does it feel to see a historical character “revive”? Dalí Lives, a project created with artificial intelligence. Source: The Dalí Museum. Image generation with a text command Capturing the representation of a face by exposing thousands of faces is already a milestone. But if we can also capture the relationship between different data inputs, such as the combined image and text input, we will have gone even further. We can achieve this type of multimodal representation with AI architectures such as the Dall-E2 system. Dall-E is an acronym that fuses Wall-E, the famous Disney movie, and the painter Salvador Dalí. Social networks have already been flooded with his creations. What can they draw? Most. The only limit is the imagination. For example, a medieval man sitting at a computer. It is now possible to try, and for free, DALL-E mini, a model inspired by DALL-E that is used to experience the level of associative capacity between the textual input and a visual result. There are ways to achieve one result or another depending on the words used or how they are arranged in the text. What kind of positive and negative consequences can this kind of generation of realistic images by AI algorithms have for society? Can we get used to living surrounded by ghosts?