Have you ever settled yourself down in front of an animated movie and marvelled at how the 3D figures are brought to life?
From Sulley’s wind-ruffled fur as he strides across the ‘Monsters’ University’ campus to the heart-wrenching fade-out of Riley’s imaginary friend, Bing Bong, in ‘Inside Out’, it’s the play of light across these 3D scenes that brings the characters so vividly to life. Each moment is painstakingly animated, textured and rendered to give a carefully crafted illusion of reality.
In these more recent productions, a technique called ‘ray tracing’ maps out each ray of light in a scene, giving rise to the shadows, reflections and 3D appearance of characters. Even with the help of vast banks of powerful computers, the rendering process takes hundreds of thousands of computing hours, and films can take years to finish.
There are shortcuts that animators can take to limit production time, however, and one method involves using fewer light rays to render the images. While this does speed up the whole process, inaccuracies and ‘noise’ can show up in the final picture, reducing the quality of the film.
Joining forces with engineers from Disney Research and the University of California, Santa Barbara, the brains behind Pixar Animation Studios think they may have found a solution.
‘Finding Dory’ trains machine ‘memory’
Researchers from the three institutes pooled their knowledge to develop a new technology based on artificial intelligence and a process called ‘deep learning’. Dory, the forgetful Regal blue tang who first appeared in ‘Finding Nemo’, assumed the unlikely role of the machine’s tutor.
Using millions of images from 2016’s ‘Finding Dory’, the team trained a deep-learning model to recognise ‘noisy’, poor quality frames. The algorithm then transformed them into the sharper pictures like those rendered with significantly more light rays. Once trained, the system could even pick out and replace grainy test images from other films, such as Pixar’s latest release, ‘Cars 3’, despite having a vastly different colour palette.
The new technology denotes a giant leap forwards from previously high-end ‘de-noising’ methods. While these could make a start on cleaning up the images, many images would need to be retouched by artists to make them movie-ready.
Markus Gross, vice president for research at Disney Research said: “Other approaches for removing image noise have grown increasingly complex, with diminishing returns. By leveraging deep learning, this work presents an important step forward for removing undesirable artefacts from animated films.”
The algorithm allows artists to quickly and automatically remove noise, while making sure they keep the detail in scenes.
Disney and Pixar plan to use the technology in their upcoming block-busters to speed up the movie-making process. In a bid to continue to the forward push of research, they are even making the code available to the research community.