Culture

This 4K, 60 FPS, AI-enhanced video from 1896 is absolutely mind-blowing

The technique includes neural networks, algorithms, massive amounts of training data, and a healthy dose of imagination.

Denis Shiryaev

In the middle of makeup tutorials, reaction commentary, drama channels, cake terrors, and other chaotic content, YouTube has little nooks where nerds go wild on their niche hobbies. One of them, Denis Shiryaev, applied artificial intelligence wizardry to old footage and created mesmerizing masterpieces with the help of publicly available algorithms. We've previously seen the kind of insane magic neural networks can breathe into simple 2D images, and the results that are possible when you combine photographs with modern 3D techniques to bring deceased composers' likenesses to life.

Old films like A Trip Down Market Street, Lumiere brothers' clips, and footage from early 20th-century Japan all get Shiryaev's frame interpolation, color, speed, and upscaling treatment. The results aren't merely a visual treat but also a modern-day philosophical project: are these algorithms merely enhancing old footage or are they changing our understanding of the past and altering the narratives the clips contain? It's likely all of the above.

Playing with the past — When you juxtapose the original clips with Shiryaev's enhanced footage, you'll notice several changes. For one, Shiryaev's enhancement boosts the film to 60 frames per second. So, it's faster for modern viewers and more akin to our expectations for video content. Another result of Shiryaev's artificial intelligence-input is that the faces are enhanced.

Shiryaev's work is tricky. When you're adding color to and enhancing the speed of inanimate objects, there is the risk of imposing your personal narrative on them. Algorithms like DeOldify are typically trained on millions and millions of images, subsequently allowing the network to understand that, for example, tree leaves are almost always green, the sky is blue, and so on. But when it comes to human beings in previously black and white films, adding color isn't just facial restoration — it's a matter of colorization, upscaling, and almost fictionalizing the past.

Consult the experts — This is where artificial intelligence researchers do well to consult historians and archivists to better grasp the region, geography, racial demographics, and other contextual details of these films. Consider Shiryaev's previous project involving old footage of Tokyo, Japan, between 1913 and 1915. Shiryaev used neural networks to colorize and interpolate the film to 60 frames per second.

The result is stunning. Shiryaev also removed noise from the film, boosted the image resolution, and he made it a point to add: "Please, be aware that colorization colors are not real and fake. Colorization was made only for the ambiance and do not represent historical data."

We'd love to see more — There's a certain charm to these old, grainy, and low-resolution films from the past. With artificial intelligence geeks like Shiryaev, we get a chance to time travel. Restoration of this sort is the closest experience we can have to visit another era and see how far humanity has come. Of course, there will be critics who object to Shiryaev's work, insisting that the technical purity of these mediums are best left alone. But a little colorful and sped-up saunter down yesteryear won't hurt anyone if you ask us.