Video creators: Want to swap backgrounds? Knock yourselves out.
Google researchers have been working on a way to let you swap out your video backgrounds using a neural network—no green screen required.
It’s rolling out to YouTube Stories on mobile in a limited fashion, said TechCrunch.
Video segmentation is a widely used technique that enables movie directors and video content creators to separate the foreground of a scene from the background, and treat them as two different visual layers.
Video content creators know that a scene’s background can be separated from the background treated as two different layers. The maneuver is done to achieve a mood, or insert a fun location or punch up the impact of the message.
The operation, said the two on the Google Research site, is “a time-consuming manual process (e.g. an artist rotoscoping every frame) or requires a studio environment with a green screen for real-time background removal (a technique referred to as chroma keying).”
Google researchers know how much people like to trick others into thinking they’re on the moon, or that it’s night instead of day, and other fun shenanigans only possible if you happen to be in a movie studio in front of a green screen. So they did what any good 2018 coder would do: build a neural network that lets you do it.
This “video segmentation” tool, as they call it (well, everyone does) is rolling out to YouTube Stories on mobile in a limited fashion starting now — if you see the option, congratulations, you’re a beta tester.
“Google is developing an artificial intelligence alternative that works in real time, from a smartphone camera,” Grigonis wrote.
Google designed a network architecture and training procedure suitable for mobile phones focusing on the following requirements and constraints:
- A mobile solution should be lightweight and run at least 10-30 times faster than existing state-of-the-art photo segmentation models. For real time inference, such a model needs to provide results at 30 frames per second.
- A video model should leverage temporal redundancy (neighboring frames look similar) and exhibit temporal consistency (neighboring results should be similar)
- High quality segmentation results require high quality annotations.
This is great news for a lot of folks — removing or replacing a background is a great tool to have in your toolbox and this makes it quite easy. And hopefully it won’t kill your battery.
News Source: https://research.googleblog.com/2018/03/mobile-real-time-video-segmentation.html
Related Videos:
MIT’s Deep-learning system generates Videos that predict what will happen next in a scene | QPT
MIT’s “CornerCameras” system can detect objects or people in a Hidden Scene
