Rotoscoping and Infill tedium gets a new Machine Learning Replacement

In the back of my mind I always thought, if I had it to do over, I’d go into Visual Effects. The combination of creativity and technology that goes into VFX fascinates me. Then the reality of the industry that I read about and its financial woes, makes me rather glad my life took a different path.

But what would VFX be like if the tedium – rotoscoping and background removal – were simply a couple of clicks away. The video below compares “traditional” (up to Rotobrush 2 in After Effects) rotoscoping tools, against RunwayML for isolating moving actors, and in filling. Stick around to the end for a preview of ML creating cartoon images based on text descriptions!

Machine Learning drive RunwayML is a web service, also used in this real time – 10 minute – demo of rotoscoping three separate clips and exporting the results.

These incredibly powerful tools are only the beginning of how Machine Learning is transforming almost everything visual: from production tools like RunwayML, to deep fakes like the recently distributed one of Tom Cruise.

Machine Learning is making huge strides in almost everything visual. It’s a pity it’s not advancing as fast at deriving useful keywords!

These tools are, however, the way I see Machine Learning (a.k.a. Artificial Intelligence) integrating with human creatives to devote more time to polishing and finishing the effect, to to process effects shots that would never be done otherwise. That last part is the exciting part for me.

Subscribe to The Corridor Crew for a bunch of interesting VFX videos.

Or for more of the applied techniques ActionVFX for the “Best Stock Footage for Visual FX.”