Runway AI – What Can It Do And Will It Take Over Our Jobs?

A walkthrough of Runway’s Machine Learning tools

Can you color grade by typing out just a word or two? Is it possible to shape the look you’re after with a simple written description?

That’s the promise of Runway’s text-to-grade feature.

This certainly sounds like a solution for exploration – but (spoiler alert) don’t worry, it won’t replace colorists anytime soon.

Yet the pace of AI development is running faster than anything I’ve seen before in my couple of decades in this industry, so it is worth paying attention and keeping up-to-date on developments.

When I recorded this Insight, Runway was far more obscure than it is now, given that it’s even been on the front page of The New York Times. 

What is Runway AI?

Runwayml.com is the ‘Swiss-Army Knife’ of ‘AI-powered’ video editing and effects platforms, and in this Insight, we take a whistle-stop tour of just some of the 30 different tools at your disposal.

This includes:

• Instant rotoscoping
• motion tracking
• audio clean up
• transcription and subtitle creation
• exporting alpha mattes
• In-painting
• text to image
• Image to image
• text to 3D texture
• and more.

Runway is a browser-based tool, so every shot you want to run through it needs to be uploaded, which may or may not be a concern depending on your content security parameters.

In terms of pricing, there is a limited but still useful free tier, and the cheapest month-on-month Pro tier is just $15 (as of April 2023).

I’ve personally used Runway to rapidly rotoscope some tricky shots and have leaned on its capabilities as my ‘junior VFX assistant.’ I do see myself using it more and more often.

It is, after all, an easy way for me to leverage a new tool to deliver greater value to my clients. And who doesn’t want that?

Update: Text-to-Video – Gen 2

Since recording this insight, Runway has released ‘Gen-2’ of their text prompt to video tool, which allows you to generate short video clips just by typing some words in a box.

Text-to-video is a pretty interesting tool to play with, and the results are getting exponentially better as the model iteratively improves.

Tool covered in  this Insight

Here’s a short list of everything I look at in this quick tour of Runway’s tools:

• Text to color grade
• Slow motion
• Online Video Editing Tool
• Depth Map Extraction
• Motion Tracking
• Runway Academy – In-built Training Tutorials
• Background Replacement
• Keyboard Shortcuts
• Effects and Filters
• In-Painting (like Content-Aware Fill)
• Erase and Replace – Still Image Tool
• Text to Image Generation
• Colorise Black and White Image
• Image Up-scaling
• Erase and Replace Part 2

I don’t get too in-depth into any one tool, as in this Insight, I’m just showcasing some of what’s possible. I want to give you enough flavor to decide if you want to play with it yourself!

If you do, let us know how you get on.

External Links

• Try Runway – https://www.runwayml.com
• Discount Code – Members, log in to access a 10% discount code (listed under the Insight video)
• Runway Training Academy – https://academy.runwayml.com/
• Follow Runway’s ongoing AI research: https://research.runwayml.com/

Questions or Comments? Join the discussion!

Is this Insight useful to you? Let us know! Mixing Light is all about community discussions, and we’re curious if you found this helpful, if you have something to add, or if you need more questions answered.

– Jonny