Computer-Assisted Shot Matching with Colourlab AI – Part 1

March 24, 2021

In Part 1 of a new series, Jason Bowdach shows you how to get projects to ColourLab AI and how to build a match across a scene using this powerful tool.


Series

A Shot Matching Assistant Or Something Else?

If you’re anything like me, you’re probably somewhat skeptical of anything that claims it can do our job as a colorist.

Outside of the highly technical, numerical aspect of being a colorist, there are some very human and psychological aspects that I have doubts any machine could emulate, at least in 2021. That said, there are certain aspects that I feel a machine, especially a well-trained one, might be very helpful for, like shot matching, and it certainly made me curious when I first heard about ColourLab AI.

In this Insight series, I’ll share with you what I’ve learned testing the application & plugins over the past few months, along with some background shared with me by one of its creators, Colorist Dado Valentic.  In this Insight, we’ll start out by exploring some background of ColourLab AI in the video I’ll walk you through the basic workflow of using it.

What is Colourlab AI?

Colourlab AI is a colorist-created, standalone, MacOS application that utilizes computer vision and machine learning to enable the quick analysis and automatic matching of a look or correction across a scene or entire sequence.

You can adjust your footage within the application using a variety of industry-standard controls (such as LGG or offset) and quickly match a graded key shot to the rest of the scene. You can even import a reference from another film or visual medium and Colourlab will do its best to match the overall ‘DNA’ of the look, but don’t worry – you still have control to make some finer adjustments on a shot-to-shot basis as needed.

ColourLabAI also has several different perceptual models it uses to analyze and correct each shot, allowing you to switch between them within the application and back in within Resolve as versions (if you enable that within the settings). You’ll get a brief explanation of each one in the video below, but I’ve found they give slightly different approaches to matching which is great.

ColourLab = Computer Vision + Machine Learning + Deep Learning

During my talks with Dado Valentic to learn a bit more about the application and its background, I discovered that artificial intelligence isn’t entirely an accurate way to describe it. An amalgam of, ‘computer vision, machine learning, and artificial intelligence, ColourLab attempts to identify familiar elements of a shot similar to how human vision works – not by brute-force pixel-for-pixel processing, but through a computer-vision based approach that’s been highly trained to recognize familiar objects and tones which might grab the attention of the human visual system.

If you’re interested in learning a bit more about terms like machine learning and artificial intelligence as it refers to our purposes, Mixing Light Contributor Katie Hinsen wrote a great intro to them back in 2018.

All of this stuff on machine learning and AI is a very deep rabbit hole and I don’t even get into it very deep in this series, but luckily you don’t really need to in order to use this application or technology effectively. It’s simply worth mentioning as it explains ColourLab’s generally excellent performance, even on my 2018 Intel Mac Mini. I’ve been informed that upcoming M1-based Macs can expect even faster performance due to Apple’s CoreML platform and how easily it moves between platforms.

The Integration with Resolve

Thanks to some clever engineering, Colourlab AI’s integration with Resolve is most easily described as virtually seamless.

You prepare a sequence within Resolve, as usual, and then over in ColourLab AI click the ‘Resolve Fetch’ button to pull your media and timeline over to the standalone ColourLab AI application. Once finished with your grading in ColourLab, you click ‘Resolve Sync’, and four nodes are synced right into your Resolve node tree. Two nodes are for Input \ Output transforms, one node is where your match \ grade occurs, and the last node is for your show look LUT (which you have the option of creating in ColourLab).

CDL Compliant Matching?

In the video below, I mention that Colourlab AI generates its ‘matches’ using CDL compliant metadata and that deserves some additional explanation.

To ensure it’s flexible between a variety of workflows, Colourlab AI matches using the industry-standard lift, gamma, and gain controls & printer points you would find in an ASC CDL file. For example, contrast is calculated back down to lift, gamma, and gain as it’s not traditionally a control that can be passed via a CDL, even though it’s available within the Colourlab UI.

Being CDL compliant also means that when grades come back into DaVinci Resolve Lum Mix is set to zero – this is necessary to be CDL compliant.

Color Managed…Again

As you know when Resolve 17 recently came out it featured excellent set of upgraded tools for color management, whether you’re working in SDR or HDR.

ColourlabAI though has its own color management pipeline – you can export your work from the application using a ‘ready-to-output’ Rec709 preset or switch to a scene-referred DaVinci Wide Gamut and finish off your color management at the timeline level using Resolve’s own color management or even use Look Designer 2 OFX. You have quite a few options, including the ability to utilize your own fixed node structure, which made me smile more than I can express here – it’s the little things in life!

The big tip, which I show in the video, is to make sure you don’t accidentally ‘double your color management’ after coming back to Resolve if you’re working in a color-managed workflow.

MacOS Only – For Now

Due to the ease which Apple made developing machine-learning-based applications (via its CoreML platform), MacOS is the first OS to get the standalone Colourlab AI application but a Windows beta should be coming later this year, around Summer 2021 based on the official roadmap but just keep in mind that can always be pushed back a bit.

Windows Workaround

If you’re impatient, utilize a Postgres (Shared) Resolve Database, and would like to utilize ColourLab AI on Windows or Linux workstation, you can set-up a ‘synced’ LUT (using Resolve 17’s custom LUT folder + A Network location) folder between a secondary machine like a Mac Mini and everything else will synchronize and work seamlessly if project sharing and mapped mounts are set-up correctly between your main grading workstation and your secondary MacOS machine.

This workaround isn’t official, but it’s something I found works well and allows me to continue my main grading work using my main HP Z840 workstation after my initial ‘base’ match in Colourlab AI on my Mac Mini.

Streamdeck Integration

If you’re like me, you can’t live without your Streamdeck!  Its flexibility speeds up so many day-to-day tasks.  Fortunately, ColourLabAI has support for a StreamDeck and there are even some prebuilt profiles available.

YouTube video

Streamdeck Profile (thanks to Willian Aleman )

Download Link

Streamdeck Profile User Manual

if you’d like a reference of all the available keyboard shortcuts (including a downloadable PDF), you can find them available here:

Keyboard Shortcuts

Pricing

If you would like to learn more about Colourlab AI, you can learn more at Colourlab.ai.

Colourlab AI is only available as a subscription, but it’s not a terrible value if you find the time savings works out in your favor.

In Part 2, we’ll dive into two additional plugins that are part of the set of tools – Looks Designer 2, including the ability to build a show LUT functionality within ColourLab AI, and GrainLab.

Stock footage courtesy of RAW.FILM

As always, if you have questions or something to add to the conversation please use the comments below!

-Jason


 

Comments

Homepage Forums Computer-Assisted Shot Matching with Colourlab AI – Part 1

  • This topic has 10 replies, 1 voice, and was last updated 1 year, 1 month ago by Willian Aleman.
Viewing 9 reply threads

    • Gedaly G
      Guest

      Great insight, Jason! I purchased Colourlab a while back but hadn’t had the chance to play with it as much as I’d like until recently. Really cool tool and this motivates me to use it more


    • Scott Stacy
      Guest

      Hey Jason… great Insight! I look forward to the deeper dive in the next Insight. Once you get back into Resolve, have you noticed any clipping or degradation with further adjustments? Is there a specific spot in the node tree where it will be best to make other adjustments?


    • Jason Bowdach
      Guest

      Happy to hear, Gedaly! It’s a very interesting tool that continues to surprise me each time I use it, even if its not always what I expect. Highly encourage you to experiment with it and see how it works for you.


    • Jason Bowdach
      Guest

      Hey Scott! I split out your questions below –
      1. I haven’t seen any clipping or degradation but technically it’s prone to the same issues that you could have with any look-up tables workflow, specifically that floating point isn’t preserved, but as you . All the adjustments are performed in a scene-referred Log-C color space after the cameras have been normalized so while unlikely it’s theoretically possible. I suggest you have your LUT interpolation within Resolve settings changed to “tetrahedral” for best results.
      2. Ideally, I prefer working after my IDT (input transform) & CDL corrections but before my show look & ODT (output transform), but it’s simply a preference for the “feel” of the controls. ColourLab works in Log-C AWG between the IDT & ODT so everything responds very similarly to the way camera-native Arri Log-C footage tends to respond and controls, so it will feel very familiar. I might differ this if I’m trying to realistically simulate a photochemical negativepositive workflow, but otherwise I tend to stick pretty close what I described. I’ll show a lot more in my next insight, but always happy to answer any specific questions.


    • Mark Todd Osborne
      Guest

      Great job, Jason!


    • Carey Dissmore
      Guest

      Thanks for the review Jason. I look forward to it. I’m intrigued but have to say I would only find this workflow appropriate for some of the projects I do. Not sure I like subscription model for something like this. Justifying an ongoing subscription when there’s a likelihood that months may pass between my use cases for it is harder. But another thought, perhaps you could address in the future, is a friendly or automated way to map the nodes Colorlab generates into my existing fixed node structure at appropriate places. A lot of us are doing the fixed node structure workflow so this would be an interesting exploration.


    • Jason Bowdach
      Guest

      Hey Carey! I completely understand your feelings and agree, this workflow isn’t perfect for every project.

      As I’ll demonstrate in the next insight in this series, you can include a completely custom fixed node tree in the stand-alone application’s options.

      A subscription to ColourLab AI includes two useful OFX plug-ins, Look Designer and GrainLab, which potentially makes the overall subscription cost easier to justify, but you make a very good point. I believe Color Intelligence allows you to purchase a subscription to one or both OFX plug-ins independent from the stand-alone AI matching application, along with perpetual license for both plug-ins, if that’s your preferred license model. I also believe you can subscribe on a month-by-month basis, but I would reach out and confirm pricing with Dado and team.


    • Marc Wielage
      Guest

      Terrific presentation, Jason. I’m not sure if the Shot Match feature will work with our current feature workflow, but I am intrigued by Dado’s Look Designer and Grain Lab. I hope your future installments will cover those.


    • Jason Bowdach
      Guest

      We’ll be taking a closer look at both of those in the next insight. Thanks for your comments Marc!


    • Willian Aleman
      Guest

      Hi Jason,
      Thanks for the terrific insight.
      The link below has the recent download update to Colourlab Ai profile for Stream Deck, including a pdf user guide. They are under the description tab of the video

      YouTube video

Viewing 9 reply threads
  • You must be logged in to reply to this topic.

Hundreds of Free Tutorials

Get full access to our entire library of 900+ color tutorials for an entire week!


Start Your Free Trial
Loading...