A Shot Matching Assitant Or Something Else?
If you’re anything like me, you’re probably somewhat skeptical of anything that claims it can do our job as a colorist.
Outside of the highly technical, numerical aspect of being a colorist, there are some very human and psychological aspects that I have doubts any machine could emulate, at least in 2021. That said, there are certain aspects that I feel a machine, especially a well-trained one, might be very helpful for, like shot matching, and it certainly made me curious when I first heard about ColourLab AI.
In this Insight series, I’ll share with you what I’ve learned testing the application & plugins over the past few months, along with some background shared with me by one of its creators, Colorist Dado Valentic. In this Insight, we’ll start out by exploring some background of ColourLab AI in the video I’ll walk you through the basic workflow of using it.
What is Colourlab AI?
Colourlab AI is a colorist-created, standalone, MacOS application that utilizes computer vision and machine learning to enable the quick analysis and automatic matching of a look or correction across a scene or entire sequence.
You can adjust your footage within the application using a variety of industry-standard controls (such as LGG or offset) and quickly match a graded key shot to the rest of the scene. You can even import a reference from another film or visual medium and Colourlab will do its best to match the overall ‘DNA’ of the look, but don’t worry – you still have control to make some finer adjustments on a shot-to-shot basis as needed.
ColourLabAI also has several different perceptual models it uses to analyze and correct each shot, allowing you to switch between them within the application and back in within Resolve as versions (if you enable that within the settings). You’ll get a brief explanation of each one in the video below, but I’ve found they give slightly different approaches to matching which is great.
ColourLab = Computer Vision + Machine Learning + Deep Learning
During my talks with Dado Valentic to learn a bit more about the application and its background, I discovered that artificial intelligence isn’t entirely an accurate way to describe it. An amalgam of, ‘computer vision, machine learning, and artificial intelligence, ColourLab attempts to identify familiar elements of a shot similar to how human vision works – not by brute-force pixel-for-pixel processing, but through a computer-vision based approach that’s been highly trained to recognize familiar objects and tones which might grab the attention of the human visual system.
If you’re interested in learning a bit more about terms like machine learning and artificial intelligence as it refers to our purposes, Mixing Light Contributor Katie Hinsen wrote a great intro to them back in 2018.
All of this stuff on machine learning and AI is a very deep rabbit hole and I don’t even get into it very deep in this series, but luckily you don’t really need to in order to use this application or technology effectively. It’s simply worth mentioning as it explains ColourLab’s generally excellent performance, even on my 2018 Intel Mac Mini. I’ve been informed that upcoming M1-based Macs can expect even faster performance due to Apple’s CoreML platform and how easily it moves between platforms.
The Integration with Resolve
Thanks to some clever engineering, Colourlab AI’s integration with Resolve is most easily described as virtually seamless.
You prepare a sequence within Resolve, as usual, and then over in ColourLab AI click the ‘Resolve Fetch’ button to pull your media and timeline over to the standalone ColourLab AI application. Once finished with your grading in ColourLab, you click ‘Resolve Sync’, and four nodes are synced right into your Resolve node tree. Two nodes are for Input \ Output transforms, one node is where your match \ grade occurs, and the last node is for your show look LUT (which you have the option of creating in ColourLab).
CDL Compliant Matching?
In the video below, I mention that Colourlab AI generates its ‘matches’ using CDL compliant metadata and that deserves some additional explanation.
To ensure it’s flexible between a variety of workflows, Colourlab AI matches using the industry-standard lift, gamma, and gain controls & printer points you would find in an ASC CDL file. For example, contrast is calculated back down to lift, gamma, and gain as it’s not traditionally a control that can be passed via a CDL, even though it’s available within the Colourlab UI.
Being CDL compliant also means that when grades come back into DaVinci Resolve Lum Mix is set to zero – this is necessary to be CDL compliant.
As you know when Resolve 17 recently came out it featured excellent set of upgraded tools for color management, whether you’re working in SDR or HDR.
ColourlabAI though has its own color management pipeline – you can export your work from the application using a ‘ready-to-output’ Rec709 preset or switch to a scene-referred DaVinci Wide Gamut and finish off your color management at the timeline level using Resolve’s own color management or even use Look Designer 2 OFX. You have quite a few options, including the ability to utilize your own fixed node structure, which made me smile more than I can express here – it’s the little things in life!
The big tip, which I show in the video, is to make sure you don’t accidentally ‘double your color management’ after coming back to Resolve if you’re working in a color-managed workflow.
MacOS Only – For Now
Due to the ease which Apple made developing machine-learning-based applications (via its CoreML platform), MacOS is the first OS to get the standalone Colourlab AI application but a Windows beta should be coming later this year, around Summer 2021 based on the official roadmap but just keep in mind that can always be pushed back a bit.
If you’re impatient, utilize a Postgres (Shared) Resolve Database, and would like to utilize ColourLab AI on Windows or Linux workstation, you can set-up a ‘synced’ LUT (using Resolve 17’s custom LUT folder + A Network location) folder between a secondary machine like a Mac Mini and everything else will synchronize and work seamlessly if project sharing and mapped mounts are set-up correctly between your main grading workstation and your secondary MacOS machine.
This workaround isn’t official, but it’s something I found works well and allows me to continue my main grading work using my main HP Z840 workstation after my initial ‘base’ match in Colourlab AI on my Mac Mini.
If you’re like me, you can’t live without your Streamdeck! Its flexibility speeds up so many day-to-day tasks. Fortunately, ColourLabAI has support for a StreamDeck and there are even some prebuilt profiles available.
Streamdeck Profile (thanks to Willian Aleman )
if you’d like a reference of all the available keyboard shortcuts (including a downloadable PDF), you can find them available here:
If you would like to learn more about Colourlab AI, you can learn more at Colourlab.ai.
Colourlab AI is only available as a subscription, but it’s not a terrible value if you find the time savings works out in your favor.
In Part 2, we’ll dive into two additional plugins that are part of the set of tools – Looks Designer 2, including the ability to build a show LUT functionality within ColourLab AI, and GrainLab.
Stock footage courtesy of RAW.FILM
As always, if you have questions or something to add to the conversation please use the comments below!