Learning Silverstack Part 3: The Offload Wizard

The Offload Wizard

Now that we’ve covered why professionals choose and use Silverstack on production and have had our house tour of the interface, we can finally settle down and look at some of its functions in action and in detail. Today, we’re going to acquaint ourselves with what Pomfort refers to as the Offload Wizard.

Offloading is also commonly referred to as copying, downloading, or backing up footage. Don’t be fooled – this is not a job for a Production Assistant! The Digital Loader and Digital Imaging Technician are part of the Camera Department.

The Offload Wizard is an intelligent module that kickstarts off the offload process by determining a few key parameters of the job even before it starts. In this specific order Silverstack:

1. Identifies the folder structure on a mounted volume (“Is this RED, Arri, or Sony media?”)
2. Scans the volume for files (“How many files are in here?”)
3. Directs these files to a given destination (“Where are these files going?”)

In the video Insight below, I explain how each of these sections work, why the Ingest and Thumbnail Creation step is showing us potential errors. Also shown in more detail, how to manage more than one download destination under ‘Copy and Verify’.

WHAT’S IN A CHECKSUM?

If you’ve been working around data for long enough, you may have heard mention of something called ‘Checksum verification’. If you think that it sounds like a super technical computer term, you’re right – it is! Checksum verification comes from the world of cryptography, sure enough.

A checksum is a small-sized datum derived from a block of digital data for the purpose of detecting errors which may have been introduced during its transmission or storage. Depending on its design goals, a good checksum algorithm will usually output a significantly different value, even for small changes made to the input.

In plain English, a checksum verification works by comparing the source copy of a file to the destination copy of that same file.

Each source file by nature of random seeds gets its own unique ID number. After the copy is complete, the same algorithm which produced that unique ID number is run again on the destination file.

If the unique ID number is a match, we have a carbon-copy of the original source media. The slightest change in the destination file will result in a completely different ID number, and in that case, we have a potentially corrupted copy.

Now, this is a gross oversimplification of the process, but all you need to understand at this point is that both Loaders and Digital Imaging Technicians use checksum verification during the offload process to ensure 100% data integrity of the video and audio files. For those who are more technically inclined, Imagine Products has a fantastic series on checksum verification. I highly recommend checking out parts 2 and 3, which go into the nitty gritty details about this black magic wizardry.

So this all sounds wonderful, but nothing comes for free. Almost every form of checksum verification incurs a time penalty on your overall downloads if you were to compare to a direct copy paste without protection. There is no one single form of checksum algorithm – there are actually several! Inside of Silverstack you have the following choices:

• xxHash64 BE
• MD5
• SHA1
• File Size Verification Only

All of these algorithms vary in complexity, accuracy, and speed. Choose the right one that best fits your configuration. It’s all about finding that balance, and providing yourself and your employers the peace of mind that their files are completely backed up.

TAMING WILDCARDS

During the scanning process, Silverstack pulls attributes from the files’ metadata and presents these findings to you for use in creating a custom download directory! Those who have used % variables in Resolve before will extremely comfortable here.

Every wildcard is a bubble in which you can drag and drop to specify a contextually aware destination path. Be sure to choose and use your wildcards carefully, as missing metadata can upset your download organization. You can evaluate the usefulness of each wildcard based on the ‘Availability’ count – generally speaking, you want to base your paths on solid references.

The Jobs Detail Sheet

On the tail end of our previous insight in this series, we looked at a mystery workspace which turned out to be the Jobs Sheet. This is the area in which we can supervise the download progress, very much like the download manager you have built into your web browser.

The Jobs Sheet allows you to not only monitor overall offload progress, but also on an individual file basis. Additionally, it provides average copy and verification speeds (often expressed in MB/s) as important health indicators of your general offload performance, complete with a Job History tab so you can review past offloads and their corresponding metrics.

There are several different ways to improve your offload performance, but that is outside the scope of this insight and as mentioned in the video, I’ll be touching on topics like this in a following insight that’s going to be chock-full of tips and tricks to improve your quality of life on set. If you have something you’d like to see in this upcoming insight, be sure to leave your suggestions in the comment section.

About this Video Insight

In the video below you’ll learn:

• How to utilize the Offload Wizard.
• Save and manage offload templates.
• Use wildcards to create custom offload directories.
• Evaluate download progress in the Jobs Sheet page.

Extra Reading

• Pomfort Silverstack Knowledgebase: Checksum Verification Methods
  https://kb.pomfort.com/silverstack/offloadandbackup/checksum-verification-process/
• StudioDaily/Imagine Products: Checksums and Verification Part 3
  http://partners.studiodaily.com/imagineproducts/content/checksums-and-verification-part-3-speed-vs-security
• Pomfort Silverstack Knowledgebase: Factors for Data-Transfer Performance
  https://kb.pomfort.com/silverstack/offloadandbackup/factors-for-data-transfer-performance/
• Pomfort Silverstack Knowledgebase: Cascading Copy
  https://kb.pomfort.com/silverstack/offloadandbackup/cascading-copy
  Video: https://www.youtube.com/watch?v=ZTv-Hpg_ZrA