How Bit Torrent Sync & VPN Changed My Color Grading Life

May 21, 2016

In this Insight, Robbie explores how through VPN & BitTorrent sync he was able to leave the office earlier & still work!


Two Readily Available Technologies Improved My Life & Let Me Get More Done

You’ve probably heard the adage – ‘not enough time in the day’.  

Look up this phrase on the Internet and you might just read in the explanation ‘see Robbie Carman & other colorists’.

Joking aside, I’m lucky enough to be a very busy colorist with a loyal client base that always seems to have a project ready for me to work on.

Even though that sounds awesome (and it is), the downside is that I’m stuck at the office – ALL THE TIME.

It’s my own doing, but I typically work 12-hour color grading days + other responsibilities like training, speaking and oh yeah, being a husband & father to two beautiful, smart, creative kids.

While everyone I know (especially Dan/Pat) tell me I work too much and have a long list of ways to lessen the load, it’s the husband & father part that weighs heavy on me.

Sure, grading awesome projects, keeping clients happy and earning more business are important things, but it’s not a hard argument to make that family is most (and more) important.

For the past year, I’ve been slowly building out the capabilities of my home setup to be equal or near equal to my facility setup  – with the goal to work more from home when I’m not supervised AND to be able to get home at a consistent hour to help my kids with homework, read bedtime stories, and maybe even talk to my wife for a bit 🙂

In this Insight, I briefly want to share some details about my setup at home and then discuss two things that allowed me to have a seamless workflow from the office to home without exporting projects or shuttling drives back and forth.

The Home Studio

For years, I’ve been depending on an iMac at home.  They’re AWESOME machines.

The 5k 27in iMac is really a fantastic machine.  But after attempting to use it as an alternative to my main work setup (yeah, I know that I’m a year or so late on that part!) it just fell flat on its face.

Real-time NR performance?  Yeah right, it’s a mobile GPU.

RAW performance? Not great even with an i7.  The best parts of the 27″ iMac are the screen, the all-in-oneness and of course the familiarity and ease of use of Mac OS.

Having spent a year or so with Windows and Linux at the office, I decided that I needed to step up my game at home, and after a cursory thought about a ‘new’ Mac Pro I just couldn’t justify 10k on a system that is currently using 3+ year old CPUs and often problematic GPUs.

Our audio team lead by partner Cheryl Ottenritter , invested heavily in systems from ADK, and has been ridiculously happy with them.

ADK is a PC system builder focusing on the audio and pro video markets.  Bang for the buck they were hard to beat. With that said, while they’re able to build systems that are often thousands less than HP, Dell, and others they’re using off the shelf components.

One reason you pay more for an HP, Dell or Supermicro system is the R&D that goes into building those systems.

ADK builds are no frills – meaning you’ll have to use a screwdriver for a lot of things. The designs aren’t revolutionary or even sexy.

To me, those things simply don’t matter – I wanted a robust system, cheaper than I could get from the main PC builders.  I’ve been very impressed with the build that I got.

 

homesetup
I’ve been very happy with my home setup. No, I do not have the warm tungsten light as a bias light! I just need a little bit more fill for the photo!

 

Here are the highlights:

  • Asus X99-WS USB 3.1 Motherboard
  • Single 14 Core 2.6ghz Xeon
  • Full Rackmount Case (27in deep, yikes but fans are quite)
  • 128GB Ram
  • 2X Titan X
  • Asus Thunderbolt 2 PCI card
  • Blu-Ray Burner
  • 2 TB Samsung 850 Pro System Drive
  • 2 x 2TB Samsung 850 Pro (Raid 0) Render Cache Drives
  • Decklink SDI 4k
  • Atto R680 Raid Controller (taken from office machine)
  • Sonnett SAS RAID 8×3 TB (also taken from office machine)
  • Drobo 5D – Backup (5x3TB + M2 Card)

Other items in my home studio build include:

I’m very happy with this system. Admittedly, I know this is not a typical ‘home’ system. I sunk quite a bit of money and time into the build, but my goal was always to replicate what I work on day to day at the office.

Now that you know what I’m working with let’s dive into the difference makers of this setup.

A Few Words About PostGres SQL

I know many of you depend on disk databases. If you’re a Resolve (non-studio) user you have to as there is no support for shared PostGres SQL databases.

In my facility, we rely on PostgresSQL databases for a few reasons:

First, you can share them – get the same database in any room and that ability is the heart of this Insight. The details in this Insight DO NOT APPLY to sharing Disk Databases.  

Next, PostGresSQL databases unlock Resolve’s collaborative features including remote rendering.

Before you continue, there is some required reading/watching here on mixinglight.com.

First is Dan’s approach to setting up a shared SQL database HERE.  I produced an article showing my approach to the shared database HERE, and most importantly, Dan wrote a great article on automated SQL database backup HERE.

I would like to also raise three red flags about SQL databases:

  • Everything I describe below about setting up and working with VPN and shared databases assumes a Mac Postgres Database Server.  I have no experience setting this up on Linux (should be similar) or Windows (no idea),
  • Some other applications namely the Mac OS X server app also install (sometimes a newer version) Postgres. I can’t say it strongly enough USE ONE MACHINE FOR A RESOLVE SQL DATABASE SERVER, ANOTHER FOR WEB SERVICES INCLUDING VPN.
  • For the love of all that is holy, please backup your SQL databases – manually or with Dan’s script detailed in his Insight. Since SQL databases are not something you ‘see’ compared to disk databases, it’s easy to forget about them.

PostRes SQL is awesome with Resolve, but it takes some extra attention.

The Need For Speed

Who doesn’t love fast Internet?

One of the most crucial parts of my setup is fast Internet – both for VPN but most importantly for Sync (BitTorrent Sync) file sharing.

I live about a block from a Verizon Fios head-end and I can get 300/300 mbps speeds (they offer 500/500 but it’s stupidly expensive).

In my testing, the threshold for the workflow I describe below is much lower, and it’s all about download speed which is the higher number in most Internet plans.

I think 50/25 or similar is a good base level.

Of course, if faster Internet is available, jump on it as your budget allows.

Member Content

Sorry... the rest of this content is for members only. You'll need to login or sign up to continue (we hope you do!).

Membership options
Member Login

Comments

19 thoughts on “How Bit Torrent Sync & VPN Changed My Color Grading Life”

  1. Nice home setup and great insight Robbie, loves me some BTsync! Clients are always blown away by it when introduced to that method vs. dropbox or ftp use alone. Cheers!

    1. Indeed! I find thought with clients it can be one to many steps compared to something they know like Drive or Dropbox but I’m really digging it!

  2. So this is a dumb question but does the source machine always have to be on to receive the files on the other end? So if I tried to use this for a disk database and turned my work computer off at night or my san off at night that has my media, would I not be able to sync on the other end?

    1. yep – the machines and the BitTorrent Sync app need to be running to sync. If you had one end off when you boot and launch sync again it’ll start syncing again.

      I seldom turn off machines so this scenario is not one I face often.

  3. Hi Robbie, slightly off topic but are both your GPU cards housed internally in your PC or are they in a GPU expander? after reading the config guide it gave me the impression that it was only advised to install 1 gpu on a generic PC build and then more through an expander. Is this correct? I really want to instal a second GTX 1080 but just wanted to check on this.
    I have a single GTX 1080 in one of the x16 pcie slots, a Decklink 4k studio in one of the x4 slots and a U.2 SSD drive for cache using another x4 of bandwidth.
    I still have another x16 slot free – my question is if I add another GTX 1080 my overall pcie bandwidth will be at the total x40 – will this actually slow my machine down or will I see the benefit of an extra GTX 1080 ? Im keen to get another one 🙂
    Cheers, Tim

    1. Tim –

      This totally depends on a few things. 1. Is the CPU you’re using 28 or 40 lanes 2. Does the Motherboard you’re using have PLX chips in it. The PLX chips essentially expand the lane capabilities. The x99-E WS I have in my home machine has two PLX chips that do allow me to run up to 4 cards at full 16x bandwidth.

      Here is a good description of this MoBo and PLX

      https://www.pugetsystems.com/parts/Motherboard/ASUS-X99-E-WS-USB-3-1-11240#advice

      1. Thanks Robbie, the processor I am using is the Core i7 6900K 8 core, which iim pretty sure is 40 lanes compatible, the motherboard is a Gigabyte GA-X99-Designare EX – this one – http://www.gigabyte.com.au/products/product-page.aspx?pid=5812#ov

        On the product page it says it supports Premium 3-Way PCIe x16 Multi-Graphics Support – I think it is referring to SLI mode which I would obviously not want SLI for Resolve.

        From what I can see I think it has 1 x PLX chip, will this be enough to run 2 x GTX 1080, Black magic studio 4k, and the U.2 SSD – total bandwidth would be using all 40 lanes.

        If I understand correctly if I add another GTX 1080 to slot 3 and move the Decklink studio 4k to slot 5, I should be able to run both GTX 1080’s at full16x bandwidth – does that sound right to you?

        Cheers, Tim.

        1. HI Tim.

          Same basic deal as the X99 e-WS USB 3.1 I have in my home build (my board can do 4 x 16 but has dual PLX controllers).

          I ‘think’ you’re A-Ok as again the PLX controllers are fancy way to defeat the 40 Lane CPU limitation. However after looking the product page for you board, and downloading the manual, I’ll admit it I’m a bit confused about how they’re sharing bandwidth.

          On my board all the slots are mechanically 16x (just like yours) 1,3,5,7 operate at full 16X thanks to the PLX controllers. But slots 2, 4 ,6 would slow down to 8x in my box if I put 16x cards in all the odd slots.

          On your MoBo I ‘think’ you’d be fine putting putting 1 1080 in Slot one and the 2nd in Slot 3 but I”m not positive slot 3 will run at 16x because of the NVMe (GB makes specfic mention of that sharing bandwidth with slot 3).

          The decklink card will work fine were ever you put it.

          I’d give Gigabyte a call just to confirm.

          1. Cool, thanks Robbie, I will give them a call. I was just looking at the manual and couldn’t find the part about the NVMe sharing bandwidth with slot 3, what page of the manual does it mention this? thanks 🙂

          2. wasn’t in the manual – its right on the specifications page of the link you sent

            ‘* The M2_32G connector shares bandwidth with the PCIEX16_3 slot. When the M2_32G connector is populated, the PCIEX16_3 slot operates at up to x8 mode’

          3. Ah ok yes I did see that, thanks, I don’t have a M.2 SSD connected so I am pretty sure that the M2.32G connector is empty and not populated – which would seem I should get the full 16x bandwidth from this slot, would you agree?

          4. my bad – I read your original post as a M.2 SSD. Not a regular PCI SSD. Yeah I think that would be full 16x then.

          5. Hey Robbie, just to follow up on this – I installed a second 1080 and all is working at full bandwidth, super quick in Resolve now! Thanks again 🙂

  4. Hey don’t know if anyone could answer this for me or not but I work from home and currently do not have the option for a static ip. My service provider does not offer one. Such a bummer 🙁 Would something like this work? https://www.noip.com/remote-access. If so would it be a big headache? I really need the vpn for the connected sql database. My wife and I both use the shared sql database for our business and it works awesome but would love access to it from anywhere.

    Thanks!

Leave a Reply

Hundreds of Free Tutorials

Get full access to our entire library of 900+ color tutorials for an entire week!


Start Your Free Trial
Loading...