gear-related Storage workflow advice?

[Tabonga Films]

Hi, lengthy post here. I’m looking for a second opinion on my storage plan, would love to hear people’s thoughts. I’ve only started to look into storage solutions in the past 2 weeks. And I’m a student, on a student’s budget. I’m willing to spend up to $1000-1500 AUD.

Previous system:
I stored all my personal work on a 2TB Toshiba external drive. I also worked off this drive. I backed this drive up onto a 2TB Seagate external drive using Time Machine.

This is the new system I’m planning on implementing (I haven’t bought any of these items yet):

PROJECT DRIVE:
Samsung T7 SSD (1 or 2TB).
Edit all current projects on this drive – when the projects are completed they will be deleted off the drive.

PERSONAL WORK_VOL. 1:
10TB Seagate Iron Wolf 3.5” HDD in 2 or 4 bay enclosure.
Back-up current projects regularly to this drive, which is always connected to the computer. This drive contains all my past projects as well as my libraries, stock footage and other archival materials.

ARCHIVE_PERSONAL WORK_VOL. 1:
10TB Seagate Iron Wolf 3.5” HDD
OR
10TB WD My Book desktop drive.
This is a duplicate of the Personal Work_Vol. 1 drive. Either manually or using a program like Time Machine I’ll back up the 10TB Seagate Iron Wolf to this drive. This drive will then be disconnected and placed in another room in a waterproof and shockproof storage case. Back-ups will be at weekly or monthly intervals, depending on the rate I’m working at.

If I use the Iron Wolf, I would be using a docking station like the Sabrent USB 3.0 to SATA External Hard Drive Lay-Flat Docking Station for 2.5 or 3.5in HDD, SSD (Original w/Fan).


I had originally considered just using 2x 3.5” HDDs in the enclosure with a RAID 1 configuration, but I wasn’t comfortable with the idea of the only 2 drives containing all my past projects and archival material both living in the same enclosure, always running, always connected to my computer. Thus the 2nd drive living in a protective case in another room.
Also I just can’t afford to have 2x 10TB drives in the enclosure AND have a 3rd external back-up drive.

For the enclosure, I’m considering options from QNAP or Icy Box, specifically the QNAP TR-004 or the Icy Box IB-RD3640SU3. Again, I don’t think I’ll be needing hardware RAID capabilities as I want the back-ups to be physically separate for safety.
Not sure whether to go with 2 or 4 bays, as I will want to always keep the Personal Work_Vol. 1 (and onwards) drive in the enclosure to allow me to access archival material, which I re-use.

https://www.amazon.com.au/TR-004-St...qid=1656381171&sprefix=qnap+00,aps,826&sr=8-1

https://www.skycomp.com.au/icy-box-...L32naj-0rynW5G3lhI357dgxoluJFBGhoC_FYQAvD_BwE

I should also note that I’m currently working on a late 2013 Mac Pro with a 3.5 GHz 6-Core Intel Xeon E5 processor, 32GB of RAM and a 512GB SSD.
Apologies for the rambling post, thanks for reading all the way to the end. Any suggestions would be much appreciated

 

[Nate North]

It actually sounds like you are headed towards the correct configuration already.

The setup you outlined is pretty much what I use for professional work. A fast SSD 2TB for current use files, a raid for bulk assets and storage, and in my case, a number of disconnected archive drives, where long term data storage of output and redundancy backups are stored.

Hard drives do go bad, and you do suddenly loose all your data in some instances, so you're headed down the right path setting up to always have 2 or 3 copies of anything you can't afford to loose.

Film use can be hard on drives, with a lot of heavy read write all the time, so my average experience is to expect any drive to die between 4 and 8 years after use begins.

Something that will save you a lot of time and money over the long haul is to compress out of use footage (like a movie you shot 5 years ago and no longer work on) and store it as highbit 264 or similar. You really only need the Raw footage through post production, and after that it can start taking up huge amounts of space. Compress your old files and 10TB will last you through at least 1 or two feature films.

As best as I can determine, your question is about whether to purchase an archive drive or a raid first, not having the funds for everything at once.

1. Both raid drives dying at once is extremely rare. This is the whole point of raids, that if one drive dies you already have an intact copy. I'd say your odds of that happening are pretty low, though it does cascade on someone once in a great while.

2. Drive failure rates increase over use, so this first year is the safest of all years. You can find drive failure stats for brand x online.

If you're making student films in college, I'd forgo the RAID and just back up anything irreplaceable onto a usb external. It's a good way to keep track of what files are actually important. If lightning strikes your computer, you've still got your most critical files in the closet, without wearing down a second drive with redundant daily use. (since the raid drive is mirroring EVERY change on your drive)

Take it with a grain of salt, because I'm paranoid about storage failure. Mostly, I buy multiple backup drives, and the first drive never fails, but better safe than sorry.

 

[Tabonga Films]

It actually sounds like you are headed towards the correct configuration already.

The setup you outlined is pretty much what I use for professional work. A fast SSD 2TB for current use files, a raid for bulk assets and storage, and in my case, a number of disconnected archive drives, where long term data storage of output and redundancy backups are stored.

Hard drives do go bad, and you do suddenly loose all your data in some instances, so you're headed down the right path setting up to always have 2 or 3 copies of anything you can't afford to loose.

Film use can be hard on drives, with a lot of heavy read write all the time, so my average experience is to expect any drive to die between 4 and 8 years after use begins.

Something that will save you a lot of time and money over the long haul is to compress out of use footage (like a movie you shot 5 years ago and no longer work on) and store it as highbit 264 or similar. You really only need the Raw footage through post production, and after that it can start taking up huge amounts of space. Compress your old files and 10TB will last you through at least 1 or two feature films.

As best as I can determine, your question is about whether to purchase an archive drive or a raid first, not having the funds for everything at once.

1. Both raid drives dying at once is extremely rare. This is the whole point of raids, that if one drive dies you already have an intact copy. I'd say your odds of that happening are pretty low, though it does cascade on someone once in a great while.

2. Drive failure rates increase over use, so this first year is the safest of all years. You can find drive failure stats for brand x online.

If you're making student films in college, I'd forgo the RAID and just back up anything irreplaceable onto a usb external. It's a good way to keep track of what files are actually important. If lightning strikes your computer, you've still got your most critical files in the closet, without wearing down a second drive with redundant daily use. (since the raid drive is mirroring EVERY change on your drive)

Take it with a grain of salt, because I'm paranoid about storage failure. Mostly, I buy multiple backup drives, and the first drive never fails, but better safe than sorry.

Hey Nate, thanks! I'm glad to hear that I'm not way off in my approach. Your system is what I aim for when I can afford it. Definitely agree with your point about compressing older projects.
I'm paranoid about storage failure too, ultimately I reckon I'll go with the above set-up as I do not want to lose the past 10 years of my work.

 

[Nate North]

If you're looking to be a lifer filmmaker like myself, you may at some point consider permanent storage disks, which are somewhat expensive, but never degrade. They are only for archiving, and can't be used directly. Long term though, it's the only way to store large scale data permanently.