Storage-Hard Drives

There are many different issues that need to be taken into consideration when evaluating data storage options for post-production.  Cost is a primary factor to consider, since without that issue, there would be little to weight against high end systems.  Performance is key as well, for efficiency as well as creativity.  Reliability is a frequently undervalued consideration, that effects everything from insurance prices to stress levels.  The ability to share data effectively can be an important consideration, since post-production is usually a collabrative effort.

The industry has responded with many different solutions, that vary in concept beyond recognition and in price by many orders of magnitude.  The earliest solutions involved video tape, analog replaced by digital recording.  Hard disks were introduced for random access to data, and now those are slowly beginning to be replaced by solid state flash chips.  Since this site is targeted to PC users, we will focus on hard disk based solutions, and the interfaces with which they can be accessed by a media workstation.

Hard disks are produced with five popular interfaces:  IDE/ATAPI, Serial-ATA (SATA), Small Computer System Interface (SCSI), Serial Attached SCSI (SAS), and Fibre Channel (FC).  IDE and SCSI interfaces are currently being phased out and replaced by their more capable and flexible Serial varients.  I know little of true Fibre Channel hard disks, but that format is rarely used in this industry.  That leaves only two options, which are now somewhat similar and compatible, SATA and SAS.  With identical connection cables, and both offered in 3.5″ and 2.5″ form factors, it is hard to tell the two options apart visually.  Their interfaces both support 300MB/s, dedicated buses for each drive, and port splitting when that is not required.

The biggest differences between SATA and SAS are performance and cost, which eventually distill down to one issue: size.  SAS disks have slightly more capable and efficient electronics, run fewer platter, with less data, and much higher RPMs and faster I/O and transfer rates.  SATA drives usually have much more storage capacity, lower speeds, and are always much cheaper.  At first glance, high end post production work would seem suited for SAS drives, since moving picture footage requires a higher data transfer rate than almost any other application of computing technology.

There are four other factors, which when combined, weight much more heavily in favor of SATA.  The first is price.  Since the difference in price per Gigabyte is currently so great, and SATA drives are not that different in their design or performance, a few quick calculations will reveal that while SAS disks have higher performance per drive, SATA disks deliver more performance per dollar, regardless of their storage capacity.  Second is that the infrastructure needed to aggregate the performance of multiple disks (Raid arrays) will be required, regardless of which disk solution we choose.  This is due to the fact that HD resolutions and larger require much higher data transfer rates than any single drive can provide (unless compressed, and even then, fast disk access is beneficial).  The marginal cost to increase the number of drives being aggregated will be low in many cases.  The third factor is that digitalized footage requires a tremendous amount of storage space, once again contributing to the need for many hard drives to be combined.  Lastly, most of the popular solutions to improve reliability, do so by utilizing even more capacity, to store redundant information in the form of parity, or straight backups.

These factors, when combined make a strong case for SATA disks, which have higher capacity at the expense of performance per drive.  If we are combining drives anyway, the performance benefits of SAS will usually be negated by combining more SATA drives for less money.  This is a case where quantity can clearly overcome quality in most instances.  As a side benefit, SATA drives usually have much greater capacities.

The only time when SAS may be favorable, will be when there is little need for high capacity, and when there is value to smaller solutions.  Fewer SAS disks are required to reach a given level of performance, and will therefore be more portable, require less power, and frequently generate less heat and noise.  For visual effects, were a few seconds of footage are manipulated at very high quality, or short commercials, SAS may be a more efficient option.

In most cases though, the numbers come down in favor of SATA by along shot.  Let’s imagine a two hour movie, with a 10:1 shooting ratio, giving us 20 hours of footage, and for the sake of example, let’s assume a data rate of 100MB/s.  With 3600 seconds in an hour, that is 360000MB an hour, or 360GB.  20 hours of footage would require 7.2TB of storage.  Add 10% to avoid disk fragmentation, and you need an 8TB array.  With 1TB SATA disks you need 8, plus two more to support Raid 50.  You will have the bandwidth of eight drives, and assuming 50MB/s each for SATA disks, and an efficient controller interface, that is 400 MB/s, more than enough for our 100MB/s files.  10 SATA drives at 1TB currently costs ago $3,000, and the Raid hardware will be required by both SATA and SAS, so it does not necessarily need to be factored in.  Now when onlining a production, not all footage is usually captured, but when you factor in captures, conformed exports, film and video colored versions, testless and texted masters, a 10:1 ratio will not be an inaccurate estimate.  Now I used round numbers, so that if the datarate of your format of choice is higher or lower, you can ajust accordingly. 200MB/s footage would need 20 disks, but could get double performance.  50MB/s footage would only need 5 disks, but could still expect 200MB/s of performance.  Have less footage, I left a 4x overhead in this example with 20 hours of source, but I also used 1TB drives for my calculations.  With 10 hours, 500GB drives show SATA to be even more economically favorable.

Now for a quick comparison to SAS, we start by noting that the maximum capacity is 300GB, and you can expect to pay at least $500 per disk.  Our 8TB example would require about thirty disks, assuming a Raid 50, striping together three Raid 5 arrays of ten disks. 27 data disks is 8.1TB for a cost of $15,000 in drives alone, not counting that it requires hardware for three 10 bay array enclosures instead of one.  From a performance perspective, assuming 80MB/s per disk, you can get over 2GB/s if you want to pay for an interface that fast, but remember that this is all for footage that is 100MB/s.  2GB/s might be good if you want to share it between multiple systems, but with that many users, usually multiple productions will be processed concurrently, requiring much more storage capacity anyway.  By multiplying up and down for different formats, it becomes clear that there is no way that SAS can economically catch up.

So I hope this successfully establishes that SATA disk drives will almost always be the drive type of choice for post-production environments.  I plan to examine the different options for connecting these drive arrays to a workstation or group of systems in my next post.

Tags: Fibre Channel, SAS, SATA