Post Production Media Storage and Drive Arrays

After my last article explained why USB3 is not ready to be used as the interface for primary media editing drives, this has led to the question of: what interface should be used for attaching media arrays to editing workstations for maximum system performance?  Even more so than with basic external hard drives, there are a number of competing options, none of which have emerged as clearly superior.  At this point the answer really varies a lot depending on your needs and budget.  It has been three years since my last article on the subject, so the market has changed considerably since then.  The primary options as I see it, are eSATA, SAS, external PCIe, and Fibre Channel.  While older parallel SCSI based devices are technically still available, the interface has no real advantages over even eSATA, let alone the more expensive options.

eSATA is of course based on the popular SATA interface, and connects an array of drives to a system in their native interface.  Most RAID redundancy is usually accomplished at the controller level, with a PCI expansion card inside the workstation.  Some arrays are capable of creating integrated RAID sets, which are presented to the host system as single large volumes.  One of the advantages of this approach is that the array can be connected to other systems without needing to match the internal RAID controller card for the array to function properly.  While a single 300MB/s SATA channel will be sufficient for simpler compressed HD workflows on a budget, most professional arrays for high end editing systems will need more bandwidth than that, so most large eSATA arrays connect to the workstation with 1 or 2 four channel cables, using a variety of different physical interfaces depending on the vendor.

External SAS connected arrays function in much the same way as SATA based ones, but with a few advantages, that usually come at a significantly higher cost.  SAS is a full duplex interface, and the command set is based on SCSI instead of IDE, allowing higher performance and throughput.  More expensive SAS arrays also support multipath signaling, for greater redundancy in the supporting electronics. (As opposed to the redundancy provided at the disk level by RAID configurations)  SAS also supports much longer cable lengths, up to 10 meters or 30 feet.  This can be advantagious for quiet video editing rooms, since the disk array, which is usually the loudest part of the system, can be located farther away from the users.

A number of vendors have now begun offering external arrays that interface with the host workstation via a direct extension of the PCIe bus.  This allows all of the RAID functionality to be contained within the array, and gives full speed access to the data as if it was contained within the machine.  Among the advantages of removing the RAID functionality from an internal add-on card, are that it can be attached to a laptop via an ExpressCard, which uses the same signaling protocol as PCIe, and that with addition of a few cheap pass-thru cards, an array can easily be moved between systems.  This is definitely not a hot swappable solution, since it accesses the PCIe bus directly, which is initialized at bootup on most systems.  But if your main edit system has a total OS meltdown at a critical point in your project, it should be much easier to access your data from a different system than if you needed to reinstall the PCI SATA RAID card somewhere else, and allow you use your laptop as a backup edit system in certain instances.

Fibre Channel is by far the most expensive option.  Every part of the system is more expensive, the PCIe HBA cards, the fiber cables, and the disk array controllers.  On the otherhand, Fibre Channel offers capabilities that none of the other storage options really do.  It is a hot swappable interface, running on fiber cables that can extend access thousands of feet if desired, and can easily be networked and shared.  Devices can be connected directly together, shared in an Arbitrated Loop, or all attached to a central fibre switch for simplified management.  It is an efficient and low latency interface, and is available in speeds of 1,2,4, or 8Gb per second, and multiple channels can be combined for higher performance.  Higher speed devices are usually backwards compatible with older hardware, similar to the way ethernet works, allowing you to upgrade your storage network one piece at a time.

Choosing the right storage solution depends on your immediate media needs, your available budget, and the direction you anticipate growing in the future. SATA based solutions offer all of the speed you could need if scaled large enough.  SAS can offer similar performance in a smaller package, but at a higher cost.  Sharing data beyond gigabit network speeds requires a storage system that can interface with multiple computers, but that comes at a significantly increased initial cost.  Investing in Fibre Channel storage is usually only worth the expense if you anticipate the need to share your data on a SAN, either immediately or at some point in the future.  I will examine a few popular shared SAN options in my next post.

Tags: Compression, Fibre Channel, PCIe, SAS, SATA