REVIEW-CALDIGIT HD PRO
Not long ago...January 2007...I was at MacWorld in San Francisco. There was a LOT of buzz going around about a great many things. The iPhone was announced, Leopard was demoed, a company converted a MacBook into a tablet computer...there was no end to the cool things to see. But all the companies that were there showing off storage solutions...RAID solutions...were hit by one big question.
"Does this do Raid 5?"
Well, I can tell you, that a VAST majority of the answers were "Sorry, no. Only Raid 0 or Raid 1." This turned many people away. The only people who could say "Why yes, we do!" were the Fibrechannel solutions. But, as you might suspect, those solutions were EXPENSIVE. Restrictively so for many of us.
My how technology flies.
Here we are...8 months later...and the companies who were at MacWorld (and NAB) heard the cry. One such company was CalDigit. Their offering, the HD PRO, offers what everyone desires...hardware Raid 5 support. For a device to offer “Hardware RAID” it must have 2 things: An IO Processor (CPU) and a Cache Memory. This unit has both.
What is Raid 5? Well, let us first start with the lower numbers.
RAID 0 is what is called PERFORMANCE RAID. What you do is take two or more drives and "stripe" them together so they act as ONE DRIVE. "Striping" essentially means you format multiple hard drives together as one large "virtual" hard drive for your system." For example, if you take four 250GB drives and stripe them RAID 0, you'd have a 1TB Raid. In doing this, you spread out the information that is being written onto all the drives so that you get faster data transfers. This means that you can capture uncompressed video formats and play them back. The more drives you add, the faster the raid, the better the performance.
RAID 1 is known as a PROTECTION RAID. You take 2 or more drives (need to be in pairs) and group them so that they again appear as one drive, but this time the information is MIRRORED. If you took two 250GB drives and striped them as RAID 1, you'd have a 250GB drive, but the information would be secure because the information on one drive is copied to the other drive as well. The drives are exact duplicates of each other. The good thing is that your media is safe, the drawback being that your drive performance isn't as great, and you cannot really work with uncompressed formats.
RAID 5 is the best of both worlds. Performance with protection. RAID 5 stripes both data and parity information across three or more drives. It doesn't have one dedicated parity drive that contains all the information, but rather distributes the data and parity blocks across all the drives in the array. "What the heck did you just say Shane? What does all that mean?" To tell you the honest truth...I don't understand the inner workings of the engineering. I am not an engineer, I am a creative editor. But I DO know that it basically means that your footage and the backup information is not on one drive, but spread across many. So that if one drive fails (and drives are known to do this, more often than we care to admit) your information is still safe.
If you want to know more about the RAID types, go to the Wikipedia. Enough with the schooling...let's get to the HD Pro itself.
First off, I must mention that currently the HD Pro ONLY WORKS WITH PCIe MACHINES (PCI-X support due by mid-September I hear). This means it works only on the newer MacPros, and the older G5 Quads withe a PCIe slot. Now, I don't have one of these machines, I have a PCI-X G5 with a Kona LH capture card. I needed a PCIe machine with capture card in order to do this test. So for the purpose of testing, I was loaned a MacPro (Dual-core 2.66Ghz model) with a Decklink Multibridge Pro capture card.
When I went to unpack the unit, I encountered something similar to what I encountered when I unpacked the S2VR Duo unit. The hard drives were not inside the unit. They were all tucked into little cut out sections of the packing foam that encased the drive chassis. This is clever, for it helps prevent damage that can occur by jostling the unit. We know how gently delivery services are with packages, right? So I pulled out all the drives from their safe cubbies, removed them from the plastic wrap, and installed them into the chassis. They were all conveniently labelled A0, A1, A2 so I knew what order to install them.
Another interesting thing is that the card that you install inside the MacPro is a VERY small and basic card. It is a 4-lane one port card, with no real electronics to speak of. The card doesn't have the controller on it. Nope...it is simply a BRIDGE from the board to an external connection that a PCI Express cable connects to. The controller is INSIDE the chassis. To quote the website, " This direct link eliminates latency introduced by the conversion of other interfaces to PCI Express and provides superior bandwidth, availability and deployment flexibility over earlier-generation SCSI and Fibre Channel technology." The reason for this is two-fold. First, it doesn't contribute to the heat inside your MacPro...since you will have plenty with your DOUBLE-WIDE graphics card and your capture card. And second, it allows for controlling MULTIPLE units with the aid of a PCIe switch (not out yet). Meaning that your machine can control more than one HD Pro box...you can add more units as the need arises. Just like I can now with my eSATA boxes and my port multiplier card. Room to grow.
The test unit I received was a 4TB unit. Eight 500GB drives. When I fired up the unit it appeared on the desktop, ready to go. It was already configured...Raid 5 and everything. So I ran the AJA test with the following result:
331 Write / 372 Read. RAID 5!! Plenty fast for uncompressed HD...a couple streams of uncompressed HD. Niiiiice. But, me being me and not trusting the integrity of a drive format after shipping...and, well, to see what is involved in setting up a RAID 5, I did it myself. CalDigit includes a handy little pamphlet on how to do this, so I won't go into the boring details. But I will mention that it did take 3 hours to build the RAID...which is slightly faster than I hear RAID 5 takes to build on an XServe. Do note that the size of your Raid determines how long it will take to build. 8TB will take longer to build than 2TB. Makes sense. So, I let it do it's thing while I ate dinner, played with the kids, and tucked them into bed.
When the building was done, I ran the AJA test again:
311 Write and 402 Read....again, niiiiice. The Write was a little slower, but who really needs more than 250 anyway? I am not CAPTURING multiple streams...so 300 to 340 is fine. The Read increased, which was cool. And I suppose the average will be about 320 Write and 385 Read...which is VERY nice indeed.
I captured a few different ways. First DVCPRO HD native via firewire. No sweat...but I can do that with my G-Raid. So then I captured uncompressed 8-bit 720p...no problems. Uncompressed 10-bit 720p...no problems. Uncompressed 10-bit HD...passed again without one dropped frame.
OK, time to abuse this thing. Again I set it up to capture uncompressed 10-bit, and about 5 minutes in...I YANKED A DRIVE OUT! Just so you know, this isn't something you can do by accident. There is a pinhole that you need to push an included tool into (or a paperclip if you lost the tool) and that pops open the handle that you then use to disengage the drive and pull it from the chassis. When I did this it did give me this nice little warning:
But, it kept merrily match capturing...not missing a beat. When it was done I saved my project and set about fixing what I intentionally broke. I pushed the drive back in and went into the RAID SHIELD software and unlocked the drive and the software went about rebuilding the Raid. Again, that took about 3 hours. But the footage I captured was still intact and played fine.
OK, now let's look at playback. I was able to get 9 streams of DVCPRO HD playback (both firewire and Decklink encoded DVCPRO HD)...I resized my footage to look like the opening to THE BRADY BUNCH. The first 4 streams gave me no render bar whatsoever. When I hit 5 streams I encountered the dark green render bar, which means high quality playback. Eight streams got me the light green render bar....low quality playback...and that held up until I added the 10th stream...when it went RED. So i stuck with nine. Since it was getting late, and I had work in the morning, I just let it play this footage as I went to bed, setting Final Cut Pro to loop the footage(did I mention I was testing this with Final Cut Pro? No? Sorry).
I woke up, went to work, came back home, had dinner, then went "oh yeah, I have something going on in my office." I turned my monitors on and there the footage was, happily playing along. No dropped frames.
But hey...seriously....DVCPRO HD footage? Was that a REAL test? OF COURSE NOT! So I went to the 8-bit uncompressed 720p footage. I started layering that. I got the dark green render bar after two layers, light green after 3 and the red render bar when I hit five. So four streams it is. I tried 10-bit uncompressed and couldn't get more than one. But really...uncompressed 10-bit...I am not offline editing with this codec. I am FINISHING with it, so one stream is plenty. I am offlining with DVCPRO HD, and getting nine RT streams is a nice thing to have. OH, and I was able to add the 3-way color corrector to seven streams and stay RT.
Time for bed again, so I resized the uncompressed 8-bit footage into four windows, and pressed play. This time I let it run for 3 days. Every time I came back, it was running. On the third day when I got back from work, I walked in, opened the unit, and yanked a drive. I turned on the monitor and again, saw the warning, but playback wasn't affected. Kept going and going and going. So I let it go. Went to bed. Woke up the next morning to see it still humming along...footage playing. I stopped...saved...quit...and set the raid to rebuild again. And again, when it was done, all my footage was still there...safe and sound.
The HD Pro unit was running for the entire 8 days that I had it...I didn't turn it off. When I finally did, and I removed a drive and felt it...is was warm to the touch, but not overly hot. The RAID SHIELD software has temperature monitoring capabilities, so you can always check and see how hot things are. OH, and I forgot to mention that you can set up e-mail notification so when a drive does happen to die and you are...somewhere else...you'll get an e-mail telling you a drive failed. Very handy.
So I am rather impressed with this. And I would easily consider it for use in my edit bay. The piece of mind that RAID 5 gives you is something EVERYONE should experience. Currently I am running a RAID 0 box, and have all the footage backed up to a couple firewire drives. A sort of manual RAID 1. With this box I wouldn't need to waste the drive space. It would be an invaluable asset to any edit bay.
OH! OH! OH! AND...oh I love this...AND CalDigit makes an Express34 card for this unit, so you can use it with your MACBOOK PRO! You won't get nearly the numbers you get with the tower, but the numbers you do get are QUITE impressive:
168 Write and 195 Read. PLENTY for uncompressed SD and even uncompressed 8-bit HD work. And for DVCPRO HD and the new ProRes format, that would be plenty for several streams of playback...ON A LAPTOP!
Man...time for me to get a MacPro...or MacBook Pro.
8 comments:
Thanks for the write-up, Shane! Sounds like a great box. Maybe someday I'll get past my reservations about Raid 5 products, which are:
Reservation #1: Raid 5 doesn't protect against accidental erases/changes, or corrupt files.
Reservation #2: Vendors are now selling boxes with hard drives built in, and jacking up the prices by quite a bit. The HDPro is $7999 for 6 TB. That's with 8 750 GB drives, but I can buy my own 750GB models from MacSales.com for about $220 each, which comes out to about $1760. So really, is the encloser and the RAID 5 capability really worth an additional $6239, especially when I'm still paranoid about accidentally erasing a file, or changing it in some unwanted way?
That's why I tend to favor Raid 0 boxes, backed up nightly to another box. It's not perfect, but it's a lot cheaper.
About Reservation #1 - is there a RAID version that does this? Not that I know of. So if you need that, sticking to the RAID 0 with backup will be your best bet. And if it works for you (as it does for me currently), then stick with it. With Raid 5 you can still back up for that ACCIDENTAL CHANGE protection.
About Reservation #2 - I think you can find boxes out there that you can get your own drives for. But the reason that the companies do this isn't to gouge the customer, it is for legitimate reasons. The companies make sure that the firmware on the drives match the firmware of the Raid chassis...that ensures better performance and stability. Also, TECH SUPPORT, which is huge. If something doesn't work, you have one company to call, not several. What typically happens when you have mulitple companies involved, is that they all to the "blame game." The chassis manufacturer will blame the drives, or the cables...or the card. The card manufacturer will say that the drives firmware isn't current...and so on.
And yes, I'll admit this is an expensive option. This is why I haven't gotten one yet. I built my Raid 0 tower for $1800 (a year ago...so more expensive) and currently use that. When the job comes along that will pay for the Raid 5 unit...I will LEAP on it. My Raid 0 unit did have a drive failure (soft failure, not dead drive) which made the raid go bad, so I had to copy the info back from my backups. Which was fine, but that is a LOT OF DRIVES used for this project. If I had the Raid 5, I'd be fine with that, and then I can start using the other drives to archive my P2 footage, or to keep footage from past projects online longer, JUST IN CASE changes need to happen.
But hey, if what you have works for you, go for it man. I do what you do now too, and I like it. Not knocking it.
My big question is how this stacks up to the Dulce Quad. It seems to ba a similar set up, and I don't see a big difference in functionality except the price, and that the controller is PCI-e or X. And there's no laptop controller.
I'm like to be recommending such a system soon, and not having to get an expensive X-Serve is really sweet.
Well, Bob Zelin did a review of the Dulce Systems unit, so you can look at both that and this. We both haven't tested the other units, so...that makes things difficult. But he might be getting one soon to test.
Big differences I saw just reading? Dulce systems RAID buids faster. I hear it builds the parity as it captures. Because of that, it runs a bit slower than the Caldigit HD Pro. Also, the Dulce Systems uses a third party card....the ATTO. If you have an issue with the system (as I mentioned) then finger pointing might occur. CalDigit makes everything. Oh, and CalDigit guys told me PCI-X support will come soon....mid September. Which is good, because I might need one of these in December for my next gig.
OH...and the HD Pro doesn't have a 2 meter cable limit. Much longer. I'll have to get more numbers when I meet up with the Caldigit guys at IBC.
Greetings,
It’s Trish Buendia from CalDigit Inc. Came across Shane’s comments and I’m really noticing the same question with all end-users… “What’s the difference?” Well, there are quite a few differences, especially with this new product.
The HDPro is way more than just a box with eight drives. The engineering is complex, but the concept is simple; RAID Storage that provides performance, scalability, and security.
How is it scalable? Unlike most of the RAID card solution in the market right now, HDPro is expandable… you can add as many units as you want so it is virtually unlimited capacity. It is SAN ready; our engineering team is working on a SAN software. Once the SAN software is ready, you can do the SAN using multiple HDPro, multiple workstations doing the sharing. Because it is upgradeable, the value will not decrease from your investment.
How is it secure? Everything is designed to work 24/7. For example, the HDPro provides hot-swappable fan module and redundant power supply. If any one of the power supplies or fan dies out on you, you can continue your workflow with zero downtime. HDPro is protected by RAID Shield; if anything goes wrong it will send an E-mail notification. Without having to add an additional cable, the RAID Shield will give an instant read-out detail status of the: RAID, drive, fan, power supply, etc. You can easily change the RAID setting and update the RAID firmware through the RAID Shield.
How do you get performance? One HDPro under RAID 5 can easily handle multiple streams of uncompressed HD and up to 2K workflow.
Here's a link to the page for the product: www.caldigit.com/hdpro.asp
Hope this helps :)
Trish Buendia |CalDigit Inc.
I had a RAID 5 setup, like you I was so happy with the idea you are SAFE from drive failure and dropped all my media on the 1TB RAID (5x250GB drives, biggest drives you could buy at the time)
But here is the thing:
A RAID 5 can die as horrible a death as any RAID 0, the thing is: 5 drives means 5 chances of failure and with all drives being exactly the same age the chance is if one fails the chance they will all fail around the same time is sooner then later.
RAID 5 is cool, the speed is fantastic but the chance you will trust a system like this without triple copies after a failure is zero. You will still need to backup a system like this wwith the added problem of: how on earth do you backup 4TB or even 1TB without spending double the money.
Why make this point? Well simple: with all the P2 and soon X2X HD shoots you will no longer have a tape backup to fall back on. You will have to think about a workflow that consists of making a work copy and a archive copy both at the same time.
I still have a lovely computer here with a brilliant RAID5 controller that doesn't use any CPU and gave me over 150MB / sec read and write but i will not use it again and replaced it with a Mac Pro with 4xx 500GB non raid setup.
I'm waiting for a RAID 10 setup, no seriously no joke :)
Well, I'd NEVER store my master P2 footage on my RAID drives. I keep my P2 files on a separate hard drive (still exploring other more stable archiving options) and IMPORT them into FCP and store the imported files on the Raid. Or, if I opted to use Raylight, I'd copy the files to the Raid, and retain a copy elsewhere. Just like tape...you don't capture the tapes then toss the masters.
SO when a drive fails, I simply reimport the P2 footage, and recapture the tapes.
Hi Shane,
Thanks for this detailed write up. I invested in the S2VR HD a little while back on your advice and it seems when I get the finla go-ahead on the new series. I'll have to plan on this unit for the second edit bay.
Thanks again
Tim
Birdman Productions
Choteau, Mt
Post a Comment