Monday, August 10, 2009

WTB External Storage Advice, PST

My brother is a photographer, and recently had a hard drive take a hard crap on him. There are wedding photos for a client involved, and the drive is currently in 'the clean room' of a data recovery place, being picked apart. Gnomes will go in to grab the data (hopefully), and all will be well in the end. His lesson learned from this adventure is that external hard drives are no good unless the data exists somewhere else as well. He's new at this 'being paid to shoot pictures' biz, so this experience is really driving the 'have your client's data in two places at all times' point home.

None of this is news to me. I did IT for 6 years, and was the one to push for a tape system for my office when they had none. It was such a huge up front cost for the hardware, and was the kind of thing that's a really hard sell to the bean counters. You buy these huge robot tape drives, and hope you never need to have them.

But if anything SHOULD happen, they pay for themselves immediately. Our tape library paid for itself many times over, because our employees made dumb mistakes. Hooray for me.

So now here I am explaining the concepts of RAID 5 setups and data redundancy to my brother and father (dad's an illustrator), and trying to drive home how grabbing four drives nets you three drives worth of space, and if any one single drive should fail, then the system is fine as long as you replace the dead drive with a new one before another goes out. Then I realize two things:

1) they're both on laptops, so PCI raid cards aren't an option.

2) I've been out of the hardware loop, playing in elementary schools, for way too long. Raid 10 and 50 are new to me, and I need to dig around on wikipedia to catch up.

So I'm turning to you guys to throw me a bone.

I've basically done some research, and realized that, without a tower to throw a raid card in, I'm looking for an external, self contained, storage unit capable of providing something better than just mirroring. This leads to products like this or this, which are actually NAS boxes. You fill them up with HDs, plop them on a gigabit hub, and jack into them through ethernet. The problem is that most of those are aimed at frat houses full of dudes sharing porn and mp3s to Beta Kappa Beta or whatever, and we really don't need anything like that. I'd even prefer to just have a USB cable poking out of the back for sustained writes and file copies. Is there anything out there you guys know of that:

a) support at least RAID5 (that is, at least 3 or 4 drive bays)

b) don't require a raid card (are fully self contained)

c) won't break a bank.

1TB drives are about 99 bucks these days, so I'm looking in the 500 dollar range. I had a $399 2TB Buffalo unit (4x 500GB) and gigabit hub in a physical shopping cart today at the local shop, then thought better of it, and decided to research more. It turns out that was the right choice, since reviews are saying that sustained transfers drag their feet on these. My brother will be dumping scads of 12+ megapixel RAW images on it at a time, and my father will be opening huge photoshop documents across the connection.

If NAS is my only option, I can train them to work locally, then shove the data across at the end of each session, but these guys are noobs, and I just want to teach them to 'use the RAID device for anything with dollars attached to it', everything else can go wherever.

Dinner's getting put on the table, but I wanted to throw this bottle out into the internets sooner than later, and hopefully get some feedback for any kind of options I haven't come across yet.



Geoffrey said...

If it was me I would not do a NAS. For a little more, you can just build an inexpensive server. Use hardware RAID1 or RAID5 or whatever, bootleg Windows 2003 or 2008 server, and you are set. In addition to RAID, you can do shadow copies and a bunch of other neat server things. You can set up a terminal server or vpn or Active Directory, Exchange, whatever.

Ixobelle said...

yeah, the problem is the budget, and the lack of any know how on these guys' parts. For me, definitely, a Linux Samba server all the way. I could throw apache on there with MySQL and host guild sites, use it as a firewall/router, get all kinds of fruity with it. But even a windows server that runs itself would require more setup on my end (and random phonecalls at all hours), and again... the budget. A cheap system is still a full system.

I want a Raid Array with a USB connection.

Joe said...

What you want is a Drobo. But, it's going to cost you. $399 with no drives and no network functionality. Add $100 for the NAS portion. Then add your drives.

$500 is a little tight for an external RAID system for someone with insufficient nerd points to roll their own. You can also look into some of the Windows Home Server things, but they're about $500 with minimal non-redundant storage.

Geoffrey said...

I would just get them to sign up for then. For $50 per year, it's pretty cheap and you will have zero support.

The problem with the NAS things and the way you are proposing what they do, they are still asking for trouble. Except that now the point of failure is the NAS, not the local hard drive. I understand RAID redundancy, but still, the controller or the motherboard on the NAS or something like that could go out. So now you have to back up the NAS to have a good system.

Anyway, as a past network admin, I am sure you realize that RAID is no replacement for a backup, as it does not allow you to go back in time.

So now you are left with the option of just letting them use their laptop for primary storage and just attach an external USB hard drive and try to maintain backups, or go with an online system. Carbonite works well enough for me and about 4 people I know to where I can recommend it.

Richard Dallimore said...

Check out FreeNAS

its BSD based, buts boots from a liveCD - you can store the config on a USB stick. Take any old PC and throw in as many drives as you can, FreeNAS can be configured to use the drives as whatever RAID you like, has lots of other cool features too. The good news is its Opensource - just supply the old hardware and drives

Jesse said...

Joe has is it right. Drobo costs about the same as a linux-based NAS, but is much easier for non-tech people to set up.

Hatch said...

I feel sorry for your brother, since my sister's wedding photos got lost exactly the same way, and she, her husband, and our families were really angry with the photographer and never forgave him. Let's just say it wasn't in his best interests to be in the same room as anyone in my family after that.

At least the families were taking so many photos that she ended up with a nice collection anyway.

Unfortunately, I can't help you at all with your technical issue. I'm one of those artist types who stopped doing math as soon as I got out of high school. I'm computer-savvy (and even used the command prompt to change the file system on my external hard drive the other day so it could accept very large files!), but not technically knowledgeable. Good luck!

Ixobelle said...

those drobos do look pretty hawt, and would be basically what we were looking for, but I kind of came to the conclusion that I was being too fancy about it.

Once I come across some sort of IT problem, I tackle it with my own level of knowledge, which is demanding raid 5 in this situation.

Then I realized the bottom line is that they just need raid 1, and are fine to get some dual disk externals. They like having the simple plug and play, and I can explain that one drive being 'wasted' out of a 2TB drive (giving you 1TB of space that's duplicated) is the easiest way to go about it.

thanks for the links. I've got Drobo and FreeNAS bookmarked for my own tinkering later. Especially FreeNAS looks like a cool project for me, since I have almost enough leftover parts from all my upgrades to make a NAS for myself.


oshin said...

Does your brothers laptop have an esata port ? Newish laptops should have one. Esata is basically sata with a rugged connector. It be possible to setup some kind of array with it, but there might be some fooling around etc. But I know you love that shit really !

Anonymous said...

Good call on not getting a buffalo.
We had a 2TB rackmount Pro model shit itself at work. it was in raid 5 mode. 1 drive died and it failed to recover.
Buffalo was little help on recovering it. they basically gave me a link to a wiki page with some crazy hacks, none worked. even mounting the array manually on a windows box with special software would get the data back.
Fun fact is it uses software raid not a hardware controller. XFS filesystem. the thing was also painfully slow.

Some of those NAS devices you see might include usb/firewire connectivity. for direct access.

unfortunately i do not have any recommendations for you. the drobo always looked like a nice idea because you don't have to put matched drives into it like most others. and it is direct connect only. unless you buy the addon.