RAID-5 Crash Video: How Not to Keep Data Recovery Possible

After a RAID-5 crash, your data is especially vulnerable. How you treat your crashed server before you bring it to a data recovery lab can have a huge effect on a data recovery lab’s chances of success at recovering your data.

This RAID-5 data loss scenario was about as bad as it gets. The prospective client came to us after their five-drive RAID-5 array had crashed. But unfortunately, they didn’t come to us right after the server had crashed. They had taken the three remaining healthy hard drives in the array, arranged them into a new three-drive RAID-5 array, and started loading their backups onto the array. No harm, no foul, no need for data recovery—until they realized their backups were out of date. There was important data on that crashed RAID-5 array that didn’t live anywhere else on their system!

And so, the client had five hard drives. Two had failed. The remaining three had all been filled about two-thirds of the way with next-to-useless data. When you write data to a sector on a hard drive’s platters, the hard drive doesn’t keep a backup of that sector. If you had important data living there, and then wrote new data to that sector, there is no way to get the old sector back. Period. End of story.

And so this would be a situation in which maybe one-third of the data on the three healthy drives would be relevant to the client. Of the two failed hard drives, one would be filled with “stale” data, months out of date compared to its peers (depending on when the first drive failed). In addition, there was no telling what condition these two failed drives were in. There could be any degree of platter damage affecting them, from tiny scratches to severe scoring. To make matters worse, the client’s critical files lived inside a Hyper-V virtual hard disk file.

This RAID-5 crash was a nightmare. Watch Brian Gill’s breakdown of the situation to understand just how much of a complete and unmitigated catastrophe the prospective client’s situation was:

Low Odds for This RAID-5 Crash

Rarely will Gillware explicitly turn prospective clients away like this. We pride ourselves on the incredible skills and talents of our data recovery experts, who have successfully salvaged data from thousands of crashed RAID arrays of every stripe over the past 12 years. But given the prospective client’s uniquely terrible situation, we had to be frank with them and let them know just how astronomically low the chances of a successful recovery would have been. Odds were that the vast majority of their most critical data had been completely overwritten at worst and partially overwritten at best, with maybe a handful of small files escaping the cataclysm unscathed.

When you’re dealing with a crashed RAID-5 array, one wrong move can turn your whole situation around. And not in a good way. Before the prospective client had loaded up their (out-of-date and incomplete) backups, this would have been an ordinary RAID-5 data recovery case. Our RAID recovery experts would have eaten it for breakfast. At worst, one or both of the array’s hard drives might have suffered severely scoring, potentially drastically reducing the quality of the recovered data. But even that would have been an enormously preferable scenario to this.

Keeping Data Recovery Possible After a RAID-5 Crash

We here at Gillware have a few pieces of advice regarding RAID-5 arrays to make sure that this kind of RAID-5 recovery nightmare scenario never graces your doorstep. If you follow our advice when setting up and maintaining your array, you won’t have to worry about irretrievably losing your data if your server crashes—and may not ever ever have to worry about sending your server to our data recovery lab at all.

As a final word of advice: Take care of your backups. Far too many people running RAID-5 arrays forget about backups. They lapse into thinking the single-drive fault tolerance of a RAID-5 is enough to protect them from disaster. But it isn’t. And while this client did have backups, they’d neglected to keep them current. As their data center’s needs and their setups changed, they fell prey to configuration drift. With each passing week, their backups drifted farther and farther behind their current state of the union. When the time came and they needed their backups, the backups they had were useless.