Though so many things have gotten more reliable over the last 20 years in the land of computing, hard drives are not one of them. The failure rates are about the same as they have always been, when talking about unrecoverable read errors, and the other components in a computer system have progressed by levels of magnitude.
A story alluding to problems with this was written by Robin Harris on ZDNet a few months ago, and not many seemed to take notice. Now, another notice about the impending problems with RAID arrays, simply because of the massive size of the drives, and the mathematical surety that problems will occur beyond the capabilities of the system to repair. (A system of guessing has not yet been implemented!)
storagedude sends in an article claiming that RAID is nearing the end of the line because of soaring rebuild times and the growing risk of data loss. “The concept of parity-based RAID (levels 3, 5 and 6) is now pretty old in technological terms, and the technology’s limitations will become pretty clear in the not-too-distant future — and are probably obvious to some users already. In my opinion, RAID-6 is a reliability Band Aid for RAID-5, and going from one parity drive to two is simply delaying the inevitable. The bottom line is this: Disk density has increased far more than performance and hard error rates haven’t changed much, creating much greater RAID rebuild times and a much higher risk of data loss. In short, it’s a scenario that will eventually require a solution, if not a whole new way of storing and protecting data.”
While we speak of the sadness we feel that Apple has not yet included ZFS, from Sun, into OS X, and that there is simply no equivalent in the Windows world, it is something that is nearing a critical point for many businesses and other centers that need reliable data. ZFS is one way of elegantly taking care of the problem, so we must hope that it finds its way into some mainstream operating systems soon.
People who store things like music, movies, family pictures, and anything else that is archival in nature, will be needing this soon. As drive sizes of more than a terabyte become commonplace, and many of us have several of them in one case, with no wish to lose any of the data they hold, a solution will be needed and RAID 5 is close to incapable, and RAID 6 not much better, there really is no RAID level that is designed to take up the slack. We’ve run out of a way to throw more drives at the problem, because the drives themselves are the problem.
Software solutions, like ZFS, are the fix, but they do extract quite a penalty in overhead. Until drives get better, which is not very likely, software will have to heal the flaws in hardware.
Doublethink – it’s the mindset of the Right, from 1984.