Sunday, April 18, 2010

Installing a new QNAP TS559 PRO NAS – An Unpleasant Experience.

On Friday night, I threw 6 2TB drives into a QNAP NAS configured as RAID 6. Since then I've lost connectivity to the box a couple of times (which may have been caused by some load balancing and jumbo frame settings I played around with) and reset the unit twice. I reset the box cause I had no way of knowing exactly what it was doing until I could get access to it again, and after leaving it for 8 or 9 hours on both occasions, things just didn't look right. I managed to get to the web interface each time the box was rebooted and although the RAID 6 Volume was still there it was usually trying to simply re-build or re-synchronize. I had plugged in a console and saw the bootup process squawk about not having access to a formatted volume to start a couple of the services, so after the third reset I went straight in and formatted the volume, even though it was in the process of rebuilding. I couldn't see how this would make a difference, but it couldn't hurt. That was last night around 11:30pm. It's now midday and I still have access to the box and the lights are all flashing at a reasonable rate (as opposed to one occasion where four stayed lit, and the other two were flashing extremely quickly. The Disk volume manager says it is in the process of rebuilding and is at 13%, with a percentage increase every 10 minutes or so. That means it's probably been rebuilding for a couple of hours. I have no idea what was going on for the previous 10 hours. Maybe the formatting took a an hour or so followed by a resynchronization of some sort.

So this really had me hesitating on whether going with RAID 6 was a good choice, as my initial choice was a RAID10. I had stumbled across a very detailed testing of the QNAP TS509 using the Intel NAS Performance Kit. From what I could see, the performance hit of RAID5 over RAID0 didn't seem that bad, and the same from RAID5 to RAID6, so I thought I may as well benefit from the extra 2TB of storage I would save by going with RAID6 plus the safety net of supporting 2 failed drives. I've always found the performance of RAID5 very poor on Motherboards with build in RAID controllers, but the QNAP is a high quality NAS and all performance tests indicate it is able to handle the load placed on it by RAID6 with a relatively small performance penalty. Nevertheless, the whole rebuilding process with RAID was even more time consuming than I expected, after this whole process I still don't have a lot of faith in the whole hot swappable automatic rebuild process, hence why a full backup strategy is also very important. The only problem  is that I need an equivalent amount disk storage for the back-up process!

Therefore, as of about 20 minutes ago I cancelled the RAID6 build and decided to go with a RAID0 on four drives, and use the other two drives, along with some 500MB drives I have lying around as part of my backup solution. In ten minutes, I had removed the RAID6 set, created, initialized and formatted the stripe set. It does mean my backup strategy becomes even more important factor, but I have a much better feeling about it all. My backup drives will all be used individually so I don't depend on another striped set and the backup set will be synchronized instead of a using a compressed backup image So hopefully this works all works out much better!

**Update 2010.04.19 8:12am

Well everything looked like it was going well, but after leaving the box for a few hours, all the lights turned read, the console mentioned something about a Kernel Panic and was frozen. I rebooted, but then the box just froze after another kernel panic. I rebooted again and the console didn't even come up. I waited an hour or so and tried it once more. It started up again, but as soon as the GRUB loader came up I went in and selected the backup option. Everything booted up fine, so I went in and set everything to factory defaults. I rebooted, selected the other option and everything seemed to come up ok. I went the through the initial configuration wizard, pulling out two of the drives so only the four I wanted to stripe were left, and was able to reconfigured the settings and re-create the stripe set. After this I was once again able to access the web interface, and even got so far as creating new shares and copying some files over to the unit. Unfortunately half way through the copying of about 25GB, windows lost its connectivity to the QNAP, and the QNAP rebooted itself and froze. So my next step is to start working with support, although I'm pretty damn close to returning this unit.

1 comment:

GUEST said...

fyi, qnap's disk build sequence is build the array, resync the array. so the process you described in the first paragraph is correct. it takes a while. after it is done syncing you can enable bitmap which will make any future resync a lot faster. not sure what happed with the panics... i have not had the experience except when vmware 4 tried to open a cifs connection to it (weird).

good luck!