How-to Convert RAID5 to RAID6

[cruiseoveride]

Note: This how-to assumes you are using LVM on top of the RAID5

Example scenario:
VG0 <-> md0 = RAID5(3x500GB) = sda1, sdb1, sdc1
Volume Group VG0 is made up of a single Physical Volume (md0), which is made up of 3x500GB hard drives in RAID5.

You want to add 2 more drives (sdd1, sde1) and convert the underlying RAID5 to RAID6, thus giving you a 5x500GB RAID6

Methodology
Because mdadm does not currently support converting a RADI5 into RAID6, we need to dance around this problem by:
1. Taking offline one of the drives in md0
2. Using this drive, plus the 2 new drives and creating a fully degraded RAID6 (md1), ie consisting of 5 drives but 2 drives are missing
3. Create a PV on md1,
4. Add md1 to VG0,
5. Ask LVM to move md0 out of VG0
6. Shutdown md0
7. Add the drives from md0 into md1 to complete the RAID6

Ok this does sound long and complicated. It IS long. It is NOT complicated. It IS 100% safe and can be done without taking LVM offline. ie Data stays online during conversion.

Lets begin:

1. Take offline one of the drives in md0

mdadm /dev/md0 --fail /dev/sdc1 --remove /dev/sdc1

- where sdc is the drive we want to remove

2. Using this removed drive, plus the 2 new drives, create a fully degraded RAID6 (md1)

mdadm --create /dev/md1 --level=6 --raid-devices=5 missing missing /dev/sdc1 /dev/sdd1 /dev/sde1

- Note the word "missing:" for each drive we are not adding now.
- Also note you may receive a warning about sdc1 already being part of a RAID5 (we already know this)

mdadm: /dev/sdc1 appears to contain an ext2fs file system
    size=17414428K  mtime=Tue Apr 29 22:53:23 2008
mdadm: /dev/sdc1 appears to be part of a raid array:
    level=raid5 devices=3 ctime=Wed Apr 30 15:41:15 2008
Continue creating array? yes
mdadm: array /dev/md1 started.

3. Create a Physical Volume out of the new degraded RAID6 (md1)

pvcreate /dev/md1

4. Add md1 to VG0

vgextend /dev/VG0 /dev/md1

5. Setup md0 for removal from VG0 (ie Ask LVM to copy all the extents currently allocated on md0 to other physical volumes)

pvmove -v /dev/md0

Note: This can take a LOOOONG time. It took 24hrs for me. Dont worry if this process gets aborted. It can be resumed using the same command.

5.1 Remove md0 from VG0

vgreduce VG0 /dev/md0

5.2 Wipe the LVM label from md0 (not really essential)

pvremove /dev/md0

6. Shutdown md0

 mdadm --stop /dev/md0

7. Add the drives from md0 into md1, thus completing the RAID6

 mdadm /dev/md1 --add /dev/sda1 /dev/sdb1

Ta da! You have now successfully migrated from a 3 drive RAID5 to a 5 drive RAID6 without stopping the music for even a second.

The reason I did this was, I swear by Western Digital Hard drives, but i couldnt afford them anymore, so I bought some el-cheapo Seagates, which i dont trust for ****, and decided I needed to up the redundancy a level.

BE CAREFUL - During this procedure, md0 is in a very critical condition. If the two remaining drives dont come back up after a mid-step crash. You are ****ed. If the system crashes, there is a 1% chance that mdadm will not be able to bring back the degraded RAID5 (md0).

God Speed.

[cruiseoveride]

Remember pvmove doesnt really "move" anything, so in case something happens during the pvmove, you can use vgcfgrestore --list and pick an old automatic backup file and use it to restore the VG.