Raid5 looses the spare in reboot

[Paavo]

Hi,

I get this mail after every reboot:

This is an automatically generated mail message from mdadm
running on yyy.zzz.com

A SparesMissing event had been detected on md device /dev/md0.

Faithfully yours, etc.

P.S. The /proc/mdstat file currently contains the following:

Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sdb1[0] sdc1[1] sdd1[4]
781419520 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]

unused devices: <none>

/etc/mdadm.conf looks like this:

MAILADDR root
ARRAY /dev/md0 level=raid5 num-devices=3 metadata=1.2 spares=1 name=yyy.zzz.com:0 UUID=7e634bd2:6780596e:8456192c:55513501
devices=/dev/sdb1,/dev/sdc1,/dev/sdd1,/dev/sde1

So after every reboot In need to

mdadm --add /dev/md0 /dev/sde1

After this the md0 looks like this:

# mdadm --detail /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Sun Feb 5 22:25:51 2012
Raid Level : raid5
Array Size : 781419520 (745.22 GiB 800.17 GB)
Used Dev Size : 390709760 (372.61 GiB 400.09 GB)
Raid Devices : 3
Total Devices : 4
Persistence : Superblock is persistent

Update Time : Fri Mar 30 10:44:18 2012
State : clean
Active Devices : 3
Working Devices : 4
Failed Devices : 0
Spare Devices : 1

Layout : left-symmetric
Chunk Size : 512K

Name : yyy.zzz.com:0 (local to host yyy.zzz.com)
UUID : 7e634bd2:6780596e:8456192c:55513501
Events : 112

Number Major Minor RaidDevice State
0 8 17 0 active sync /dev/sdb1
1 8 33 1 active sync /dev/sdc1
4 8 49 2 active sync /dev/sdd1

3 8 65 - spare /dev/sde1

Device /dev/sde1 details:

# mdadm --examine /dev/sde1
/dev/sde1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 7e634bd2:6780596e:8456192c:55513501
Name : yyy.zzz.com:0 (local to host yyy.zzz.com)
Creation Time : Sun Feb 5 22:25:51 2012
Raid Level : raid5
Raid Devices : 3

Avail Dev Size : 781420719 (372.61 GiB 400.09 GB)
Array Size : 1562839040 (745.22 GiB 800.17 GB)
Used Dev Size : 781419520 (372.61 GiB 400.09 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : bfc1c6cc:a574f73f:1824f131:d3390a24

Update Time : Fri Mar 30 10:26:57 2012
Checksum : ee90accc - correct
Events : 112

Layout : left-symmetric
Chunk Size : 512K

Device Role : spare
Array State : AAA ('A' == active, '.' == missing)

What is causing this? What can I do so that md0 remembers the spare device /dev/sde1?

Fedora 16 i686 installed on /dev/sda1 (swap=/dev/sda2, no other partitions on sda, and no other disks than sd[abcde]). Up-to date, running 3.3.0-4.fc16.i686 kernel currently.

-Paavo

PS. some info about the disks /dev/sd[bcde]1:

parted "p u chs p q"

Model: ATA ST3400620AS (scsi)
Disk /dev/sdb: 400GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number Start End Size Type File system Flags
1 512B 400GB 400GB primary raid

Model: ATA ST3400620AS (scsi)
Disk /dev/sdb: 48641,80,62
Sector size (logical/physical): 512B/512B
BIOS cylinder,head,sector geometry: 48641,255,63. Each cylinder is 8225kB.
Partition Table: msdos

Number Start End Type File system Flags
1 0,0,1 48641,80,62 primary raid

Model: ATA ST3400620AS (scsi)
Disk /dev/sdc: 400GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number Start End Size Type File system Flags
1 512B 400GB 400GB primary raid

Model: ATA ST3400620AS (scsi)
Disk /dev/sdc: 48641,80,62
Sector size (logical/physical): 512B/512B
BIOS cylinder,head,sector geometry: 48641,255,63. Each cylinder is 8225kB.
Partition Table: msdos

Number Start End Type File system Flags
1 0,0,1 48641,80,62 primary raid

Model: ATA ST3400620AS (scsi)
Disk /dev/sdd: 400GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number Start End Size Type File system Flags
1 512B 400GB 400GB primary raid

Model: ATA ST3400620AS (scsi)
Disk /dev/sdd: 48641,80,62
Sector size (logical/physical): 512B/512B
BIOS cylinder,head,sector geometry: 48641,255,63. Each cylinder is 8225kB.
Partition Table: msdos

Number Start End Type File system Flags
1 0,0,1 48641,80,62 primary raid

Model: ATA ST3400620AS (scsi)
Disk /dev/sde: 400GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number Start End Size Type File system Flags
1 512B 400GB 400GB primary raid

Model: ATA ST3400620AS (scsi)
Disk /dev/sde: 48641,80,62
Sector size (logical/physical): 512B/512B
BIOS cylinder,head,sector geometry: 48641,255,63. Each cylinder is 8225kB.
Partition Table: msdos

Number Start End Type File system Flags
1 0,0,1 48641,80,62 primary raid

fdisk "-l"

Disk /dev/sdb: 400.1 GB, 400088457216 bytes
255 heads, 63 sectors/track, 48641 cylinders, total 781422768 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xeecea0da

Device Boot Start End Blocks Id System
/dev/sdb1 1 781422767 390711383+ fd Linux raid autodetect

Disk /dev/sdc: 400.1 GB, 400088457216 bytes
255 heads, 63 sectors/track, 48641 cylinders, total 781422768 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xeecea0dd

Device Boot Start End Blocks Id System
/dev/sdc1 1 781422767 390711383+ fd Linux raid autodetect

Disk /dev/sdd: 400.1 GB, 400088457216 bytes
255 heads, 63 sectors/track, 48641 cylinders, total 781422768 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xeecea0d8

Device Boot Start End Blocks Id System
/dev/sdd1 1 781422767 390711383+ fd Linux raid autodetect

Disk /dev/sde: 400.1 GB, 400088457216 bytes
255 heads, 63 sectors/track, 48641 cylinders, total 781422768 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xeecea0db

Device Boot Start End Blocks Id System
/dev/sde1 1 781422767 390711383+ fd Linux raid autodetect