xfs: 'Can't read superblock' on boot but otherwise fine

[David Becker]

I have a RAID 1 setup using xfs.

The array is mounted on boot (though it's obviously not a boot partition) but during the last boot I got the message (something like):

'/dev/md1: xfs: Can't read superblock'.

I boot the machine in rescue mode and the array is mounted. I unmount the filesystem, run xfs_repair which doesn't report anything wrong. From the rescue session I can mount and unmount the partition, no problems whatsoever.

Only when 'normally' booting do I get the above message and the filesystem doesn't get mounted.

The 2.16.18-2798 kernel is still on the system and I believe this is the same kernel used for resuce mode, but even when I boot (nonrescue) to this kernel version I get the same 'can't read superblock' message and the filesystem remains unmounted.

I've disabled selinux for normal boots but still the 'can't read superblock' persists. I checked the partition tables for the disks involved in the array and there's no inconsistency there (both partitions on the different disks have equal size).

I ran smartctl on the disks and there's nothing (physically) wrong with them.

Regretfully '/var' resides on this partition so the machine isn't able to completely boot and I thus can't try to mount the filesystem from a normal boot.

I have no idea why the filesystem can be (cleanly) mounted from rescue mode while the normal boot of fedora gives me this message. This is an fc6, i386 system.


Any tips appreciated,

thanks,

David

[scott32746]

Check and see if your VG is in /dev
If not try # vgchange VG -a y

[David Becker]

Originally Posted by scott32746

Check and see if your VG is in /dev
If not try # vgchange VG -a y

He Scott,

I'm not using LVM or otherwise. I think I discovered what was wrong. It wasn't xfs to blame, rather the superblock belonging to the raid partition. I had backed up the data on the partition in question, formatted it ext3 and restored the data. Even with ext3 I was getting a superblock (or otherwise) error preventing a mount which suggests the problem had nothing to do with the actual filesystem, rather the underlying raid partition.

After booting to rescue mode I reassembled the array with the 'resync' option and the array once again became available/mountable. It's now ext3 instead of xfs but otherwise the same.

rescue mode was able to mount the (xfs or ext3) raid partition, but on a normal boot the mount fails. So is rescue mode getting it's info from the raid partition superblock or is a normal boot getting it in this manner? In other words, who's reading /etc/mdadm.conf (and who's not)?

On another note, I then upgraded the machine to f7 through a 'yum update' call. This was a bit of a disaster as the device labels apparently changed from fc6 (hde,hdg) to f7 (sdb,sdc). I had to go into rescue mode, change the labels in raid configuration and /etc/fstab, reassemble the arrays and after that it's working again.

Thanks,

David

[scott32746]

You may need to update mdadm.conf with your new mdX

cd /etc
cp mdadm.conf mdadm.conf.`date +%y%m%d`
echo "DEVICE partitions" > mdadm.conf
mdadm --detail --scan >> mdadm.conf