PDA

View Full Version : Help with LVM


lanrat
7th July 2008, 02:18 AM
Hi, I had my server setup using a HD's one was SDA which held the OS, and SDB which was mounted at /data which held all my data. I was using LVM but I did not know it.

During the re-install I took out SDB just to make sure that I would not erase it.

Now afterwards I was realized I cant mount SDB anymore, because I was using LVM. But I need to get that data off. Is there anything I can do to get it off?

Here is the output of several commands if it helps:

[root@world0 mapper]# fdisk -l

Disk /dev/sda: 18.4 GB, 18400000000 bytes
255 heads, 63 sectors/track, 2237 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 83 Linux
/dev/sda2 14 2237 17864280 8e Linux LVM

Disk /dev/sdb: 73.5 GB, 73543163904 bytes
255 heads, 63 sectors/track, 8941 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdb1 * 1 8941 71818551 8e Linux LVM
[root@world0 mapper]#

[root@world0 mapper]# pvs
/dev/hda: open failed: No medium found
PV VG Fmt Attr PSize PFree
/dev/sda2 VolGroup01 lvm2 a- 17.03G 0
[root@world0 mapper]# vgs
/dev/hda: open failed: No medium found
VG #PV #LV #SN Attr VSize VFree
VolGroup01 1 2 0 wz--n- 17.03G 0
[root@world0 mapper]# lvs
/dev/hda: open failed: No medium found
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
LogVol00 VolGroup01 -wi-ao 16.03G
LogVol01 VolGroup01 -wi-ao 1.00G
[root@world0 mapper]# mount
/dev/mapper/VolGroup01-LogVol00 on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/sda1 on /boot type ext3 (rw)
tmpfs on /dev/shm type tmpfs (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
[root@world0 mapper]#


[root@world0 mapper]# pvdisplay /dev/sdb1
/dev/hda: open failed: No medium found
Couldn't find device with uuid 'z3cJc9-4RXQ-zNFl-Exiu-orqx-vX9s-0D3AZK'.
Couldn't find all physical volumes for volume group VolGroup00.
Couldn't find device with uuid 'z3cJc9-4RXQ-zNFl-Exiu-orqx-vX9s-0D3AZK'.
Couldn't find all physical volumes for volume group VolGroup00.
get_pv_from_vg_by_id: vg_read failed to read VG VolGroup00
Couldn't find device with uuid 'z3cJc9-4RXQ-zNFl-Exiu-orqx-vX9s-0D3AZK'.
Couldn't find device with uuid 'z3cJc9-4RXQ-zNFl-Exiu-orqx-vX9s-0D3AZK'.
Couldn't find device with uuid 'z3cJc9-4RXQ-zNFl-Exiu-orqx-vX9s-0D3AZK'.
Couldn't find all physical volumes for volume group VolGroup00.
Couldn't find device with uuid 'z3cJc9-4RXQ-zNFl-Exiu-orqx-vX9s-0D3AZK'.
Couldn't find all physical volumes for volume group VolGroup00.
get_pv_from_vg_by_id: vg_read failed to read VG VolGroup00
"/dev/sdb1" is a new physical volume of "68.49 GB"
--- NEW Physical volume ---
PV Name /dev/sdb1
VG Name
PV Size 68.49 GB
Allocatable NO
PE Size (KByte) 0
Total PE 0
Free PE 0
Allocated PE 0
PV UUID CpaFSg-hNNp-nB0D-mgWn-Fxy3-f3FR-rCl6aN

[root@world0 mapper]#

Thanks...

SlowJet
7th July 2008, 05:31 AM
http://forums.fedoraforum.org/forum/showthread.php?p=1042498#post1042498

Read that post and make a choice.

Re-install with custom partitioning or continue on your release (until you need to install the next one.)


SJ

lanrat
7th July 2008, 06:09 AM
I understand that, but is it possible for me to get my data off that drive?

SlowJet
7th July 2008, 06:36 AM
sdb was pv /dev/sdb1 and part of VolGroup00.(or it would not be missing)
VolGroup00 spanned both drives.
/mnt/data was an LV ext3 f/s and was not necessarily on sdb1 until all of sda2 lv ? was full or out of contiguous free space or a something else?.

So how did you create the f/s for /mnt/data? (because these 2 conditions contradict one another)

If sdb1 has a separate vgname and lv f/s then it can be mounted again (but the missing volume message would indicate that was NOT a separate vg-lv and so the re-install knows nothing about it and the /data extents on sdb1 are unknown and inaccessible.

So I think the answer is no.

SJ

lanrat
7th July 2008, 06:49 AM
I thought that sdb was its own Volgroup (02?) on my previous install, But I do not remember how I set it up. All I know now is that is is no longer mountable. And I need that data back.

How would I try mounting it as its own Volume group?

SlowJet
7th July 2008, 07:14 AM
If it is in the machime in a root term do
vgchange -a y
vgs
lvs

if you see it and the parts then do

mkdir /mnt/data
chown -R someuser:someuser /mnt/data # whatever you need
chmod -R 750 /mount/data # whatever you need
mount /dev/VolGroup02/LogVol?? /mnt/data

#if it mounts ok
cd /mnt/data
ls

#if you see files and dir's then do
mount
#edit fstab to add mount for
/dev/mapper/VolGroup02-LogVol?? /mnt/data

SJ

lanrat
7th July 2008, 06:21 PM
I get this:

Couldn't find device with uuid 'z3cJc9-4RXQ-zNFl-Exiu-orqx-vX9s-0D3AZK'.

What does that mean? I have little understanding of LVM.

SlowJet
7th July 2008, 06:40 PM
I get this:

Couldn't find device with uuid 'z3cJc9-4RXQ-zNFl-Exiu-orqx-vX9s-0D3AZK'.

What does that mean? I have little understanding of LVM.


That is from the fstab/mounting process
or the maybe the LVM metadata is not sync'd yet.

Is this before or after trying to mount the sdb1 again.
It is there before because the sdb disk was gone.

A UUID is just a very semi-unique label. It can be made unique within a system with the
UUIDD package from e2fsprogs

If you get the sdb drive in and sdb1 mounted, fix the fstab, and do a vgscan, it should go away.

SJ

lanrat
7th July 2008, 06:59 PM
This is what I get:
[root@world0 ~]# vgchange -a y
Couldn't find device with uuid 'z3cJc9-4RXQ-zNFl-Exiu-orqx-vX9s-0D3AZK'.
Couldn't find all physical volumes for volume group VolGroup00.
Couldn't find device with uuid 'z3cJc9-4RXQ-zNFl-Exiu-orqx-vX9s-0D3AZK'.
Couldn't find all physical volumes for volume group VolGroup00.
Volume group "VolGroup00" not found
2 logical volume(s) in volume group "VolGroup01" now active
[root@world0 ~]# vgs
Couldn't find device with uuid 'z3cJc9-4RXQ-zNFl-Exiu-orqx-vX9s-0D3AZK'.
Couldn't find all physical volumes for volume group VolGroup00.
Couldn't find device with uuid 'z3cJc9-4RXQ-zNFl-Exiu-orqx-vX9s-0D3AZK'.
Couldn't find all physical volumes for volume group VolGroup00.
Volume group "VolGroup00" not found
VG #PV #LV #SN Attr VSize VFree
VolGroup01 1 2 0 wz--n- 17.03G 0
[root@world0 ~]# lvs
Couldn't find device with uuid 'z3cJc9-4RXQ-zNFl-Exiu-orqx-vX9s-0D3AZK'.
Couldn't find all physical volumes for volume group VolGroup00.
Couldn't find device with uuid 'z3cJc9-4RXQ-zNFl-Exiu-orqx-vX9s-0D3AZK'.
Couldn't find all physical volumes for volume group VolGroup00.
Volume group "VolGroup00" not found
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
LogVol00 VolGroup01 -wi-ao 16.03G
LogVol01 VolGroup01 -wi-ao 1.00G
[root@world0 ~]#

sundaraz
7th July 2008, 07:35 PM
lanrat: Looking at your output, I'm confused. why are you getting "/dev/hda: no medium found"? Did you have a device by that name as part of LVM? Also your old volumegroup seems to be VolGroup00. Also did you run pvcreate on /dev/sdb1? pvdisplay seems to recogonize it as a new physical volume that has not been allocated to any volume groups. If you had run pvcreate on this, you probably lost the LVM metadata on it. Below has some info on restoring it.

http://www.tldp.org/HOWTO/LVM-HOWTO/recovermetadata.html

If nothing works, you can try using pvcreate with --uuid option to set the uuid manually to the string from the error (z3cJc9-4RXQ-zNFl-Exiu-orqx-vX9s-0D3AZK) and see. Use your discretion though.

SlowJet
7th July 2008, 07:57 PM
Are you sure that the disk connections, cables, jumper switches
match what the bios says is on primary/secondary positions?


With the disk removed.
pvdisplay /dev/sdb1
/dev/hda: open failed: No medium found

Assuming your last post was with the second disk back in the machine?

Couldn't find all physical volumes for volume group VolGroup00.
Volume group "VolGroup00" not found

2 logical volume(s) in volume group "VolGroup01" now active

sda and sdb switched places?

SJ

lanrat
8th July 2008, 05:29 PM
I have no idea what hda is, I have never had a hda in this machine. Both HD's are SCSI. When I set it up last time I think I created sdb in its own VolGroup that is why I am almost certain all my data is on it.

What would the syntax of pvcreate with --uuid option be for me?

Thanks.

sundaraz
8th July 2008, 06:00 PM
Did you check for any possible backups you had? Below would be the best solution if you had any backup

pvcreate --uuid "z3cJc9-4RXQ-zNFl-Exiu-orqx-vX9s-0D3AZK" --restorefile /etc/lvm/archive/VolumeGroupName_XXXXX.vg /dev/sdb1

If its not there, just run

pvcreate --uuid "z3cJc9-4RXQ-zNFl-Exiu-orqx-vX9s-0D3AZK" /dev/sdb1

Above would work if the LVM data was not lost.

sundaraz
8th July 2008, 06:13 PM
There seems to be better information here..check this

http://www.redhat.com/docs/manuals/csgfs/browse/4.6/Cluster_Logical_Volume_Manager/mdatarecover.html

Try restoring the metadata from possible backup instead of just setting the UUID..

lanrat
8th July 2008, 06:52 PM
I do not have a backup, So I ran the command

[root@world0]# pvcreate --uuid "z3cJc9-4RXQ-zNFl-Exiu-orqx-vX9s-0D3AZK" /dev/sdb1
Physical volume "/dev/sdb1" successfully created

What Do I do now?

sundaraz
8th July 2008, 07:04 PM
Hmm..not much to do. Can you post the output of vgs, lvs and pvs? If this doesn't get it back, I really don't know how to restore it back without having a backup of the configuration.

lanrat
8th July 2008, 07:14 PM
[root@world0 ~]# vgs
VG #PV #LV #SN Attr VSize VFree
VolGroup01 1 2 0 wz--n- 17.03G 0
[root@world0 ~]# lvs
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
LogVol00 VolGroup01 -wi-ao 16.03G
LogVol01 VolGroup01 -wi-ao 1.00G
[root@world0 ~]# pvs
PV VG Fmt Attr PSize PFree
/dev/sda2 VolGroup01 lvm2 a- 17.03G 0
/dev/sdb1 lvm2 -- 68.49G 68.49G
[root@world0 ~]#

So it looks like my data is still there, there must be a way to get it off...

sundaraz
8th July 2008, 07:33 PM
Your previous vgs output showed
"Couldn't find all physical volumes for volume group VolGroup00"

I'm surprised that it did not even report an error with VolGroup00 now. Can you try

vgimport -v VolGroup00 /dev/sdb1

Also, could you post pvdisplay and pvdata?

lanrat
8th July 2008, 07:38 PM
[root@world0 ~]# vgimport -v VolGroup00 /dev/sdb1
Using volume group(s) on command line
Finding volume group "VolGroup00"
Wiping cache of LVM-capable devices
Volume group "VolGroup00" not found
Finding volume group "sdb1"
Volume group "sdb1" not found
[root@world0 ~]# pvdisplay
--- Physical volume ---
PV Name /dev/sda2
VG Name VolGroup01
PV Size 17.04 GB / not usable 5.59 MB
Allocatable yes (but full)
PE Size (KByte) 32768
Total PE 545
Free PE 0
Allocated PE 545
PV UUID GFcSVU-BS8D-SCcH-vAUe-K42Q-nUVh-ml3hox

"/dev/sdb1" is a new physical volume of "68.49 GB"
--- NEW Physical volume ---
PV Name /dev/sdb1
VG Name
PV Size 68.49 GB
Allocatable NO
PE Size (KByte) 0
Total PE 0
Free PE 0
Allocated PE 0
PV UUID z3cJc9-4RXQ-zNFl-Exiu-orqx-vX9s-0D3AZK

[root@world0 ~]# pvdata
-bash: pvdata: command not found

Looks like pvdata is not the right command...
And that sdb1 is not in a group, do we need to put it in one? (I'm guessing)

sundaraz
8th July 2008, 07:59 PM
lanrat: I wish I could help. Your pvdisplay from the beginning post seems to show the PV size as full, implying that no area has been used for the metadata, meaning you probably lost your LVM metadata. Only way I see, is to find some backup configuration and restore it. I seriously doubt if creating a new volume group, logical volume and and mounting to it would help. It might actually overwrite some actual data area with some metadata and vice versa, and I am not sure what logic it uses to do that. Someone who knows better might be able to help you.

Just curious, can you post the output of vgdisplay --partial --verbose

lanrat
8th July 2008, 08:19 PM
[root@world0 ~]# vgdisplay --partial --verbose
Partial mode. Incomplete volume groups will be activated read-only.
Finding all volume groups
Finding volume group "VolGroup01"
--- Volume group ---
VG Name VolGroup01
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 3
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 2
Open LV 2
Max PV 0
Cur PV 1
Act PV 1
VG Size 17.03 GB
PE Size 32.00 MB
Total PE 545
Alloc PE / Size 545 / 17.03 GB
Free PE / Size 0 / 0
VG UUID A3oYYT-TG0c-DmgQ-Dh7K-jiU9-Mku9-xiKHGo

--- Logical volume ---
LV Name /dev/VolGroup01/LogVol00
VG Name VolGroup01
LV UUID 15bBT0-UtU4-K5fc-GjYQ-WIfa-IEsC-7tfRWY
LV Write Access read/write
LV Status available
# open 1
LV Size 16.03 GB
Current LE 513
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:0

--- Logical volume ---
LV Name /dev/VolGroup01/LogVol01
VG Name VolGroup01
LV UUID DZBLfu-uqhb-Xhk0-StsW-DiE3-3g98-60AnJl
LV Write Access read/write
LV Status available
# open 1
LV Size 1.00 GB
Current LE 32
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:1

--- Physical volumes ---
PV Name /dev/sda2
PV UUID GFcSVU-BS8D-SCcH-vAUe-K42Q-nUVh-ml3hox
PV Status allocatable
Total PE / Free PE 545 / 0

[root@world0 ~]#

Anything else? I feel horrible giving up knowing my data is still there.
I have backups of some of my config files, but not any LVM ones, if it helps here is my backup fstab

/dev/VolGroup00/LogVol00 / ext3 defaults 1 1
LABEL=/boot /boot ext3 defaults 1 2
devpts /dev/pts devpts gid=5,mode=620 0 0
tmpfs /dev/shm tmpfs defaults 0 0
proc /proc proc defaults 0 0
sysfs /sys sysfs defaults 0 0
/dev/VolGroup00/LogVol01 swap swap defaults 0 0
/dev/VolGroup00/LogVol02 /data ext3 user,suid,dev,exec 0 0

Does that help at all?

sundaraz
8th July 2008, 08:48 PM
It seems to have no idea of your previous volume group. fstab is only for initialising your filesystem. It wouldn't help. Looking at your fstab, you had three logical volumes on VolGroup00 before, with a swap filesystem too on it.

Below article has some info on restoring LVM without having a backup configuration. Hopefully your /dev/sdb1 initial sectors have not been overwritten. Relevanr information is from "Recovering and Renaming the LVM2 Volume" section.

http://www.linuxjournal.com/article/8874

Try

dd if=/dev/sdb1 bs=512 count=255 skip=1 of=/tmp/md2-raw-start

This does a raw copy of the bytes.

vi /tmp/md2-raw-start or any text editor

and see if there is anything useful apart from the binary junk. If you find something, clean it up and build a configuration file as mentioned. Change the physical device name to the new one. You can then try to restore this configuration.

algej
8th July 2008, 08:55 PM
Anything else? I feel horrible giving up knowing my data is still there.
I have backups of some of my config files, but not any LVM ones, if it helps here is my backup fstab

/dev/VolGroup00/LogVol00 / ext3 defaults 1 1
LABEL=/boot /boot ext3 defaults 1 2
devpts /dev/pts devpts gid=5,mode=620 0 0
tmpfs /dev/shm tmpfs defaults 0 0
proc /proc proc defaults 0 0
sysfs /sys sysfs defaults 0 0
/dev/VolGroup00/LogVol01 swap swap defaults 0 0
/dev/VolGroup00/LogVol02 /data ext3 user,suid,dev,exec 0 0

Does that help at all?

Ouch. The backup config file would have lived in /etc/lvm/backups, and I would assume that it was destroyed when you re-installed your operating system.

Having said that, it's possible that not all is lost - a copy of the config file exists on your /dev/sdb1 disk, albeit in a circular buffer immediately following the disk label that's a bit of a pain to parse. Maybe a bit of dd magic can save you.

Try this:

dd if=/dev/sdb1 bs=512 skip=1 count=384 | less
Edit: Changed skip to 1 (I use LVM on top of MD Raid)

You should be able to scroll around and pick out something like the following:
# Generated by LVM2: Sun Jul 6 08:35:04 2008

contents = "Text Format Volume Group"
version = 1

description = " something might be here, it might be empty, it doesn't matter"

creation_host = "your host name would be here" # Linux uname -a output here
#1 SMP Fri Jun 27 16:05:49 EDT 2008 x86_64
creation_time = 1215347704 # Sun Jul 6 08:35:04 2008

VolGroup00 {
id = "6T0rHz-HQDX-ltOe-b7HP-Cf1e-YDlK-wW4Fwf"
seqno = 351
status = ["RESIZEABLE", "READ", "WRITE"]
extent_size = 65536 # 32 Megabytes
max_lv = 0
max_pv = 0

physical_volumes {

pv0 {
id = "whatever the uuid was"
device = "/dev/sdb1"

status = ["ALLOCATABLE"]
dev_size = xxxxxxxxx # xx Gigabytes
pe_start = yyy
pe_count = zzzzz # xx Gigabytes
}

pv1 {
id = "if you had one it went here"
device = "/dev/whatever" # Hint only

status = ["ALLOCATABLE"]
dev_size =
pe_start =
pe_count =
}
}

logical_volumes {
LogVol00 {
id = "some-uuid-string-here"
status = ["READ", "WRITE", "VISIBLE"]
segment_count = 1

segment1 {
start_extent = X
extent_count = XXX

type = "striped"
stripe_count = 1 # linear

stripes = [
"pv1", 0
]
}
}
LogVol01 {
id = "some-uuid-string-here"
status = ["READ", "WRITE", "VISIBLE"]
segment_count = 1

segment1 {
start_extent = X
extent_count = XXX

type = "striped"
stripe_count = 1 # linear

stripes = [
"pv1", 0
]
}
}

LogVol02 {
id = "some-uuid-string-here"
status = ["READ", "WRITE", "VISIBLE"]
segment_count = 1

segment1 {
start_extent = X
extent_count = XXX

type = "striped"
stripe_count = 1 # linear

stripes = [
"pv1", 0
]
}
}
}
}

Anyway, open a second window, and using your favorite text editor, copy and paste from the window running dd piped into less to reconstruct your LVM configuration file, using the example I gave you as a template to make sure you get everything. There will likely be some extraneous junk in there, just ignore it and make your new LVM configuration syntactically correct. Then save it somewhere (e.g. ~/VolGroup00)

Then you can rerun pvcreate like this (or post your new VolGroup00 file for review first if all of this made you queasy):

pvcreate --uuid "z3cJc9-4RXQ-zNFl-Exiu-orqx-vX9s-0D3AZK" --restorefile ~/VolGroup00 /dev/sdb1

Once you get to that point, please post the ouput of vgdisplay --partial --verbose again.

Edit: sundarez beat me to it :)

lanrat
8th July 2008, 09:44 PM
Attached is the output of "dd if=/dev/sdb1 bs=512 skip=1 count=384 | less"

What should I make my VolGroup00 look like?

Thanks!!!

sundaraz
9th July 2008, 12:07 AM
Thats pretty good. I have attached the modified file.

Below restores data on the PV appropriately

pvcreate --uuid="CpaFSg-hNNp-nB0D-mgWn-Fxy3-f3FR-rCl6aN" --restore_file=new_sdb.txt /dev/sdb1

Now

vgcfgrestore -f new_sdb.txt VolGroup00

Do vgs and pvs to make sure they are right. Make the volume group available with

vgchange -a y VolGroup00

lvscan should give you the logical volume LogVol02. Mount it as /data for use.

lanrat
9th July 2008, 12:57 AM
THANKS! IT WORKED! :)

Now how should I format the drive so that is is plain old ext3 mountable with "mount -t ext3 /dev/sdb1 /data" with no LVM involved to make sure this does not happen again?

sundaraz
9th July 2008, 01:25 AM
Great! Are you able to access the files. I was thinking it may not work as the original logical group had a segment count of 2 and was spanning both sda1 and sdb1. I am not sure if you can just convert it into a normal partition. You can backup the data, use fdisk/parted to create a partition and format it to ext3, copy back the data and use it.

lanrat
9th July 2008, 01:46 AM
I am currently backing up the entire disk. so once I have it backed up, how should I format it to ext3? I don't need to convert it.

sundaraz
9th July 2008, 12:21 PM
Once you are done, unmount it and do the following

lvremove VolGroup00
vgremove VolGroup00
pvremove /dev/sdb2

Then

fdisk /dev/sdb
Type 'p' to list the partitions.You'll see /dev/sdb1
Type 'd' to delete and give 1 as partition number.
Type 'n' to create a new partition
Choose primary and choose the partition number as 1
Type 'w' to save and exit

Do the below to format the filesystem to ext3
mkfs.ext3 -L "/data" /dev/sdb1
Now you can mount the /dev/sdb1 as /data

You can add the below line to fstab to automount if you want
LABEL=/data /data ext3 defaults 0 2