question on dd command line

gurutech · #1 30th August 2012, 02:25 AM

Trying to figure out the best way to do things here...

I have a 500gb hard drive, using about 115gb (about 450gb free). Not sure why Linux is reporting this, since 450 + 115 is 565, which is more than 500...

Anyway, I want to copy the contents of this drive into either an ISO file, or into a VMDK file, and then reformat the drive, and restore contents.

Question about using dd - if I run a dd with if=sda and of=/path/isofile or using something vmware converter to convert it to a vmdk, am I going to have a 500gb "file", or will it only make the file out of the data itself?

Or am I better off doing something like a "cp -a" command as root, from the / folder and go from there?

If I go the dd route, will I have to boot from a Live CD, or is it ok to do this from a running system? (My concern here is network access, as the ISO or vmdk file would have to be on the network somewhere, since I will be reformatting this drive.)

cryogenic666 · #2 30th August 2012, 03:20 AM

Here's the thing: If you want to reformat and copy the contents back over, you'd almost be better off using one of the many backup tools and copying them back over after you've reformatted. If you back it up to any sort of image (using partimage, clonezilla, dd, etc), it's going to overwrite and ignore any new partitioning you've already done. If you dd it to an ISO, I'm PRETTY sure you're going to end up with an ISO that's the same size as whatever the partition is because it doesn't have the ability to ignore unused blocks. Partimage can do that, but I'm not sure if you can image a partition (or partitions) and mount them to copy stuff.

That being said, why are you reformatting?

gurutech · #3 30th August 2012, 03:34 AM

Looking to reformat because I'm not using as much of the drive as I was hoping to, and want to dual boot with another OS. just not sure which one yet. Figure split the drive 50/50 (250gb each), and go from there.

Is partimage part of the default installation, or is it something I would need to install first?

cryogenic666 · #4 30th August 2012, 03:46 AM

It's probably going to be a messy process. You'll probably have to shrink your existing partitions down to the size that you'll want them to be after the fact before you image them. Partimage can't restore an image of a large partition to a smaller partition, from my understanding. If I had to guess, you'll need a Clonezilla live CD to do the imaging since it can't image mounted partitions. You may also need a Gparted live CD to repartition since your partitions are going to be mounted.

It might actually be easier to just use a Gparted CD to repartition, then install the other OS, then boot to a liveCD to reinstall Grub if you're not going to be dual booting two Linuxes.

flyingfsck · #5 30th August 2012, 11:07 AM

Howdy,

Data Definition is a low level command. If you copy the data off, format it and then copy it back using dd, the only thing you have achieved is to waste a whole lot of time and inconvenience a lot of electrons...

stevea · #6 30th August 2012, 11:59 AM

To answer the original question - dd does a literal image copy and will produce a 500GB result.
You mean "Data Description" f'fsck.

The first question is whether the OP is using LVM or not.
If so then a copy, reinstall and copy-over is probably the best direction.

If not (hopefully not), just boot a live CD and then you can resize and move partitions easily and graphically using 'gparted'.
So you could for example boot a Fedora Live, become root with 'su -' then install gparted 'yum install gparted' .
Using gparted is mindlessly obvious. Shrinking an (unmounted) ext4 is a bit slow, moving a partition can take a very long time (it does a dd internally, copies entire partition from/to same disk). So shrink first then move. Increasing a partition & fs size is pretty fast.

*IF *you move the /boot or / (rootfs if you have no /boot) partition then you'll likely need to re-install grub.
Boot the DVD in rescue mode, chroot as directed, then 'grub2-install ...'

LongTimeDabbler · #7 30th August 2012, 03:54 PM

A bit off topic but isn't gparted broken in current Fedora?

gurutech · #8 30th August 2012, 05:41 PM

was not planning on dd back to the drive, just a straight copy. But if dd is going to give me a 500gb file, I'm not gonna do that.

Not sure if I have LVM on here, how can I tell?

cryogenic666 · #9 30th August 2012, 05:49 PM

Quote:

Originally Posted by gurutech

was not planning on dd back to the drive, just a straight copy. But if dd is going to give me a 500gb file, I'm not gonna do that.

Not sure if I have LVM on here, how can I tell?

just sudo fdisk -l and see if you see anything pertaining to LVM. It'll be pretty obvious. If you have a UEFI system then you'd have to use gdisk, but it's basically the same and fdisk will complain if you're using GPT.

---------- Post added at 12:49 PM ---------- Previous post was at 12:48 PM ----------

Quote:

Originally Posted by LongTimeDabbler

A bit off topic but isn't gparted broken in current Fedora?

From my experience, gparted isn't broken. I've used it a couple of times already. It's possible it's broken for GPT/UEFI systems, I suppose.

LongTimeDabbler · #10 30th August 2012, 06:02 PM

You're right - I just started gparted and it fired right up. Seems like less than a week ago I had run into a conflict whereby gparted need libparted.so.0 and FC17 had libparted.so.1 and the later library had removed functionality that the older one had provided but I guess things are all good now. Maybe one of the updates fixed things, I don't know.

cryogenic666 · #11 30th August 2012, 06:14 PM

Quote:

Originally Posted by LongTimeDabbler

You're right - I just started gparted and it fired right up. Seems like less than a week ago I had run into a conflict whereby gparted need libparted.so.0 and FC17 had libparted.so.1 and the later library had removed functionality that the older one had provided but I guess things are all good now. Maybe one of the updates fixed things, I don't know.

Yep, sounds like it got stuck in limbo somehow. That happens with packages on occasion. I've taught myself just to wait a few days and it all settles out.

aurabindo · #12 31st August 2012, 08:12 PM

by the way, again a bit off topic, Even if you want a 500 GB iso file, is that possible at all ? I think there are some limits to the size of an ISO.

Or is it not ?

gurutech · #13 31st August 2012, 08:29 PM

Quote:

Originally Posted by aurabindo

by the way, again a bit off topic, Even if you want a 500 GB iso file, is that possible at all ? I think there are some limits to the size of an ISO.

Or is it not ?

I'm not about to find out... lol

I wondered if dd would remove any "free space" from the iso or not. Guess it won't.

sidebrnz · #14 31st August 2012, 11:17 PM

I'd like to answer a question you asked in your original post: why the amount of data plus free space is greater than the partition's size. It's quite possible that you have a large number of symlinks so that the same file can be accessed from several different directories. (It's not uncommon for programs to expect common libraries to be in odd locations and creating links not only saves space, it makes sure that all "copies" get updated together.)

As an example, I have a 16GB flash drive that I use for backups. The backup utility that I use (Back In Time) saves space, when possible, by using symlinks whenever a file hasn't been changed since the last backup, so I reformatted the drive as ext4. (You can't create symlinks on a VFAT filesystem.) Currently, the system reports that the drive has 10.5GB of data on it and 9.5GB freespace.

jpollard · #15 31st August 2012, 11:39 PM

Quote:

Originally Posted by aurabindo

by the way, again a bit off topic, Even if you want a 500 GB iso file, is that possible at all ? I think there are some limits to the size of an ISO.

Or is it not ?

Sure it is.

There aren't any effective limit to the size other than the size limits of the filesystem where you save it. You can even mount it (via loopback mounts) and update it.

The problem is that dd makes an image copy, bypassing any filesystem. It goes directly to the partition and copies the data within the partition. The resulting file size is the size of the partition.

If you copy from the disk, then dd will make an image copy of the disk - including the partition table. This copy can't easily be used for a loopback mount (you have to figure out what the offsets are to get to the filesystem first).