use dd to copy 1 hd to another?

znarus · #1 2nd May 2011, 07:09 PM

Hi.

My main hd is 500 gb. I have an usb-hd that also is 500 gb.
I want to copy the main hd to the usb-hd so if
the main hd should crash i can boot from the backup hd.
I think dd is the command to use. I have checked the man-page but there
are so many options.. Anyone knows how to use dd to do this?

Regards Znarus

jpollard · #2 2nd May 2011, 07:39 PM

The simple way is "dd if=<src> of=<dest> bs=1M"

But the issue is going to be the time it takes to copy.

For 500GB it will take 3/4 hours, and since going through USB is usually (not always) slower than the main controller (SATA likely), it can take over a day.

And during that time, it would NOT be a good idea to use the disk...

In fact, it would be best to do this by booting a rescue CD to do the copy.

dd does not respect open files -- This means that any activity on the disk being copied will not be copied appropriately, leaving possibly corrupted data on the copy.

If the disks are NOT full to the brim, it is actually faster to create target filesystems and copy them. None of the unused blocks will copied - speeding things up significantly.

The other problem depends on how the system is configured. Logical volumes will cause problems as you cannot install the disks to copy single files. The logical structure is duplicated, causing confusion with two disks/partitions that have the same identifier. This can prevent filesystems from being mounted, or even usable.

znarus · #3 2nd May 2011, 08:35 PM

Aha.. thank you for your reply.. Not a good idea to use dd, I guess.. I know how to create file systems on the usb-disk and then copy ..
but i dont know how to make the usb-disk bootable..

hefeweizen · #4 3rd May 2011, 12:20 PM

dd was built for exactly this kind of thing (designed in a day when a 1 megabyte file was huge). The first caution noted was about time and regardless of what tool you use, USB will be slow. A tool you make like better is rsync. If your copy (rsync) gets interrupted, you can kick it off again and rsync is smart enough to determine what already exists in the new location and picks up from there. Likewise once your rsync finishes, you can kick it off again and it will update only those files that have changed.

As for booting off a usb drive, it should not be insurmountable. I would proceed thusly:
- make sure bios can boot from usb hdd
- install new os into usb drive
- copy over important data

I suggest this because of a hesitancy involving grub (or other boot loader) detecting the usb disk properly. Theoretically it shouldn't care; you should be able to treat it as any other disk. However, this speculation leads me from recommending it outright and instead advising the above.

jpollard · #5 3rd May 2011, 12:38 PM

check out grub-install.

If the BIOS can boot USB, then grub-install should be able to boot it.
NOTE: the disk identification may cause problems if, for instance, you needed to remove an internal disk (I have hot-plug disks on SATA), then it is possible that the root disk (hd0,0) by default points to the /boot partition, may/will change. requiring you to change it using the grub shell to boot a USB.

A follow up on rsync - some notes I've had with it..

rsync will do the job... but if you have many many files (passing the millions, so this usually isn't an issue)....

rsync has to compare the entire directory tree being backed up with the tree on the destination before it starts copying the first file. When I worked with large file systems having millions of files and many subdirectories I found that this can take several days. Even when rsync is being restarted, this step must complete before the first file new file is copied. It is faster to use an incremental copy - that way new files are copied without needing a directory analysis first. The downside is that this depends on the application doing the copy as to whether it can handle deleting files that have been removed.

assen · #6 3rd May 2011, 10:31 PM

Hi,

dd is very fast and will do the job at "line speed", preserving your filesystem with all permissions et al. You only need to take care of one small thing: make sure the target drive has at least same number of bytes (blocks) as the source; check exactly in this unit as relocated blocks or simply some poor manufacturer's idea what a "gigabyte" is can spoil your day.

WWell,