Transferring to a new machine

[Henry2630]

:confused:
I just bought a new machine to replace the tired old box that my Fedora Linux 5 is running on.

It's a P4 with a 250GB Hard Drive and 2GB RAM so it should be much better than the (2001) P4 with 740MB RAM that it's on now.

OK the box is all assembled and working perfectly so all I have to do now is move everything across to it and there is a lot of stuff on the old machine including a (live) Forum and quite a big MySQL database. If I take the thing offline for more than a day my boss will have my guts for a necktie!

I think to avoid any updating hassles and the system being off-line for long, that I'll stick with FC5 (or do you think I should move it to FC6? - maybe some things won't run? Maybe it will be easier to move?)

I don't have a Tape Drive, but I have a gigabit ethernet setup so I can easily connect the two machines up.

I'm rather new to Linux, learning steadily, but I need some advice on a quick and easy idiotproof way to do this,

As I see it, the first job will be to install FC5 (or FC6) onto the new box, conect them to the switch and get them talking and then....

What's the best way to transfer the data. The old box had TWO Hard Drives a 20GB and an 80GB. The new box has a single 250GB Hard Drive.

I would really appreciate some tips here from someone who has already done this perhaps.

[offcenter77]

I would just take the old hard drive and put it in the new machine as a slave, copy the files you want, then wipe the old drive completely.

[sej7278]

running a website on fedora is a bit weird, but i'd say:

1. setup your new machine (i'd say go with centos5, not fc6), configured how you like it, get the forum software installed, security harden it etc.

2. copy the forum database onto the new machine, test it to death. this might take a while as fc5 and fc6/centos5 will have vastly different versions of mysql and php etc.

3. make the forum read-only for an hour so your database won't get out-of-sync whilst you switch over.

4. plug the new machine in place of the old one, and make the forum writable again.

this way you have no downtime - only an hour of read-only, plus you still have the old machine to plug back in if anything goes wrong.

no offence offcenter77, but your strategy is a really bad idea.

[offcenter77]

Why is my strategy a bad idea? That will be the fastest transfer rate than trying to pull it across the network. A little down time is not that bad.

You can easily rework the MySQL database to work on the new box, as well as the forums. If you want, you can leave the original drive as is while you get the new system up and running, then wipe it later.

[theforest]

Assuming you decide to transfer the files from the old server to the new server via network; once you have both machines connected to the network, you could use ssh/scp to physically copy the desired files from the old to the new. If the old server has ftp opened up and running an ftp server, you could use ftp. You'll have to readup on scp to understand the commands.

You should not have to be offline for more than how long it takes to make the final cutover, should be just the ip/hostname change; a couple minutes. This assumes you have already configured the new server just as the old, copied (backuped up/restored...whatever) over all application files, and have fully tested. Treat the new server as a "test" server until its ready, then "make" it the production server.

[sej7278]

offcenter - your strategy involves no testing stage and offers no fallback plan - you wipe the old drive and you're left with two useless servers and no website.

theforest's getting the idea - test, test, test, then just switch the two running servers. it's the old development-staging-production method, sysadmins and developers have been using for years.

how you get files between the two machines is largely irrelevant.

as far as i can remember, fc5 would have had mysql 4.0 or 4.1 and php4, pretty large differences between fc6/centos5's mysql5/php5 - database password syntax for a start.

[rpstitz]

I trained as IT tech/guru at my workplace. We're a local manufacturing plant of one of the largest auto parts suppliers in the world (ArvinMeritor). We have our own server with about 300 computers locally, and stay connected to Flint, MI via secure internet. The method we used when replacing the old server was by adding the new machine to the network without changing/interrupting the old server. We called it something like 'dexter02.arvinmeritor.com', robbed a hard drive from the old server (part of a Raid set), copied it's contents to the new server, and tested for about a month (from just about every PC in the entire plant). When we were sure all was ok, we removed the old server and renamed the new one. Not quite a easy as it sounds for me, but the guy training me made it look like a piece of cake. The plant was off line for about 15 minutes and we did it after hours for most office people (2nd shift).

[sej7278]

rpstitz, the point you made about swapping drives is slightly different - you're talking about breaking a raid set to use as a backup mechanism, bit of a no-no, but it works (i'm doing just that with a couple of suns and a couple of compaqs right now).

essentially when you break the mirror and put it in another machine that machine will recreate its own mirror from that drive. it only really works with identical servers, if you're changing hardware and os at the same time, that ain't gonna work.

but yeah, you have to test stuff to death, anyone who does server migration overnight is just reckless.

[rpstitz]

we didn't put the hd in a new raid set. We only copied the data, not the OS or software. But Henry doesn't seem to be using Raid anyway, and like SEJ mentioned earlier, how you copy the data is irrelevant. The basic procedure of leaving the original server online until after testing, testing, testing is the point.

[offcenter77]

offcenter - your strategy involves no testing stage and offers no fallback plan - you wipe the old drive and you're left with two useless servers and no website.



Then I guess either you didn't read my post or I didn't explain it well enough. You move the old hard drive over to the new machine. You copy all the files from the old hard drive to the new one, reconfigure it all and bring it up. The old hard drive can stay there as long as you want, and you can wipe it later if things go well.

You can even set up the machine to dual boot to the second hard drive. That way, your fallback plan is to simply reboot and bring up the old system.

To me, this sounds a lot easier than trying to muck around with two boxes for a week and a half before you can officially go live on the new box. It's just a hardware upgrade for crying out loud.

[Henry2630]

WOW! Well there's some food for thought! I get the idea. In the meantime Maybe I can get hold of a Tape Drive. Thanks guys. If I get any problems I'll yell.

I prefer keeping the old server intact until I'm 100% sure there are no problems with the new one.

[A.Serbinski]

Getting hold of another hard disk is much easier and less expensive than a tape. You can get a NEW 80GB disk for under $45.

Now as for everybody suggesting to pull a disk to make transfer faster.... yeah right. Your gigabit ethernet is as fast or faster than the PEAK transfer rate of the disks in the old server. Your PCI bus supports maximum transfer rate of 133 MB/s, or 1064 Mb/s (1 Gb/s), so ASSUMING that the disk and adapter supported 133 MB/s, AND that the disk could actually SUSTAIN that speed (not a chance), then the network adapter and disk speed would be equal.

Just don't do anything stupid, like transferring the files over SFTP (SSH), since that method has significant overhead in encrypting and decrypting the data sent. Use NFS to transfer the data.

[icy-flame]

I done something like that not long ago, the following is my notes, hope it helps

Because I was moving a busy server, I tried to reduce the down time as much as possible. In short:


Have the new box up and running with the latest update etc.
Do a data transfer (which takes a long time)
Test and make sure everyting on the new box is working fine
Take the existing box offline (public service wise)
Transfer those data changed since step 2 via rsync (Much quicker, reduce down time)
Have the new box online


#########################
# Server migration note #
#########################

use sshfs to mount the existing server to the a directory of the new one:
sshfs root@hostname:/ mountpoint

To unmount it when you finish copying all the files:
fusermount -u moutpoint

Use rsync to mirror files across once the exisiting server is mounted as a local directory.

It is a good idea to make a 'base' backup of data from the existing machine (which can take
a long time depend on your network connection and amount of data), then build the new one
from there. Once everything is running smoothly on the new server, rsync again to update
those files changed since the initial backup.

rsync -av /SrvDirectory/ /LocalDirectory/

Here is a list of data files you might want to move over:

/home/
/var/www/
/var/mail/
/var/lib/mysql/
/var/lib/awstats/
/var/lib/denyhosts/
/var/lib/geoip/
/var/lib/mysql/
/var/log/httpd


Use cp instead of rsync to guarentee overwrite.
cp -vrf --preserve=all

Here is a list of setting you will want to overwitre instead of just update (which wont happen
because the time stamp is most likely newer than your existing settings.):

/etc/awstats/
/etc/cron.d/
/etc/cron.daily/
/etc/cron.deny/
/etc/cron.hourly/
/etc/cron.monthly/
/etc/crontab
/etc/cron.weekly/
/etc/cups/
/etc/denyhosts.conf
/etc/dovecot.conf
/etc/firestarter/
/etc/GeoIP.conf
/etc/httpd/
/etc/my.cnf
/etc/skel/
/etc/squid/
/etc/ssh/


/etc/pki/tls/certs/*.crt
/etc/pki/tls/private/*.key


Copy only the real user accounts over for the following files:
/etc/passewd
/etc/shadow
/etc/group
/etc/gshadow

[A.Serbinski]

icy-flame: You're using ssh/sftp to transfer files. Thats why your step #2 is extremely slow. See my previous post.

[icy-flame]

icy-flame: You're using ssh/sftp to transfer files. Thats why your step #2 is extremely slow. See my previous post.

The services are still online, so speed wasnt a problem to me. In fact it works better that way for me, a fast transfer means more load on the old box, and the old box is struggling to keep up with the services anyway.

You might have the luxury of having the servers locally, for me, both of them are remote. I am transfering files from a US server to a DK server, while I am sitting in my office in the UK.

I just copy pasted my note, it is by no mean a direct step to step guide, I only hope my notes will be useful in reference, especially what files to transfer and what to take special care of.

[tw56]

I have recently done it in a test non critical environment going from fedora4 to fedora6. If time is not a factor I would say install a newer distro and work at getting things like httpd and mysql setup on the new box and use the old one as a guide. Copying the folders for websites and mysql database folders to the new box is not hard and you can test that everything is working. It will take a lot more time to set it up.

[Henry2630]

Well, I provoked a lively debate, but I want to throw something else at this.

I've installed Fedora 5 onto the new box and the new drive. It detected all the hardware and worked fine.

This being so, what would be the result of taking the 2 hard drives out of the OLD box and putting them into the NEW box, and booting the NEW box from the OLD boot Drive?

Would Fedora 'find' and install all the new hardware like sysprep is SUPPOSED to do with Windows (note I used 'supposed' to do).

If I could boot from the OLD Hard Drives I could then add the NEW 400GB drive and maybe then have lots of time to either set it up as a dual boot and simply install FC6 and spend as long as it takes (at night) reinstalling everything on the FC6 Partition or copy everything across to the new drive.

I'm not too worried about the OLD Drives going down. They are the newest part of the OLD box. I can take the system offline over a long weekend without a problem.

What I would like to know is what are the chances of FC5 trashing the OLD drives in the NEW system so it wouldn't run on either?

BTW, I don't really run a website, just use the website function for testing. What I DO run is a SimpleMachines Forum and a TroubleTicket Express HelpDesk using Sendmail (which gave me white hair trying to setup) and ProFTPD and GProFTPd, that's all.

If I COULD simply transfer the 2 hard drives and boot them on the new box, it would solve ALL my problems, but having tried to do this on Micro****e. I know how easy it is to lose the entire drive!

[offcenter77]

That was my original suggestion.

[offcenter77]

As long as the old drives are set up as sda, sdb (or hda, hdb), etc, then it should find grub just fine and boot up without a problem.

[Henry2630]

I know it was your original suggestion and the one I liked the most, but all I'm asking you is what is the risk factor to a Newbie like me. I know that some of the Linux / UNIX guys I have worked with would hit the keyboard for 10 minutes and everything would be fine!

It will take a LONG time before I'm that good. I can do that with Novell eDirectory, Zenworks and GroupWise, etc but I have BIG L-Plates for Linux.

I once tried putting a Hard Drive from and OLD Windows XP machine into a NEW Box and it trashed the drive so badly that we lost the entire drive. Fortunately I had a Zen Image so I could put it back into the old box, re-image it and all was well. Zenworks doesn't do that for Fedora - YET, but I'm sure it will. I just can't wait in hope.

[offcenter77]

If you're really that concerned, then take a backup image of the original drives. You can just use the dd command and create a file image of the drive and store it wherever you want:

dd if=/dev/hda of=/path/hda.img

[A.Serbinski]

It shouldn't trash the old drive, but it may make some configuration changes that could have significant effects on the function of the system and make it impossible to just dump it back. By this, I mean things like network device configurations could be changed when it detects removal of old network devices and addition of new devices.

[offcenter77]

But if you move them back to the old box, wouldn't the same thing happen? Detect all the existing devices?

[Henry2630]

:) I like this suggestion. Can I dump the backup onto a Windows XP-Share without any risks?

i.e dd if=/dev/hda of=/xp-box/share/hda.img

If you're really that concerned, then take a backup image of the original drives. You can just use the dd command and create a file image of the drive and store it wherever you want:

dd if=/dev/hda of=/path/hda.img

[offcenter77]

Don't see why not.