Howto: Install VMware Server 2.0.2 on Fedora 15 & 16
 

UPDATE: I've changed the links to the updated patch that will work with the latest kernel updates.

Official support for VMware server ended on June 30th, 2011. Although it is generally recommended that anyone using VMware server migrate to VMware ESXi or another VMware product, in some cases it may still be relevant to use VMware server. However, due to the lack of support, the last version of VMware server (2.0.2) has become severely out of date and no longer runs as intended “out of the box” on modern Linux distributions with newer kernels.

There have been numerous efforts in the past by the community to fix the various issues which enabled us to continue using VMware server even on modern Linux distributions. Notably, the wrapper script and patches by Rado Cotescu (See http://radu.cotescu.com/vmware-server-kernel-2-6-38/) enabled VMware server to function on the Linux 2.6.3x kernels up until 2.6.39 when BKL (Big Kernel Lock) was removed from the kernel causing the VMware modules to no longer compile. There are also countless valuable contributions online by many other members of the community that are not mentioned here.

What is presented here are just my own recent efforts built on top of the VMware server user community and the likes of Rado Cotescu to show that VMware server 2.0.2 can still be used even today and works on Fedora 15/16 and the newer 3.x kernels. A lot of credit goes to those who came before me.

Setting up Fedora 15/16 system

For this tutorial, we use either Fedora 15 or Fedora 16 on the x86_64 architecture (64bit). The initial release of Fedora 15 used a kernel that still had BKL and hence Rado Cotescu's wrapper scripts still worked. But the kernel changed in Fedora 15 through updates and the latest F15 kernel no longer has BKL, which broke VMware Server. The tutorial here, along with the updated code patches will fix this problem. So our first steps are:

[1] Install Fedora 15/16 x86_64 on hardware with virtualization support.
[2] Apply all current updates with “yum -y update”.

Ensure that your installation includes 'make', 'patch', the C compiler 'gcc', and the kernel source and headers. These packages are usually named 'make', 'patch', 'gcc', 'kernel-headers', and 'kernel-devel'. When you install those via the 'yum' utility, their dependencies will be pulled in. These are needed to patch the code and rebuild the kernel modules for VMware server. You can install these tools with:

[3] Install packages with:

# yum -y install make patch gcc kernel-headers kernel-devel

I'm sure there is a way to setup SELinux to work with VMware Server, but I haven't figured that one out and that's not an obstacle I want to to deal with just yet. SELinux is enabled by default, so in the interim, we disable SELinux:

[4] Disable “SELinux” by editing /etc/sysconfig/selinux and rebooting.

Next, there is a problem with VMware and the NMI watchdog feature of certain Linux kernels. (See http://kb.vmware.com/selfservice/mic...externalId=924) Check to see if you have this feature enabled by running this command:

$ cat /proc/sys/kernel/nmi_watchdog

If you get a '1', then the feature is enabled and must be turned off or VMs will mysteriously crash. To turn off nmi_watchdog is different for Fedora 15 and Fedora 16:

[5] Fedora 15: Edit /boot/grub/grub.conf and add “nmi_watchdog=0” to the kernel line.
[5] Fedora 16: Edit /etc/default/grub and add “nmi_watchdog=0” to the GRUB_CMDLINE_LINUX. Then run:

# grub2-mkconfig -o /boot/grub2/grub.cfg”

to rebuild the grub configuration.

Now, reboot the system and check the /proc/sys/kernel/nmi_watchdog parameter again to make sure you see a '0'.

The final step is to open up the necessary ports on the firewall or disable the firewall completely. The three ports that need to be opened are 902/TCP, 8222/TCP, and 8333/TCP.

[6] Care must be taken in how you add the firewall rules. The iptables commands below are just guidelines.

# iptables -I INPUT 5 -p tcp --dport 902 -m state --state NEW -j ACCEPT
# iptables -I INPUT 6 -p tcp --dport 8222 -m state --state NEW -j ACCEPT
# iptables -I INPUT 7 -p tcp --dport 8333 -m state --state NEW -j ACCEPT

Installing VMware Server

VMware server is available from www.vwmare.com in both a .tar.gz and RPM format. Since we are on Fedora, download the RPM format and then install it:

[7] Install VMware server RPM package:

yum localinstall /path/to/VMware-server-2.0.2-203138.x86_64.rpm

Normally, at this point, you can run 'vmware-config.pl' to finish the setup. But unfortunately, if you went ahead and did that, you would run into all sorts of problems like the Perl script not being able to detect your kernel sources and the missing “smp_lock.h” header file which was part of BKL that is no longer available. In order to fix these problems, we need to perform a few more steps.

First, download the patches (http://www.jlbond.com/downloads/vmwa....6.42+.tar.bz2) and untar the archive:

[8] Download the archive:

[9] Untar the archive:

# tar jxf vmware-server-2.0.2-203138-linux_2.6.42+.tar.bz2

You will find two files: vmware-config.pl.patch and vmware-server-2.0.2-203138-linux_2.6.42.patch. The first one is to fix the vmware-config.pl script and the second one is to patch the VMware kernel module sources. The 2nd patch is based on Rado Cotescu's patch but with additional fixes for some deprecated syntax and naively replaces BKL with mutexes.

[10] patch vmware-config.pl script:

# patch -d /usr/bin -p0 < /path/to/vmware-config.pl.patch

Patching the kernel modules is a bit more tedious (at some point, i may write a script to do this):

[11] patch the VMware module sources:

# pushd /usr/lib/vmware/modules/source
# mkdir backups; cp -a *.tar backups/
# mkdir working; ls *.tar | xargs -n 1 tar -C working/ -xf
# pushd working
# patch -p1 < /path/to/vmware-server-2.0.2-203138-linux_2.6.42.patch
# tar cf ../vmci.tar vmci-only
# tar cf ../vmmon.tar vmmon-only
# tar cf ../vmnet.tar vmnet-only
# tar cf ../vsock.tar vsock-only
# popd; rm -rf working/ ;popd

Next, there appears to be a problem between the bundled libxml2 with the system's zlib package. Either you can use an older zlib library or you can make vmware use the system's libxml2 which doesn't have problems with zlib. (see http://communities.vmware.com/message/1670102) We will move the bundled libxml2 out of the way so that vmware uses the native libxml2:

[12] Move bundled libxml2 out of the way so vmware uses native libxml2:

# pushd /usr/lib/vmware/lib; mv libxml2.so.2 SKIP-libxml2.so.2 ; popd

Finally, run 'vmware-config.pl' and answer the questions to finish up the installation. Once VMware server starts up, you should be able to access the management console at https://<your_ip_address>:8333.

Remark 1: Certain browsers like Firefox don't always work with VMware server's HTTPS. If you experience such problems, disabling TLS (and forcing SSL3) seems to help. Or, you can try another browser like Chrome.

Remark 2: When using NAT or HostOnly networking, the interface counters show enormous numbers in the packet counters. Since I mostly use Bridged networking which seems to work fine, I haven't looked into this further.

Re: Howto: Install VMware Server 2.0.2 on Fedora 15 & 16
 

doesn't work on fedora 16. I get error:
/tmp/vmware-config10/vmnet-only/userif.c: In function ‘VNetCsumCopyDatagram’:
/tmp/vmware-config10/vmnet-only/userif.c:579:3: error: incompatible type for argument 1 of ‘kmap’
/usr/src/kernels/3.2.2-1.fc16.i686.PAE/arch/x86/include/asm/highmem.h:60:7: note: expected ‘struct page *’ but argument is of type ‘const struct <anonymous>’
/tmp/vmware-config10/vmnet-only/userif.c:582:3: error: incompatible type for argument 1 of ‘kunmap’
/usr/src/kernels/3.2.2-1.fc16.i686.PAE/arch/x86/include/asm/highmem.h:61:6: note: expected ‘struct page *’ but argument is of type ‘const struct <anonymous>’
make[2]: *** [/tmp/vmware-config10/vmnet-only/userif.o] Error 1
make[1]: *** [_module_/tmp/vmware-config10/vmnet-only] Error 2
make[1]: Leaving directory `/usr/src/kernels/3.2.2-1.fc16.i686.PAE'
make: *** [vmnet.ko] Error 2
make: Leaving directory `/tmp/vmware-config10/vmnet-only'

I did notice that I dont have /usr/lib/vmware/modules/sources. I have /usr/lib/vmware/modules/source??? maybe that is causing a problem with the patch????

Re: Howto: Install VMware Server 2.0.2 on Fedora 15 & 16
 

you're right about "source" vs "sources", that was my typo and I have corrected it. Thanks!

I'm guessing that the patch didn't really take. What was the output when you ran the patch?

Re: Howto: Install VMware Server 2.0.2 on Fedora 15 & 16
 

Patched seemed to work fine. Maybe because I'm not using 64bit vmware server package???

Re: Howto: Install VMware Server 2.0.2 on Fedora 15 & 16
 

that may be true, i've only tested on 64bit thinking no one runs in 32 anymore...

Re: Howto: Install VMware Server 2.0.2 on Fedora 15 & 16
 

I'm still running 32 because I figured there would still be application compatability issues. 64bit is just now becoming standard even though cpus capable of the 64bit instruction set have been out for a long time. anyway the scrpts can be adapted??? Unless you know how I can just run a 64bit kernel so I can get vmware server running!!:D Googles something about it but I cant find the dir /etc/rpm/platform in fedora 16.

Re: Howto: Install VMware Server 2.0.2 on Fedora 15 & 16
 

it depends on what you're doing... most server purposes or VM hosting purposes rarely have compatibility issues in 64bit environments. for the most part, 64bit desktop environments are pretty good across the board, but there's a lot more diversity of applications so I can't speak for everything out there. personally, I've been running Fedora 64bit since Fedora 7 and now on Fedora 15 as a desktop while never re-installing for 5+ yrs now.

but, nonetheless, it doesn't mean i should ignore the problem on 32bit. i'll need to find some time to look over the patches to see where it's broken with 32bit kernels. give me some time and I'll see what i can do.

Re: Howto: Install VMware Server 2.0.2 on Fedora 15 & 16
 

Thanks. I'll probably just reinstall to 64bit next weekend. Real pain in the but but don't have another choice

Re: Howto: Install VMware Server 2.0.2 on Fedora 15 & 16
 

Regards picasso_1.2.13, those patches saved real amount of time for me, thanks very much!

Almost all compiles OK.

But task really isn't complete yet, problem with vmnet-only imho does not depend on 32/64 differrences but Fedora 15/16, for I have identical error on my Fedora 16 x86_64

Would be great if there could be a solution for this, for not sure I would be able to manage an issue myself (in term).

Re: Howto: Install VMware Server 2.0.2 on Fedora 15 & 16
 

Picasso_1.2.13 are you running Fedora 15 or 16? I would like to install the version it is known to work with.

Re: Howto: Install VMware Server 2.0.2 on Fedora 15 & 16
 

Hi AlexKey: thanks for reporting the problem. Are you on Fedora 16 x86_64? I think F16 went from 2.6.x kernel to 3.x didn't they? That might have something to do with it... when I tested with F16, it was with the DVD install kernel, which was 2.6.x i believe. I'll see if I can resolve the issue.. need to setup F16 w/ all kernel updates and see what the issue is...

I tested on F15 x86_64 and F16 x86_64, but the kernel versions were all 2.6.x, not in the 3.x kernels. I run this on F15 all the time, but I only tested F16 64bit in a VM which I don't run all the time so kernel updates in F16 can break this. I'll have to look into it... I'll update my F16 VM's kernel to the latest and see....

Anyway, appreciate you guys reporting back on problems; I don't run F16 until F17 comes out.

Re: Howto: Install VMware Server 2.0.2 on Fedora 15 & 16
 

I'm going back to 15 since I usually stay one version behind for stability sake. Is anyone aware of a way to downgrade the kernel.

Re: Howto: Install VMware Server 2.0.2 on Fedora 15 & 16
 

Hi, thanks for quick answer

Yes, I'm on Fedora 16 x86_64, and I guess yes, they did. I have now kernel 3.2.5, would be some pain to downgrade specially for that..

Fedora have been already installed before I had to deal with that serv, so I don't know for sure with which kernel it was shipped, though
http://docs.fedoraproject.org/en-US/..._Sysadmin.html
now say that
Though I did all as you wrote in first post, all other modules compiled OK, on 3.2.5, except for last one

Re: Howto: Install VMware Server 2.0.2 on Fedora 15 & 16
 

Guys here http://weltall.heliohost.org/wordpre...l-3-2-and-3-3/
managed to to this for 3.x with Workstation and Player , maybe it can be helpful with server too someway..

Re: Howto: Install VMware Server 2.0.2 on Fedora 15 & 16
 

Guess it was..

Weltall's solution for workstation and player resolved my installation issue with server vmnet module, though patch can't be automatically applied, particularly to netif.c - some minor handwork was done.

From what I've seen, it must be compatible with all 2.6.x through 3.3.x RC kernels too.

I'll post here diff file when I'll have little more time though, for it could be included in the first post.