Help: NFS Failing (Connection refused) after Fedora 16 upgrade

[urilabob]

I have a fedora cluster which had been successfully sharing mounts via nfs since FC 4. Until last Thursday, everything was working sweetly under F15. I upgraded a client to F16 first, nfs mounting was working OK. However upgrading the server to F16 did not go so well. Initially, nfs-server was not enabled, but I fixed this, and also (just in case) reinstalled nfs, but no joy. Attempting to mount from an F16 client gave:

mount -v kaya:/home /home
mount: no type was given - I'll assume nfs because of the colon
mount.nfs: timeout set for Sun Nov 27 14:01:51 2011
mount.nfs: trying text-based options 'vers=4,addr=192.168.0.1,clientaddr=192.168.0.14'
mount.nfs: mount(2): Connection refused

The F16 host showed the following rpc procedures with rpcinfo -p:

rpcinfo -p kaya
   program vers proto   port  service
    100000    4   tcp    111  portmapper
    100000    4   udp    111  portmapper
    100024    1   udp  56898  status
    100024    1   tcp  35354  status
    100011    1   udp    875  rquotad
    100011    1   tcp    875  rquotad
    100005    1   udp  20048  mountd
    100005    1   tcp  20048  mountd
(note: duplicates omitted)

Comparing with F15 and a newly installed F16, it is missing nfs, nfs-acl and nlockmgr. However the nfs server appears to start OK:

service nfs-server status
Redirecting to /bin/systemctl  status nfs-server.service
nfs-server.service - NFS Server
	  Loaded: loaded (/lib/systemd/system/nfs-server.service; enabled)
	  Active: active (running) since Sun, 27 Nov 2011 13:22:30 +0900; 51min ago
	 Process: 17929 ExecStopPost=/usr/sbin/exportfs -f (code=exited, status=0/SUCCESS)
	 Process: 17927 ExecStop=/usr/sbin/rpc.nfsd 0 (code=exited, status=0/SUCCESS)
	 Process: 17939 ExecStartPost=/usr/lib/nfs-utils/scripts/nfs-server.postconfig (code=exited, status=0/SUCCESS)
	 Process: 17937 ExecStartPost=/usr/sbin/rpc.mountd $RPCMOUNTDOPTS (code=exited, status=0/SUCCESS)
	 Process: 17936 ExecStart=/usr/sbin/rpc.nfsd $RPCNFSDARGS ${RPCNFSDCOUNT} (code=exited, status=0/SUCCESS)
	 Process: 17935 ExecStartPre=/usr/sbin/exportfs -r (code=exited, status=0/SUCCESS)
	 Process: 17933 ExecStartPre=/usr/sbin/rpc.rquotad $RPCRQUOTADOPTS (code=exited, status=0/SUCCESS)
	 Process: 17931 ExecStartPre=/usr/lib/nfs-utils/scripts/nfs-server.preconfig (code=exited, status=0/SUCCESS)
	Main PID: 17934 (rpc.rquotad)
	  CGroup: name=systemd:/system/nfs-server.service
		  \u251c 17934 /usr/sbin/rpc.rquotad
		  \u2514 17938 /usr/sbin/rpc.mountd

Nothing showed up in /var/log/messages. However in the machine with the newly-installed (and apparently working) nfs-server, I saw in the logs

Nov 27 15:19:21 bobslinux kernel: [98770.332132] Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
Nov 27 15:19:21 bobslinux kernel: [98770.588016] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
Nov 27 15:19:21 bobslinux kernel: [98770.616891] NFSD: starting 90-second grace period
Nov 27 15:19:21 bobslinux rpc.mountd[24678]: Version 1.2.5 starting

The problem was the result of a newly-installed config file, which was actually not installed because my nfs config file had been edited. To fix:
mv /etc/sysconfig/nfs /etc/sysconfig/nfs.bak
mv /etc/sysconfig/nfs.rpmnew /etc/sysconfig/nfs
Then edit /etc/sysconfig/nfs to include whatever you actually need from /etc/sysconfig/nfs.bak.

[dwrice]

thank you very much

[lightman47]

Thank you from me as well - it was driving me crazy trying to figure out what went wrong!!!

[ThumbTack]

I had the same problem after upgrading a machine from F14 to F16. I fussed and fumed, checking the /etc/exports file, the Selinux settings, the nfs exports settings, the systemctl enables, etc., all to no avail. It was obvious that something was wrong because "rpcinfo -p localhost" did not show port 2049 as in play. Finally, I found this note and the comment about nfs.rpmnew!!! Thank you very much for this posting.

This has to be regarded as a preupgrade and upgrade bug. As part of the upgrade, there is no reason not to save the old nfs file as nfs.0 or some such, pick up any non-standard settings from it (usually there won't even be any) and add them to the nfs.rpmnew file, creating a new nfs file. That is what I did by hand, and lo and behold.....now the mounts from another machine are working again.

But at the same time, this was a very very frustrating experience!!!

[marko]

Yes, it's an important part of administration of a machine to search for new *.rpmnew files. When a package gets updated, if rpm sees the old config file was changed, it puts the new config file down as suffixed to rpmnew.

It's up to the admin to find these and merge the old edits into the new file and rename it

I mostly just do this occasionally:

find /etc -type f -name "*rpmnew"

to find them (config files tend to go under /etc )

[urilabob]

I have to say, I agree with Thumbtack, rpm could do a whole lot better on config file update. It's essentially a three-way merge problem, between fred.cfg.orig, fred.cfg.usermods and fred.cfg.rpmnew. Three-way merge is a fairly-well-understood problem (see wikipedia). Of course there would be merges that a reasonable three-way merge system couldn't handle without intervention, but the vast bulk of cases could be handled automatically (in 15 years of linux admin, I can't recall a single case of a conflict that couldn't have been - i.e. one where I actually had to think about what to do). The utility diff3 is part of diffutils. Seems to me that rpm could first check if diff3 is present, then use it to merge the cfg files. If diff3 is absent, or the merge gives warnings, rpm could revert to its present behaviour, but if diff3 succeeds, rpm could silently, and presumably safely, generate the merged cfg file. It's notable that a closely-related RFE has already been submitted (bug 710233) but appears to have been ignored, which is rather sad.