System boots & network broken until service network restart. RTNETLINK complains!

msodrew · #1 2nd July 2008, 05:19 AM

So I've got a new Dell 1U that has a quad core opteron, 4 gigs of ram... very new machine. Installed the latest release of FC9 with minimal options selected. Everything installed fine, but the machine's networking fails on boot. When you login to the machine (of course being next to it), and do "service network restart", it complains about "RTNETLINK: file exists" but then networking suddenly works (can ssh in from outside, etc).

This is a huge problem, because this means that I cannot ever reboot the machine unless I'm right next to it. The machine is in a datacenter, leaving the chance of being near it highly unlikely.

A few things to mention since I've googled this for hours to no avail.

- only eth0 is enabled (no other interface)
- the config in ifcfg-eth0 is proper... the IP address is correct and not used by another machine, gateway correct, etc.
- I know the above are true because days before, the machine had FC6 on it with the exact same config file and no problems. I updated to FC9 to hopefully fix a kernel bug involving my particular hardware (unrelated to networking) and this networking issue arose out of nowhere that I cannot get rid of.

Here is my ifcfg-eth0 file (i replaced the real ip with xxx):

Code:

# Broadcom Corporation NetXtreme BCM5721 Gigabit Ethernet PCI Express
DEVICE=eth0
BOOTPROTO=static
BROADCAST=66.160.140.191
HWADDR=00:1e:c9:46:07:12
IPADDR=66.160.140.xxx
NETMASK=255.255.255.224
NETWORK=66.160.140.160
ONBOOT=yes

Here is what "service network restart" outputs (i replaced the real ip with xxx)

Code:

Shutting down interface eth0:                              [  OK  ]
Shutting down loopback interface:                          [  OK  ]
Bringing up loopback interface:                            [  OK  ]
Bringing up interface eth0:  RTNETLINK says: file exists Error adding address 66.160.140.xxx for eth0.
                                                           [  OK  ]

Does anyone have any remote idea what could be the culprit? Thank you.

stevea · #2 2nd July 2008, 09:25 AM

What is the state of the interface when you login ?
Login while the network is broken and collect the result of "ifconfig" "ifconfig -a" "ip addr show" "ip route show". Where is your GATEWAY= entry ? You can only see your LAN without a GATEWAY (but the route table should tell us.

The message probably means you are creating a duplicate or overlapping route entry.

stevea · #3 2nd July 2008, 09:47 AM

My hunch is 66.160.140.165 and you installed the 64bit F9.

BTW - check your services "chkconfig NetworkManager off", and look through /var/log/messages to see you you can spot the failure at startup problem.

msodrew · #4 2nd July 2008, 09:48 AM

Quote:

Originally Posted by stevea

My hunch is 66.160.140.165 and you installed the 64bit F9.

I did install the 64-bit F9... whats wrong with it?

stevea · #5 2nd July 2008, 09:49 AM

You have to tell me. You haven't posted enough info.

msodrew · #6 2nd July 2008, 09:58 AM

Quote:

Originally Posted by stevea

You have to tell me. You haven't posted enough info.

damn. unfortunately i thought i might have 'fixed' the problem, so i tried to reboot the machine but it never came back so i cannot shell into it right now. going to head to the datacenter tomorrow. fail. what info did you need exactly?

stevea · #7 2nd July 2008, 10:37 AM

read post #2.

msodrew · #8 3rd July 2008, 05:30 AM

OK ive got it all, here's my output... this is all from after i restarted the network and the machine complatins about RTNETLINK. I had to do it after otherwise there'd be no way i could copy and paste all this quickly short of scp'ing a bunch of files over to another machine. See if you can tell me anything from them...

ifconfig

Code:

eth0      Link encap:Ethernet  HWaddr 00:1E:C9:46:07:12  
          inet addr:66.160.140.164  Bcast:66.160.140.191  Mask:255.255.255.224
          inet6 addr: fe80::21e:c9ff:fe46:712/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2158 errors:0 dropped:0 overruns:0 frame:0
          TX packets:133 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:301283 (294.2 KiB)  TX bytes:20732 (20.2 KiB)
          Interrupt:33 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:30 errors:0 dropped:0 overruns:0 frame:0
          TX packets:30 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:5467 (5.3 KiB)  TX bytes:5467 (5.3 KiB)

ifconfig -a

Code:

eth0      Link encap:Ethernet  HWaddr 00:1E:C9:46:07:12  
          inet addr:66.160.140.164  Bcast:66.160.140.191  Mask:255.255.255.224
          inet6 addr: fe80::21e:c9ff:fe46:712/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1928 errors:0 dropped:0 overruns:0 frame:0
          TX packets:118 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:267903 (261.6 KiB)  TX bytes:15954 (15.5 KiB)
          Interrupt:33 

eth1      Link encap:Ethernet  HWaddr 00:1E:C9:46:07:13  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
          Interrupt:37 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:30 errors:0 dropped:0 overruns:0 frame:0
          TX packets:30 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:5467 (5.3 KiB)  TX bytes:5467 (5.3 KiB)

pan0      Link encap:Ethernet  HWaddr 22:49:39:B4:C7:53  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

sit0      Link encap:IPv6-in-IPv4  
          NOARP  MTU:1480  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

ip addr show

Code:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:1e:c9:46:07:12 brd ff:ff:ff:ff:ff:ff
    inet 66.160.140.164/27 brd 66.160.140.191 scope global eth0
    inet6 fe80::21e:c9ff:fe46:712/64 scope link 
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:1e:c9:46:07:13 brd ff:ff:ff:ff:ff:ff
4: pan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop 
    link/ether 22:49:39:b4:c7:53 brd ff:ff:ff:ff:ff:ff
5: sit0: <NOARP> mtu 1480 qdisc noop 
    link/sit 0.0.0.0 brd 0.0.0.0

ip route show

Code:

66.160.140.160/27 dev eth0  proto kernel  scope link  src 66.160.140.164 
169.254.0.0/16 dev eth0  scope link 
default via 66.160.140.161 dev eth0

thanks

msodrew · #9 3rd July 2008, 05:41 AM

Alright so this is the result of "cat /var/log/messages | grep NetworkManager" BEFORE I did a restart on the network (ie: network is broken).

Code:

Jul  2 21:32:27 megaphone NetworkManager: <info>  starting...
Jul  2 21:32:27 megaphone NetworkManager: <info>  eth1: Device is fully-supported using driver 'tg3'.
Jul  2 21:32:27 megaphone NetworkManager: <info>  Found new Ethernet device 'eth1'.
Jul  2 21:32:27 megaphone NetworkManager: <info>  (eth1): exported as /org/freedesktop/Hal/devices/net_00_1e_c9_46_07_13
Jul  2 21:32:27 megaphone NetworkManager: <info>  eth0: Device is fully-supported using driver 'tg3'.
Jul  2 21:32:27 megaphone NetworkManager: <info>  Found new Ethernet device 'eth0'.
Jul  2 21:32:27 megaphone NetworkManager: <info>  (eth0): exported as /org/freedesktop/Hal/devices/net_00_1e_c9_46_07_12
Jul  2 21:32:27 megaphone NetworkManager: <info>  Trying to start the supplicant...
Jul  2 21:32:27 megaphone NetworkManager: <info>  Trying to start the system settings daemon...
Jul  2 21:32:27 megaphone nm-system-settings: Loaded plugin ifcfg-fedora: (c) 2007 - 2008 Red Hat, Inc.  To report bugs please use the NetworkManager mailing list.
Jul  2 21:32:31 megaphone NetworkManager: <info>  (eth1): device state change: 1 -> 2
Jul  2 21:32:31 megaphone NetworkManager: <info>  (eth1): bringing up device.
Jul  2 21:32:31 megaphone NetworkManager: <info>  (eth1): preparing device.
Jul  2 21:32:31 megaphone NetworkManager: <info>  (eth1): deactivating device.
Jul  2 21:32:31 megaphone NetworkManager: <info>  (eth1): carrier now ON (device state 2)
Jul  2 21:32:31 megaphone NetworkManager: <info>  (eth1): device state change: 2 -> 3
Jul  2 21:32:31 megaphone NetworkManager: <info>  (eth0): device state change: 1 -> 2
Jul  2 21:32:31 megaphone NetworkManager: <info>  (eth0): preparing device.
Jul  2 21:32:31 megaphone NetworkManager: <info>  (eth0): deactivating device.
Jul  2 21:32:31 megaphone NetworkManager: <info>  (eth1): carrier now OFF (device state 3)
Jul  2 21:32:31 megaphone NetworkManager: <info>  (eth1): device state change: 3 -> 2
Jul  2 21:32:31 megaphone NetworkManager: <info>  (eth1): deactivating device.

why does it do "preparing device" then "deactivating device" ?

msodrew · #10 3rd July 2008, 06:00 AM

OH GOD... FIXED.

I had to turn off something called the avahi-daemon ... it was ****ing **** up. What the hell fedora?!

For anyone else with this problem do:

Code:

chkconfig avahi-daemon off
reboot

Thanks for the help anyway man!