Fedora Linux Support Community & Resources Center
  #1  
Old 27th March 2017, 09:33 PM
Porcelain Mouse Offline
Registered User
 
Join Date: Aug 2011
Posts: 35
linuxfedorafirefox
Networking unusable, slow after dnf upgrade NEED HELP

It has only been two weeks since my previous dnf update, but last night I did another dnf upgrade operation and, after reboot, the networking was extremely slow, I mean, FF failed to load the simple start.fedoraproject.org homepage because it timed out. Even local network is crazy slow. When I loging remotely with SSH, it looks like it's gong to work, then, when I get to the command line, it's frozen...for about 3 minutes, then my packet flow again, suddenly, then stall for a 3 minutes again. It's like there is some burst, then everything is queued up, then it releases, but it's so late, that the connection could time out in the delay. I see network resets if SYN didn't complete in time.

When I do simple ping, I dont' see any delays; it's 1 pkt/s, no trouble. But, TCP connections like SSH and HTTP, which also use a lot more data, are complete messed up.

There are no errors or warnings anywhere in the journal. System is pretty old Reltek on-bard LAN that has worked for years. The system was working just fine immediately before update.

I thought it might be kernel, but going back to old kernel *did not help.* Exact same symptoms.

It's like the driver got messed up, but it's not the driver. Nothing else related to networking was updated, AFAICT, and I considered every package in dnf log.

So confused. And seriously screwed. Does anyone have any idea what this could be?!
Reply With Quote
  #2  
Old 28th March 2017, 07:23 AM
Porcelain Mouse Offline
Registered User
 
Join Date: Aug 2011
Posts: 35
linuxfedorafirefox
Re: Networking unusable, slow after dnf upgrade NEED HELP

Even stranger! I left the system alone for 12 hours...and the symptoms went away!

It doesn't make any sense. I was trying to figure it out what was wrong for like 3 hours, so it wasn't a short blip event.

If anyone has any thoughts on what happened, please let me know. This is extremely concerning. (Virus?)
Reply With Quote
  #3  
Old 28th March 2017, 01:33 PM
flyingdutchman Online
Registered User
 
Join Date: Jan 2015
Location: Al Ain, UAE
Posts: 674
linuxfedorafirefox
Re: Networking unusable, slow after dnf upgrade NEED HELP

To measure, is to know.

So, next time you have an issue, do this:
# ifconfig
Check for packet errors
Check the IP address

# route
Check the routing table
Is there a default route to the router

# ping www.yahoo.com
Does the DNS resolve the IP address
Is the ping speed OK

# tcpdump -nlX localhost
What is happening on the LAN?
Does it make sense?

BTW, there are no Linux viruses. If you ever do find one, please send it to me.
__________________
--
Have fun!
http://www.aeronetworks.ca
Reply With Quote
  #4  
Old 28th March 2017, 01:37 PM
rexrf Offline
Registered User
 
Join Date: Aug 2016
Location: Dallas
Posts: 45
linuxfedorachrome
Re: Networking unusable, slow after dnf upgrade NEED HELP

Could it have been your actual internet connection that was running slow (try testing your bandwidth and compare it to what you should be receiving http://speedtest.googlefiber.net/ ) Also, assuming you're using fedora with the default desktop environment (gnome) you should be using the software application to update your system.
Reply With Quote
  #5  
Old 28th March 2017, 02:47 PM
antikythera Offline
Administrator
 
Join Date: Dec 2013
Location: United Kingdom
Posts: 4,778
linuxchrome
Re: Networking unusable, slow after dnf upgrade NEED HELP

Quote:
Originally Posted by flyingdutchman View Post

BTW, there are no Linux viruses. If you ever do find one, please send it to me.
penguins are susceptable to H11N2...

seriously though, there was one such incident with Linux Mint. Someone uploaded an infected ISO (containing the "Kaiten" backdoor) to a Bulgarian mirror and managed to compromise the downloads page to point users to it.
__________________
Download, Install and Share Fedora - Official ISO Torrents | Live ISO Respins containing post-release updates
Reply With Quote
  #6  
Old 3rd April 2017, 09:10 AM
Porcelain Mouse Offline
Registered User
 
Join Date: Aug 2011
Posts: 35
linuxfedorafirefox
Re: Networking unusable, slow after dnf upgrade NEED HELP

Quote:
Originally Posted by rexrf View Post
Could it have been your actual internet connection ...
No, I asked myself the same thing several times. I had the same problem with local connections to my internal router interface while my laptop didn't have the same symptoms.
Reply With Quote
  #7  
Old 3rd April 2017, 09:38 AM
Porcelain Mouse Offline
Registered User
 
Join Date: Aug 2011
Posts: 35
linuxfedorafirefox
Re: Networking unusable, slow after dnf upgrade NEED HELP

Quote:
Originally Posted by flyingdutchman View Post
To measure, is to know.

So, next time you have an issue, do this:
# ifconfig
Check for packet errors
Check the IP address
I checked ips and they looked normal. I checked netstat --stats, but I didn't see anything odd; it doesn't report errors. I didn't try ifconfig, and I see now that would have been useful. There were retransmits, but that's normal; I see >6000 now and I just rebooted so I don't know what would be a lot.

Quote:
Originally Posted by flyingdutchman View Post
# route
Check the routing table
Is there a default route to the router
Yes, routes looked good.

Quote:
Originally Posted by flyingdutchman View Post
# ping www.yahoo.com
Does the DNS resolve the IP address
Is the ping speed OK
Yes, but not always quickly. (Ha! I still use yahoo.com for ping, too!) And ping times were great, totally normal and *constant.* That's when I started to realize that it wasn't upstream, but local. Whatever it was, I don't think it affected ICMP, whatever that suggests. I ran many tracepath tests and got extremely inconsistent results, it would stall at different points each pass, meaning, sometimes it would enumerate 10 hops quickly (before getting no reply), but the next time it would get only 3 before stalling for a few minutes, then report the rest. It was much like HTTP, which was getting some data, then stalling for a long time, then getting more data.

Quote:
Originally Posted by flyingdutchman View Post
# tcpdump -nlX localhost
What is happening on the LAN?
Does it make sense?
I used iftop and saw a few things going on, mostly local chatter, but total traffic was tiny. I also have pcap captures between me and my router. I see retransmit bursts, but I didn't see immediately what was the problem. Perhaps I can revisit it now or get someone to look at it who does it more often. It's harder because it's HTTPS and I can't see where the stall happened, not that that would really be helpful.
Reply With Quote
  #8  
Old 3rd April 2017, 12:55 PM
flyingdutchman Online
Registered User
 
Join Date: Jan 2015
Location: Al Ain, UAE
Posts: 674
linuxfedorafirefox
Re: Networking unusable, slow after dnf upgrade NEED HELP

So, your LAN is hickety pickety...

Check whether you perhaps have an ethernet loop - a cable plugged into the router at both ends, or two ethernet cables running to the same machine, or a machine both on a wire and WiFi. A loop will cause broadcast storms: ARP messages will go round and round clogging up the works.

Check the DNS response time with nslookup or dig.

You should be able to see what is the matter with tcpdump or ettercap if it is a LAN problem.

For example:
a. If there are bazillions of ARPs, then you have a broken switch or a loop.
b. If your primary DNS never replies causing a fail over to the secondary, then obviously you have a lame DNS in your resolv.conf due to a bad DHCP configuration.
__________________
--
Have fun!
http://www.aeronetworks.ca
Reply With Quote
  #9  
Old 12th April 2017, 11:14 PM
Porcelain Mouse Offline
Registered User
 
Join Date: Aug 2011
Posts: 35
linuxfedorafirefox
Re: Networking unusable, slow after dnf upgrade NEED HELP

First of all, many thanks for helping me and thinking about this. I know it's not easy and you've never seen my network.

Quote:
Originally Posted by flyingdutchman View Post
So, your LAN is hickety pickety...
Hmm, not sure why you say that. I've had the same network topology for five years. And I haven't changed any aspect of the network in over one year. Everything was fine, I did dnf update, reboot, and immediately noticed the problem. There is no reason to suspect anything external to the box unless is was some interaction between network and some updated software, but how would it resolve itself.

Quote:
Check whether you perhaps have an ethernet loop - a cable plugged into the router at both ends, or two ethernet cables running to the same machine, or a machine both on a wire and WiFi. A loop will cause broadcast storms: ARP messages will go round and round clogging up the works.
Sure, I follow that, but I don't see how I could get an Ethernet loop. I can see all the ports and I didn't plug in or unplug anything before the problem started. Network is tiny; I think you are imagining something bigger. There was only like five nodes on the whole network at this time.

Quote:
Check the DNS response time with nslookup or dig.
I don't remember checking this specifically, but as I said, I use www.yahoo.com for ping test and, if DNS was slow, that wouldn't have resolved quickly, which I don't remember happening. I've noticed such delays before when DNS wasn't work and I don't remember that in this case, and I think I would.

Quote:
You should be able to see what is the matter with tcpdump or ettercap if it is a LAN problem.

For example:
a. If there are bazillions of ARPs, then you have a broken switch or a loop.
b. If your primary DNS never replies causing a fail over to the secondary, then obviously you have a lame DNS in your resolv.conf due to a bad DHCP configuration.
Okay, you could be on to something here. I noticed odd ARPs immediately, too, but couldn't think of an explanation. Check this out: Yes, I see some periods where there is a flurry of ARPs. All get answered immediately. But, the suspicious part is that they repeat shortly later, but there are not "bazillions" of APRs, either. For example, there's one, it's answered, and then 55 seconds later the same ARP from the same requester, even though the original one was answered correctly by the same NIC.

Another thing is that I see FIN ACK retransmits and several RST from the client to the server with no reply (as seen by the client snooping). This is from the period after I noticed the HTTP request was stalled. Makes sense that client would be trying to reset, but why no reply. At the same time, I could get to the same web service from another device and didn't notice any problems. Same network devices alone the path for both clients, but the client that didn't have problems was wireless, so that's a little different.
Reply With Quote
  #10  
Old 13th April 2017, 06:47 AM
flyingdutchman Online
Registered User
 
Join Date: Jan 2015
Location: Al Ain, UAE
Posts: 674
linuxfedorafirefox
Re: Networking unusable, slow after dnf upgrade NEED HELP

Hmm, are you sure all the IP addresses are unique - you don't perhaps have two devices with the same IP address?

Note that tcpdump will only show the traffic on the machine that you are running it on.

With ettercap, you can see the traffic to/from any machine on your LAN - it uses ARP spoofing to reroute the traffic to your host and back again to the real destination - a 'man on the middle attack' for debugging:
# ettercap -i eth0 –T –M arp /victim_ip_A/ /victim_ip_B/

There may be another faulty device and with ettercap you may be able to get some more clues.
__________________
--
Have fun!
http://www.aeronetworks.ca
Reply With Quote
  #11  
Old 19th April 2017, 11:53 PM
Porcelain Mouse Offline
Registered User
 
Join Date: Aug 2011
Posts: 35
linuxfedorafirefox
Re: Networking unusable, slow after dnf upgrade NEED HELP

Quote:
Originally Posted by flyingdutchman View Post
Hmm, are you sure all the IP addresses are unique - you don't perhaps have two devices with the same IP address?
Oh, no, oops, I wasn' t clear. The NIC that answered was not the NIC that requested. I mean that NIC1 asks who has NIC2, NIC2 replies immediately, then 55 seconds laps, then NIC1 asks who ask NIC2, again. The same exchange recurs in 55 seconds. But what would cause the requester's ARP entry for NIC2 to be cleaned in 55 seconds? I don't remember what the normal cache period is, but I'm pretty sure it's more than 55 seconds. Right?

I do remember that some situations can result in the removal of ARP table entries.

Oh, I see, yeah, if someone else on the network was answering for NIC2 or spoofing NIC2, then NIC1 would keep getting confused and have to ARP a lot. right. Hmm, I'll have to think about that. I don't know how that would happen. I've assigned all IPs myself, manually. If it really was malware spoofing IPs, a la Ethercap, that would do it.

How would I check for dups. DHCP server could tell; don't they log that sort of thing. Oh, right, I remember: I can broadcast ARP and watch for dup replies. yeah. I've done that before. Is there a tool that does it. NMAP?
Reply With Quote
  #12  
Old 29th May 2017, 03:25 AM
Porcelain Mouse Offline
Registered User
 
Join Date: Aug 2011
Posts: 35
linuxfedorafirefox
Re: Networking unusable, slow after dnf upgrade NEED HELP

Just wanted to follow up. I found the problem: bad switch/router.

As I said, my network hasn't changed in a long time. I finally tracked the problem down to a Netgear router which I was just using as a switch (no FW, passive). It guess it was just eating packets intermittently. It would go through periods of high packet loss and then be fine the next few days.
Reply With Quote
Reply

Tags
dnf, networking, slow, unusable, upgrade

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
tiger-vncserver: unusable after upgrade to fedora 25 petasis Using Fedora 6 21st December 2016 09:25 AM
Slow Scrolling (even in Terminal) , Firefox unusable thisismyname Installation, Upgrades and Live Media 2 27th October 2011 01:09 PM
F10 networking really slow khkachn Servers & Networking 2 20th January 2009 12:59 AM
Sorry but a bit slow here.just networking help muzz51 Servers & Networking 3 3rd March 2007 05:28 AM
networking VERY slow in FC2 test1 randy Fedora Core 2 Test Releases 11 1st June 2004 12:02 PM


Current GMT-time: 00:33 (Sunday, 20-08-2017)

TopSubscribe to XML RSS for all Threads in all ForumsFedoraForumDotOrg Archive
logo

All trademarks, and forum posts in this site are property of their respective owner(s).
FedoraForum.org is privately owned and is not directly sponsored by the Fedora Project or Red Hat, Inc.

Privacy Policy | Term of Use | Posting Guidelines | Archive | Contact Us | Founding Members

Powered by vBulletin® Copyright ©2000 - 2012, vBulletin Solutions, Inc.

FedoraForum is Powered by RedHat