Why is yum so dog slow these days?

Nowwhat · #1 12th November 2011, 06:07 PM

...and I don't mean the download speed. I mean the whole transaction business when it's already local. No fast mirror would help.

Posting this in rawhide, because I'm on rawhide, and I'm using yum 3.4.3 from there.

Let's say I have a relatively modern machine. It's a Thinkpad Edge laptop, dual Neo L325 CPU, eight (8) gigs of RAM (so no one gets me started on too little RAM), and a generic species of 5400rpm hard drives.

Take, for example, installing a small package.

Code:

time sudo yum -y install telnet

The thing starts thrashing my hard drive for no apparent reasons, then install a package which has got 61 KiB in size (while thrashing furiously, mind you), then thrash again for… (drum roll) the whole total of (drum roll) (it's still thrashing as I'm writing this)

Code:

real	7m8.694s
user	0m5.067s
sys	0m2.399s

It's SEVEN. Freaking. Minutes. All of them. To install 61 kilobytes worth of just one package. On a modern machine.

I suspect yum developers have never ever tried to use it on anything weaker than an OCZ PCIe SSD, or a SAS RAID array.

Yes, I know about that bug. Unfortunately, it's almost two years since it was open, it was closed since, but the cause for which it was closed was long ago forgotten, it seems.

Because the cause for slowdowns remains the same: the whole RPM database is being read all over, over and over.

Quoting from the bug comments:

Quote:

2. The new rpmdb caching requires a "what state/version/whatever is the rpmdb"
so we can know when we have to invalidate our cache (either because we screwed
up, or someone altered the rpmdb from outside yum).

which kinda defeats the purpose of caching, which is SPEED in the first place. What else makes me suspicious is that this information is queried several times (namely, the version of rpmdb) during a yum run. During that run, the rpmdb is locked and it's quite obvious nobody, ever, would muck around with it.

Of course someone would defend the scheme by going on with technobabble on how the rpmdb works, but I seem remembering that for the same task, which is packages' installation, obsoleting and removal, Debian spends a fraction of that time. And if Fedora's scheme must require seven minutes for a single package, it means the scheme is totally broken.

Which makes me wonder just how competent yum developers are.
[/rant]

Adunaic · #2 12th November 2011, 06:35 PM

Before ranting on about things, it might be worth checking if it is a fault on your end and not one with yum as a whole. On my machine the same command:

Code:

time yum -y install telnet

runs very nicely

Code:

real	0m6.019s
user	0m2.201s
sys	0m0.301s

perhaps the error is not actually with yum (which I have always found to be fast) but with something on your end.

Nowwhat · #3 12th November 2011, 06:36 PM

Adunaic, can you post your yum version and the result of command

Code:

rpm -qa | wc -l

before assuming anything wrong on my side? Mine is 3.4.3 and 2500.

Adunaic · #4 12th November 2011, 06:47 PM

I am not assuming anything wrong on your side. Just stating that I do not find it that slow, and if it was I think there would have been more complaints. I have version 3.2.29-9 and the result of

Code:

rpm -qa | wc -l

is 2096.

I am not sure how the later is of help. What version of Fedora are you running?

jenaniston · #5 12th November 2011, 06:52 PM

Quote:

Originally Posted by Nowwhat

. . .
eight (8) gigs of RAM (so no one gets me started on too little RAM),

Which makes me wonder just how competent yum developers are.
[/rant]

Yeh . . the rants / stresses / frustrations of being on the cutting (oops, I mean bleeding) edge.
Yum - and Yum Extender is -imho - one of the best parts about Fedora - much better than Ubuntu's Synaptic Package Manager and the Debian's apt-get.

But sorry to hear of your frustrating problem - and maybe a suggestion.

First though, about those too little RAM arguments (which you obviously don't have),
just refer them folks to the linux swap space if they are newbies coming from the Bill Gates monopoly operating system.

Of course with that strategy, use mkswap command instead of the partioning with say cfdisk
(rarely a good swap in my experience, even if easy to appear that you actually have made a swap that can augment low RAM)
and then . . . verify that the swap is actually usable with the free command (important!)

My point is that I have run Fedora 14 in ridiculously low RAM machines (Windows98) although also adding RAM with the swap certainly improved performance further. I feel I could ignore the recommended minimal RAM with enough swap.

For your particular situation - and I'd doubt it could be that much of a problem . . . but if it really is . . .

Maybe try Yum Extender (yikes . . . the easiest way to get is to use yum to get it !!!)

Code:

yum install yumex

or maybe try and use PackageKit, which may already installed ?

Honestly, good luck.

Nowwhat · #6 12th November 2011, 06:54 PM

As I previously said, I'm running rawhide and yum is 3.4.3.
The bug I have referenced above was "fixed" in 3.2. Then they just reintroduced slowness again.

Adunaic · #7 12th November 2011, 06:56 PM

Also, just realised that this is in the rawhide forum. So I would assume you are using rawhide. If memory serves me correctly, then this means all of the debugging code that is possible will be turned on - it ain't gona run fast at all. Also its nowhere near beta - file a bug report if you really think there is an issue, but do not rant about the devs being useless because you are using software that is not for general release.

Nowwhat · #8 12th November 2011, 07:10 PM

yum 3.4.3 is considered a stable version. Please do not mix "unstable distribution" (which means mainly installation/dependency/integration kinks to iron out) and "unstable software" (alpha, beta, developer preview etc).

zarthon · #9 12th November 2011, 08:05 PM

There is not doubt that is terrible performance alpha beta , even a dev test ! lol !
I am not sure it's fair to question all the yum devs as a group about this and it could just be a mistake.
Very annoying!

What yum plugins have you installed ?
What happens when you run with no plugins and only from cache ?

Code:

yum -C --noplugins install telnet

Is this a fresh install or an upgraded upgraded upgraded upgrade? ! ?

What else do you have loaded and running and what is your disk setup?

What are the results of:
yum repolist
uptime
free -m

The community needs people willing to run Rawhide and report bugs. I'd only do this if you like that sort of thing, it makes you happy and does not get in the way of having fun or getting other things done. Yum is in python and the code is fairly readable. You could even track down the bug with the python debugger by seeing what it's doing during the disk thrashing.

Personally I'd only run Rawhide in a virtual machine. That is beside the point about yum and what can make it slow or what could cause disk thrashing. I am stumped without looking at the code itself.
You might check for root kits with rkhunter and the like. It just seems like very weird behavior.

Quote:

Originally Posted by Nowwhat

yum 3.4.3 is considered a stable version. Please do not mix "unstable distribution" (which means mainly installation/dependency/integration kinks to iron out) and "unstable software" (alpha, beta, developer preview etc).

Rawhide is the version that is intended for developers and QA. You can call it what you like but it is equivalent to alpha. Rawhide after a RC/ stable release is often more unstable than later in the cycle.

I see your point that upstream packages are usually "stable" unto themselves.
There is a process of selection and inclusion that screens out 'alpha' application binaries however
packages and components that are "driven" by the Fedora project and are central to the distro like yum may not be so stable.

Also this is an operating system, with apps libs, and utilities that all must interrelate and be configured together. "Configuration" and "kinks" can be complete system failures loss of data, etc.

RHamel · #10 12th November 2011, 09:08 PM

It's not even Alpha. It's Pre-Alpha. What's there is going to become F17, which isn't scheduled to go public for another 6 months. It's not unstable. It's slow. Well, I would expect all the debug and diagnostic routines to be turned on at this point. So slow should be a normal state.

Nowwhat · #11 12th November 2011, 09:13 PM

The superstable F16 also has this bug.

Adunaic · #12 12th November 2011, 09:33 PM

From what I can tell that bug and yours are not the same.

Also, no-one said that Fedora 16 was super stable (its not) just that rawhide is not stable.

Nowwhat · #13 13th November 2011, 12:03 AM

Quote:

From what I can tell that bug and yours are not the same.

I know they are the same because I did all the debugging it needed.
The bug indeed is in RPM's innards, which doesn't make yum any faster.

I'm interested in profiling it more after the bug is fixed. As of now, even simple getting the list of packages takes ridiculous time and obscures any other interesting things that might have caught attention otherwise.

YeOK · #14 13th November 2011, 01:12 PM

I have tested for this bug but I can't seem to reproduce it,

Fedora 16 x86_64.

Code:

time rpm --rebuilddb -v

real	0m8.179s
user	0m5.575s
sys	0m0.922s

Code:

[root@yeok ~]#time yum -y install telnet
Loaded plugins: fastestmirror, langpacks, presto, refresh-packagekit
Loading mirror speeds from cached hostfile
 * fedora: mirror01.th.ifl.net
 * rpmfusion-free: mirror01.th.ifl.net
 * rpmfusion-free-updates: mirror01.th.ifl.net
 * rpmfusion-nonfree: mirror01.th.ifl.net
 * rpmfusion-nonfree-updates: mirror01.th.ifl.net
 * updates: mirror.ovh.net
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package telnet.x86_64 1:0.17-51.fc16 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package         Arch            Version                  Repository       Size
================================================================================
Installing:
 telnet          x86_64          1:0.17-51.fc16           fedora           61 k

Transaction Summary
================================================================================
Install       1 Package

Total download size: 61 k
Installed size: 61 k
Downloading Packages:
telnet-0.17-51.fc16.x86_64.rpm                           |  61 kB     00:00     
Running Transaction Check
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : 1:telnet-0.17-51.fc16.x86_64                                 1/1 

Installed:
  telnet.x86_64 1:0.17-51.fc16                                                  

Complete!

real	0m3.130s
user	0m1.737s
sys	0m0.167s

I only run Fedora 17 in a VM and having just changed VM to kvm I would need to install Fedora 16 to compare. I will download an ISO and give it a go.

RahulSundaram · #15 13th November 2011, 09:53 PM

Quote:

Originally Posted by Nowwhat

As I previously said, I'm running rawhide and yum is 3.4.3.
The bug I have referenced above was "fixed" in 3.2. Then they just reintroduced slowness again.

If you believe this, then it should be reported as a regression in bugzilla. Posting it here won't fix it.