DNF eats up bandwidth
FedoraForum.org - Fedora Support Forums and Community
Results 1 to 8 of 8
  1. #1
    Join Date
    Jun 2015
    Location
    Lithuania
    Posts
    5
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    DNF eats up bandwidth

    Dear forum members,

    I am very happy with the installation of Fedora 22 on my laptop, and am quickly learning the differences with the previous editions.

    One of them is the dnf repository manager for software. It works well, but I have one problem.

    Everytime I want to do
    Code:
    dnf search
    or even
    Code:
    dnf install
    it will connect to the network and refresh the metadata. This is fine perhaps for many people, but here the internet is actually fairly expensive - around 10 $USD for 2 GB.

    Thus even if I want to just search for a certain package, I have to pay money.... Why can't dnf just use the metadata it has? Can't it just search the current database? Can it be configured so that it only updates this metadata when I want it to? I thought that was the whole purpose of
    Code:
    dnf update
    was to update this metadata....

    I've looked at the documentation, and I put
    Code:
    metadata_timer_sync=0
    in /etc/dnf/dnf.conf but this makes no difference whatsoever. Just now I restarted the machine, tried to
    Code:
    dnf install offlineimap
    and the same thing.... only that it goes to 44MB downloading and just hangs there.

    44MB is an enormous file -- why on earth so much metadata? Even in FreeBSD the *entire* ports tree of over 22000 package descriptions and make files is at most 65MB. What is going on with this huge consumption of bandwidth.

    Please remember that not all of us have broadband and we have to pay for this data.... I really hate dnf as I fear every time I just want to see if a package exists I end up paying a fortune!

    HELP ME CONFIGURE THIS MONSTER SO IT DOESN'T DO THIS ANYMORE!

    Thank you

  2. #2
    PabloTwo's Avatar
    PabloTwo is offline "Registered User" T-Shirt Winner
    Join Date
    Mar 2007
    Location
    Seville, FL
    Posts
    8,069
    Mentioned
    9 Post(s)
    Tagged
    0 Thread(s)

    Re: DNF eats up bandwidth

    One of things I learned long ago about yum, the predecessor to dnf, was that running yum as a regular user and as root created two entirely separate sets of yum databases. I believe that is the same now with dnf. I adopted the practice of issuing any yum (and now dnf) command only as root, eliminating the duplicate files and cutting down significantly on bandwidth usage.

  3. #3
    Join Date
    Jun 2015
    Location
    Lithuania
    Posts
    5
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: DNF eats up bandwidth

    I found a solution for short term:

    One can invoke dnf and force it to use cache data:

    dnf -C search offlineimap

    And then it will simply give what is in the cache.

    However, I noticed that there is a systemd process always sychronizing the repos - so even here I am getting enormous amount of data used for no real purpose. In fact I calculate that just this background task is about 1G a month. This is simply unacceptable!

    I have been searching around Google for answers, and I stumbled upon this link:

    Why I prefer YUM

    ---------- Post added at 08:48 PM ---------- Previous post was at 08:43 PM ----------

    Some more research shows that this is a big problem:

    BIG BUG THEY WON'T FIX

    Seriously, for something as critical as a software management application, this is simply FAIL.

    This is simply an enormous design flaw, it needs to be corrected ASAP.

    This flaw is costing people a lot of money, so saying that Fedora is Free software is only half correct - it will end up costing me much more than Windows if this bug is not fixed!

  4. #4
    Join Date
    Jun 2015
    Location
    Lithuania
    Posts
    5
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: DNF eats up bandwidth

    Some more research:

    How to disable automatic updates by systemd:

    Code:
    sudo systemctl disable dnf-makecache.timer
    And while you are at it, might as well tell Gnome not to do this either:

    Code:
    gsettings set org.gnome.software download-updates false
    Very annoying. These things should be option-in, not defaults. It is no wonder the internet is so slow somedays - everyone is just constantly downloading updates -- or worse, just metadata of updates ;/

  5. #5
    Join Date
    Jun 2004
    Location
    Maryland, US
    Posts
    8,152
    Mentioned
    17 Post(s)
    Tagged
    0 Thread(s)

    Re: DNF eats up bandwidth

    Resolving all these issues is covered in the dnf man page.
    I'm on my cell phone now so I can't look but see in
    'man dnf.conf' and see metadata_expire and the systemd related timer
    Settings exist for both so they never happen.

  6. #6
    Join Date
    Jun 2015
    Location
    Lithuania
    Posts
    5
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: DNF eats up bandwidth

    Dear Marko,

    Thank you very much for the reply! I looked in the man page and was able to set those variables to something appropriate for my needs.

    I also found a way to disable the automatic updates as well.

    However, I am really wondering why the official repo needs to download 44+ MB of data for each update to the metadata. This seems to be very excessive.

    For instance, in FreeBSD (which I use everywhere else but on my laptop - video drivers not yet finished for the haswell), there is an initial "portsnap" which is around 65 MB and the subsequent updates to the ports tree are only diff files. The entire ports tree are simply text files [which compress well] and the diffs are simply diffs to these files. Thus a portsnap fetch can be just a few kilobytes.

    In fact you can even use Subversion to update the ports tree, which is what I usually do. This is very efficient and requires minimal resources. You also know immediately the last commit to the tree and you can see if you really need to update it or not.

    The actual metadata for a repo cannot be much more than what it takes to actually build a program from source. It seems to me a serious design flaw to require so much data on every update to the metadata.

    Why doesn't DNF store metadata in a simple git repo? It would make updates very easy - and automatic compression, history and the rest almost for free.

    If I wanted to contribute to DNF how would I do so?
    Last edited by jsrjenkins; 17th September 2015 at 08:47 PM. Reason: small typo

  7. #7
    Join Date
    Jun 2004
    Location
    Maryland, US
    Posts
    8,152
    Mentioned
    17 Post(s)
    Tagged
    0 Thread(s)

    Re: DNF eats up bandwidth

    Actually the git idea is being considered, I found a forum where
    It was being talked about, but I can't seem to find it again

  8. #8
    Join Date
    Jan 2010
    Posts
    7,561
    Mentioned
    1 Post(s)
    Tagged
    0 Thread(s)

    Re: DNF eats up bandwidth

    I know that at one point, dnf didn't use deltarpm by default, but figured that had changed. (A quick look at google shows that yes, that has changed, if deltarpm is available, dnf will use it.)

    That should cut down on bandwidth usage by dnf. Make sure you have deltarpm installed.

Similar Threads

  1. Skype eats whole RAM!!
    By rahularjun86 in forum Using Fedora
    Replies: 2
    Last Post: 25th May 2012, 11:16 AM
  2. [SOLVED]
    windows eats mbr. ugh.
    By adempewolff in forum Using Fedora
    Replies: 0
    Last Post: 20th June 2010, 03:52 AM
  3. xvnc eats 100% cpu
    By carlainz in forum Using Fedora
    Replies: 0
    Last Post: 14th August 2007, 09:47 AM
  4. yum eats up all the memory
    By Botond in forum Fedora Core 5 - Dev
    Replies: 3
    Last Post: 2nd February 2006, 02:13 PM
  5. mysterious java thread eats all bandwidth
    By nandowong in forum Using Fedora
    Replies: 5
    Last Post: 23rd January 2006, 11:46 PM

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •