Suddenly rsyncd fails to load at boot
FedoraForum.org - Fedora Support Forums and Community
Page 1 of 2 1 2 LastLast
Results 1 to 15 of 18

Hybrid View

Previous Post Previous Post   Next Post Next Post
  1. #1
    Join Date
    Feb 2009
    Location
    Island of Roatan in the Caribbean
    Posts
    281

    Suddenly rsyncd fails to load at boot

    Yesterday, I had an F26 box die due to drive failure. I replaced the drive, installed F26, completely patched it and then restored the user data from backup.

    This box always runs via WiFi. Now, suddenly, rsyncd wants to start before there's a WiFi NIC available and therefore fails during boot. The nightly backup for this box relies on rsync and that's how I noticed an issue with rsyncd. The journal clearly shows rsync starting and failing before there's a WiFi connection. Once the boot process allows for a root login, I can systemctl start rsyncd successfully.

    I want to know what changed to cause this, as this box has been running F26 for months with rsyncd apparently loading successfully during boot because I never had a backup issue before.

    What provides the sequence of loading instructions to tell rsyncd to wait for NIC(s) to finish becoming available before attempting to use one?

  2. #2
    Join Date
    Jun 2005
    Location
    Montreal, Que, Canada
    Posts
    5,300

    Re: Suddenly rsyncd fails to load at boot

    sudo dnf install rsync*
    will bring in the daemon part for rsync

    rsync --daemon

    You could make an alias command

    alias rsyncd='rsync --daemon'
    Leslie in Montreal

    Interesting web sites list
    http://forums.fedoraforum.org/showth...40#post1697840

  3. #3
    Join Date
    Feb 2009
    Location
    Island of Roatan in the Caribbean
    Posts
    281

    Re: Suddenly rsyncd fails to load at boot

    I have the daemon installed.

    It's a sequence of events issue. rsyncd shouldn't attempt to start till after the WiFi nic is up.

  4. #4
    Join Date
    Oct 2011
    Posts
    1,841

    Re: Suddenly rsyncd fails to load at boot

    I don't see any service dependencies on the network Maybe you've had its socket enabled before, meaning it would start only when there is need for it. Network would be up and running by that time. If there is no such service dependency rsync should be able to handle no network issue and try to reconnect on its own. It is possible that that changed with new rsync version.

    If rsync --daemon fails without a network, service should include previous mentioned dependencies, then that is a bug in the service file. You can report it at bugzilla.redhat.com .

    I think the service should at least include a dependency on network.target
    Last edited by srakitnican; 14th March 2018 at 07:10 AM.

  5. #5
    Join Date
    Feb 2009
    Location
    Island of Roatan in the Caribbean
    Posts
    281

    Re: Suddenly rsyncd fails to load at boot

    The box has a wired NIC in it that I'm not using but I can test to see what happens if I enable it and disable WiFi. I've got other boxes that are hard wired but have WiFi capability, so I'll switch them over to WiFi to see if I can reproduce the problem. They're all patched to the limit so should all be running the same software.

    What I want to know is what changed. This box has run F26, F25, F24, etc and never had a problem.

  6. #6
    Join Date
    Oct 2011
    Posts
    1,841

    Re: Suddenly rsyncd fails to load at boot

    Dices rolled wrong perhaps. When there are no service dependencies set, services start in any order. I guess you could take a look at other machines journalctl log if network.target starts before rsyncd.

  7. #7
    Join Date
    Oct 2006
    Location
    CN99CF Agassiz BC Canada
    Posts
    397

    Re: Suddenly rsyncd fails to load at boot

    I noticed something interesting with respect to rsync-daemon from the release notes
    Code:
           Release : FEDORA-2018-034101216d
              Type : Security
            Issued : 2018-03-15 09:48:22
    
          Bugzilla : 1536662 - CVE-2018-5764 rsync: sanitization bypass in parse_argument in oprions.c [fedora-all]
                   : 1539466 - rsync-3.1.3 is available
    
       Description : Removing dependencies on systemd-units
                     
                     ----
                     
                     New version 3.1.3, includes security fix for  CVE-2018-5764
    that appears to support the type of behavior you are experiencing.

    Could your please let us know which version level of rsync-daemon you have on you system?

    With the removal of the systemd unit dependencies, you may have lost your dependence on having your network (WiFi in your case) available.

    In my case, I have version 3.1.3 installed on my system. Issuing a 'systemctl status rsyncd.service' command shows:
    Code:
    ● rsyncd.service - fast remote file copy program daemon
       Loaded: loaded (/usr/lib/systemd/system/rsyncd.service; disabled; vendor preset: disabled)
       Active: inactive (dead)
    Note the 'disabled' state of the rsyncd.service. This would explain why you can issue the command manually after your WiFi is operational and why rsyncd starts out of sequence before the WiFi is operational.

    I have read elsewhere that you can condition the start-up of rsyncd by using xinetd and I am thinking this is what the developers / maintainers my have been trying to achieve and did not complete the change. The link for that article is at https://toic.org/blog/2009/howto-create-rsync-server/.

    Let's start by looking at the version and seeing if downgrading on your system will return your backup server to functioning.
    -----
    f26 x86_64 Acer Predator G5910 Quad core Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz

  8. #8
    Join Date
    Feb 2009
    Location
    Island of Roatan in the Caribbean
    Posts
    281

    Re: Suddenly rsyncd fails to load at boot

    I patched all my boxes yesterday so they are all running rsyncd 3.1.3 .

    I disabled the wired NIC in my box and started the WiFi connection. Rebooted and WiFi comes up and so does rsyncd. The broken box is still broken.

    Here's what the broken box says:
    [root@xxxxx ~]# systemctl status rsyncd
    ● rsyncd.service - fast remote file copy program daemon
    Loaded: loaded (/usr/lib/systemd/system/rsyncd.service; enabled; vendor preset: disabled)
    Active: failed (Result: exit-code) since Fri 2018-03-16 07:46:51 CST; 12min ago
    Process: 796 ExecStart=/usr/bin/rsync --daemon --no-detach $OPTIONS (code=exited, status=10)
    Main PID: 796 (code=exited, status=10)

    Mar 16 07:46:51 xxxxx.lmnop.xyz systemd[1]: Started fast remote file copy program daemon.
    Mar 16 07:46:51 xxxxx.lmnop.xyz systemd[1]: rsyncd.service: Main process exited, code=exited, status=10/n/a
    Mar 16 07:46:51 xxxxx.lmnop.xyz systemd[1]: rsyncd.service: Unit entered failed state.
    Mar 16 07:46:51 xxxxx.lmnop.xyz systemd[1]: rsyncd.service: Failed with result 'exit-code'.

    It's something peculiar with this box's O/S. I may blow the box away and reinstall just out of curiosity. I used to use xinetd but boxes have so much RAM and so much power these days that xinetd is so last century now.

  9. #9
    Join Date
    Oct 2006
    Location
    CN99CF Agassiz BC Canada
    Posts
    397

    Re: Suddenly rsyncd fails to load at boot

    A thought ...

    • Is there any difference between the .service file on the 'broken' box and one on a functioning box?
    • Are the .service files enabled on the 'working' boxes?
    -----
    f26 x86_64 Acer Predator G5910 Quad core Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz

  10. #10
    Join Date
    Jun 2005
    Location
    Montreal, Que, Canada
    Posts
    5,300

    Re: Suddenly rsyncd fails to load at boot

    I have a root crontab and therein I have my rsync backup scheduled. Backup once per day. I use a script that has two parts. One checks the day of the year versus a stored value for the day of the year. If they match, rsync is not run. If they do not match, rsync is run and step2 is to update that day of the year value.

    date "+%j" (today is 075)
    Leslie in Montreal

    Interesting web sites list
    http://forums.fedoraforum.org/showth...40#post1697840

  11. #11
    Join Date
    Feb 2009
    Location
    Island of Roatan in the Caribbean
    Posts
    281

    Re: Suddenly rsyncd fails to load at boot

    For the files :
    /etc/systemd/system/multi-user.target.wants/rsyncd.service
    /usr/lib/systemd/system/rsyncd.service
    /usr/lib/systemd/system/rsyncd@.service
    both the box that works (my personal box) as well as the broken box have identical files according to cksum.

    rsyncd is enabled on both boxes. I can see no difference between the boxes.

    I have a set of master scripts that I use to do installs. I just run one script after the other, some on the master (my box), some on the target, checking to make sure all went OK, and I've never seen this before. Those scripts guarantee that one box is the same as the next box and it's always worked before. I've been doing this since Fedora Core 2.

    Right now, I have a cron job that runs every hour to 'restart' rsyncd on the broken box so when the user turns the box on some time during the day, I'm confident that rsyncd will be up by time the backup runs at 7PM for that machine. Backups are now working as usual because rsyncd is up when the backup runs.

  12. #12
    Join Date
    Dec 2013
    Location
    United Kingdom
    Posts
    6,279

    Re: Suddenly rsyncd fails to load at boot

    that seems like a nice approach but what happens if the backup runs for more than an hour, where there's either a large amount of data to process and/or the hardware involved is old so data transfer bandwidth is a limiting factor? would that daemon restart kill the running backup process rendering it an incomplete backup or would the restart be denied while it's active with a task?

    flyingdutchman's approach is a nicer 'crude' hack

  13. #13
    Join Date
    Jan 2015
    Location
    Al Ain, UAE
    Posts
    815

    Re: Suddenly rsyncd fails to load at boot

    I like your solution to restart rsyncd from cron. I always like crude and simple hacks!

    Another crude way to fix it, is to rename the rsyncd file, then make a script by that name and in it, do a 'sleep 10', before calling the real rsyncd.
    --
    Have fun!
    http://www.aeronetworks.ca

  14. #14
    Join Date
    Feb 2009
    Location
    Island of Roatan in the Caribbean
    Posts
    281

    Re: Suddenly rsyncd fails to load at boot

    An rsync backup of the box takes about 2 minutes since all it's moving is what's changed. I don't know what the daemon would do if it was asked to restart while it's being used. I don't know if that's even the daemons call. In this case it's an issue that doesn't matter.

    I'm not worried about long running backup times.

    I don't like hacks at all. I'd like to understand why and how this is happening, but then I weigh the research time against my schedule and it comes up low priority, especially since F28 will be out within 90 days when I blow all the boxes away and start over.

  15. #15
    Join Date
    Feb 2009
    Location
    Florida
    Posts
    638

    Re: Suddenly rsyncd fails to load at boot

    I have been running a server using NetworkManger and offering services to my home network.

    Occasionally over the years I have had a similar situation where a service tries to start before the network is up. ( in my case "dnsmasq" )

    Adding "After=NetworkManager-wait-online.service" to the service definition has helped me. This is a systemd service which effectively signals that the network is online.

    Look in "/usr/lib/systemd/system/" to find the "rsyncd" service definition.
    In the "rsyncd.service" file you should see a "[Unit]" section with a line "After=network.target". This waits to start "rsyncd" until after the network is defined.

    Add the line "After=NetworkManager-wait-online.service" and the "rsyncd" service will not be started until the network is online.

    It may be that the startup of the wireless network takes enough time to affect its availability to "rsyncd".

    Support info:
    https://www.freedesktop.org/wiki/Sof...NetworkTarget/
    Last edited by Kobuck; 17th March 2018 at 03:33 PM.
    Laptop: Toshiba / Intel B960 2.20Ghz x2/ 4GB/ 320GB SataII/ Intel HD/ fc28.x86_64
    Tower: GigaByte (970A) / AMD FX 8320 3.5Ghz x8/ 16GB/ 9TB Sata III/ AMD 6770HD/ fc28.x86_64
    Bookshelf: Shuttle DS61 (H61)/ i3-3225 3.3Ghz x2/ 16GB/ 320GB Sata II/ Intel HD 4000/ fc27.x86_64
    Embedded: BeagleBone Blk / ARM AM3358 1 GHz x1/ 512MB/ 2GB eMMC/ PowerVR SGX530/ fc27.armv7hl

Page 1 of 2 1 2 LastLast

Similar Threads

  1. akmod fails to load at boot
    By iHorse in forum Using Fedora
    Replies: 4
    Last Post: 8th January 2012, 04:23 PM
  2. new to Linux eth0 fails to load at boot nic is rtl8139
    By Tgilyard in forum Servers & Networking
    Replies: 8
    Last Post: 30th December 2006, 11:00 AM
  3. Loopback fails to load on kernel boot
    By RaXeT in forum Using Fedora
    Replies: 2
    Last Post: 14th October 2005, 02:52 PM
  4. CUPS fails to load during boot.
    By LaVacaLoca in forum Using Fedora
    Replies: 4
    Last Post: 12th May 2005, 11:13 PM
  5. Network works at boot, then fails on load
    By fleece in forum Servers & Networking
    Replies: 1
    Last Post: 7th April 2005, 03:03 PM

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •