FedoraForum.org - Fedora Support Forums and Community
Results 1 to 15 of 15
  1. #1
    Join Date
    Sep 2016
    Location
    Ohio
    Posts
    11
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    F24 system kernel panicked on shutdown. Now it won't boot.

    Hello. I have a Fedora 24 system (x86_64) that recently kernel panicked on shutdown from the Gnome 3 desktop. I've been keeping up with updates as I'm notified, so I think it is up to date. I tried to restart it after the panic but it would not complete the boot up process, stopping just short of the blue f. Sometimes it will get to the blue f but never any further.

    I'm hoping to bring the system back to life before I break down and reinstall. It's been a good system since I think F16.

    If I press Esc during boot up, the last two messages that I can see are

    A start job is running for Wait for Plymouth Boot Screen to Quit.
    A start job is running for Builds and install new kmods from akmod packages. (I think this is the culprit, but I'm not positive.)

    I've tried letting the system run like this overnight but I only succeeded in making my power meter spin.

    I can reboot into run levels 1 and 3 but even using the system from the terminal is a chore. At run level 3 I get a login prompt and I can log in, but then I get a lot of

    /usr/bin/uname: fork: retry: No child processes
    /usr/bin/uname: fork: retry: Resource temporarily unavailable
    /usr/bin/uname: fork: Resource temporarily unavailable

    And it is difficult to use the terminal.

    From what I've read, these messages can sometimes suggest a problem with Nvidia graphics drivers but I'm on an AMD card and always have been. I've made no hardware changes recently.

    I'm at a loss now and I'm hoping someone here can help. Thank you!
    Last edited by ahaning; 3rd October 2016 at 06:38 AM. Reason: Problem solved.

  2. #2
    Join Date
    Mar 2012
    Location
    München, Deutschland
    Posts
    897
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Wink Re: F24 system kernel panicked on shutdown. Now it won't boot.

    aHanging,

    It is easy. When you go to next reboot, grub 2.0 will give you (in kernel boot screen) at least 3 kernels (latest 3) to choose from. Please, choose NOT the latest, just previous one.

    _nobody_

  3. #3
    Join Date
    Sep 2016
    Location
    Ohio
    Posts
    11
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: F24 system kernel panicked on shutdown. Now it won't boot.

    Thank you for the response, nobody. I should have mentioned this in the OP. I've tried the three kernels as well as the rescue kernel and all of them produce the same result, so I don't think the problem is necessarily with the kernel.

    The four kernels available are

    4.7.3-200.fc24.x86_64
    4.7.2-201.fc24.x86_64
    4.6.7-300.fc24.x86_64
    3.13.5-202.fc20.x86_64

    I'm also on grub 1.99. Not sure if that matters.
    And when I ctrl alt del to reboot from the stuck boot process, I get

    A stop job is running for Restore /run/initramfs on shutdown

    Which halts for the max 1 minute 30 seconds and then proceeds to reboot.

  4. #4
    Join Date
    Mar 2012
    Location
    München, Deutschland
    Posts
    897
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Talking Re: F24 system kernel panicked on shutdown. Now it won't boot.

    Quote Originally Posted by ahaning
    The four kernels available are:

    4.7.3-200.fc24.x86_64
    4.7.2-201.fc24.x86_64
    4.6.7-300.fc24.x86_64
    3.13.5-202.fc20.x86_64
    Until I (re)think more about this problem, there are few things you can do in the meantime:
    [1] #dnf upgrade (to upgrade to latest 4.7.4-200.fc24.x86_64)
    [2] Please, read the following and try to act upon: http://forums.fedoraforum.org/showpo...73&postcount=9
    [3] Please, find grub.cfg in your /boot directory, and from the kernel you are using remove the following parameters: rhgb quiet (to avoid splash/Plymouth screen).

    Please, report your findings here.

    Thank you,
    _nobody_

  5. #5
    Join Date
    Sep 2016
    Location
    Ohio
    Posts
    11
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: F24 system kernel panicked on shutdown. Now it won't boot.

    I tried your suggestions in reverse order. After [3], I get a login prompt and I can log in as root, but I continuously see the uname fork retry No child processes errors as reported in the OP.

    For [2], I checked my /etc/passwd and cannot see any duplicate UIDs.

    After logging in as root in run level 3, I tried running dnf but I get a traceback ending in

    ImportError: /lib64/libcurl.so.4:undefined symbol: libssh2_userauth_list

  6. #6
    Join Date
    Mar 2012
    Location
    München, Deutschland
    Posts
    897
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Talking Re: F24 system kernel panicked on shutdown. Now it won't boot.

    Quote Originally Posted by ahaning
    ImportError: /lib64/libcurl.so.4:undefined symbol: libssh2_userauth_list
    I called my best friend to help me in the matter: www.google.com
    https://www.google.de/search?sourcei...69i58.1129j0j8

    It seems that your F24 is missing some packages, some dependencies are (most likely) screwed.

    Please, check the following:
    https://bugzilla.redhat.com/show_bug.cgi?id=1140188
    http://forums.fedoraforum.org/archiv.../t-230730.html

    On my fedora 24:
    PHP Code:
    [root@localhost grub2]# dnf whatprovides libcurl.so.4
    Last metadata expiration check: 0:48:48 ago on Sun Sep 25 06:05:24 2016.
    libcurl
    -7.47.1-4.fc24.i686 : A library for getting files from web servers
    Repo        
    : fedora

    libcurl
    -7.47.1-8.fc24.i686 : A library for getting files from web servers
    Repo        
    : updates

    [root@localhost grub2]# 
    _nobody_

  7. #7
    Join Date
    Sep 2016
    Location
    Ohio
    Posts
    11
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: F24 system kernel panicked on shutdown. Now it won't boot.

    Well, I was able to boot the system with a F24 netinstall USB disk and use the Rescue a Fedora System option to make some changes to the system. When I get to the "rescue system will attempt to find your Linux installation", it tells me I don't have one so I have to skip to the shell and load it manually

    Code:
    # lvm vgscan -v
    # lvm vgchange -a y
    # lvm lvs -a
    # mount /dev/VolGroup/lv_root /mnt/sysimage
    # mount /dev/sda1 /mnt/sysimage/boot
    # for i in /dev /dev/pts /sys /proc /run; do mount -B $i /mnt/sysimage$i; done;
    # chroot /mnt/sysimage
    At this point I can access the root volume of my system and I did a dnf update to update many packages, but now it will generally hang before completing.

    No matter what I do, I cannot run like this for very long as my shell gets "taken over" with these messages over and over, making it difficult at best to even reboot back into the rescue environment

    Code:
    /usr/bin/uname: fork: retry: No child processes
    /usr/bin/uname: fork: retry: Resource temporarily unavailable
    /usr/bin/uname: fork: Resource temporarily unavailable
    I also was able to do an fsck on my root volume and had it make some corrections. The good news is that my home volume was clean.

    I wonder if there is a way to install packages (with dnf or rpm) without chrooting into the environment?

  8. #8
    Join Date
    Mar 2012
    Location
    München, Deutschland
    Posts
    897
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Arrow Re: F24 system kernel panicked on shutdown. Now it won't boot.

    Hello aHanging (there),

    Here is what I would do immediately being you (from my F24 VM):

    [root@localhost etc]# pwd (go to /etc directory)
    [root@localhost etc]# cat fstab (check what content you have in fstab file, and edit if necessary)
    [root@localhost etc]# dnf upgrade --refresh (to invalidate dnf local caches before upgrading)

    Here is what these 3 CLI Linux commands give on my F24 VM (on the top of VXWorkstation 12.5 VMM):

    PHP Code:
    [root@localhost etc]# pwd
    /etc

    [root@localhost etc]# cat fstab

    #
    # /etc/fstab
    # Created by anaconda on Wed Mar 16 11:59:42 2016
    #
    # Accessible filesystems, by reference, are maintained under '/dev/disk'
    # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
    #
    /dev/mapper/fedora-root /                       ext4    defaults        1 1
    UUID
    =8010ee03-159e-4250-bac1-26b484c95c84 /boot                   ext4    defaults        1 2
    /dev/mapper/fedora-swap swap                    swap    defaults        0 0

    [root@localhost etc]# dnf upgrade --refresh
    RPM Fusion for Fedora 24 - Nonfree - Updates                                                   416 kB/s | 110 kB     00:00    
    RPM Fusion 
    for Fedora 24 - Free                                                                787 kB/s | 326 kB     00:00    
    google
    -chrome                                                                                   47 kB/s | 3.8 kB     00:00    
    RPM Fusion 
    for Fedora 24 - Nonfree                                                             268 kB/s |  64 kB     00:00    
    RPM Fusion 
    for Fedora 24 - Free - Updates                                                      784 kB/s | 320 kB     00:00    
    Dependencies resolved
    .
    Nothing to do.
    Complete!
    [
    root@localhost etc]# 
    _nobody_
    Last edited by nobody; 27th September 2016 at 10:51 AM.

  9. #9
    Join Date
    Sep 2016
    Location
    Ohio
    Posts
    11
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: F24 system kernel panicked on shutdown. Now it won't boot.

    I tried these commands. In my fstab, I have basically the same as you. root, boot (listed with UUID), swap, and home volumes. I also had three RAID arrays listed that I've physically removed for now, so I commented them out.

    When I did dnf upgrade --refresh, it offers to remove two versions of kernel-core which seems to run fine, HDD chugging along, until the Running transaction stage when the HDD activity light stops blinking and no messages appear on the screen. When I press Ctrl-C, I see a couple of 'fork: retry: No child processes' messages and then

    Error in PREUN scriptlet in rpm package kernel-core

    At this point my rescue session is basically useless with almost any command I run producing 'bash: fork: retry: No child processes' and eventually 'Resource temporarily unavailable'.

    I'm very confused about these fork messages. They are making this very tiresome :-)

  10. #10
    Join Date
    Mar 2012
    Location
    München, Deutschland
    Posts
    897
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Exclamation Re: F24 system kernel panicked on shutdown. Now it won't boot.

    Hang in there, aHaning!

    My best friend google showed me lot of results for your: Error in PREIN scriptlet in rpm package kernel-core .

    Here is one very good read, it runs more indepth, but if you do understand it in nutshell, you will be able to work something out of this error.

    Really, I would like to experience similar error, so I could apply some of these written here (especially using strace command to trace forking child process): http://users.fedoraproject.narkive.c...n-of-fedora-23

    Basically, you need to run rpm -q --scripts kernel-core and take it from there.

    Please, let us know if this helps, and in which way, indeed. I'll also more in-depth analyze this pointer, to try to understand more!

    _nobody_

  11. #11
    Join Date
    Mar 2012
    Location
    München, Deutschland
    Posts
    897
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Arrow Re: F24 system kernel panicked on shutdown. Now it won't boot.

    Quote Originally Posted by nobody
    Please, let us know if this helps, and in which way, indeed. I'll also more in-depth analyze this pointer, to try to understand more!
    Yep, I did understand. It is all very clearly written here: https://access.redhat.com/documentat...mode-boot.html

    Now, I completely understood the following: http://users.fedoraproject.narkive.c...n-of-fedora-23

    In other words, what Andrej Podzimek meant by his replays to himself!

    Thank you, Andrej, U R really Linux Cool Cat!
    _nobody_

  12. #12
    Join Date
    Sep 2016
    Location
    Ohio
    Posts
    11
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: F24 system kernel panicked on shutdown. Now it won't boot.

    OK. I'm still confused.

    I ran rpm -q --scripts kernel-core and I can see the preinstall and preuninstall scripts that are run. I'm focusing on the preuninstall scripts as they are the ones causing my error. One of the preuninstall scripts for kernel-core runs uname -i in a conditional statement. If I run this manually from the shell, I get a stream of /usr/bin/uname: fork: retry: No child processes scrolling up my screen and my shell is unusable.

    I tried mounting /sys/fs/selinux before chrooting into the environment but no luck. Still streams of /usr/bin/uname: fork: retry: No child processes.

    My next guess is that I need to reinstall uname but I'm not sure what package provides it?

  13. #13
    Join Date
    Mar 2012
    Location
    München, Deutschland
    Posts
    897
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Wink Re: F24 system kernel panicked on shutdown. Now it won't boot.

    Hello aHaning,

    Here is how you'll do it (this is what I understood yesterday, and this is what Andrej really meant by his last post)!

    You need to do rescue boot of your system. Everything you have to do, you need to follow by this link: https://access.redhat.com/documentat...mode-boot.html

    You have to get to the following from this pointer:

    Once you have booted using one of the described methods, add the keyword rescue as a kernel parameter. For example, for an x86 system, type the following command at the installation boot prompt:

    linux rescue

    You are prompted to answer a few basic questions, including which language to use. It also prompts you to select where a valid rescue image is located. Select from Local CD-ROM, Hard Drive, NFS image, FTP, or HTTP. The location selected must contain a valid installation tree, and the installation tree must be for the same version of Red Hat Enterprise Linux as the Red Hat Enterprise Linux CD-ROM #1 from which you booted. If you used a boot CD-ROM or diskette to start rescue mode, the installation tree must be from the same tree from which the media was created. For more information about how to setup an installation tree on a hard drive, NFS server, FTP server, or HTTP server, refer to the Red Hat Enterprise Linux Installation Guide.

    If you select a rescue image that does not require a network connect, you are asked whether or not you want to establish a network connection. A network connection is useful if you need to backup files to a different computer or install some RPM packages from a shared network location, for example.
    Once you get Linux rescue system, your REAL root directory will be here:

    The rescue environment will now attempt to find your Red Hat Linux installation and mount it under the directory /mnt/sysimage. You can then make any changes required to your system.
    You have to read very carefully THE WHOLE PAGE: 11.2. Booting into Rescue Mode -> https://access.redhat.com/documentat...mode-boot.html

    Now, when you'll have this achieved, you need to do the following (per Andrej's sentence):
    Short answer: First, you must "mount -o bind /sys/fs/selinux /mnt/sysimage/sys/fs/selinux". Second, also mount /mnt/sysimage/proc *before* attempting to install 'filesystem'. That's it. That's all.

    In other words, you need to have achieved the following (in linux rescue):
    [1] To have your original root system mounted under /mnt/sysimage (cd-ing into it you should see your own /)
    [2] You also need to do: mount -o bind /sys/fs/selinux /mnt/sysimage/sys/fs/selinux
    [3] Then: mount -o bind /proc /mnt/sysimage/proc
    [4] # chroot /mnt/sysimage
    [5] Once you have your / tree, do again # dnf upgrade!

    I really hope my "translation" helps!

    _nobody_

  14. #14
    Join Date
    Sep 2016
    Location
    Ohio
    Posts
    11
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: F24 system kernel panicked on shutdown. Now it won't boot.

    nobody,

    Thank you for your patient assistance. I'm typing this message from my previously dead F24 system. (Previous messages were from my iPad 2 and my 7 year old eeepc; both painful systems to use compared to my i5 desktop!)

    After using your and Andrej's instructions, I kept getting errors from uname. Eventually, I less'd /usr/bin/uname and noticed it was a shell script while the rescue system's uname was a binary. Odd. A bit of googling pointed me to the coreutils package, so I did dnf reinstall coreutils and bingo! No more fork: retry, blah blah, errors like before.

    That was only the beginning, but the rest went quickly. Every so often when reinstalling a package, I'd get a bunch of errors from ldconfig that a file wasn't an ELF executable (I'm not remembering the exact error at the moment). dnf reinstall *package* was the trick I used to find and reinstall the necessary libraries and packages.

    Thanks again! I'm going to try rebooting now and see if I come back. Fingers crossed!

  15. #15
    Join Date
    Mar 2012
    Location
    München, Deutschland
    Posts
    897
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Thumbs up Re: F24 system kernel panicked on shutdown. Now it won't boot.

    Quote Originally Posted by ahaning
    Thanks again! I'm going to try rebooting now and see if I come back. Fingers crossed!
    ahaning, good news, actually excellent, I should say! Fingers crossed!

    Thank you for being patient. Do not forget, helping/assisting to you, I am also learning a lot. Indeed!

    _nobody_

Similar Threads

  1. Kernel upgrade broke system - kernel panic and unable to boot
    By itsjustarumour in forum Using Fedora
    Replies: 13
    Last Post: 29th May 2012, 06:35 PM
  2. Replies: 3
    Last Post: 25th August 2009, 08:46 PM
  3. Replies: 6
    Last Post: 9th October 2008, 04:34 PM
  4. Kernel Panic on boot system
    By kacperix in forum Using Fedora
    Replies: 13
    Last Post: 23rd January 2007, 04:00 PM
  5. Kernel Panic During System Boot
    By Besnell in forum Using Fedora
    Replies: 1
    Last Post: 12th September 2005, 09:58 AM

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •