PDA

View Full Version : Boot goes to emergency mode due to....



Gh0zt36
24th June 2014, 06:57 PM
a file expected containing a certain amount of data being empty . or something I couldnt screenshot caus OS wasnt booted . I ran journalctl -XB and theres thousands of line of code for the log but a couple that came upin the red were in order of appearance on the log


The start-up result is done.
Jun 24 13:41:13 localhost.localdomain pvscan[1085]: 3 logical volume(s) in volume group "fedora" now active
Jun 24 13:41:14 localhost.localdomain kernel: sd 6:0:0:0: [sdb] Test WP failed, assume Write Enabled
Jun 24 13:41:14 localhost.localdomain kernel: sd 6:0:0:0: [sdb] Asking for cache data failed
Jun 24 13:41:14 localhost.localdomain kernel: sd 6:0:0:0: [sdb] Assuming drive cache: write through
Jun 24 13:41:21 localhost.localdomain systemd[1]: Found device /dev/mapper/fedora-home.
-- Subject: Unit dev-mapper-fedora\x2dhome.device has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit dev-mapper-fedora\x2dhome.device has finished starting up.
--
-- The start-up result is done.
Jun 24 13:41:21 localhost.localdomain systemd[1]: Started LVM2 PV scan on device 253:0.
-- Subject: Unit lvm2-pvscan@253:0.service has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit lvm2-pvscan@253:0.service has finished starting up.
--
-- The start-up result is done.
Jun 24 13:41:22 localhost.localdomain systemd-fsck[1005]: /dev/mapper/fedora-root: Inodes that were part of a corrupted orphan linked list found.
Jun 24 13:41:22 localhost.localdomain systemd-fsck[1005]: /dev/mapper/fedora-root: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
Jun 24 13:41:22 localhost.localdomain systemd-fsck[1005]: (i.e., without -a or -p options)
Jun 24 13:41:23 localhost.localdomain systemd-fsck[1005]: fsck failed with error code 4.
Jun 24 13:41:23 localhost.localdomain systemd-fsck[1005]: Running request emergency.target/start/replace
Jun 24 13:41:23 localhost.localdomain systemd[1]: Stopped Daily Cleanup of Temporary Directories.
-- Subject: Unit systemd-tmpfiles-clean.timer has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel



]Jun 24 13:41:49 localhost.localdomain systemd[1]: Unit rngd.service entered failed state.
Jun 24 13:41:49 localhost.localdomain rpcbind[1243]: warning: /etc/hosts.deny, line 1: missing ":" separator

Jun 24 13:42:56 localhost.localdomain kernel: sd 6:0:0:0: [sdb] Test WP failed, assume Write Enabled
Jun 24 13:42:56 localhost.localdomain kernel: sd 6:0:0:0: [sdb] Asking for cache data failed
Jun 24 13:42:56 localhost.localdomain kernel: sd 6:0:0:0: [sdb] Assuming drive cache: write through
Jun 24 13:43:07 localhost.localdomain PackageKit[2140]: get-updates transaction /2397_abeaebca from uid 1000 finished with success after 12314ms
Jun 24 13:43:26 localhost.localdomain NetworkManager[1335]: <warn> nl_recvmsgs() error: (-33) Dump inconsistency detected, interrupted
Jun 24 13:43:48 localhost.localdomain kernel: sd 6:0:0:0: [sdb] Test WP failed, assume Write Enabled
Jun 24 13:43:48 localhost.localdomain kernel: sd 6:0:0:0: [sdb] Asking for cache data failed
Jun 24 13:43:48 localhost.localdomain kernel: sd 6:0:0:0: [sdb] Assuming drive cache: write through
Jun 24 13:44:40 localhost.localdomain kernel: sd 6:0:0:0: [sdb] Test WP failed, assume Write Enabled
Jun 24 13:44:40 localhost.localdomain kernel: sd 6:0:0:0: [sdb] Asking for cache data failed
Jun 24 13:44:40 localhost.localdomain kernel: sd 6:0:0:0: [sdb] Assuming drive cache: write through
Jun 24 13:45:31 localhost.localdomain kernel: sd 6:0:0:0: [sdb] Test WP failed, assume Write Enabled
Jun 24 13:45:31 localhost.localdomain kernel: sd 6:0:0:0: [sdb] Asking for cache data failed
Jun 24 13:45:31 localhost.localdomain kernel: sd 6:0:0:0: [sdb] Assuming drive cache: write through
Jun 24 13:45:58 localhost.localdomain kernel: EXT4-fs (dm-2): error count: 3
Jun 24 13:45:58 localhost.localdomain kernel: EXT4-fs (dm-2): initial error at 1401730272: ext4_journal_check_start:56
Jun 24 13:45:58 localhost.localdomain kernel: EXT4-fs (dm-2): last error at 1401730272: ext4_journal_check_start:56
Jun 24 13:46:23 localhost.localdomain kernel: sd 6:0:0:0: [sdb] Test WP failed, assume Write Enabled
Jun 24 13:46:23 localhost.localdomain kernel: sd 6:0:0:0: [sdb] Asking for cache data failed
Jun 24 13:46:23 localhost.localdomain kernel: sd 6:0:0:0: [sdb] Assuming drive cache: write through
Jun 24 13:47:15 localhost.localdomain kernel: sd 6:0:0:0: [sdb] Test WP failed, assume Write Enabled
Jun 24 13:47:15 localhost.localdomain kernel: sd 6:0:0:0: [sdb] Asking for cache data failed
Jun 24 13:47:15 localhost.localdomain kernel: sd 6:0:0:0: [sdb] Assuming drive cache: write through
lines 3745-3798/3798 (END)






ctrl d resumes boot and system starts .

Any ideas ?

Skull One
24th June 2014, 09:10 PM
Jun 24 13:41:22 localhost.localdomain systemd-fsck[1005]: /dev/mapper/fedora-root: Inodes that were part of a corrupted orphan linked list found.
Jun 24 13:41:22 localhost.localdomain systemd-fsck[1005]: /dev/mapper/fedora-root: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
Jun 24 13:41:22 localhost.localdomain systemd-fsck[1005]: (i.e., without -a or -p options)
Jun 24 13:41:23 localhost.localdomain systemd-fsck[1005]: fsck failed with error code 4.

The root filesystem is damaged and requires a manual intervention. For instance, in the emergency mode you can try:


fsck -c -p /dev/mapper/fedora-root

Gh0zt36
24th June 2014, 09:35 PM
The root filesystem is damaged and requires a manual intervention. For instance, in the emergency mode you can try:


fsck -c -p /dev/mapper/fedora-root


Thank you for the reply . What do you think could cause this?

Gh0zt36
24th June 2014, 09:52 PM
Ok from emergency mode I entered root password to try the fsck -c -p and it said it couldnt proceed cause the disk was mouted . Should I be doing this from a liveCD ???

nsnbm
25th June 2014, 02:25 AM
Gh0zt36 asked:


Should I be doing this from a liveCD ?

Yes ... it's worked for me.

Skull One
25th June 2014, 08:13 AM
A live CD is fine. Otherwise, you can add to the grub command line:


rd.break=pre-mount

to drop in the emergency shell before the mount.

Gh0zt36
25th June 2014, 05:07 PM
A live CD is fine. Otherwise, you can add to the grub command line:


rd.break=pre-mount

to drop in the emergency shell before the mount.

mmmm now you are getting beyond my talent here. How would I go about adding to the grub command line ? And that wouldnt fix the root partition would it ? just not goto emergency shell ? Id rather fix this if possible.

Im on a livecd now and it doesn't recognize /DEV/MAPPER/FEDORA-ROOT AS an existing partition. Or its not reading the HDD ? I donno

Skull One
25th June 2014, 05:20 PM
mmmm now you are getting beyond my talent here. How would I go about adding to the grub command line ? And that wouldnt fix the root partition would it ? just not goto emergency shell ? Id rather fix this if possible.

The idea here is to go in the emergency shell _before_ the mount to fix the filesystem. It is useful when you do not have a livecd, for instance.



Im on a livecd now and it doesn't recognize /DEV/MAPPER/FEDORA-ROOT AS an existing partition. Or its not reading the HDD ? I donno
Maybe lvm was not set. Try:


lvm vgscan
lvm vgchange -ay

Gh0zt36
25th June 2014, 05:27 PM
The idea here is to go in the emergency shell _before_ the mount to fix the filesystem. It is useful when you do not have a livecd, for instance.


Maybe lvm was not set. Try:


lvm vgscan
lvm vgchange -ay


crap lol I wasnt expecting such a quick reply so I got off live cd . Ill put it back in in a sec. and try that

Gh0zt36
25th June 2014, 06:32 PM
[root@localhost liveuser]# lvm vgchange -ay
No volume groups found

[root@localhost liveuser]# lvm vgscan
Reading all physical volumes. This may take a while...
No volume groups found



This is what I got . I dunno

Skull One
25th June 2014, 06:42 PM
Huh? What is the output of:


ls /dev/sda*
mount |grep dev

Gh0zt36
25th June 2014, 06:46 PM
Huh? What is the output of:


ls /dev/sda*
/dev/sda /dev/sda1 /dev/sda2

mount |grep dev
devtmpfs on /dev type devtmpfs (rw,nosuid,seclabel,size=1940264k,nr_inodes=485066 ,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,relatime,seclabel)
devpts on /dev/pts type devpts (rw,relatime,seclabel,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,seclabel,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (rw,nosuid,nodev,noexec,seclabel,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_age nt=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/net_cls type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
/dev/sr0 on /run/initramfs/live type iso9660 (ro,relatime)
/dev/mapper/live-rw on / type ext4 (rw,noatime,seclabel,data=ordered)
mqueue on /dev/mqueue type mqueue (rw,relatime,seclabel)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,seclabel)





This here is what I got. It's like its completely ignoring the HDD and only recognizing the livecd as a legit partition.

Skull One
25th June 2014, 06:52 PM
And what is the output of:


blkid /dev/sda*

Gh0zt36
25th June 2014, 06:53 PM
/dev/sr0: UUID="2013-12-12-02-27-19-00" LABEL="Fedora-Live-KDE-x86_64-20-1" TYPE="iso9660" PTUUID="19288ddb" PTTYPE="dos"
/dev/sda1: UUID="4868d58c-2cca-4995-a990-3a12f02af61d" TYPE="ext4" PARTUUID="000d1193-01"
/dev/sda2: UUID="b663d93b-ca90-4e5a-a3b6-aea8a8195f78" TYPE="crypto_LUKS" PARTUUID="000d1193-02"
/dev/loop0: TYPE="squashfs"
/dev/loop1: TYPE="DM_snapshot_cow"
/dev/loop2: TYPE="squashfs"
/dev/loop3: LABEL="_Fedora-Live-KDE" UUID="f55efb5e-3b29-4bae-8684-87cd5b0eae75" TYPE="ext4"
/dev/loop4: TYPE="DM_snapshot_cow"
/dev/mapper/live-rw: LABEL="_Fedora-Live-KDE" UUID="f55efb5e-3b29-4bae-8684-87cd5b0eae75" TYPE="ext4"
/dev/mapper/live-base: LABEL="_Fedora-Live-KDE" UUID="f55efb5e-3b29-4bae-8684-87cd5b0eae75" TYPE="ext4"
/dev/mapper/live-osimg-min: LABEL="_Fedora-Live-KDE" UUID="f55efb5e-3b29-4bae-8684-87cd5b0eae75" TYPE="ext4"

This is the output of blkid . See ? /devmapper/fedora-root doesnt even show up . Funny enough it shows my encrypted home folder? that what /dev/sda2 is right ? I encrypted @ install.

Strange it shows that but not root

Skull One
25th June 2014, 07:10 PM
There is no lvm? Great then! The root partition should be /dev/sda1.
Use directly;


fsck -c /dev/sda1

Gh0zt36
25th June 2014, 07:14 PM
There is no lvm? Great then! The root partition should be /dev/sda1.
Use directly;


fsck -c /dev/sda1

output of

e2fsck 1.42.8 (20-Jun-2013)
Checking for bad blocks (read-only test): 0.00% done, 0:00 elapsed. (0/0/0 errdone
/dev/sda1: Updating bad block inode.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information

/dev/sda1: ***** FILE SYSTEM WAS MODIFIED *****
/dev/sda1: 385/128016 files (1.8% non-contiguous), 174427/512000 blocks
[root@localhost /]#


What exactly is lvm and is it good there isnt one ? and why isnt /sda1 labeled as root ?

Skull One
25th June 2014, 07:17 PM
I let you read: https://wiki.archlinux.org/index.php/LVM
In a nutshell, it is a powerfull tool but it makes simple things a little more complicated. A standard user don't really need it.

The filesystem should be clean now. Sometimes I run a second pass:


fsck -cp /dev/sda1

Gh0zt36
25th June 2014, 07:21 PM
I let you read: https://wiki.archlinux.org/index.php/LVM
In a nutshell, it is a powerfull tool th

i haz no powerful tool :(


On the bright side looks like the rogue inode was re routed .

Gh0zt36
25th June 2014, 07:31 PM
Well thank you again for all the help. I would be stuck as a perpetual noob without the amazing fedora community and I learned quite a bit today

Thanks skully

Gh0zt36
25th June 2014, 07:36 PM
ughh still booting into emergency shell :/

lemme re journalctl


same thing :/



Jun 25 14:33:43 localhost.localdomain systemd-fsck[641]: /dev/mapper/fedora-root: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
Jun 25 14:33:43 localhost.localdomain systemd-fsck[641]: (i.e., without -a or -p options)
Jun 25 14:33:43 localhost.localdomain systemd-fsck[641]: fsck failed with error code 4.
Jun 25 14:33:43 localhost.localdomain systemd-fsck[641]: Running request emergency.target/start/replace

Skull One
25th June 2014, 07:53 PM
Damn!
Ok, this time try without the livecd.
When the grub starts, press "e" on the kernel line to edit it.
Scroll down to find the line starting by 'linux', and append 'rd.break=pre-mount'
Press ctrl-X to boot. You'll get the emergency shell. Run fcsk on /dev/mapper/fedora-root this time.

Gh0zt36
25th June 2014, 07:58 PM
Damn!
Ok, this time try without the livecd.
When the grub starts, press "e" on the kernel line to edit it.
Scroll down to find the line starting by 'linux', and append 'rd.break=pre-mount'
Press ctrl-X to boot. You'll get the emergency shell. Run fcsk on /dev/mapper/fedora-root this time.

ok , This is getting into area im totally unfamiliar with . where is the kernel line ? I boot , goto bios pre boot ..... then select instance of fedora to run . then goes to my encryption field put in pass word then emergency shell .

Where in this process do I hit e ?

Skull One
25th June 2014, 08:12 PM
ok , This is getting into area im totally unfamiliar with . where is the kernel line ? I boot , goto bios pre boot ..... then select instance of fedora to run . then goes to my encryption field put in pass word then emergency shell .

Where in this process do I hit e ?

Hum... it is probably this one, where you choose the kernel version. It should be like these screenshots https://ask.fedoraproject.org/en/question/23949/i-need-to-change-grub2s-appearance/

Instead of pressing 'enter' or let the counter timeout, press 'e'.
You will have something like this http://bcl.fedorapeople.org/grub2/grub2-duplicate.png

Gh0zt36
25th June 2014, 08:25 PM
Ok so to recap make sure I get it right .

boot hit e at grub menu .

find line beginning with linux

add rd.break=pre-mount to the end of the line

reboot and then from there do I goto maintenance / cmd line to do fsck /dev/mapper/fedora-root ?

Sound bout right ?

Skull One
25th June 2014, 08:31 PM
Ok so to recap make sure I get it right .

boot hit e at grub menu .

find line beginning with linux

add rd.break=pre-mount to the end of the line

reboot and then from there do I goto maintenance / cmd line to do fsck /dev/mapper/fedora-root ?

Sound bout right ?

It is not reboot! It is Ctrl-X to boot! It should be written on the screen.
Use the options -c and -p with fsck.

Gh0zt36
25th June 2014, 08:39 PM
It is not reboot! It is Ctrl-X to boot! It should be written on the screen.
Use the options -c and -p with fsck.

right on . I got ya . fsck -c -p /dev/mapper/fedora-root after ctrl x ... Ill report back how it went .

Skull One
25th June 2014, 08:48 PM
I have to leave for a moment. I'm crossing fingers for you :).

Gh0zt36
25th June 2014, 08:53 PM
I have to leave for a moment. I'm crossing fingers for you :).

thanks bud lol Uhmm heres what i got . Then it just stalled at cache write through failed . after couple lines of that i ctrl C out of there cause it didnt appear to progress

http://i.imgur.com/oydm0Cr.jpg

nsnbm
26th June 2014, 12:22 AM
I haven't had this particular problem, but I am beginning to wonder whether there is some issue with the hard disk itself ... and thanks Skull One for that nifty grub option.

Skull One
26th June 2014, 07:52 AM
thanks bud lol Uhmm heres what i got . Then it just stalled at cache write through failed . after couple lines of that i ctrl C out of there cause it didnt appear to progress

http://i.imgur.com/oydm0Cr.jpg

Never ctrl-c when repairing a filesystem! You can make things worse!
But it is not the case here, the command reported an error and stopped, asking you to use it without the -p option.
The other lines were just some kernel messages: they are printed on the terminal, so it can confuse you when you are not aware of that. If you have pressed 'enter', you would have seen the prompt again.

So, do the same thing without the -p opton!
What annoys me is that badblocks is not included in the emergency environment, I don't know why.
Let's hope there is not any on the partition!

Gh0zt36
26th June 2014, 09:09 AM
Never ctrl-c when repairing a filesystem! You can make things worse!
But it is not the case here, the command reported an error and stopped, asking you to use it without the -p option.
The other lines were just some kernel messages: they are printed on the terminal, so it can confuse you when you are not aware of that. If you have pressed 'enter', you would have seen the prompt again.

So, do the same thing without the -p opton!
What annoys me is that badblocks is not included in the emergency environment, I don't know why.
Let's hope there is not any on the partition!

SUCCESS ! Skull you are a scholar and a gentleman. I ran fsck /dev/mapper/fedora-root with no attributes and it fixed the problem .


Thank you sir. And thank you Fedoraforums . As I said before without the wealth of knowledge and the spirit to help your fellow man I would be a perpetual n00b!! :dance:

Skull One
26th June 2014, 10:32 AM
Very good!
Now, what I want to know is what failed with the livecd: why the root partition was identified as a standard one whereas il looks like a lvm one on the system. I will be really grateful to anybody shedding some light here.

JordanWells
28th December 2016, 05:36 AM
When you rebooted the system, was your external media left connected? I disconnected it and I didn't get the Emergency mode. You might want to try that?

dreamcarrior
3rd May 2017, 10:03 PM
Thank you very much!

fsck -c -p /dev/mapper/fedora-root says says that the partition has inconsistent inodes

I then run

fsck -c -f /dev/mapper/fedora-root to get the partition fixed.

After reboot, the system is back to normal.

I am running Fedora 25.


The root filesystem is damaged and requires a manual intervention. For instance, in the emergency mode you can try:


fsck -c -p /dev/mapper/fedora-root