EXT4 Data Corruption Bug Hits Stable Linux Kernels

[Cygn]

from phoronix:

http://www.phoronix.com/scan.php?pag...tem&px=MTIxNDQ

[glennzo]

Your link doesn't work. I got to that page with this: http://www.phoronix.com/scan.php?pag...tem&px=MTIxNDQ

[bigflopper2]

"Well, the problem won't show up if the journal has wrapped. So it will only show up if the system has been rebooted twice in fairly quick succession."

Just don't reboot twice.

[mmix]

http://www.phoronix.com/scan.php?pag...tem&px=MTIxNDQ

As a warning for those who are normally quick to upgrade to the latest stable vanilla kernel releases, a serious EXT4 data corruption bug worked its way into the stable Linux 3.4, 3.5, and 3.6 kernel series.

Being discussed recently on the Linux kernel mailing list was an "apparent serious progressive ext4 data corruption bug in 3.6.3." Theodore Ts'o was able to successfully bisect the kernel and found the serious bug, which first appeared within the Linux 3.6.2 kernel and was since back-ported to older stable kernels.

[glennzo]

Threads merged.

Originally Posted by glennzo

Threads merged.

Can you move this lame thread to linux chat as it's not fedora related.

[dsmithhfx]

Originally Posted by Yellowman

Can you move this lame thread to linux chat as it's not fedora related.

Are you saying that no Fedora distributions are affected by this bug?

[Dan]

It may effect all linux distros using those kernels. Moved to Linux Chat.

[stoat]

To me, it's worth paying attention to and maybe doing something about it. My understanding of the situation is that the affected versions are the last two of v3.4, the last one of v3.5, and last two of v3.6. In other words, it appears to me, any kernel or patch since October 12.

http://lwn.net/Articles/521022/

Like so at kernel.org...

linux-3.4.14.tar.bz2     12-Oct-2012 21:12   76M  
linux-3.4.14.tar.gz      12-Oct-2012 21:12   96M  
linux-3.4.14.tar.sign    12-Oct-2012 21:12  836   
linux-3.4.14.tar.xz      12-Oct-2012 21:12   64M  

linux-3.4.15.tar.bz2     21-Oct-2012 16:36   76M  
linux-3.4.15.tar.gz      21-Oct-2012 16:36   96M  
linux-3.4.15.tar.sign    21-Oct-2012 16:36  836   
linux-3.4.15.tar.xz      21-Oct-2012 16:36   64M  

linux-3.5.7.tar.bz2      12-Oct-2012 21:12   77M  
linux-3.5.7.tar.gz       12-Oct-2012 21:12   98M  
linux-3.5.7.tar.sign     12-Oct-2012 21:12  836   
linux-3.5.7.tar.xz       12-Oct-2012 21:12   64M  

linux-3.6.2.tar.bz2      12-Oct-2012 21:12   78M  
linux-3.6.2.tar.gz       12-Oct-2012 21:12   99M  
linux-3.6.2.tar.sign     12-Oct-2012 21:12  836   
linux-3.6.2.tar.xz       12-Oct-2012 21:12   65M  

linux-3.6.3.tar.bz2      21-Oct-2012 16:42   78M  
linux-3.6.3.tar.gz       21-Oct-2012 16:42   99M  
linux-3.6.3.tar.sign     21-Oct-2012 16:42  836   
linux-3.6.3.tar.xz       21-Oct-2012 16:42   65M

I had these three installed in Fedora 17...

Sep 30 23:38:44 Installed: kernel-PAE-3.5.4-2.fc17.i686
Oct 13 13:00:45 Installed: kernel-PAE-3.6.1-1.fc17.i686
Oct 20 11:56:08 Installed: kernel-PAE-3.6.2-4.fc17.i686

Everybody should do what they think is best. The scenario required to cause the issue sounds sort of unlikely. But I often do need to do a bunch of reboots. For me, I decided to go back to v3.5.4-2, remove the other two, and exclude kernels from updates for the time being. It's a completely trivial thing to do. And it's not the first time either.

[Dan]

Gee, thaaaat's neat. <....>

If I understand all that correctly, these are the kernels with the cooked books.

... users would be well advised to avoid versions 3.4.14, 3.4.15, 3.5.7, 3.6.2, and 3.6.3; they all contain a patch which can, in some situations, cause filesystem corruption.

Which means, if it is indeed restricted to just those kernels ... the ext4 file systems my new laptop SSDs are likely pooched. It's had 3.6.2-4.fc17.x86_64 in it for a week or so, and it's been re-booted a dozen or so times. (With the boot speed of the SSD, I usually shutdown over night rather than suspend on that one.)

If my assumptions are correct, this desktop is running 3.4.11-1.fc16.x86_64, so with luck, it isn't crackerjacked as well. It's been re-booted maybe a half dozen times in the last month. But it's ext4 as well.

Are my assumptions completely erroneous?

[BBQdave]

Originally Posted by Dan

the ext4 file systems my new laptop SSDs are likely pooched. It's had 3.6.2-4.fc17.x86_64 in it for a week or so, and it's been re-booted a dozen or so times.

Dan I am running F17 with Kernel Linux 3.6.2-4.fc17.x86_64 on Toshiba Satellite C655. I shut down and reboot several times daily, to conserve battery power - as I am on the go. I have not had any issues or data corruption that I am aware of. And I am pushing data on the web and on my network without issue. My work on this notebook is sent and received by other hardware - and works on other hardware.

Is there something I am missing, or should be concerned about - though there are no apparent issues with my distro and machine? Yellowman are you saying this is rare and/or no-concern?

[bigflopper2]

First of all, it appears that my initial analysis was wrong; it appears that it's probably related to a unclean shutdown, and probably some other required exacerbating factor, since I haven't been able to trivially reproduce it. Secondly, Fedora 17 is currently using 3.6.2. So if this was an easily triggered bug, (a) lots of people would have been complaining, since the results of this bug is not subtle, and (b) Eric and I probably would have found a reliable repro for the problem by now.

Update: It now looks like the reproduction involved something very esoteric indeed, involving using umount -l and shutdowns while the file system was still being unmounted --- and the user had nobarrier specified in the mount options as well.

https://plus.google.com/117091380454...ts/Wcc5tMiCgq7

[dsmithhfx]

Originally Posted by Dan

the ext4 file systems my new laptop SSDs are likely pooched.

Not necessarily, but to be perfectly safe you might consider booting from a live cd and running fsck on potentially affected file systems first, and then booting into a different kernel.

Bollocks to their over hyped claims of doom

[leigh@main-pc F-17]$ uname -r
3.6.3-1.fc17.x86_64
[leigh@main-pc F-17]$

For the record I've been running 3.6.2 + since kernel.org released it and haven't seen any corruption.

[Dan]

Originally Posted by BBQDave

Is there something I am missing, or should be concerned about - though there are no apparent issues with my distro and machine? Yellowman are you saying this is rare and/or no-concern?

As I understand it, the issue won't become a problem unless a remote file system is removed without being unmounted first. And that wouldn't normally be a problem, except I frequently shuttle files between myself and the War Department, and she's got a nasty habit of suspending without telling me ... which wouldn't be a problem, except I sometimes forget to log out/unmount from her machine when I'm finished shoving electrons back and forth.

Originally Posted by dsmithhfx

Not necessarily, but to be perfectly safe you might consider booting from a live cd and running fsck on potentially affected file systems first, and then booting into a different kernel.

That's on this morning's agenda ... along with another sticky note reminding me to log out of her box whenever I've finished moving stuff. <....>