Ext4 and 32.1-9

[SlowJet]

Regression or simulated ~ barriers?

On a Pent Iv 2.4 F12 with F13 kernel, LVM, rpm, and a few other required packages;.

Coping large 4 vdi (22.5GB) from /home//me/.Viruatlbox/HardDisks to another lv on sdb1
the speed has been around 27-29MB/s, down from the F12 updated release of 56-59MB/s

Tar of LV snapshots have not changed at all. Very slow process so disk speed is not taxed. About 5.6MB/s for root(less boot and home).

Under normal Desktop activity, the speed does not seem to be a factor.
Several batches of RPM's from 20 to 90 did not seem to be any different in spped.
Yum updates were restricted by download speed more than updating, speed seem normal.

tar: /mnt/snaproot/tmp/seahorse-ZyFMMm/S.gpg-agent: socket ignored
Total bytes written: 4557406208 (4.3GiB, 5.6MiB/s)

[root@Ruthie-07 bkup]# mount /dev/mapper/VolGroup71-snaphome /mnt/snaphome
[root@Ruthie-07 bkup]# tar -c -f bkuphome711214.tar /mnt/snaphome -P --preserve-permission --preserve-order --one-file-system --xattrs --totals -b32 --exclude=*.iso --exclude=*.vdi --exclude=/home/darwinhwebb/Pictures/xmas2005/*

Total bytes written: 731103232 (698MiB, 15MiB/s)

[root@Ruthie-07 bkup]# tar -c -f bkupboot711214.tar /boot -P --preserve-permission --preserve-order --one-file-system --xattrs --totals -b32 --exclude=*.iso --exclude=*.vdi

Total bytes written: 54247424 (52MiB, 37MiB/s)


SJ

[rsingh]

Its a feature but causes performance problems, I have written a short article on my blog about it:

http://stratazure.blogspot.com/2009/...egression.html

[SlowJet]

No, no.
Barriers is set by default, but the new modern drives are write cashed by default (and possibly LVM is can not be barrierd).
Therefore almost no F11/F13 default fedora has barriers actually on.
But the real code in ext 4 is not yet complete for barriers even if the write cache on the disk drive is turned off.

The point here is, a 50% drop in large file copy is a lot of debugging turned on for the stable released Kernel or something else has occurred at the kernel level.
The slow down occurred before 32.1-9, I was expecting things to fly, rather than flop, since the 32.1-9 had even more ext4 fixes (unless those fixes were some of the new tricks to make barriers work, like checksum journal, checksum metadata?)
i.e. ~ barriers but not all cases. But only the coders know for sure.
So if this is the real state of ext4, then turning off the write cache would be a good move as it is the only barrier to full barriers protection and having it on does nothing for write speed.
But only the coders know for sure.

SJ

[rsingh]

Originally Posted by SlowJet

No, no.
Barriers is set by default, but the new modern drives are write cashed by default (and possibly LVM is can not be barrierd).
Therefore almost no F11/F13 default fedora has barriers actually on.
But the real code in ext 4 is not yet complete for barriers even if the write cache on the disk drive is turned off.

The point here is, a 50% drop in large file copy is a lot of debugging turned on for the stable released Kernel or something else has occurred at the kernel level.
The slow down occurred before 32.1-9, I was expecting things to fly, rather than flop, since the 32.1-9 had even more ext4 fixes (unless those fixes were some of the new tricks to make barriers work, like checksum journal, checksum metadata?)
i.e. ~ barriers but not all cases. But only the coders know for sure.
So if this is the real state of ext4, then turning off the write cache would be a good move as it is the only barrier to full barriers protection and having it on does nothing for write speed.
But only the coders know for sure.

SJ

Are you sure barriers are not on by default??

I say this because when I do this on a default installation of F12:

dmesg | grep barriers

I get:

EXT4-fs (sda6): barriers enabled
EXT4-fs (sda5): barriers enabled
EXT4-fs (sda3): barriers enabled

and apps like liferea that use sqlite as their backend are EXTREMELY slow. Alternatively when I apply barrier=0 and then use liferea, such lag is no more visible and sqlite operation is smooth again.

[SlowJet]

I just truned my write cashe off on the sdb and copied 17.5 GB (4 vdi's) from sda and got 25.8 to 26.3 MB/s, a little less than the 32.1-9 with ext4 fixes.

So I think the ext4 code fixes approaches full barriers, but not all cases, so the write cache needs to be turned off (to prevent out of order writes from the cache).

I truned off sda (the /boot and system, home LV's with ext4).
I copied a folder of 19 OOo rpm's to another folder on the /home LV ext4 of 134MB. It did not seem any slower than usual.

Yes, barriers are turned on by default, but as I indicated in the previous thread, they are not used when conditions are detected that prevent them from occurring.
In conclusion, ext4 with 32.1-9 comes closer to barrier state, but barriers are not actually used unless the write cache is off.

How-To: (Some Distos turn write cache on at boot up with hard to find scripts. I could not find any in Fedora for hdparm but I haven;t rebooted yet.

# hdparm -i /dev/sda

/dev/sda:

Model=ST3500641A, FwRev=3.AAE, SerialNo=3PM0DBT1
Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs RotSpdTol>.5% }
RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=4
BuffType=unknown, BuffSize=16384kB, MaxMultSect=16, MultSect=16
CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=976773168
IORDY=on/off, tPIO={min:240,w/IORDY:120}, tDMA={min:120,rec:120}
PIO modes: pio0 pio1 pio2 pio3 pio4
DMA modes: mdma0 mdma1 mdma2
UDMA modes: udma0 udma1 udma2 udma3 udma4 *udma5
AdvancedPM=no WriteCache=enabled
Drive conforms to: Unspecified: ATA/ATAPI-1,2,3,4,5,6,7

* signifies the current active mode

[root@Ruthie-07 /]# hdparm -W0 /dev/sda

/dev/sda:
setting drive write-caching to 0 (off)
write-caching = 0 (off)
[root@Ruthie-07 /]# hdparm -i /dev/sda

/dev/sda:

Model=ST3500641A, FwRev=3.AAE, SerialNo=3PM0DBT1
Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs RotSpdTol>.5% }
RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=4
BuffType=unknown, BuffSize=16384kB, MaxMultSect=16, MultSect=16
CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=976773168
IORDY=on/off, tPIO={min:240,w/IORDY:120}, tDMA={min:120,rec:120}
PIO modes: pio0 pio1 pio2 pio3 pio4
DMA modes: mdma0 mdma1 mdma2
UDMA modes: udma0 udma1 udma2 udma3 udma4 *udma5
AdvancedPM=no WriteCache=disabled
Drive conforms to: Unspecified: ATA/ATAPI-1,2,3,4,5,6,7

* signifies the current active mode

SJ

---------- Post added at 03:44 AM CST ---------- Previous post was at 03:05 AM CST ----------

OK, after rebooting
the system sda write cache was enabled again.
The sdb write cache was still disabled, it is mounted in rc.local.

So now to find what enabled the system disk?

SJ

---------- Post added at 04:33 AM CST ---------- Previous post was at 03:44 AM CST ----------

I used rc.local for
hdparm -Wo /dev/sda

Also /dev/mapper is waht prevents LVM from using barriers so write cache must be off to insure f/s integrity.

SJ