Extremely slow networking

[gizenshu]

F14 running 2.6.35-14.96 64 bit kernel on Gigabyte mobo with AMD 890 chipset that has built in Realtek 8168 gigabit NIC.

Windows 7 64 bit is running on ASUS mobo with 890 chipset and same NIC
Its shared drives are mounted on F14 using cifs in fstab.

Copying files from Win7 to F14 machine at about 200-300 kB/s.

Sometimes it is possible to get better speed for a short time by starting another network transfer from F14 and another host, for example d/l from the web.

Same PC dual-booted into WinXP or Win7 will copy files at 50-60 MB/s.

What am I doing wrong?

---------- Post added at 10:35 PM ---------- Previous post was at 04:02 PM ----------

More information on the issue:

After unmounting the share and mounting it using Gnome interface, transfer speed reached 26-27 MB/s.
That's not as good as under Windowze, but quite an improvement compared to CIFS.

How can I improve CIFS transfers? I am kind of used to having drive permanently mounted in the same location, and having to click Network/share/mount sounds a bit of overkill.

[flyingfsck]

Howdy,

CIFS is a very chatty protocol and the Samba implementation leaves much to be desired. So, even if you have it installed and configured right, it will always be slow.
http://www.google.com/search?hl=en&q...cifs+very+slow

Do you really have to use CIFS? I'd go with NFS instead.

[stevea]

Compare here ....
http://forums.fedoraforum.org/showthread.php?t=246702

The OP should be able to get much better than 0.2 - 0.3 MBPS.

I strongly suggest the OP run 'nuttcp' tests between systems (it's available for Windows) and see what sort of bandwidth he gets.
Something seems very wrong here.

[thony sasak]

f16 kde. cannot connect to wireless. all chipset has been detected. its happen when iam update kernel header. Now using 3.2.9-2.fc16.i686.before that everythings is ok. iam try to running my dmesg | fpaste. in here is the link .
http://fpaste.org/Nt51/
is it something wrong on my system. thank you before..

[gizenshu]

I would have gladly moved to NFS or any other more efficient network FS, but for that I have to have authority on Windows machines that share files with me, and I don't. Have to put up with CIFS.

It wasn't always like that. Between 1997 and about 2003 I was getting technically possible speed with file xfers between Slackware and RH machines on my side and WinNT/WinXP machines on another.

But it was smbfs back then, not cifs. As for

even if you have it installed and configured right, it will always be slow.

...F14 did it, not me. If you know how to configure CIFS, please share, as I can't see where and how it could be done.
The last part is obviously not true, as Microsoft, quite obviously, is not having slowness issue.

Also I came across this http://ubuntuforums.org/showthread.php?t=1578068&page=2 and this http://oreilly.com/catalog/samba/chapter/book/appb.pdf however I don't see how tuning smb.conf is going to help, because in my case smbd is not even running.

I won't be allowed to run sniffer tools on Windowze. This has to be troubleshot on F14 because infrastructure considers Win7 running fine and I can't find fault in their thinking.
Don't know if this gives anyone more food for thought, just showing you how I mount shares (tried w/ and w/o async last night with no obvious success)

//10.10.10.2/drived	/mnt/Win764D		cifs	async,user=userid,password=pass,uid=userid,gid=userid,file_mode=0644,dir_mode=0777	0 0
//10.10.10.2/drivee	/mnt/Win764E		cifs	async,user=userid,password=pass,uid=userid,gid=userid,file_mode=0644,dir_mode=0777	0 0

[gizenshu]

Does anybody know what is the difference between mounting Windows shares using mount.cifs and in Gnome GUI?

[flyingfsck]

man mount.cifs

directio
Do not do inode data caching on files opened on this mount. This
precludes mmaping files on this mount. In some cases with fast
networks and little or no caching benefits on the client (e.g. when
the application is doing large sequential reads bigger than page
size without rereading the same data) this can provide better
performance than the default behavior which caches reads
(readahead) and writes (writebehind) through the local Linux client
pagecache if oplock (caching token) is granted and held. Note that
direct allows write operations larger than page size to be sent to
the server. On some kernels this requires the cifs.ko module to be
built with the CIFS_EXPERIMENTAL configure option.

rsize=bytes
Maximum amount of data that the kernel will request in a read
request in bytes. Prior to kernel 3.2.0, the default was 16k, and
the maximum size was limited by the CIFSMaxBufSize module
parameter. As of kernel 3.2.0, the behavior varies according to
whether POSIX extensions are enabled on the mount and the server
supports large POSIX reads. If they are, then the default is 1M,
and the maxmimum is 16M. If they are not supported by the server,
then the default is 60k and the maximum is around 127k. The reason
for the 60k is because it's the maximum size read that windows
servers can fill. Note that this value is a maximum, and the client
may settle on a smaller size to accomodate what the server
supports. In kernels prior to 3.2.0, no negotiation is performed.

wsize=bytes
Maximum amount of data that the kernel will send in a write request
in bytes. Prior to kernel 3.0.0, the default and maximum was 57344
(14 * 4096 pages). As of 3.0.0, the default is 1M, and the maximum
allowed is 16M. Note that this value is just a starting point for
negotiation in 3.0.0 and up. The client and server may negotiate
this size downward according to the server's capabilities. In
kernels prior to 3.0.0, no negotiation is performed. It can end up
with an existing superblock if this value isn't specified or it's
greater or equal than the existing one.

fsc
Enable local disk caching using FS-Cache for CIFS. This option
could be useful to improve performance on a slow link, heavily
loaded server and/or network where reading from the disk is faster
than reading from the server (over the network). This could also
impact the scalability positively as the number of calls to the
server are reduced. But, be warned that local caching is not
suitable for all workloads, for e.g., read-once type workloads. So,
you need to consider carefully the situation/workload before using
this option. Currently, local disk caching is enabled for CIFS
files opened as read-only.

NOTE: This feature is available only in the recent kernels that
have been built with the kernel config option CONFIG_CIFS_FSCACHE.
You also need to have cachefilesd daemon installed and running to
make the cache operational.

[gizenshu]

Tried directio option.
It gives me 10 MB/s, which is lots better than default behavior, but still less than 50% of GUI mounted shares and less than 20% of Windows.
Further,

modprobe cifs CIFSMaxBufSize=130048

with same rsize is not giving any improvement.

[flyingfsck]

rsize and wsize do nothing? These used to make a huge difference years ago.

[gizenshu]

Actually, more precise measurement is 11.8 MB/s. That's the maximum I've gotten with directio and rsize combined with CIFSMaxBufSize=130048 made no difference.
smbclient in Gnome is giving me 26-28 MB/s from the same share.

I've taken the value 130048 from here https://bbs.archlinux.org/viewtopic.php?id=115788 not sure if it's optimal for my system, but it certainly makes no difference whatsoever.

Writing to the Win share is at 7.6 MB/s.

[flyingfsck]

According to the man page the default is 1M. Too big a block size will make it slower. Long ago, I used 64K, when the default was 16k.

The local caching may be more useful.

[gizenshu]

Originally Posted by flyingfsck

According to the man page the default is 1M. Too big a block size will make it slower. Long ago, I used 64K, when the default was 16k.

The local caching may be more useful.

Dunno, here is what I see under F14 x86_64:

rsize=arg
           default network read size (usually 16K). The client currently can not use rsize larger than CIFSMaxBufSize. CIFSMaxBufSize defaults to 16K and may be
           changed (from 8K to the maximum kmalloc size allowed by your kernel) at module install time for cifs.ko. Setting CIFSMaxBufSize to a very large value will
           cause cifs to use more memory and may reduce performance in some cases. To use rsize greater than 127K (the original cifs protocol maximum) also requires
           that the server support a new Unix Capability flag (for very large read) which some newer servers (e.g. Samba 3.0.26 or later) do. rsize can be set from a
           minimum of 2048 to a maximum of 130048 (127K or CIFSMaxBufSize, whichever is smaller)

Not sure I understand local caching part - haven't I just turned it off with directio to obtain 50x times better results? I am so confused now!

[gizenshu]

Just an update: wth fsc option copy speed increased further 25%, from 12 to 15 MB/s
Still twice lower than smbclient thru Nautilus and 4 times lower than in Windowze

Specifying 127K rsize might have improved things by 1-2 more MB/s, but I can't be certain, as that could be due to file fragmentation on Win7 share (not getting sustained result, just peaks at 16.8-17 MB/s).
Again, I am not 100% sure, but it looks like specifying a larger value for module parameter and rsize than 127k removes those peaks.

[gizenshu]

Is it possible to mount the shares using Nautilus/smbclient to the same locations as I used to with mount.cifs?

[gizenshu]

The changes I've made above to achieve 15 MB/s have an nasty side effect: now some programs can't load files from the shares, for instance Acrobat Reader reports that every other file is corrupt, though the files are fine.

Where is this world going? I more and more think that I should go back to RH5 or Slackware 2. They did not have these blatant bugs.