Messages in this thread | | | Date | Tue, 01 Feb 2000 18:44:45 +0100 | From | Richard Ems <> | Subject | BUG REPORT: NFS/SMP file corruption |
| |
[1.] One line summary of the problem:
NFS/SMP file corruption.
[2.] Full description of the problem/report:
Host A: 2.2.14 on a P III 700 Dual SMP machine, 512 MB SDRAM, 512 MB swap, ... Host B: 2.2.14 on a P III 700 machine, 256 MB SDRAM, 128 MB swap, ... (See more information on network cards and statistics under [7.5] )
After running a FORTRAN program on host A and writing results on a NFS-mounted partition on host B, I noticed that the (binary) output file was corrupted. After running some tests, I suppose that this is a NFS/SMP related problem. I wrote a small 6 lines FORTRAN program for my test, it writes 2 files, one binary (fort.11) the second one (fort.12) ascii:
program nfs c do 1 i=1,20*2000*10,1 write (11) ( float(i+j), j=0,10,1 ) write (12,*) ( float(i+j), j=0,10,1 ) 1 continue c end
I also wrote a program for reading the binary file and writing again the input data from fort.11 on a new ascii file fort.13:
program nfsread c real z(0:10) c do 1 i=1,20*2000*10,1 read (11) ( z(j), j=0,10,1 ) write (13,*) ( z(j), j=0,10,1 ) 1 continue c end
I started my test programs 20 times on host A with the files being written also on host A. Everything looks fine, same results. BUT starting the programs on the same host A but writing on a nfs-mounted partition of a second host B gives me DIFFERENT RESULTS !!! This phenomena doesn't appear every time I run the programs; sometimes the results are correct, sometimes they aren't.
I don't get any errors on /var/log/messages or /var/log/warn.
I'm using userland nfsd an still not knfsd.
autofs v 3.1.3, rsize and wsize were not set, so the defaults are both 1024, or not? I changed both rsize and wsize now to 8192, should I use other values?
The problem seems to be NFS/SMP related, only under heavy/long network load. Both network cards are 100Mbit but both a switch and a hub between them are only 10 Mbit.
I'm still running a couple of tests, ... Where could I look for errors? Why does iptraf show so many "IP checksum errors", or is this "normal" and what does this error mean? (look under [7.5])
Too many collisions?
[3.] Keywords (i.e., modules, networking, kernel):
NFS, SMP, 2.2.14
[4.] Kernel version (from /proc/version):
Linux version 2.2.14 (root@diablo) (gcc version egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)) #2 SMP Tue Jan 11 17:04:46 MET 2000
[5.] Output of Oops.. message (if applicable) with symbolic information resolved (see Documentation/oops-tracing.txt)
No Oops.
[6.] A small shell script or example program which triggers the problem (if possible)
See above.
[7.] Environment
PWD=/tmp/ems/nfs/5 PAGER=less HOSTNAME=diablo LS_OPTIONS=-N --color=tty -T 0 ignoreeof=0 POVRAYOPT=-l/usr/lib/povray/include SUSE_DOC_HOST=diablo.mtg-marinetechnik.de QTDIR=/usr/lib/qt LESSKEY=/etc/lesskey.bin LESSOPEN=|lesspipe.sh %s MANPATH=/usr/local/man:/usr/share/man:/usr/man:/usr/X11R6/man:/usr/openwin/man:/usr/man/allman:/usr/local/lib/blas/man:/usr/local/lib/lapack/man
PS1=\u@\h:\w > gks=/usr/local/newgks PS2=> NNTPSERVER=news KDEDIR=/opt/kde LESS=-M -S -I USER=ems LS_COLORS= HISTCONTROL=ignoredups MACHTYPE=i686-pc-linux-gnu XKEYSYMDB=/usr/X11R6/lib/X11/XKeysymDB MAIL=/var/spool/mail/ems OLDPWD=/tmp/ems/nfs/4 LANG=POSIX GNOMEDIR=/opt/gnome COLORTERM=1 INFOPATH=/usr/local/info:/usr/info DISPLAY=bingo:0.0 LOGNAME=ems SHLVL=1 TEXINPUTS=/usr/local/pstricks/generic::~/.TeX:/usr/doc/.TeX:/usr/doc/.TeX: MINICOM=-c on INFODIR=/usr/local/info:/usr/info DVIPSHEADERS=/usr/local/pstricks/dvips: SHELL=/bin/bash HOSTTYPE=i686 OSTYPE=linux-gnu WINDOWMANAGER=/usr/X11R6/bin/kde TERM=xterm HOME=/net/bingo/ems XNLSPATH=/usr/X11R6/lib/X11/nls no_proxy=localhost PATH=/opt/jdk1.2/bin:/opt/java/bin:/net/bingo/ems/bin:/usr/local/bin:/usr/bin:/usr/X11R6/bin:/bin:/usr/lib/java/bin:/usr/games/bin:/usr/games:/opt/gnome/bin:/opt/kde/bin:.:/net/bingo/ems/bin:/net/bingo/ems/bin/scripts:/net/diablo/pereira/bin/simbelbin
LESSCHARSET=latin1 FROM_HEADER=YAST_ASK JIKESPATH=/opt/java/jre/lib/ext/iiimp.jar:/opt/java/jre/lib/ext/vecmath.jar:/opt/java/jre/lib/ext/j3dcore.jar:/opt/java/jre/lib/ext/j3daudio.jar:/opt/java/jre/lib/ext/j3dutils.jar:/opt/java/jre/lib/i18n.jar:/opt/java/jre/lib/rt.jar:/opt/java/lib/tools.jar:/opt/java/lib/dt.jar:/opt/java/lib/jpda.jar:/opt/java/demo/jfc/SwingSet/SwingSet.jar:/opt/java/demo/jfc/FileChooserDemo/FileChooserDemo.jar:/opt/java/demo/jfc/Metalworks/Metalworks.jar:/opt/java/demo/jfc/SimpleExample/SimpleExample.jar:/opt/java/demo/jfc/Notepad/Notepad.jar:/opt/java/demo/jfc/Stylepad/Stylepad.jar:/opt/java/demo/jfc/SampleTree/SampleTree.jar:/opt/java/demo/jfc/SwingApplet/SwingApplet.jar:/opt/java/demo/jfc/TableExample/TableExample.jar:/opt/java/demo/jfc/Java2D/Java2Demo.jar:/opt/java/src.jar:/opt/java/java3d-utils-src.jar
_=/usr/bin/env
[7.1.] Software (add the output of the ver_linux script here)
Linux diablo 2.2.14 #2 SMP Tue Jan 11 17:04:46 MET 2000 i686 unknown Kernel modules 2.3.6 Gnu C egcs-2.91.66 Binutils 2.9.1.0.25 Linux C Library x 1 root root 4223971 Nov 6 17:22 /lib/libc.so.6 Dynamic linker ldd (GNU libc) 2.1.2 Procps 2.0.2 Mount 2.9z Net-tools 1.53 Kbd 0.99 Sh-utils 2.0 Modules Loaded parport_probe parport_pc lp parport serial
[7.2.] Processor information (from /proc/cpuinfo):
processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 8 model name : Pentium III (Coppermine) stepping : 1 cpu MHz : 701.605749 cache size : 256 KB fdiv_bug : no hlt_bug : no sep_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 2 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr xmm bogomips : 701.24
processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 8 model name : Pentium III (Coppermine) stepping : 1 cpu MHz : 701.605749 cache size : 256 KB fdiv_bug : no hlt_bug : no sep_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 2 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr xmm bogomips : 699.60
[7.3.] Module information (from /proc/modules):
parport_probe 3204 0 (autoclean) parport_pc 7440 1 (autoclean) lp 5380 0 (autoclean) (unused) parport 7468 1 (autoclean) [parport_probe parport_pc lp] serial 19508 0 (autoclean)
[7.4.] SCSI information (from /proc/scsi/scsi)
Attached devices: Host: scsi0 Channel: 00 Id: 00 Lun: 00 Vendor: IBM Model: DMVS18V Rev: 0100 Type: Direct-Access ANSI SCSI revision: 03 Host: scsi0 Channel: 00 Id: 01 Lun: 00 Vendor: IBM Model: DMVS18V Rev: 0100 Type: Direct-Access ANSI SCSI revision: 03 Host: scsi0 Channel: 00 Id: 02 Lun: 00 Vendor: PIONEER Model: DVD-ROM DVD-303 Rev: 1.10 Type: CD-ROM ANSI SCSI revision: 02 Host: scsi0 Channel: 00 Id: 04 Lun: 00 Vendor: HP Model: C1533A Rev: 9503 Type: Sequential-Access ANSI SCSI revision: 02 Host: scsi0 Channel: 00 Id: 05 Lun: 00 Vendor: YAMAHA Model: CDR100 Rev: 1.10 Type: WORM ANSI SCSI revision: 02
[7.5.] Other information that might be relevant to the problem (please look in /proc and include all information that you think to be relevant):
Output from dmesg on host A: 3c59x.c:v0.99H 11/17/98 Donald Becker http://cesdis.gsfc.nasa.gov/linux/drivers/vortex.html eth0: 3Com 3c905B Cyclone 100baseTx at 0xe400, 00:10:5a:ef:d9:f0, IRQ 12 8K byte-wide RAM 5:3 Rx:Tx split, autoselect/Autonegotiate interface. MII transceiver found at address 24, status 786d. MII transceiver found at address 0, status 786d. Enabling bus-master transmits and whole-frame receives.
Output of ifconfig eth0 on host A: eth0 Link encap:Ethernet HWaddr 00:50:DA:79:9B:99 inet addr:128.1.3.200 Bcast:128.1.255.255 Mask:255.255.0.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:14656245 errors:0 dropped:0 overruns:0 frame:0 TX packets:10591497 errors:0 dropped:0 overruns:0 carrier:0 collisions:1303598 txqueuelen:100 Interrupt:18 Base address:0xb800
Output from dmesg on host B: ... 3c59x.c:v0.99H 11/17/98 Donald Becker http://cesdis.gsfc.nasa.gov/linux/drivers/vortex.html eth0: 3Com 3c905B Cyclone 100baseTx at 0xb800, 00:50:da:79:9b:99, IRQ 18 8K byte-wide RAM 5:3 Rx:Tx split, autoselect/Autonegotiate interface. MII transceiver found at address 24, status 786d. MII transceiver found at address 0, status 786d. Enabling bus-master transmits and whole-frame receives. ...
Output of ifconfig eth0 on host B: eth0 Link encap:Ethernet HWaddr 00:10:5A:EF:D9:F0 inet addr:128.1.3.96 Bcast:128.1.255.255 Mask:255.255.0.0 UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:47231427 errors:29 dropped:0 overruns:16 frame:34 TX packets:11987070 errors:0 dropped:0 overruns:0 carrier:606 collisions:1535611 txqueuelen:100 Interrupt:12 Base address:0xe400
Too many collisions? 29 errors on host B?
statistics form iptraf running over night doing some tests: 488 IP Checksum Errors ???
IPTraf 1.2.0 l Statistics for eth0 qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk x x x Packets Bytes x x Total: 30613492 -3915173578 x x IP: 30342496 -8357790507 x x TCP: 10087103 -2791074268 Activity: 135.60 kbits/sec x x UDP: 20254710 -5566881849 80.20 packets/sec x x ICMP: 192 16464 x x Other IP: 3 96 IP Checksum Errors: 488 x x Non-IP: 270996 19334387 x x x x x x Pkt Size (bytes) Count Pkt Size (bytes) Count x x 1 to 100: 5635351 801 to 900: 11827 x x 101 to 200: 17406087 901 to 1000: 5218 x x 201 to 300: 1241546 1001 to 1100: 2291 x x 301 to 400: 222220 1101 to 1200: 1158 x x 401 to 500: 30912 1201 to 1300: 232354 x x 501 to 600: 78606 1301 to 1400: 764 x x 601 to 700: 4333406 1401 to 1500: 457959 x x 701 to 800: 682309 1500+: 0 x x x m Elapsed time: 14:29 qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj
[X.] Other notes, patches, fixes, workarounds:
Nothing.
OK, enough for now ...
Thanks, rems
-- Richard Ems ... e-mail: r.ems@gmx.net ... Computer Science, University of Hamburg
Unix IS user friendly. It's just selective about who its friends are.
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |