This message generated a parse failure. Raw output follows here. Please use 'back' to navigate. From devnull@lkml.org Thu Apr 25 17:44:08 2024 Received: from nic.funet.fi (nic.funet.fi [128.214.248.6]) by herbie.ucs.indiana.edu (8.9.3/8.9.3) with ESMTP id EAA15663 for ; Sat, 15 May 1999 04:43:09 -0500 (EST) Received: from vger.rutgers.edu ([128.6.190.2]:54546 "EHLO vger.rutgers.edu" ident: "NO-IDENT-SERVICE[2]") by nic.funet.fi with ESMTP id <20488-14153>; Sat, 15 May 1999 03:54:37 +0300 Received: by vger.rutgers.edu via listexpand id ; Fri, 14 May 1999 12:25:21 -0400 Received: by vger.rutgers.edu id ; Fri, 14 May 1999 12:24:09 -0400 Received: from penguin.e-mind.com ([195.223.140.120]:22080 "EHLO penguin.e-mind.com") by vger.rutgers.edu with ESMTP id ; Fri, 14 May 1999 06:26:48 -0400 Received: from laser.random (isdn53-imo.e-mind.com [195.223.140.63]) by penguin.e-mind.com (8.9.1a/8.9.1/Debian/GNU) with ESMTP id NAA08963; Fri, 14 May 1999 13:11:56 +0200 Received: from localhost (andrea@localhost) by laser.random (8.8.8/8.8.8) with ESMTP id MAA00834; Fri, 14 May 1999 12:54:01 +0200 X-Authentication-Warning: laser.random: andrea owned process doing -bs Date: Fri, 14 May 1999 12:54:01 +0200 (CEST) From: Andrea Arcangeli X-Sender: andrea@laser.random To: Steve Willer Cc: Zack Weinberg , linux-kernel@vger.rutgers.edu, Linus Torvalds , lm@bitmover.com, pcg@goof.com, pavel@atrey.karlin.mff.cuni.cz, mhw@wittsend.com Subject: Re: All the problems with 2.2.8/2.3.x and bdflush/update In-Reply-To: Message-Id: X-Public-Key-URL: http://e-mind.com/~andrea/aa.asc Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=iso-8859-1 Sender: owner-linux-kernel@vger.rutgers.edu Precedence: bulk X-Loop: majordomo@vger.rutgers.edu X-Orcpt: rfc822;linux-kernel-outgoing-dig X-UIDL: 926699116.830967.17442 On Thu, 13 May 1999, Steve Willer wrote: >Ideally, the flushing algorithm would be tightly coupled to the wakeup >time calculation. update is quite simplistic, and also isn't part of the update can run every msec but the number of dirty buffer flushed/sec has to remain the same. That doesn't happen due the current flushtime design but this is a completly different issue (and I just spotted and fixed it some time ago), you don't need to kill update if there's a bug somewhere else. Even if you run update very frequently you won't flush more data to disk. The only point of update is trying to be friendly under crash. Nothing more. Really please try out: ftp://e-mind.com/pub/andrea/kernel/2.2.8_andrea1.bz2 or ftp://e-mind.com/pub/andrea/kernel/2.3.1_andrea1.bz2 and do some test under heavy load. >Having an atime means that a system (on a laptop) that reads files, but >doesn't write to any, will still want to wake up the disk. Some people With my current tree having atime or not is really a minor issue. I know it's an issue with the old flushtime handling. Here I have atime set but: Script started on Fri May 14 12:40:37 1999 andrea@laser:~$ /usr/bin/time cvs diff -u /usr/src/linux >/dev/null cvs diff: Diffing /usr/src/linux cvs diff: Diffing /usr/src/linux/Documentation cvs diff: Diffing /usr/src/linux/Documentation/arm cvs diff: Diffing /usr/src/linux/Documentation/arm/nwfpe cvs diff: Diffing /usr/src/linux/Documentation/cdrom cvs diff: Diffing /usr/src/linux/Documentation/fb cvs diff: Diffing /usr/src/linux/Documentation/filesystems cvs diff: Diffing /usr/src/linux/Documentation/i386 cvs diff: Diffing /usr/src/linux/Documentation/isdn cvs diff: Diffing /usr/src/linux/Documentation/kbuild cvs diff: Diffing /usr/src/linux/Documentation/m68k cvs diff: Diffing /usr/src/linux/Documentation/networking cvs diff: Diffing /usr/src/linux/Documentation/networking/ip_masq cvs diff: Diffing /usr/src/linux/Documentation/powerpc cvs diff: Diffing /usr/src/linux/Documentation/sound cvs diff: Diffing /usr/src/linux/Documentation/sysctl cvs diff: Diffing /usr/src/linux/Documentation/video4linux cvs diff: Diffing /usr/src/linux/Documentation/video4linux/bttv cvs diff: Diffing /usr/src/linux/arch cvs diff: Diffing /usr/src/linux/arch/alpha cvs diff: Diffing /usr/src/linux/arch/alpha/boot cvs diff: Diffing /usr/src/linux/arch/alpha/boot/tools cvs diff: Diffing /usr/src/linux/arch/alpha/kernel cvs diff: Diffing /usr/src/linux/arch/alpha/lib cvs diff: Diffing /usr/src/linux/arch/alpha/math-emu cvs diff: Diffing /usr/src/linux/arch/alpha/mm cvs diff: Diffing /usr/src/linux/arch/arm cvs diff: Diffing /usr/src/linux/arch/arm/boot cvs diff: Diffing /usr/src/linux/arch/arm/boot/compressed cvs diff: Diffing /usr/src/linux/arch/arm/kernel cvs diff: Diffing /usr/src/linux/arch/arm/lib cvs diff: Diffing /usr/src/linux/arch/arm/mm cvs diff: Diffing /usr/src/linux/arch/i386 cvs diff: Diffing /usr/src/linux/arch/i386/boot cvs diff: Diffing /usr/src/linux/arch/i386/boot/compressed cvs diff: Diffing /usr/src/linux/arch/i386/boot/tools cvs diff: Diffing /usr/src/linux/arch/i386/kernel cvs diff: Diffing /usr/src/linux/arch/i386/kernel/debug cvs diff: Diffing /usr/src/linux/arch/i386/lib cvs diff: Diffing /usr/src/linux/arch/i386/math-emu cvs diff: Diffing /usr/src/linux/arch/i386/mm cvs diff: Diffing /usr/src/linux/arch/m68k cvs diff: Diffing /usr/src/linux/arch/m68k/amiga cvs diff: Diffing /usr/src/linux/arch/m68k/apollo cvs diff: Diffing /usr/src/linux/arch/m68k/atari cvs diff: Diffing /usr/src/linux/arch/m68k/bvme6000 cvs diff: Diffing /usr/src/linux/arch/m68k/fpsp040 cvs diff: Diffing /usr/src/linux/arch/m68k/hp300 cvs diff: Diffing /usr/src/linux/arch/m68k/ifpsp060 cvs diff: Diffing /usr/src/linux/arch/m68k/kernel cvs diff: Diffing /usr/src/linux/arch/m68k/lib cvs diff: Diffing /usr/src/linux/arch/m68k/mac cvs diff: Diffing /usr/src/linux/arch/m68k/mm cvs diff: Diffing /usr/src/linux/arch/m68k/mvme147 cvs diff: Diffing /usr/src/linux/arch/m68k/mvme16x cvs diff: Diffing /usr/src/linux/arch/m68k/q40 cvs diff: Diffing /usr/src/linux/arch/m68k/sun3x cvs diff: Diffing /usr/src/linux/arch/m68k/tools cvs diff: Diffing /usr/src/linux/arch/m68k/tools/amiga cvs diff: Diffing /usr/src/linux/arch/mips cvs diff: Diffing /usr/src/linux/arch/mips/boot cvs diff: Diffing /usr/src/linux/arch/mips/jazz cvs diff: Diffing /usr/src/linux/arch/mips/kernel cvs diff: Diffing /usr/src/linux/arch/mips/lib cvs diff: Diffing /usr/src/linux/arch/mips/mm cvs diff: Diffing /usr/src/linux/arch/mips/sgi cvs diff: Diffing /usr/src/linux/arch/mips/sgi/kernel cvs diff: Diffing /usr/src/linux/arch/mips/sgi/prom cvs diff: Diffing /usr/src/linux/arch/mips/sni cvs diff: Diffing /usr/src/linux/arch/mips/tools cvs diff: Diffing /usr/src/linux/arch/ppc cvs diff: Diffing /usr/src/linux/arch/ppc/8xx_io cvs diff: Diffing /usr/src/linux/arch/ppc/amiga cvs diff: Diffing /usr/src/linux/arch/ppc/boot cvs diff: Diffing /usr/src/linux/arch/ppc/chrpboot cvs diff: Diffing /usr/src/linux/arch/ppc/coffboot cvs diff: Diffing /usr/src/linux/arch/ppc/kernel cvs diff: Diffing /usr/src/linux/arch/ppc/lib cvs diff: Diffing /usr/src/linux/arch/ppc/mbxboot cvs diff: Diffing /usr/src/linux/arch/ppc/mm cvs diff: Diffing /usr/src/linux/arch/sparc cvs diff: Diffing /usr/src/linux/arch/sparc/ap1000 cvs diff: Diffing /usr/src/linux/arch/sparc/boot cvs diff: Diffing /usr/src/linux/arch/sparc/kernel cvs diff: Diffing /usr/src/linux/arch/sparc/lib cvs diff: Diffing /usr/src/linux/arch/sparc/math-emu cvs diff: Diffing /usr/src/linux/arch/sparc/mm cvs diff: Diffing /usr/src/linux/arch/sparc/prom cvs diff: Diffing /usr/src/linux/arch/sparc64 cvs diff: Diffing /usr/src/linux/arch/sparc64/boot cvs diff: Diffing /usr/src/linux/arch/sparc64/kernel cvs diff: Diffing /usr/src/linux/arch/sparc64/lib cvs diff: Diffing /usr/src/linux/arch/sparc64/math-emu cvs diff: Diffing /usr/src/linux/arch/sparc64/mm cvs diff: Diffing /usr/src/linux/arch/sparc64/prom cvs diff: Diffing /usr/src/linux/arch/sparc64/solaris cvs diff: Diffing /usr/src/linux/drivers cvs diff: Diffing /usr/src/linux/drivers/acorn cvs diff: Diffing /usr/src/linux/drivers/acorn/block cvs diff: Diffing /usr/src/linux/drivers/acorn/char cvs diff: Diffing /usr/src/linux/drivers/acorn/net cvs diff: Diffing /usr/src/linux/drivers/acorn/scsi cvs diff: Diffing /usr/src/linux/drivers/ap1000 cvs diff: Diffing /usr/src/linux/drivers/block cvs diff: Diffing /usr/src/linux/drivers/block/paride cvs diff: Diffing /usr/src/linux/drivers/cdrom cvs diff: Diffing /usr/src/linux/drivers/char cvs diff: Diffing /usr/src/linux/drivers/char/ftape cvs diff: Diffing /usr/src/linux/drivers/char/ftape/compressor cvs diff: Diffing /usr/src/linux/drivers/char/ftape/lowlevel cvs diff: Diffing /usr/src/linux/drivers/char/ftape/zftape cvs diff: Diffing /usr/src/linux/drivers/char/hfmodem cvs diff: Diffing /usr/src/linux/drivers/char/joystick cvs diff: Diffing /usr/src/linux/drivers/dio cvs diff: Diffing /usr/src/linux/drivers/fc4 cvs diff: Diffing /usr/src/linux/drivers/isdn cvs diff: Diffing /usr/src/linux/drivers/isdn/act2000 cvs diff: Diffing /usr/src/linux/drivers/isdn/avmb1 cvs diff: Diffing /usr/src/linux/drivers/isdn/hisax cvs diff: Diffing /usr/src/linux/drivers/isdn/icn cvs diff: Diffing /usr/src/linux/drivers/isdn/isdnloop cvs diff: Diffing /usr/src/linux/drivers/isdn/pcbit cvs diff: Diffing /usr/src/linux/drivers/isdn/sc cvs diff: Diffing /usr/src/linux/drivers/macintosh cvs diff: Diffing /usr/src/linux/drivers/misc cvs diff: Diffing /usr/src/linux/drivers/net cvs diff: Diffing /usr/src/linux/drivers/net/hamradio cvs diff: Diffing /usr/src/linux/drivers/net/hamradio/soundmodem cvs diff: Diffing /usr/src/linux/drivers/net/irda cvs diff: Diffing /usr/src/linux/drivers/nubus cvs diff: Diffing /usr/src/linux/drivers/pci cvs diff: Diffing /usr/src/linux/drivers/pnp cvs diff: Diffing /usr/src/linux/drivers/sbus cvs diff: Diffing /usr/src/linux/drivers/sbus/audio cvs diff: Diffing /usr/src/linux/drivers/sbus/char cvs diff: Diffing /usr/src/linux/drivers/scsi cvs diff: Diffing /usr/src/linux/drivers/scsi/aic7xxx cvs diff: Diffing /usr/src/linux/drivers/sgi cvs diff: Diffing /usr/src/linux/drivers/sgi/char cvs diff: Diffing /usr/src/linux/drivers/sound cvs diff: Diffing /usr/src/linux/drivers/sound/lowlevel cvs diff: Diffing /usr/src/linux/drivers/usb cvs diff: Diffing /usr/src/linux/drivers/usb/maps cvs diff: Diffing /usr/src/linux/drivers/video cvs diff: Diffing /usr/src/linux/drivers/zorro cvs diff: Diffing /usr/src/linux/fs cvs diff: Diffing /usr/src/linux/fs/adfs cvs diff: Diffing /usr/src/linux/fs/affs cvs diff: Diffing /usr/src/linux/fs/autofs cvs diff: Diffing /usr/src/linux/fs/coda cvs diff: Diffing /usr/src/linux/fs/devpts cvs diff: Diffing /usr/src/linux/fs/ext2 cvs diff: Diffing /usr/src/linux/fs/fat cvs diff: Diffing /usr/src/linux/fs/hfs cvs diff: Diffing /usr/src/linux/fs/hpfs cvs diff: Diffing /usr/src/linux/fs/isofs cvs diff: Diffing /usr/src/linux/fs/lockd cvs diff: Diffing /usr/src/linux/fs/minix cvs diff: Diffing /usr/src/linux/fs/msdos cvs diff: Diffing /usr/src/linux/fs/ncpfs cvs diff: Diffing /usr/src/linux/fs/nfs cvs diff: Diffing /usr/src/linux/fs/nfsd cvs diff: Diffing /usr/src/linux/fs/nls cvs diff: Diffing /usr/src/linux/fs/ntfs cvs diff: Diffing /usr/src/linux/fs/proc cvs diff: Diffing /usr/src/linux/fs/qnx4 cvs diff: Diffing /usr/src/linux/fs/romfs cvs diff: Diffing /usr/src/linux/fs/smbfs cvs diff: Diffing /usr/src/linux/fs/sysv cvs diff: Diffing /usr/src/linux/fs/ufs cvs diff: Diffing /usr/src/linux/fs/umsdos cvs diff: Diffing /usr/src/linux/fs/vfat cvs diff: Diffing /usr/src/linux/include cvs diff: Diffing /usr/src/linux/include/asm-alpha cvs diff: Diffing /usr/src/linux/include/asm-arm cvs diff: Diffing /usr/src/linux/include/asm-arm/arch-arc cvs diff: Diffing /usr/src/linux/include/asm-arm/arch-ebsa110 cvs diff: Diffing /usr/src/linux/include/asm-arm/arch-ebsa285 cvs diff: Diffing /usr/src/linux/include/asm-arm/arch-nexuspci cvs diff: Diffing /usr/src/linux/include/asm-arm/arch-rpc cvs diff: Diffing /usr/src/linux/include/asm-arm/arch-vnc cvs diff: Diffing /usr/src/linux/include/asm-arm/proc-armo cvs diff: Diffing /usr/src/linux/include/asm-arm/proc-armv cvs diff: Diffing /usr/src/linux/include/asm-generic cvs diff: Diffing /usr/src/linux/include/asm-i386 cvs diff: Diffing /usr/src/linux/include/asm-m68k cvs diff: Diffing /usr/src/linux/include/asm-mips cvs diff: Diffing /usr/src/linux/include/asm-ppc cvs diff: Diffing /usr/src/linux/include/asm-sparc cvs diff: Diffing /usr/src/linux/include/asm-sparc/ap1000 cvs diff: Diffing /usr/src/linux/include/asm-sparc64 cvs diff: Diffing /usr/src/linux/include/linux cvs diff: Diffing /usr/src/linux/include/linux/byteorder cvs diff: Diffing /usr/src/linux/include/linux/lockd cvs diff: Diffing /usr/src/linux/include/linux/nfsd cvs diff: Diffing /usr/src/linux/include/linux/sunrpc cvs diff: Diffing /usr/src/linux/include/net cvs diff: Diffing /usr/src/linux/include/net/irda cvs diff: Diffing /usr/src/linux/include/scsi cvs diff: Diffing /usr/src/linux/include/video cvs diff: Diffing /usr/src/linux/init cvs diff: Diffing /usr/src/linux/ipc cvs diff: Diffing /usr/src/linux/kernel cvs diff: Diffing /usr/src/linux/kernel/debug cvs diff: Diffing /usr/src/linux/lib cvs diff: Diffing /usr/src/linux/mm cvs diff: Diffing /usr/src/linux/net cvs diff: Diffing /usr/src/linux/net/802 cvs diff: Diffing /usr/src/linux/net/802/pseudo cvs diff: Diffing /usr/src/linux/net/802/transit cvs diff: Diffing /usr/src/linux/net/appletalk cvs diff: Diffing /usr/src/linux/net/ax25 cvs diff: Diffing /usr/src/linux/net/bridge cvs diff: Diffing /usr/src/linux/net/core cvs diff: Diffing /usr/src/linux/net/decnet cvs diff: Diffing /usr/src/linux/net/econet cvs diff: Diffing /usr/src/linux/net/ethernet cvs diff: Diffing /usr/src/linux/net/ipv4 cvs diff: Diffing /usr/src/linux/net/ipv6 cvs diff: Diffing /usr/src/linux/net/ipx cvs diff: Diffing /usr/src/linux/net/irda cvs diff: Diffing /usr/src/linux/net/irda/compressors cvs diff: Diffing /usr/src/linux/net/irda/ircomm cvs diff: Diffing /usr/src/linux/net/irda/irlan cvs diff: Diffing /usr/src/linux/net/irda/irlpt cvs diff: Diffing /usr/src/linux/net/irda/irobex cvs diff: Diffing /usr/src/linux/net/lapb cvs diff: Diffing /usr/src/linux/net/netlink cvs diff: Diffing /usr/src/linux/net/netrom cvs diff: Diffing /usr/src/linux/net/packet cvs diff: Diffing /usr/src/linux/net/rose cvs diff: Diffing /usr/src/linux/net/sched cvs diff: Diffing /usr/src/linux/net/sunrpc cvs diff: Diffing /usr/src/linux/net/unix cvs diff: Diffing /usr/src/linux/net/wanrouter cvs diff: Diffing /usr/src/linux/net/x25 cvs diff: Diffing /usr/src/linux/scripts cvs diff: Diffing /usr/src/linux/scripts/ksymoops cvs diff: Diffing /usr/src/linux/scripts/lxdialog cvs diff: Diffing /usr/src/linux/scripts/memleak 1.66user 0.64system 0:02.29elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (150major+6093minor)pagefaults 0swaps andrea@laser:~$ ]0;andrea@laser:~exit Script done on Fri May 14 12:40:53 1999 2sec to cvs diff a whole 2.3.1 kernel tree with all kernels since 2.0.0 (and most pre-patches) in the repository. I know this is not possible in the stock kenrel but it's due the current design of flushtime, not because you run uptime every 5 sec. I can run uptime every msec and I get the same numbers. What you should care are _only_ the numbers in /proc/sys/vm/bdflush relative to the buffer and super-buffer ages but only once you'll get flushtime fixed. >The per-wakeup write limit was too small, and there was no hard limit of >dirty buffer age. The result was that in situations where there's a lot of >write I/O happening, the disk would be underutilized, but then the buffer >would eventually hit its max dirty buffer percentage and it would start >writing like crazy. It was really bad with update running, doing its sync >every 30s. System capacity would drop to 50% for 5s every 30s. Very bad. >In short, the writes weren't being paced. I agree and I just fixed the flushtime handling last month. If you want to get it fixed start using my current tree (that btw rejects completly the 2.2.8 buffer changes). I think I'll post a patch against 2.2.8 (hopefully today) that will port by current buffer code to 2.2.8. Now I ask you to try out (and benchmark) one of my last two andrea patches. Then I'll ask you to try my new buffer code alone over 2.2.8. Thanks. Andrea Arcangeli - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/