This message generated a parse failure. Raw output follows here. Please use 'back' to navigate. From devnull@lkml.org Sun Apr 28 04:39:22 2024 Received: from entropy.muc.muohio.edu (entropy.muc.muohio.edu [134.53.213.10]) by herbie.ucs.indiana.edu (8.8.8/8.8.8) with ESMTP id PAA03470 for ; Tue, 23 Feb 1999 15:57:21 -0500 (EST) Received: from vger.rutgers.edu (vger.rutgers.edu [128.6.190.2]) by entropy.muc.muohio.edu (Postfix) with ESMTP id 2C33723DCD; Tue, 23 Feb 1999 13:45:22 -0500 (EST) Received: by vger.rutgers.edu via listexpand id <157266-25206>; Tue, 23 Feb 1999 12:48:01 -0500 Received: by vger.rutgers.edu id <157087-25206>; Tue, 23 Feb 1999 12:47:51 -0500 Received: from idiom.com ([209.157.64.1]:1511 "EHLO idiom.com" ident: "root") by vger.rutgers.edu with ESMTP id <155563-25206>; Tue, 23 Feb 1999 12:47:33 -0500 Received: from idiom.com (reiser.dial.idiom.com [209.157.70.56]) by idiom.com (8.8.8/8.8.5) with ESMTP id KAA02956; Tue, 23 Feb 1999 10:54:32 -0800 (PST) Message-Id: <36D2F9F6.7F6B29F6@idiom.com> Date: Tue, 23 Feb 1999 10:56:54 -0800 From: Hans Reiser Organization: Namesys X-Mailer: Mozilla 4.5 [en] (X11; U; Linux 2.2.0-pre6 i686) X-Accept-Language: en Mime-Version: 1.0 To: reiserfs , "linux-kernel@vger.rutgers.edu" Subject: Reiserfs faster on fsync benchmarked some more Content-Type: multipart/mixed; boundary="------------BE9387A5C2D81E0792BF1DD4" Sender: owner-linux-kernel@vger.rutgers.edu Precedence: bulk X-Loop: majordomo@vger.rutgers.edu X-Orcpt: rfc822;linux-kernel-outgoing-dig This is a multi-part message in MIME format. --------------BE9387A5C2D81E0792BF1DD4 Content-Type: multipart/alternative; boundary="------------D463034941420CC3C4658412" --------------D463034941420CC3C4658412 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit I suggest you try writing the file, flushing the cache by cating /dev/hda, then dirtying 1% of the file and fsync'ing. This would resemble what happens with a file that is large and has been around for a while. Best, Hans -- Don't be locked out of the source, and doomed to life in the slow lane. Dump NT! Get Linux (http://www.kernel.org) plus ReiserFS (http://devlinux.org/namesys). If you sell an OS or internet appliance, buy a port of ReiserFS! Speed matters. Trees are fast. Go faster! --------------D463034941420CC3C4658412 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit I suggest you try writing the file, flushing the cache by cating /dev/hda, then dirtying 1% of the file and fsync'ing.
This would resemble what happens with a file that is large and has been around for a while.

Best,

Hans

-- 
Don't be locked out of the source, and doomed to life in the slow lane.
Dump NT! Get Linux (http://www.kernel.org) plus ReiserFS
 (http://devlinux.org/namesys).  If you sell an OS or
internet appliance, buy a port of ReiserFS!
Speed matters.  Trees are fast.  Go faster!
  --------------D463034941420CC3C4658412-- --------------BE9387A5C2D81E0792BF1DD4 Content-Type: message/rfc822 Content-Transfer-Encoding: 7bit Content-Disposition: inline Return-Path: yura@namesys.botik.ru Received: from select.botik.ru (root@select.botik.ru [193.232.174.49]) by idiom.com (8.8.8/8.8.5) with ESMTP id JAA03315 for ; Tue, 23 Feb 1999 09:10:41 -0800 (PST) Received: from namesys.botik.ru (really [127.0.0.1]) by select.botik.ru via smail with esmtp (ident root using rfc1413) id (Debian Smail3.2.0.101) for ; Tue, 23 Feb 1999 21:00:13 +0300 (MSK) Sender: root@namesys.botik.ru Message-ID: <36D2ECA1.8A64E254@namesys.botik.ru> Date: Tue, 23 Feb 1999 21:00:02 +0300 From: "Yury Y.Rupasov" X-Mailer: Mozilla 4.5 [en] (X11; I; Linux 2.2.0 i586) X-Accept-Language: en MIME-Version: 1.0 To: Hans Reiser , vs@namesys.botik Subject: Re: fsync on large files References: <36CC0AD0.E4C7D3C8@bibliotech.co.uk> <7ahnvm$4an$1@palladium.transmeta.com> <36CC71BB.BCD4A086@step.es> <36CD2E45.F2EF418F@idiom.com> <36CDA9E3.B4A04F18@namesys.botik.ru> <36CF896D.E1DDD886@idiom.com> <36D169A2.B4DAF12E@namesys.botik.ru> <36D1AE11.A7180BED@idiom.com> Content-Type: multipart/alternative; boundary="------------D86624808F200806661362A7" X-Mozilla-Status2: 00000000 --------------D86624808F200806661362A7 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Hans Reiser wrote: > What about when you write the 500mb file, flush the cache, modify > (dirty) a few of its blocks (e.g. the last one), and then fsync? Here are the results of new benchmark: fsync time for the 500MB file when a part of the file buffers are dirty buffers. 3Mb of dirty buffers: 0.6 % of the file size reiserfs fsync time (in sec): 1 ext2fs fsync time (in sec): 1 6Mb of dirty buffers: 1.2% of the file size reiserfs fsync time (in sec): 1 ext2fs fsync time (in sec): 1 15Mb of dirty buffers: 3% of the file size reiserfs fsync time (in sec): 2 ext2fs fsync time (in sec): 16 60Mb of dirty buffers: 12% of the file size reiserfs fsync time (in sec): 2 ext2fs fsync time (in sec): 16 I changed fsync.c so that it can do lseek() and write() before fsync(). The program writes N * 1024 of 1KB buffers in the beginnig , in the middle and in the end of the file (500MB). fsync.c int main(int argc, char** argv) { int fd, i, N; long time_before_fsync; char buf[1024]; int file_size = 500 * 1024 * 1024; if (argc < 2 ) {prinf("Usage: fsync "); return 0;} sscanf(argv[2],"%i",&N); N = N *1024; fd = open (argv[1], O_RDWR, 0777); for(i=0; i > > "Yury Y.Rupasov" wrote: > >> Hans Reiser wrote: >> >> > This is not what I meant. I meant what is the time to use the >> > system call fsync on a 500mb file when a few of its blocks are >> > dirty. >> > This affects database performance, syslog performance, etc. >> >> Here is the fsync time of 500mb file when any file parts are out of >> cache. >> >> reiserfs 1.768 seconds >> ext2 15.625 seconds >> ext2fs_fsync_time / reiserfs_fsync_time = 8.838 >> >> Fsync time are almost the same for both file systems when file has >> been accessed recently. >> So, for the case, when there are only few dirty blocks of file the >> results will be between these points. >> >> I wrote the program fsync, did mkreiserfs, mount, make_file f500 >> (size=500MB), then : >> >> $cat /dev/hda5 >/dev/null >> $time fsync /testfs/f500 >> >> Then I did the same for ext2fs. >> >> Without "cat /dev/hda5 >/dev/null " reiserfs and ext2fs do >> fsync < 1 second (reiserfs=0.273, ext2fs=0.554). >> >> fsync.c >> int main(int argc, char** argv) >> { >> int fd; >> fd = open (argv[1], O_RDWR, 0777); >> >> printf("time_before_fsync: %u\n",time(NULL)); >> fsync(fd); >> printf("time_after_fsync: %u\n",time(NULL)); >> >> close(fd); >> } >> >> best, >> Yura. >> >> > >> > >> >> Hello, Hans >> >> >> >> See attached files for test source. >> >> >> >> reiserfs create 500mb file >> >> real 159.42 >> >> user 19.08 >> >> sys 59.83 >> >> >> >> reiserfs remove 500mb file >> >> real 0.78 >> >> user 0.03 >> >> sys 0.35 >> >> >> >> ext2fs create 500mb file >> >> real 161.10 >> >> user 29.46 >> >> sys 71.53 >> >> >> >> ext2fs remove 500mb file >> >> real 2.30 >> >> user 0.04 >> >> sys 0.61 >> >> >> >> best, >> >> Yura. >> >> >> >> Hans Reiser wrote: >> >> >> >> > Yura, would you measure our performance when doing fsync on a >> >> > 500mb file, and compare it to ext2? >> >> > fsync performance is a current linux kernel thread.... >> >> > I have no idea what it is for reiserfs.... >> >> > >> >> > Thanks, >> >> > >> >> > Hans >> >> > >> >> > "Manuel J. Galan" wrote: >> >> > >> >> >> What happened with ReiserFS ? >> >> >> >> >> >> - >> >> >> To unsubscribe from this list: send the line "unsubscribe >> >> >> linux-kernel" in >> >> >> the body of a message to majordomo@vger.rutgers.edu >> >> >> Please read the FAQ at http://www.tux.org/lkml/ >> >> > >> >> > Hans >> >> > >> >> > -- >> >> > Don't be locked out of the source, and doomed to life in the slow lane. >> >> > Dump NT! Get Linux (http://www.kernel.org) plus ReiserFS >> >> > (http://devlinux.org/namesys). If you sell an OS or >> >> > internet appliance, buy a port of ReiserFS! >> >> > Speed matters. Trees are fast. Go faster! >> >> > >> >> > >> >> >> >> ----------------------------------------------------------- >> >> #!/bin/bash >> >> >> >> if [ $# -lt 1 ] >> >> then >> >> echo Usage: sync_test log_file_name >> >> exit >> >> fi >> >> >> >> LOGFILE=$1 >> >> >> >> umount /testfs >> >> mkreiserfs /dev/hda5 >> >> mount -t reiserfs /dev/hda5 /testfs >> >> >> >> echo "reiserfs create 500mb file">> $LOGFILE >> >> (time -p create_file )>> $LOGFILE 2>&1 >> >> >> >> echo "reiserfs remove 500mb file">> $LOGFILE >> >> (time -p remove_file )>> $LOGFILE 2>&1 >> >> >> >> sleep 30 >> >> umount /testfs >> >> mke2fs /dev/hda5 >> >> mount -t ext2 /dev/hda5 /testfs >> >> >> >> echo "ext2fs create 500mb file">> $LOGFILE >> >> (time -p create_file )>> $LOGFILE 2>&1 >> >> >> >> echo "ext2fs remove 500mb file">> $LOGFILE >> >> (time -p remove_file )>> $LOGFILE 2>&1 >> >> >> >> ----------------------------------------------------------- >> >> #!/bin/bash >> >> mkfile /testfs/f500mb 524288000 1048576 >> >> sync >> >> >> >> ----------------------------------------------------------- >> >> #!/bin/bash >> >> rm /testfs/f500mb >> >> sync >> >> >> > -- >> > Don't be locked out of the source, and doomed to life in the slow lane. >> > Dump NT! Get Linux (http://www.kernel.org) plus ReiserFS >> > (http://devlinux.org/namesys). If you sell an OS or >> > internet appliance, buy a port of ReiserFS! >> > Speed matters. Trees are fast. Go faster! >> > >> > >> > -- > Don't be locked out of the source, and doomed to life in the slow lane. > Dump NT! Get Linux (http://www.kernel.org) plus ReiserFS > (http://devlinux.org/namesys). If you sell an OS or > internet appliance, buy a port of ReiserFS! > Speed matters. Trees are fast. Go faster! > > --------------D86624808F200806661362A7 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit Hans Reiser wrote:
What about when you write the 500mb file, flush the cache, modify (dirty) a few of its blocks (e.g. the last one), and then fsync?
Here are the results of new benchmark:  fsync time  for  the 500MB file when a part of the file buffers are dirty buffers.
 
3Mb of dirty buffers:   0.6 %  of the file size
reiserfs fsync time  (in sec): 1
ext2fs   fsync time  (in sec): 1

6Mb of dirty buffers:  1.2%  of the file size
reiserfs fsync time  (in sec): 1
ext2fs   fsync time  (in sec): 1

15Mb of dirty buffers:  3% of the file size
reiserfs fsync time  (in sec): 2
ext2fs   fsync time  (in sec): 16

60Mb of dirty buffers:  12% of the file size
reiserfs fsync time  (in sec): 2
ext2fs   fsync time  (in sec): 16

I changed fsync.c so that it can do lseek() and write() before fsync().
The program writes  N * 1024 of 1KB buffers in the beginnig , in the middle and  in the end  of the file (500MB).

fsync.c
int main(int argc, char** argv)
{
  int fd, i, N;
  long time_before_fsync;
  char buf[1024];
  int file_size = 500 * 1024 * 1024;
 
  if (argc < 2 ) {prinf("Usage: fsync  <name_of_file>  <nr_buffers_to_dirty>"); return 0;}
  sscanf(argv[2],"%i",&N);
  N =  N *1024;
 
  fd = open (argv[1], O_RDWR, 0777);
  for(i=0;  i<N;  i++) { lseek(fd, i*1024, SEEK_SET); write(fd, buf, sizeof(buf));  }
  for(i=0;  i<N;  i++) { lseek(fd, file_size / 2+ i*1024, SEEK_SET); write(fd, buf, sizeof(buf)); }
  for(i=0;  i<N;  i++) { lseek(fd, file_size-i*1024, SEEK_SET); write(fd, buf, sizeof(buf)); }
 
  time_before_fsync=time(NULL));
  fsync(fd);
  printf("fsync time (in sec): %u\n",(time(NULL)-time_bfore_fsync) );
 
  close(fd);
}
 

best,
Yura.

 

"Yury Y.Rupasov" wrote:

Hans Reiser wrote:
This is not what I meant.  I meant what is the time to use the system call fsync on a 500mb file when a few of its blocks are dirty.
This affects database performance, syslog performance, etc.
Here is the fsync time of 500mb file when any file parts are out of cache.

reiserfs    1.768 seconds
ext2        15.625 seconds
ext2fs_fsync_time  /  reiserfs_fsync_time = 8.838

Fsync time are almost the same for both file systems when file has been accessed recently.
So, for the case, when there are only few dirty blocks of file the results will be between these points.

I wrote the program fsync,  did mkreiserfs, mount, make_file f500 (size=500MB), then  :

$cat  /dev/hda5 >/dev/null
$time fsync /testfs/f500

Then I did the same for ext2fs.

Without  "cat  /dev/hda5  >/dev/null "  reiserfs and ext2fs do fsync  < 1 second (reiserfs=0.273, ext2fs=0.554).

fsync.c
int main(int argc, char** argv)
{
  int fd;
  fd = open (argv[1], O_RDWR, 0777);

  printf("time_before_fsync: %u\n",time(NULL));
  fsync(fd);
  printf("time_after_fsync: %u\n",time(NULL));

  close(fd);
}

best,
Yura.

 
Hello, Hans

See attached files for test source.

reiserfs create 500mb file
real 159.42
user 19.08
sys 59.83

reiserfs remove 500mb file
real 0.78
user 0.03
sys 0.35

ext2fs create 500mb file
real 161.10
user 29.46
sys 71.53

ext2fs remove 500mb file
real 2.30
user 0.04
sys 0.61

best,
Yura.

Hans Reiser wrote:

Yura, would you measure our performance when doing fsync on a 500mb file, and compare it to ext2?
fsync performance is a current linux kernel thread....
I have no idea what it is for reiserfs....

Thanks,

Hans

"Manuel J. Galan" wrote:

What happened with ReiserFS ?

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

Hans
-- 
Don't be locked out of the source, and doomed to life in the slow lane.
Dump NT! Get Linux (http://www.kernel.org) plus ReiserFS
 (http://devlinux.org/namesys).  If you sell an OS or
internet appliance, buy a port of ReiserFS!
Speed matters.  Trees are fast.  Go faster!
 

#!/bin/bash if [ $# -lt 1 ] then        echo Usage: sync_test log_file_name         exit fi LOGFILE=$1 umount /testfs mkreiserfs /dev/hda5 mount -t reiserfs /dev/hda5 /testfs echo "reiserfs create 500mb file">> $LOGFILE (time -p create_file )>> $LOGFILE 2>&1 echo "reiserfs remove 500mb file">> $LOGFILE (time -p remove_file )>> $LOGFILE 2>&1 sleep 30 umount /testfs mke2fs /dev/hda5 mount -t ext2 /dev/hda5 /testfs echo "ext2fs create 500mb file">> $LOGFILE (time -p create_file )>> $LOGFILE 2>&1 echo "ext2fs remove 500mb file">> $LOGFILE (time -p remove_file )>> $LOGFILE 2>&1

#!/bin/bash mkfile /testfs/f500mb 524288000 1048576 sync

#!/bin/bash rm /testfs/f500mb  sync
-- 
Don't be locked out of the source, and doomed to life in the slow lane.
Dump NT! Get Linux (http://www.kernel.org) plus ReiserFS
 (http://devlinux.org/namesys).  If you sell an OS or
internet appliance, buy a port of ReiserFS!
Speed matters.  Trees are fast.  Go faster!
 
-- 
Don't be locked out of the source, and doomed to life in the slow lane.
Dump NT! Get Linux (http://www.kernel.org) plus ReiserFS
 (http://devlinux.org/namesys).  If you sell an OS or
internet appliance, buy a port of ReiserFS!
Speed matters.  Trees are fast.  Go faster!
 
--------------D86624808F200806661362A7-- --------------BE9387A5C2D81E0792BF1DD4-- - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/