Messages in this thread | | | From | jlnance@unity ... | Date | Tue, 20 Jan 2004 08:28:03 -0500 | Subject | Re: Awful NFS performance with attached test program |
| |
On Mon, Jan 19, 2004 at 04:53:46PM -0500, Trond Myklebust wrote:
> So you are surprised that writing the same dataset by putting one > integer onto each kernel page takes much more time than placing the > entire dataset onto just a few kernel pages? 'cos I'm not...
I must admit that I am. I could see that it would take somewhat longer because a logicial way for the kernel to implement this would be as a read-modify-write operation. So a 2X slowdown would not supprise me. But the slowdown is more than 10X, and that does.
Also, for what its worth, Solaris performs like this:
flame> ./a.out Creating file: 3.886 seconds Updating file: 1.259 seconds
While Linux performs like this:
jesse> ./a.out Creating file: 43.042 seconds Updating file: 555.796 seconds
Both machines are writing to the same directory (not at the same time) on an x86_64 Linux server running a kernel which identifies itself as 2.4.21-4.ELsmp. The network connection between the Solaris machine and the NFS server actually seems to be slightly slower than that between the Linux machine and the NFS server:
sledge> ping flame PING flame.synopsys.com (10.48.2.191) 56(84) bytes of data. 64 bytes from flame.synopsys.com (10.48.2.191): icmp_seq=0 ttl=255 time=0.174 ms 64 bytes from flame.synopsys.com (10.48.2.191): icmp_seq=1 ttl=255 time=0.171 ms 64 bytes from flame.synopsys.com (10.48.2.191): icmp_seq=2 ttl=255 time=0.168 ms
sledge> ping jesse PING jesse.synopsys.com (10.48.2.120) 56(84) bytes of data. 64 bytes from jesse.synopsys.com (10.48.2.120): icmp_seq=0 ttl=64 time=0.115 ms 64 bytes from jesse.synopsys.com (10.48.2.120): icmp_seq=1 ttl=64 time=0.113 ms 64 bytes from jesse.synopsys.com (10.48.2.120): icmp_seq=2 ttl=64 time=0.104 ms
so I dont think the descrepency is due to network problems.
I should note that Solaris takes 135 seconds to close the file while Linux takes almost no time. This probably means that Solaris has postponed some of the work it needs to do until close time. However, even with the 135 seconds factored in, Solaris is still killing us.
I would like to try this with a 2.6 kernel, but that is difficult. Ill see what I can put together though.
> Have a look at the nfsstat output and you'll see what I mean. Doing the > former will necessarily end up generating *a lot* more NFS write > requests.
Thanks for the info. I didnt know about nfsstat. It looks like a useful utility.
Thanks,
Jim - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |