Messages in this thread | | | From | Andrew Theurer <> | Subject | Re: [PATCH] zerocopy NFS updated | Date | Fri, 19 Apr 2002 16:12:37 -0500 |
| |
With all this talk on serialization on UDP, and have a question. first, let me explain the situation. I have an NFS test which calls 48 clients to read the same 200 MB file on the same server. I record the time for all the clients to finish and then calculate the total throughput. The server is a 4-way IA32. (I used this test to measure the zerocopy/tcp/nfs patch) Now, right before the test, the 200 MB file is created on the server, so there is no disk IO at all during the test. It's just an very simple cached read. Now, when the clients use udp, I can only get a run queue length of 1, and I have confirmed there is only one nfsd thread in svc_process() at one time, and I am 65% idle. With tcp, I can get all nfsd threads running, and max all CPUs. Am I experiencing a bottleneck/serialization due to a single UDP socket?
Regards,
Andrew Theurer
jakob> Won't this serialize too much ? I mean, consider the situation where we jakob> have file-A and file-B completely in cache, while file-C needs to be jakob> read from the physical disk. jakob> jakob> Three different clients (A, B and C) request file-A, file-B and file-C jakob> respectively. The send of file-C is started first, and the sends of files jakob> A and B (which could commence immediately and complete at near wire-speed) jakob> will now have to wait (leaving the NIC idle) until file-C is read from jakob> the disks. jakob> jakob> Even if it's not the entire file but only a single NFS request (probably 8kB), jakob> one disk seek (7ms) is still around 85 kB, or 10 8kB NFS requests (at 100Mbit). jakob> jakob> Or am I misunderstanding ? Will your UDP sendpage() queue the requests ?
No problem. On my implementation, at the beginning a knfsd grabs all pages -- a part of file-C -- to reply to the NFS client. After that the knfsd starts to send them. It won't block any other knfsds during disk I/Os.
Thank you, Hirokazu Takahashi.
On Friday 19 April 2002 04:18 am, Trond Myklebust wrote: > On Friday 19. April 2002 05:21, Hirokazu Takahashi wrote: > > And it seems to be more important on UDP sendfile(). > > processes or threads sharing the same UDP socket would affect each other, > > while processes or threads on TCP sockets don't care about it as TCP > > connection is peer to peer. > > No. It is not the lack of peer-to-peer connections that gives rise to the > bottleneck, but the idea of several threads multiplexing sendfile() through > a single socket. Given a bad program design, it can be done over TCP too. > > The conclusion is that the programmer really ought to choose a different > design. For multimedia streaming, for instance, it makes sense to use 1 UDP > socket per thread rather than to multiplex the output through one socket. > > Cheers, > Trond > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |