lkml.org 
[lkml]   [2002]   [Apr]   [19]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
SubjectRe: [PATCH] zerocopy NFS updated
Date
With all this talk on serialization on UDP, and have a question.  first, let 
me explain the situation. I have an NFS test which calls 48 clients to read
the same 200 MB file on the same server. I record the time for all the
clients to finish and then calculate the total throughput. The server is a
4-way IA32. (I used this test to measure the zerocopy/tcp/nfs patch) Now,
right before the test, the 200 MB file is created on the server, so there is
no disk IO at all during the test. It's just an very simple cached read.
Now, when the clients use udp, I can only get a run queue length of 1, and I
have confirmed there is only one nfsd thread in svc_process() at one time,
and I am 65% idle. With tcp, I can get all nfsd threads running, and max all
CPUs. Am I experiencing a bottleneck/serialization due to a single UDP
socket?

Regards,

Andrew Theurer

jakob> Won't this serialize too much ? I mean, consider the situation where
we
jakob> have file-A and file-B completely in cache, while file-C needs to be
jakob> read from the physical disk.
jakob>
jakob> Three different clients (A, B and C) request file-A, file-B and file-C
jakob> respectively. The send of file-C is started first, and the sends of
files
jakob> A and B (which could commence immediately and complete at near
wire-speed)
jakob> will now have to wait (leaving the NIC idle) until file-C is read from
jakob> the disks.
jakob>
jakob> Even if it's not the entire file but only a single NFS request
(probably 8kB),
jakob> one disk seek (7ms) is still around 85 kB, or 10 8kB NFS requests (at
100Mbit).
jakob>
jakob> Or am I misunderstanding ? Will your UDP sendpage() queue the
requests ?
No problem.
On my implementation, at the beginning a knfsd grabs all pages -- a part
of file-C -- to reply to the NFS client. After that the knfsd starts to
send them. It won't block any other knfsds during disk I/Os.

Thank you,
Hirokazu Takahashi.


On Friday 19 April 2002 04:18 am, Trond Myklebust wrote:
> On Friday 19. April 2002 05:21, Hirokazu Takahashi wrote:
> > And it seems to be more important on UDP sendfile().
> > processes or threads sharing the same UDP socket would affect each other,
> > while processes or threads on TCP sockets don't care about it as TCP
> > connection is peer to peer.
>
> No. It is not the lack of peer-to-peer connections that gives rise to the
> bottleneck, but the idea of several threads multiplexing sendfile() through
> a single socket. Given a bad program design, it can be done over TCP too.
>
> The conclusion is that the programmer really ought to choose a different
> design. For multimedia streaming, for instance, it makes sense to use 1 UDP
> socket per thread rather than to multiplex the output through one socket.
>
> Cheers,
> Trond
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 13:25    [from the cache]
©2003-2011 Jasper Spaans