lkml.org 
[lkml]   [2005]   [Nov]   [7]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRE: rpc-srv/tcp: nfsd: sent only -107 bytes (fwd)


On Mon, 7 Nov 2005, Roger Heflin wrote:
>>
>> I am seeing repeated errors of rpc-srv/tcp: nfsd: sent only
>> -107 bytes in the /var/log/messages of my machine. Full
>> configuration info is below. Only suggestion I have seen
>> thus far increase the number of nfsd that are running. we
>> have done this, raising from 8 to 64, the problem persists.
>> Are there any other suggestions that could help this problem?
>>
>> Thanks
>>
>
> I have only seen this problem with large numbers of NFS clients, given
> your address I suspect that would be the issue.

wc -l on /var/lib/nfs/rmtab gives 744 lines. There are 203 client
machines, each mounting on the average of 3 mounts, but obviously
some mount a few more. Most of the mounts are done via automount, which
as you can guess is timing out on the average of 10-12 attempts in
the course of the day.
>
> What kernel are you running? I saw this issue on the 2.4 series, the
> large 2.6 things we have built avoided using TCP given that we had
> seen this issue.

Currently running 2.4.21-32.0.1 as put out by RedHat with XFS file
system patches. will upgrade to 2.4.21-37 later this week.
>
> There is a calculation someplace in nfs that determines how many of these
> things exist, is is some base + so many per nfs thread. If you
> search for message you have there are some posts on the NFS lists
> about it that I made last year.
>
> On the client side each separate mount against a server counts as
> one, so if each client is mounting /opt and /home and /data and
> you have 100 machines you need at least 300.
>
Are you suggesting in such a scenario we would need 300 nfsd?

> The solution that I came to was to use UDP mounts, as this limit
> is not there. In the situation I had we would have had to change
> the number of nfsd to 256 and even that was going to be close,
> and the 256 caused some other failures. To not have the issue you
> will need to use UDP mounts everywere if you have enough tcp mounts
> to cause the error it will affect the udp mounts in a similar bad way.
>
> We also could have changed the thread to resource count, but we had
> some other process starvation issue with TCP that seemed to not be
> duplicatable with UDP.

We had changed from UDP to TCP due to our network's propensity to drop
UDP packets. The same mount options we are using now against a Linux
server were working fine when our NFS server was a much-less-powerful SGI
IRIX server. No doubt, as you say, that switching back to UDP would
make this rpc-srv/tcp message go away.

Steve Timm



>
> Roger
> Atipa Technologies
>
>

--
------------------------------------------------------------------
Steven C. Timm, Ph.D (630) 840-8525 timm@fnal.gov http://home.fnal.gov/~timm/
Fermilab Computing Div/Core Support Services Dept./Scientific Computing Section
Assistant Group Leader, Farms and Clustered Systems Group
Lead of Computing Farms Team
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-11-07 17:02    [W:0.073 / U:0.152 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site