lkml.org 
[lkml]   [2009]   [Nov]   [3]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
SubjectRe: [PATCH 1/3] sysfs directory scaling: rbtree for dirent name lookups
From
Date
Greg KH <greg@kroah.com> writes:

> On Tue, Nov 03, 2009 at 07:14:33AM +0100, Eric Dumazet wrote:
>> Greg KH a ?crit :
>> > On Sun, Nov 01, 2009 at 11:31:30AM -0500, Benjamin LaHaise wrote:
>> >> Use an rbtree in sysfs_dirent to speed up file lookup times
>> >>
>> >> Systems with large numbers (tens of thousands and more) of network
>> >> interfaces stress the sysfs code in ways that make the linear search for
>> >> a name match take far too long. Avoid this by using an rbtree.
>> >
>> > What kind of speedups are you seeing here? And do these changes cause a
>> > memory increase due to the structure changes which outweigh the
>> > speedups?
>> >
>> > What kind of test are you doing to reproduce this?
>> >
>>
>> Its curious because in my tests the biggest problems come from
>> kernel/sysctl.c (__register_sysctl_paths) consuming 80% of cpu
>> in following attempt to create 20.000 devices
>>
>> (disable hotplug before trying this, and ipv6 too !)
>> modprobe dummy numdummies=20000
>>
>> I believe we should address __register_sysctl_paths() scalability
>> problems too.
>
> But registering 20000 devices is a far different problem from using
> those 20000 devices :)
>
> I think the "use the device" path should be the one we care the most
> about fixing up, as that is much more common than the register path for
> all users.

Definitely. Of the three proc sysctl and sysfs. sysctl tends to have
the worst costs across the board. They are all rarely used so a lot
of what gets hit when scaling are rare path events that even the most
horrible code works fine on small systems.

Usually slow registration times indicate an O(N^2) or worse data
structure for filename lookup.

Eric


\
 
 \ /
  Last update: 2009-11-03 23:31    [W:0.154 / U:0.164 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site