Messages in this thread | | | Date | Thu, 1 Jul 2010 10:35:35 -0700 | Subject | Re: [patch 00/52] vfs scalability patches updated | From | Linus Torvalds <> |
| |
On Wed, Jun 30, 2010 at 5:40 AM, Nick Piggin <npiggin@suse.de> wrote: >> >> That's a pretty big ouch. Why does RCU freeing of inodes cause that >> much regression? The RCU freeing is out of line, so where does the big >> impact come from? > > That comes mostly from inability to reuse the cache-hot inode structure, > and the cost to go over the deferred RCU list and free them after they > get cache cold.
I do wonder if this isn't a big design bug.
Most of the time with RCU, we don't need to wait to actually do the _freeing_ of the individual data structure, we only need to make sure that the data structure remains of the same _type_. IOW, we can free it (and re-use it), but the backing storage cannot be released to the page cache. That's what SLAB_DESTROY_BY_RCU should give us.
Is that not possible in this situation? Do we really need to keep the inode _identity_ around for RCU?
If you use just SLAB_DESTROY_BY_RCU, then inode re-use remains, and cache behavior would be much improved. The usual requirement for SLAB_DESTROY_BY_RCU is that you only touch a lock (and perhaps re-validate the identity) in the RCU-reader paths. Could that be made to work?
Because that 27% drop really is pretty distressing.
That said, open (of the non-creating kind), close, and stat are certainly more important than creating and freeing files. So as a trade-off, it's probably the right thing to do. But if we can get all the improvement _without_ that big downside, that would obviously be better yet.
Linus
| |