Messages in this thread | | | Date | Sun, 4 Jun 2006 22:52:58 +0200 | From | Willy TARREAU <> | Subject | Re: [patch] epoll use unlocked wqueue operations ... |
| |
Hi Davide,
On Sat, Jun 03, 2006 at 10:35:52AM -0700, Davide Libenzi wrote: > On Sat, 3 Jun 2006, Willy Tarreau wrote: > > >Hi Davide, > > > >On Fri, Jun 02, 2006 at 04:28:25PM -0700, Davide Libenzi wrote: > >> > >>A few days ago Arjan signaled a lockdep red flag on epoll locks, and > >>precisely between the epoll's device structure lock (->lock) and the > >>wait > >>queue head lock (->lock). Like I explained in another email, and > >>directly > >>to Arjan, this can't happen in reality because of the explicit check at > >>eventpoll.c:592, that does not allow to drop an epoll fd inside the same > >>epoll fd. Since lockdep is working on per-structure locks, it will never > >>be able to know of policies enforced in other parts of the code. It was > >>decided time ago of having the ability to drop epoll fds inside other > >>epoll fds, that triggers a very trick wakeup operations (due to possibly > >>reentrant callback-driven wakeups) handled by the ep_poll_safewake() > >>function. > >>While looking again at the code though, I noticed that all the > >>operations > >>done on the epoll's main structure wait queue head (->wq) are already > >>protected by the epoll lock (->lock), so that locked-style functions can > >>be used to manipulate the ->wq member. This makes both a lock-acquire > >>save, and lockdep happy. > >>Running totalmess on my dual opteron for a while did not reveal any > >>problem so far: > >> > >>http://www.xmailserver.org/totalmess.c > > > >Shouldn't we notice a tiny performance boost by avoiding those useless > >locks, or do you consider they are not located in the fast path anyway ? > > Well, we take a lock less but I can't say if it'll be measureable. The > test program above is not a performance thing though, just some code to > verify multiple threads doing waits on the same epoll fd.
OK, so I ported your patch to 2.4 (+epoll-lt-0.22) because I have some code using it right there. At first, I thought I was observing measuring errors, but after about 6 reboots, I seem to observe a consistent 6.5% increase in the number of sessions/s on my fake web server on my dual athlon. It jumps from 14350 hits/s with epoll-lt-0.22 alone to 15300 with your patch. It seems much to me, but I'm sure I'm not dreaming (yet).
I'll send you (offlist) an update to 2.4-epoll-lt-0.22 which incorporates this patch.
> - Davide
Cheers, Willy
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |