lkml.org 
[lkml]   [2000]   [Aug]   [28]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
Date
From
Subject[patch] waitqueue optimization, 2.4.0-test7

a substantial percentage of __wake_up() calls are done on empty waitqueues
- the percentage in a typical system is around 50% but can be higher under
various loads. The attached waitqueue-2.4.0-test7-A1 patch optimizes
wake_up() to check wether the waitqueue is empty before calling
__wake_up(). This optimization is especially useful on SMP systems - the
waitqueue spinlock is not taken thus causes no cacheline ping-pong. But
there are benefits on UP systems as well, the cli/sti pair in __wake_up is
not executed in these cases and we avoid the function call as well.

the wake_up() variants could be optimized further if we didnt allow NULL
pointers being passed to wake_up() - but this i think is a 2.5 item as it
changes the wake_up() interface. I've done this optimization too, and
there are not that many places that pass NULL to wake_up() - but it
happens often enough to cause trouble if done now.

there is the question of synchronization with the waitqueue lock on SMP
systems - the waitqueue_active() check goes outside the waitqueue
spinlock, but i think this is safe. I didnt see any problems whatsoever.

Ingo
--- linux/include/linux/sched.h.orig Mon Aug 28 15:05:02 2000
+++ linux/include/linux/sched.h Mon Aug 28 15:11:41 2000
@@ -534,13 +534,31 @@
signed long timeout));
extern void FASTCALL(wake_up_process(struct task_struct * tsk));

-#define wake_up(x) __wake_up((x),TASK_UNINTERRUPTIBLE | TASK_INTERRUPTIBLE | TASK_EXCLUSIVE)
-#define wake_up_all(x) __wake_up((x),TASK_UNINTERRUPTIBLE | TASK_INTERRUPTIBLE)
-#define wake_up_sync(x) __wake_up_sync((x),TASK_UNINTERRUPTIBLE | TASK_INTERRUPTIBLE | TASK_EXCLUSIVE)
-#define wake_up_interruptible(x) __wake_up((x),TASK_INTERRUPTIBLE | TASK_EXCLUSIVE)
-#define wake_up_interruptible_all(x) __wake_up((x),TASK_INTERRUPTIBLE)
-#define wake_up_interruptible_sync(x) __wake_up_sync((x),TASK_INTERRUPTIBLE | TASK_EXCLUSIVE)
+/*
+ * Subtle. We skip the wakeup if the queue is empty, but we do
+ * not synchronize with the waitqueue spinlock. The reason for
+ * this is performance, a big percentage of wakeups goes to empty
+ * waitqueues. The effect of this is that we might not notice
+ * 'just being added' entries, but this is not a problem, it's
+ * effectively the same as if this CPU was 'very fast'.
+ */
+#define _wake_up(q,mode) \
+ do { if (q && waitqueue_active(q)) __wake_up((q), (mode)); } while (0)
+#define _wake_up_sync(q,mode) \
+ do { if (q && waitqueue_active(q)) __wake_up_sync((q), (mode)); } while (0)

+#define wake_up(q) \
+ _wake_up((q),TASK_UNINTERRUPTIBLE | TASK_INTERRUPTIBLE | TASK_EXCLUSIVE)
+#define wake_up_all(q) \
+ _wake_up((q),TASK_UNINTERRUPTIBLE | TASK_INTERRUPTIBLE)
+#define wake_up_sync(q) \
+ _wake_up_sync((q),TASK_UNINTERRUPTIBLE | TASK_INTERRUPTIBLE | TASK_EXCLUSIVE)
+#define wake_up_interruptible(q) \
+ _wake_up((q),TASK_INTERRUPTIBLE | TASK_EXCLUSIVE)
+#define wake_up_interruptible_all(q) \
+ _wake_up((q),TASK_INTERRUPTIBLE)
+#define wake_up_interruptible_sync(q) \
+ _wake_up_sync((q),TASK_INTERRUPTIBLE | TASK_EXCLUSIVE)
extern int in_group_p(gid_t);
extern int in_egroup_p(gid_t);
\
 
 \ /
  Last update: 2005-03-22 12:38    [from the cache]
©2003-2011 Jasper Spaans