Messages in this thread Patch in this message | | | Date | Mon, 28 Aug 2000 15:49:24 +0200 (CEST) | From | Ingo Molnar <> | Subject | [patch] waitqueue optimization, 2.4.0-test7 |
| |
a substantial percentage of __wake_up() calls are done on empty waitqueues - the percentage in a typical system is around 50% but can be higher under various loads. The attached waitqueue-2.4.0-test7-A1 patch optimizes wake_up() to check wether the waitqueue is empty before calling __wake_up(). This optimization is especially useful on SMP systems - the waitqueue spinlock is not taken thus causes no cacheline ping-pong. But there are benefits on UP systems as well, the cli/sti pair in __wake_up is not executed in these cases and we avoid the function call as well.
the wake_up() variants could be optimized further if we didnt allow NULL pointers being passed to wake_up() - but this i think is a 2.5 item as it changes the wake_up() interface. I've done this optimization too, and there are not that many places that pass NULL to wake_up() - but it happens often enough to cause trouble if done now.
there is the question of synchronization with the waitqueue lock on SMP systems - the waitqueue_active() check goes outside the waitqueue spinlock, but i think this is safe. I didnt see any problems whatsoever.
Ingo --- linux/include/linux/sched.h.orig Mon Aug 28 15:05:02 2000 +++ linux/include/linux/sched.h Mon Aug 28 15:11:41 2000 @@ -534,13 +534,31 @@ signed long timeout)); extern void FASTCALL(wake_up_process(struct task_struct * tsk)); -#define wake_up(x) __wake_up((x),TASK_UNINTERRUPTIBLE | TASK_INTERRUPTIBLE | TASK_EXCLUSIVE) -#define wake_up_all(x) __wake_up((x),TASK_UNINTERRUPTIBLE | TASK_INTERRUPTIBLE) -#define wake_up_sync(x) __wake_up_sync((x),TASK_UNINTERRUPTIBLE | TASK_INTERRUPTIBLE | TASK_EXCLUSIVE) -#define wake_up_interruptible(x) __wake_up((x),TASK_INTERRUPTIBLE | TASK_EXCLUSIVE) -#define wake_up_interruptible_all(x) __wake_up((x),TASK_INTERRUPTIBLE) -#define wake_up_interruptible_sync(x) __wake_up_sync((x),TASK_INTERRUPTIBLE | TASK_EXCLUSIVE) +/* + * Subtle. We skip the wakeup if the queue is empty, but we do + * not synchronize with the waitqueue spinlock. The reason for + * this is performance, a big percentage of wakeups goes to empty + * waitqueues. The effect of this is that we might not notice + * 'just being added' entries, but this is not a problem, it's + * effectively the same as if this CPU was 'very fast'. + */ +#define _wake_up(q,mode) \ + do { if (q && waitqueue_active(q)) __wake_up((q), (mode)); } while (0) +#define _wake_up_sync(q,mode) \ + do { if (q && waitqueue_active(q)) __wake_up_sync((q), (mode)); } while (0) +#define wake_up(q) \ + _wake_up((q),TASK_UNINTERRUPTIBLE | TASK_INTERRUPTIBLE | TASK_EXCLUSIVE) +#define wake_up_all(q) \ + _wake_up((q),TASK_UNINTERRUPTIBLE | TASK_INTERRUPTIBLE) +#define wake_up_sync(q) \ + _wake_up_sync((q),TASK_UNINTERRUPTIBLE | TASK_INTERRUPTIBLE | TASK_EXCLUSIVE) +#define wake_up_interruptible(q) \ + _wake_up((q),TASK_INTERRUPTIBLE | TASK_EXCLUSIVE) +#define wake_up_interruptible_all(q) \ + _wake_up((q),TASK_INTERRUPTIBLE) +#define wake_up_interruptible_sync(q) \ + _wake_up_sync((q),TASK_INTERRUPTIBLE | TASK_EXCLUSIVE) extern int in_group_p(gid_t); extern int in_egroup_p(gid_t);
| |