lkml.org 
[lkml]   [2015]   [Sep]   [29]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
From
Subject[PATCH 3/5] percpu_ref: unify staggered atomic switching wait behavior
Date
When an atomic or percpu switching starts before the previous atomic
switching finishes, the taken behaviors are

* If the new atomic switching has confirmation callback, it waits
for the previous atomic switching to complete.

* If the new percpu switching is the first percpu switching following
the previous atomic switching, it waits the previous atomic
switching to complete.

No percpu_ref user depends on these subtleties. The only meaningful
part is that, if the caller ensures that atomic switching isn't in
progress, mode switching operations can be issued from any context.

This patch pulls the wait logic to the top of both switching functions
so that they always wait for the previous atomic switching to
complete. This makes the behavior simpler and consistent for both
directions and will help allowing concurrent invocations of mode
switching functions.

Signed-off-by: Tejun Heo <tj@kernel.org>
---
lib/percpu-refcount.c | 31 +++++++++++++++++++++----------
1 file changed, 21 insertions(+), 10 deletions(-)

diff --git a/lib/percpu-refcount.c b/lib/percpu-refcount.c
index 622e93b..6a36597 100644
--- a/lib/percpu-refcount.c
+++ b/lib/percpu-refcount.c
@@ -161,15 +161,19 @@ static void percpu_ref_noop_confirm_switch(struct percpu_ref *ref)
static void __percpu_ref_switch_to_atomic(struct percpu_ref *ref,
percpu_ref_func_t *confirm_switch)
{
+ /*
+ * If the previous ATOMIC switching hasn't finished yet, wait for
+ * its completion. If the caller ensures that ATOMIC switching
+ * isn't in progress, this function can be called from any context.
+ * Do an extra confirm_switch test to circumvent the unconditional
+ * might_sleep() in wait_event().
+ */
+ if (ref->confirm_switch)
+ wait_event(percpu_ref_switch_waitq, !ref->confirm_switch);
+
if (ref->percpu_count_ptr & __PERCPU_REF_ATOMIC) {
- if (confirm_switch) {
- /*
- * Somebody else already set ATOMIC. Wait for its
- * completion and invoke @confirm_switch() directly.
- */
- wait_event(percpu_ref_switch_waitq, !ref->confirm_switch);
+ if (confirm_switch)
confirm_switch(ref);
- }
return;
}

@@ -180,7 +184,6 @@ static void __percpu_ref_switch_to_atomic(struct percpu_ref *ref,
* Non-NULL ->confirm_switch is used to indicate that switching is
* in progress. Use noop one if unspecified.
*/
- WARN_ON_ONCE(ref->confirm_switch);
ref->confirm_switch = confirm_switch ?: percpu_ref_noop_confirm_switch;

percpu_ref_get(ref); /* put after confirmation */
@@ -192,13 +195,21 @@ static void __percpu_ref_switch_to_percpu(struct percpu_ref *ref)
unsigned long __percpu *percpu_count = percpu_count_ptr(ref);
int cpu;

+ /*
+ * If the previous ATOMIC switching hasn't finished yet, wait for
+ * its completion. If the caller ensures that ATOMIC switching
+ * isn't in progress, this function can be called from any context.
+ * Do an extra confirm_switch test to circumvent the unconditional
+ * might_sleep() in wait_event().
+ */
+ if (ref->confirm_switch)
+ wait_event(percpu_ref_switch_waitq, !ref->confirm_switch);
+
BUG_ON(!percpu_count);

if (!(ref->percpu_count_ptr & __PERCPU_REF_ATOMIC))
return;

- wait_event(percpu_ref_switch_waitq, !ref->confirm_switch);
-
atomic_long_add(PERCPU_COUNT_BIAS, &ref->count);

/*
--
2.4.3


\
 
 \ /
  Last update: 2015-09-30 00:01    [W:0.056 / U:23.440 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site