Messages in this thread |  | | | Date | Sun, 27 Aug 2006 16:36:58 +0530 | | From | Dipankar Sarma <> | | Subject | Re: [RFC][PATCH 0/4] Redesign cpu_hotplug locking. |
| |
On Sun, Aug 27, 2006 at 12:42:13AM -0700, Andrew Morton wrote: > On Sun, 27 Aug 2006 12:41:16 +0530 > Dipankar Sarma <dipankar@in.ibm.com> wrote: > > > On Sat, Aug 26, 2006 at 11:46:18PM -0700, Andrew Morton wrote: > > > Yes you do. Please, read _cpu_up(), _cpu_down() and the example in > > > workqueue_cpu_callback(). It's really very simple. > > > > What are you talking about here ? > > Did you look? workqueue_mutex is used to protect per-cpu workqueue > resources. The lock is taken prior to modification of per-cpu resources > and is released after their modification. Very very simple.
I did and there is no lock named workqueue_mutex. workqueue_cpu_callback() is farily simple and doesn't have the issues in cpufreq that we are talking about (lock_cpu_hotplug() in cpu callback path).
> > > That is the write side. You are > > *not* supposed to do lock_cpu_hotplug() in cpu callbacks paths AFAICT. > > The workqueue code doesn't use lock_cpu_hotplug().
And that is the right thing to do.
> > I am talking about readsides here - you read cpu_online_map and > > block then reuse the map and make some calls to another subsystem > > that may again do a similar read-side cpu_hotplug lock. > > Two unrelated subsystems which have both independent and interdependent CPU > hotplug locking requirements and neither of which can protect per-cpu > resources via preempt_disable()? Sounds unlikely and undesirable.
I would worry about situations where we have to use set_cpus_allowed() with cpumasks. IIRC, those weren't trivial to handle and can happen due to interaction between unrelated subsystems one using services of the other - rtasd -> set_cpus_allowed() for example.
> > 1. If you are in cpu hotplug callback path, don't take any lock. > > That rule is wrong. The CPU_UP_PREPARE and CPU_DOWN_PREPARE notification > entrypoints are the logical place for a subsystem to lock any per-cpu resources > which another thread/cpu might presently be using.
I meant lock_cpu_hotplug(), not any lock. Of course, susbsystems may need to use their own lock there to handle per-cpu data there.
> > 2. If you are in a non-hotplug path reading cpu_online_map and you don't > > block, you just disable preemption and you are safe from hotplug. > > Sure. > > > 3. If you are in a non-hotplug path and you use cpu_online_map and > > you *really* need to block, you use lock_cpu_hotplug() or > > cpu_hotplug_disable whatever it is called. > > > > Is this too difficult for people to follow ? > > Apparently. What's happening is that lock_cpu_hotplug() is seen as some > amazing thing which will prevent an *event* from occurring. > > There's an old saying "lock data, not code". What data is being locked > here? It's the subsystem's per-cpu resources which we want to lock. We > shouldn't consider the lock as being some way of preventing an event from > happening.
That is what I argued for earlier, but I was given some examples where they really needed to disable the asynchronous event of cpu hotplug - otherwise they would have need to use very complex multi-layer locking.
> > > > seem to have just got lazy with lock_cpu_hotplug(). > > > > > > That's because lock_cpu_hotplug() purports to be some magical thing which > > > makes all your troubles go away. > > > > No it doesn't. Perhaps we should just document the rules better > > and put some static checks for people to get it right. > > Yes, we could probably fix cpufreq using the existing lock_cpu_hotplug(). > But we have a quite large amount of racy-wrt-cpu-hotplug code in the kernel > and although a lot of it can be fixed with preempt_disable(), it's possible > that we'll get into scalability problems. > > If we do have scalability problems, they can be fixed on a per-subsystem > basis: the affected subsystem can use per-cpu locking of its per-cpu data > within its CPU_UP_PREPARE and CPU_DOWN_PREPARE handlers. That's a local, > contained issue, and addressing it this way is better than inventing (and > debugging) some fancy new lock type.
I would suggest an audit of lock_cpu_hotlpug() users to start with.
Thanks Dipankar - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
|  |