Messages in this thread | | | From | Kevin Hilman <> | Subject | Re: [RFC PATCH 0/3] workqueue: Control cpu affinity of !WQ_SYSFS unbound workqueues | Date | Fri, 14 Mar 2014 11:06:06 -0700 |
| |
Frederic Weisbecker <fweisbec@gmail.com> writes:
> There are several types of workqueues. Some of them are bound to specific > CPUs, some others are unbound and can be executed on any CPU. > > A tiny subset of the unbound workqueues have a sysfs representation > in /sys/devices/virtual/workqueue/ and have a cpumask file than can > be used to tweak their CPU affinity. > > But the vast majority of unbound workqueues aren't visible in sysfs. > > They are a problem nowaday because people who work on CPU isolation > (HPC, Real time, etc...) want to be able to migrate all the unbound > workqueues away from specific CPUs. > > There are several possible solutions to solve this: > > 1) Affine the !WQ_SYSFS unbound workqueues to the CPUs outside the > full dynticks mask. Full dynticks is expected to be a component in > many CPU isolation configurations and its CPU mapping can be a good > way to retrieve the desired set of isolated CPUs. > > On the drawbacks though we can notice the lack of consistency with > WQ_SYSFS workqueue affinity interface, issues with ordering between > workqueue and dynticks subsystem initialization, intrusion from > the workqueue subsystem on dynticks internals. > > 2) Implement a sysfs directory for each unbound !WQ_SYSFS. That sounds > like a very nice solution as it uses existing and known interface. > But workqueues appearing in the sysfs hierarchy are subject to become > stable ABIs. And this is definetly not what we want. > > This could be worked around with a specific Kconfig to make sure that > these workqueues won't be considered as a stable ABI. But we all know > that all distros will enable this Kconfig symbol and that nobody > reads, nor care about, warnings in Kconfig help text which thereby won't > protect us against anything. > > 3) Implement a single sysfs directory to map properties of all !WQ_SYSFS > unbound workqueues. It would contain only the cpumask file to control > the affinity of all these workqueues. But more can be added later. > > This complexifies the code a bit although not that much compared to > solution 2 which requires some plumbling to cope with workqueues created > before sysfs, as reported by Mike (I played with that a bit as well, as I > took that direction initially). But it deals with all issues previously > described.
Since I also tinkered with (1) and (2) and ran into some of the same issues, I think (3) is a good way. It also doesn't tie the affinity to full_nohz, and leaves it up to userspace which addresses a concern of Mike's in earlier proposals as well.
Kevin
| |