Messages in this thread | | | From | (Eric W. Biederman) | Date | Wed, 14 Dec 2011 17:25:19 -0800 | Subject | Re: IRQ migration on CPU offline path |
| |
Will Deacon <will.deacon@arm.com> writes:
> Hi Thomas, > > I've been looking at the IRQ migration code on x86 (fixup_irqs) for the CPU > hotplug path in order to try and fix a bug we have on ARM with the > desc->affinity mask not getting updated. Compared to irq_set_affinity, the code > is pretty whacky (I guess because it's called via stop_machine) so I wondered if > you could help me understand a few points:
There is a lot of craziness on that path because of poor hardware design on x86 we can't know when an irq has actually be migrated, and other nasties.
There is also the issue that I expect is still the case that we have the generic layer asking us to cpu migration and the associated irq migrations with the irqs disabled which at least for the bits of poorly designed hardware made the entire path a best effort beast.
> (1) Affinity notifiers - we seem to ignore these and I guess they don't expect > to be called from this context. It could lead to the cpu_rmap stuff being > bogus, but that's not used for anything much. Do we just depend on people > having hotplug notifiers to deal with this?
> (2) Threaded handlers - I can't see where we update irqaction->thread_flags with > IRQTF_AFFINITY for handlers that are migrated by the scheduler when a CPU > goes down. Is this required? > > (3) On x86, we rely on the irq_chip updating the desc->affinity mask in > ->irq_set_affinity. It seems like we could use IRQ_SET_MASK_OK{_NOCOPY} for > this and, in the case of the ioapic, return IRQ_SET_MASK_OK_NOCOPY (removing > a redundant copy from the usual affinity path).
Possibly. I forget which direction this code has gone, but we have the interesting case that frequently on x86 we can be pinned to multiple and the hardware only supports being pinned to a single cpu at a time. So the original logic was to put in the affinity mask what we had actually done instead of what was requested. And in that case the mask changes so a NOCOPY doesn't feel appropriate but I don't understand that code.
> Of course, I could just be completely confused, which is why I haven't started > hacking code just yet :)
If x86 becomes a good clean example in this corner case I would be amazed. Last I looked I almost marked it all as CONFIG_BROKEN because we were trying to do the impossible. Unfortunately peoples laptops go through this path when they suspend and so it was more painful to disable hacky racy mess than to keep living with it.
There has been an increase in the number of cases where it is possible to actually perform the migration with irqs disabled so on a good day that code might even work.
Eric
| |