lkml.org 
[lkml]   [2008]   [Oct]   [24]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: default IRQ affinity change in v2.6.27 (breaking several SMP PPC based systems)
Kumar Gala wrote:
>
> On Oct 24, 2008, at 11:09 AM, Chris Snook wrote:
>
>> Kumar Gala wrote:
>>> On Oct 24, 2008, at 10:17 AM, Chris Snook wrote:
>>>> Kumar Gala wrote:
>>>>> It appears the default IRQ affinity changes from being just cpu 0
>>>>> to all cpu's. This breaks several PPC SMP systems in which only a
>>>>> single processor is allowed to be selected as the destination of
>>>>> the IRQ.
>>>>> What is the right answer in fixing this? Should we:
>>>>> cpumask_t irq_default_affinity = 1;
>>>>> instead of
>>>>> cpumask_t irq_default_affinity = CPU_MASK_ALL?
>>>>
>>>> On those systems, perhaps, but not universally. There's plenty of
>>>> hardware where the physical topology of the machine is abstracted
>>>> away from the OS, and you need to leave the mask wide open and let
>>>> the APIC figure out where to map the IRQs. Ideally, we should
>>>> probably make this decision based on the APIC, but if there's no PPC
>>>> hardware that uses this technique, then it would suffice to make
>>>> this arch-specific.
>>> What did those systems do before this patch? Its one thing to expose
>>> a mask in the ability to change the default mask in
>>> /proc/irq/default_smp_affinity. Its another (and a regression in my
>>> opinion) to change the mask value itself.
>>
>> Before the patch they took an extremely long time to boot if they had
>> storage attached to each node of a multi-chassis system, performed
>> poorly unless special irqbalance hackery or manual assignment was
>> used, and imposed artificial restrictions on the granularity of
>> hardware partitioning to ensure that CPU 0 would always be a CPU that
>> could service all interrupts necessary to boot the OS.
>>
>>> As for making it ARCH specific, that doesn't really help since not
>>> all PPC hw has the limitation I spoke of. Not even all MPIC (in our
>>> cases) have the limitation.
>>
>> What did those systems do before this patch? :)
>>
>> Making it arch-specific is an extremely simple way to solve your
>> problem without making trouble for the people who wanted this patch in
>> the first place. If PPC needs further refinement to handle particular
>> *PICs, you can implement that without touching any arch-generic code.
>
>
> So why not just have x86 startup code set irq_default_affinity =
> CPU_MASK_ALL than?

It's an issue on Itanium as well, and potentially any SMP architecture with a
non-trivial interconnect.

-- Chris


\
 
 \ /
  Last update: 2008-10-24 19:53    [W:0.058 / U:1.640 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site