lkml.org 
[lkml]   [2022]   [Apr]   [21]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    Date
    From
    SubjectRe: [PATCH] irqchip/armada-370-xp: Enable MSI affinity configuration
    Hi Nathan,

    On Thu, 21 Apr 2022 02:57:28 +0100,
    Nathan Rossi <nathan@nathanrossi.com> wrote:
    >
    > From: Nathan Rossi <nathan.rossi@digi.com>
    >
    > With multiple devices attached via PCIe to an Armada 385 it is possible
    > to overwhelm a single CPU with MSI interrupts. Under certain scenarios
    > configuring the interrupts to be handled by more than one CPU would
    > prevent the system from being overwhelmed. However the
    > irqchip-aramada-370-xp driver is configured to only handle MSIs on the
    > boot CPU, and provides no affinity configuration.
    >
    > This change adds support to the armada-370-xp driver to allow for
    > configuring the affinity of specific MSI irqs and to generate the
    > interrupts on secondary CPUs. This is done by enabling the private
    > doorbell for all online CPUs and configures all CPUs to unmask MSI
    > specific private doorbell bits. The CPU affinity selection of the
    > interrupt is handled by the target list of the software triggered
    > interrupt value, which is provided as the MSI message. The message has
    > the associated CPU bit set for the target CPU. For private doorbell
    > interrupts only one bit can be set otherwise all CPUs will receive the
    > interrupt, so the lowest CPU in the affinity mask is used. This means
    > that by default the first CPU will handle all the interrupts as was the
    > case before.
    >
    > Signed-off-by: Nathan Rossi <nathan.rossi@digi.com>
    > ---
    > drivers/irqchip/irq-armada-370-xp.c | 34 ++++++++++++++++++++++++++++++++--
    > 1 file changed, 32 insertions(+), 2 deletions(-)
    >
    > diff --git a/drivers/irqchip/irq-armada-370-xp.c b/drivers/irqchip/irq-armada-370-xp.c
    > index 5b8d571c04..42c257f576 100644
    > --- a/drivers/irqchip/irq-armada-370-xp.c
    > +++ b/drivers/irqchip/irq-armada-370-xp.c
    > @@ -209,15 +209,37 @@ static struct msi_domain_info armada_370_xp_msi_domain_info = {
    >
    > static void armada_370_xp_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
    > {
    > +#ifdef CONFIG_SMP
    > + unsigned int cpu = cpumask_first(irq_data_get_effective_affinity_mask(data));
    > +
    > + msg->data = (1 << (cpu + 8)) | (data->hwirq + PCI_MSI_DOORBELL_START);

    BIT(cpu + 8) | ...

    > +#else
    > + msg->data = 0xf00 | (data->hwirq + PCI_MSI_DOORBELL_START);

    This paints the existing code a bit differently. This seems to target
    all 4 CPUs. Why is that? I'd expect only bit 8 to be set, and the
    whole #ifdefery to go away.

    > +#endif
    > msg->address_lo = lower_32_bits(msi_doorbell_addr);
    > msg->address_hi = upper_32_bits(msi_doorbell_addr);
    > - msg->data = 0xf00 | (data->hwirq + PCI_MSI_DOORBELL_START);
    > }
    >
    > static int armada_370_xp_msi_set_affinity(struct irq_data *irq_data,
    > const struct cpumask *mask, bool force)
    > {
    > - return -EINVAL;
    > +#ifdef CONFIG_SMP
    > + unsigned int cpu;
    > +
    > + if (!force)
    > + cpu = cpumask_any_and(mask, cpu_online_mask);
    > + else
    > + cpu = cpumask_first(mask);
    > +
    > + if (cpu >= nr_cpu_ids)
    > + return -EINVAL;
    > +
    > + irq_data_update_effective_affinity(irq_data, cpumask_of(cpu));
    > +
    > + return IRQ_SET_MASK_OK;
    > +#else
    > + return -EINVAL;
    > +#endif
    > }
    >
    > static struct irq_chip armada_370_xp_msi_bottom_irq_chip = {
    > @@ -482,6 +504,7 @@ static void armada_xp_mpic_smp_cpu_init(void)
    > static void armada_xp_mpic_reenable_percpu(void)
    > {
    > unsigned int irq;
    > + u32 reg;
    >
    > /* Re-enable per-CPU interrupts that were enabled before suspend */
    > for (irq = 0; irq < ARMADA_370_XP_MAX_PER_CPU_IRQS; irq++) {
    > @@ -501,6 +524,13 @@ static void armada_xp_mpic_reenable_percpu(void)
    > }
    >
    > ipi_resume();
    > +
    > + /* Enable MSI doorbell mask and combined cpu local interrupt */
    > + reg = readl(per_cpu_int_base + ARMADA_370_XP_IN_DRBEL_MSK_OFFS)
    > + | PCI_MSI_DOORBELL_MASK;
    > + writel(reg, per_cpu_int_base + ARMADA_370_XP_IN_DRBEL_MSK_OFFS);
    > + /* Unmask local doorbell interrupt */
    > + writel(1, per_cpu_int_base + ARMADA_370_XP_INT_CLEAR_MASK_OFFS);

    This is a duplicate of what is already in armada_370_xp_msi_init().
    Please refactor it so that this doesn't happen twice on the first CPU.

    This otherwise seem like a valuable improvement on the current
    behaviour,

    M.

    --
    Without deviation from the norm, progress is not possible.
    \
     
     \ /
      Last update: 2022-04-21 08:56    [W:2.868 / U:0.024 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site