Messages in this thread | | | Date | Wed, 4 Jan 2023 07:47:03 -0600 | Subject | Re: [PATCH v2] arm64: gic: increase the number of IRQ descriptors | From | Shanker Donthineni <> |
| |
Hi Marc,
On 1/4/23 03:14, Marc Zyngier wrote: > External email: Use caution opening links or attachments > > > On Wed, 04 Jan 2023 02:37:38 +0000, > Shanker Donthineni <sdonthineni@nvidia.com> wrote: >> >> The default value of NR_IRQS is not sufficient to support GICv4.1 >> features and ~56K LPIs. This parameter would be too small for certain >> server platforms where it has many IO devices and is capable of >> direct injection of vSGI and vLPI features. >> >> Currently, maximum of 64 + 8192 (IRQ_BITMAP_BITS) IRQ descriptors >> are allowed. The vCPU creation fails after reaching count ~400 with >> kvm-arm.vgic_v4_enable=1. >> >> This patch increases NR_IRQS to 1^19 to cover 56K LPIs and 262144 >> vSGIs (16K vPEs x 16). >> >> Signed-off-by: Shanker Donthineni <sdonthineni@nvidia.com> >> --- >> Changes since v1: >> -create from v6.2-rc1 and edit commit text >> >> arch/arm64/include/asm/irq.h | 4 ++++ >> 1 file changed, 4 insertions(+) >> >> diff --git a/arch/arm64/include/asm/irq.h b/arch/arm64/include/asm/irq.h >> index fac08e18bcd5..3fffc0b8b704 100644 >> --- a/arch/arm64/include/asm/irq.h >> +++ b/arch/arm64/include/asm/irq.h >> @@ -4,6 +4,10 @@ >> >> #ifndef __ASSEMBLER__ >> >> +#if defined(CONFIG_ARM_GIC_V3_ITS) >> +#define NR_IRQS (1 << 19) >> +#endif >> + >> #include <asm-generic/irq.h> >> >> struct pt_regs; > > Sorry, but I don't think this is an acceptable change. This is a large > overhead that affects *everyone*, and that will eventually be too > small anyway with larger systems and larger interrupt spaces. > > A better way to address this would be to move to a more dynamic > allocation, converting the irqdesc rb-tree into an xarray, getting rid > of the bitmaps (the allocation bitmap and the resend one), and track > everything in the xarray.
The actual memory allocation for IRQ descriptors is still dynamic for ARM64. This change increases static memory for variable 'allocated_irqs' by 64KB, feel not a noticeable overhead.
If 64KB is too high, can we change NR_IRQS to 65536.
#ifdef CONFIG_SPARSE_IRQ # define IRQ_BITMAP_BITS (NR_IRQS + 8196) #else # define IRQ_BITMAP_BITS NR_IRQS #endif
static DECLARE_BITMAP(allocated_irqs, IRQ_BITMAP_BITS);
For ARM64, CONFIG_SPARSE_IR is set to y.
| |