Messages in this thread | | | Date | Tue, 18 Aug 2015 22:28:47 -0700 | Subject | Re: [PATCH v3 22/51] PCI: Add alt_size allocation support | From | Yinghai Lu <> |
| |
On Mon, Aug 17, 2015 at 5:03 PM, Bjorn Helgaas <bhelgaas@google.com> wrote: > On Mon, Jul 27, 2015 at 04:29:40PM -0700, Yinghai Lu wrote: >> On system with several pcie switches, BIOS allocate very tight resources >> to the bridge bar, and it is not aligned to min_align as kernel allocation >> code. > > I can't parse this.
BIOS allocate resource in different way.
kernel is trying to find smallest align (min_align) and use it to get aligned min_size.
> >> For example: >> 02:03.0---0c:00.0---0d:04.0---18:00.0 >> 18:00.0 need 0x10000000, and 0x00010000. >> BIOS only allocate 0x10100000 to 0d:04.0 and above bridges. > > Do you mean the BIOS only allocated 0x10010000?
I can not find the exact bus layout on hand. only one similar ...
23 13:15:49 kernel: pci_bus 0000:10: scanning bus Jun 23 13:15:49 kernel: pci 0000:10:00.0: [xxxx:xxxx] type 00 class 0x028000 Jun 23 13:15:49 kernel: pci 0000:10:00.0: reg 0x10: [mem 0xb0000000-0xbfffffff 64bit pref] Jun 23 13:15:49 kernel: pci 0000:10:00.0: reg 0x18: [mem 0xc0000000-0xc000ffff 64bit pref] Jun 23 13:15:49 kernel: pci_bus 0000:10: fixups for bus Jun 23 13:15:49 kernel: pci 0000:05:04.0: PCI bridge to [bus 10-17] Jun 23 13:15:49 kernel: pci 0000:05:04.0: bridge window [mem 0xb0000000-0xc00fffff] Jun 23 13:15:49 kernel: pci_bus 0000:10: bus scan returning with max=10
so device is using 0x10000000 and 0x00010000 and bridge is 0x10100000
As the bridge MMIO need to be aligned to 1M.
> >> Later after using /sys/bus/pci/devices/0000:0c:00.0/remove to remove 0c:00.0, >> rescan with /sys/bus/pci/rescan can not allocate 0x18000000 to 0c:00.0. >> >> another example: >> 00:1c.0-[02-21]----00.0-[03-21]--+-01.0-[04-12]----00.0-[05-12]----19.0-[06-12]----00.0 >> +-05.0-[13]-- >> +-07.0-[14-20]----00.0-[15-20]--+-08.0-[16]--+-00.0 >> | | \-00.1 >> | +-14.0-[17]----00.0 >> | \-19.0-[18-20]----00.0 >> \-09.0-[21]-- >> 06:00.0 need 0x4000000 and 0x800000. >> BIOS only allocate 0x4800000 to 05:19.0 and 04:00.0. >> when 05:19.0 get removed via /sys/bus/pci/devices/0000:05:19.0/remove, >> rescan with /sys/bus/pci/rescan will fail. >> pci 0000:05:19.0: BAR 14: no space for [mem size 0x06000000] >> pci 0000:05:19.0: BAR 14: failed to assign [mem size 0x06000000] >> pci 0000:06:00.0: BAR 2: no space for [mem size 0x04000000 64bit] >> pci 0000:06:00.0: BAR 2: failed to assign [mem size 0x04000000 64bit] >> pci 0000:06:00.0: BAR 0: no space for [mem size 0x00800000] >> pci 0000:06:00.0: BAR 0: failed to assign [mem size 0x00800000] >> current code try to use align 0x2000000 and size 0x6000000, but parent >> bridge only have 0x4800000. > > I *think* you're saying: > - BIOS assigned space for device X > - We remove X via sysfs > - We rescan via sysfs and discover X > - We try to assign space for X > - We fail because we don't use the same algorithm as BIOS did > > If there is an optimal way to assign space for an arbitrary number of > BARs, we could just adopt it. I don't know what that is, and I don't > know whether an optimal algorithm exists even in principle. > > If there is no single optimal algorithm, there will always be cases where > we fail because we use a different algorithm than the firmware did.
That is what this patch try to do.
alt_size solution that is preferring smaller size and big alignment.
Use it together with min_align solution that is used in kernel.
> >> Introduce alt_align/alt_size and store them in realloc list in addition >> to addon info, and will try it after min_align/min_size allocation fails. > > What does "alt" mean? >
alternative
| |