lkml.org 
[lkml]   [2011]   [Oct]   [25]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [PATCH 3/3] Add NumaChip quirk
On 10/25/2011 16:38, Bjorn Helgaas wrote:
> On Tue, Oct 18, 2011 at 2:22 AM, Daniel J Blueman
> <daniel@numascale-asia.com> wrote:
>> Add quirk for Numascale's NumaChip to prevent resource conflicts.
[]
> This feels like a band-aid ... what's the background here? I can see
> that you're disabling PCI resources, and I can read that this avoids
> conflicts, but what's the underlying cause of the conflict?
>
> I wonder if there's a more generic problem that should be fixed
> differently. Presumably the NumaChip designers put those BARs there
> for a reason, and often when we report "conflicts," it's really a clue
> that we're doing something wrong in host bridge discovery or in
> generic PCI.
>
> Can you post a complete dmesg log showing the conflict?
>

Hi Bjorn,

The issue is a bit complicated, but here's the story; NumaChip is a
coherent NorthBridge device on AMD systems (i.e part of the coherent
fabric) but the BIOS does *not* assign any resources to it, in fact the
BIOS skips our device entirely leaving our BAR registers at HW init
values (0x00000000). This is in fact by design (AMD AGESA code). This
isn't really a big issue for us anyway because we have other means of
reaching our CSR logic (not going into detail, but the other patches in
this patchset would reveal how we do that).

Linux however, when scanning the PCI buses finds our device (because it
is responding to config space requests) it thinks that we got a BAR0
that starts at 0x00000000 which obviously isn't correct. In addition, in
the bootloader that we've written for NumaChip systems (to bring them
all together as a huge coherent system) we had to use the expansion rom
config space register (F0x030) as kind of a "scratch register".

The end result is :

[ 4.636297] pci 0000:00:1a.0: reg 10: [mem 0x00000000-0x000fffff]
[ 4.640317] pci 0000:00:1a.0: reg 30: [mem 0x3fff0000-0x3fffffff pref]
[ 4.644369] pci 0000:00:1a.1: [1b47:0602] type 0 class 0x000600

Neither of these resources are real, our device will not respond to
requests to any of those windows.

The conflict we refer to in the patch is that since Linux thinks we have
those windows assigned to us, we get conflicts later on with real devices :

[ 5.887856] pnp 00:0e: disabling [mem 0x00000000-0x0009ffff] because
it overlaps 0000:00:1a.0 BAR 0 [mem 0x00000000-0x000fffff]
[ 5.899525] pnp 00:0e: disabling [mem 0x000c0000-0x000cffff] because
it overlaps 0000:00:1a.0 BAR 0 [mem 0x00000000-0x000fffff]
[ 5.911002] pnp 00:0e: disabling [mem 0x000e0000-0x000fffff] because
it overlaps 0000:00:1a.0 BAR 0 [mem 0x00000000-0x000fffff]


I guess technically, the Linux PCI bus probing code should check the
Command register (offset 0x4) to see if MemorySpace is enabled (which in
our case it won't be) before checking the BAR registers.

From my perspective however, it seemed easier and less intrusive to
just hook in a quirk for our device since we do not use those BAR
settings anyway in any form once Linux is booted.

I hope this clear things up a bit. If you have any other questions don't
hesitate to ask.

Kind regards,
--
Steffen Persvold, Chief Architect NumaChip
Numascale AS - www.numascale.com
Tel: +47 92 49 25 54 Skype: spersvold


\
 
 \ /
  Last update: 2011-10-25 18:05    [W:0.080 / U:0.204 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site