lkml.org 
[lkml]   [2010]   [Aug]   [26]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    From
    Subject[RFC PATCH v2] Modularize IOMMUs detection/init for X86
    Date
    Per our discussion at: http://lkml.org/lkml/2010/8/2/282
    I've implemented an RFC set of patches to address the problem.

    This patch set adds a mechanism to "modularize" the IOMMUs we have
    on X86. Currently the count of IOMMUs is up to six and they have a complex
    relationship that requires careful execution order. 'pci_iommu_alloc'
    does that today, but most folks are unhappy with how it does it.
    This patch set addresses this and also paves a mechanism to jettison
    unused IOMMUs during run-time.

    The first solution that comes to mind is to convert wholesale
    the IOMMU detection routines to be called during initcall
    time frame. Unfortunately that misses the dependency relationship
    that some of the IOMMUs have (for example: for AMD-Vi IOMMU to work,
    GART detection MUST run first, and before all of that SWIOTLB MUST run).

    The second solution would be to introduce a registration call wherein
    the IOMMU would provide its detection/init routines and as well on what
    MUST run before it. That would work, except that the 'pci_iommu_alloc'
    which would run through this list, is called during mem_init. This means we
    don't have any memory allocator, and it is so early that we haven't yet
    started running through the initcall_t list.

    This solution borrows concepts from the 2nd idea and from how
    MODULE_INIT works. A macro is provided that each IOMMU uses to define
    it's detect function and early_init (before the memory allocate is
    active), and as well what other IOMMU MUST run before us. Since most IOMMUs
    depend on having SWIOTLB run first ("pci_swiotlb_detect") a convenience macro
    to depends on that is also provided.

    This macro is similar in design to MODULE_PARAM macro wherein
    we setup a .iommu_table section in which we populate it with the values
    that match a struct iommu_table_entry. During bootup we will sort
    through the array so that the IOMMUs that MUST run before us are first
    elements in the array. And then we just iterate through them calling the
    detection routine and if appropriate, the init routines.

    Testing:
    I've done testing on machines with Intel VT-d, SWIOTLB, and GART for regressions.
    I sadly don't have machines with the Calgary nor the AMD-Vi chipsets.

    Enhancements:

    Jeremy suggested that we could get rid of the macro and instead provide
    an registration API, such as:

    static void __init_early register_my_iommu() {

    register_iommu(&my_iommu_details);
    }

    where the __init_early would put the address of the function in an
    -1 level of .initcall list. This -1 level of init that would run
    right before the pci_iommu_alloc is called (or perhaps during earlier
    setup). The register_iommu would have a static list allocated, and the
    structure passed in would have a 'list_head' value that we would use to
    stitch the structs together. Then the 'sort_iommu' function would take care of
    sorting the elements in order of dependency.

    This patchset is also available on git:
    git://git.kernel.org/pub/scm/linux/kernel/git/konrad/swiotlb-2.6.git devel/iommu-0.2

    arch/x86/include/asm/amd_iommu.h | 4 +-
    arch/x86/include/asm/calgary.h | 4 +-
    arch/x86/include/asm/gart.h | 5 +-
    arch/x86/include/asm/iommu_table.h | 101 ++++++++++++++++++++++++++++++++++++
    arch/x86/include/asm/swiotlb.h | 13 ++++-
    arch/x86/kernel/Makefile | 1 +
    arch/x86/kernel/amd_iommu_init.c | 16 ++++--
    arch/x86/kernel/aperture_64.c | 11 +++--
    arch/x86/kernel/pci-calgary_64.c | 18 ++++---
    arch/x86/kernel/pci-dma.c | 44 ++++++++--------
    arch/x86/kernel/pci-gart_64.c | 2 +
    arch/x86/kernel/pci-iommu_table.c | 90 ++++++++++++++++++++++++++++++++
    arch/x86/kernel/pci-swiotlb.c | 45 +++++++++++++---
    arch/x86/kernel/vmlinux.lds.S | 7 +++
    arch/x86/xen/pci-swiotlb-xen.c | 5 ++
    drivers/pci/dmar.c | 6 ++-
    include/linux/dmar.h | 6 +-
    17 files changed, 321 insertions(+), 57 deletions(-)


    \
     
     \ /
      Last update: 2010-08-26 20:01    [W:6.340 / U:0.008 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site