[lkml]   [2006]   [Jan]   [31]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    SubjectRe: iommu_alloc failure and panic
    On Tue, 2006-01-31 at 08:41 -0500, James Smart wrote:
    > >> 2) The emulex driver has been prone to problems in the past where it's
    > >> been very aggressive at starting DMA operations, and I think it can
    > >> be avoided with tuning. What I don't know is if it's because of this,
    > >> or simply because of the large number of targets you have. Cc:ing James
    > >> Smart.
    > I don't have data points for the 2.6 kernel, but I can comment on what I
    > have seen on the 2.4 kernel.
    > The issue that I saw on the 2.4 kernel was that the pci dma alloc routine
    > was inappropriately allocating from the dma s/g maps. On systems with less
    > than 4Gig of memory, or on those with no iommmu (emt64), the checks around
    > adapter-supported dma masks were off (I'm going to be loose in terms to not
    > describe it in detail). The result was, although the adapter could support
    > a fully 64bit address and/or although the physical dma address would be under
    > 32-bits, the logic forced allocation from the mapped dma pool. On some
    > systems, this pool was originally only 16MB. Around 2.4.30, the swiotlb was
    > introduced, which reduced issue, but unfortunately, still never solved the
    > allocation logic. It fails less as the swiotlb simply had more space.
    > As far as I know, this problem doesn't exist in the 2.6 kernel. I'd have to
    > go look at the dma map functions to make sure.
    > Why was the lpfc driver prone to the dma map exhaustion failures ? Due to the
    > default # of commands per lun and max sg segments reported by the driver to
    > the scsi midlayer, the scsi mid-layer's preallocation of dma maps for commands
    > for each lun, and the fact that our FC configs were usually large, had lots
    > of luns, and replicated the resources for each path to the same storage.
    > Ultimately, what I think is the real issue here is the way the scsi mid-layer
    > is preallocating dma maps for the luns. 16000 luns is a huge number.
    > Multiply this by a max sg segment count of 64 by the driver, and a number
    > between 3 and 30 commands per lun, and you can see the numbers. Scsi does do
    > some interesting allocation algorithms once it hits an allocation failure.
    > One side effect of this is that it is fairly efficient at allocating the
    > bulk of the dma pool.


    Thanks for the information. I tried loading the lpfc driver with
    lpfc_lun_queue_depth=1 and haven't seen iommu_alloc failures. I'm still
    curious why the alloc failures lead to a panic though.


    > -- james s
    Mark Haverkamp <>

    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to
    More majordomo info at
    Please read the FAQ at

     \ /
      Last update: 2006-01-31 22:36    [W:0.022 / U:64.712 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site