lkml.org 
[lkml]   [2019]   [Feb]   [24]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
SubjectRe: [PATCH] iommu/vt-d: Handle hotplug devices' default identity mapping setting
From
Date
Hi James,

On 2/22/19 3:36 PM, James Dong wrote:
> Baolu:
>
> Sorry that my last reply email seems not text format. Resend it now.
>
> Thanks for your comments and your patch. Please find below our responses to
> each of your comments:
>
>> What does "I/O operation won't work" exactly mean here? Do you see any
>> IOMMU fault message? Or, something doesn't work as expected?
>
> Yes, DMAR fault messages as following came out:
> [ 354.939896] DMAR: DMAR:[DMA Read] Request device [03:00.1]fault addr 1fdfe80000
> [ 354.939896] DMAR:[fault reason 02] Present bit in context entry is clear
>
>
>> Do you mind checking this?
>>
>> index 6ecdcf8fc8c0..f62f30bc1339 100644
>> --- a/drivers/iommu/intel-iommu.c
>> +++ b/drivers/iommu/intel-iommu.c
>> @@ -2632,6 +2632,9 @@ static struct dmar_domain
>> *find_or_alloc_domain(struct device *dev, int gaw)
>> goto out;
>> }
>>
>> + if (!iommu_should_identity_map(dev, 0))
>> + return si_domain;
>> +
>> /* Allocate and initialize new domain for the device */
>> domain = alloc_domain(0);
>> if (!domain)
>
> Tried this patch, and the same DMAR fault message came out.

Thank you!

>
> Guess it is because of the iommu code path for hotplug devices. If a hotplug
> device is rescanned after removal, iommu_bus_notifier will be called as part
> of the notifier chains to handle BUS_NOTIFY_ADD_DEVICE event. Along the code
> path, intel_iommu_ops->add_device() created an iommu group for this hotplug
> device, but failed to create an iommu domain because of the default domain
> type IOMMU_DOMAIN_IDENTITY imposed by current IOMMU command line option got
> declined by intel_iommu_ops->domain_alloc().

The Intel IOMMU driver hasn't switched to default domain yet although
it's in the pipe line. So, there should be no domain allocated when a
group is allocated for the device.

The problem is we need to check whether a hot-added device requires
identity map instead of allocating a normal domain blindly.

>
> Since si_domain is type of "struct dmar_domain", which is platform dependent,
> it is hard to make this change in intel_iommu_ops->domain_alloc().
>
> In your patch, function find_or_alloc_domain() is not in the code path of
> BUS_NOTIFY_ADD_DEVICE event notifier chain.
>
> Please let us know if your have more concerns and suggestions.

Can you please try the patch attached? I think this is a generic issue
as I described in the commit message.

Best regards,
Lu Baolu
From d942e60557fc7ea6fee535fb9a0a7d334d65b636 Mon Sep 17 00:00:00 2001
From: Lu Baolu <baolu.lu@linux.intel.com>
Date: Sun, 24 Feb 2019 10:01:03 +0800
Subject: [PATCH 1/1] iommu/vt-d: Check identity map for hot-added devices

The Intel IOMMU driver will put devices into a static identity
mapped domain during boot if the kernel parameter "iommu=pt" is
used. That means the IOMMU hardware will translate a DMA address
into the same memory address.

Unfortunately, a hot-added device doesn't subject to this. That
results in some devices not working properly after hot added. A
quick way to reproduce this issue is to boot a system with

iommu=pt

and, remove then readd the pci device with

echo 1 > /sys/bus/pci/devices/[pci_source_id]/remove
echo 1 > /sys/bus/pci/rescan

You will find the identity mapped domain was replaced with a
normal domain.

Cc: Ashok Raj <ashok.raj@intel.com>
Cc: Jacob Pan <jacob.jun.pan@linux.intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
drivers/iommu/intel-iommu.c | 27 +++++++++++++++------------
1 file changed, 15 insertions(+), 12 deletions(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 6ecdcf8fc8c0..730ee29d561b 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -3001,9 +3001,9 @@ static int iommu_should_identity_map(struct device *dev, int startup)
}

/*
- * At boot time, we don't yet know if devices will be 64-bit capable.
- * Assume that they will — if they turn out not to be, then we can
- * take them out of the 1:1 domain later.
+ * At boot time or hot added, we don't yet know if devices will be
+ * 64-bit capable. Assume that they will — if they turn out not to
+ * be, then we can take them out of the 1:1 domain later.
*/
if (!startup) {
/*
@@ -4807,16 +4807,19 @@ static int device_notifier(struct notifier_block *nb,
if (iommu_dummy(dev))
return 0;

- if (action != BUS_NOTIFY_REMOVED_DEVICE)
- return 0;
-
- domain = find_domain(dev);
- if (!domain)
- return 0;
+ if (action == BUS_NOTIFY_REMOVED_DEVICE) {
+ domain = find_domain(dev);
+ if (!domain)
+ return 0;

- dmar_remove_one_dev_info(dev);
- if (!domain_type_is_vm_or_si(domain) && list_empty(&domain->devices))
- domain_exit(domain);
+ dmar_remove_one_dev_info(dev);
+ if (!domain_type_is_vm_or_si(domain) &&
+ list_empty(&domain->devices))
+ domain_exit(domain);
+ } else if (action == BUS_NOTIFY_ADD_DEVICE) {
+ if (iommu_should_identity_map(dev, 1))
+ domain_add_dev_info(si_domain, dev);
+ }

return 0;
}
--
2.17.1
\
 
 \ /
  Last update: 2019-02-24 10:05    [W:0.111 / U:0.796 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site