summaryrefslogtreecommitdiff
path: root/drivers/iommu
AgeCommit message (Collapse)Author
2021-03-18iommu/tegra-smmu: Make tegra_smmu_probe_device() to handle all IOMMU phandlesDmitry Osipenko
The tegra_smmu_probe_device() handles only the first IOMMU device-tree phandle, skipping the rest. Devices like 3D module on Tegra30 have multiple IOMMU phandles, one for each h/w block, and thus, only one IOMMU phandle is added to fwspec for the 3D module, breaking GPU. Previously this problem was masked by tegra_smmu_attach_dev() which didn't use the fwspec, but parsed the DT by itself. The previous commit to tegra-smmu driver partially reverted changes that caused problems for T124 and now we have tegra_smmu_attach_dev() that uses the fwspec and the old-buggy variant of tegra_smmu_probe_device() which skips secondary IOMMUs. Make tegra_smmu_probe_device() not to skip the secondary IOMMUs. This fixes a partially attached IOMMU of the 3D module on Tegra30 and now GPU works properly once again. Fixes: 765a9d1d02b2 ("iommu/tegra-smmu: Fix mc errors on tegra124-nyan") Signed-off-by: Dmitry Osipenko <digetx@gmail.com> Tested-by: Nicolin Chen <nicoleotsuka@gmail.com> Link: https://lore.kernel.org/r/20210312155439.18477-1-digetx@gmail.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2021-03-18iommu/amd: Keep track of amd_iommu_irq_remap stateJoerg Roedel
The amd_iommu_irq_remap variable is set to true in amd_iommu_prepare(). But if initialization fails it is not set to false. Fix that and correctly keep track of whether irq remapping is enabled or not. References: https://bugzilla.kernel.org/show_bug.cgi?id=212133 References: https://bugzilla.suse.com/show_bug.cgi?id=1183132 Fixes: b34f10c2dc59 ("iommu/amd: Stop irq_remapping_select() matching when remapping is disabled") Cc: stable@vger.kernel.org # v5.11 Signed-off-by: Joerg Roedel <jroedel@suse.de> Link: https://lore.kernel.org/r/20210317091037.31374-4-joro@8bytes.org Acked-by: Huang Rui <ray.huang@amd.com>
2021-03-18iommu/amd: Don't call early_amd_iommu_init() when AMD IOMMU is disabledJoerg Roedel
Don't even try to initialize the AMD IOMMU hardware when amd_iommu=off has been passed on the kernel command line. References: https://bugzilla.kernel.org/show_bug.cgi?id=212133 References: https://bugzilla.suse.com/show_bug.cgi?id=1183132 Cc: stable@vger.kernel.org # v5.11 Signed-off-by: Joerg Roedel <jroedel@suse.de> Link: https://lore.kernel.org/r/20210317091037.31374-3-joro@8bytes.org Acked-by: Huang Rui <ray.huang@amd.com>
2021-03-18iommu/amd: Move Stoney Ridge check to detect_ivrs()Joerg Roedel
The AMD IOMMU will not be enabled on AMD Stoney Ridge systems. Bail out even earlier and refuse to even detect the IOMMU there. References: https://bugzilla.kernel.org/show_bug.cgi?id=212133 References: https://bugzilla.suse.com/show_bug.cgi?id=1183132 Cc: stable@vger.kernel.org # v5.11 Signed-off-by: Joerg Roedel <jroedel@suse.de> Link: https://lore.kernel.org/r/20210317091037.31374-2-joro@8bytes.org Acked-by: Huang Rui <ray.huang@amd.com>
2021-03-05Merge tag 'iommu-fixes-v5.12-rc1' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu Pull iommu fixes from Joerg Roedel: - Fix a sleeping-while-atomic issue in the AMD IOMMU code - Disable lazy IOTLB flush for untrusted devices in the Intel VT-d driver - Fix status code definitions for Intel VT-d - Fix IO Page Fault issue in Tegra IOMMU driver * tag 'iommu-fixes-v5.12-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: iommu/vt-d: Fix status code for Allocate/Free PASID command iommu: Don't use lazy flush for untrusted device iommu/tegra-smmu: Fix mc errors on tegra124-nyan iommu/amd: Fix sleeping in atomic in increase_address_space()
2021-03-04iommu/vt-d: Fix status code for Allocate/Free PASID commandZenghui Yu
As per Intel vt-d spec, Rev 3.0 (section 10.4.45 "Virtual Command Response Register"), the status code of "No PASID available" error in response to the Allocate PASID command is 2, not 1. The same for "Invalid PASID" error in response to the Free PASID command. We will otherwise see confusing kernel log under the command failure from guest side. Fix it. Fixes: 24f27d32ab6b ("iommu/vt-d: Enlightened PASID allocation") Signed-off-by: Zenghui Yu <yuzenghui@huawei.com> Acked-by: Lu Baolu <baolu.lu@linux.intel.com> Link: https://lore.kernel.org/r/20210227073909.432-1-yuzenghui@huawei.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2021-03-04iommu: Don't use lazy flush for untrusted deviceLu Baolu
The lazy IOTLB flushing setup leaves a time window, in which the device can still access some system memory, which has already been unmapped by the device driver. It's not suitable for untrusted devices. A malicious device might use this to attack the system by obtaining data that it shouldn't obtain. Fixes: c588072bba6b5 ("iommu/vt-d: Convert intel iommu driver to the iommu ops") Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Link: https://lore.kernel.org/r/20210225061454.2864009-1-baolu.lu@linux.intel.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2021-03-04iommu/tegra-smmu: Fix mc errors on tegra124-nyanNicolin Chen
Commit 25938c73cd79 ("iommu/tegra-smmu: Rework tegra_smmu_probe_device()") removed certain hack in the tegra_smmu_probe() by relying on IOMMU core to of_xlate SMMU's SID per device, so as to get rid of tegra_smmu_find() and tegra_smmu_configure() that are typically done in the IOMMU core also. This approach works for both existing devices that have DT nodes and other devices (like PCI device) that don't exist in DT, on Tegra210 and Tegra3 upon testing. However, Page Fault errors are reported on tegra124-Nyan: tegra-mc 70019000.memory-controller: display0a: read @0xfe056b40: EMEM address decode error (SMMU translation error [--S]) tegra-mc 70019000.memory-controller: display0a: read @0xfe056b40: Page fault (SMMU translation error [--S]) After debugging, I found that the mentioned commit changed some function callback sequence of tegra-smmu's, resulting in enabling SMMU for display client before display driver gets initialized. I couldn't reproduce exact same issue on Tegra210 as Tegra124 (arm-32) differs at arch-level code. Actually this Page Fault is a known issue, as on most of Tegra platforms, display gets enabled by the bootloader for the splash screen feature, so it keeps filling the framebuffer memory. A proper fix to this issue is to 1:1 linear map the framebuffer memory to IOVA space so the SMMU will have the same address as the physical address in its page table. Yet, Thierry has been working on the solution above for a year, and it hasn't merged. Therefore, let's partially revert the mentioned commit to fix the errors. The reason why we do a partial revert here is that we can still set priv in ->of_xlate() callback for PCI devices. Meanwhile, devices existing in DT, like display, will go through tegra_smmu_configure() at the stage of bus_set_iommu() when SMMU gets probed(), as what it did before we merged the mentioned commit. Once we have the linear map solution for framebuffer memory, this change can be cleaned away. [Big thank to Guillaume who reported and helped debugging/verification] Fixes: 25938c73cd79 ("iommu/tegra-smmu: Rework tegra_smmu_probe_device()") Reported-by: Guillaume Tucker <guillaume.tucker@collabora.com> Signed-off-by: Nicolin Chen <nicoleotsuka@gmail.com> Tested-by: Guillaume Tucker <guillaume.tucker@collabora.com> Acked-by: Thierry Reding <treding@nvidia.com> Link: https://lore.kernel.org/r/20210218220702.1962-1-nicoleotsuka@gmail.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2021-03-04iommu/amd: Fix sleeping in atomic in increase_address_space()Andrey Ryabinin
increase_address_space() calls get_zeroed_page(gfp) under spin_lock with disabled interrupts. gfp flags passed to increase_address_space() may allow sleeping, so it comes to this: BUG: sleeping function called from invalid context at mm/page_alloc.c:4342 in_atomic(): 1, irqs_disabled(): 1, pid: 21555, name: epdcbbf1qnhbsd8 Call Trace: dump_stack+0x66/0x8b ___might_sleep+0xec/0x110 __alloc_pages_nodemask+0x104/0x300 get_zeroed_page+0x15/0x40 iommu_map_page+0xdd/0x3e0 amd_iommu_map+0x50/0x70 iommu_map+0x106/0x220 vfio_iommu_type1_ioctl+0x76e/0x950 [vfio_iommu_type1] do_vfs_ioctl+0xa3/0x6f0 ksys_ioctl+0x66/0x70 __x64_sys_ioctl+0x16/0x20 do_syscall_64+0x4e/0x100 entry_SYSCALL_64_after_hwframe+0x44/0xa9 Fix this by moving get_zeroed_page() out of spin_lock/unlock section. Fixes: 754265bcab ("iommu/amd: Fix race in increase_address_space()") Signed-off-by: Andrey Ryabinin <arbn@yandex-team.com> Acked-by: Will Deacon <will@kernel.org> Cc: <stable@vger.kernel.org> Link: https://lore.kernel.org/r/20210217143004.19165-1-arbn@yandex-team.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2021-02-24Merge tag 'dma-mapping-5.12' of git://git.infradead.org/users/hch/dma-mappingLinus Torvalds
Pull dma-mapping updates from Christoph Hellwig: - add support to emulate processing delays in the DMA API benchmark selftest (Barry Song) - remove support for non-contiguous noncoherent allocations, which aren't used and will be replaced by a different API * tag 'dma-mapping-5.12' of git://git.infradead.org/users/hch/dma-mapping: dma-mapping: remove the {alloc,free}_noncoherent methods dma-mapping: benchmark: pretend DMA is transmitting
2021-02-22Merge tag 'iommu-updates-v5.12' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu Pull iommu updates from Joerg Roedel: - ARM SMMU and Mediatek updates from Will Deacon: - Support for MT8192 IOMMU from Mediatek - Arm v7s io-pgtable extensions for MT8192 - Removal of TLBI_ON_MAP quirk - New Qualcomm compatible strings - Allow SVA without hardware broadcast TLB maintenance on SMMUv3 - Virtualization Host Extension support for SMMUv3 (SVA) - Allow SMMUv3 PMU perf driver to be built independently from IOMMU - Some tidy-up in IOVA and core code - Conversion of the AMD IOMMU code to use the generic IO-page-table framework - Intel VT-d updates from Lu Baolu: - Audit capability consistency among different IOMMUs - Add SATC reporting structure support - Add iotlb_sync_map callback support - SDHI support for Renesas IOMMU driver - Misc cleanups and other small improvments * tag 'iommu-updates-v5.12' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: (94 commits) iommu/amd: Fix performance counter initialization MAINTAINERS: repair file pattern in MEDIATEK IOMMU DRIVER iommu/mediatek: Fix error code in probe() iommu/mediatek: Fix unsigned domid comparison with less than zero iommu/vt-d: Parse SATC reporting structure iommu/vt-d: Add new enum value and structure for SATC iommu/vt-d: Add iotlb_sync_map callback iommu/vt-d: Move capability check code to cap_audit files iommu/vt-d: Audit IOMMU Capabilities and add helper functions iommu/vt-d: Fix 'physical' typos iommu: Properly pass gfp_t in _iommu_map() to avoid atomic sleeping iommu/vt-d: Fix compile error [-Werror=implicit-function-declaration] driver/perf: Remove ARM_SMMU_V3_PMU dependency on ARM_SMMU_V3 MAINTAINERS: Add entry for MediaTek IOMMU iommu/mediatek: Add mt8192 support iommu/mediatek: Remove unnecessary check in attach_device iommu/mediatek: Support master use iova over 32bit iommu/mediatek: Add iova reserved function iommu/mediatek: Support for multi domains iommu/mediatek: Add get_domain_id from dev->dma_range_map ...
2021-02-21Merge tag 'hyperv-next-signed-20210216' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/hyperv/linux Pull Hyper-V updates from Wei Liu: - VMBus hardening patches from Andrea Parri and Andres Beltran. - Patches to make Linux boot as the root partition on Microsoft Hypervisor from Wei Liu. - One patch to add a new sysfs interface to support hibernation on Hyper-V from Dexuan Cui. - Two miscellaneous clean-up patches from Colin and Gustavo. * tag 'hyperv-next-signed-20210216' of git://git.kernel.org/pub/scm/linux/kernel/git/hyperv/linux: (31 commits) Revert "Drivers: hv: vmbus: Copy packets sent by Hyper-V out of the ring buffer" iommu/hyperv: setup an IO-APIC IRQ remapping domain for root partition x86/hyperv: implement an MSI domain for root partition asm-generic/hyperv: import data structures for mapping device interrupts asm-generic/hyperv: introduce hv_device_id and auxiliary structures asm-generic/hyperv: update hv_interrupt_entry asm-generic/hyperv: update hv_msi_entry x86/hyperv: implement and use hv_smp_prepare_cpus x86/hyperv: provide a bunch of helper functions ACPI / NUMA: add a stub function for node_to_pxm() x86/hyperv: handling hypercall page setup for root x86/hyperv: extract partition ID from Microsoft Hypervisor if necessary x86/hyperv: allocate output arg pages if required clocksource/hyperv: use MSR-based access if running as root Drivers: hv: vmbus: skip VMBus initialization if Linux is root x86/hyperv: detect if Linux is the root partition asm-generic/hyperv: change HV_CPU_POWER_MANAGEMENT to HV_CPU_MANAGEMENT hv: hyperv.h: Replace one-element array with flexible-array in struct icmsg_negotiate hv_netvsc: Restrict configurations on isolated guests Drivers: hv: vmbus: Enforce 'VMBus version >= 5.2' on isolated guests ...
2021-02-12Merge branches 'arm/renesas', 'arm/smmu', 'x86/amd', 'x86/vt-d' and 'core' ↵Joerg Roedel
into next
2021-02-12iommu/amd: Fix performance counter initializationSuravee Suthikulpanit
Certain AMD platforms enable power gating feature for IOMMU PMC, which prevents the IOMMU driver from updating the counter while trying to validate the PMC functionality in the init_iommu_perf_ctr(). This results in disabling PMC support and the following error message: "AMD-Vi: Unable to read/write to IOMMU perf counter" To workaround this issue, disable power gating temporarily by programming the counter source to non-zero value while validating the counter, and restore the prior state afterward. Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Tested-by: Tj (Elloe Linux) <ml.linux@elloe.vision> Link: https://lore.kernel.org/r/20210208122712.5048-1-suravee.suthikulpanit@amd.com Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=201753 Signed-off-by: Joerg Roedel <jroedel@suse.de>
2021-02-11iommu/hyperv: setup an IO-APIC IRQ remapping domain for root partitionWei Liu
Just like MSI/MSI-X, IO-APIC interrupts are remapped by Microsoft Hypervisor when Linux runs as the root partition. Implement an IRQ domain to handle mapping and unmapping of IO-APIC interrupts. Signed-off-by: Wei Liu <wei.liu@kernel.org> Acked-by: Joerg Roedel <joro@8bytes.org> Reviewed-by: Michael Kelley <mikelley@microsoft.com> Link: https://lore.kernel.org/r/20210203150435.27941-17-wei.liu@kernel.org
2021-02-09dma-mapping: remove the {alloc,free}_noncoherent methodsChristoph Hellwig
It turns out allowing non-contigous allocations here was a rather bad idea, as we'll now need to define ways to get the pages for mmaping or dma_buf sharing. Revert this change and stick to the original concept. A different API for the use case of non-contigous allocations will be added back later. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Tested-by: Ricardo Ribalda <ribalda@chromium.org>:wq
2021-02-08iommu/mediatek: Fix error code in probe()Dan Carpenter
This error path is supposed to return -EINVAL. It used to return directly but we added some clean up and accidentally removed the error code. Also I fixed a typo in the error message. Fixes: c0b57581b73b ("iommu/mediatek: Add power-domain operation") Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com> Reviewed-by: Yong Wu <yong.wu@mediatek.com> Link: https://lore.kernel.org/r/YB0+GU5akSdu29Vu@mwanda Signed-off-by: Joerg Roedel <jroedel@suse.de>
2021-02-05iommu/mediatek: Fix unsigned domid comparison with less than zeroColin Ian King
Currently the check for domid < 0 is always false because domid is unsigned. Fix this by casting domid to an int before making the comparison. Signed-off-by: Colin Ian King <colin.king@canonical.com> Acked-by: Will Deacon <will@kernel.org> Link: https://lore.kernel.org/r/20210204150001.102672-1-colin.king@canonical.com Addresses-Coverity: ("Unsigned comparison against 0") Signed-off-by: Joerg Roedel <jroedel@suse.de>
2021-02-04iommu/vt-d: Parse SATC reporting structureYian Chen
Software should parse every SATC table and all devices in the tables reported by the BIOS and keep the information in kernel list for further reference. Signed-off-by: Yian Chen <yian.chen@intel.com> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Link: https://lore.kernel.org/r/20210203093329.1617808-1-baolu.lu@linux.intel.com Link: https://lore.kernel.org/r/20210204014401.2846425-7-baolu.lu@linux.intel.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2021-02-04iommu/vt-d: Add iotlb_sync_map callbackLu Baolu
Some Intel VT-d hardware implementations don't support memory coherency for page table walk (presented by the Page-Walk-coherency bit in the ecap register), so that software must flush the corresponding CPU cache lines explicitly after each page table entry update. The iommu_map_sg() code iterates through the given scatter-gather list and invokes iommu_map() for each element in the scatter-gather list, which calls into the vendor IOMMU driver through iommu_ops callback. As the result, a single sg mapping may lead to multiple cache line flushes, which leads to the degradation of I/O performance after the commit <c588072bba6b5> ("iommu/vt-d: Convert intel iommu driver to the iommu ops"). Fix this by adding iotlb_sync_map callback and centralizing the clflush operations after all sg mappings. Fixes: c588072bba6b5 ("iommu/vt-d: Convert intel iommu driver to the iommu ops") Reported-by: Chuck Lever <chuck.lever@oracle.com> Link: https://lore.kernel.org/linux-iommu/D81314ED-5673-44A6-B597-090E3CB83EB0@oracle.com/ Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Cc: Robin Murphy <robin.murphy@arm.com> [ cel: removed @first_pte, which is no longer used ] Signed-off-by: Chuck Lever <chuck.lever@oracle.com> Link: https://lore.kernel.org/linux-iommu/161177763962.1311.15577661784296014186.stgit@manet.1015granger.net Link: https://lore.kernel.org/r/20210204014401.2846425-5-baolu.lu@linux.intel.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2021-02-04iommu/vt-d: Move capability check code to cap_audit filesKyung Min Park
Move IOMMU capability check and sanity check code to cap_audit files. Also implement some helper functions for sanity checks. Signed-off-by: Kyung Min Park <kyung.min.park@intel.com> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Link: https://lore.kernel.org/r/20210130184452.31711-1-kyung.min.park@intel.com Link: https://lore.kernel.org/r/20210204014401.2846425-4-baolu.lu@linux.intel.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2021-02-04iommu/vt-d: Audit IOMMU Capabilities and add helper functionsKyung Min Park
Audit IOMMU Capability/Extended Capability and check if the IOMMUs have the consistent value for features. Report out or scale to the lowest supported when IOMMU features have incompatibility among IOMMUs. Report out features when below features are mismatched: - First Level 5 Level Paging Support (FL5LP) - First Level 1 GByte Page Support (FL1GP) - Read Draining (DRD) - Write Draining (DWD) - Page Selective Invalidation (PSI) - Zero Length Read (ZLR) - Caching Mode (CM) - Protected High/Low-Memory Region (PHMR/PLMR) - Required Write-Buffer Flushing (RWBF) - Advanced Fault Logging (AFL) - RID-PASID Support (RPS) - Scalable Mode Page Walk Coherency (SMPWC) - First Level Translation Support (FLTS) - Second Level Translation Support (SLTS) - No Write Flag Support (NWFS) - Second Level Accessed/Dirty Support (SLADS) - Virtual Command Support (VCS) - Scalable Mode Translation Support (SMTS) - Device TLB Invalidation Throttle (DIT) - Page Drain Support (PDS) - Process Address Space ID Support (PASID) - Extended Accessed Flag Support (EAFS) - Supervisor Request Support (SRS) - Execute Request Support (ERS) - Page Request Support (PRS) - Nested Translation Support (NEST) - Snoop Control (SC) - Pass Through (PT) - Device TLB Support (DT) - Queued Invalidation (QI) - Page walk Coherency (C) Set capability to the lowest supported when below features are mismatched: - Maximum Address Mask Value (MAMV) - Number of Fault Recording Registers (NFR) - Second Level Large Page Support (SLLPS) - Fault Recording Offset (FRO) - Maximum Guest Address Width (MGAW) - Supported Adjusted Guest Address Width (SAGAW) - Number of Domains supported (NDOMS) - Pasid Size Supported (PSS) - Maximum Handle Mask Value (MHMV) - IOTLB Register Offset (IRO) Signed-off-by: Kyung Min Park <kyung.min.park@intel.com> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Link: https://lore.kernel.org/r/20210130184452.31711-1-kyung.min.park@intel.com Link: https://lore.kernel.org/r/20210204014401.2846425-3-baolu.lu@linux.intel.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2021-02-02iommu: Properly pass gfp_t in _iommu_map() to avoid atomic sleepingDouglas Anderson
Sleeping while atomic = bad. Let's fix an obvious typo to try to avoid it. The warning that was seen (on a downstream kernel with the problematic patch backported): BUG: sleeping function called from invalid context at mm/page_alloc.c:4726 in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 9, name: ksoftirqd/0 CPU: 0 PID: 9 Comm: ksoftirqd/0 Not tainted 5.4.93-12508-gc10c93e28e39 #1 Call trace: dump_backtrace+0x0/0x154 show_stack+0x20/0x2c dump_stack+0xa0/0xfc ___might_sleep+0x11c/0x12c __might_sleep+0x50/0x84 __alloc_pages_nodemask+0xf8/0x2bc __arm_lpae_alloc_pages+0x48/0x1b4 __arm_lpae_map+0x124/0x274 __arm_lpae_map+0x1cc/0x274 arm_lpae_map+0x140/0x170 arm_smmu_map+0x78/0xbc __iommu_map+0xd4/0x210 _iommu_map+0x4c/0x84 iommu_map_atomic+0x44/0x58 __iommu_dma_map+0x8c/0xc4 iommu_dma_map_page+0xac/0xf0 Fixes: d8c1df02ac7f ("iommu: Move iotlb_sync_map out from __iommu_map") Signed-off-by: Douglas Anderson <dianders@chromium.org> Reviewed-by: Yong Wu <yong.wu@mediatek.com> Acked-by: Will Deacon <will@kernel.org> Link: https://lore.kernel.org/r/20210201170611.1.I64a7b62579287d668d7c89e105dcedf45d641063@changeid Signed-off-by: Joerg Roedel <jroedel@suse.de>
2021-02-02iommu/vt-d: Fix compile error [-Werror=implicit-function-declaration]Lu Baolu
trace_qi_submit() could be used when interrupt remapping is supported, but DMA remapping is not. In this case, the following compile error occurs. ../drivers/iommu/intel/dmar.c: In function 'qi_submit_sync': ../drivers/iommu/intel/dmar.c:1311:3: error: implicit declaration of function 'trace_qi_submit'; did you mean 'ftrace_nmi_exit'? [-Werror=implicit-function-declaration] trace_qi_submit(iommu, desc[i].qw0, desc[i].qw1, ^~~~~~~~~~~~~~~ ftrace_nmi_exit Fixes: f2dd871799ba5 ("iommu/vt-d: Add qi_submit trace event") Reported-by: kernel test robot <lkp@intel.com> Reported-by: Randy Dunlap <rdunlap@infradead.org> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Link: https://lore.kernel.org/r/20210130151907.3929148-1-baolu.lu@linux.intel.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2021-02-01Merge branch 'for-joerg/mtk' into for-joerg/arm-smmu/updatesWill Deacon
Merge in Mediatek support from Yong Wu which introduces significant changes to the TLB invalidation and Arm short-descriptor code in the io-pgtable layer. * for-joerg/mtk: (40 commits) MAINTAINERS: Add entry for MediaTek IOMMU iommu/mediatek: Add mt8192 support iommu/mediatek: Remove unnecessary check in attach_device iommu/mediatek: Support master use iova over 32bit iommu/mediatek: Add iova reserved function iommu/mediatek: Support for multi domains iommu/mediatek: Add get_domain_id from dev->dma_range_map iommu/mediatek: Add iova_region structure iommu/mediatek: Move geometry.aperture updating into domain_finalise iommu/mediatek: Move domain_finalise into attach_device iommu/mediatek: Adjust the structure iommu/mediatek: Support report iova 34bit translation fault in ISR iommu/mediatek: Support up to 34bit iova in tlb flush iommu/mediatek: Add power-domain operation iommu/mediatek: Add pm runtime callback iommu/mediatek: Add device link for smi-common and m4u iommu/mediatek: Add error handle for mtk_iommu_probe iommu/mediatek: Move hw_init into attach_device iommu/mediatek: Update oas for v7s iommu/mediatek: Add a flag for iova 34bits case ...
2021-02-01iommu/mediatek: Add mt8192 supportYong Wu
Add mt8192 iommu support. Signed-off-by: Yong Wu <yong.wu@mediatek.com> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Link: https://lore.kernel.org/r/20210111111914.22211-33-yong.wu@mediatek.com Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01iommu/mediatek: Remove unnecessary check in attach_deviceYong Wu
This priv_data is set in the of_xlate. if of_xlate failed, it should not enter attach_device. remove the unnecessary check. Signed-off-by: Yong Wu <yong.wu@mediatek.com> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Link: https://lore.kernel.org/r/20210111111914.22211-32-yong.wu@mediatek.com Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01iommu/mediatek: Support master use iova over 32bitYong Wu
After extending v7s, our pagetable already support iova reach 16GB(34bit). the master got the iova via dma_alloc_attrs may reach 34bits, but its HW register still is 32bit. then how to set the bit32/bit33 iova? this depend on a SMI larb setting(bank_sel). we separate whole 16GB iova to four banks: bank: 0: 0~4G; 1: 4~8G; 2: 8-12G; 3: 12-16G; The bank number is (iova >> 32). We will preassign which bank the larbs belong to. currently we don't have a interface for master to adjust its bank number. Each a bank is a iova_region which is a independent iommu-domain. the iova range for each iommu-domain can't cross 4G. Signed-off-by: Yong Wu <yong.wu@mediatek.com> Acked-by: Krzysztof Kozlowski <krzk@kernel.org> #for memory part Reviewed-by: Tomasz Figa <tfiga@chromium.org> Link: https://lore.kernel.org/r/20210111111914.22211-31-yong.wu@mediatek.com Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01iommu/mediatek: Add iova reserved functionYong Wu
For multiple iommu_domains, we need to reserve some iova regions. Take a example, If the default iova region is 0 ~ 4G, but the 0x4000_0000 ~ 0x43ff_ffff is only for the special CCU0 domain. Thus we should exclude this region for the default iova region. Signed-off-by: Anan sun <anan.sun@mediatek.com> Signed-off-by: Chao Hao <chao.hao@mediatek.com> Signed-off-by: Yong Wu <yong.wu@mediatek.com> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Link: https://lore.kernel.org/r/20210111111914.22211-30-yong.wu@mediatek.com Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01iommu/mediatek: Support for multi domainsYong Wu
Some HW IP(ex: CCU) require the special iova range. That means the iova got from dma_alloc_attrs for that devices must locate in his special range. In this patch, we prepare a iommu group(domain) for each a iova range requirement. Meanwhile we still use one pagetable which support 16GB iova. After this patch, If the iova range of a master is over 4G, the master should: a) Declare its special dma-ranges in its dtsi node. For example, If we preassign the iova 4G-8G for vcodec, then the vcodec dtsi node should add this: /* * iova start at 0x1_0000_0000, pa still start at 0x4000_0000 * size is 0x1_0000_0000. */ dma-ranges = <0x1 0x0 0x0 0x40000000 0x1 0x0>; /* 4G ~ 8G */ Note: we don't have a actual bus concept here. the master doesn't have its special parent node, thus this dma-ranges can only be put in the master's node. b) Update the dma_mask: dma_set_mask_and_coherent(dev, DMA_BIT_MASK(33)); Signed-off-by: Yong Wu <yong.wu@mediatek.com> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Link: https://lore.kernel.org/r/20210111111914.22211-29-yong.wu@mediatek.com Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01iommu/mediatek: Add get_domain_id from dev->dma_range_mapYong Wu
Add a new interface _get_domain_id from dev->dma_range_map, The iommu consumer device will use dma-ranges in dtsi node to indicate its dma address region requirement. In this iommu driver, we will get the requirement and decide which iova domain it should locate. In the lastest SoC, there will be several iova-regions(domains), we will compare and calculate which domain is right. If the start/end of device requirement equal some region. it is best fit of course. If it is inside some region, it is also ok. the iova requirement of a device should not be inside two or more regions. Signed-off-by: Yong Wu <yong.wu@mediatek.com> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Link: https://lore.kernel.org/r/20210111111914.22211-28-yong.wu@mediatek.com Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01iommu/mediatek: Add iova_region structureYong Wu
Add a new structure for the iova_region. Each a region will be a independent iommu domain. For the previous SoC, there is single iova region(0~4G). For the SoC that need support multi-domains, there will be several regions. Signed-off-by: Yong Wu <yong.wu@mediatek.com> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Link: https://lore.kernel.org/r/20210111111914.22211-27-yong.wu@mediatek.com Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01iommu/mediatek: Move geometry.aperture updating into domain_finaliseYong Wu
Move the domain geometry.aperture updating into domain_finalise. This is a preparing patch for updating the domain region. We know the detailed iova region in the attach_device. Signed-off-by: Yong Wu <yong.wu@mediatek.com> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Link: https://lore.kernel.org/r/20210111111914.22211-26-yong.wu@mediatek.com Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01iommu/mediatek: Move domain_finalise into attach_deviceYong Wu
Currently domain_alloc only has a parameter(type), We have no chance to input some special data. This patch moves the domain_finalise into attach_device which has the device information, then could update the domain's geometry.aperture ranges for each a device. Strictly, I should use the data from mtk_iommu_get_m4u_data as the parameter of mtk_iommu_domain_finalise in this patch. but dom->data only is used in tlb ops in which the data is get from the m4u_list, thus it is ok here. Signed-off-by: Yong Wu <yong.wu@mediatek.com> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Link: https://lore.kernel.org/r/20210111111914.22211-25-yong.wu@mediatek.com Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01iommu/mediatek: Adjust the structureYong Wu
Add "struct mtk_iommu_data *" in the "struct mtk_iommu_domain", reduce the call mtk_iommu_get_m4u_data(). No functional change. Signed-off-by: Yong Wu <yong.wu@mediatek.com> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Link: https://lore.kernel.org/r/20210111111914.22211-24-yong.wu@mediatek.com Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01iommu/mediatek: Support report iova 34bit translation fault in ISRYong Wu
If the iova is over 32bit, the fault status register bit is a little different. Signed-off-by: Yong Wu <yong.wu@mediatek.com> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Link: https://lore.kernel.org/r/20210111111914.22211-23-yong.wu@mediatek.com Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01iommu/mediatek: Support up to 34bit iova in tlb flushYong Wu
If the iova is 34bit, the iova[32][33] is the bit0/1 in the tlb flush register. Add a new macro for this. In the macro, since (iova + size - 1) may be end with 0xfff, then the bit0/1 always is 1, thus add a mask. Signed-off-by: Yong Wu <yong.wu@mediatek.com> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Link: https://lore.kernel.org/r/20210111111914.22211-22-yong.wu@mediatek.com Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01iommu/mediatek: Add power-domain operationYong Wu
In the previous SoC, the M4U HW is in the EMI power domain which is always on. the latest M4U is in the display power domain which may be turned on/off, thus we have to add pm_runtime interface for it. When the engine work, the engine always enable the power and clocks for smi-larb/smi-common, then the M4U's power will always be powered on automatically via the device link with smi-common. Note: we don't enable the M4U power in iommu_map/unmap for tlb flush. If its power already is on, of course it is ok. if the power is off, the main tlb will be reset while M4U power on, thus the tlb flush while m4u power off is unnecessary, just skip it. Therefore, we increase the ref_count for pm when pm status is ACTIVE, otherwise, skip it. Meanwhile, the tlb_flush_range is called so often, thus, update pm ref_count while the SoC has power-domain to avoid touch the dev->power.lock. and the tlb_flush_all only is called when boot, so no need check if the SoC has power-domain to keep code clean. There will be one case that pm runctime status is not expected when tlb flush. After boot, the display may call dma_alloc_attrs before it call pm_runtime_get(disp-dev), then the m4u's pm status is not active inside the dma_alloc_attrs. Since it only happens after boot, the tlb is clean at that time, I also think this is ok. Signed-off-by: Yong Wu <yong.wu@mediatek.com> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Link: https://lore.kernel.org/r/20210111111914.22211-21-yong.wu@mediatek.com Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01iommu/mediatek: Add pm runtime callbackYong Wu
In pm runtime case, all the registers backup/restore and bclk are controlled in the pm_runtime callback, Rename the original suspend/resume to the runtime_suspend/resume. Use pm_runtime_force_suspend/resume as the normal suspend/resume. iommu should suspend after iommu consumer devices, thus use _LATE_. Signed-off-by: Yong Wu <yong.wu@mediatek.com> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Link: https://lore.kernel.org/r/20210111111914.22211-20-yong.wu@mediatek.com Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01iommu/mediatek: Add device link for smi-common and m4uYong Wu
In the lastest SoC, M4U has its special power domain. thus, If the engine begin to work, it should help enable the power for M4U firstly. Currently if the engine work, it always enable the power/clocks for smi-larbs/smi-common. This patch adds device_link for smi-common and M4U. then, if smi-common power is enabled, the M4U power also is powered on automatically. Normally M4U connect with several smi-larbs and their smi-common always are the same, In this patch it get smi-common dev from the last smi-larb device, then add the device_link. Signed-off-by: Yong Wu <yong.wu@mediatek.com> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Link: https://lore.kernel.org/r/20210111111914.22211-19-yong.wu@mediatek.com Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01iommu/mediatek: Add error handle for mtk_iommu_probeYong Wu
In the original code, we lack the error handle. This patch adds them. Signed-off-by: Yong Wu <yong.wu@mediatek.com> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Link: https://lore.kernel.org/r/20210111111914.22211-18-yong.wu@mediatek.com Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01iommu/mediatek: Move hw_init into attach_deviceYong Wu
In attach device, it will update the pagetable base address register. Move the hw_init function also here. Then it only need call pm_runtime_get/put one time here if m4u has power domain. Signed-off-by: Yong Wu <yong.wu@mediatek.com> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Link: https://lore.kernel.org/r/20210111111914.22211-17-yong.wu@mediatek.com Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01iommu/mediatek: Update oas for v7sYong Wu
This patch only updates oas in different SoCs. If the SoC supports 4GB-mode and current dram size is 4GB, the oas is 33. otherwise, it's still 32. In the lastest SoC, the oas is 35bits. Signed-off-by: Yong Wu <yong.wu@mediatek.com> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Link: https://lore.kernel.org/r/20210111111914.22211-16-yong.wu@mediatek.com Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01iommu/mediatek: Add a flag for iova 34bits caseYong Wu
Add a HW flag for if the HW support 34bit IOVA. the previous SoC still use 32bit. normally the lvl1 pgtable size is 16KB when ias == 32. if ias == 34, lvl1 pgtable size is 16KB * 4. The purpose of this patch is to save 16KB*3 continuous memory for the previous SoC. Signed-off-by: Yong Wu <yong.wu@mediatek.com> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Link: https://lore.kernel.org/r/20210111111914.22211-15-yong.wu@mediatek.com Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01iommu/io-pgtable-arm-v7s: Quad lvl1 pgtable for MediaTekYong Wu
The standard input iova bits is 32. MediaTek quad the lvl1 pagetable (4 * lvl1). No change for lvl2 pagetable. Then the iova bits can reach 34bit. Signed-off-by: Yong Wu <yong.wu@mediatek.com> Reviewed-by: Robin Murphy <robin.murphy@arm.com> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Link: https://lore.kernel.org/r/20210111111914.22211-14-yong.wu@mediatek.com Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01iommu/io-pgtable-arm-v7s: Add cfg as a param in some macrosYong Wu
Add "cfg" as a parameter for some macros. This is a preparing patch for mediatek extend the lvl1 pgtable. No functional change. Signed-off-by: Yong Wu <yong.wu@mediatek.com> Acked-by: Will Deacon <will@kernel.org> Reviewed-by: Robin Murphy <robin.murphy@arm.com> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Link: https://lore.kernel.org/r/20210111111914.22211-13-yong.wu@mediatek.com Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01iommu/io-pgtable-arm-v7s: Clarify LVL_SHIFT/BITS macroYong Wu
The current _ARM_V7S_LVL_BITS/ARM_V7S_LVL_SHIFT use a formula to calculate the corresponding value for level1 and level2 to pretend the code sane. Actually their level1 and level2 values are different from each other. This patch only clarify the two macro. No functional change. Suggested-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Yong Wu <yong.wu@mediatek.com> Reviewed-by: Robin Murphy <robin.murphy@arm.com> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Link: https://lore.kernel.org/r/20210111111914.22211-12-yong.wu@mediatek.com Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01iommu/io-pgtable-arm-v7s: Extend PA34 for MediaTekYong Wu
MediaTek extend the bit5 in lvl1 and lvl2 descriptor as PA34. Signed-off-by: Yong Wu <yong.wu@mediatek.com> Acked-by: Will Deacon <will@kernel.org> Reviewed-by: Robin Murphy <robin.murphy@arm.com> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Link: https://lore.kernel.org/r/20210111111914.22211-11-yong.wu@mediatek.com Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01iommu/io-pgtable-arm-v7s: Use ias to check the valid iova in unmapYong Wu
Use the ias for the valid iova checking in arm_v7s_unmap. This is a preparing patch for supporting iova 34bit for MediaTek. Signed-off-by: Yong Wu <yong.wu@mediatek.com> Reviewed-by: Robin Murphy <robin.murphy@arm.com> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Link: https://lore.kernel.org/r/20210111111914.22211-10-yong.wu@mediatek.com Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01iommu/mediatek: Use the common mtk-memory-port.hYong Wu
Use the common memory header(larb-port) in the source code. Signed-off-by: Yong Wu <yong.wu@mediatek.com> Acked-by: Krzysztof Kozlowski <krzk@kernel.org> Reviewed-by: Tomasz Figa <tfiga@chromium.org> Link: https://lore.kernel.org/r/20210111111914.22211-9-yong.wu@mediatek.com Signed-off-by: Will Deacon <will@kernel.org>
generated by cgit v1.2.3-70-g09d2 (git 2.47.1) at 2024-12-29 09:57:45 +0000