aboutsummaryrefslogtreecommitdiffstats
path: root/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c (follow)
AgeCommit message (Collapse)AuthorFilesLines
2017-09-01drm/amdgpu: fix moved list handling in the VMChristian König1-3/+6
Only move BOs to the moved/relocated list when they aren't already on a list. This prevents accidential removal from the evicted list. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-09-01drm/amdgpu: handle all fragment sizes v4Roger He1-29/+26
This can improve performance for some cases. v2 (chk): handle all sizes, simplify the patch quite a bit v3 (chk): adjust dw estimation as well v4 (chk): use single loop, make end mask 64bit Signed-off-by: Roger He <Hongbo.He@amd.com> Signed-off-by: Christian König <christian.koenig@amd.com> Tested-by: Roger He <Hongbo.He@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Chunming Zhou <david1.zhou@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-08-31drm/amdgpu: add support for per VM BOs v2Christian König1-23/+56
Per VM BOs are handled like VM PDs and PTs. They are always valid and don't need to be specified in the BO lists. v2: validate PDs/PTs first Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-08-31drm/amdgpu: fix new PD update code for Vega10 v2Christian König1-2/+2
We need to refer to the parent instead of the root BO for multi level page tables on Vega10. Also don't set the PDE_PTE bit. v2: Don't set the PDE_PTE bit either. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-and-Tested-by: Roger He <Hongbo.He@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-08-31drm/amdgpu: cleanup the VM code a bit moreChristian König1-15/+5
The src isn't used any more after GART hack removal. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Chunming Zhou <david1.zhou@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-08-31drm/amdgpu: rework page directory filling v2Christian König1-29/+60
Keep track off relocated PDs/PTs instead of walking and checking all PDs. v2: fix root PD handling Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> (v1) Reviewed-by: Chunming Zhou <david1.zhou@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-08-29drm/amdgpu: track evicted page tables v2Christian König1-124/+103
Instead of validating all page tables when one was evicted, track which one needs a validation. v2: simplify amdgpu_vm_ready as well Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> (v1) Reviewed-by: Chunming Zhou <david1.zhou@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-08-29drm/amdgpu: add bo_va cleared flag again v2Christian König1-22/+13
We changed this to use an extra list a while back, but for the next series I need a separate flag again. v2: reorder to avoid unlocked list access Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Chunming Zhou <david1.zhou@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-08-29drm/amdgpu: rework moved handling in the VM v2Christian König1-3/+10
Instead of using the vm_state use a separate flag to note that the BO was moved. v2: reorder patches to avoid temporary lockless access Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Chunming Zhou <david1.zhou@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-08-29drm/amdgpu: fix and cleanup VM ready checkChristian König1-0/+32
Stop checking the mapped BO itself, cause that one is certainly not a page table. Additional to that move the code into amdgpu_vm.c Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Chunming Zhou <david1.zhou@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-08-29drm/amdgpu: fix amdgpu_vm_bo_map trace pointChristian König1-0/+2
That somehow got lost. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Chunming Zhou <david1.zhou@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-08-29drm/amdgpu: remove the GART copy hackChristian König1-17/+5
This isn't used since we don't map evicted BOs to GART any more. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Reviewed-by: Roger He <Hongbo.He@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-08-29drm/amdgpu: fix and cleanup shadow handlingChristian König1-8/+0
Set the shadow flag on the shadow and not the parent, always bind shadow BOs during allocation instead of manually, use the reservation_object wrappers to grab the lock. This fixes a couple of issues with binding the shadow BOs as well as correctly evicting them when memory becomes tight. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Chunming Zhou <david1.zhou@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-08-29drm/amdgpu: Fix huge page updates with CPUFelix Kuehling1-1/+15
Correctly detect system memory mappings when using CPU and don't use huge pages for them. Avoid incorrectly translating a physical page table GPU address when splitting a huge page while mapping system memory. Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Christian König <christian.koenig@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-08-24drm/amdgpu: remove the GART copy hackChristian König1-17/+5
This isn't used since we don't map evicted BOs to GART any more. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Reviewed-by: Roger He <Hongbo.He@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-08-24drm/amdgpu: fix and cleanup shadow handlingChristian König1-8/+0
Set the shadow flag on the shadow and not the parent, always bind shadow BOs during allocation instead of manually, use the reservation_object wrappers to grab the lock. This fixes a couple of issues with binding the shadow BOs as well as correctly evicting them when memory becomes tight. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Chunming Zhou <david1.zhou@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-08-23drm/amdgpu: Fix huge page updates with CPUFelix Kuehling1-1/+15
Correctly detect system memory mappings when using CPU and don't use huge pages for them. Avoid incorrectly translating a physical page table GPU address when splitting a huge page while mapping system memory. Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Christian König <christian.koenig@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-08-17drm/amd/amdgpu: expose fragment size as module parameter (v2)Roger He1-4/+21
Allow overrides on the command line. v2: agd: sqaush in spelling fix and bogus default value warning Reviewed-by: Christian König <christian.koenig@amd.com> Signed-off-by: Roger He <Hongbo.He@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-08-17drm/amd/amdgpu: store fragment_size in vm_managerRoger He1-3/+1
adds fragment_size in the vm_manager structure and implements hardware setup for it. Reviewed-by: Christian König <christian.koenig@amd.com> Signed-off-by: Roger He <Hongbo.He@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-08-17drm/amdgpu: rename VM invalidated to movedChristian König1-8/+8
That better describes what happens here with the BO. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-08-17drm/amdgpu: separate bo_va structureChristian König1-38/+43
Split that into vm_bo_base and bo_va to allow other uses as well. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-08-17drm/amdgpu: drop the extra VM huge page flag v2Christian König1-9/+9
Just add the flags to the addr field as well. v2: add some more comments that the flag is for huge pages. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-08-17drm/amdgpu: remove superflous amdgpu_bo_kmap in the VMChristian König1-19/+9
We now properly kmap all BOs after validation. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-08-17drm/amdgpu: only bind VM shadows after validation v2Christian König1-14/+8
No need to do this on every CS. v2: remove all other bind, reorder code Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Chunming Zhou <david1.zhou@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-08-17drm/amdgpu: only move VM BOs in the LRU during validation v2Christian König1-47/+11
This should save us a bunch of command submission overhead. v2: move the LRU move to the right place to avoid the move for the root BO and handle the shadow BOs as well. This turned out to be a bug fix because the move needs to happen before the kmap. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Chunming Zhou <david1.zhou@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-08-15drm/amdgpu: Support IOMMU on RavenYong Zhao1-5/+24
We achieved that by setting S(SYSTEM) and P(PDE as PTE) bit to 1 for PDEs and setting S bit to 1 for PTEs when the corresponding addresses are not occupied by gpu driver allocated buffers. Signed-off-by: Yong Zhao <Yong.Zhao@amd.com> Reviewed-by: Christian König <christian.koenig@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-08-15drm/amdgpu: Add a parameter to amdgpu_bo_create()Yong Zhao1-2/+2
The parameter init_value contains the value to which we initialized VRAM bo when AMDGPU_GEM_CREATE_VRAM_CLEARED flag is set. Signed-off-by: Yong Zhao <Yong.Zhao@amd.com> Reviewed-by: Christian König <christian.koenig@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-08-15drm/amdgpu: cleanup kptr handlingChristian König1-2/+2
Don't keep around the same pointer twice. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-07-25drm/amdgpu: enable huge page handling in the VM v5Alex Deucher1-20/+100
The hardware can use huge pages to map 2MB of address space with only one PDE. v2: few cleanups and rebased v3: skip PT updates if we are using the PDE v4: rebased, added support for CPU based updates v5: fix CPU based updates once more v6: fix ndw estimation Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-and-tested-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-07-25drm/amdgpu: increase fragmentation size for Vega10 v2Christian König1-2/+3
The fragment bits work differently for Vega10 compared to previous generations. Increase the fragment size to 2MB for now to better handle that. v2: handle the hardware setup as well Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-and-tested-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-07-14drm/amdgpu: map VM BOs for CPU based updates only onceChristian König1-36/+51
No need to try to map them every time. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-07-14drm/amdgpu: flush the HDP only once for CPU based VM updatesChristian König1-4/+12
No need to do this after every single update. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-07-14drm/amdgpu: trace setting VM page tables with the CPU as wellChristian König1-0/+2
Handy for debugging. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-07-14drm/amdgpu: remove VM shadow WARN_ONsAlex Deucher1-5/+1
Printing a warning into the logs that we will certainly run into a BUG() is completely nonsense, the BUG() is more than noisy enough. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-07-14drm/amdgpu: fix amdgpu_vm_bo_waitChristian König1-8/+9
We need to wait with the correct owner on unmap operations or otherwise can run into VM faults. Also always wait for the page directory since this is where the reservation object comes from. So rename the function to amdgpu_vm_wait_pd instead as well. Signed-off-by: Christian König <christian.koenig@amd.com> Acked-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-07-14drm/amdgpu: fix VM flush for CPU based updatesChristian König1-3/+3
We don't have any update fence in that case, so the need for flushing isn't detected automatically. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-07-14drm/amdgpu:fix world switch hangMonk Liu1-2/+5
for SR-IOV, we must keep the pipeline-sync in the protection of COND_EXEC, otherwise the command consumed by CPG is not consistent when world switch triggerd, e.g.: world switch hit and the IB frame is skipped so the fence won't signal, thus CP will jump to the next DMAframe's pipeline-sync command, and it will make CP hang foever. after pipelin-sync moved into COND_EXEC the consistency can be guaranteed Signed-off-by: Monk Liu <Monk.Liu@amd.com> Reviewed-by: Christian König <christian.koenig@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-07-14drm/amdgpu: simplify VM shadow handling v2Christian König1-20/+11
Now that we don't join PTE updates any more we don't need to call the update function twice for this. v2: rebased Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Chunming Zhou <david1.zhou@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-06-16Merge branch 'drm-next-4.13' of git://people.freedesktop.org/~agd5f/linux into drm-nextDave Airlie1-154/+472
New radeon and amdgpu features for 4.13: - Lots of Vega10 bug fixes - Preliminary Raven support - KIQ support for compute rings - MEC queue management rework from Andres - Audio support for DCE6 - SR-IOV improvements - Improved module parameters for controlling radeon vs amdgpu support for SI and CIK - Bug fixes - General code cleanups [airlied: dropped drmP.h header from one file was needed and build broke] * 'drm-next-4.13' of git://people.freedesktop.org/~agd5f/linux: (362 commits) drm/amdgpu: Fix compiler warnings drm/amdgpu: vm_update_ptes remove code duplication drm/amd/amdgpu: Port VCN over to new SOC15 macros drm/amd/amdgpu: Port PSP v10.0 over to new SOC15 macros drm/amd/amdgpu: Port PSP v3.1 over to new SOC15 macros drm/amd/amdgpu: Port NBIO v7.0 driver over to new SOC15 macros drm/amd/amdgpu: Port NBIO v6.1 driver over to new SOC15 macros drm/amd/amdgpu: Port UVD 7.0 over to new SOC15 macros drm/amd/amdgpu: Port MMHUB over to new SOC15 macros drm/amd/amdgpu: Cleanup gfxhub read-modify-write patterns drm/amd/amdgpu: Port GFXHUB over to new SOC15 macros drm/amd/amdgpu: Add offset variant to SOC15 macros drm/amd/powerplay: add avfs control for Vega10 drm/amdgpu: add virtual display support for raven drm/amdgpu/gfx9: fix compute ring doorbell index drm/amd/amdgpu: Rename KIQ ring to avoid spaces drm/amd/amdgpu: gfx9 tidy ups (v2) drm/amdgpu: add contiguous flag in ucode bo create drm/amdgpu: fix missed gpu info firmware when cache firmware during S3 drm/amdgpu: export test ib debugfs interface ...
2017-06-15drm/amdgpu: Fix compiler warningsHarish Kasiviswanathan1-4/+4
Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-06-15drm/amdgpu: vm_update_ptes remove code duplicationHarish Kasiviswanathan1-57/+16
CPU and GPU paths were mostly the same. Acked-by: Christian König <christian.koenig@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-06-09drm/amdgpu: Support page table update via CPUHarish Kasiviswanathan1-1/+87
v2: Fix logical mistake. If CPU update failed amdgpu_vm_bo_update_mapping() would not return and instead fall through to SDMA update. Minor change due to amdgpu_vm_bo_wait() prototype change Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com> Reviewed-by: Christian König <christian.koenig@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-06-09drm/amdgpu: Support page directory update via CPUHarish Kasiviswanathan1-42/+119
If amdgpu.vm_update_context param is set to use CPU, then Page Directories will be updated by CPU instead of SDMA v2: Call amdgpu_vm_bo_wait before updating the page tables to ensure the PD/PT BOs are free v3: Minor changes - due to amdgpu_vm_bo_wait() prototype change, local variable declaration order and function comments. Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com> Reviewed-by: Christian König <christian.koenig@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-06-09drm/amdgpu: Add vm context module paramHarish Kasiviswanathan1-1/+35
Add VM update mode module param (amdgpu.vm_update_mode) that can used to control how VM pde/pte are updated for Graphics and Compute. BIT0 controls Graphics and BIT1 Compute. BIT0 [= 0] Graphics updated by SDMA [= 1] by CPU BIT1 [= 0] Compute updated by SDMA [= 1] by CPU By default, only for large BAR system vm_update_mode = 2, indicating that Graphics VMs will be updated via SDMA and Compute VMs will be updated via CPU. And for all all other systems (by default) vm_update_mode = 0 Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com> Reviewed-by: Christian König <christian.koenig@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-06-01drm/amdgpu: Move compute vm bug logic to amdgpu_vm.cAlex Xie1-2/+36
In review, Christian would like to keep the logic inside amdgpu_vm.c with a cost of slightly slower. The loop is still optimized out with this patch. v2: remove the if statement. Now it is not slower. Signed-off-by: Alex Xie <AlexBin.Xie@amd.com> Reviewed-by: Christian König <christian.koeng@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-05-31drm/amdgpu: stop joining VM PTE updatesChristian König1-54/+7
This isn't beneficial any more since VRAM allocations are now split so that they fits into a single page table. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com> Reviewed-by: Chunming Zhou <david1.zhou@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-05-31drm/amdgpu: cache the complete pdeChristian König1-11/+5
Makes it easier to update the PDE with huge pages. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com> Reviewed-by: Chunming Zhou <david1.zhou@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-05-31drm/amdgpu: Remove two ! operations in an if conditionAlex Xie1-3/+2
Make the code easier to understand. Signed-off-by: Alex Xie <AlexBin.Xie@amd.com> Reviewed-by: Michel Dänzer <michel.daenzer@amd.com> Reviewed-by: Chunming Zhou <david1.zhou@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-05-31drm/amdgpu: Optimize a function called by every IB shedulingAlex Xie1-27/+1
Move several if statements and a loop statment from run time to initialization time. Signed-off-by: Alex Xie <AlexBin.Xie@amd.com> Reviewed-by: Chunming Zhou <david1.zhou@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-05-31drm/amdgpu: Return EINVAL if no PT BOHarish Kasiviswanathan1-18/+30
This change is also useful for the upcoming changes where page tables can be updated by CPU. Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com> Reviewed-by: Christian König <christian.koenig@amd.com> Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Copyright © 1996 – 2023 Jason A. Donenfeld. All Rights Reverse Engineered.