aboutsummaryrefslogtreecommitdiffstats
path: root/drivers/gpu/drm/i915/intel_breadcrumbs.c (follow)
AgeCommit message (Collapse)AuthorFilesLines
2019-04-24drm/i915: Move GraphicsTechnology files under gt/Chris Wilson1-337/+0
Start partitioning off the code that talks to the hardware (GT) from the uapi layers and move the device facing code under gt/ One casualty is s/intel_ringbuffer.h/intel_engine.h/ with the plan to subdivide that header and body further (and split out the submission code from the ringbuffer and logical context handling). This patch aims to be simple motion so git can fixup inflight patches with little mess. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Acked-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Acked-by: Jani Nikula <jani.nikula@intel.com> Acked-by: Rodrigo Vivi <rodrigo.vivi@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20190424174839.7141-1-chris@chris-wilson.co.uk
2019-04-16drm/i915: Drop bool return from breadcrumbs signalerChris Wilson1-11/+3
Since removal of the "missed interrupt detection" nobody used the result of whether or not we signaled anybody during that invocation, so now remove the return value. References: 789659f4307a ("drm/i915: Drop fake breadcrumb irq") Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20190416085218.431-1-chris@chris-wilson.co.uk
2019-03-04drm/i915: Acquire breadcrumb ref before cancellingChris Wilson1-10/+8
We may race the interrupt signaling with retirement, in which case the order in which we acquire the reference inside the interrupt is vital to provide the correct barrier against the request being freed in retirement, i.e. we need to acquire our reference before marking the breadcrumb as cancelled (as soon as the breadcrumb is cancelled retirement may drop its reference to the request without serialisation with the interrupt handler). <3>[ 683.372226] BUG i915_request (Tainted: G U ): Object already free <3>[ 683.372269] ----------------------------------------------------------------------------- <4>[ 683.372323] Disabling lock debugging due to kernel taint <3>[ 683.372393] INFO: Allocated in i915_request_alloc+0x169/0x810 [i915] age=0 cpu=2 pid=1420 <3>[ 683.372412] kmem_cache_alloc+0x21c/0x280 <3>[ 683.372478] i915_request_alloc+0x169/0x810 [i915] <3>[ 683.372540] i915_gem_do_execbuffer+0x84e/0x1ae0 [i915] <3>[ 683.372603] i915_gem_execbuffer2_ioctl+0x11b/0x420 [i915] <3>[ 683.372617] drm_ioctl_kernel+0x83/0xf0 <3>[ 683.372626] drm_ioctl+0x2f3/0x3b0 <3>[ 683.372636] do_vfs_ioctl+0xa0/0x6e0 <3>[ 683.372645] ksys_ioctl+0x35/0x60 <3>[ 683.372654] __x64_sys_ioctl+0x11/0x20 <3>[ 683.372664] do_syscall_64+0x55/0x190 <3>[ 683.372675] entry_SYSCALL_64_after_hwframe+0x49/0xbe <3>[ 683.372740] INFO: Freed in i915_request_retire_upto+0xfb/0x2e0 [i915] age=0 cpu=0 pid=1419 <3>[ 683.372807] i915_request_retire_upto+0xfb/0x2e0 [i915] <3>[ 683.372870] i915_request_add+0x3bd/0x9d0 [i915] <3>[ 683.372931] i915_gem_do_execbuffer+0x141c/0x1ae0 [i915] <3>[ 683.372991] i915_gem_execbuffer2_ioctl+0x11b/0x420 [i915] <3>[ 683.373001] drm_ioctl_kernel+0x83/0xf0 <3>[ 683.373008] drm_ioctl+0x2f3/0x3b0 <3>[ 683.373015] do_vfs_ioctl+0xa0/0x6e0 <3>[ 683.373023] ksys_ioctl+0x35/0x60 <3>[ 683.373030] __x64_sys_ioctl+0x11/0x20 <3>[ 683.373037] do_syscall_64+0x55/0x190 <3>[ 683.373045] entry_SYSCALL_64_after_hwframe+0x49/0xbe <3>[ 683.373054] INFO: Slab 0x0000000079bcdd71 objects=30 used=2 fp=0x000000006d77b8af flags=0x8000000000010201 <3>[ 683.373069] INFO: Object 0x000000006d77b8af @offset=24000 fp=0x000000007b061eab <3>[ 683.373083] Redzone 00000000ee47ef28: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb ................ <3>[ 683.373097] Redzone 000000000cb91471: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb ................ <3>[ 683.373111] Redzone 00000000cf2b86ee: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb ................ <3>[ 683.373125] Redzone 00000000f1f5a2cd: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb ................ <3>[ 683.373139] Object 000000006d77b8af: 00 00 00 00 5a 5a 5a 5a 00 3c 49 c0 ff ff ff ff ....ZZZZ.<I..... <3>[ 683.373153] Object 000000006f9b6204: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ <3>[ 683.373167] Object 0000000091410ffb: e0 dd 6b fa 87 9f ff ff e0 dd 6b fa 87 9f ff ff ..k.......k..... <3>[ 683.373181] Object 000000004cdf799d: 20 de 6b fa 87 9f ff ff 3d 00 00 00 00 00 00 00 .k.....=....... <3>[ 683.373195] Object 00000000545afebc: aa b3 00 00 00 00 00 00 0f 00 00 00 00 00 00 00 ................ <3>[ 683.373209] Object 00000000e4a394a8: 25 bd bd 1b 9f 00 00 00 00 00 00 00 5a 5a 5a 5a %...........ZZZZ <3>[ 683.373223] Object 0000000029a7878a: 00 00 00 00 ad 4e ad de ff ff ff ff 5a 5a 5a 5a .....N......ZZZZ <3>[ 683.373237] Object 00000000d37797b3: ff ff ff ff ff ff ff ff e8 6e 57 c0 ff ff ff ff .........nW..... <3>[ 683.373251] Object 00000000d50414f6: 00 b3 c8 8e ff ff ff ff 80 b0 c8 8e ff ff ff ff ................ <3>[ 683.373265] Object 00000000c28e8847: 41 01 4b c0 ff ff ff ff 00 00 88 8e 88 9f ff ff A.K............. <3>[ 683.373279] Object 00000000c74212ab: 38 c1 6d 8a 88 9f ff ff 58 21 74 8a 88 9f ff ff 8.m.....X!t..... <3>[ 683.373293] Object 000000000d8012cf: c0 c1 6d 8a 88 9f ff ff 58 79 dd d9 87 9f ff ff ..m.....Xy...... <3>[ 683.373306] Object 00000000c9900b91: 98 d0 4e 8a 88 9f ff ff 58 3c e8 9b 88 9f ff ff ..N.....X<...... <3>[ 683.373320] Object 0000000044bb8c3d: 58 3c e8 9b 88 9f ff ff 64 f5 04 00 00 00 00 00 X<......d....... <3>[ 683.373334] Object 00000000180c4cca: 00 00 00 00 ad 4e ad de ff ff ff ff 5a 5a 5a 5a .....N......ZZZZ <3>[ 683.373348] Object 00000000c9044498: ff ff ff ff ff ff ff ff e0 6e 57 c0 ff ff ff ff .........nW..... <3>[ 683.373362] Object 0000000072d0dfb3: 00 00 00 00 00 00 00 00 c0 b1 c8 8e ff ff ff ff ................ <3>[ 683.373376] Object 0000000081f198b9: 55 01 4b c0 ff ff ff ff d8 de 6b fa 87 9f ff ff U.K.......k..... <3>[ 683.373390] Object 000000006a375a13: d8 de 6b fa 87 9f ff ff cc 05 39 c0 ff ff ff ff ..k.......9..... <3>[ 683.373404] Object 00000000b8392dd1: ff ff ff ff 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ....ZZZZZZZZZZZZ <3>[ 683.373418] Object 00000000e5c1bbcb: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ <3>[ 683.373432] Object 00000000199feccd: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ <3>[ 683.373446] Object 0000000020f5e08b: 20 df 6b fa 87 9f ff ff 20 df 6b fa 87 9f ff ff .k..... .k..... <3>[ 683.373460] Object 0000000090591b0f: 30 df 6b fa 87 9f ff ff 30 df 6b fa 87 9f ff ff 0.k.....0.k..... <3>[ 683.373473] Object 00000000232f7cd0: 40 df 6b fa 87 9f ff ff 40 df 6b fa 87 9f ff ff @.k.....@.k..... <3>[ 683.373487] Object 0000000060458027: 50 df 6b fa 87 9f ff ff 50 df 6b fa 87 9f ff ff P.k.....P.k..... <3>[ 683.373501] Object 00000000e3c82ce2: 06 00 00 00 00 00 00 00 5a 5a 5a 5a 5a 5a 5a 5a ........ZZZZZZZZ <3>[ 683.373515] Object 00000000ec804eb8: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ <3>[ 683.373529] Object 00000000ce7ccc08: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ <3>[ 683.373543] Object 000000002dbc575c: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ <3>[ 683.373557] Object 00000000b86d3417: 5a 5a 5a 5a 5a 5a 5a 5a 00 de 6b fa 87 9f ff ff ZZZZZZZZ..k..... <3>[ 683.373571] Object 00000000d1e82276: b8 61 dd d9 87 9f ff ff a0 06 00 00 d0 06 00 00 .a.............. <3>[ 683.373585] Object 00000000cc53f969: e8 06 00 00 20 07 00 00 28 07 00 00 00 00 00 00 .... ...(....... <3>[ 683.373599] Object 00000000ea2426d2: 40 0c 8c 7b 88 9f ff ff 00 00 00 00 00 00 00 00 @..{............ <3>[ 683.373613] Object 00000000b860c1c3: 68 0d 8c 7b 88 9f ff ff 68 25 8c 7b 88 9f ff ff h..{....h%.{.... <3>[ 683.373627] Object 0000000016455ea0: 96 d5 05 00 01 00 00 00 00 5a 5a 5a 5a 5a 5a 5a .........ZZZZZZZ <3>[ 683.373640] Object 00000000e66ede82: 00 e0 6b fa 87 9f ff ff 00 e0 6b fa 87 9f ff ff ..k.......k..... <3>[ 683.373654] Object 0000000080964939: 10 e0 6b fa 87 9f ff ff 10 e0 6b fa 87 9f ff ff ..k.......k..... <3>[ 683.373668] Object 00000000e7ffc5dd: 00 00 00 00 00 00 00 00 00 01 00 00 00 00 ad de ................ <3>[ 683.373682] Object 000000000ce9d6ca: 00 02 00 00 00 00 ad de 5a 5a 5a 5a 5a 5a 5a 5a ........ZZZZZZZZ <3>[ 683.373696] Object 00000000386659d0: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ <3>[ 683.373710] Redzone 0000000075d2069d: bb bb bb bb bb bb bb bb ........ <3>[ 683.373723] Padding 0000000054e14c6b: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ <3>[ 683.373737] Padding 00000000425e5b34: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ <3>[ 683.373751] Padding 00000000ad3d4db9: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ <4>[ 683.373767] CPU: 1 PID: 151 Comm: kworker/1:2 Tainted: G BU 5.0.0-rc8-g39139489403b-drmtip_236+ #1 <4>[ 683.373769] Hardware name: Intel Corporation Ice Lake Client Platform/IceLake Y LPDDR4x T4 RVP TLC, BIOS ICLSFWR1.R00.3087.A00.1902250334 02/25/2019 <4>[ 683.373773] Workqueue: events delayed_fput <4>[ 683.373775] Call Trace: <4>[ 683.373777] <IRQ> <4>[ 683.373781] dump_stack+0x67/0x9b <4>[ 683.373783] free_debug_processing+0x344/0x370 <4>[ 683.373832] ? intel_engine_breadcrumbs_irq+0x2e4/0x380 [i915] <4>[ 683.373836] __slab_free+0x337/0x4f0 <4>[ 683.373840] ? _raw_spin_unlock_irqrestore+0x39/0x60 <4>[ 683.373844] ? debug_check_no_obj_freed+0x132/0x210 <4>[ 683.373889] ? intel_engine_breadcrumbs_irq+0x2e4/0x380 [i915] <4>[ 683.373892] ? kmem_cache_free+0x275/0x2e0 <4>[ 683.373894] kmem_cache_free+0x275/0x2e0 <4>[ 683.373939] intel_engine_breadcrumbs_irq+0x2e4/0x380 [i915] <4>[ 683.373984] gen8_cs_irq_handler+0x4e/0xa0 [i915] <4>[ 683.374026] gen11_irq_handler+0x24b/0x330 [i915] <4>[ 683.374032] __handle_irq_event_percpu+0x41/0x2d0 <4>[ 683.374034] ? handle_irq_event+0x27/0x50 <4>[ 683.374038] handle_irq_event_percpu+0x2b/0x70 <4>[ 683.374040] handle_irq_event+0x2f/0x50 <4>[ 683.374044] handle_edge_irq+0xe7/0x190 <4>[ 683.374048] handle_irq+0x67/0x160 <4>[ 683.374051] do_IRQ+0x5e/0x130 <4>[ 683.374054] common_interrupt+0xf/0xf Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=109827 Fixes: 52c0fdb25c7c ("drm/i915: Replace global breadcrumbs with per-context interrupt tracking") Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20190304114113.371-1-chris@chris-wilson.co.uk
2019-01-29drm/i915: Drop fake breadcrumb irqChris Wilson1-144/+3
Missed breadcrumb detection is defunct due to the tight coupling with dma_fence signaling and the myriad ways we may signal fences from everywhere but from an interrupt, i.e. we frequently signal a fence before we even see its interrupt. This means that even if we miss an interrupt for a fence, it still is signaled before our breadcrumb hangcheck fires, so simplify the breadcrumb hangchecking by moving it into the GPU hangcheck and forgo fake interrupts. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20190129205230.19056-3-chris@chris-wilson.co.uk
2019-01-29drm/i915: Replace global breadcrumbs with per-context interrupt trackingChris Wilson1-590/+228
A few years ago, see commit 688e6c725816 ("drm/i915: Slaughter the thundering i915_wait_request herd"), the issue of handling multiple clients waiting in parallel was brought to our attention. The requirement was that every client should be woken immediately upon its request being signaled, without incurring any cpu overhead. To handle certain fragility of our hw meant that we could not do a simple check inside the irq handler (some generations required almost unbounded delays before we could be sure of seqno coherency) and so request completion checking required delegation. Before commit 688e6c725816, the solution was simple. Every client waiting on a request would be woken on every interrupt and each would do a heavyweight check to see if their request was complete. Commit 688e6c725816 introduced an rbtree so that only the earliest waiter on the global timeline would woken, and would wake the next and so on. (Along with various complications to handle requests being reordered along the global timeline, and also a requirement for kthread to provide a delegate for fence signaling that had no process context.) The global rbtree depends on knowing the execution timeline (and global seqno). Without knowing that order, we must instead check all contexts queued to the HW to see which may have advanced. We trim that list by only checking queued contexts that are being waited on, but still we keep a list of all active contexts and their active signalers that we inspect from inside the irq handler. By moving the waiters onto the fence signal list, we can combine the client wakeup with the dma_fence signaling (a dramatic reduction in complexity, but does require the HW being coherent, the seqno must be visible from the cpu before the interrupt is raised - we keep a timer backup just in case). Having previously fixed all the issues with irq-seqno serialisation (by inserting delays onto the GPU after each request instead of random delays on the CPU after each interrupt), we can rely on the seqno state to perfom direct wakeups from the interrupt handler. This allows us to preserve our single context switch behaviour of the current routine, with the only downside that we lose the RT priority sorting of wakeups. In general, direct wakeup latency of multiple clients is about the same (about 10% better in most cases) with a reduction in total CPU time spent in the waiter (about 20-50% depending on gen). Average herd behaviour is improved, but at the cost of not delegating wakeups on task_prio. v2: Capture fence signaling state for error state and add comments to warm even the most cold of hearts. v3: Check if the request is still active before busywaiting v4: Reduce the amount of pointer misdirection with list_for_each_safe and using a local i915_request variable inside the loops v5: Add a missing pluralisation to a purely informative selftest message. References: 688e6c725816 ("drm/i915: Slaughter the thundering i915_wait_request herd") Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20190129205230.19056-2-chris@chris-wilson.co.uk
2019-01-22drm/i915: Tidy common test_bit probing of i915_request->fence.flagsChris Wilson1-2/+1
A repeated pattern is to test the signaled bit of our request->fence.flags. Make this an inline to shorten a few lines and remove unnecessary line continuations. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20190121222117.23305-20-chris@chris-wilson.co.uk
2019-01-18drm/i915: Use b->irq_enable() as predicate for mock engineChris Wilson1-13/+4
Since commit d4ccceb05591 ("drm/i915/icl: Ringbuffer interrupt handling") we have required a mechanism to avoid touching the interrupt hardware for breadcrumbs, superseding our mock interface for selftests. The residual problem (ideas welcome) is in probing the mock ring registers for ring_is_idle. Hmm, maybe we should just install mock handlers for i915->uncore.mmio__write and friends? Only problem being is that we would to truly mock some expected reads. :( References: d4ccceb05591 ("drm/i915/icl: Ringbuffer interrupt handling") Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20190118112225.13780-1-chris@chris-wilson.co.uk
2019-01-18drm/i915/breadcrumbs: Drop assertion that we've already enabled irqsChris Wilson1-17/+11
The motivation for introducing the check that we only enable breadcrumb irqs if the device's irq was installed was once upon a time we waited during suspend after disabling interrupts (which was quite slow until the bug was discovered). Since then we have the notion of pinning the breadcrumb irq, broadening it from the sole purpose of user interrupt notification and waiting, and more importantly decoupling it from a very defined time period during which enabling the irq was expected. So stop insisting the irq is installed before we setup our IMR masks, if the IER isn't yet enabled, nothing will happen and we will timeout instead, revealing the lack of irq in the hang debug messages. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20190117233126.30165-1-chris@chris-wilson.co.uk
2018-12-31drm/i915: Drop unused engine->irq_seqno_barrier w/aChris Wilson1-26/+0
Now that we have eliminated the CPU-side irq_seqno_barrier by moving the delays on the GPU before emitting the MI_USER_INTERRUPT, we can remove the engine->irq_seqno_barrier infrastructure. Though intentionally slowing down the GPU is nasty, so is the code we can now remove! Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20181228171641.16531-6-chris@chris-wilson.co.uk
2018-12-03drm/i915/breadcrumbs: Reduce missed-breadcrumb false positive rateChris Wilson1-5/+1
Change the on-cpu check to on-runqueue to catch if the waiter has been woken (and reset its current_state back to TASK_UNINTERRUPTIBLE to perform the seqno check) but is sleeping due to being preempted off the cpu. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20181203113701.12106-1-chris@chris-wilson.co.uk
2018-08-07drm/i915: Pull seqno started checks togetherChris Wilson1-4/+2
We have a few instances of checking seqno-1 to see if the HW has started the request. Pull those together under a helper. v2: Pull the !seqno assertion higher, as given seqno==1 we may indeed check to see if we have started using seqno==0. Suggested-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20180806112605.20725-1-chris@chris-wilson.co.uk
2018-06-28drm/i915: Move the irq_counter inside the spinlockChris Wilson1-4/+7
Rather than have multiple locked instructions inside the notify_ring() irq handler, move them inside the spinlock and reduce their intrinsic locking. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20180627201304.15817-3-chris@chris-wilson.co.uk
2018-05-19drm/i915: Allow init_breadcrumbs to be used from irq contextChris Wilson1-2/+3
In order to support engine reset from irq (timer) context, we need to be able to re-initialise the breadcrumbs. So we need to promote the plain spin_lock_irq to a safe spin_lock_irqsave. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20180518090212.5349-3-chris@chris-wilson.co.uk
2018-04-26drm/i915: Compile out engine debug for releaseChris Wilson1-1/+1
The majority of the engine state dumping is too voluminous to be useful outside of a controlled setup, though a few do accompany severe errors. Keep the debug dumps next to the errors, but hide the others behind a CI compile flag. This becomes more useful when adding more dumps to latency sensitive paths. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20180426103219.22181-1-chris@chris-wilson.co.uk
2018-04-24drm/i915/breadcrumbs: Keep the fake irq armed across resetChris Wilson1-9/+20
Instead of synchronously cancelling the timer and re-enabling it inside the reset callbacks, keep the timer enabled and let it die on its next wakeup if no longer required. This allows intel_engine_reset_breadcrumbs() to be used from an atomic (timer/softirq) context such as required for resetting an engine. It also allows us to react better to the user poking around debugfs for testing missed irqs. v2: Tighten the order of del_timer_sync as the fake_irq timer may trigger the hangcheck timer, and so we should cancel it first and then cancel the hangcheck (Mika) Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com> Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20180424142945.6787-1-chris@chris-wilson.co.uk
2018-03-12drm/i915: Remove the impedance mismatch around intel_engine_enable_signalingChris Wilson1-8/+13
There is some redundancy between dma_fence->ops->enable_signaling (via i915_fence_enable_signaling) and our backend, intel_engine_enable_signaling() in that both levels recheck the fence status multiple times. If we convert intel_engine_enable_signaling() to return the information desired by dma_fence->ops->enable_signaling, we can reduce i915_fence_enable_signaling to a simple stub and avoid trying to reinterpret the same information. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Mika Kuoppala <mika.kuoppala@intel.com> Cc: Michal Winiarski <michal.winiarski@intel.com> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20180308140732.25090-1-chris@chris-wilson.co.uk
2018-03-06drm/i915: Flush waiters on seqno wraparoundChris Wilson1-19/+0
Previously, we would spin waiting for all waiters to wake up and notice their request had completed before we would reset the seqno upon wraparound. However, we can mark their waits as complete and wake them up directly using the existing machinery for handling the flushing of missed wakeups when idling. Suggested-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20180306130143.13312-2-chris@chris-wilson.co.uk
2018-03-06drm/i915: Stop kicking the signaling thread on seqno wraparoundChris Wilson1-5/+0
Since commit fd10e2ce9905 ("drm/i915/breadcrumbs: Ignore unsubmitted signalers"), we cancel the signaler when retiring the request and so upon wraparound, where we wait for all requests to be retired, we no longer need to spin waiting for the signaling thread to release its references to the in-flight requests, and so we can assert that the signaler is idle. References: fd10e2ce9905 ("drm/i915/breadcrumbs: Ignore unsubmitted signalers") Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20180306130143.13312-1-chris@chris-wilson.co.uk
2018-03-06drm/i915/breadcrumbs: Assert all missed breadcrumbs were signaledChris Wilson1-0/+2
When parking the engines and their breadcrumbs, if we have waiters left then they missed their wakeup. Verify that each waiter's seqno did complete. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20180222092545.17216-2-chris@chris-wilson.co.uk
2018-03-06drm/i915/breadcrumbs: Reduce signaler rbtree to a sorted listChris Wilson1-148/+113
The goal here is to try and reduce the latency of signaling additional requests following the wakeup from interrupt by reducing the list of to-be-signaled requests from an rbtree to a sorted linked list. The original choice of using an rbtree was to facilitate random insertions of request into the signaler while maintaining a sorted list. However, if we assume that most new requests are added when they are submitted, we see those new requests in execution order making a insertion sort fast, and the reduction in overhead of each signaler iteration significant. Since commit 56299fb7d904 ("drm/i915: Signal first fence from irq handler if complete"), we signal most fences directly from notify_ring() in the interrupt handler greatly reducing the amount of work that actually needs to be done by the signaler kthread. All the thread is then required to do is operate as the bottom-half, cleaning up after the interrupt handler and preparing the next waiter. This includes signaling all later completed fences in a saturated system, but on a mostly idle system we only have to rebuild the wait rbtree in time for the next interrupt. With this de-emphasis of the signaler's role, we want to rejig it's datastructures to reduce the amount of work we require to both setup the signal tree and maintain it on every interrupt. References: 56299fb7d904 ("drm/i915: Signal first fence from irq handler if complete") Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20180222092545.17216-1-chris@chris-wilson.co.uk
2018-03-05drm/i915/icl: Ringbuffer interrupt handlingTvrtko Ursulin1-6/+10
On Gen11 interrupt masks need to be clear to allow C6 entry. We keep them all enabled knowing that we generate extra interrupts. v2: Rebase. v3: Remove gen 11 extra check in logical_render_ring_init. v4: Rebase fixes. v5: Rebase/refactor. v6: Rebase. v7: Rebase. v8: Update comment and commit message (Daniele) Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com> Cc: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com> Reviewed-by: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com> Signed-off-by: Mika Kuoppala <mika.kuoppala@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20180302161501.28594-1-mika.kuoppala@linux.intel.com
2018-02-21drm/i915: Rename drm_i915_gem_request to i915_requestChris Wilson1-16/+15
We want to de-emphasize the link between the request (dependency, execution and fence tracking) from GEM and so rename the struct from drm_i915_gem_request to i915_request. That is we may implement the GEM user interface on top of requests, but they are an abstraction for tracking execution rather than an implementation detail of GEM. (Since they are not tied to HW, we keep the i915 prefix as opposed to intel.) In short, the spatch: @@ @@ - struct drm_i915_gem_request + struct i915_request A corollary to contracting the type name, we also harmonise on using 'rq' shorthand for local variables where space if of the essence and repetition makes 'request' unwieldy. For globals and struct members, 'request' is still much preferred for its clarity. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Michał Winiarski <michal.winiarski@intel.com> Cc: Michal Wajdeczko <michal.wajdeczko@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20180221095636.6649-1-chris@chris-wilson.co.uk Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com> Reviewed-by: Michał Winiarski <michal.winiarski@intel.com> Acked-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
2018-02-07drm/i915/breadcrumbs: Ignore unsubmitted signalersChris Wilson1-16/+4
When a request is preempted, it is unsubmitted from the HW queue and removed from the active list of breadcrumbs. In the process, this however triggers the signaler and it may see the clear rbtree with the old, and still valid, seqno, or it may match the cleared seqno with the now zero rq->global_seqno. This confuses the signaler into action and signaling the fence. Fixes: d6a2289d9d6b ("drm/i915: Remove the preempted request from the execution queue") Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Cc: <stable@vger.kernel.org> # v4.12+ Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20180206094633.30181-1-chris@chris-wilson.co.uk
2018-02-05drm/i915/breadcrumbs: Drop request reference for the signaler threadChris Wilson1-69/+81
If we remember to cancel the signaler on a request when retiring it (after we know that the request has been signaled), we do not need to carry an additional request in the signaler itself. This prevents an issue whereby the signaler threads may be delayed and hold on to thousands of request references, causing severe memory fragmentation and premature oom (most noticeable on 32b snb due to the limited GFP_KERNEL and frequent use of inter-engine fences). v2: Rename first_signal(), document reads outside of locks. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20180203101914.24880-1-chris@chris-wilson.co.uk Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
2018-01-31drm/i915: Always run hangcheck while the GPU is busyChris Wilson1-11/+0
Previously, we relied on only running the hangcheck while somebody was waiting on the GPU, in order to minimise the amount of time hangcheck had to run. (If nobody was watching the GPU, nobody would notice if the GPU wasn't responding -- eventually somebody would care and so kick hangcheck into action.) However, this falls apart from around commit 4680816be336 ("drm/i915: Wait first for submission, before waiting for request completion"), as not all waiters declare themselves to hangcheck and so we could switch off hangcheck and miss GPU hangs even when waiting under the struct_mutex. If we enable hangcheck from the first request submission, and let it run until the GPU is idle again, we forgo all the complexity involved with only enabling around waiters. We just have to remember to be careful that we do not declare a GPU hang when idly waiting for the next request to be come ready, as we will run hangcheck continuously even when the engines are stalled waiting for external events. This should be true already as we should only be tracking requests submitted to hardware for execution as an indicator that the engine is busy. Fixes: 4680816be336 ("drm/i915: Wait first for submission, before waiting for request completion" Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=104840 Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Chris Wilson <chris@chris-wilson.co.uk> Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20180129144104.3921-1-chris@chris-wilson.co.uk Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
2018-01-03drm/i915: Assert we do not try to wait on an invalid seqnoChris Wilson1-0/+2
We should never insert the invalid seqno into the wait tree, so assert we do not. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Michał Winiarski <michal.winiarski@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20180102192500.20364-1-chris@chris-wilson.co.uk
2017-12-11drm/i915: Only report a wakeup if the waiter was truly asleepChris Wilson1-4/+22
If we attempt to wake up a waiter, who is currently checking the seqno it will be in the TASK_INTERRUPTIBLE state and ttwu will report success. However, it is actually awake and functioning -- so delay reporting the actual wake up until it sleeps. This fixes some spurious claims of missed_breadcrumbs when running under heavy load; i.e. sufficient load to preempt away the newly woken waiter before they complete their checks. However, it does so at the cost of a rare false negative; where the waiter changes between the check and ttwu -- the only way to fix that would be to extend the reporting from ttwu where the check could be done atomically. v2: Defend against !CONFIG_SMP v3: Don't filter out calls to wake_up_process v4: Drop risky microoptimisation to skip wakeups Testcase: igt/drv_missed_irq # sanity check we do detect missed_breadcrumb() Testcase: igt/gem_concurrent_blit # for generating false positives References: https://bugs.freedesktop.org/show_bug.cgi?id=100007 Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20171209124710.1606-1-chris@chris-wilson.co.uk Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
2017-12-11drm/i915: Stop listening to request resubmission from the signaler kthreadChris Wilson1-11/+11
The intent here was that we would be listening to i915_gem_request_unsubmit in order to cancel the signaler quickly and release the reference on the request. Cancelling the signaler is done directly via intel_engine_cancel_signaling (called from unsubmit), but that does not directly wake up the signaling thread, and neither does setting the request->global_seqno back to zero wake up listeners to the request->execute waitqueue. So the only time that listening to the request->execute waitqueue would wake up the signaling kthread would be on the request resubmission, during which time we would already receive wake ups from rejoining the global breadcrumbs wait rbtree. Trying to wake up to release the request remains an issue. If the signaling was cancelled and no other request required signaling, then it is possible for us to shutdown with the reference on the request still held. To ensure that we do not try to shutdown, leaking that request, we kick the signaling threads whenever we disarm the breadcrumbs, i.e. on parking the engine when idle. v2: We do need to be sure to release the last reference on stopping the kthread; asserting that it has been dropped already is insufficient. Fixes: d6a2289d9d6b ("drm/i915: Remove the preempted request from the execution queue") Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com> Cc: Michał Winiarski <michal.winiarski@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20171208121033.5236-1-chris@chris-wilson.co.uk Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>
2017-12-08drm/i915: Include engine state on detecting a missed breadcrumb/seqnoChris Wilson1-11/+14
Now that we have a common engine state pretty printer, we can use that instead of the adhoc information printed when we miss a breadcrumb. v2: Rearrange intel_engine_disarm_breadcrumbs() to avoid calling intel_engine_dump() under the rb spinlock (Mika) and to pretty-print the error state early so that we include the full list of waiters. v3: Pass missed breadcrumb msg to pretty-printer as the header v4: Preserve DRM_DEBUG_DRIVER filtering. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Mika Kuoppala <mika.kuoppala@intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Reviewed-by: Mika Kuoppala <mika.kuoppala@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20171208012303.25504-3-chris@chris-wilson.co.uk
2017-11-16drm/i915: Clear breadcrumb node when cancelling signalingChris Wilson1-0/+1
When we call intel_engine_cancel_signaling() to stop reporting when a request is completed via an asynchronous signal, we remove that request from the breadcrumb wait queue. However, we may be concurrently processing that request in the signaler itself, the actual operations on the request's node itself are serialised but we do not actually clear the waiter after removing it from the tree allowing both parties to attempt to do so and corrupting the rbtree. (Previously removing from the breadcrumb wait queue could only be done on behalf of i915_wait_request, so this race could not happen). Reported-by: "He, Bo" <bo.he@intel.com> Fixes: 9eb143bbec7d ("drm/i915: Allow a request to be cancelled") Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: "He, Bo" <bo.he@intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Michał Winiarski <michal.winiarski@intel.com> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20171115121458.24655-1-chris@chris-wilson.co.uk Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
2017-11-10drm/i915: Move irqs enabled assertion deeper for mock breadcrumbsChris Wilson1-2/+8
In order to allow the mock breadcrumbs tests to run without device irqs being enabled, move the intel_irqs_enabled() assert deeper to just before we commit to enabling the HW irq. v2: Add a FIXME explaining that placing the assertion so deep is not ideal, but a compromise for mock breadcrumbs. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Cc: Matthew Auld <matthew.william.auld@gmail.com> Link: https://patchwork.freedesktop.org/patch/msgid/20171107102003.1802-1-chris@chris-wilson.co.uk Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
2017-11-01drm/i915: Check that the breadcrumb wasn't disarmed automatically before parkingChris Wilson1-1/+2
We will disarm the breadcrumb interrupt if we see a user interrupt whilst no one is waiting. This may race with the call to intel_engine_disarm_breadcrumbs() triggering an assert that we aren't trying to do the same job twice. Prevent this by checking that the irq is still armed after flushing the interrupt (for the irq spinlock). Fixes: bcbd5c33a342 ("drm/i915/guc: Always enable the breadcrumbs irq") Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Cc: Michał Winiarski <michal.winiarski@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20171031122235.1395-1-chris@chris-wilson.co.uk Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
2017-10-25drm/i915/guc: Always enable the breadcrumbs irqChris Wilson1-10/+35
The execlists emulation on top of the GuC (used for scheduling and preemption) depends on the MI_USER_INTERRUPT for its notifications and tasklet action. As we always employ the irq, there is no advantage in ever disabling it while we are using the GuC, so allow us to arm the breadcrumb irq when enabling GuC submission and disarm upon disabling. The impact should be lessened by the delayed irq disabling we do (we only disable after receiving an interrupt for which no one was wanting), but allowing guc to explicitly manage the irq in relation to itself is simpler and prevents an issue with losing an interrupt for preemption as it is not coupled to an active request. Internally, we add a reference counter (breadcrumbs.irq_enabled) as a simple mechanism to allow GuC to keep the breadcrumb irq enabled. To improve upon always enabling the irq while guc is selected, we need to hook into the parking facility of intel_engines so that we only enable the breadcrumbs while the GT is active (one step better would be to individually park/unpark each engine). In effect, this means that we keep the breadcrumb irq always enabled for the entire duration the guc is busy, whereas before we would try to switch it off whenever we idled for more than interrupt with no associated waiters. The difference *should* be negligible in practice! v2: Stop abusing fence signaling (and its auxiliary data structures) to enable the breadcrumbs irqs. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Cc: Michał Winiarski <michal.winiarski@intel.com>, Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com> Reviewed-by: Michał Winiarski <michal.winiarski@intel.com>, Link: https://patchwork.freedesktop.org/patch/msgid/20171025143943.7661-3-chris@chris-wilson.co.uk
2017-10-18drm/i915: Convert timers to use timer_setup()Kees Cook1-10/+8
In preparation for unconditionally passing the struct timer_list pointer to all timer callbacks, switch to using the new timer_setup() and from_timer() to pass the timer pointer explicitly. Cc: Daniel Vetter <daniel.vetter@intel.com> Cc: Jani Nikula <jani.nikula@linux.intel.com> Cc: David Airlie <airlied@linux.ie> Cc: Chris Wilson <chris@chris-wilson.co.uk> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Oscar Mateo <oscar.mateo@intel.com> Cc: intel-gfx@lists.freedesktop.org Cc: dri-devel@lists.freedesktop.org Signed-off-by: Kees Cook <keescook@chromium.org> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Signed-off-by: Jani Nikula <jani.nikula@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20171017065304.3358-1-joonas.lahtinen@linux.intel.com
2017-09-27i915: Use %pS printk format for direct addressesHelge Deller1-1/+1
Use the %pS printk format for printing symbols from direct addresses. This is important for the ia64, ppc64 and parisc64 architectures, while on other architectures there is no difference between %pS and %pF. Fix it for consistency across the kernel. Signed-off-by: Helge Deller <deller@gmx.de> Cc: Jani Nikula <jani.nikula@linux.intel.com> Cc: David Airlie <airlied@linux.ie> Cc: intel-gfx@lists.freedesktop.org Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch> Link: https://patchwork.freedesktop.org/patch/msgid/1504729681-3504-6-git-send-email-deller@gmx.de
2017-06-08drm/i915: Skip adding the request to the signal tree is completeChris Wilson1-21/+28
Enabling the interrupt for the signaler takes a finite amount of time (a few microseconds) during which it is possible for the request to complete. Check afterwards and skip adding the request to the signal rbtree if it complete. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170608111405.16466-3-chris@chris-wilson.co.uk
2017-06-08drm/i915: Report back whether the irq was armed when adding the waiterChris Wilson1-9/+15
The important condition that we need to check after enabling the interrupt for signaling is whether the request completed in the process (and so we missed that interrupt). A large cost in enabling the signaling (rather than waiters) is in waking up the auxiliary signaling thread, but we only need to do so to catch that missed interrupt. If we know we didn't miss any interrupts (because we didn't arm the interrupt) then we can skip waking the auxiliary thread. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Reviewed-by: Mika Kuoppala <mika.kuoppala@intel.com> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170608111405.16466-2-chris@chris-wilson.co.uk
2017-04-26drm/i915: Skip waking the signaler when enabling before request submissionChris Wilson1-3/+4
If we are enabling the breadcrumbs signaling prior to submitting the request, we know that we cannot have missed the interrupt and can therefore skip immediately waking the signaler to check. This reduces a significant chunk of the __i915_gem_request_submit() overhead for inter-engine synchronisation, for example in gem_exec_whisper. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170426080659.28771-1-chris@chris-wilson.co.uk Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
2017-04-24drm/i915: Include interesting seqno in the missed breadcrumb debugChris Wilson1-2/+4
Knowing the neighbouring seqno (current on hw, last submitted to hw) provide some useful breadcrumbs to the debug log. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Link: http://patchwork.freedesktop.org/patch/msgid/20170423170619.7156-4-chris@chris-wilson.co.uk Reviewed-by: Mika Kuoppala <mika.kuoppala@intel.com>
2017-04-04drm/i915: Apply a cond_resched() to the saturated signalerChris Wilson1-1/+14
If the engine is continually completing nops, we can saturate the signaler and keep it working indefinitely. This angers the NMI watchdog! A good example is to disable semaphores on snb and run igt/gem_exec_nop - the parallel, multi-engine workloads are more than sufficient to hog the CPU, preventing the system from even processing ICMP echo replies. v2: Tvrtko dug into cond_resched() on x86 and found that it only depended upon preempt_count and not tif_need_resched() - which means that we would always call schedule() at that point. Fixes: c81d46138da6 ("drm/i915: Convert trace-irq to the breadcrumb waiter") Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Mika Kuoppala <mika.kuoppala@intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170404120531.10737-1-chris@chris-wilson.co.uk Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
2017-04-04drm/i915: Park the signaler before sleepingChris Wilson1-3/+3
If the signal to park arrives before we sleep, then we need to check kthread_should_park() before sleeping to avoid missing the signal. Otherwise, if the signal arrives whilst we are processing completed requests, we will reset the current->state back to TASK_INTERRUPTIBLE and so miss the wakeup. Fixes: fe3288b5da2c ("drm/i915: Park the breadcrumbs signaler across a GPU reset") Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Mika Kuoppala <mika.kuoppala@intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170403105124.8969-1-chris@chris-wilson.co.uk Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
2017-03-21drm/i915: Protect intel_engine_wakeup() for call from irq contextChris Wilson1-2/+3
intel_engine_wakeup() is called by nop_request_submit() which is installed to handle third party fences completed from within irq context. As such, it needs the full irqsave/irqrestore and not the partial spin_irq_lock handling. [18942.714467] ================================= [18942.719076] [ INFO: inconsistent lock state ] [18942.723522] 4.11.0-rc2-CI-CI_DRM_2368+ #1 Tainted: G U W [18942.729970] --------------------------------- [18942.734466] inconsistent {IN-SOFTIRQ-W} -> {SOFTIRQ-ON-W} usage. [18942.740594] gem_eio/1275 [HC0[0]:SC0[0]:HE1:SE1] takes: [18942.745932] (&(&fence->lock)->rlock){+.?...}, at: [<ffffffff815ec100>] dma_fence_signal+0x100/0x 230 [18942.755331] {IN-SOFTIRQ-W} state was registered at: [18942.760356] __lock_acquire+0x5d0/0x1bb0 [18942.764444] lock_acquire+0xc9/0x220 [18942.768196] _raw_spin_lock_irqsave+0x41/0x60 [18942.772747] dma_fence_signal+0x100/0x230 [18942.776927] vgem_fence_timeout+0x9/0x10 [vgem] [18942.781701] call_timer_fn+0x92/0x380 [18942.785557] expire_timers+0x150/0x1f0 [18942.789491] run_timer_softirq+0x7c/0x160 [18942.793705] __do_softirq+0x116/0x4c0 [18942.797560] irq_exit+0xa9/0xc0 [18942.800873] smp_apic_timer_interrupt+0x38/0x50 [18942.805611] apic_timer_interrupt+0x90/0xa0 [18942.810008] cpuidle_enter_state+0x135/0x380 [18942.814503] cpuidle_enter+0x12/0x20 [18942.818250] call_cpuidle+0x1e/0x40 [18942.821906] do_idle+0x17e/0x1f0 [18942.825333] cpu_startup_entry+0x18/0x20 [18942.829463] rest_init+0x127/0x130 [18942.833025] start_kernel+0x3f1/0x3fe [18942.836908] x86_64_start_reservations+0x2a/0x2c [18942.841733] x86_64_start_kernel+0x173/0x186 [18942.846234] verify_cpu+0x0/0xfc [18942.849604] irq event stamp: 30568 [18942.853140] hardirqs last enabled at (30567): [<ffffffff8110b81f>] ktime_get+0xef/0x120 [18942.861468] hardirqs last disabled at (30568): [<ffffffff81876377>] _raw_spin_lock_irqsave+0x17/0 x60 [18942.870812] softirqs last enabled at (30462): [<ffffffff81085cd9>] __do_softirq+0x1d9/0x4c0 [18942.879443] softirqs last disabled at (30439): [<ffffffff81086139>] irq_exit+0xa9/0xc0 [18942.887616] [18942.887616] other info that might help us debug this: [18942.894279] Possible unsafe locking scenario: [18942.894279] [18942.900336] CPU0 [18942.902851] ---- [18942.905362] lock(&(&fence->lock)->rlock); [18942.909647] <Interrupt> [18942.912330] lock(&(&fence->lock)->rlock); [18942.916821] [18942.916821] *** DEADLOCK *** [18942.916821] [18942.922862] 1 lock held by gem_eio/1275: [18942.926859] #0: (&(&fence->lock)->rlock){+.?...}, at: [<ffffffff815ec100>] dma_fence_signal+0x1 00/0x230 [18942.936651] [18942.936651] stack backtrace: [18942.941142] CPU: 3 PID: 1275 Comm: gem_eio Tainted: G U W 4.11.0-rc2-CI-CI_DRM_2368+ # 1 [18942.950367] Hardware name: Gigabyte Technology Co., Ltd. Z170X-UD5/Z170X-UD5-CF, BIOS F21 01/06/2 017 [18942.959756] Call Trace: [18942.962244] dump_stack+0x67/0x92 [18942.965626] print_usage_bug.part.23+0x259/0x268 [18942.970362] mark_lock+0x12c/0x6f0 [18942.973851] ? check_usage_forwards+0x130/0x130 [18942.978487] mark_held_locks+0x6f/0xa0 [18942.982329] ? _raw_spin_unlock_irq+0x27/0x50 [18942.986797] trace_hardirqs_on_caller+0x150/0x200 [18942.991599] trace_hardirqs_on+0xd/0x10 [18942.995515] _raw_spin_unlock_irq+0x27/0x50 [18942.999796] intel_engine_wakeup+0x26/0x30 [i915] [18943.004670] intel_engine_init_global_seqno+0x131/0x1a0 [i915] [18943.010745] nop_submit_request+0x2e/0x40 [i915] [18943.015476] submit_notify+0x3f/0x5c [i915] [18943.019763] __i915_sw_fence_complete+0x176/0x220 [i915] [18943.025234] ? try_to_del_timer_sync+0x4d/0x60 [18943.029825] i915_sw_fence_complete+0x25/0x40 [i915] [18943.034887] dma_i915_sw_fence_wake+0x26/0x60 [i915] [18943.039959] dma_fence_signal+0x146/0x230 [18943.044109] vgem_fence_signal_ioctl+0x6c/0xc0 [vgem] [18943.049275] drm_ioctl+0x200/0x450 [18943.052758] ? vgem_fence_attach_ioctl+0x270/0x270 [vgem] [18943.058334] do_vfs_ioctl+0x90/0x6e0 [18943.061991] ? entry_SYSCALL_64_fastpath+0x5/0xb1 [18943.066843] ? __this_cpu_preempt_check+0x13/0x20 [18943.071643] ? trace_hardirqs_on_caller+0xe7/0x200 [18943.076532] SyS_ioctl+0x3c/0x70 [18943.079842] entry_SYSCALL_64_fastpath+0x1c/0xb1 [18943.084558] RIP: 0033:0x7f0dfcc14357 [18943.088240] RSP: 002b:00007ffeb4628da8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 [18943.095996] RAX: ffffffffffffffda RBX: ffffffff8147eb93 RCX: 00007f0dfcc14357 [18943.103311] RDX: 00007ffeb4628de0 RSI: 0000000040086442 RDI: 0000000000000005 [18943.110574] RBP: ffffc9000176ff88 R08: 0000000000000004 R09: 0000000000000000 [18943.117845] R10: 0000000000000029 R11: 0000000000000246 R12: 0000000000000001 [18943.125168] R13: 0000000000000005 R14: 0000000040086442 R15: 0000000000000000 [18943.132520] ? __this_cpu_preempt_check+0x13/0x20 Fixes: cdc3a4539034 ("drm/i915: No need to save/restore irq status in intel_engine_wakeup") Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Mika Kuoppala <mika.kuoppala@intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170320143133.1507-1-chris@chris-wilson.co.uk Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
2017-03-16drm/i915/breadcrumbs: Tweak commentaryChris Wilson1-8/+17
Tvrtko spotted a stale reference to b->lock (now b->rb_lock) so review the comments and try to improve them in passing. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170315222259.1469-1-chris@chris-wilson.co.uk Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
2017-03-15drm/i915/breadcrumbs: Assert that we do not shortcut the current bottom-halfChris Wilson1-1/+4
We need to ensure that we always serialize updates to the bottom-half using the breadcrumbs.irq_lock so that we don't race with a concurrent interrupt handler. This is most important just prior to leaving the waiter (when the intel_wait will be overwritten), so make sure we are not the current bottom-half when skipping the irq locks. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170315210726.12095-4-chris@chris-wilson.co.uk
2017-03-15drm/i915/breadcrumbs: Disable interrupt bottom-half first on idlingChris Wilson1-7/+8
Before walking the rbtree of waiters (marking them as complete and waking them), decouple the interrupt handler. This prevents a race between the missed waiter waking up and removing its intel_wait (which skips checking the lock) and the interrupt handler dereferencing the intel_wait. (Though we do not expect to encounter waiters during idle!) Fixes: e1c0c91bdaec ("drm/i915: Wake up all waiters before idling") Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Mika Kuoppala <mika.kuoppala@intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170315210726.12095-3-chris@chris-wilson.co.uk
2017-03-15drm/i915/breadcrumbs: Update bottom-half before marking as completeChris Wilson1-17/+17
When adding a new request to the breadcrumb rbtree, we mark all those requests inside the rbtree that are already completed as complete. This wakes those waiters up and allows them to skip the spinlock before returning to userspace. If one of those is the current bottom-half and allocated its intel_wait on the stack, it may then overwrite the b->irq_wait upon exiting i915_wait_request() just as the interrupt handler dereferences it. Fixes: 56299fb7d904 ("drm/i915: Signal first fence from irq handler if complete") Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170315210726.12095-2-chris@chris-wilson.co.uk
2017-03-15drm/i915/breadcrumbs: Use booleans for intel_breadcrumbs_busy()Chris Wilson1-2/+2
Since commit 9b6586ae9f6b ("drm/i915: Keep a global seqno per-engine") converted intel_breadcrumbs_busy() to reporting a single boolean, we need only compute a boolean internally (and not needlessly compute the flag). Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Mika Kuoppala <mika.kuoppala@intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: http://patchwork.freedesktop.org/patch/msgid/20170315210726.12095-1-chris@chris-wilson.co.uk
2017-03-08Merge remote-tracking branch 'airlied/drm-next' into drm-intel-next-queuedDaniel Vetter1-0/+1
Backmerge drm-next to get at all the good stuff in drm-misc. We need that because: - drm_connector_list_iter conversion for i915 needs the core patches. - Maarten's patches to use the new atomic state iterators also need the core patches. - We need the new link status property to complete the DP retraining work, merging through 2 branches wasn't a good idea and we had to partially backtrack. - Chris needs reservation_object_trylock and we want to roll out kref_read everywhere. Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
2017-03-08Merge tag 'drm-intel-next-2017-03-06' of git://anongit.freedesktop.org/git/drm-intel into drm-nextDave Airlie1-176/+352
4 weeks worth of stuff since I was traveling&lazy: - lspcon improvements (Imre) - proper atomic state for cdclk handling (Ville) - gpu reset improvements (Chris) - lots and lots of polish around fences, requests, waiting and everything related all over (both gem and modeset code), from Chris - atomic by default on gen5+ minus byt/bsw (Maarten did the patch to flip the default, really this is a massive joint team effort) - moar power domains, now 64bit (Ander) - big pile of in-kernel unit tests for various gem subsystems (Chris), including simple mock objects for i915 device and and the ggtt manager. - i915_gpu_info in debugfs, for taking a snapshot of the current gpu state. Same thing as i915_error_state, but useful if the kernel didn't notice something is stick. From Chris. - bxt dsi fixes (Umar Shankar) - bxt w/a updates (Jani) - no more struct_mutex for gem object unreference (Chris) - some execlist refactoring (Tvrtko) - color manager support for glk (Ander) - improve the power-well sync code to better take over from the firmware (Imre) - gem tracepoint polish (Tvrtko) - lots of glk fixes all around (Ander) - ctx switch improvements (Chris) - glk dsi support&fixes (Deepak M) - dsi fixes for vlv and clanups, lots of them (Hans de Goede) - switch to i915.ko types in lots of our internal modeset code (Ander) - byt/bsw atomic wm update code, yay (Ville) * tag 'drm-intel-next-2017-03-06' of git://anongit.freedesktop.org/git/drm-intel: (432 commits) drm/i915: Update DRIVER_DATE to 20170306 drm/i915: Don't use enums for hardware engine id drm/i915: Split breadcrumbs spinlock into two drm/i915: Refactor wakeup of the next breadcrumb waiter drm/i915: Take reference for signaling the request from hardirq drm/i915: Add FIFO underrun tracepoints drm/i915: Add cxsr toggle tracepoint drm/i915: Add VLV/CHV watermark/FIFO programming tracepoints drm/i915: Add plane update/disable tracepoints drm/i915: Kill level 0 wm hack for VLV/CHV drm/i915: Workaround VLV/CHV sprite1->sprite0 enable underrun drm/i915: Sanitize VLV/CHV watermarks properly drm/i915: Only use update_wm_{pre,post} for pre-ilk platforms drm/i915: Nuke crtc->wm.cxsr_allowed drm/i915: Compute proper intermediate wms for vlv/cvh drm/i915: Skip useless watermark/FIFO related work on VLV/CHV when not needed drm/i915: Compute vlv/chv wms the atomic way drm/i915: Compute VLV/CHV FIFO sizes based on the PM2 watermarks drm/i915: Plop vlv/chv fifo sizes into crtc state drm/i915: Plop vlv wm state into crtc_state ...
2017-03-07drm/i915: No need to save/restore irq status in intel_engine_wakeupTvrtko Ursulin1-3/+2
It is called from either the process or timer context so it is correct to always disable interrupts. Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Chris Wilson <chris@chris-wilson.co.uk> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Link: http://patchwork.freedesktop.org/patch/msgid/20170306150321.29024-1-tvrtko.ursulin@linux.intel.com
Copyright © 1996 – 2023 Jason A. Donenfeld. All Rights Reverse Engineered.