diff options
| author |   Frederic Weisbecker <frederic@kernel.org> | 2021-02-23 01:10:11 +0100 |
|---|---|---|
| committer |   Paul E. McKenney <paulmck@kernel.org> | 2021-05-12 12:10:23 -0700 |
| commit | e75bcd48e2c4026b1f3feda916a2327b1744d664 (patch) | |
| tree | 1891eba686d079afc2ed4370282e2d58cb624b97 /include/trace/events/rcu.h | |
| parent | rcu/nocb: Prepare for fine-grained deferred wakeup (diff) | |
| download | linux-dev-e75bcd48e2c4026b1f3feda916a2327b1744d664.tar.xz linux-dev-e75bcd48e2c4026b1f3feda916a2327b1744d664.zip | |
rcu/nocb: Unify timers
Now that ->nocb_timer and ->nocb_bypass_timer have become quite similar,
this commit merges them together. A new RCU_NOCB_WAKE_BYPASS wake level
is introduced. As a result, timers perform all kinds of deferred wake
ups but other deferred wakeup callsites only handle non-bypass wakeups
in order not to wake up rcuo too early.
The timer also unconditionally executes a full barrier so as to order
timer_pending() and callback enqueue although the path performing
RCU_NOCB_WAKE_FORCE that makes use of it is debatable. It should also
test against the rdp leader instead of the current rdp.
This unconditional full barrier shouldn't bring visible overhead since
these timers almost never fire.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Josh Triplett <josh@joshtriplett.org>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Joel Fernandes <joel@joelfernandes.org>
Cc: Neeraj Upadhyay <neeraju@codeaurora.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Diffstat (limited to 'include/trace/events/rcu.h')
| -rw-r--r-- | include/trace/events/rcu.h | 1 |
1 files changed, 1 insertions, 0 deletions
diff --git a/include/trace/events/rcu.h b/include/trace/events/rcu.h index 6768b64bc738..670e41783edd 100644 --- a/include/trace/events/rcu.h +++ b/include/trace/events/rcu.h @@ -278,6 +278,7 @@ TRACE_EVENT_RCU(rcu_exp_funnel_lock, * "WakeNot": Don't wake rcuo kthread. * "WakeNotPoll": Don't wake rcuo kthread because it is polling. * "WakeOvfIsDeferred": Wake rcuo kthread later, CB list is huge. + * "WakeBypassIsDeferred": Wake rcuo kthread later, bypass list is contended. * "WokeEmpty": rcuo CB kthread woke to find empty list. */ TRACE_EVENT_RCU(rcu_nocb_wake, |