diff options
| author |   Mel Gorman <mgorman@techsingularity.net> | 2021-08-04 12:58:56 +0100 |
|---|---|---|
| committer |   Peter Zijlstra <peterz@infradead.org> | 2021-08-04 15:16:44 +0200 |
| commit | 89aafd67f28c9e3b725aa30b44b7f61ad3e348ce (patch) | |
| tree | d4a91efe0fcb31601011a3f24476dfb4e0c0e480 /kernel/sched/fair.c | |
| parent | sched: Don't report SCHED_FLAG_SUGOV in sched_getattr() (diff) | |
| download | linux-dev-89aafd67f28c9e3b725aa30b44b7f61ad3e348ce.tar.xz linux-dev-89aafd67f28c9e3b725aa30b44b7f61ad3e348ce.zip | |
sched/fair: Use prev instead of new target as recent_used_cpu
After select_idle_sibling, p->recent_used_cpu is set to the
new target. However on the next wakeup, prev will be the same as
recent_used_cpu unless the load balancer has moved the task since the
last wakeup. It still works, but is less efficient than it could be.
This patch preserves recent_used_cpu for longer.
The impact on SIS efficiency is tiny so the SIS statistic patches were
used to track the hit rate for using recent_used_cpu. With perf bench
pipe on a 2-socket Cascadelake machine, the hit rate went from 57.14%
to 85.32%. For more intensive wakeup loads like hackbench, the hit rate
is almost negligible but rose from 0.21% to 6.64%. For scaling loads
like tbench, the hit rate goes from almost 0% to 25.42% overall. Broadly
speaking, on tbench, the success rate is much higher for lower thread
counts and drops to almost 0 as the workload scales to towards saturation.
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20210804115857.6253-2-mgorman@techsingularity.net
Diffstat (limited to 'kernel/sched/fair.c')
| -rw-r--r-- | kernel/sched/fair.c | 4 |
1 files changed, 1 insertions, 3 deletions
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 13c3fd40bd34..7ee394b4b4cd 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -6347,6 +6347,7 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target) /* Check a recently used CPU as a potential idle candidate: */ recent_used_cpu = p->recent_used_cpu; + p->recent_used_cpu = prev; if (recent_used_cpu != prev && recent_used_cpu != target && cpus_share_cache(recent_used_cpu, target) && @@ -6873,9 +6874,6 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int wake_flags) } else if (wake_flags & WF_TTWU) { /* XXX always ? */ /* Fast path */ new_cpu = select_idle_sibling(p, prev_cpu, new_cpu); - - if (want_affine) - current->recent_used_cpu = cpu; } rcu_read_unlock(); |