diff options
| author |   I Hsin Cheng <richard120310@gmail.com> | 2025-02-10 18:30:18 +0800 |
|---|---|---|
| committer |   Peter Zijlstra <peterz@infradead.org> | 2025-02-14 10:32:01 +0100 |
| commit | d34e798094ca7be935b629a42f8b237d4d5b7f1d (patch) | |
| tree | 4f52b9ef6456619ef1dc381ae5b0814fb7d88d1f | |
| parent | sched/eevdf: Force propagating min_slice of cfs_rq when {en,de}queue tasks (diff) | |
| download | wireguard-linux-d34e798094ca7be935b629a42f8b237d4d5b7f1d.tar.xz wireguard-linux-d34e798094ca7be935b629a42f8b237d4d5b7f1d.zip | |
sched/fair: Refactor can_migrate_task() to elimate looping
The function "can_migrate_task()" utilize "for_each_cpu_and" with a
"if" statement inside to find the destination cpu. It's the same logic
to find the first set bit of the result of the bitwise-AND of
"env->dst_grpmask", "env->cpus" and "p->cpus_ptr".
Refactor it by using "cpumask_first_and_and()" to perform bitwise-AND
for "env->dst_grpmask", "env->cpus" and "p->cpus_ptr" and pick the
first cpu within the intersection as the destination cpu, so we can
elimate the need of looping and multiple times of branch.
After the refactoring this part of the code can speed up from ~115ns
to ~54ns, according to the test below.
Ran the test for 5 times and the result is showned in the following
table, and the test script is paste in next section.
-------------------------------------------------------
|Old method| 130| 118| 115| 109| 106| avg ~115ns|
-------------------------------------------------------
|New method| 58| 55| 54| 48| 55| avg ~54ns|
-------------------------------------------------------
Signed-off-by: I Hsin Cheng <richard120310@gmail.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20250210103019.283824-1-richard120310@gmail.com
Diffstat (limited to '')
| -rw-r--r-- | kernel/sched/fair.c | 11 |
1 files changed, 5 insertions, 6 deletions
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 9279bfb74040..857808da23d8 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -9439,12 +9439,11 @@ int can_migrate_task(struct task_struct *p, struct lb_env *env) return 0; /* Prevent to re-select dst_cpu via env's CPUs: */ - for_each_cpu_and(cpu, env->dst_grpmask, env->cpus) { - if (cpumask_test_cpu(cpu, p->cpus_ptr)) { - env->flags |= LBF_DST_PINNED; - env->new_dst_cpu = cpu; - break; - } + cpu = cpumask_first_and_and(env->dst_grpmask, env->cpus, p->cpus_ptr); + + if (cpu < nr_cpu_ids) { + env->flags |= LBF_DST_PINNED; + env->new_dst_cpu = cpu; } return 0; |