diff options
| author |   Nicholas Piggin <npiggin@gmail.com> | 2020-12-17 23:47:30 +1000 |
|---|---|---|
| committer |   Michael Ellerman <mpe@ellerman.id.au> | 2021-02-09 01:09:45 +1100 |
| commit | 032b7f08932c9b212952d6d585e45b2941b3e8be (patch) | |
| tree | b177ec1a30b3623584c124d6957b580afe663ee6 /arch/powerpc/mm/book3s64/pgtable.c | |
| parent | powerpc/64s/radix: occasionally attempt to trim mm_cpumask (diff) | |
| download | linux-dev-032b7f08932c9b212952d6d585e45b2941b3e8be.tar.xz linux-dev-032b7f08932c9b212952d6d585e45b2941b3e8be.zip | |
powerpc/64s/radix: serialize_against_pte_lookup IPIs trim mm_cpumask
serialize_against_pte_lookup() performs IPIs to all CPUs in mm_cpumask.
Take this opportunity to try trim the CPU out of mm_cpumask. This can
reduce the cost of future serialize_against_pte_lookup() and/or the
cost of future TLB flushes.
Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
Link: https://lore.kernel.org/r/20201217134731.488135-7-npiggin@gmail.com
Diffstat (limited to 'arch/powerpc/mm/book3s64/pgtable.c')
| -rw-r--r-- | arch/powerpc/mm/book3s64/pgtable.c | 13 |
1 files changed, 10 insertions, 3 deletions
diff --git a/arch/powerpc/mm/book3s64/pgtable.c b/arch/powerpc/mm/book3s64/pgtable.c index 5b3a3bae21aa..78c492e86752 100644 --- a/arch/powerpc/mm/book3s64/pgtable.c +++ b/arch/powerpc/mm/book3s64/pgtable.c @@ -79,10 +79,17 @@ void set_pmd_at(struct mm_struct *mm, unsigned long addr, return set_pte_at(mm, addr, pmdp_ptep(pmdp), pmd_pte(pmd)); } -static void do_nothing(void *unused) -{ +void exit_lazy_flush_tlb(struct mm_struct *mm, bool always_flush); +static void do_serialize(void *arg) +{ + /* We've taken the IPI, so try to trim the mask while here */ + if (radix_enabled()) { + struct mm_struct *mm = arg; + exit_lazy_flush_tlb(mm, false); + } } + /* * Serialize against find_current_mm_pte which does lock-less * lookup in page tables with local interrupts disabled. For huge pages @@ -96,7 +103,7 @@ static void do_nothing(void *unused) void serialize_against_pte_lookup(struct mm_struct *mm) { smp_mb(); - smp_call_function_many(mm_cpumask(mm), do_nothing, NULL, 1); + smp_call_function_many(mm_cpumask(mm), do_serialize, mm, 1); } /* |