diff options
| author |   Jisheng Zhang <jszhang@kernel.org> | 2021-04-17 00:37:22 +0800 |
|---|---|---|
| committer |   Palmer Dabbelt <palmerdabbelt@google.com> | 2021-05-25 22:50:47 -0700 |
| commit | 3332f4190674114e08daaf6859c11a7e464bceff (patch) | |
| tree | 1cc6c4dac63d80494f602016964b91833353b992 /arch/riscv/include/asm/pgtable.h | |
| parent | riscv: mm: add THP support on 64-bit (diff) | |
| download | linux-dev-3332f4190674114e08daaf6859c11a7e464bceff.tar.xz linux-dev-3332f4190674114e08daaf6859c11a7e464bceff.zip | |
riscv: mremap speedup - enable HAVE_MOVE_PUD and HAVE_MOVE_PMD
HAVE_MOVE_PUD enables remapping pages at the PUD level if both the source
and destination addresses are PUD-aligned.
HAVE_MOVE_PMD does similar speedup on the PMD level.
With HAVE_MOVE_PUD enabled, there is about a 143x improvement on qemu
With HAVE_MOVE_PMD enabled, there is about a 5x improvement on qemu
Signed-off-by: Jisheng Zhang <jszhang@kernel.org>
Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
Diffstat (limited to 'arch/riscv/include/asm/pgtable.h')
| -rw-r--r-- | arch/riscv/include/asm/pgtable.h | 11 |
1 files changed, 11 insertions, 0 deletions
diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 4b708ae08910..3b72862a83fa 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -241,6 +241,11 @@ static inline pte_t pmd_pte(pmd_t pmd) return __pte(pmd_val(pmd)); } +static inline pte_t pud_pte(pud_t pud) +{ + return __pte(pud_val(pud)); +} + /* Yields the page frame number (PFN) of a page table entry */ static inline unsigned long pte_pfn(pte_t pte) { @@ -570,6 +575,12 @@ static inline void set_pmd_at(struct mm_struct *mm, unsigned long addr, return set_pte_at(mm, addr, (pte_t *)pmdp, pmd_pte(pmd)); } +static inline void set_pud_at(struct mm_struct *mm, unsigned long addr, + pud_t *pudp, pud_t pud) +{ + return set_pte_at(mm, addr, (pte_t *)pudp, pud_pte(pud)); +} + #ifdef CONFIG_TRANSPARENT_HUGEPAGE static inline int pmd_trans_huge(pmd_t pmd) { |