aboutsummaryrefslogtreecommitdiffstats
path: root/arch/arm/include/asm/memory.h (follow)
AgeCommit message (Collapse)AuthorFilesLines
2021-08-10ARM: 9104/2: Fix Keystone 2 kernel mapping regressionLinus Walleij1-3/+4
This fixes a Keystone 2 regression discovered as a side effect of defining an passing the physical start/end sections of the kernel to the MMU remapping code. As the Keystone applies an offset to all physical addresses, including those identified and patches by phys2virt, we fail to account for this offset in the kernel_sec_start and kernel_sec_end variables. Further these offsets can extend into the 64bit range on LPAE systems such as the Keystone 2. Fix it like this: - Extend kernel_sec_start and kernel_sec_end to be 64bit - Add the offset also to kernel_sec_start and kernel_sec_end As passing kernel_sec_start and kernel_sec_end as 64bit invariably incurs BE8 endianness issues I have attempted to dry-code around these. Tested on the Vexpress QEMU model both with and without LPAE enabled. Fixes: 6e121df14ccd ("ARM: 9090/1: Map the lowmem and kernel separately") Reported-by: Nishanth Menon <nmenon@kernel.org> Suggested-by: Russell King <rmk+kernel@armlinux.org.uk> Tested-by: Grygorii Strashko <grygorii.strashko@ti.com> Tested-by: Nishanth Menon <nmenon@kernel.org> Signed-off-by: Linus Walleij <linus.walleij@linaro.org> Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
2021-06-21ARM: 9097/1: mmu: Declare section start/end correctlyLinus Walleij1-2/+2
The kernel test robot reported an interesting bug: A debug print was using %08x with kernel_sec_start and kernel_sec_end being phys_addr_t which can be either u32 or u64 (possibly more). Actually these should just be declared as u32 to begin with: they are declared as such in the assembly in head.S and the kernel definitely boots in a 32 bit physical address space. Redeclare the kernel_sec_start and kernel_sec_end to rid the bug. Reported-by: kernel test robot <lkp@intel.com> Fixes: 6e121df14ccd ("ARM: 9090/1: Map the lowmem and kernel separately") Signed-off-by: Linus Walleij <linus.walleij@linaro.org> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
2021-06-13ARM: 9089/1: Define kernel physical section start and endLinus Walleij1-0/+7
When we are mapping the initial sections in head.S we know very well where the start and end of the kernel image in physical memory is placed. Later on it gets hard to determine this. Save the information into two variables named kernel_sec_start and kernel_sec_end for convenience for later work involving the physical start and end of the kernel. These variables are section-aligned corresponding to the early section mappings set up in head.S. Signed-off-by: Linus Walleij <linus.walleij@linaro.org> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
2021-06-13ARM: 9088/1: Split KERNEL_OFFSET from PAGE_OFFSETLinus Walleij1-1/+7
We want to be able to compile the kernel into an address different from PAGE_OFFSET (start of lowmem) + TEXT_OFFSET, so start to pry apart the address of where the kernel is located from the address where the lowmem is located by defining and using KERNEL_OFFSET in a few key places. Signed-off-by: Linus Walleij <linus.walleij@linaro.org> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
2021-03-09ARM: 9059/1: cache-v7: get rid of mini-stackArd Biesheuvel1-15/+0
Now that we have reduced the number of registers that we need to preserve when calling v7_invalidate_l1 from the boot code, we can use scratch registers to preserve the remaining ones, and get rid of the mini stack entirely. This works around any issues regarding cache behavior in relation to the uncached accesses to this memory, which is hard to get right in the general case (i.e., both bare metal and under virtualization) While at it, switch v7_invalidate_l1 to using ip as a scratch register instead of r4. This makes the function AAPCS compliant, and removes the need to stash r4 in ip across the call. Acked-by: Nicolas Pitre <nico@fluxnic.net> Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
2020-12-21Merge branch 'devel-stable' into for-nextRussell King1-17/+40
2020-10-28ARM: p2v: reduce p2v alignment requirement to 2 MiBArd Biesheuvel1-5/+8
The ARM kernel's linear map starts at PAGE_OFFSET, which maps to a physical address (PHYS_OFFSET) that is platform specific, and is discovered at boot. Since we don't want to slow down translations between physical and virtual addresses by keeping the offset in a variable in memory, we implement this by patching the code performing the translation, and putting the offset between PAGE_OFFSET and the start of physical RAM directly into the instruction opcodes. As we only patch up to 8 bits of offset, yielding 4 GiB >> 8 == 16 MiB of granularity, we have to round up PHYS_OFFSET to the next multiple if the start of physical RAM is not a multiple of 16 MiB. This wastes some physical RAM, since the memory that was skipped will now live below PAGE_OFFSET, making it inaccessible to the kernel. We can improve this by changing the patchable sequences and the patching logic to carry more bits of offset: 11 bits gives us 4 GiB >> 11 == 2 MiB of granularity, and so we will never waste more than that amount by rounding up the physical start of DRAM to the next multiple of 2 MiB. (Note that 2 MiB granularity guarantees that the linear mapping can be created efficiently, whereas less than 2 MiB may result in the linear mapping needing another level of page tables) This helps Zhen Lei's scenario, where the start of DRAM is known to be occupied. It also helps EFI boot, which relies on the firmware's page allocator to allocate space for the decompressed kernel as low as possible. And if the KASLR patches ever land for 32-bit, it will give us 3 more bits of randomization of the placement of the kernel inside the linear region. For the ARM code path, it simply comes down to using two add/sub instructions instead of one for the carryless version, and patching each of them with the correct immediate depending on the rotation field. For the LPAE calculation, which has to deal with a carry, it patches the MOVW instruction with up to 12 bits of offset (but we only need 11 bits anyway) For the Thumb2 code path, patching more than 11 bits of displacement would be somewhat cumbersome, but the 11 bits we need fit nicely into the second word of the u16[2] opcode, so we simply update the immediate assignment and the left shift to create an addend of the right magnitude. Suggested-by: Zhen Lei <thunder.leizhen@huawei.com> Acked-by: Nicolas Pitre <nico@fluxnic.net> Acked-by: Linus Walleij <linus.walleij@linaro.org> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
2020-10-28ARM: p2v: switch to MOVW for Thumb2 and ARM/LPAEArd Biesheuvel1-12/+32
In preparation for reducing the phys-to-virt minimum relative alignment from 16 MiB to 2 MiB, switch to patchable sequences involving MOVW instructions that can more easily be manipulated to carry a 12-bit immediate. Note that the non-LPAE ARM sequence is not updated: MOVW may not be supported on non-LPAE platforms, and the sequence itself can be updated more easily to apply the 12 bits of displacement. For Thumb2, which has many more versions of opcodes, switch to a sequence that can be patched by the same patching code for both versions. Note that the Thumb2 opcodes for MOVW and MVN are unambiguous, and have no rotation bits in their immediate fields, so there is no need to use placeholder constants in the asm blocks. While at it, drop the 'volatile' qualifiers from the asm blocks: the code does not have any side effects that are invisible to the compiler, so it is free to omit these sequences if the outputs are not used. Suggested-by: Russell King <linux@armlinux.org.uk> Acked-by: Nicolas Pitre <nico@fluxnic.net> Reviewed-by: Linus Walleij <linus.walleij@linaro.org> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
2020-10-28ARM: p2v: use relative references in patch site arraysArd Biesheuvel1-3/+3
Free up a register in the p2v patching code by switching to relative references, which don't require keeping the phys-to-virt displacement live in a register. Acked-by: Nicolas Pitre <nico@fluxnic.net> Reviewed-by: Linus Walleij <linus.walleij@linaro.org> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
2020-10-28ARM: p2v: drop redundant 'type' argument from __pv_stubArd Biesheuvel1-4/+4
We always pass the same value for 'type' so pull it into the __pv_stub macro itself. Acked-by: Nicolas Pitre <nico@fluxnic.net> Reviewed-by: Linus Walleij <linus.walleij@linaro.org> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
2020-10-28ARM: 9020/1: mm: use correct section size macro to describe the FDT virtual addressArd Biesheuvel1-3/+3
Commit 149a3ffe62b9dbc3 ("9012/1: move device tree mapping out of linear region") created a permanent, read-only section mapping of the device tree blob provided by the firmware, and added a set of macros to get the base and size of the virtually mapped FDT based on the physical address. However, while the mapping code uses the SECTION_SIZE macro correctly, the macros use PMD_SIZE instead, which means something entirely different on ARM when using short descriptors, and is therefore not the right quantity to use here. So replace PMD_SIZE with SECTION_SIZE. While at it, change the names of the macro and its parameter to clarify that it returns the virtual address of the start of the FDT, based on the physical address in memory. Tested-by: Joel Stanley <joel@jms.id.au> Tested-by: Marek Szyprowski <m.szyprowski@samsung.com> Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
2020-10-27ARM: 9015/2: Define the virtual space of KASan's shadow regionLinus Walleij1-0/+5
Define KASAN_SHADOW_OFFSET,KASAN_SHADOW_START and KASAN_SHADOW_END for the Arm kernel address sanitizer. We are "stealing" lowmem (the 4GB addressable by a 32bit architecture) out of the virtual address space to use as shadow memory for KASan as follows: +----+ 0xffffffff | | | | |-> Static kernel image (vmlinux) BSS and page table | |/ +----+ PAGE_OFFSET | | | | |-> Loadable kernel modules virtual address space area | |/ +----+ MODULES_VADDR = KASAN_SHADOW_END | | | | |-> The shadow area of kernel virtual address. | |/ +----+-> TASK_SIZE (start of kernel space) = KASAN_SHADOW_START the | | shadow address of MODULES_VADDR | | | | | | | | |-> The user space area in lowmem. The kernel address | | | sanitizer do not use this space, nor does it map it. | | | | | | | | | | | | | |/ ------ 0 0 .. TASK_SIZE is the memory that can be used by shared userspace/kernelspace. It us used for userspace processes and for passing parameters and memory buffers in system calls etc. We do not need to shadow this area. KASAN_SHADOW_START: This value begins with the MODULE_VADDR's shadow address. It is the start of kernel virtual space. Since we have modules to load, we need to cover also that area with shadow memory so we can find memory bugs in modules. KASAN_SHADOW_END This value is the 0x100000000's shadow address: the mapping that would be after the end of the kernel memory at 0xffffffff. It is the end of kernel address sanitizer shadow area. It is also the start of the module area. KASAN_SHADOW_OFFSET: This value is used to map an address to the corresponding shadow address by the following formula: shadow_addr = (address >> 3) + KASAN_SHADOW_OFFSET; As you would expect, >> 3 is equal to dividing by 8, meaning each byte in the shadow memory covers 8 bytes of kernel memory, so one bit shadow memory per byte of kernel memory is used. The KASAN_SHADOW_OFFSET is provided in a Kconfig option depending on the VMSPLIT layout of the system: the kernel and userspace can split up lowmem in different ways according to needs, so we calculate the shadow offset depending on this. When kasan is enabled, the definition of TASK_SIZE is not an 8-bit rotated constant, so we need to modify the TASK_SIZE access code in the *.s file. The kernel and modules may use different amounts of memory, according to the VMSPLIT configuration, which in turn determines the PAGE_OFFSET. We use the following KASAN_SHADOW_OFFSETs depending on how the virtual memory is split up: - 0x1f000000 if we have 1G userspace / 3G kernelspace split: - The kernel address space is 3G (0xc0000000) - PAGE_OFFSET is then set to 0x40000000 so the kernel static image (vmlinux) uses addresses 0x40000000 .. 0xffffffff - On top of that we have the MODULES_VADDR which under the worst case (using ARM instructions) is PAGE_OFFSET - 16M (0x01000000) = 0x3f000000 so the modules use addresses 0x3f000000 .. 0x3fffffff - So the addresses 0x3f000000 .. 0xffffffff need to be covered with shadow memory. That is 0xc1000000 bytes of memory. - 1/8 of that is needed for its shadow memory, so 0x18200000 bytes of shadow memory is needed. We "steal" that from the remaining lowmem. - The KASAN_SHADOW_START becomes 0x26e00000, to KASAN_SHADOW_END at 0x3effffff. - Now we can calculate the KASAN_SHADOW_OFFSET for any kernel address as 0x3f000000 needs to map to the first byte of shadow memory and 0xffffffff needs to map to the last byte of shadow memory. Since: SHADOW_ADDR = (address >> 3) + KASAN_SHADOW_OFFSET 0x26e00000 = (0x3f000000 >> 3) + KASAN_SHADOW_OFFSET KASAN_SHADOW_OFFSET = 0x26e00000 - (0x3f000000 >> 3) KASAN_SHADOW_OFFSET = 0x26e00000 - 0x07e00000 KASAN_SHADOW_OFFSET = 0x1f000000 - 0x5f000000 if we have 2G userspace / 2G kernelspace split: - The kernel space is 2G (0x80000000) - PAGE_OFFSET is set to 0x80000000 so the kernel static image uses 0x80000000 .. 0xffffffff. - On top of that we have the MODULES_VADDR which under the worst case (using ARM instructions) is PAGE_OFFSET - 16M (0x01000000) = 0x7f000000 so the modules use addresses 0x7f000000 .. 0x7fffffff - So the addresses 0x7f000000 .. 0xffffffff need to be covered with shadow memory. That is 0x81000000 bytes of memory. - 1/8 of that is needed for its shadow memory, so 0x10200000 bytes of shadow memory is needed. We "steal" that from the remaining lowmem. - The KASAN_SHADOW_START becomes 0x6ee00000, to KASAN_SHADOW_END at 0x7effffff. - Now we can calculate the KASAN_SHADOW_OFFSET for any kernel address as 0x7f000000 needs to map to the first byte of shadow memory and 0xffffffff needs to map to the last byte of shadow memory. Since: SHADOW_ADDR = (address >> 3) + KASAN_SHADOW_OFFSET 0x6ee00000 = (0x7f000000 >> 3) + KASAN_SHADOW_OFFSET KASAN_SHADOW_OFFSET = 0x6ee00000 - (0x7f000000 >> 3) KASAN_SHADOW_OFFSET = 0x6ee00000 - 0x0fe00000 KASAN_SHADOW_OFFSET = 0x5f000000 - 0x9f000000 if we have 3G userspace / 1G kernelspace split, and this is the default split for ARM: - The kernel address space is 1GB (0x40000000) - PAGE_OFFSET is set to 0xc0000000 so the kernel static image uses 0xc0000000 .. 0xffffffff. - On top of that we have the MODULES_VADDR which under the worst case (using ARM instructions) is PAGE_OFFSET - 16M (0x01000000) = 0xbf000000 so the modules use addresses 0xbf000000 .. 0xbfffffff - So the addresses 0xbf000000 .. 0xffffffff need to be covered with shadow memory. That is 0x41000000 bytes of memory. - 1/8 of that is needed for its shadow memory, so 0x08200000 bytes of shadow memory is needed. We "steal" that from the remaining lowmem. - The KASAN_SHADOW_START becomes 0xb6e00000, to KASAN_SHADOW_END at 0xbfffffff. - Now we can calculate the KASAN_SHADOW_OFFSET for any kernel address as 0xbf000000 needs to map to the first byte of shadow memory and 0xffffffff needs to map to the last byte of shadow memory. Since: SHADOW_ADDR = (address >> 3) + KASAN_SHADOW_OFFSET 0xb6e00000 = (0xbf000000 >> 3) + KASAN_SHADOW_OFFSET KASAN_SHADOW_OFFSET = 0xb6e00000 - (0xbf000000 >> 3) KASAN_SHADOW_OFFSET = 0xb6e00000 - 0x17e00000 KASAN_SHADOW_OFFSET = 0x9f000000 - 0x8f000000 if we have 3G userspace / 1G kernelspace with full 1 GB low memory (VMSPLIT_3G_OPT): - The kernel address space is 1GB (0x40000000) - PAGE_OFFSET is set to 0xb0000000 so the kernel static image uses 0xb0000000 .. 0xffffffff. - On top of that we have the MODULES_VADDR which under the worst case (using ARM instructions) is PAGE_OFFSET - 16M (0x01000000) = 0xaf000000 so the modules use addresses 0xaf000000 .. 0xaffffff - So the addresses 0xaf000000 .. 0xffffffff need to be covered with shadow memory. That is 0x51000000 bytes of memory. - 1/8 of that is needed for its shadow memory, so 0x0a200000 bytes of shadow memory is needed. We "steal" that from the remaining lowmem. - The KASAN_SHADOW_START becomes 0xa4e00000, to KASAN_SHADOW_END at 0xaeffffff. - Now we can calculate the KASAN_SHADOW_OFFSET for any kernel address as 0xaf000000 needs to map to the first byte of shadow memory and 0xffffffff needs to map to the last byte of shadow memory. Since: SHADOW_ADDR = (address >> 3) + KASAN_SHADOW_OFFSET 0xa4e00000 = (0xaf000000 >> 3) + KASAN_SHADOW_OFFSET KASAN_SHADOW_OFFSET = 0xa4e00000 - (0xaf000000 >> 3) KASAN_SHADOW_OFFSET = 0xa4e00000 - 0x15e00000 KASAN_SHADOW_OFFSET = 0x8f000000 - The default value of 0xffffffff for KASAN_SHADOW_OFFSET is an error value. We should always match one of the above shadow offsets. When we do this, TASK_SIZE will sometimes get a bit odd values that will not fit into immediate mov assembly instructions. To account for this, we need to rewrite some assembly using TASK_SIZE like this: - mov r1, #TASK_SIZE + ldr r1, =TASK_SIZE or - cmp r4, #TASK_SIZE + ldr r0, =TASK_SIZE + cmp r4, r0 this is done to avoid the immediate #TASK_SIZE that need to fit into a limited number of bits. Cc: Andrey Ryabinin <aryabinin@virtuozzo.com> Cc: Alexander Potapenko <glider@google.com> Cc: Dmitry Vyukov <dvyukov@google.com> Cc: kasan-dev@googlegroups.com Cc: Mike Rapoport <rppt@linux.ibm.com> Reviewed-by: Ard Biesheuvel <ardb@kernel.org> Tested-by: Ard Biesheuvel <ardb@kernel.org> # QEMU/KVM/mach-virt/LPAE/8G Tested-by: Florian Fainelli <f.fainelli@gmail.com> # Brahma SoCs Tested-by: Ahmad Fatoum <a.fatoum@pengutronix.de> # i.MX6Q Reported-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Abbott Liu <liuwenliang@huawei.com> Signed-off-by: Florian Fainelli <f.fainelli@gmail.com> Signed-off-by: Linus Walleij <linus.walleij@linaro.org> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
2020-10-27ARM: 9012/1: move device tree mapping out of linear regionArd Biesheuvel1-0/+5
On ARM, setting up the linear region is tricky, given the constraints around placement and alignment of the memblocks, and how the kernel itself as well as the DT are placed in physical memory. Let's simplify matters a bit, by moving the device tree mapping to the top of the address space, right between the end of the vmalloc region and the start of the the fixmap region, and create a read-only mapping for it that is independent of the size of the linear region, and how it is organized. Since this region was formerly used as a guard region, which will now be populated fully on LPAE builds by this read-only mapping (which will still be able to function as a guard region for stray writes), bump the start of the [underutilized] fixmap region by 512 KB as well, to ensure that there is always a proper guard region here. Doing so still leaves ample room for the fixmap space, even with NR_CPUS set to its maximum value of 32. Tested-by: Linus Walleij <linus.walleij@linaro.org> Reviewed-by: Linus Walleij <linus.walleij@linaro.org> Reviewed-by: Nicolas Pitre <nico@fluxnic.net> Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
2019-06-19treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500Thomas Gleixner1-4/+1
Based on 2 normalized pattern(s): this program is free software you can redistribute it and or modify it under the terms of the gnu general public license version 2 as published by the free software foundation this program is free software you can redistribute it and or modify it under the terms of the gnu general public license version 2 as published by the free software foundation # extracted by the scancode license scanner the SPDX license identifier GPL-2.0-only has been chosen to replace the boilerplate/reference in 4122 file(s). Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Enrico Weigelt <info@metux.net> Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org> Reviewed-by: Allison Randal <allison@lohutok.net> Cc: linux-spdx@vger.kernel.org Link: https://lkml.kernel.org/r/20190604081206.933168790@linutronix.de Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2018-04-11linux/const.h: move UL() macro to include/linux/const.hMasahiro Yamada1-6/+0
ARM, ARM64 and UniCore32 duplicate the definition of UL(): #define UL(x) _AC(x, UL) This is not actually arch-specific, so it will be useful to move it to a common header. Currently, we only have the uapi variant for linux/const.h, so I am creating include/linux/const.h. I also added _UL(), _ULL() and ULL() because _AC() is mostly used in the form either _AC(..., UL) or _AC(..., ULL). I expect they will be replaced in follow-up cleanups. The underscore-prefixed ones should be used for exported headers. Link: http://lkml.kernel.org/r/1519301715-31798-4-git-send-email-yamada.masahiro@socionext.com Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Guan Xuetao <gxt@mprc.pku.edu.cn> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Russell King <rmk+kernel@armlinux.org.uk> Cc: David Howells <dhowells@redhat.com> Cc: Geert Uytterhoeven <geert@linux-m68k.org> Cc: Will Deacon <will.deacon@arm.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2018-01-21ARM: 8739/1: NOMMU: Setup VBAR/Hivecs for secondaries coresVladimir Murzin1-0/+1
With switch to dynamic exception base address setting, VBAR/Hivecs set only for boot CPU, but secondaries stay unaware of that. That might lead to weird effects when trying up to bring up secondaries. Fixes: ad475117d201 ("ARM: 8649/2: nommu: remove Hivecs configuration is asm") Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com> Acked-by: afzal mohammed <afzal.mohd.ma@gmail.com> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
2017-02-28ARM: 8648/2: nommu: display vectors baseAfzal Mohammed1-2/+7
VECTORS_BASE displays the exception base address. Now on no-MMU as the exception base address is dynamically estimated, define VECTORS_BASE to the variable holding it. As it is the case, limit VECTORS_BASE constant definition to MMU. Suggested-by: Russell King <rmk+kernel@armlinux.org.uk> Signed-off-by: afzal mohammed <afzal.mohd.ma@gmail.com> Tested-by: Vladimir Murzin <vladimir.murzin@arm.com> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
2017-02-28ARM: 8646/1: mmu: decouple VECTORS_BASE from KconfigAfzal Mohammed1-0/+2
For MMU configurations, VECTORS_BASE is always 0xffff0000, a macro definition will suffice. For no-MMU, exception base address is dynamically determined in subsequent patches. To preserve bisectability, now make the macro applicable for no-MMU scenario too. Thanks to 0-DAY kernel test infrastructure that found the bisectability issue. This macro will be restricted to MMU case upon dynamically determining exception base address for no-MMU. Once exception address is handled dynamically for no-MMU, VECTORS_BASE can be removed from Kconfig. Signed-off-by: afzal mohammed <afzal.mohd.ma@gmail.com> Tested-by: Vladimir Murzin <vladimir.murzin@arm.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2017-02-28ARM: 8640/1: Add support for CONFIG_DEBUG_VIRTUALFlorian Fainelli1-2/+13
x86 has an option: CONFIG_DEBUG_VIRTUAL to do additional checks on virt_to_phys calls. The goal is to catch users who are calling virt_to_phys on non-linear addresses immediately. This includes caller using __virt_to_phys() on image addresses instead of __pa_symbol(). This is a generally useful debug feature to spot bad code (particulary in drivers). Acked-by: Russell King <rmk+kernel@armlinux.org.uk> Acked-by: Laura Abbott <labbott@redhat.com> Signed-off-by: Florian Fainelli <f.fainelli@gmail.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2017-02-28ARM: 8639/1: Define KERNEL_START and KERNEL_ENDFlorian Fainelli1-0/+7
In preparation for adding CONFIG_DEBUG_VIRTUAL support, define a set of common constants: KERNEL_START and KERNEL_END which abstract CONFIG_XIP_KERNEL vs. !CONFIG_XIP_KERNEL. Update the code where relevant. Acked-by: Russell King <rmk+kernel@armlinux.org.uk> Signed-off-by: Florian Fainelli <f.fainelli@gmail.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2016-08-12ARM: 8589/1: asm/memory.h: remove dead definitionsNicolas Pitre1-12/+3
The last ad-hoc __phys_to_virt definition was removed in commit fd0053c9 ("ARM: realview: remove sparsemem hack"). Therefore we can remove the unneeded definitions and unduplicate the virt_to_pfn macro from asm/memory.h. Signed-off-by: Nicolas Pitre <nico@linaro.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2016-05-03ARM: provide arm_has_idmap_alias() helperRussell King1-0/+5
Provide a helper to indicate whether we need to perform special handling for boot identity mapping aliases or not. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk> Reviewed-by: Pratyush Anand <panand@redhat.com>
2016-05-03ARM: provide improved virt_to_idmap() functionalityRussell King1-7/+26
For kexec, we need more functionality from the IDMAP system. We need to be able to convert physical addresses to their identity mappped versions as well as virtual addresses. Convert the existing arch_virt_to_idmap() to deal with physical addresses instead. Acked-by: Santosh Shilimkar <ssantosh@kernel.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2016-02-22ARM: 8530/1: remove VIRT_TO_BUSArnd Bergmann1-14/+0
All drivers that are relevant for rpc or footbridge have stopped using virt_to_bus a while ago, so we can remove it and avoid some harmless randconfig warnings for drivers that we do not care about: drivers/atm/zatm.c: In function 'poll_rx': drivers/atm/zatm.c:401:18: warning: 'bus_to_virt' is deprecated [-Wdeprecated-declarations] skb = ((struct rx_buffer_head *) bus_to_virt(here[2]))->skb; FWIW, the remaining drivers using this are: ATM: firestream, zatm, ambassador, horizon ISDN: hisax/netjet V4L: STA2X11, zoran Net: Appletalk LTPC, Tulip DE4x5, Toshiba IrDA WAN: comtrol sv11, cosa, lanmedia, sealevel SCSI: DPT_I2O, buslogic VME: CA91C142 My best guess is that all of the above are so hopelessly obsolete that we are best off removing all of them form the kernel, but that can be done another time. Signed-off-by: Arnd Bergmann <arnd@arndb.de> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2016-02-17ARM: make the physical-relative calculation more obviousRussell King1-4/+11
The physical-relative calculation between the XIP text and data sections introduced by the previous patch was far from obvious. Let's simplify it by turning it into a macro which takes the two (virtual) addresses. This allows us to arrange the calculation in a more obvious manner - we can make it two sub-expressions which calculate the physical address for each symbol, and then takes the difference of those physical addresses. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2016-02-16ARM: 8512/1: proc-v7.S: Adjust stack address when XIP_KERNELNicolas Pitre1-0/+8
When XIP_KERNEL is enabled, the virt to phys address translation for RAM is not the same as the virt to phys address translation for .text. The only way to know where physical RAM is located is to use PLAT_PHYS_OFFSET. The MACRO will be useful for other places where there is a similar problem. Signed-off-by: Nicolas Pitre <nico@linaro.org> Signed-off-by: Chris Brandt <chris.brandt@renesas.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2016-02-08ARM: make virt_to_idmap() return unsigned longRussell King1-3/+3
Make virt_to_idmap() return an unsigned long rather than phys_addr_t. Returning phys_addr_t here makes no sense, because the definition of virt_to_idmap() is that it shall return a physical address which maps identically with the virtual address. Since virtual addresses are limited to 32-bit, identity mapped physical addresses are as well. Almost all users already had an implicit narrowing cast to unsigned long so let's make this official and part of this interface. Tested-by: Grygorii Strashko <grygorii.strashko@ti.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2015-09-22ARM: 8430/1: use default ioremap alignment for SMP or LPAESergey Dyasly1-0/+2
16MB alignment for ioremap mappings was added by commit a069c896d0d6 ("[ARM] 3705/1: add supersection support to ioremap()") in order to support supersection mappings. But __arm_ioremap_pfn_caller uses section and supersection mappings only in !SMP && !LPAE case. There is no need for such big alignment if either SMP or LPAE is enabled. After this change, ioremap will use default maximum alignment of 128 pages. Link: https://lkml.kernel.org/g/1419328813-2211-1-git-send-email-d.safonov@partner.samsung.com Cc: Guan Xuetao <gxt@mprc.pku.edu.cn> Cc: Nicolas Pitre <nicolas.pitre@linaro.org> Cc: James Bottomley <JBottomley@parallels.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Arnd Bergmann <arnd.bergmann@linaro.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Dmitry Safonov <d.safonov@partner.samsung.com> Signed-off-by: Sergey Dyasly <s.dyasly@samsung.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2015-08-27mm: move __phys_to_pfn and __pfn_to_phys to asm/generic/memory_model.hChristoph Hellwig1-6/+0
Three architectures already define these, and we'll need them genericly soon. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2015-07-17ARM: fix __virt_to_idmap build error on !MMURussell King1-1/+1
Fengguang Wu reports that building ARM with !MMU results in the following build error: arch/arm/kernel/built-in.o: In function `__soft_restart': >> :(.text+0x1624): undefined reference to `arch_virt_to_idmap' Fix this by adding an appropriate IS_ENABLED(CONFIG_MMU) into the __virt_to_idmap() inline function. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2015-07-07Merge branches 'fixes' and 'ioremap' into for-linusRussell King1-16/+0
2015-06-29ARM: 8396/1: use phys_addr_t in pfn_to_kaddr()Vitaly Andrianov1-1/+1
This patch fixes pfn_to_kaddr() to use phys_addr_t. Without this, this macro is broken on LPAE systems. For physical addresses above first 4GB result of shifting pfn with PAGE_SHIFT may be truncated. Signed-off-by: Vitaly Andrianov <vitalya@ti.com> Acked-by: Nicolas Pitre <nico@linaro.org> Acked-by: Santosh Shilimkar <ssantosh@kernel.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2015-06-01ARM: redo TTBR setup code for LPAERussell King1-16/+0
Re-engineer the LPAE TTBR setup code. Rather than passing some shifted address in order to fit in a CPU register, pass either a full physical address (in the case of r4, r5 for TTBR0) or a PFN (for TTBR1). This removes the ARCH_PGD_SHIFT hack, and the last dangerous user of cpu_set_ttbr() in the secondary CPU startup code path (which was there to re-set TTBR1 to the appropriate high physical address space on Keystone2.) Tested-by: Murali Karicheri <m-karicheri2@ti.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2014-11-10ARM: Use include/asm-generic/io.hThierry Reding1-0/+4
Include the generic I/O header file so that duplicate implementations can be removed. This will also help to establish consistency across more architectures regarding which accessors they support. Signed-off-by: Thierry Reding <treding@nvidia.com>
2014-08-05Merge branch 'devel-stable' into for-nextRussell King1-3/+1
Conflicts: arch/arm/kernel/perf_event_cpu.c
2014-07-29ARM: 8113/1: remove remaining definitions of PLAT_PHYS_OFFSET from <mach/memory.h>Uwe Kleine-König1-4/+2
The platforms selecting NEED_MACH_MEMORY_H defined the start address of their physical memory in the respective <mach/memory.h>. With ARM_PATCH_PHYS_VIRT=y (which is quite common today) this is useless though because the definition isn't used but determined dynamically. So remove the definitions from all <mach/memory.h> and provide the Kconfig symbol PHYS_OFFSET with the respective defaults in case ARM_PATCH_PHYS_VIRT isn't enabled. This allows to drop the dependency of PHYS_OFFSET on !NEED_MACH_MEMORY_H which prevents compiling an integrator nommu-kernel. (CONFIG_PAGE_OFFSET which has "default PHYS_OFFSET if !MMU" expanded to "0x" because CONFIG_PHYS_OFFSET doesn't exist as INTEGRATOR selects NEED_MACH_MEMORY_H.) Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2014-07-01ARM: nommu: don't limit TASK_SIZEUwe Kleine-König1-3/+1
With TASK_SIZE set to the maximal RAM address booting in some XIP configurations fails (e.g. on efm32 DK3750). The problem is that strncpy_from_user et al. check for the address not being above TASK_SIZE (since 8c56cc8be5b3 (ARM: 7449/1: use generic strnlen_user and strncpy_from_user functions)) and this makes booting fail if the XIP flash is above the RAM address space. This change is in line with blackfin, frv and m68k which also use 0xffffffff for TASK_SIZE with !MMU. Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
2014-04-23ARM: 8023/1: remove remnants of the static DMA mappingNicolas Pitre1-2/+0
It looks like the static mapping area for DMA was replaced by dynamic allocation into the vmalloc area by commit e9da6e9905e6 but the information in Documentation/arm/memory.txt was not removed accordingly. CONSISTENT_END in arch/arm/include/asm/memory.h has no more users and can be removed as well. Signed-off-by: Nicolas Pitre <nico@linaro.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2014-04-04Merge branches 'amba', 'fixes', 'misc', 'mmci', 'unstable/omap-dma' and 'unstable/sa11x0' into for-nextRussell King1-23/+27
2014-04-03ARM: Better virt_to_page() handlingRussell King1-17/+24
virt_to_page() is incredibly inefficient when virt-to-phys patching is enabled. This is because we end up with this calculation: page = &mem_map[asm virt_to_phys(addr) >> 12 - __pv_phys_offset >> 12] in assembly. The asm virt_to_phys() is equivalent this this operation: addr - PAGE_OFFSET + __pv_phys_offset and we can see that because this is assembly, the compiler has no chance to optimise some of that away. This should reduce down to: page = &mem_map[(addr - PAGE_OFFSET) >> 12] for the common cases. Permit the compiler to make this optimisation by giving it more of the information it needs - do this by providing a virt_to_pfn() macro. Another issue which makes this more complex is that __pv_phys_offset is a 64-bit type on all platforms. This is needlessly wasteful - if we store the physical offset as a PFN, we can save a lot of work having to deal with 64-bit values, which sometimes ends up producing incredibly horrid code: a4c: e3009000 movw r9, #0 a4c: R_ARM_MOVW_ABS_NC __pv_phys_offset a50: e3409000 movt r9, #0 ; r9 = &__pv_phys_offset a50: R_ARM_MOVT_ABS __pv_phys_offset a54: e3002000 movw r2, #0 a54: R_ARM_MOVW_ABS_NC __pv_phys_offset a58: e3402000 movt r2, #0 ; r2 = &__pv_phys_offset a58: R_ARM_MOVT_ABS __pv_phys_offset a5c: e5999004 ldr r9, [r9, #4] ; r9 = high word of __pv_phys_offset a60: e3001000 movw r1, #0 a60: R_ARM_MOVW_ABS_NC mem_map a64: e592c000 ldr ip, [r2] ; ip = low word of __pv_phys_offset Reviewed-by: Nicolas Pitre <nico@linaro.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2014-03-07ARM: fix noMMU kallsyms symbol filteringRussell King1-6/+3
With noMMU, CONFIG_PAGE_OFFSET was not being set correctly. As there's no MMU, PAGE_OFFSET should be equal to PHYS_OFFSET in all cases. This commit makes that explicit. Since we do this, we don't need to mess around in asm/memory.h with ifdefs to sort this out, so let's get rid of that, and there's no point offering the "Memory split" option for noMMU as that's meaningless there. Fixes: b9b32bf70f2f ("ARM: use linker magic for vectors and vector stubs") Cc: <stable@vger.kernel.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2013-12-29ARM: 7931/1: Correct virt_addr_validLaura Abbott1-1/+2
The definition of virt_addr_valid is that virt_addr_valid should return true if and only if virt_to_page returns a valid pointer. The current definition of virt_addr_valid only checks against the virtual address range. There's no guarantee that just because a virtual address falls bewteen PAGE_OFFSET and high_memory the associated physical memory has a valid backing struct page. Follow the example of other architectures and convert to pfn_valid to verify that the virtual address is actually valid. The check for an address between PAGE_OFFSET and high_memory is still necessary as vmalloc/highmem addresses are not valid with virt_to_page. Cc: Will Deacon <will.deacon@arm.com> Cc: Nicolas Pitre <nico@linaro.org> Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Laura Abbott <lauraa@codeaurora.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2013-12-13ARM: fix asm/memory.h build errorRussell King1-17/+14
Jason Gunthorpe reports a build failure when ARM_PATCH_PHYS_VIRT is not defined: In file included from arch/arm/include/asm/page.h:163:0, from include/linux/mm_types.h:16, from include/linux/sched.h:24, from arch/arm/kernel/asm-offsets.c:13: arch/arm/include/asm/memory.h: In function '__virt_to_phys': arch/arm/include/asm/memory.h:244:40: error: 'PHYS_OFFSET' undeclared (first use in this function) arch/arm/include/asm/memory.h:244:40: note: each undeclared identifier is reported only once for each function it appears in arch/arm/include/asm/memory.h: In function '__phys_to_virt': arch/arm/include/asm/memory.h:249:13: error: 'PHYS_OFFSET' undeclared (first use in this function) Fixes: ca5a45c06cd4 ("ARM: mm: use phys_addr_t appropriately in p2v and v2p conversions") Tested-By: Jason Gunthorpe <jgunthorpe@obsidianresearch.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2013-11-14ARM: 7882/1: mm: fix __phys_to_virt to work with 64 bit phys_addr_t in BE caseVictor Kamensky1-1/+8
Make sure that inline assembler that expects 'r' operand receives 32 bit value. Before this fix in case of CONFIG_ARCH_PHYS_ADDR_T_64BIT and CONFIG_ARM_PATCH_PHYS_VIRT __phys_to_virt function passed 64 bit value to __pv_stub inline assembler where 'r' operand is expected. Compiler behavior in such case is not well specified. It worked in little endian case, but in big endian case incorrect code was generated, where compiler confused which part of 64 bit value it needed to modify. For example BE snippet looked like this: N:0x80904E08 : MOV r2,#0 N:0x80904E0C : SUB r2,r2,#0x81000000 when LE similar code looked like this N:0x808FCE2C : MOV r2,r0 N:0x808FCE30 : SUB r2,r2,#0xc0, 8 ; #0xc0000000 Note 'r0' register is va that have to be translated into phys To avoid this situation use explicit cast to 'unsigned long', which explicitly discard upper part of phys address and convert value to 32 bit. Also add comment so such cast will not be removed in the future. Signed-off-by: Victor Kamensky <victor.kamensky@linaro.org> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2013-10-29ARM: fix misplaced arch_virt_to_idmap()Russell King1-1/+2
Olof Johansson reported: In file included from arch/arm/include/asm/page.h:163:0, from include/linux/mm_types.h:16, from include/linux/sched.h:24, from arch/arm/kernel/asm-offsets.c:13: arch/arm/include/asm/memory.h: In function '__virt_to_idmap': arch/arm/include/asm/memory.h:300:6: error: 'arch_virt_to_idmap' undeclared (first use in this function) caused by arch_virt_to_idmap being placed inside a different preprocessor conditional to its user. Move it along side its user. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2013-10-10ARM: mm: Correct virt_to_phys patching for 64 bit physical addressesSricharan R1-3/+34
The current phys_to_virt patching mechanism works only for 32 bit physical addresses and this patch extends the idea for 64bit physical addresses. The 64bit v2p patching mechanism patches the higher 8 bits of physical address with a constant using 'mov' instruction and lower 32bits are patched using 'add'. While this is correct, in those platforms where the lowmem addressable physical memory spawns across 4GB boundary, a carry bit can be produced as a result of addition of lower 32bits. This has to be taken in to account and added in to the upper. The patched __pv_offset and va are added in lower 32bits, where __pv_offset can be in two's complement form when PA_START < VA_START and that can result in a false carry bit. e.g 1) PA = 0x80000000; VA = 0xC0000000 __pv_offset = PA - VA = 0xC0000000 (2's complement) 2) PA = 0x2 80000000; VA = 0xC000000 __pv_offset = PA - VA = 0x1 C0000000 So adding __pv_offset + VA should never result in a true overflow for (1). So in order to differentiate between a true carry, a __pv_offset is extended to 64bit and the upper 32bits will have 0xffffffff if __pv_offset is 2's complement. So 'mvn #0' is inserted instead of 'mov' while patching for the same reason. Since mov, add, sub instruction are to patched with different constants inside the same stub, the rotation field of the opcode is using to differentiate between them. So the above examples for v2p translation becomes for VA=0xC0000000, 1) PA[63:32] = 0xffffffff PA[31:0] = VA + 0xC0000000 --> results in a carry PA[63:32] = PA[63:32] + carry PA[63:0] = 0x0 80000000 2) PA[63:32] = 0x1 PA[31:0] = VA + 0xC0000000 --> results in a carry PA[63:32] = PA[63:32] + carry PA[63:0] = 0x2 80000000 The above ideas were suggested by Nicolas Pitre <nico@linaro.org> as part of the review of first and second versions of the subject patch. There is no corresponding change on the phys_to_virt() side, because computations on the upper 32-bits would be discarded anyway. Cc: Russell King <linux@arm.linux.org.uk> Reviewed-by: Nicolas Pitre <nico@linaro.org> Signed-off-by: Sricharan R <r.sricharan@ti.com> Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
2013-10-10ARM: mm: Introduce virt_to_idmap() with an arch hookSantosh Shilimkar1-0/+16
On some PAE systems (e.g. TI Keystone), memory is above the 32-bit addressable limit, and the interconnect provides an aliased view of parts of physical memory in the 32-bit addressable space. This alias is strictly for boot time usage, and is not otherwise usable because of coherency limitations. On such systems, the idmap mechanism needs to take this aliased mapping into account. This patch introduces virt_to_idmap() and a arch function pointer which can be populated by platform which needs it. Also populate necessary idmap spots with now available virt_to_idmap(). Avoided #ifdef approach to be compatible with multi-platform builds. Most architecture won't touch it and in that case virt_to_idmap() fall-back to existing virt_to_phys() macro. Cc: Russell King <linux@arm.linux.org.uk> Acked-by: Nicolas Pitre <nico@linaro.org> Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
2013-10-10ARM: mm: use phys_addr_t appropriately in p2v and v2p conversionsSantosh Shilimkar1-6/+16
Fix remainder types used when converting back and forth between physical and virtual addresses. Cc: Russell King <linux@arm.linux.org.uk> Acked-by: Nicolas Pitre <nico@linaro.org> Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
2013-07-03mm/ARM: fix stale comment about VALID_PAGE()Jiang Liu1-6/+0
VALID_PAGE() has been removed from kernel long time ago, so fix the comment. Signed-off-by: Jiang Liu <jiang.liu@huawei.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Will Deacon <will.deacon@arm.com> Cc: Nicolas Pitre <nico@linaro.org> Cc: Stephen Boyd <sboyd@codeaurora.org> Cc: Giancarlo Asnaghi <giancarlo.asnaghi@st.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-05-30ARM: fix type of PHYS_PFN_OFFSET to unsigned longCyril Chemparathy1-1/+1
On LPAE machines, PHYS_OFFSET evaluates to a phys_addr_t and this type is inherited by the PHYS_PFN_OFFSET definition as well. Consequently, the kernel build emits warnings of the form: init/main.c: In function 'start_kernel': init/main.c:588:7: warning: format '%lx' expects argument of type 'long unsigned int', but argument 2 has type 'phys_addr_t' [-Wformat] This patch fixes this warning by pinning down the PFN type to unsigned long. Signed-off-by: Cyril Chemparathy <cyril@ti.com> Acked-by: Nicolas Pitre <nico@linaro.org> Tested-by: Santosh Shilimkar <santosh.shilimkar@ti.com> Tested-by: Subash Patel <subash.rp@samsung.com> Signed-off-by: Will Deacon <will.deacon@arm.com>