wireguard-linux/arch/mips/include/asm/cmpxchg.h, branch jd/unified-crypt-queue

Merge tag 'mips_fixes_5.4_3' into mips-next

2019-11-01T21:36:44Z

Pull in mips-fixes primarily to gain build fixes in order to allow better testing of mips-next. A few MIPS fixes: - Fix VDSO time-related function behavior for systems where we need to fall back to syscalls, but were instead returning bogus results. - A fix to TLB exception handlers for Cavium Octeon systems where they would inadvertently clobber the $1/$at register. - A build fix for bcm63xx configurations. - Switch to using my @kernel.org email address. Signed-off-by: Paul Burton

MIPS: include: Mark xchg as always_inline

2019-10-09T19:45:31Z

Commit ac7c3e4ff401 ("compiler: enable CONFIG_OPTIMIZE_INLINING forcibly") allows compiler to uninline functions marked as 'inline'. In cace of __xchg this would cause to reference function __xchg_called_with_bad_pointer, which is an error case for catching bugs and will not happen for correct code, if __xchg is inlined. Signed-off-by: Thomas Bogendoerfer Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Paul Burton Cc: Ralf Baechle Cc: James Hogan Cc: linux-mips@vger.kernel.org Cc: linux-kernel@vger.kernel.org

MIPS: cmpxchg: Omit redundant barriers for Loongson3

2019-10-07T16:43:01Z

When building a kernel configured to support Loongson3 LL/SC workarounds (ie. CONFIG_CPU_LOONGSON3_WORKAROUNDS=y) the inline assembly in __xchg_asm() & __cmpxchg_asm() already emits completion barriers, and as such we don't need to emit extra barriers from the xchg() or cmpxchg() macros. Add compile-time constant checks causing us to omit the redundant memory barriers. Signed-off-by: Paul Burton Cc: linux-mips@vger.kernel.org Cc: Huacai Chen Cc: Jiaxun Yang Cc: linux-kernel@vger.kernel.org

MIPS: cmpxchg: Emit Loongson3 sync workarounds within asm

2019-10-07T16:43:00Z

Generate the sync instructions required to workaround Loongson3 LL/SC errata within inline asm blocks, which feels a little safer than doing it from C where strictly speaking the compiler would be well within its rights to insert a memory access between the separate asm statements we previously had, containing sync & ll instructions respectively. Signed-off-by: Paul Burton Cc: linux-mips@vger.kernel.org Cc: Huacai Chen Cc: Jiaxun Yang Cc: linux-kernel@vger.kernel.org

MIPS: Unify sc beqz definition

2019-10-07T16:42:13Z

We currently duplicate the definition of __scbeqz in asm/atomic.h & asm/cmpxchg.h. Move it to asm/llsc.h & rename it to __SC_BEQZ to fit better with the existing __SC macro provided there. We include a tab in the string in order to avoid the need for users to indent code any further to include whitespace of their own after the instruction mnemonic. Signed-off-by: Paul Burton Cc: linux-mips@vger.kernel.org Cc: Huacai Chen Cc: Jiaxun Yang Cc: linux-kernel@vger.kernel.org

MIPS: include: Mark cmpxchg as always_inline

2019-10-07T16:37:01Z

Commit ac7c3e4ff401 ("compiler: enable CONFIG_OPTIMIZE_INLINING forcibly") allows compiler to uninline functions marked as 'inline'. In cace of cmpxchg this would cause to reference function __cmpxchg_called_with_bad_pointer, which is a error case for catching bugs and will not happen for correct code, if __cmpxchg is inlined. Signed-off-by: Thomas Bogendoerfer [paul.burton@mips.com: s/__cmpxchd/__cmpxchg in subject] Signed-off-by: Paul Burton Cc: Ralf Baechle Cc: James Hogan Cc: linux-mips@vger.kernel.org Cc: linux-kernel@vger.kernel.org

mips/atomic: Fix smp_mb__{before,after}_atomic()

2019-08-31T10:06:02Z

Recent probing at the Linux Kernel Memory Model uncovered a 'surprise'. Strongly ordered architectures where the atomic RmW primitive implies full memory ordering and smp_mb__{before,after}_atomic() are a simple barrier() (such as MIPS without WEAK_REORDERING_BEYOND_LLSC) fail for: *x = 1; atomic_inc(u); smp_mb__after_atomic(); r0 = *y; Because, while the atomic_inc() implies memory order, it (surprisingly) does not provide a compiler barrier. This then allows the compiler to re-order like so: atomic_inc(u); *x = 1; smp_mb__after_atomic(); r0 = *y; Which the CPU is then allowed to re-order (under TSO rules) like: atomic_inc(u); r0 = *y; *x = 1; And this very much was not intended. Therefore strengthen the atomic RmW ops to include a compiler barrier. Reported-by: Andrea Parri Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Paul Burton

mips/atomic: Fix loongson_llsc_mb() wreckage

2019-08-31T10:05:17Z

The comment describing the loongson_llsc_mb() reorder case doesn't make any sense what so ever. Instruction re-ordering is not an SMP artifact, but rather a CPU local phenomenon. Clarify the comment by explaining that these issue cause a coherence fail. For the branch speculation case; if futex_atomic_cmpxchg_inatomic() needs one at the bne branch target, then surely the normal __cmpxch_asm() implementation does too. We cannot rely on the barriers from cmpxchg() because cmpxchg_local() is implemented with the same macro, and branch prediction and speculation are, too, CPU local. Fixes: e02e07e3127d ("MIPS: Loongson: Introduce and use loongson_llsc_mb()") Cc: Huacai Chen Cc: Huang Pei Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Paul Burton

mips/atomic: Fix cmpxchg64 barriers

2019-08-31T10:03:46Z

There were no memory barriers on the 32bit implementation of cmpxchg64(). Fix this. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Paul Burton

MIPS: Fix set_pte() for Netlogic XLR using cmpxchg64()

2019-02-06T22:59:24Z

Commit 46011e6ea392 ("MIPS: Make set_pte() SMP safe.") introduced an open-coded version of cmpxchg() within set_pte(), that always operated on a value the size of an unsigned long. That is, it used ll/sc instructions when CONFIG_32BIT=y or lld/scd instructions when CONFIG_64BIT=y. This was broken for configurations in which pte_t is larger than an unsigned long (with the exception of XPA configurations which have a different implementation of set_pte()), because we no longer update the whole PTE. Indeed commit 46011e6ea392 ("MIPS: Make set_pte() SMP safe.") notes: > The case of CONFIG_64BIT_PHYS_ADDR && CONFIG_CPU_MIPS32 is *not* > handled. In practice this affects Netlogic XLR/XLS systems including nlm_xlr_defconfig. Commit 82f4f66ddf11 ("MIPS: Remove open-coded cmpxchg() in set_pte()") then replaced this open-coded version of cmpxchg() with an actual call to cmpxchg(). Unfortunately the configurations mentioned above then fail to build because cmpxchg() can only operate on values 32 bits or smaller in size, resulting in: arch/mips/include/asm/cmpxchg.h:166:11: error: call to '__cmpxchg_called_with_bad_pointer' declared with attribute error: Bad argument size for cmpxchg One option that would fix the build failure & restore the previous behaviour would be to cast the pte pointer to a pointer to unsigned long, so that cmpxchg() would operate on just 32 bits of the PTE as it has been since commit 46011e6ea392 ("MIPS: Make set_pte() SMP safe."). That feels like an ugly hack though, and the behaviour of set_pte() is likely a little broken. Instead we take advantage of the fact that the affected configurations already know at compile time that the CPU will support 64 bits (ie. have hardcoded cpu_has_64bits in cpu-feature-overrides.h) in order to allow cmpxchg64() to be used in these configurations. set_pte() then makes use of cmpxchg64() when necessary. Signed-off-by: Paul Burton Fixes: 46011e6ea392 ("MIPS: Make set_pte() SMP safe.") Fixes: 82f4f66ddf11 ("MIPS: Remove open-coded cmpxchg() in set_pte()")