linux-dev/arch/powerpc/kernel/tm.S, branch linus/master

powerpc/tm: Fix more userspace r13 corruption

2022-03-16T00:59:24Z

Commit cf13435b730a ("powerpc/tm: Fix userspace r13 corruption") fixes a problem in treclaim where a SLB miss can occur on the thread_struct->ckpt_regs while SCRATCH0 is live with the saved user r13 value, clobbering it with the kernel r13 and ultimately resulting in kernel r13 being stored in ckpt_regs. There is an equivalent problem in trechkpt where the user r13 value is loaded into r13 from chkpt_regs to be recheckpointed, but a SLB miss could occur on ckpt_regs accesses after that, which will result in r13 being clobbered with a kernel value and that will get recheckpointed and then restored to user registers. The same memory page is accessed right before this critical window where a SLB miss could cause corruption, so hitting the bug requires the SLB entry be removed within a small window of instructions, which is possible if a SLB related MCE hits there. PAPR also permits the hypervisor to discard this SLB entry (because slb_shadow->persistent is only set to SLB_NUM_BOLTED) although it's not known whether any implementations would do this (KVM does not). So this is an extremely unlikely bug, only found by inspection. Fix this by also storing user r13 in a temporary location on the kernel stack and don't change the r13 register from kernel r13 until the RI=0 critical section that does not fault. The SCRATCH0 change is not strictly part of the fix, it's only used in the RI=0 section so it does not have the same problem as the previous SCRATCH0 bug. Fixes: 98ae22e15b43 ("powerpc: Add helper functions for transactional memory context switching") Cc: stable@vger.kernel.org # v3.9+ Signed-off-by: Nicholas Piggin Acked-by: Michael Neuling Signed-off-by: Michael Ellerman Link: https://lore.kernel.org/r/20220311024733.48926-1-npiggin@gmail.com

powerpc: flexible GPR range save/restore macros

2021-11-29T12:15:20Z

Introduce macros that operate on a (start, end) range of GPRs, which reduces lines of code and need to do mental arithmetic while reading the code. Signed-off-by: Nicholas Piggin Reviewed-by: Segher Boessenkool Reviewed-by: Christophe Leroy Signed-off-by: Michael Ellerman Link: https://lore.kernel.org/r/20211022061322.2671178-1-npiggin@gmail.com

powerpc/tm: Save and restore AMR on treclaim and trechkpt

2020-10-06T12:22:25Z

Althought AMR is stashed in the checkpoint area, currently we don't save it to the per thread checkpoint struct after a treclaim and so we don't restore it either from that struct when we trechkpt. As a consequence when the transaction is later rolled back the kernel space AMR value when the trechkpt was done appears in userspace. That commit saves and restores AMR accordingly on treclaim and trechkpt. Since AMR value is also used in kernel space in other functions, it also takes care of stashing kernel live AMR into the stack before treclaim and before trechkpt, restoring it later, just before returning from tm_reclaim and __tm_recheckpoint. Is also fixes two nonrelated comments about CR and MSR. Signed-off-by: Gustavo Romero Tested-by: Aneesh Kumar K.V Signed-off-by: Michael Ellerman Link: https://lore.kernel.org/r/20200919150025.9609-1-gromero@linux.ibm.com

powerpc/tm: update comment about interrupt re-entrancy

2019-07-02T11:39:49Z

Since the system reset interrupt began to use its own stack, and machine check interrupts have done so for some time, r1 can be changed without clearing MSR[RI], provided no other interrupts (including SLB misses) are taken. MSR[RI] does have to be cleared when using SCRATCH0, however. Signed-off-by: Nicholas Piggin Signed-off-by: Michael Ellerman

Merge branch 'fixes' into next

2018-10-09T05:51:05Z

Merge our fixes branch. It has a few important fixes that are needed for futher testing and also some commits that will conflict with content in next.

powerpc/tm: Reformat comments

2018-10-03T05:40:07Z

The comments in this file don't conform to the coding style so take them to "Comment Formatting Re-Education Camp". Suggested-by: Michael "Camp Drill Sergeant" Ellerman Signed-off-by: Michael Neuling [mpe: Reflow some comments and add full stops, fix spelling of Sergeant.] Signed-off-by: Michael Ellerman

powerpc/tm: Avoid possible userspace r1 corruption on reclaim

2018-09-25T12:51:32Z

Current we store the userspace r1 to PACATMSCRATCH before finally saving it to the thread struct. In theory an exception could be taken here (like a machine check or SLB miss) that could write PACATMSCRATCH and hence corrupt the userspace r1. The SLB fault currently doesn't touch PACATMSCRATCH, but others do. We've never actually seen this happen but it's theoretically possible. Either way, the code is fragile as it is. This patch saves r1 to the kernel stack (which can't fault) before we turn MSR[RI] back on. PACATMSCRATCH is still used but only with MSR[RI] off. We then copy r1 from the kernel stack to the thread struct once we have MSR[RI] back on. Suggested-by: Breno Leitao Signed-off-by: Michael Neuling Signed-off-by: Michael Ellerman

powerpc/tm: Fix userspace r13 corruption

2018-09-25T12:51:08Z

When we treclaim we store the userspace checkpointed r13 to a scratch SPR and then later save the scratch SPR to the user thread struct. Unfortunately, this doesn't work as accessing the user thread struct can take an SLB fault and the SLB fault handler will write the same scratch SPRG that now contains the userspace r13. To fix this, we store r13 to the kernel stack (which can't fault) before we access the user thread struct. Found by running P8 guest + powervm + disable_1tb_segments + TM. Seen as a random userspace segfault with r13 looking like a kernel address. Signed-off-by: Michael Neuling Reviewed-by: Breno Leitao Signed-off-by: Michael Ellerman

powerpc/tm: Fix HTM documentation

2018-09-17T11:17:25Z

This patch simply fix part of the documentation on the HTM code. This fixes reference to old fields that were renamed in commit 000ec280e3dd ("powerpc: tm: Rename transct_(*) to ck(\1)_state") It also documents better the flow after commit eb5c3f1c8647 ("powerpc: Always save/restore checkpointed regs during treclaim/trecheckpoint"), where tm_recheckpoint can recheckpoint what is in ck{fp,vr}_state blindly. Signed-off-by: Breno Leitao Acked-by: Michael Neuling Signed-off-by: Michael Ellerman

powerpc: clean inclusions of asm/feature-fixups.h

2018-07-30T12:48:17Z

files not using feature fixup don't need asm/feature-fixups.h files using feature fixup need asm/feature-fixups.h Signed-off-by: Christophe Leroy Signed-off-by: Michael Ellerman