linux-dev/arch/s390/include/asm/ftrace.h, branch linus/master

s390/ftrace: fix arch_ftrace_get_regs implementation

2022-03-01T19:41:28Z

arch_ftrace_get_regs is supposed to return a struct pt_regs pointer only if the pt_regs structure contains all register contents, which means it must have been populated when created via ftrace_regs_caller. If it was populated via ftrace_caller the contents are not complete (the psw mask part is missing), and therefore a NULL pointer needs be returned. The current code incorrectly always returns a struct pt_regs pointer. Fix this by adding another pt_regs flag which indicates if the contents are complete, and fix arch_ftrace_get_regs accordingly. Fixes: 894979689d3a ("s390/ftrace: provide separate ftrace_caller/ftrace_regs_caller implementations") Reported-by: Christophe Leroy Reported-by: Naveen N. Rao Reviewed-by: Sven Schnelle Acked-by: Ilya Leoshkevich Signed-off-by: Heiko Carstens Signed-off-by: Vasily Gorbik

s390/ftrace: add HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALL support

2021-10-19T13:39:53Z

This is the s390 variant of commit 562955fe6a55 ("ftrace/x86: Add register_ftrace_direct() for custom trampolines"). Acked-by: Ilya Leoshkevich Reviewed-by: Sven Schnelle Signed-off-by: Heiko Carstens Acked-by: Steven Rostedt (VMware) Link: https://lore.kernel.org/r/20211012133802.2460757-2-hca@linux.ibm.com Signed-off-by: Vasily Gorbik

s390/ftrace: provide separate ftrace_caller/ftrace_regs_caller implementations

2021-10-11T18:55:58Z

ftrace_regs_caller is an alias to ftrace_caller - making ftrace_caller quite heavyweight. Split the function and provide an ftrace_caller implementation which comes with fewer instructions. Especially getting rid of 'stosm' on each function entry should help here, e.g. to have less performance impact on live patched functions. Signed-off-by: Heiko Carstens Signed-off-by: Vasily Gorbik

s390/ftrace: add ftrace_instruction_pointer_set() helper function

2021-10-11T18:55:58Z

Add ftrace_instruction_pointer_set() helper function to match x86. See commit 2860cd8a2353 ("livepatch: Use the default ftrace_ops instead of REGS when ARGS is available"). Signed-off-by: Heiko Carstens Signed-off-by: Vasily Gorbik

s390/ftrace: add HAVE_DYNAMIC_FTRACE_WITH_ARGS support

2021-10-11T18:55:58Z

Add HAVE_DYNAMIC_FTRACE_WITH_ARGS support similar to commit 02a474ca266a ("ftrace/x86: Allow for arguments to be passed in to ftrace_regs by default"). s390's ftrace implementation always provides all registers with pt_regs, therefore this is trivial. Signed-off-by: Heiko Carstens Signed-off-by: Vasily Gorbik

s390/ftrace: add FTRACE_GEN_NOP_ASM macro

2021-10-04T07:49:37Z

FTRACE_GEN_NOP_ASM(name) can be used to generate assembly functions with the required information added to allow tracing via kprobes/ftrace. It adds the nop instruction which will be patched by ftrace later. If the compiler supports -mnop-mcount it will also add an entry to the __mcount_loc section. Signed-off-by: Sven Schnelle Acked-by: Heiko Carstens Signed-off-by: Vasily Gorbik

s390/ftrace: implement hotpatching

2021-08-03T12:31:40Z

s390 allows hotpatching the mask of a conditional jump instruction. Make use of this feature in order to avoid the expensive stop_machine() call. The new trampolines are split in 3 stages: - A first stage is a 6-byte relative conditional long branch located at each function's entry point. Its offset always points to the second stage for the corresponding function, and its mask is either all 0s (ftrace off) or all 1s (ftrace on). The code for flipping the mask is borrowed from ftrace_{enable,disable}_ftrace_graph_caller. After flipping, ftrace_arch_code_modify_post_process() syncs with all the other CPUs by sending SIGPs. - Second stages for vmlinux are stored in a separate part of the .text section reserved by the linker script, and in dynamically allocated memory for modules. This prevents the icache pollution. The total size of second stages is about 1.5% of that of the kernel image. Putting second stages in the .bss section is possible and decreases the size of the non-compressed vmlinux, but splits the kernel 1:1 mapping, which is a bad tradeoff. Each second stage contains a call to the third stage, a pointer to the part of the intercepted function right after the first stage, and a pointer to an interceptor function (e.g. ftrace_caller). Second stages are 8-byte aligned for the future direct calls implementation. - There are only two copies of the third stage: in the .text section for vmlinux and in dynamically allocated memory for modules. It can be an expoline, which is relatively large, so inlining it into each second stage is prohibitively expensive. As a result of this organization, phoronix-test-suite with ftrace off does not show any performance degradation. Suggested-by: Sven Schnelle Suggested-by: Vasily Gorbik Co-developed-by: Heiko Carstens Signed-off-by: Ilya Leoshkevich Link: https://lore.kernel.org/r/20210728212546.128248-3-iii@linux.ibm.com Signed-off-by: Heiko Carstens

s390/ftrace: fix ftrace_update_ftrace_func implementation

2021-07-15T10:54:58Z

s390 enforces DYNAMIC_FTRACE if FUNCTION_TRACER is selected. At the same time implementation of ftrace_caller is not compliant with HAVE_DYNAMIC_FTRACE since it doesn't provide implementation of ftrace_update_ftrace_func() and calls ftrace_trace_function() directly. The subtle difference is that during ftrace code patching ftrace replaces function tracer via ftrace_update_ftrace_func() and activates it back afterwards. Unexpected direct calls to ftrace_trace_function() during ftrace code patching leads to nullptr-dereferences when tracing is activated for one of functions which are used during code patching. Those function currently are: copy_from_kernel_nofault() copy_from_kernel_nofault_allowed() preempt_count_sub() [with debug_defconfig] preempt_count_add() [with debug_defconfig] Corresponding KASAN report: BUG: KASAN: nullptr-dereference in function_trace_call+0x316/0x3b0 Read of size 4 at addr 0000000000001e08 by task migration/0/15 CPU: 0 PID: 15 Comm: migration/0 Tainted: G B 5.13.0-41423-g08316af3644d Hardware name: IBM 3906 M04 704 (LPAR) Stopper: multi_cpu_stop+0x0/0x3e0 <- stop_machine_cpuslocked+0x1e4/0x218 Call Trace: [<0000000001f77caa>] show_stack+0x16a/0x1d0 [<0000000001f8de42>] dump_stack+0x15a/0x1b0 [<0000000001f81d56>] print_address_description.constprop.0+0x66/0x2e0 [<000000000082b0ca>] kasan_report+0x152/0x1c0 [<00000000004cfd8e>] function_trace_call+0x316/0x3b0 [<0000000001fb7082>] ftrace_caller+0x7a/0x7e [<00000000006bb3e6>] copy_from_kernel_nofault_allowed+0x6/0x10 [<00000000006bb42e>] copy_from_kernel_nofault+0x3e/0xd0 [<000000000014605c>] ftrace_make_call+0xb4/0x1f8 [<000000000047a1b4>] ftrace_replace_code+0x134/0x1d8 [<000000000047a6e0>] ftrace_modify_all_code+0x120/0x1d0 [<000000000047a7ec>] __ftrace_modify_code+0x5c/0x78 [<000000000042395c>] multi_cpu_stop+0x224/0x3e0 [<0000000000423212>] cpu_stopper_thread+0x33a/0x5a0 [<0000000000243ff2>] smpboot_thread_fn+0x302/0x708 [<00000000002329ea>] kthread+0x342/0x408 [<00000000001066b2>] __ret_from_fork+0x92/0xf0 [<0000000001fb57fa>] ret_from_fork+0xa/0x30 The buggy address belongs to the page: page:(____ptrval____) refcount:1 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x1 flags: 0x1ffff00000001000(reserved|node=0|zone=0|lastcpupid=0x1ffff) raw: 1ffff00000001000 0000040000000048 0000040000000048 0000000000000000 raw: 0000000000000000 0000000000000000 ffffffff00000001 0000000000000000 page dumped because: kasan: bad access detected Memory state around the buggy address: 0000000000001d00: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 0000000000001d80: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 >0000000000001e00: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 ^ 0000000000001e80: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 0000000000001f00: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 ================================================================== To fix that introduce ftrace_func callback to be called from ftrace_caller and update it in ftrace_update_ftrace_func(). Fixes: 4cc9bed034d1 ("[S390] cleanup ftrace backend functions") Cc: stable@vger.kernel.org Reviewed-by: Heiko Carstens Signed-off-by: Vasily Gorbik Signed-off-by: Heiko Carstens

s390/ftrace: assume -mhotpatch or -mrecord-mcount always available

2020-11-20T18:19:11Z

Currently the kernel minimal compiler requirement is gcc 4.9 or clang 10.0.1. * gcc -mhotpatch option is supported since 4.8. * A combination of -pg -mrecord-mcount -mnop-mcount -mfentry flags is supported since gcc 9 and since clang 10. Drop support for old -pg function prologues. Which leaves binary compatible -mhotpatch / -mnop-mcount -mfentry prologues in a form: brcl 0,0 Which are also do not require initial nop optimization / conversion and presence of _mcount symbol. Signed-off-by: Vasily Gorbik Reviewed-by: Heiko Carstens Signed-off-by: Heiko Carstens

s390/ftrace: use HAVE_FUNCTION_GRAPH_RET_ADDR_PTR

2019-05-02T11:54:11Z

Make the call chain more reliable by tagging the ftrace stack entries with the stack pointer that is associated with the return address. Signed-off-by: Martin Schwidefsky