diff options
| author |   Kenny Yu <kennyyu@fb.com> | 2022-01-24 10:54:00 -0800 |
|---|---|---|
| committer |   Alexei Starovoitov <ast@kernel.org> | 2022-01-24 19:55:40 -0800 |
| commit | b77fb25dcb342788d72ad7533163c34b8b823a1d (patch) | |
| tree | e2da08693c1dd3f620eb6f53d67e15145a8a63de /kernel/bpf/bpf_iter.c | |
| parent | Merge https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next (diff) | |
| download | linux-dev-b77fb25dcb342788d72ad7533163c34b8b823a1d.tar.xz linux-dev-b77fb25dcb342788d72ad7533163c34b8b823a1d.zip | |
bpf: Add support for bpf iterator programs to use sleepable helpers
This patch allows bpf iterator programs to use sleepable helpers by
changing `bpf_iter_run_prog` to use the appropriate synchronization.
With sleepable bpf iterator programs, we can no longer use
`rcu_read_lock()` and must use `rcu_read_lock_trace()` instead
to protect the bpf program.
Signed-off-by: Kenny Yu <kennyyu@fb.com>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/r/20220124185403.468466-2-kennyyu@fb.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Diffstat (limited to 'kernel/bpf/bpf_iter.c')
| -rw-r--r-- | kernel/bpf/bpf_iter.c | 20 |
1 files changed, 15 insertions, 5 deletions
diff --git a/kernel/bpf/bpf_iter.c b/kernel/bpf/bpf_iter.c index b7aef5b3416d..110029ede71e 100644 --- a/kernel/bpf/bpf_iter.c +++ b/kernel/bpf/bpf_iter.c @@ -5,6 +5,7 @@ #include <linux/anon_inodes.h> #include <linux/filter.h> #include <linux/bpf.h> +#include <linux/rcupdate_trace.h> struct bpf_iter_target_info { struct list_head list; @@ -684,11 +685,20 @@ int bpf_iter_run_prog(struct bpf_prog *prog, void *ctx) { int ret; - rcu_read_lock(); - migrate_disable(); - ret = bpf_prog_run(prog, ctx); - migrate_enable(); - rcu_read_unlock(); + if (prog->aux->sleepable) { + rcu_read_lock_trace(); + migrate_disable(); + might_fault(); + ret = bpf_prog_run(prog, ctx); + migrate_enable(); + rcu_read_unlock_trace(); + } else { + rcu_read_lock(); + migrate_disable(); + ret = bpf_prog_run(prog, ctx); + migrate_enable(); + rcu_read_unlock(); + } /* bpf program can only return 0 or 1: * 0 : okay |