diff options
| author |   Daniel Borkmann <daniel@iogearbox.net> | 2018-10-19 15:51:03 +0200 |
|---|---|---|
| committer |   Alexei Starovoitov <ast@kernel.org> | 2018-10-19 13:43:08 -0700 |
| commit | a64af0ef1c1dbd1e8be65a6ebbf5950305b27e48 (patch) | |
| tree | dce79eaf0705b6bc3eaf0c8592fb36893427246b /tools/lib | |
| parent | tools, perf: add and use optimized ring_buffer_{read_head, write_tail} helpers (diff) | |
| download | linux-dev-a64af0ef1c1dbd1e8be65a6ebbf5950305b27e48.tar.xz linux-dev-a64af0ef1c1dbd1e8be65a6ebbf5950305b27e48.zip | |
bpf, libbpf: use correct barriers in perf ring buffer walk
Given libbpf is a generic library and not restricted to x86-64 only,
the compiler barrier in bpf_perf_event_read_simple() after fetching
the head needs to be replaced with smp_rmb() at minimum. Also, writing
out the tail we should use WRITE_ONCE() to avoid store tearing.
Now that we have the logic in place in ring_buffer_read_head() and
ring_buffer_write_tail() helper also used by perf tool which would
select the correct and best variant for a given architecture (e.g.
x86-64 can avoid CPU barriers entirely), make use of these in order
to fix bpf_perf_event_read_simple().
Fixes: d0cabbb021be ("tools: bpf: move the event reading loop to libbpf")
Fixes: 39111695b1b8 ("samples: bpf: add bpf_perf_event_output example")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Diffstat (limited to 'tools/lib')
| -rw-r--r-- | tools/lib/bpf/libbpf.c | 10 |
1 files changed, 4 insertions, 6 deletions
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index bd71efcc53be..0c21355f04a7 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -27,6 +27,7 @@ #include <linux/list.h> #include <linux/limits.h> #include <linux/perf_event.h> +#include <linux/ring_buffer.h> #include <sys/stat.h> #include <sys/types.h> #include <sys/vfs.h> @@ -2418,13 +2419,12 @@ bpf_perf_event_read_simple(void *mem, unsigned long size, unsigned long page_size, void **buf, size_t *buf_len, bpf_perf_event_print_t fn, void *priv) { - volatile struct perf_event_mmap_page *header = mem; + struct perf_event_mmap_page *header = mem; + __u64 data_head = ring_buffer_read_head(header); __u64 data_tail = header->data_tail; - __u64 data_head = header->data_head; int ret = LIBBPF_PERF_EVENT_ERROR; void *base, *begin, *end; - asm volatile("" ::: "memory"); /* in real code it should be smp_rmb() */ if (data_head == data_tail) return LIBBPF_PERF_EVENT_CONT; @@ -2467,8 +2467,6 @@ bpf_perf_event_read_simple(void *mem, unsigned long size, data_tail += ehdr->size; } - __sync_synchronize(); /* smp_mb() */ - header->data_tail = data_tail; - + ring_buffer_write_tail(header, data_tail); return ret; } |