| Commit message (Collapse) | Author | Files | Lines |
|
This should fix a performance problem when two (or more) producers run
on the same CPU.
[ TODO: more text ]
previously:
Thread A Thread B
producer_head=1
write item
<preemptive context switch>
producer_head=2
write item
<stalled because producer_tail
is still 0>
<preemptive context switch>
producer_tail=1
producer_head=3
write item
<stalled because producer_tail is still 1>
...
with this patch:
Thread A Thread B
producer_head=1
write item
<preemptive context switch>
producer_head=2
write item
producer_head=3
write item
producer_head=4
write item
producer_head=5
write item
<preemptive context switch>
producer_head=6
write item
producer_head=7
write item
...
|
|
|
|
|
|
Signed-off-by: Jonathan Neuschäfer <j.neuschaefer@gmx.net>
|
|
The main function, mpmc_ring_selftest, is already marked with __init,
but the callback functions and the ring pointer were not.
|
|
mpmc_ptr_ring_produce takes a void*.
This fixes the following warning:
net/wireguard/selftest/mpmc_ring.h: In function ‘producer_function’:
net/wireguard/selftest/mpmc_ring.h:43:38: warning: passing argument 2 of ‘mpmc_ptr_ring_produce’ disc]
while (mpmc_ptr_ring_produce(ring, (const void *) count))
^
|
|
mpmc_ptr_ring.h should include the headers that it needs explicitly.
This commit adds the following:
- <asm/barrier.h>: smp_rmb, smp_wmb, smp_mb__before_atomic
- <linux/atomic.h>: atomic_t, atomic_read, atomic_set, atomic_cmpxchg, atomic_inc
- <linux/cache.h>: ____cacheline_aligned_in_smp
- <linux/compiler.h>: READ_ONCE, WRITE_ONCE
- <linux/errno.h>: ENOMEM
- <linux/log2.h>: is_power_of_2
- <linux/processor.h>: cpu_relax
- <linux/slab.h>: kcalloc, kfree
- <linux/stddef.h>: NULL
I'm not sure if all of them are really needed because I wasn't about to
provoke a compile error due to the missing includes.
Signed-off-by: Jonathan Neuschäfer <j.neuschaefer@gmx.net>
|
|
|
|
|
|
|
|
For producers/consumers we use atomic_t which is really int, so size_t
could either be too large, wasting memory, or too small (unlikely).
For size, we also want to be using unsigned int, since the mask that we
derive from it is ANDed with producer/consumer.
|
|
|
|
From https://www.kernel.org/doc/Documentation/memory-barriers.txt:
> SMP memory barriers are reduced to compiler barriers on uniprocessor
> compiled systems because it is assumed that a CPU will appear to be
> self-consistent, and will order overlapping accesses correctly with
> respect to itself.
Since we only order CPU memory accesses with the memory barriers in
mpmc_ptr_ring.h, smp_[rw]mb() should be sufficient.
Signed-off-by: Jonathan Neuschäfer <j.neuschaefer@gmx.net>
|
|
I'm not completely sure about this.
It also doesn't fix all the errors: sometimes the test suite reports
that it fails to send packets.
|
|
They are updated together, so it doesn't make much sense to keep them
separate in the cache.
|
|
The index range doesn't get large enough to warrant 64-bit indices.
|
|
|
|
|
|
|
|
TODO: actually use the right memory barriers in the right places
TODO: eliminate false sharing between mpmc_ptr_ring members
TODO: reconsider atomic_long_t vs. atomic_t, vs. the type of size in _init()
TODO: sprinkle likely/unlikely on some branches
FIXME: it still crashes
|
|
|
|
|
|
`ndc users add` eventually invokes SOCK_DESTROY on user sockets, causing
them to reconnect. By delaying this until after routes are set, we
ensure that the sockets reconnect using the tunnel, rather than the old
route.
|
|
|
|
|
|
|
|
The new flow offloading feature at the moment does not set the dst. We
have a patch pending to fix this upstream, but in the meantime, work
around it here.
|
|
This reduces memory access and the total opaque size.
Signed-off-by: René van Dorst <opensource@vdorst.com>
|
|
|
|
|
|
|
|
This works around a race condition in macOS's network daemons, while
also adding one in the form of possibly calling kill -ALRM on a stale
PID; unfortunately bash can't wait from a trap.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Our previous heuristic wasn't good enough, since CopperheadOS backported
CANARY_MASK without backporting get_random_u32, as Qualcomm did, so now
we just entirely rename all invocations of the function.
|
|
|
|
|
|
These OSes have an unpriv'd ifconfig, so this isn't an even larger info
leak.
|
|
|
|
|
|
This lets crippled OSes sed out our -v more easily.
|