| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This should fix a performance problem when two (or more) producers run
on the same CPU.
[ TODO: more text ]
previously:
Thread A Thread B
producer_head=1
write item
<preemptive context switch>
producer_head=2
write item
<stalled because producer_tail
is still 0>
<preemptive context switch>
producer_tail=1
producer_head=3
write item
<stalled because producer_tail is still 1>
...
with this patch:
Thread A Thread B
producer_head=1
write item
<preemptive context switch>
producer_head=2
write item
producer_head=3
write item
producer_head=4
write item
producer_head=5
write item
<preemptive context switch>
producer_head=6
write item
producer_head=7
write item
...
|
| |
|
| |
|
|
|
|
| |
Signed-off-by: Jonathan Neuschäfer <j.neuschaefer@gmx.net>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
mpmc_ptr_ring.h should include the headers that it needs explicitly.
This commit adds the following:
- <asm/barrier.h>: smp_rmb, smp_wmb, smp_mb__before_atomic
- <linux/atomic.h>: atomic_t, atomic_read, atomic_set, atomic_cmpxchg, atomic_inc
- <linux/cache.h>: ____cacheline_aligned_in_smp
- <linux/compiler.h>: READ_ONCE, WRITE_ONCE
- <linux/errno.h>: ENOMEM
- <linux/log2.h>: is_power_of_2
- <linux/processor.h>: cpu_relax
- <linux/slab.h>: kcalloc, kfree
- <linux/stddef.h>: NULL
I'm not sure if all of them are really needed because I wasn't about to
provoke a compile error due to the missing includes.
Signed-off-by: Jonathan Neuschäfer <j.neuschaefer@gmx.net>
|
| |
|
| |
|
|
|
|
|
|
|
|
| |
For producers/consumers we use atomic_t which is really int, so size_t
could either be too large, wasting memory, or too small (unlikely).
For size, we also want to be using unsigned int, since the mask that we
derive from it is ANDed with producer/consumer.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
From https://www.kernel.org/doc/Documentation/memory-barriers.txt:
> SMP memory barriers are reduced to compiler barriers on uniprocessor
> compiled systems because it is assumed that a CPU will appear to be
> self-consistent, and will order overlapping accesses correctly with
> respect to itself.
Since we only order CPU memory accesses with the memory barriers in
mpmc_ptr_ring.h, smp_[rw]mb() should be sufficient.
Signed-off-by: Jonathan Neuschäfer <j.neuschaefer@gmx.net>
|
|
|
|
|
|
|
| |
I'm not completely sure about this.
It also doesn't fix all the errors: sometimes the test suite reports
that it fails to send packets.
|
|
|
|
|
| |
They are updated together, so it doesn't make much sense to keep them
separate in the cache.
|
|
|
|
| |
The index range doesn't get large enough to warrant 64-bit indices.
|
| |
|
| |
|
| |
|
|
TODO: actually use the right memory barriers in the right places
TODO: eliminate false sharing between mpmc_ptr_ring members
TODO: reconsider atomic_long_t vs. atomic_t, vs. the type of size in _init()
TODO: sprinkle likely/unlikely on some branches
FIXME: it still crashes
|