diff options
| author |   Eric Dumazet <edumazet@google.com> | 2016-11-16 14:54:50 -0800 |
|---|---|---|
| committer |   David S. Miller <davem@davemloft.net> | 2016-11-16 18:32:02 -0500 |
| commit | 89c4b442b78bdba388337cc746fe63caba85f46c (patch) | |
| tree | 825a8d6e04cdc937950e9d8a79b52eeb249236c3 /include/linux/netpoll.h | |
| parent | cadence: Add LSO support. (diff) | |
| download | wireguard-linux-89c4b442b78bdba388337cc746fe63caba85f46c.tar.xz wireguard-linux-89c4b442b78bdba388337cc746fe63caba85f46c.zip | |
netpoll: more efficient locking
Callers of netpoll_poll_lock() own NAPI_STATE_SCHED
Callers of netpoll_poll_unlock() have BH blocked between
the NAPI_STATE_SCHED being cleared and poll_lock is released.
We can avoid the spinlock which has no contention, and use cmpxchg()
on poll_owner which we need to set anyway.
This removes a possible lockdep violation after the cited commit,
since sk_busy_loop() re-enables BH before calling busy_poll_stop()
Fixes: 217f69743681 ("net: busy-poll: allow preemption in sk_busy_loop()")
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Diffstat (limited to 'include/linux/netpoll.h')
| -rw-r--r-- | include/linux/netpoll.h | 13 |
1 files changed, 7 insertions, 6 deletions
diff --git a/include/linux/netpoll.h b/include/linux/netpoll.h index b25ee9ffdbe6..1828900c9411 100644 --- a/include/linux/netpoll.h +++ b/include/linux/netpoll.h @@ -78,8 +78,11 @@ static inline void *netpoll_poll_lock(struct napi_struct *napi) struct net_device *dev = napi->dev; if (dev && dev->npinfo) { - spin_lock(&napi->poll_lock); - napi->poll_owner = smp_processor_id(); + int owner = smp_processor_id(); + + while (cmpxchg(&napi->poll_owner, -1, owner) != -1) + cpu_relax(); + return napi; } return NULL; @@ -89,10 +92,8 @@ static inline void netpoll_poll_unlock(void *have) { struct napi_struct *napi = have; - if (napi) { - napi->poll_owner = -1; - spin_unlock(&napi->poll_lock); - } + if (napi) + smp_store_release(&napi->poll_owner, -1); } static inline bool netpoll_tx_running(struct net_device *dev) |