diff options
| author |   Eric Dumazet <edumazet@google.com> | 2016-12-03 11:14:54 -0800 |
|---|---|---|
| committer |   David S. Miller <davem@davemloft.net> | 2016-12-05 13:32:23 -0500 |
| commit | 75eefc6c59fd2c5f1ab95a3a113c217237d12a31 (patch) | |
| tree | f5559676f497c4e30b71f58e9df43a944da88641 /net/ipv4 | |
| parent | tcp: tsq: avoid one atomic in tcp_wfree() (diff) | |
| download | linux-dev-75eefc6c59fd2c5f1ab95a3a113c217237d12a31.tar.xz linux-dev-75eefc6c59fd2c5f1ab95a3a113c217237d12a31.zip | |
tcp: tsq: add a shortcut in tcp_small_queue_check()
Always allow the two first skbs in write queue to be sent,
regardless of sk_wmem_alloc/sk_pacing_rate values.
This helps a lot in situations where TX completions are delayed either
because of driver latencies or softirq latencies.
Test is done with no cache line misses.
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Diffstat (limited to 'net/ipv4')
| -rw-r--r-- | net/ipv4/tcp_output.c | 9 |
1 files changed, 9 insertions, 0 deletions
diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c index 0db63efe5b8b..d5c46749adab 100644 --- a/net/ipv4/tcp_output.c +++ b/net/ipv4/tcp_output.c @@ -2091,6 +2091,15 @@ static bool tcp_small_queue_check(struct sock *sk, const struct sk_buff *skb, limit <<= factor; if (atomic_read(&sk->sk_wmem_alloc) > limit) { + /* Always send the 1st or 2nd skb in write queue. + * No need to wait for TX completion to call us back, + * after softirq/tasklet schedule. + * This helps when TX completions are delayed too much. + */ + if (skb == sk->sk_write_queue.next || + skb->prev == sk->sk_write_queue.next) + return false; + set_bit(TSQ_THROTTLED, &tcp_sk(sk)->tsq_flags); /* It is possible TX completion already happened * before we set TSQ_THROTTLED, so we must |