diff options
| author |   Pavel Begunkov <asml.silence@gmail.com> | 2021-09-02 00:38:23 +0100 |
|---|---|---|
| committer |   Jens Axboe <axboe@kernel.dk> | 2021-09-03 06:16:15 -0600 |
| commit | 8d4ad41e3e8e4b907f088f25aee4a92f3f864027 (patch) | |
| tree | f6ea340ceae16ae71d53b281b9966f08b8561518 /fs | |
| parent | io_uring: don't disable kiocb_done() CQE batching (diff) | |
| download | wireguard-linux-8d4ad41e3e8e4b907f088f25aee4a92f3f864027.tar.xz wireguard-linux-8d4ad41e3e8e4b907f088f25aee4a92f3f864027.zip | |
io_uring: prolong tctx_task_work() with flushing
io_submit_flush_completions() may enqueue linked requests for task_work
execution, so don't leave tctx_task_work() right after the tw list is
exhausted, but try to flush and then retry.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/0755d4c2c36301447c63bdd4146c10477cea4249.1630539342.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Diffstat (limited to 'fs')
| -rw-r--r-- | fs/io_uring.c | 3 |
1 files changed, 3 insertions, 0 deletions
diff --git a/fs/io_uring.c b/fs/io_uring.c index 9f3f8a802abd..3a7145a38653 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -2102,6 +2102,9 @@ static void tctx_task_work(struct callback_head *cb) while (1) { struct io_wq_work_node *node; + if (!tctx->task_list.first && locked && ctx->submit_state.compl_nr) + io_submit_flush_completions(ctx); + spin_lock_irq(&tctx->task_lock); node = tctx->task_list.first; INIT_WQ_LIST(&tctx->task_list); |