diff options
| author |   Jens Axboe <axboe@fb.com> | 2015-04-17 16:23:59 -0600 |
|---|---|---|
| committer | Jens Axboe <axboe@fb.com> | 2015-05-05 13:32:49 -0600 |
| commit | dac56212e8127dbc0bff7be35c508bc280213309 (patch) | |
| tree | a1a066d7f6deb7d66403137b8edafb394ce0ca6a /block | |
| parent | bio: skip atomic inc/dec of ->bi_remaining for non-chains (diff) | |
| download | linux-dev-dac56212e8127dbc0bff7be35c508bc280213309.tar.xz linux-dev-dac56212e8127dbc0bff7be35c508bc280213309.zip | |
bio: skip atomic inc/dec of ->bi_cnt for most use cases
Struct bio has a reference count that controls when it can be freed.
Most uses cases is allocating the bio, which then returns with a
single reference to it, doing IO, and then dropping that single
reference. We can remove this atomic_dec_and_test() in the completion
path, if nobody else is holding a reference to the bio.
If someone does call bio_get() on the bio, then we flag the bio as
now having valid count and that we must properly honor the reference
count when it's being put.
Tested-by: Robert Elliott <elliott@hp.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
Diffstat (limited to 'block')
| -rw-r--r-- | block/bio.c | 18 |
1 files changed, 11 insertions, 7 deletions
diff --git a/block/bio.c b/block/bio.c index 117da319afb6..c2ff8a88aef1 100644 --- a/block/bio.c +++ b/block/bio.c @@ -271,7 +271,7 @@ void bio_init(struct bio *bio) memset(bio, 0, sizeof(*bio)); bio->bi_flags = 1 << BIO_UPTODATE; atomic_set(&bio->__bi_remaining, 1); - atomic_set(&bio->bi_cnt, 1); + atomic_set(&bio->__bi_cnt, 1); } EXPORT_SYMBOL(bio_init); @@ -524,13 +524,17 @@ EXPORT_SYMBOL(zero_fill_bio); **/ void bio_put(struct bio *bio) { - BIO_BUG_ON(!atomic_read(&bio->bi_cnt)); - - /* - * last put frees it - */ - if (atomic_dec_and_test(&bio->bi_cnt)) + if (!bio_flagged(bio, BIO_REFFED)) bio_free(bio); + else { + BIO_BUG_ON(!atomic_read(&bio->__bi_cnt)); + + /* + * last put frees it + */ + if (atomic_dec_and_test(&bio->__bi_cnt)) + bio_free(bio); + } } EXPORT_SYMBOL(bio_put); |