Age | Commit message (Collapse) | Author | Files | Lines |
|
Pull rdma updates from Jason Gunthorpe:
"Not a big list of changes this cycle, mostly small things. The new
MANA rdma driver should come next cycle along with a bunch of work on
rxe.
Summary:
- Small bug fixes in mlx5, efa, rxe, hns, irdma, erdma, siw
- rts tracing improvements
- Code improvements: strlscpy conversion, unused parameter, spelling
mistakes, unused variables, flex arrays
- restrack device details report for hns
- Simplify struct device initialization in SRP
- Eliminate the never-used service_mask support in IB CM
- Make rxe not print to the console for some kinds of network packets
- Asymetric paths and router support in the CM through netlink
messages
- DMABUF importer support for mlx5devx umem's"
* tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma: (84 commits)
RDMA/rxe: Remove error/warning messages from packet receiver path
RDMA/usnic: fix set-but-not-unused variable 'flags' warning
IB/hfi1: Use skb_put_data() instead of skb_put/memcpy pair
RDMA/hns: Unified Log Printing Style
RDMA/hns: Replacing magic number with macros in apply_func_caps()
RDMA/hns: Repacing 'dseg_len' by macros in fill_ext_sge_inl_data()
RDMA/hns: Remove redundant 'max_srq_desc_sz' in caps
RDMA/hns: Remove redundant 'num_mtt_segs' and 'max_extend_sg'
RDMA/hns: Remove redundant 'phy_addr' in hns_roce_hem_list_find_mtt()
RDMA/hns: Remove redundant 'use_lowmem' argument from hns_roce_init_hem_table()
RDMA/hns: Remove redundant 'bt_level' for hem_list_alloc_item()
RDMA/hns: Remove redundant 'attr_mask' in modify_qp_init_to_init()
RDMA/hns: Remove unnecessary brackets when getting point
RDMA/hns: Remove unnecessary braces for single statement blocks
RDMA/hns: Cleanup for a spelling error of Asynchronous
IB/rdmavt: Add __init/__exit annotations to module init/exit funcs
RDMA/rxe: Remove redundant num_sge fields
RDMA/mlx5: Enable ATS support for MRs and umems
RDMA/mlx5: Add support for dmabuf to devx umem
RDMA/core: Add UVERBS_ATTR_RAW_FD
...
|
|
Given that rnbd_srv_sess_dev already has an open_flags member, there
is no need for the rnbd_dev indirection as a simple block_device pointer
works just as well.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
Acked-by: Jack Wang <jinpu.wang@ionos.com>
Link: https://lore.kernel.org/r/20220909131509.3263924-5-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
These can be trivially open coded in the callers.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
Acked-by: Jack Wang <jinpu.wang@ionos.com>
Link: https://lore.kernel.org/r/20220909131509.3263924-4-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Fold rnbd_endio into the only caller.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
Acked-by: Jack Wang <jinpu.wang@ionos.com>
Link: https://lore.kernel.org/r/20220909131509.3263924-3-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Remove all the wrappers and just get the information directly from
the block device, or where no such helpers exist the request_queue.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
Acked-by: Jack Wang <jinpu.wang@ionos.com>
Link: https://lore.kernel.org/r/20220909131509.3263924-2-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Change the return type to void given it always returns 0.
Signed-off-by: Guoqing Jiang <guoqing.jiang@linux.dev>
Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
Acked-by: Md Haris Iqbal <haris.iqbal@ionos.com>
Link: https://lore.kernel.org/r/20220902100055.25724-3-guoqing.jiang@linux.dev
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Let's add some explanations here given the err handling is not obvious.
Signed-off-by: Guoqing Jiang <guoqing.jiang@linux.dev>
Acked-by: Md Haris Iqbal <haris.iqbal@ionos.com>
Link: https://lore.kernel.org/r/20220902100055.25724-2-guoqing.jiang@linux.dev
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Since process_{read,write} already prints direction info if ctx->ops.rdma_ev
fails, no need to pass 'dir'.
Link: https://lore.kernel.org/r/20220826081117.21687-1-guoqing.jiang@linux.dev
Signed-off-by: Guoqing Jiang <guoqing.jiang@linux.dev>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Add event tracing mechanism for following routines:
- create_sess()
- destroy_sess()
- process_rdma()
- process_msg_sess_info()
- process_msg_open()
- process_msg_close()
How to use:
1. Load the rnbd_server module
2. cd /sys/kernel/debug/tracing
3. If all the events need to be enabled:
echo 1 > events/rnbd_srv/enable
4. OR only speific routine/event needs to be enabled e.g.
echo 1 > events/rnbd_srv/create_sess/enable
5. cat trace
5. Run some workload which can trigger create_sess() routine/event
Signed-off-by: Santosh Pradhan <santosh.pradhan@ionos.com>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com>
Link: https://lore.kernel.org/r/20220818105551.110490-2-haris.iqbal@ionos.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
The structure rnbd_srv_session maintains a list and an xarray of
rnbd_srv_dev. There is no need to keep both as one of them can serve the
purpose.
Since one of the places where the lookup of rnbd_srv_dev using
rnbd_srv_session is IO path, an xarray would serve us better than a list
traversal. Hence remove sess_dev_list from rnbd_srv_session, and replace
its uses from xarray.
Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com>
Reviewed-by: Aleksei Marov <aleksei.marov@ionos.com>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
Link: https://lore.kernel.org/r/20220707143122.460362-3-haris.iqbal@ionos.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
After setting keep_id if the mutex trylock fails, the keep_id stays set
for the rest of the sess_dev lifetime.
Therefore, set keep_id to true after mutex_trylock succeeds, so that a
failure of trylock does'nt touch keep_id.
Fixes: b168e1d85cf3 ("block/rnbd-srv: Prevent a deadlock generated by accessing sysfs in parallel")
Cc: gi-oh.kim@ionos.com
Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
Link: https://lore.kernel.org/r/20220707143122.460362-2-haris.iqbal@ionos.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Just print the block device name directly using the %pg format specifier.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Acked-by: Jack Wang <jinpu.wang@ionos.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Link: https://lore.kernel.org/r/20220713055317.1888500-7-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Add a helper to check the FUA flag based on the block_device instead of
having to poke into the block layer internal request_queue.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Martin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
Link: https://lore.kernel.org/r/20220415045258.199825-14-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Add a helper to check the write cache flag based on the block_device
instead of having to poke into the block layer internal request_queue.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Martin K. Petersen <martin.petersen@oracle.com>
Acked-by: David Sterba <dsterba@suse.com> [btrfs]
Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
Link: https://lore.kernel.org/r/20220415045258.199825-13-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Pull SCSI updates from James Bottomley:
"This series consists of the usual driver updates (qla2xxx, pm8001,
libsas, smartpqi, scsi_debug, lpfc, iscsi, mpi3mr) plus minor updates
and bug fixes.
The high blast radius core update is the removal of write same, which
affects block and several non-SCSI devices. The other big change,
which is more local, is the removal of the SCSI pointer"
* tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi: (281 commits)
scsi: scsi_ioctl: Drop needless assignment in sg_io()
scsi: bsg: Drop needless assignment in scsi_bsg_sg_io_fn()
scsi: lpfc: Copyright updates for 14.2.0.0 patches
scsi: lpfc: Update lpfc version to 14.2.0.0
scsi: lpfc: SLI path split: Refactor BSG paths
scsi: lpfc: SLI path split: Refactor Abort paths
scsi: lpfc: SLI path split: Refactor SCSI paths
scsi: lpfc: SLI path split: Refactor CT paths
scsi: lpfc: SLI path split: Refactor misc ELS paths
scsi: lpfc: SLI path split: Refactor VMID paths
scsi: lpfc: SLI path split: Refactor FDISC paths
scsi: lpfc: SLI path split: Refactor LS_RJT paths
scsi: lpfc: SLI path split: Refactor LS_ACC paths
scsi: lpfc: SLI path split: Refactor the RSCN/SCR/RDF/EDC/FARPR paths
scsi: lpfc: SLI path split: Refactor PLOGI/PRLI/ADISC/LOGO paths
scsi: lpfc: SLI path split: Refactor base ELS paths and the FLOGI path
scsi: lpfc: SLI path split: Introduce lpfc_prep_wqe
scsi: lpfc: SLI path split: Refactor fast and slow paths to native SLI4
scsi: lpfc: SLI path split: Refactor lpfc_iocbq
scsi: lpfc: Use kcalloc()
...
|
|
On client side, the device is a network device. There is no reason
to set rotational even-if the target device on server is rotational.
Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com>
Link: https://lore.kernel.org/r/20220114155855.984144-3-haris.iqbal@ionos.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
REQ_OP_WRITE_SAME was only ever submitted by the legacy Linux zeroing code,
which has switched to use REQ_OP_WRITE_ZEROES long before rnbd was even
merged.
Link: https://lore.kernel.org/r/20220209082828.2629273-4-hch@lst.de
Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
Acked-by: Jack Wang <jinpu.wang@ionos.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
Pass the block_device and operation that we plan to use this bio for to
bio_alloc to optimize the assignment. NULL/0 can be passed, both for the
passthrough case on a raw request_queue and to temporarily avoid
refactoring some nasty code.
Also move the gfp_mask argument after the nr_vecs argument for a much
more logical calling convention matching what most of the kernel does.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
Link: https://lore.kernel.org/r/20220124091107.642561-18-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Only the priv field of rnbd_dev_blk_io is used, so store the value of
that in bio->bi_private directly and remove the entire bio_set overhead.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Jack Wang <jinpu.wang@ionos.com>
Link: https://lore.kernel.org/r/20220124091107.642561-12-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
The memory mapped in process_rdma is contiguous, so there is no need
to loop over bio_add_page. Remove rnbd_bio_map_kern and just open code
the bio allocation and mapping in the caller.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Jack Wang <jinpu.wang@ionons.com>
Tested-by: Jack Wang <jinpu.wang@ionos.com>
Link: https://lore.kernel.org/r/20220124091107.642561-11-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Structure rtrs_srv is used for sessions so in order to avoid confusions
rename it to rtrs_srv_sess.
All changes were done with the help of following Coccinelle script:
@@
@@
struct
- rtrs_srv
+ rtrs_srv_sess
Link: https://lore.kernel.org/r/20220105180708.7774-5-jinpu.wang@ionos.com
Signed-off-by: Vaishali Thakkar <vaishali.thakkar@ionos.com>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
rtrs_srv_sess is used for paths and not sessions on the server side. This
creates confusion so let's rename it to rtrs_srv_path. Also, rename
related variables and functions.
Coccinelle is used to do the transformations for most of the occurrences
and remaining ones were handled manually.
Link: https://lore.kernel.org/r/20220105180708.7774-3-jinpu.wang@ionos.com
Signed-off-by: Vaishali Thakkar <vaishali.thakkar@ionos.com>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
The IO performance test with fio after removing the likely and
unlikely macros in all if-statement shows no performance drop.
They do not help for the performance of rnbd.
The fio test did random read on 32 rnbd devices and 64 processes.
Test environment:
- AMD Opteron(tm) Processor 6386 SE
- 125G memory
- kernel version: 5.4.86
- gcc version: gcc (Debian 8.3.0-6) 8.3.0
- Infiniband controller: InfiniBand: Mellanox Technologies MT26428
[ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE] (rev b0)
before
read: IOPS=549k, BW=2146MiB/s
read: IOPS=544k, BW=2125MiB/s
read: IOPS=553k, BW=2158MiB/s
read: IOPS=535k, BW=2089MiB/s
read: IOPS=543k, BW=2122MiB/s
read: IOPS=552k, BW=2154MiB/s
average: IOPS=546k, BW=2132MiB/s
after
read: IOPS=556k, BW=2172MiB/s
read: IOPS=561k, BW=2191MiB/s
read: IOPS=552k, BW=2156MiB/s
read: IOPS=551k, BW=2154MiB/s
read: IOPS=562k, BW=2194MiB/s
-----------
average: IOPS=556k, BW=2173MiB/s
The IOPS and bandwidth got better slightly after removing
likely/unlikely. (IOPS= +1.8% BW= +1.9%) But we cannot make sure
that removing the likely/unlikely help the performance because it
depends on various situations. We only make sure that removing the
likely/unlikely does not drop the performance.
Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com>
Reviewed-by: Md Haris Iqbal <haris.iqbal@ionos.com>
Link: https://lore.kernel.org/r/20210428061359.206794-5-gi-oh.kim@ionos.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
During checkpatch analyzing the following warning message was found:
WARNING:STRLCPY: Prefer strscpy over strlcpy - see:
https://lore.kernel.org/r/CAHk-=wgfRnXz0W3D37d01q3JFkr_i_uTL=V6A6G1oUZcprmknw@mail.gmail.com/
Fix it by using strscpy calls instead of strlcpy.
Signed-off-by: Dima Stepanov <dmitrii.stepanov@cloud.ionos.com>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com>
Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com>
Link: https://lore.kernel.org/r/20210419073722.15351-20-gi-oh.kim@ionos.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
struct rtrs_srv is not used when handling rnbd_srv_rdma_ev messages, so
cleaned up
rdma_ev function pointer in rtrs_srv_ops also is changed.
Cc: Leon Romanovsky <leonro@nvidia.com>
Cc: linux-rdma@vger.kernel.org
Signed-off-by: Aleksei Marov <aleksei.marov@ionos.com>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com>
Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com>
Acked-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Leon Romanovsky <leonro@nvidia.com>
Link: https://lore.kernel.org/r/20210419073722.15351-16-gi-oh.kim@ionos.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
We changed the rnbd_srv_sess_dev_force_close to use try-lock
because rnbd_srv_sess_dev_force_close and process_msg_close
can generate a deadlock.
Now rnbd_srv_sess_dev_force_close would do nothing
if it fails to get the lock. So removing the force_close
file should be moved to after the lock. Or the force_close
file is removed but the others are not removed.
Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com>
Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
Link: https://lore.kernel.org/r/20210419073722.15351-11-gi-oh.kim@ionos.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
We got a warning message below.
When server tries to close one session by force, it locks the sysfs
interface and locks the srv_sess lock.
The problem is that client can send a request to close at the same time.
By close request, server locks the srv_sess lock and locks the sysfs
to remove the sysfs interfaces.
The simplest way to prevent that situation could be just use
mutex_trylock.
[ 234.153965] ======================================================
[ 234.154093] WARNING: possible circular locking dependency detected
[ 234.154219] 5.4.84-storage #5.4.84-1+feature+linux+5.4.y+dbg+20201216.1319+b6b887b~deb10 Tainted: G O
[ 234.154381] ------------------------------------------------------
[ 234.154531] kworker/1:1H/618 is trying to acquire lock:
[ 234.154651] ffff8887a09db0a8 (kn->count#132){++++}, at: kernfs_remove_by_name_ns+0x40/0x80
[ 234.154819]
but task is already holding lock:
[ 234.154965] ffff8887ae5f6518 (&srv_sess->lock){+.+.}, at: rnbd_srv_rdma_ev+0x144/0x1590 [rnbd_server]
[ 234.155132]
which lock already depends on the new lock.
[ 234.155311]
the existing dependency chain (in reverse order) is:
[ 234.155462]
-> #1 (&srv_sess->lock){+.+.}:
[ 234.155614] __mutex_lock+0x134/0xcb0
[ 234.155761] rnbd_srv_sess_dev_force_close+0x36/0x50 [rnbd_server]
[ 234.155889] rnbd_srv_dev_session_force_close_store+0x69/0xc0 [rnbd_server]
[ 234.156042] kernfs_fop_write+0x13f/0x240
[ 234.156162] vfs_write+0xf3/0x280
[ 234.156278] ksys_write+0xba/0x150
[ 234.156395] do_syscall_64+0x62/0x270
[ 234.156513] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 234.156632]
-> #0 (kn->count#132){++++}:
[ 234.156782] __lock_acquire+0x129e/0x23a0
[ 234.156900] lock_acquire+0xf3/0x210
[ 234.157043] __kernfs_remove+0x42b/0x4c0
[ 234.157161] kernfs_remove_by_name_ns+0x40/0x80
[ 234.157282] remove_files+0x3f/0xa0
[ 234.157399] sysfs_remove_group+0x4a/0xb0
[ 234.157519] rnbd_srv_destroy_dev_session_sysfs+0x19/0x30 [rnbd_server]
[ 234.157648] rnbd_srv_rdma_ev+0x14c/0x1590 [rnbd_server]
[ 234.157775] process_io_req+0x29a/0x6a0 [rtrs_server]
[ 234.157924] __ib_process_cq+0x8c/0x100 [ib_core]
[ 234.158709] ib_cq_poll_work+0x31/0xb0 [ib_core]
[ 234.158834] process_one_work+0x4e5/0xaa0
[ 234.158958] worker_thread+0x65/0x5c0
[ 234.159078] kthread+0x1e0/0x200
[ 234.159194] ret_from_fork+0x24/0x30
[ 234.159309]
other info that might help us debug this:
[ 234.159513] Possible unsafe locking scenario:
[ 234.159658] CPU0 CPU1
[ 234.159775] ---- ----
[ 234.159891] lock(&srv_sess->lock);
[ 234.160005] lock(kn->count#132);
[ 234.160128] lock(&srv_sess->lock);
[ 234.160250] lock(kn->count#132);
[ 234.160364]
*** DEADLOCK ***
[ 234.160536] 3 locks held by kworker/1:1H/618:
[ 234.160677] #0: ffff8883ca1ed528 ((wq_completion)ib-comp-wq){+.+.}, at: process_one_work+0x40a/0xaa0
[ 234.160840] #1: ffff8883d2d5fe10 ((work_completion)(&cq->work)){+.+.}, at: process_one_work+0x40a/0xaa0
[ 234.161003] #2: ffff8887ae5f6518 (&srv_sess->lock){+.+.}, at: rnbd_srv_rdma_ev+0x144/0x1590 [rnbd_server]
[ 234.161168]
stack backtrace:
[ 234.161312] CPU: 1 PID: 618 Comm: kworker/1:1H Tainted: G O 5.4.84-storage #5.4.84-1+feature+linux+5.4.y+dbg+20201216.1319+b6b887b~deb10
[ 234.161490] Hardware name: Supermicro H8QG6/H8QG6, BIOS 3.00 09/04/2012
[ 234.161643] Workqueue: ib-comp-wq ib_cq_poll_work [ib_core]
[ 234.161765] Call Trace:
[ 234.161910] dump_stack+0x96/0xe0
[ 234.162028] check_noncircular+0x29e/0x2e0
[ 234.162148] ? print_circular_bug+0x100/0x100
[ 234.162267] ? register_lock_class+0x1ad/0x8a0
[ 234.162385] ? __lock_acquire+0x68e/0x23a0
[ 234.162505] ? trace_event_raw_event_lock+0x190/0x190
[ 234.162626] __lock_acquire+0x129e/0x23a0
[ 234.162746] ? register_lock_class+0x8a0/0x8a0
[ 234.162866] lock_acquire+0xf3/0x210
[ 234.162982] ? kernfs_remove_by_name_ns+0x40/0x80
[ 234.163127] __kernfs_remove+0x42b/0x4c0
[ 234.163243] ? kernfs_remove_by_name_ns+0x40/0x80
[ 234.163363] ? kernfs_fop_readdir+0x3b0/0x3b0
[ 234.163482] ? strlen+0x1f/0x40
[ 234.163596] ? strcmp+0x30/0x50
[ 234.163712] kernfs_remove_by_name_ns+0x40/0x80
[ 234.163832] remove_files+0x3f/0xa0
[ 234.163948] sysfs_remove_group+0x4a/0xb0
[ 234.164068] rnbd_srv_destroy_dev_session_sysfs+0x19/0x30 [rnbd_server]
[ 234.164196] rnbd_srv_rdma_ev+0x14c/0x1590 [rnbd_server]
[ 234.164345] ? _raw_spin_unlock_irqrestore+0x43/0x50
[ 234.164466] ? lockdep_hardirqs_on+0x1a8/0x290
[ 234.164597] ? mlx4_ib_poll_cq+0x927/0x1280 [mlx4_ib]
[ 234.164732] ? rnbd_get_sess_dev+0x270/0x270 [rnbd_server]
[ 234.164859] process_io_req+0x29a/0x6a0 [rtrs_server]
[ 234.164982] ? rnbd_get_sess_dev+0x270/0x270 [rnbd_server]
[ 234.165130] __ib_process_cq+0x8c/0x100 [ib_core]
[ 234.165279] ib_cq_poll_work+0x31/0xb0 [ib_core]
[ 234.165404] process_one_work+0x4e5/0xaa0
[ 234.165550] ? pwq_dec_nr_in_flight+0x160/0x160
[ 234.165675] ? do_raw_spin_lock+0x119/0x1d0
[ 234.165796] worker_thread+0x65/0x5c0
[ 234.165914] ? process_one_work+0xaa0/0xaa0
[ 234.166031] kthread+0x1e0/0x200
[ 234.166147] ? kthread_create_worker_on_cpu+0xc0/0xc0
[ 234.166268] ret_from_fork+0x24/0x30
[ 234.251591] rnbd_server L243: </dev/loop1@close_device_session>: Device closed
[ 234.604221] rnbd_server L264: RTRS Session close_device_session disconnected
Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com>
Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
Link: https://lore.kernel.org/r/20210419073722.15351-10-gi-oh.kim@ionos.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
We can use destroy_device directly since destroy_device_cb is just the
wrapper of destroy_device.
Signed-off-by: Guoqing Jiang <guoqing.jiang@ionos.com>
Reviewed-by: Danil Kipnis <danil.kipnis@ionos.com>
Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com>
Link: https://lore.kernel.org/r/20210419073722.15351-8-gi-oh.kim@ionos.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
KASAN detect following BUG:
[ 778.215311] ==================================================================
[ 778.216696] BUG: KASAN: use-after-free in rnbd_srv_sess_dev_force_close+0x38/0x60 [rnbd_server]
[ 778.219037] Read of size 8 at addr ffff88b1d6516c28 by task tee/8842
[ 778.220500] CPU: 37 PID: 8842 Comm: tee Kdump: loaded Not tainted 5.10.0-pserver #5.10.0-1+feature+linux+next+20201214.1025+0910d71
[ 778.220529] Hardware name: Supermicro Super Server/X11DDW-L, BIOS 3.3 02/21/2020
[ 778.220555] Call Trace:
[ 778.220609] dump_stack+0x99/0xcb
[ 778.220667] ? rnbd_srv_sess_dev_force_close+0x38/0x60 [rnbd_server]
[ 778.220715] print_address_description.constprop.7+0x1e/0x230
[ 778.220750] ? freeze_kernel_threads+0x73/0x73
[ 778.220896] ? rnbd_srv_sess_dev_force_close+0x38/0x60 [rnbd_server]
[ 778.220932] ? rnbd_srv_sess_dev_force_close+0x38/0x60 [rnbd_server]
[ 778.220994] kasan_report.cold.9+0x37/0x7c
[ 778.221066] ? kobject_put+0x80/0x270
[ 778.221102] ? rnbd_srv_sess_dev_force_close+0x38/0x60 [rnbd_server]
[ 778.221184] rnbd_srv_sess_dev_force_close+0x38/0x60 [rnbd_server]
[ 778.221240] rnbd_srv_dev_session_force_close_store+0x6a/0xc0 [rnbd_server]
[ 778.221304] ? sysfs_file_ops+0x90/0x90
[ 778.221353] kernfs_fop_write+0x141/0x240
[ 778.221451] vfs_write+0x142/0x4d0
[ 778.221553] ksys_write+0xc0/0x160
[ 778.221602] ? __ia32_sys_read+0x50/0x50
[ 778.221684] ? lockdep_hardirqs_on_prepare+0x13d/0x210
[ 778.221718] ? syscall_enter_from_user_mode+0x1c/0x50
[ 778.221821] do_syscall_64+0x33/0x40
[ 778.221862] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 778.221896] RIP: 0033:0x7f4affdd9504
[ 778.221928] Code: 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b3 0f 1f 80 00 00 00 00 48 8d 05 f9 61 0d 00 8b 00 85 c0 75 13 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 41 54 49 89 d4 55 48 89 f5 53
[ 778.221956] RSP: 002b:00007fffebb36b28 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
[ 778.222011] RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007f4affdd9504
[ 778.222038] RDX: 0000000000000002 RSI: 00007fffebb36c50 RDI: 0000000000000003
[ 778.222066] RBP: 00007fffebb36c50 R08: 0000556a151aa600 R09: 00007f4affeb1540
[ 778.222094] R10: fffffffffffffc19 R11: 0000000000000246 R12: 0000556a151aa520
[ 778.222121] R13: 0000000000000002 R14: 00007f4affea6760 R15: 0000000000000002
[ 778.222764] Allocated by task 3212:
[ 778.223285] kasan_save_stack+0x19/0x40
[ 778.223316] __kasan_kmalloc.constprop.7+0xc1/0xd0
[ 778.223347] kmem_cache_alloc_trace+0x186/0x350
[ 778.223382] rnbd_srv_rdma_ev+0xf16/0x1690 [rnbd_server]
[ 778.223422] process_io_req+0x4d1/0x670 [rtrs_server]
[ 778.223573] __ib_process_cq+0x10a/0x350 [ib_core]
[ 778.223709] ib_cq_poll_work+0x31/0xb0 [ib_core]
[ 778.223743] process_one_work+0x521/0xa90
[ 778.223773] worker_thread+0x65/0x5b0
[ 778.223802] kthread+0x1f2/0x210
[ 778.223833] ret_from_fork+0x22/0x30
[ 778.224296] Freed by task 8842:
[ 778.224800] kasan_save_stack+0x19/0x40
[ 778.224829] kasan_set_track+0x1c/0x30
[ 778.224860] kasan_set_free_info+0x1b/0x30
[ 778.224889] __kasan_slab_free+0x108/0x150
[ 778.224919] slab_free_freelist_hook+0x64/0x190
[ 778.224947] kfree+0xe2/0x650
[ 778.224982] rnbd_destroy_sess_dev+0x2fa/0x3b0 [rnbd_server]
[ 778.225011] kobject_put+0xda/0x270
[ 778.225046] rnbd_srv_sess_dev_force_close+0x30/0x60 [rnbd_server]
[ 778.225081] rnbd_srv_dev_session_force_close_store+0x6a/0xc0 [rnbd_server]
[ 778.225111] kernfs_fop_write+0x141/0x240
[ 778.225140] vfs_write+0x142/0x4d0
[ 778.225169] ksys_write+0xc0/0x160
[ 778.225198] do_syscall_64+0x33/0x40
[ 778.225227] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 778.226506] The buggy address belongs to the object at ffff88b1d6516c00
which belongs to the cache kmalloc-512 of size 512
[ 778.227464] The buggy address is located 40 bytes inside of
512-byte region [ffff88b1d6516c00, ffff88b1d6516e00)
The problem is in the sess_dev release function we call
rnbd_destroy_sess_dev, and could free the sess_dev already, but we still
set the keep_id in rnbd_srv_sess_dev_force_close, which lead to use
after free.
To fix it, move the keep_id before the sysfs removal, and cache the
rnbd_srv_session for lock accessing,
Fixes: 786998050cbc ("block/rnbd-srv: close a mapped device from server side.")
Signed-off-by: Jack Wang <jinpu.wang@cloud.ionos.com>
Reviewed-by: Guoqing Jiang <guoqing.jiang@cloud.ionos.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
The rnbd-client always sets the write-back cache and fua attributes
of the rnbd device queue regardless of the target device on the server.
That generates IO hang issue when the target device does not
support both of write-back cacne and fua.
This patch adds more fields for the cache policy and fua into the
device opening message. The rnbd-server sends the information
if the target device supports the write-back cache and fua
and rnbd-client recevives it and set the device queue accordingly.
Signed-off-by: Gioh Kim <gi-oh.kim@cloud.ionos.com>
[jwang: some minor change, rename a few varables, remove unrelated comments.]
Signed-off-by: Jack Wang <jinpu.wang@cloud.ionos.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Since the removal of the session sysfs can also be called from the
function destroy_sess, there is a need to protect the call from the
function rnbd_srv_sess_dev_force_close
Fixes: 786998050cbc ("block/rnbd-srv: close a mapped device from server side.")
Signed-off-by: Md Haris Iqbal <haris.iqbal@cloud.ionos.com>
Reviewed-by: Guoqing Jiang <guoqing.jiang@cloud.ionos.com>
Signed-off-by: Jack Wang <jinpu.wang@cloud.ionos.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
The forceful close of an exported device is required
for the use case, when the client side hangs, is crashed,
or is not accessible.
There have been cases observed, where only some of
the devices are to be cleaned up, but the session shall
remain.
When the device is to be exported to a different
client host, server side cleanup is required.
Signed-off-by: Lutz Pogrell <lutz.pogrell@cloud.ionos.com>
Signed-off-by: Jack Wang <jinpu.wang@cloud.ionos.com>
Reviewed-by: Gioh Kim <gi-oh.kim@cloud.ionos.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Clang warns:
drivers/block/rnbd/rnbd-srv.c:150:6: warning: variable 'err' is used
uninitialized whenever 'if' condition is true
[-Wsometimes-uninitialized]
if (IS_ERR(bio)) {
^~~~~~~~~~~
drivers/block/rnbd/rnbd-srv.c:177:9: note: uninitialized use occurs here
return err;
^~~
drivers/block/rnbd/rnbd-srv.c:150:2: note: remove the 'if' if its
condition is always false
if (IS_ERR(bio)) {
^~~~~~~~~~~~~~~~~~
drivers/block/rnbd/rnbd-srv.c:126:9: note: initialize the variable 'err'
to silence this warning
int err;
^
= 0
1 warning generated.
err is indeed uninitialized when this statement is taken. Ensure that it
is assigned the error value of bio before jumping to the error handling
label.
Fixes: 735d77d4fd28 ("rnbd: remove rnbd_dev_submit_io")
Reported-by: Brooke Basile <brookebasile@gmail.com>
Signed-off-by: Nathan Chancellor <natechancellor@gmail.com>
Acked-by: Jack Wang <jinpu.wang@cloud.ionos.com>
Link: https://github.com/ClangBuiltLinux/linux/issues/1134
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
The function only has one caller, so let's open code it in process_rdma.
Another bonus is we can avoid push/pop stack, since we need to pass 8
arguments to rnbd_dev_submit_io.
Signed-off-by: Guoqing Jiang <guoqing.jiang@cloud.ionos.com>
Acked-by: Danil Kipnis <danil.kipnis@cloud.ionos.com>
Acked-by: Jack Wang <jinpu.wang@cloud.ionos.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
This is main functionality of rnbd-server module, which handles RTRS
events and rnbd protocol requests, like map (open) or unmap (close)
device. Also server side is responsible for processing incoming IBTRS IO
requests and forward them to local mapped devices.
Link: https://lore.kernel.org/r/20200511135131.27580-21-danil.kipnis@cloud.ionos.com
Signed-off-by: Danil Kipnis <danil.kipnis@cloud.ionos.com>
Signed-off-by: Jack Wang <jinpu.wang@cloud.ionos.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Acked-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
|