aboutsummaryrefslogtreecommitdiffstats
path: root/drivers/infiniband/ulp/srpt (follow)
AgeCommit message (Collapse)AuthorFilesLines
2019-11-13RDMA/srpt: Report the SCSI residual to the initiatorBart Van Assche1-0/+24
The code added by this patch is similar to the code that already exists in ibmvscsis_determine_resid(). This patch has been tested by running the following command: strace sg_raw -r 1k /dev/sdb 12 00 00 00 60 00 -o inquiry.bin |& grep resid= Link: https://lore.kernel.org/r/20191105214632.183302-1-bvanassche@acm.org Fixes: a42d985bd5b2 ("ib_srpt: Initial SRP Target merge for v3.3-rc1") Signed-off-by: Bart Van Assche <bvanassche@acm.org> Acked-by: Honggang Li <honli@redhat.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2019-11-06Revert "RDMA/srpt: Postpone HCA removal until after configfs directory removal"Bart Van Assche1-5/+1
Although the mentioned patch fixes a use-after-free bug, it introduces a hang during shutdown. Since the latter is worse, revert this patch. Link: https://lore.kernel.org/r/20191101204756.182162-1-bvanassche@acm.org Reported-by: Honggang Li <honli@redhat.com> Fixes: 9b64f7d0bb0a ("RDMA/srpt: Postpone HCA removal until after configfs directory removal") Signed-off-by: Bart Van Assche <bvanassche@acm.org> Acked-by: Honggang Li <honli@redhat.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2019-10-28RDMA/srpt: Fix TPG creationBart Van Assche2-29/+73
Unlike the iSCSI target driver, for the SRP target driver it is sufficient if a single TPG can be associated with each RDMA port name. However, users started associating multiple TPGs with RDMA port names. Support this by converting the single TPG in struct srpt_port_id into a list. This patch fixes the following list corruption issue: list_add corruption. prev->next should be next (ffffffffc0a080c0), but was ffffa08a994ce6f0. (prev=ffffa08a994ce6f0). WARNING: CPU: 2 PID: 2597 at lib/list_debug.c:28 __list_add_valid+0x6a/0x70 CPU: 2 PID: 2597 Comm: targetcli Not tainted 5.4.0-rc1.3bfa3c9602a7 #1 RIP: 0010:__list_add_valid+0x6a/0x70 Call Trace: core_tpg_register+0x116/0x200 [target_core_mod] srpt_make_tpg+0x3f/0x60 [ib_srpt] target_fabric_make_tpg+0x41/0x290 [target_core_mod] configfs_mkdir+0x158/0x3e0 vfs_mkdir+0x108/0x1a0 do_mkdirat+0x77/0xe0 do_syscall_64+0x55/0x1d0 entry_SYSCALL_64_after_hwframe+0x44/0xa9 Link: https://lore.kernel.org/r/20191023204106.23326-1-bvanassche@acm.org Reported-by: Honggang LI <honli@redhat.com> Fixes: a42d985bd5b2 ("ib_srpt: Initial SRP Target merge for v3.3-rc1") Signed-off-by: Bart Van Assche <bvanassche@acm.org> Acked-by: Honggang Li <honli@redhat.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2019-10-22infiniband: fix ulp/srpt/ib_srpt.h kernel-doc notationrd.dunlab@gmail.com1-5/+2
Fix kernel-doc warnings (typos or renames) in ib_srpt.h: ../drivers/infiniband/ulp/srpt/ib_srpt.h:419: warning: Function parameter or member 'port_guid_id' not described in 'srpt_port' ../drivers/infiniband/ulp/srpt/ib_srpt.h:419: warning: Function parameter or member 'port_gid_id' not described in 'srpt_port' Link: https://lore.kernel.org/r/20191010035239.950150496@gmail.com Signed-off-by: Randy Dunlap <rd.dunlab@gmail.com> Reviewed-by: Jason Gunthorpe <jgg@mellanox.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2019-10-04RDMA/srpt: Postpone HCA removal until after configfs directory removalBart Van Assche1-1/+4
A shortcoming of the SCSI target core is that it does not have an API for removing tpg or wwn objects. Wait until these directories have been removed before allowing HCA removal to finish. See also Bart Van Assche, "Re: Why using configfs as the only interface is wrong for a storage target", 2011-02-07 (https://www.spinics.net/lists/linux-scsi/msg50248.html). This patch fixes the following kernel crash: ================================================================== BUG: KASAN: use-after-free in __configfs_open_file.isra.4+0x1a8/0x400 Read of size 8 at addr ffff88811880b690 by task restart-lio-srp/1215 CPU: 1 PID: 1215 Comm: restart-lio-srp Not tainted 5.3.0-dbg+ #3 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 Call Trace: dump_stack+0x86/0xca print_address_description+0x74/0x32d __kasan_report.cold.6+0x1b/0x36 kasan_report+0x12/0x17 __asan_load8+0x54/0x90 __configfs_open_file.isra.4+0x1a8/0x400 configfs_open_file+0x13/0x20 do_dentry_open+0x2b1/0x770 vfs_open+0x58/0x60 path_openat+0x5fa/0x14b0 do_filp_open+0x115/0x180 do_sys_open+0x1d4/0x2a0 __x64_sys_openat+0x59/0x70 do_syscall_64+0x6b/0x2d0 entry_SYSCALL_64_after_hwframe+0x49/0xbe RIP: 0033:0x7f2f2bd3fcce Code: 25 00 00 41 00 3d 00 00 41 00 74 48 48 8d 05 19 d7 0d 00 8b 00 85 c0 75 69 89 f2 b8 01 01 00 00 48 89 fe bf 9c ff ff ff 0f 05 <48> 3d 00 f0 ff ff 0f 87 a6 00 00 00 48 8b 4c 24 28 64 48 33 0c 25 RSP: 002b:00007ffd155f7850 EFLAGS: 00000246 ORIG_RAX: 0000000000000101 RAX: ffffffffffffffda RBX: 0000564609ba88e0 RCX: 00007f2f2bd3fcce RDX: 0000000000000241 RSI: 0000564609ba8cf0 RDI: 00000000ffffff9c RBP: 00007ffd155f7950 R08: 0000000000000000 R09: 0000000000000020 R10: 00000000000001b6 R11: 0000000000000246 R12: 0000000000000000 R13: 0000000000000003 R14: 0000000000000001 R15: 0000564609ba8cf0 Allocated by task 995: save_stack+0x21/0x90 __kasan_kmalloc.constprop.9+0xc7/0xd0 kasan_kmalloc+0x9/0x10 __kmalloc+0x153/0x370 srpt_add_one+0x4f/0x561 [ib_srpt] add_client_context+0x251/0x290 [ib_core] ib_register_client+0x1da/0x220 [ib_core] iblock_get_alignment_offset_lbas+0x6b/0x100 [target_core_iblock] do_one_initcall+0xcd/0x43a do_init_module+0x103/0x380 load_module+0x3b77/0x3eb0 __do_sys_finit_module+0x12d/0x1b0 __x64_sys_finit_module+0x43/0x50 do_syscall_64+0x6b/0x2d0 entry_SYSCALL_64_after_hwframe+0x49/0xbe Freed by task 1221: save_stack+0x21/0x90 __kasan_slab_free+0x139/0x190 kasan_slab_free+0xe/0x10 slab_free_freelist_hook+0x67/0x1e0 kfree+0xcb/0x2a0 srpt_remove_one+0x596/0x670 [ib_srpt] remove_client_context+0x9a/0xe0 [ib_core] disable_device+0x106/0x1b0 [ib_core] __ib_unregister_device+0x5f/0xf0 [ib_core] ib_unregister_driver+0x11a/0x170 [ib_core] 0xffffffffa087f666 __x64_sys_delete_module+0x1f8/0x2c0 do_syscall_64+0x6b/0x2d0 entry_SYSCALL_64_after_hwframe+0x49/0xbe The buggy address belongs to the object at ffff88811880b300 which belongs to the cache kmalloc-4k of size 4096 The buggy address is located 912 bytes inside of 4096-byte region [ffff88811880b300, ffff88811880c300) The buggy address belongs to the page: page:ffffea0004620200 refcount:1 mapcount:0 mapping:ffff88811ac0de00 index:0x0 compound_mapcount: 0 flags: 0x2fff000000010200(slab|head) raw: 2fff000000010200 dead000000000100 dead000000000122 ffff88811ac0de00 raw: 0000000000000000 0000000000070007 00000001ffffffff 0000000000000000 page dumped because: kasan: bad access detected Memory state around the buggy address: ffff88811880b580: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ffff88811880b600: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >ffff88811880b680: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ^ ffff88811880b700: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ffff88811880b780: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ================================================================== Link: https://lore.kernel.org/r/20190930231707.48259-16-bvanassche@acm.org Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2019-10-04RDMA/srpt: Make the code for handling port identities more systematicBart Van Assche2-32/+57
Introduce a new data structure for the information about an RDMA port name. This patch does not change any functionality. Link: https://lore.kernel.org/r/20190930231707.48259-15-bvanassche@acm.org Cc: Honggang LI <honli@redhat.com> Cc: Laurence Oberman <loberman@redhat.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2019-10-04RDMA/srpt: Rework the code that waits until an RDMA port is no longer in useBart Van Assche2-23/+25
The current implementation does not wait until srpt_release_channel() has finished and hence can trigger a use-after-free. Rework srpt_release_sport() such that it waits until srpt_release_channel() has finished. This patch fixes the following KASAN complaint: ================================================================== BUG: KASAN: use-after-free in srpt_free_ioctx.part.23+0x42/0x100 [ib_srpt] Read of size 8 at addr ffff888115c71100 by task kworker/4:3/807 CPU: 4 PID: 807 Comm: kworker/4:3 Not tainted 5.3.0-dbg+ #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 Workqueue: events srpt_release_channel_work [ib_srpt] Call Trace: dump_stack+0x86/0xca print_address_description+0x74/0x32d __kasan_report.cold.6+0x1b/0x36 kasan_report+0x12/0x17 __asan_load8+0x54/0x90 srpt_free_ioctx.part.23+0x42/0x100 [ib_srpt] srpt_free_ioctx_ring.part.24+0x50/0x80 [ib_srpt] srpt_release_channel_work+0x2ad/0x390 [ib_srpt] process_one_work+0x51a/0xa60 worker_thread+0x67/0x5b0 kthread+0x1dc/0x200 ret_from_fork+0x24/0x30 Allocated by task 984: save_stack+0x21/0x90 __kasan_kmalloc.constprop.9+0xc7/0xd0 kasan_kmalloc+0x9/0x10 __kmalloc+0x153/0x370 srpt_add_one+0x4f/0x570 [ib_srpt] add_client_context+0x251/0x290 [ib_core] ib_register_client+0x1da/0x220 [ib_core] iblock_get_alignment_offset_lbas+0x6b/0x100 [target_core_iblock] do_one_initcall+0xcd/0x43a do_init_module+0x103/0x380 load_module+0x3b77/0x3eb0 __do_sys_finit_module+0x12d/0x1b0 __x64_sys_finit_module+0x43/0x50 do_syscall_64+0x6b/0x2d0 entry_SYSCALL_64_after_hwframe+0x49/0xbe Freed by task 1128: save_stack+0x21/0x90 __kasan_slab_free+0x139/0x190 kasan_slab_free+0xe/0x10 slab_free_freelist_hook+0x67/0x1e0 kfree+0xcb/0x2a0 srpt_remove_one+0x569/0x5b0 [ib_srpt] remove_client_context+0x9a/0xe0 [ib_core] disable_device+0x106/0x1b0 [ib_core] __ib_unregister_device+0x5f/0xf0 [ib_core] ib_unregister_device_and_put+0x48/0x60 [ib_core] nldev_dellink+0x120/0x180 [ib_core] rdma_nl_rcv+0x287/0x480 [ib_core] netlink_unicast+0x2cc/0x370 netlink_sendmsg+0x3b1/0x630 __sys_sendto+0x1db/0x290 __x64_sys_sendto+0x80/0xa0 do_syscall_64+0x6b/0x2d0 entry_SYSCALL_64_after_hwframe+0x49/0xbe The buggy address belongs to the object at ffff888115c71100 which belongs to the cache kmalloc-4k of size 4096 The buggy address is located 0 bytes inside of 4096-byte region [ffff888115c71100, ffff888115c72100) The buggy address belongs to the page: page:ffffea0004571c00 refcount:1 mapcount:0 mapping:ffff88811ac0de00 index:0xffff888115c70000 compound_mapcount: 0 flags: 0x2fff000000010200(slab|head) raw: 2fff000000010200 ffffea00045ac408 ffffea0004593208 ffff88811ac0de00 raw: ffff888115c70000 0000000000070002 00000001ffffffff 0000000000000000 page dumped because: kasan: bad access detected Memory state around the buggy address: ffff888115c71000: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc ffff888115c71080: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc >ffff888115c71100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ^ ffff888115c71180: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ffff888115c71200: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ================================================================== Link: https://lore.kernel.org/r/20190930231707.48259-14-bvanassche@acm.org Cc: Honggang LI <honli@redhat.com> Cc: Laurence Oberman <loberman@redhat.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2019-10-04RDMA/srpt: Rework the approach for closing an RDMA channelBart Van Assche2-23/+10
Instead of relying on a waitqueue, report when the identity of an RDMA channel can be reused through a completion. Link: https://lore.kernel.org/r/20190930231707.48259-13-bvanassche@acm.org Cc: Honggang LI <honli@redhat.com> Cc: Laurence Oberman <loberman@redhat.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2019-10-04RDMA/srpt: Improve a debug messageBart Van Assche1-1/+2
The ib_srpt driver uses two different identifiers while registering a session with the LIO core. Report both identifiers if the modified pr_debug() statement is enabled. Link: https://lore.kernel.org/r/20190930231707.48259-12-bvanassche@acm.org Cc: Honggang LI <honli@redhat.com> Cc: Laurence Oberman <loberman@redhat.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2019-10-04RDMA/srpt: Fix handling of iWARP loginsBart Van Assche1-1/+2
The path_rec pointer is NULL set for IB and RoCE logins but not for iWARP logins. Hence check the path_rec pointer before dereferencing it. Link: https://lore.kernel.org/r/20190930231707.48259-11-bvanassche@acm.org Cc: Honggang LI <honli@redhat.com> Cc: Laurence Oberman <loberman@redhat.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2019-10-04RDMA/srpt: Fix handling of SR-IOV and iWARP portsBart Van Assche1-22/+19
Management datagrams (MADs) are not supported by SR-IOV VFs nor by iWARP ports. Support SR-IOV VFs and iWARP ports by only logging an error message if MAD handler registration fails. Link: https://lore.kernel.org/r/20190930231707.48259-10-bvanassche@acm.org Cc: Honggang LI <honli@redhat.com> Cc: Laurence Oberman <loberman@redhat.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2019-08-05rdma: Enable ib_alloc_cq to spread work over a device's comp_vectorsChuck Lever1-2/+2
Send and Receive completion is handled on a single CPU selected at the time each Completion Queue is allocated. Typically this is when an initiator instantiates an RDMA transport, or when a target accepts an RDMA connection. Some ULPs cannot open a connection per CPU to spread completion workload across available CPUs and MSI vectors. For such ULPs, provide an API that allows the RDMA core to select a completion vector based on the device's complement of available comp_vecs. ULPs that invoke ib_alloc_cq() with only comp_vector 0 are converted to use the new API so that their completion workloads interfere less with each other. Suggested-by: HÃ¥kon Bugge <haakon.bugge@oracle.com> Signed-off-by: Chuck Lever <chuck.lever@oracle.com> Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Cc: <linux-cifs@vger.kernel.org> Cc: <v9fs-developer@lists.sourceforge.net> Link: https://lore.kernel.org/r/20190729171923.13428.52555.stgit@manet.1015granger.net Signed-off-by: Doug Ledford <dledford@redhat.com>
2019-05-21treewide: Add SPDX license identifier - Makefile/KconfigThomas Gleixner2-0/+2
Add SPDX license identifiers to all Make/Kconfig files which: - Have no license information of any form These files fall under the project license, GPL v2 only. The resulting SPDX license identifier is: GPL-2.0-only Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2019-03-09Merge tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsiLinus Torvalds2-49/+35
Pull SCSI updates from James Bottomley: "This is mostly update of the usual drivers: arcmsr, qla2xxx, lpfc, hisi_sas, target/iscsi and target/core. Additionally Christoph refactored gdth as part of the dma changes. The major mid-layer change this time is the removal of bidi commands and with them the whole of the osd/exofs driver and filesystem. This is a major simplification for block and mq in particular" * tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi: (240 commits) scsi: cxgb4i: validate tcp sequence number only if chip version <= T5 scsi: cxgb4i: get pf number from lldi->pf scsi: core: replace GFP_ATOMIC with GFP_KERNEL in scsi_scan.c scsi: mpt3sas: Add missing breaks in switch statements scsi: aacraid: Fix missing break in switch statement scsi: kill command serial number scsi: csiostor: drop serial_number usage scsi: mvumi: use request tag instead of serial_number scsi: dpt_i2o: remove serial number usage scsi: st: osst: Remove negative constant left-shifts scsi: ufs-bsg: Allow reading descriptors scsi: ufs: Allow reading descriptor via raw upiu scsi: ufs-bsg: Change the calling convention for write descriptor scsi: ufs: Remove unused device quirks Revert "scsi: ufs: disable vccq if it's not needed by UFS device" scsi: megaraid_sas: Remove a bunch of set but not used variables scsi: clean obsolete return values of eh_timed_out scsi: sd: Optimal I/O size should be a multiple of physical block size scsi: MAINTAINERS: SCSI initiator and target tweaks scsi: fcoe: make use of fip_mode enum complete ...
2019-02-04scsi: RDMA/srpt: Fix a credit leak for aborted commandsBart Van Assche1-0/+11
Make sure that the next time a response is sent to the initiator that the credit it had allocated for the aborted request gets freed. Cc: Doug Ledford <dledford@redhat.com> Cc: Jason Gunthorpe <jgg@ziepe.ca> Cc: Nicholas Bellinger <nab@linux-iscsi.org> Cc: Mike Christie <mchristi@redhat.com> Cc: Hannes Reinecke <hare@suse.com> Cc: Christoph Hellwig <hch@lst.de> Fixes: 131e6abc674e ("target: Add TFO->abort_task for aborted task resources release") # v3.15 Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2019-02-04scsi: RDMA/srpt: Rework I/O context allocationBart Van Assche2-32/+19
Instead of maintaining a list of free I/O contexts, use an sbitmap data structure to track which I/O contexts are in use and which are free. This makes the ib_srpt driver more consistent with other LIO drivers. Cc: Doug Ledford <dledford@redhat.com> Cc: Jason Gunthorpe <jgg@ziepe.ca> Cc: Nicholas Bellinger <nab@linux-iscsi.org> Cc: Mike Christie <mchristi@redhat.com> Cc: Hannes Reinecke <hare@suse.com> Cc: Christoph Hellwig <hch@lst.de> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2019-02-04scsi: RDMA/srpt: Fix handling of TMF submission failureBart Van Assche1-3/+1
If submitting a TMF to the target core fails, send the "FUNCTION REJECTED" response to the initiator. Reviewed-by: Hannes Reinecke <hare@suse.com> Cc: Doug Ledford <dledford@redhat.com> Cc: Jason Gunthorpe <jgg@ziepe.ca> Cc: Nicholas Bellinger <nab@linux-iscsi.org> Cc: Mike Christie <mchristi@redhat.com> Cc: Christoph Hellwig <hch@lst.de> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2019-02-04scsi: RDMA/srpt: Fix handling of command / TMF submission failureBart Van Assche1-5/+4
If submitting an SRP IU to the target core fails, send the SCSI response "BUSY" to the initiator instead of not sending any response. Reviewed-by: Hannes Reinecke <hare@suse.com> Cc: Doug Ledford <dledford@redhat.com> Cc: Jason Gunthorpe <jgg@ziepe.ca> Cc: Nicholas Bellinger <nab@linux-iscsi.org> Cc: Mike Christie <mchristi@redhat.com> Cc: Christoph Hellwig <hch@lst.de> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2019-02-04scsi: target/core: Remove the write_pending_status() callback functionBart Van Assche1-9/+0
Due to the patch that makes TMF handling synchronous the write_pending_status() callback function is no longer called. Hence remove it. Acked-by: Felipe Balbi <balbi@ti.com> Reviewed-by: Sagi Grimberg <sagig@mellanox.com> Reviewed-by: Andy Grover <agrover@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Hannes Reinecke <hare@suse.com> Reviewed-by: Bryant G. Ly <bryantly@linux.vnet.ibm.com> Cc: Nicholas Bellinger <nab@linux-iscsi.org> Cc: Mike Christie <mchristi@redhat.com> Cc: Himanshu Madhani <himanshu.madhani@qlogic.com> Cc: Quinn Tran <quinn.tran@qlogic.com> Cc: Saurav Kashyap <saurav.kashyap@qlogic.com> Cc: Michael S. Tsirkin <mst@redhat.com> Cc: Juergen Gross <jgross@suse.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2019-01-25infiniband: remove unneeded header search pathsMasahiro Yamada1-1/+0
The included headers are located in include/target/. I was able to build these drivers without the extra header search paths. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2018-12-28Merge tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdmaLinus Torvalds2-93/+234
Pull rdma updates from Jason Gunthorpe: "This has been a fairly typical cycle, with the usual sorts of driver updates. Several series continue to come through which improve and modernize various parts of the core code, and we finally are starting to get the uAPI command interface cleaned up. - Various driver fixes for bnxt_re, cxgb3/4, hfi1, hns, i40iw, mlx4, mlx5, qib, rxe, usnic - Rework the entire syscall flow for uverbs to be able to run over ioctl(). Finally getting past the historic bad choice to use write() for command execution - More functional coverage with the mlx5 'devx' user API - Start of the HFI1 series for 'TID RDMA' - SRQ support in the hns driver - Support for new IBTA defined 2x lane widths - A big series to consolidate all the driver function pointers into a big struct and have drivers provide a 'static const' version of the struct instead of open coding initialization - New 'advise_mr' uAPI to control device caching/loading of page tables - Support for inline data in SRPT - Modernize how umad uses the driver core and creates cdev's and sysfs files - First steps toward removing 'uobject' from the view of the drivers" * tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma: (193 commits) RDMA/srpt: Use kmem_cache_free() instead of kfree() RDMA/mlx5: Signedness bug in UVERBS_HANDLER() IB/uverbs: Signedness bug in UVERBS_HANDLER() IB/mlx5: Allocate the per-port Q counter shared when DEVX is supported IB/umad: Start using dev_groups of class IB/umad: Use class_groups and let core create class file IB/umad: Refactor code to use cdev_device_add() IB/umad: Avoid destroying device while it is accessed IB/umad: Simplify and avoid dynamic allocation of class IB/mlx5: Fix wrong error unwind IB/mlx4: Remove set but not used variable 'pd' RDMA/iwcm: Don't copy past the end of dev_name() string IB/mlx5: Fix long EEH recover time with NVMe offloads IB/mlx5: Simplify netdev unbinding IB/core: Move query port to ioctl RDMA/nldev: Expose port_cap_flags2 IB/core: uverbs copy to struct or zero helper IB/rxe: Reuse code which sets port state IB/rxe: Make counters thread safe IB/mlx5: Use the correct commands for UMEM and UCTX allocation ...
2018-12-22RDMA/srpt: Use kmem_cache_free() instead of kfree()Wei Yongjun1-1/+1
memory allocated by kmem_cache_alloc() should be freed using kmem_cache_free(), not kfree(). Fixes: 5dabcd0456d7 ("RDMA/srpt: Add support for immediate data") Signed-off-by: Wei Yongjun <weiyongjun1@huawei.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2018-12-19RDMA: Mark if destroy address handle is in a sleepable contextGal Pressman1-2/+2
Introduce a 'flags' field to destroy address handle callback and add a flag that marks whether the callback is executed in an atomic context or not. This will allow drivers to wait for completion instead of polling for it when it is allowed. Signed-off-by: Gal Pressman <galpress@amazon.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2018-12-19RDMA/srpt: Add support for immediate dataBart Van Assche2-49/+189
Modify allocation of the non-SRQ receive queues such that immediate data is aligned on a 512 byte boundary. That alignment is necessary to pass the immediate data without copying to the block layer. When receiving an SRP_CMD with immediate data, postpone the ib_post_recv() call until target_execute_cmd() has finished. See also srpt_release_cmd(). Cc: Sergey Gorenko <sergeygo@mellanox.com> Cc: Max Gurtovoy <maxg@mellanox.com> Cc: Laurence Oberman <loberman@redhat.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Doug Ledford <dledford@redhat.com>
2018-12-19RDMA/srpt: Rework the srpt_alloc_srq() error pathBart Van Assche1-4/+6
This patch does not change any functionality but makes the next patch easier to read. Cc: Sergey Gorenko <sergeygo@mellanox.com> Cc: Max Gurtovoy <maxg@mellanox.com> Cc: Laurence Oberman <loberman@redhat.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Doug Ledford <dledford@redhat.com>
2018-12-19RDMA/srpt: Remove driver version and release dateBart Van Assche1-5/+2
Neither a driver version number nor a release data is useful in an upstream driver. Remove the word "InfiniBand" from the driver description because recently RoCE support has been added to this driver. Cc: Sergey Gorenko <sergeygo@mellanox.com> Cc: Max Gurtovoy <maxg@mellanox.com> Cc: Laurence Oberman <loberman@redhat.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Doug Ledford <dledford@redhat.com>
2018-12-19RDMA/srpt: Make kernel-doc headers completeBart Van Assche1-1/+4
Add documentation for those structure members for which it is missing. Cc: Sergey Gorenko <sergeygo@mellanox.com> Cc: Max Gurtovoy <maxg@mellanox.com> Cc: Laurence Oberman <loberman@redhat.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Doug Ledford <dledford@redhat.com>
2018-12-19RDMA/srpt: Join split stringsBart Van Assche1-17/+12
Make sure that long strings occur on a single line as required by the coding standard. Cc: Sergey Gorenko <sergeygo@mellanox.com> Cc: Max Gurtovoy <maxg@mellanox.com> Cc: Laurence Oberman <loberman@redhat.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Doug Ledford <dledford@redhat.com>
2018-12-19RDMA/srpt: Improve coding style conformanceBart Van Assche2-6/+6
Use tabs instead of spaces for indentation. Make sure that multi-line expressions have the operator at the end of a line instead of the start. Avoid a complaint about a missing space in a ternary expression by changing '(boolean) ? 1: 0' into 'boolean'. Cc: Sergey Gorenko <sergeygo@mellanox.com> Cc: Max Gurtovoy <maxg@mellanox.com> Cc: Laurence Oberman <loberman@redhat.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Doug Ledford <dledford@redhat.com>
2018-12-19RDMA/srpt: Fix a use-after-free in the channel release codeBart Van Assche1-5/+13
This patch avoids that KASAN sporadically reports the following: BUG: KASAN: use-after-free in rxe_run_task+0x1e/0x60 [rdma_rxe] Read of size 1 at addr ffff88801c50d8f4 by task check/24830 CPU: 4 PID: 24830 Comm: check Not tainted 4.20.0-rc6-dbg+ #3 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014 Call Trace: dump_stack+0x86/0xca print_address_description+0x71/0x239 kasan_report.cold.5+0x242/0x301 __asan_load1+0x47/0x50 rxe_run_task+0x1e/0x60 [rdma_rxe] rxe_post_send+0x4bd/0x8d0 [rdma_rxe] srpt_zerolength_write+0xe1/0x160 [ib_srpt] srpt_close_ch+0x8b/0xe0 [ib_srpt] srpt_set_enabled+0xe7/0x150 [ib_srpt] srpt_tpg_enable_store+0xc0/0x100 [ib_srpt] configfs_write_file+0x157/0x1d0 __vfs_write+0xd7/0x3d0 vfs_write+0x102/0x290 ksys_write+0xab/0x130 __x64_sys_write+0x43/0x50 do_syscall_64+0x71/0x210 entry_SYSCALL_64_after_hwframe+0x49/0xbe Allocated by task 13856: save_stack+0x43/0xd0 kasan_kmalloc+0xc7/0xe0 kasan_slab_alloc+0x11/0x20 kmem_cache_alloc+0x105/0x320 rxe_alloc+0xff/0x1f0 [rdma_rxe] rxe_create_qp+0x9f/0x160 [rdma_rxe] ib_create_qp+0xf5/0x690 [ib_core] rdma_create_qp+0x6a/0x140 [rdma_cm] srpt_cm_req_recv.cold.59+0x1588/0x237b [ib_srpt] srpt_rdma_cm_req_recv.isra.35+0x1d5/0x220 [ib_srpt] srpt_rdma_cm_handler+0x6f/0x100 [ib_srpt] cma_listen_handler+0x59/0x60 [rdma_cm] cma_ib_req_handler+0xd5b/0x2570 [rdma_cm] cm_process_work+0x2e/0x110 [ib_cm] cm_work_handler+0x2aae/0x502b [ib_cm] process_one_work+0x481/0x9e0 worker_thread+0x67/0x5b0 kthread+0x1cf/0x1f0 ret_from_fork+0x24/0x30 Freed by task 3440: save_stack+0x43/0xd0 __kasan_slab_free+0x139/0x190 kasan_slab_free+0xe/0x10 kmem_cache_free+0xbc/0x330 rxe_elem_release+0x66/0xe0 [rdma_rxe] rxe_destroy_qp+0x3f/0x50 [rdma_rxe] ib_destroy_qp+0x140/0x360 [ib_core] srpt_release_channel_work+0xdc/0x310 [ib_srpt] process_one_work+0x481/0x9e0 worker_thread+0x67/0x5b0 kthread+0x1cf/0x1f0 ret_from_fork+0x24/0x30 Cc: Sergey Gorenko <sergeygo@mellanox.com> Cc: Max Gurtovoy <maxg@mellanox.com> Cc: Laurence Oberman <loberman@redhat.com> Cc: <stable@vger.kernel.org> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Doug Ledford <dledford@redhat.com>
2018-12-19include/scsi/srp.h: Move response flag definitions into this fileBart Van Assche1-4/+0
This patch moves all constants that come from the SRP standard into the include/scsi/srp.h header file. Cc: Sergey Gorenko <sergeygo@mellanox.com> Cc: Max Gurtovoy <maxg@mellanox.com> Cc: Laurence Oberman <loberman@redhat.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Doug Ledford <dledford@redhat.com>
2018-11-28scsi: target: replace fabric_ops.name with fabric_aliasDavid Disseldorp1-1/+0
iscsi_target_mod is the only LIO fabric where fabric_ops.name differs from the fabric_ops.fabric_name string. fabric_ops.name is used when matching target/$fabric ConfigFS create paths, so rename it .fabric_alias and fallback to target/$fabric vs .fabric_name comparison if .fabric_alias isn't initialised. iscsi_target_mod is the only fabric module to set .fabric_alias . All other fabric modules rely on .fabric_name matching and can drop the duplicate string. Signed-off-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-11-28scsi: target: drop unnecessary get_fabric_name() accessor from fabric_opsDavid Disseldorp1-6/+1
All fabrics return a const string. In all cases *except* iSCSI the get_fabric_name() string matches fabric_ops.name. Both fabric_ops.get_fabric_name() and fabric_ops.name are user-facing, with the former being used for PR/ALUA state and the latter for ConfigFS (config/target/$name), so we unfortunately need to keep both strings around for now. Replace the useless .get_fabric_name() accessor function with a const string fabric_name member variable. Signed-off-by: David Disseldorp <ddiss@suse.de> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-11-21IB/srpt: Drop pointless static qualifier in srpt_make_tpg()Yue Haibing1-1/+1
There is no need to have the 'struct se_portal_group *tpg' variable static since new value always be assigned before use. Signed-off-by: Yue Haibing <yuehaibing@huawei.com> Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Reviewed-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2018-09-26RDMA/ulp: Use dev_name instead of ibdev->nameJason Gunthorpe1-12/+14
These return the same thing but dev_name is a more conventional use of the kernel API. Signed-off-by: Jason Gunthorpe <jgg@mellanox.com> Reviewed-by: Steve Wise <swise@opengridcomputing.com> Reviewed-by: Sagi Grimberg <sagi@grimberg.me> Reviewed-by: Dennis Dalessandro <dennis.dalessandro@intel.com>
2018-09-05IB/srp: Remove unnecessary unlikely()Igor Stoppa1-1/+1
WARN_ON() already contains an unlikely(), so it's not necessary to wrap it into another. Signed-off-by: Igor Stoppa <igor.stoppa@huawei.com> Reviewed-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2018-08-16Merge branch 'linus/master' into rdma.git for-nextJason Gunthorpe1-7/+4
rdma.git merge resolution for the 4.19 merge window Conflicts: drivers/infiniband/core/rdma_core.c - Use the rdma code and revise with the new spelling for atomic_fetch_add_unless drivers/nvme/host/rdma.c - Replace max_sge with max_send_sge in new blk code drivers/nvme/target/rdma.c - Use the blk code and revise to use NULL for ib_post_recv when appropriate - Replace max_sge with max_recv_sge in new blk code net/rds/ib_send.c - Use the net code and revise to use NULL for ib_post_recv when appropriate Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2018-08-02scsi: target: srp, vscsi, sbp, qla: use target_remove_sessionMike Christie1-2/+1
This converts the drivers that called transport_deregister_session_configfs and then immediately called transport_deregister_session to use target_remove_session. Signed-off-by: Mike Christie <mchristi@redhat.com> Reviewed-by: Bart Van Assche <bart.vanassche@wdc.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Cc: Chris Boot <bootc@bootc.net> Cc: Bryant G. Ly <bryantly@linux.vnet.ibm.com> Cc: Michael Cyr <mikecyr@linux.vnet.ibm.com> Cc: <qla2xxx-upstream@qlogic.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-08-02scsi: target: rename target_alloc_sessionMike Christie1-3/+3
Rename target_alloc_session to target_setup_session to avoid confusion with the other transport session allocation function that only allocates the session and because the target_alloc_session does so much more. It allocates the session, sets up the nacl and registers the session. The next patch will then add a remove function to match the setup in this one, so it should make sense for all drivers, except iscsi, to just call those 2 functions to setup and remove a session. iscsi will continue to be the odd driver. Signed-off-by: Mike Christie <mchristi@redhat.com> Reviewed-by: Bart Van Assche <bart.vanassche@wdc.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Cc: Chris Boot <bootc@bootc.net> Cc: Bryant G. Ly <bryantly@linux.vnet.ibm.com> Cc: Michael Cyr <mikecyr@linux.vnet.ibm.com> Cc: <qla2xxx-upstream@qlogic.com> Cc: Johannes Thumshirn <jth@kernel.org> Cc: Felipe Balbi <balbi@kernel.org> Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Cc: Andrzej Pietrasiewicz <andrzej.p@samsung.com> Cc: Michael S. Tsirkin <mst@redhat.com> Cc: Juergen Gross <jgross@suse.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-07-30RDMA/cma: Constify path record, ib_cm_event, listen_id pointersParav Pandit1-2/+3
Constify several pointers such as path_rec, ib_cm_event and listen_id pointers in several functions. Signed-off-by: Parav Pandit <parav@mellanox.com> Reviewed-by: Daniel Jurgens <danielj@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2018-07-24IB/srpt: Simplify ib_post_(send|recv|srq_recv)() callsBart Van Assche1-9/+8
Instead of declaring and passing a dummy 'bad_wr' pointer, pass NULL as third argument to ib_post_(send|recv|srq_recv)(). Signed-off-by: Bart Van Assche <bart.vanassche@wdc.com> Reviewed-by: Max Gurtovoy <maxg@mellanox.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2018-07-13IB/srpt: Fix srpt_cm_req_recv() error path (2/2)Bart Van Assche1-1/+4
If a login request was received through the RDMA/CM and if an error occurs during login, clear rdma_cm_id->context instead of ib_cm_id->context. Fixes: 63cf1a902c9d ("IB/srpt: Add RDMA/CM support") Signed-off-by: Bart Van Assche <bart.vanassche@wdc.com> Cc: <stable@vger.kernel.org> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2018-07-13IB/srpt: Fix srpt_cm_req_recv() error path (1/2)Bart Van Assche1-3/+14
Once a target session has been allocated, if an error occurs, the session must be freed. Since it is not safe to call blocking code from the context of an connection manager callback, trigger target session release in this case by calling srpt_close_ch(). Fixes: db7683d7deb2 ("IB/srpt: Fix login-related race conditions") Signed-off-by: Bart Van Assche <bart.vanassche@wdc.com> Cc: <stable@vger.kernel.org> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2018-07-10ib_srpt: use kvmalloc to allocate ring pointersJan Dakinevich1-3/+3
An array of pointers to SRPT contexts in ib_device is over 30KiB even in default case, in which an amount of contexts is 4095. The patch is intended to weed out large contigous allocation for non-DMA memory. Signed-off-by: Jan Dakinevich <jan.dakinevich@virtuozzo.com> Reviewed-by: Bart Van Assche <bart.vanassche@wdc.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2018-07-03ib_srpt: Fix a use-after-free in __srpt_close_all_ch()Bart Van Assche1-2/+2
BUG: KASAN: use-after-free in srpt_set_enabled+0x1a9/0x1e0 [ib_srpt] Read of size 4 at addr ffff8801269d23f8 by task check/29726 CPU: 4 PID: 29726 Comm: check Not tainted 4.18.0-rc2-dbg+ #4 Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.0.0-prebuilt.qemu-project.org 04/01/2014 Call Trace: dump_stack+0xa4/0xf5 print_address_description+0x6f/0x270 kasan_report+0x241/0x360 __asan_load4+0x78/0x80 srpt_set_enabled+0x1a9/0x1e0 [ib_srpt] srpt_tpg_enable_store+0xb8/0x120 [ib_srpt] configfs_write_file+0x14e/0x1d0 [configfs] __vfs_write+0xd2/0x3b0 vfs_write+0x101/0x270 ksys_write+0xab/0x120 __x64_sys_write+0x43/0x50 do_syscall_64+0x77/0x230 entry_SYSCALL_64_after_hwframe+0x49/0xbe RIP: 0033:0x7f235cfe6154 Fixes: aaf45bd83eba ("IB/srpt: Detect session shutdown reliably") Signed-off-by: Bart Van Assche <bart.vanassche@wdc.com> Cc: <stable@vger.kernel.org> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2018-07-03ib_srpt: Fix a use-after-free in srpt_close_ch()Bart Van Assche1-2/+1
Avoid that KASAN reports the following: BUG: KASAN: use-after-free in srpt_close_ch+0x4f/0x1b0 [ib_srpt] Read of size 4 at addr ffff880151180cb8 by task check/4681 CPU: 15 PID: 4681 Comm: check Not tainted 4.18.0-rc2-dbg+ #4 Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.0.0-prebuilt.qemu-project.org 04/01/2014 Call Trace: dump_stack+0xa4/0xf5 print_address_description+0x6f/0x270 kasan_report+0x241/0x360 __asan_load4+0x78/0x80 srpt_close_ch+0x4f/0x1b0 [ib_srpt] srpt_set_enabled+0xf7/0x1e0 [ib_srpt] srpt_tpg_enable_store+0xb8/0x120 [ib_srpt] configfs_write_file+0x14e/0x1d0 [configfs] __vfs_write+0xd2/0x3b0 vfs_write+0x101/0x270 ksys_write+0xab/0x120 __x64_sys_write+0x43/0x50 do_syscall_64+0x77/0x230 entry_SYSCALL_64_after_hwframe+0x49/0xbe Fixes: aaf45bd83eba ("IB/srpt: Detect session shutdown reliably") Signed-off-by: Bart Van Assche <bart.vanassche@wdc.com> Cc: <stable@vger.kernel.org> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2018-07-02scsi: target: Remove second argument from fabric_make_tpg()Bart Van Assche1-2/+0
Since most target drivers do not use the second fabric_make_tpg() argument ("group") and since it is trivial to derive the group pointer from the wwn pointer, do not pass the group pointer to fabric_make_tpg(). Signed-off-by: Bart Van Assche <bart.vanassche@wdc.com> Reviewed-by: Mike Christie <mchristi@redhat.com> Cc: Felipe Balbi <felipe.balbi@linux.intel.com> Cc: Hannes Reinecke <hare@suse.com> Cc: Christoph Hellwig <hch@lst.de> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-06-29IB/srpt: Support HCAs with more than two portsBart Van Assche2-5/+4
Since there are adapters that have four ports, increase the size of the srpt_device.port[] array. This patch avoids that the following warning is hit with quad port Chelsio adapters: WARN_ON(sdev->device->phys_port_cnt > ARRAY_SIZE(sdev->port)); Reported-by: Steve Wise <swise@opengridcomputing.com> Signed-off-by: Bart Van Assche <bart.vanassche@wdc.com> Cc: Steve Wise <swise@opengridcomputing.com> Cc: Christoph Hellwig <hch@infradead.org> Cc: <stable@vger.kernel.org> Reviewed-by: Steve Wise <swise@opengridcomputing.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2018-06-18IB/core: add max_send_sge and max_recv_sge attributesSteve Wise1-2/+4
This patch replaces the ib_device_attr.max_sge with max_send_sge and max_recv_sge. It allows ulps to take advantage of devices that have very different send and recv sge depths. For example cxgb4 has a max_recv_sge of 4, yet a max_send_sge of 16. Splitting out these attributes allows much more efficient use of the SQ for cxgb4 with ulps that use the RDMA_RW API. Consider a large RDMA WRITE that has 16 scattergather entries. With max_sge of 4, the ulp would send 4 WRITE WRs, but with max_sge of 16, it can be done with 1 WRITE WR. Acked-by: Sagi Grimberg <sagi@grimberg.me> Acked-by: Christoph Hellwig <hch@lst.de> Acked-by: Selvin Xavier <selvin.xavier@broadcom.com> Acked-by: Shiraz Saleem <shiraz.saleem@intel.com> Acked-by: Dennis Dalessandro <dennis.dalessandro@intel.com> Signed-off-by: Steve Wise <swise@opengridcomputing.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2018-06-18IB: Replace ib_query_gid/ib_get_cached_gid with rdma_query_gidParav Pandit1-2/+1
If the gid_attr argument is NULL then the functions behave identically to rdma_query_gid. ib_query_gid just calls ib_get_cached_gid, so everything can be consolidated to one function. Now that all callers either use rdma_query_gid() or ib_get_cached_gid(), ib_query_gid() API is removed. Signed-off-by: Parav Pandit <parav@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>