aboutsummaryrefslogtreecommitdiffstatshomepage
path: root/drivers/infiniband/hw/efa
AgeCommit message (Collapse)AuthorFilesLines
2026-09-10RDMA/efa: Keep EQ resources alive while IRQ is registeredLeon Romanovsky3-10/+12
The completion IRQ handler accesses the EQ state and DMA buffer. Its IRQ was registered before that state was initialized, while teardown released the buffer before free_irq() synchronized the handler. Initialize the EQ without arming it, register the IRQ, and then arm it. Reverse the resource order during teardown by freeing the IRQ before destroying the EQ. Fixes: 2a152512a155 ("RDMA/efa: CQ notifications") Link: https://patch.msgid.link/20260907-use-after-free-of-admin-queue-struct-v1-2-dd9d9267fbf4@nvidia.com Reviewed-by: Michael Margolin <mrgolin@amazon.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
2026-09-10RDMA/efa: Keep admin queues alive while IRQ is registeredLeon Romanovsky2-9/+10
The management IRQ handler accesses both the admin completion queue and the async event queue. The driver registered the IRQ before constructing these queues and destroyed them before freeing the IRQ, so the handler's lifetime was not contained by the resources it accesses. Initialize the queues with interrupts masked, request the IRQ, and then switch to interrupt mode. On removal, reset the device and free the IRQ before destroying the queues. Also reset the device before destroying the queues if IRQ registration fails, because the device already has their DMA addresses. Fixes: b7f5e880f377 ("RDMA/efa: Add the efa module") Link: https://patch.msgid.link/20260907-use-after-free-of-admin-queue-struct-v1-1-dd9d9267fbf4@nvidia.com Reviewed-by: Michael Margolin <mrgolin@amazon.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
2026-08-12RDMA/efa: Add support for 128B admin v2 SQ entryYonatan Nachum3-32/+116
Add the new admin v2 format which is 128B in size and its header extends the v1 header with checksum and payload version. On admin SQ init check if the API version reported by the device supports the admin v2 SQ entry and if so use it. Store the payload offset and max size in the SQ for quick access in admin command execution flow. Using the admin SQ v2 entry implicitly enable the checksum in its header so set it for device to validate against. Link: https://patch.msgid.link/r/20260812121718.2904349-4-ynachum@amazon.com Reviewed-by: Michael Margolin <mrgolin@amazon.com> Reviewed-by: Tom Sela <tomsela@amazon.com> Signed-off-by: Yonatan Nachum <ynachum@amazon.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2026-08-12RDMA/efa: Generalize the admin SQYonatan Nachum2-27/+24
As preparation for admin v2 entry size which is 128B, generalize the SQ ring to use a generic buffer and use the right offset into it using the configured entry size. This will allow us to choose different entry size on SQ init with minimal changes. Link: https://patch.msgid.link/r/20260812121718.2904349-3-ynachum@amazon.com Reviewed-by: Michael Margolin <mrgolin@amazon.com> Signed-off-by: Yonatan Nachum <ynachum@amazon.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2026-08-12RDMA/efa: Decouple admin command payload from admin headerYonatan Nachum5-242/+112
Remove aq_common_desc from all EFA admin command descriptors so that command structs represent pure payloads. This allows attaching different headers to the same command by copying the payload into the generic header payload field. The admin header is now constructed in a single place which improves separability. Link: https://patch.msgid.link/r/20260812121718.2904349-2-ynachum@amazon.com Reviewed-by: Daniel Kranzdorf <dkkranzd@amazon.com> Reviewed-by: Michael Margolin <mrgolin@amazon.com> Signed-off-by: Yonatan Nachum <ynachum@amazon.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2026-07-28RDMA/efa: Fix PBL chunk length computationYonatan Nachum1-4/+9
On register MR, when creating the PBL, if it's an indirect PBL we create a chunk list to hold the PBL pages pointers. Each chunk is 4KB in size and can hold 510 addresses (EFA_PTRS_PER_CHUNK) and has a 12-byte control buffer at the end of it holding the next chunk's pointer and its length. If the PBL number of pages is a multiple of EFA_PTRS_PER_CHUNK, the calculated last chunk length is wrongly computed as 0, even though that chunk is fully populated with 510 real page pointers. This wrong length is used both to DMA map the chunk and is propagated to the device, causing the device to see the chunk as empty and reject the memory registration. Fix the calculation so it will be performed only if the number of pages isn't a multiple of EFA_PTRS_PER_CHUNK, if it is, its already handled in the above loop correctly. Also prevent out-of-bounds reach in the chunks array in such scenario. Fixes: 40909f664d27 ("RDMA/efa: Add EFA verbs implementation") Reviewed-by: Firas Jahjah <firasj@amazon.com> Reviewed-by: Michael Margolin <mrgolin@amazon.com> Signed-off-by: Yonatan Nachum <ynachum@amazon.com> Link: https://patch.msgid.link/20260727090255.1175120-1-ynachum@amazon.com Signed-off-by: Leon Romanovsky <leon@kernel.org>
2026-07-23RDMA/efa: Expose 64-bit send WR ID support to userspaceYonatan Nachum1-0/+3
Currently EFA WRs support 16-bit request ID, this requires EFA to manage a translation table to translate the IB WR ID from 64-bits to 16-bits and translating it back on CQ completion. Expose a new device capability to handle 64-bit request ID for SQ WRs allowing userspace to directly post the 64-bit ID to the device. Reviewed-by: Michael Margolin <mrgolin@amazon.com> Reviewed-by: Tom Sela <tomsela@amazon.com> Signed-off-by: Yonatan Nachum <ynachum@amazon.com> Link: https://patch.msgid.link/20260722113331.2515247-3-ynachum@amazon.com Signed-off-by: Leon Romanovsky <leon@kernel.org>
2026-07-23RDMA/efa: Add CQ/QP creation with 64-bit SQ req ID supportYonatan Nachum5-5/+44
Add the support needed to propagate the user requested flags to config the CQ/QP to support 64-bit SQ request ID to the device. Reviewed-by: Michael Margolin <mrgolin@amazon.com> Reviewed-by: Tom Sela <tomsela@amazon.com> Signed-off-by: Yonatan Nachum <ynachum@amazon.com> Link: https://patch.msgid.link/20260722113331.2515247-2-ynachum@amazon.com Signed-off-by: Leon Romanovsky <leon@kernel.org>
2026-07-22RDMA/efa: Add Completion Counters supportMichael Margolin5-2/+444
Implement completion counters for the EFA device. Each completion counter is backed by two EFA event counters, one for success completions and one for error completions. The driver creates umem for counters from private descriptor ioctl attributes using core utility. Read operations are not implemented as the counter values are accessed directly from userspace through the mapped memory. Reviewed-by: Yonatan Nachum <ynachum@amazon.com> Signed-off-by: Michael Margolin <mrgolin@amazon.com> Link: https://patch.msgid.link/20260722083603.30334-7-mrgolin@amazon.com Signed-off-by: Leon Romanovsky <leon@kernel.org>
2026-07-22RDMA/efa: Update device interfaceMichael Margolin2-4/+202
Align device interface definitions. Reviewed-by: Daniel Kinsbursky <dkinsb@amazon.com> Reviewed-by: Yonatan Nachum <ynachum@amazon.com> Signed-off-by: Michael Margolin <mrgolin@amazon.com> Link: https://patch.msgid.link/20260722083603.30334-6-mrgolin@amazon.com Signed-off-by: Leon Romanovsky <leon@kernel.org>
2026-07-14RDMA: Remove redundant memset() from query_device callbacksLeon Romanovsky1-1/+0
The core always hands the driver's query_device() callback a zeroed ib_device_attr. There are only two callers of the op and both clear the structure before invoking it: setup_device() memsets &device->attrs, and ib_uverbs_ex_query_device() passes an on-stack structure initialized to {}. The open-coded memset(props, 0, sizeof(*props)) at the top of the driver callbacks is therefore redundant. Remove it from all drivers. Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
2026-07-14RDMA/efa: Add EFA 0xefa4 PCI IDAnas Mousa1-0/+2
Add support for 0xefa4 devices. Reviewed-by: Michael Margolin <mrgolin@amazon.com> Signed-off-by: Anas Mousa <anasmous@amazon.com> Link: https://patch.msgid.link/20260712134413.19226-3-mrgolin@amazon.com Signed-off-by: Leon Romanovsky <leon@kernel.org>
2026-07-14RDMA/efa: Extend page-shift field in MR registrationMichael Margolin1-3/+3
Update device interface adding one more bit from reserved to enable >4GB page sizes that can be supported on 0xefa4 devices. Reviewed-by: Yonatan Nachum <ynachum@amazon.com> Signed-off-by: Michael Margolin <mrgolin@amazon.com> Link: https://patch.msgid.link/20260712134413.19226-2-mrgolin@amazon.com Signed-off-by: Leon Romanovsky <leon@kernel.org>
2026-07-12RDMA/efa: Add AH cache handling on create and destroy AHYonatan Nachum5-8/+140
On create AH, first check if the AH cache entry already exists and if so, returns the already stored AH number. If the entry doesn't exist, the driver creates it and calls the device to create the AH. A per-entry mutex serializes concurrent device commands on the same AH cache entry, ensuring only one thread issues the device create while others wait and reuse the result. If the device create fails, the entry's user count remains zero so subsequent threads will retry the device create. On destroy AH, the user count is decremented under the entry mutex. If it reaches zero, the driver issues the device destroy command. After the device destroy completes, it removes the entry from the hashtable and frees it if no other references exist. If new users arrived during the destroy, the entry remains in the hashtable for reuse. Reviewed-by: Firas Jahjah <firasj@amazon.com> Reviewed-by: Michael Margolin <mrgolin@amazon.com> Signed-off-by: Yonatan Nachum <ynachum@amazon.com> Link: https://patch.msgid.link/20260706170008.1039417-3-ynachum@amazon.com Signed-off-by: Leon Romanovsky <leon@kernel.org>
2026-07-12RDMA/efa: Add initialization of AH cache rhashtableYonatan Nachum5-3/+93
New EFA devices don't support the creation of multiple address handles to the same remote on the same PD. To overcome this limitation, introduce an AH cache rhashtable which will store the user refcounts of the same AH creation on the same PD and will allow the driver to manage AH reuse. The hashtable key is the combination of PD and GID. Add initialization and teardown logic for the rhashtable. Each entry holds a refcount to manage the entry lifetime in the hashtable and a user count that indicates how many users are using the address handle. Reviewed-by: Firas Jahjah <firasj@amazon.com> Reviewed-by: Michael Margolin <mrgolin@amazon.com> Signed-off-by: Yonatan Nachum <ynachum@amazon.com> Link: https://patch.msgid.link/20260706170008.1039417-2-ynachum@amazon.com Signed-off-by: Leon Romanovsky <leon@kernel.org>
2026-06-18Merge tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdmaLinus Torvalds9-97/+121
Pull rdma updates from Jason Gunthorpe: "Many AI driven bug fixes, and several big driver API cleanups - Driver bug fixes and minor cleanups in mlx5, hns, rxe, efa, siw, rtrs, mana, irdma, mlx4. Commonly error path flows, integer arithmetic overflows on unsafe data, out of bounds access, and use after free issues under races. - Second half of the new udata API for drivers focusing on uAPI response - bnxt_re supports more options for QP creation that will allow a dv path in rdma-core - Untangle the module dependencies so drivers don't link to ib_uverbs.ko as was originall intended - Provide a new way to handle umems with a consistent simplified uAPI and update several drivers to use it. This brings dmabuf support to more places and more drivers - Support for mlx5 rate limit and packet pacing for UD and UC - A batch of fixes for the new shared FRMR pools infrastructure" * tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma: (148 commits) RDMA/irdma: Replace waitqueue and flag with completion RDMA/hns: Fix memory leak of bonding resources RDMA/rtrs-srv: Bound RDMA-Write length to chunk size in rdma_write_sg docs: infiniband: correct name of option to enable the ib_uverbs module RDMA/bnxt_re: Reject GET_TOGGLE_MEM when toggle page was not allocated RDMA/bnxt_re: Fail DBR related page allocation UAPIs if the feature is disabled RDMA/bnxt_re: Avoid repeated requests to allocate WC pages RDMA/bnxt_re: Proper rollback if the ioremap fails RDMA/bnxt_re: Add a max slot check for SQ RDMA/bnxt_re: Avoid displaying the kernel pointer RDMA/bnxt_re: Free CQ toggle page after firmware teardown RDMA/bnxt_re: Free SRQ toggle page after firmware teardown RDMA/bnxt_re: Initialize dpi variable to zero ABI: sysfs-class-infiniband: minor cleanup RDMA/mlx5: Release the HW‑provided UAR index rather than the SW one RDMA/mlx5: Fix undefined shift of user RQ WQE size RDMA/mlx5: Remove raw RSS QP restrack tracking RDMA/mlx5: Remove DCT restrack tracking RDMA/mlx5: Drop FRMR pool handle on UMR revoke failure RDMA/core: Add ib_frmr_pool_drop for unrecoverable handles ...
2026-06-09RDMA/efa: Implement the query port speed verbTom Sela4-3/+16
Implement the query port speed callback to report the port effective bandwidth directly in 100 Mb/s granularity. Link: https://patch.msgid.link/r/20260608083927.4116-1-tomsela@amazon.com Reviewed-by: Michael Margolin <mrgolin@amazon.com> Reviewed-by: Yonatan Nachum <ynachum@amazon.com> Signed-off-by: Tom Sela <tomsela@amazon.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2026-06-09RDMA/efa: Report 800 and 1600 Gbps link speedTom Sela1-1/+7
Add support for reporting 800 Gbps as 8X NDR and 1600 Gbps as 8X XDR link speeds. Link: https://patch.msgid.link/r/20260608083736.48454-1-tomsela@amazon.com Reviewed-by: Michael Margolin <mrgolin@amazon.com> Reviewed-by: Yonatan Nachum <ynachum@amazon.com> Signed-off-by: Tom Sela <tomsela@amazon.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2026-06-02RDMA/efa: Validate SQ ring size against max LLQ sizeYonatan Nachum1-9/+18
Validate the SQ ring size against the device's max LLQ size. This ensures that when using 128-byte WQEs, userspace cannot exceed the queue limits. On create QP, userspace provides the SQ ring size (depth x WQE size) which is validated against the max LLQ size. Fixes: 40909f664d27 ("RDMA/efa: Add EFA verbs implementation") Link: https://patch.msgid.link/r/20260526081536.1203553-1-ynachum@amazon.com Reviewed-by: Michael Margolin <mrgolin@amazon.com> Signed-off-by: Yonatan Nachum <ynachum@amazon.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2026-05-29RDMA/efa: Use ib_umem_get_cq_buf() for user CQ bufferJiri Pirko1-10/+17
Pin the user CQ buffer with ib_umem_get_cq_buf() and take ownership of the umem in the driver. Fall back to the existing kernel-DMA path on NULL. Link: https://patch.msgid.link/r/20260529134312.2836341-8-jiri@resnulli.us Signed-off-by: Jiri Pirko <jiri@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2026-05-29RDMA/umem: Rename ib_umem_get() to ib_umem_get_va()Jiri Pirko1-1/+1
The new umem getter family being introduced in follow-up patches need a fitting name for the central all-source helper that resolves attributes, legacy fillers and a UHW VA fallback. Rename the existing VA-pinning helper ib_umem_get() to ib_umem_get_va() so the name is freed up. The new name is consistent with names of rest of the helpers that are about to be introduced. Link: https://patch.msgid.link/r/20260529134312.2836341-2-jiri@resnulli.us Signed-off-by: Jiri Pirko <jiri@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2026-05-18RDMA: Convert drivers using min to ib_respond_udata()Jason Gunthorpe1-33/+11
Convert the pattern: ib_copy_to_udata(udata, &resp, min(sizeof(resp), udata->outlen)); Using Coccinelle: @@ identifier resp; expression udata; @@ - ib_copy_to_udata(udata, &resp, min(sizeof(resp), udata->outlen)) + ib_respond_udata(udata, resp) @@ identifier resp; expression udata; @@ - ib_copy_to_udata(udata, &resp, min(udata->outlen, sizeof(resp))) + ib_respond_udata(udata, resp) Run another pass with AI to propagate the return code correctly and remove redundant prints. Signed-off-by: Jason Gunthorpe <jgg@nvidia.com> Signed-off-by: Leon Romanovsky <leon@kernel.org>
2026-05-18RDMA: Use ib_is_udata_in_empty() for places calling ib_is_udata_cleared()Jason Gunthorpe1-29/+14
Convert the pattern: if (udata->inlen && !ib_is_udata_cleared(udata, 0, udata->inlen)) Using Coccinelle: virtual patch virtual context virtual report @@ expression udata; @@ ( - udata->inlen && !ib_is_udata_cleared(udata, 0, udata->inlen) + !ib_is_udata_in_empty(udata) | - udata->inlen > 0 && !ib_is_udata_cleared(udata, 0, udata->inlen) + !ib_is_udata_in_empty(udata) ) @@ expression udata; @@ - udata && udata->inlen && !ib_is_udata_cleared(udata, 0, udata->inlen) + !ib_is_udata_in_empty(udata) These cases are already checking for zeroed data that the kernel does not understand. Run another pass with AI to propagate the return code correctly and remove redundant prints. Signed-off-by: Jason Gunthorpe <jgg@nvidia.com> Signed-off-by: Leon Romanovsky <leon@kernel.org>
2026-05-18RDMA/efa: Add checksum support for admin responsesYonatan Nachum5-20/+55
EFA devices added support for CRC16 checksum on admin responses and to expose it to the driver the API version increased to 0.2. Add a check for support on device init and if supported validate the checksum on each admin response the driver receives. If the checksum validation failed, drop the CQE. Add the CRC16 module to Kconfig to have the in-tree dependency. Reviewed-by: Firas Jahjah <firasj@amazon.com> Reviewed-by: Michael Margolin <mrgolin@amazon.com> Signed-off-by: Yonatan Nachum <ynachum@amazon.com> Link: https://patch.msgid.link/20260409074905.3126023-1-ynachum@amazon.com Signed-off-by: Leon Romanovsky <leon@kernel.org>
2026-03-31RDMA: Remove redundant = {} for udata req structsJason Gunthorpe1-2/+2
Now that all of the udata request structs are loaded with the helpers the callers should not pre-zero them. The helpers all guarantee that the entire struct is filled with something. Reviewed-by: Long Li <longli@microsoft.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
2026-03-31RDMA: Use ib_copy_validate_udata_in_cm() for zero comp_maskJason Gunthorpe1-4/+4
All of these cases require a 0 comp_mask. Consolidate these into using ib_copy_validate_udata_in_cm() and remove the open coded comp_mask test. Signed-off-by: Jason Gunthorpe <jgg@nvidia.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
2026-03-31RDMA: Consolidate patterns with offsetofend() to ib_copy_validate_udata_in()Jason Gunthorpe1-43/+4
Go treewide and consolidate all existing patterns using: * offsetofend() and variations * ib_is_udata_cleared() * ib_copy_from_udata() into a direct call to the new ib_copy_validate_udata_in(). Signed-off-by: Jason Gunthorpe <jgg@nvidia.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
2026-03-30Merge branch 'master' into rdma-nextLeon Romanovsky1-48/+40
Let's bring v7.0-rc6 to the -next branch, so we can merge the DMA attributes fix [1] without merge conflicts. [1] https://lore.kernel.org/all/20260323-umem-dma-attrs-v1-1-d6890f2e6a1e@nvidia.com Signed-off-by: Leon Romanovsky <leon@kernel.org> * master: (1688 commits) Linux 7.0-rc6 ...
2026-03-17RDMA/efa: Fix possible deadlockEthan Tidmore1-0/+1
In the error path for efa_com_alloc_comp_ctx() the semaphore assigned to &aq->avail_cmds is not released. Detected by Smatch: drivers/infiniband/hw/efa/efa_com.c:662 efa_com_cmd_exec() warn: inconsistent returns '&aq->avail_cmds' Add release for &aq->avail_cmds in efa_com_alloc_comp_ctx() error path. Fixes: ef3b06742c8a2 ("RDMA/efa: Fix use of completion ctx after free") Signed-off-by: Ethan Tidmore <ethantidmore06@gmail.com> Link: https://patch.msgid.link/20260314045730.1143862-1-ethantidmore06@gmail.com Signed-off-by: Leon Romanovsky <leon@kernel.org>
2026-03-17RDMA/efa: Rename alloc_ucontext comp_mask to supported_capsMichael Margolin1-8/+9
Following discussion [1], rename the comp_mask field in efa_ibv_alloc_ucontext_cmd to supported_caps to reflect its actual usage as a capabilities handshake mechanism rather than a standard comp_mask. Rename related constants and align function and macro names. [1] https://lore.kernel.org/linux-rdma/20260312120858.GH1448102@nvidia.com/ Signed-off-by: Michael Margolin <mrgolin@amazon.com> Link: https://patch.msgid.link/20260316180846.30273-1-mrgolin@amazon.com Signed-off-by: Leon Romanovsky <leon@kernel.org>
2026-03-08RDMA/efa: Fix use of completion ctx after freeYonatan Nachum1-48/+39
On admin queue completion handling, if the admin command completed with error we print data from the completion context. The issue is that we already freed the completion context in polling/interrupts handler which means we print data from context in an unknown state (it might be already used again). Change the admin submission flow so alloc/dealloc of the context will be symmetric and dealloc will be called after any potential use of the context. Fixes: 68fb9f3e312a ("RDMA/efa: Remove redundant NULL pointer check of CQE") Reviewed-by: Daniel Kranzdorf <dkkranzd@amazon.com> Reviewed-by: Michael Margolin <mrgolin@amazon.com> Signed-off-by: Yonatan Nachum <ynachum@amazon.com> Link: https://patch.msgid.link/20260308165350.18219-1-ynachum@amazon.com Signed-off-by: Leon Romanovsky <leon@kernel.org>
2026-03-05Add support for TLP emulationLeon Romanovsky1-1/+1
This series adds support for Transaction Layer Packet (TLP) emulation response gateway regions, enabling userspace device emulation software to write TLP responses directly to lower layers without kernel driver involvement. Currently, the mlx5 driver exposes VirtIO emulation access regions via the MLX5_IB_METHOD_VAR_OBJ_ALLOC ioctl. This series extends that ioctl to also support allocating TLP response gateway channels for PCI device emulation use cases. Signed-off-by: Leon Romanovsky <leon@kernel.org>
2026-02-25RDMA/efa: Remove check for zero CQE countLeon Romanovsky1-2/+2
Since ib_core now handles validation, the device driver no longer needs to verify that the CQE count is non‑zero. Link: https://patch.msgid.link/20260213-refactor-umem-v1-9-f3be85847922@nvidia.com Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
2026-02-25RDMA/core: Prepare create CQ path for API unificationLeon Romanovsky3-14/+5
Ensure that .create_cq_umem() and .create_cq() follow the same API contract, allowing drivers to be gradually migrated to the umem-aware CQ management flow. Link: https://patch.msgid.link/20260213-refactor-umem-v1-7-f3be85847922@nvidia.com Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
2026-02-25RDMA/efa: Rely on CPU address in create‑QPLeon Romanovsky1-2/+2
Align this code with other locations where efa_free_mapped() depends on the presence of a valid CPU address, which is guaranteed when qp->rq_size != 0. Link: https://patch.msgid.link/20260213-refactor-umem-v1-6-f3be85847922@nvidia.com Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
2026-02-25RDMA/core: Manage CQ umem in core codeLeon Romanovsky1-13/+11
In the current implementation, CQ umem is handled both by ib_core and the driver. ib_core sometimes creates and destroys it, while the driver also destroys it. Store the umem in struct ib_cq and ensure that only ib_core manages its lifetime, relying solely on its internal reference counter. Link: https://patch.msgid.link/20260213-refactor-umem-v1-5-f3be85847922@nvidia.com Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
2026-02-25RDMA: Move DMA block iterator logic into dedicated filesLeon Romanovsky1-1/+1
The DMA iterator logic was mixed into verbs and umem-specific code, forcing all users to include rdma/ib_umem.h. Move the block iterator logic into iter.c and rdma/iter.h so that rdma/ib_umem.h and rdma/ib_verbs.h can be separated in a follow-up patch. Link: https://patch.msgid.link/20260213-refactor-umem-v1-1-f3be85847922@nvidia.com Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
2026-02-25RDMA/efa: Use extended inline buff size for inline validationYonatan Nachum1-2/+2
On QP creation we validate the requested max inline size is supported by the device. Use the new extended max inline size instead of the old one to support actual max inline available. Reviewed-by: Michael Margolin <mrgolin@amazon.com> Signed-off-by: Yonatan Nachum <ynachum@amazon.com> Link: https://patch.msgid.link/20260217112304.36849-4-ynachum@amazon.com Signed-off-by: Leon Romanovsky <leon@kernel.org>
2026-02-25RDMA/efa: Expose new extended max inline buff sizeYonatan Nachum4-3/+33
Add new extended max inline query and report the new value to userspace. Reviewed-by: Firas Jahjah <firasj@amazon.com> Reviewed-by: Michael Margolin <mrgolin@amazon.com> Signed-off-by: Yonatan Nachum <ynachum@amazon.com> Link: https://patch.msgid.link/20260217112304.36849-3-ynachum@amazon.com Signed-off-by: Leon Romanovsky <leon@kernel.org>
2026-02-25RDMA/efa: Rename admin queue attributes struct name for extendabilityYonatan Nachum2-24/+24
As preparation for adding a second queue attributes query, change the name of the existing queue attributes. Reviewed-by: Michael Margolin <mrgolin@amazon.com> Signed-off-by: Yonatan Nachum <ynachum@amazon.com> Link: https://patch.msgid.link/20260217112304.36849-2-ynachum@amazon.com Signed-off-by: Leon Romanovsky <leon@kernel.org>
2026-02-24RDMA/efa: Fix typo in efa_alloc_mr()Jason Gunthorpe1-1/+1
The pattern is to check the entire driver request space, not just sizeof something unrelated. Fixes: 40909f664d27 ("RDMA/efa: Add EFA verbs implementation") Signed-off-by: Jason Gunthorpe <jgg@nvidia.com> Link: https://patch.msgid.link/1-v1-83e918d69e73+a9-rdma_udata_rc_jgg@nvidia.com Acked-by: Michael Margolin <mrgolin@amazon.com> Signed-off-by: Leon Romanovsky <leon@kernel.org>
2026-02-21Convert more 'alloc_obj' cases to default GFP_KERNEL argumentsLinus Torvalds1-2/+1
This converts some of the visually simpler cases that have been split over multiple lines. I only did the ones that are easy to verify the resulting diff by having just that final GFP_KERNEL argument on the next line. Somebody should probably do a proper coccinelle script for this, but for me the trivial script actually resulted in an assertion failure in the middle of the script. I probably had made it a bit _too_ trivial. So after fighting that far a while I decided to just do some of the syntactically simpler cases with variations of the previous 'sed' scripts. The more syntactically complex multi-line cases would mostly really want whitespace cleanup anyway. Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2026-02-21Convert 'alloc_obj' family to use the new default GFP_KERNEL argumentLinus Torvalds2-4/+4
This was done entirely with mindless brute force, using git grep -l '\<k[vmz]*alloc_objs*(.*, GFP_KERNEL)' | xargs sed -i 's/\(alloc_objs*(.*\), GFP_KERNEL)/\1)/' to convert the new alloc_obj() users that had a simple GFP_KERNEL argument to just drop that argument. Note that due to the extreme simplicity of the scripting, any slightly more complex cases spread over multiple lines would not be triggered: they definitely exist, but this covers the vast bulk of the cases, and the resulting diff is also then easier to check automatically. For the same reason the 'flex' versions will be done as a separate conversion. Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2026-02-21treewide: Replace kmalloc with kmalloc_obj for non-scalar typesKees Cook2-7/+6
This is the result of running the Coccinelle script from scripts/coccinelle/api/kmalloc_objs.cocci. The script is designed to avoid scalar types (which need careful case-by-case checking), and instead replace kmalloc-family calls that allocate struct or union object instances: Single allocations: kmalloc(sizeof(TYPE), ...) are replaced with: kmalloc_obj(TYPE, ...) Array allocations: kmalloc_array(COUNT, sizeof(TYPE), ...) are replaced with: kmalloc_objs(TYPE, COUNT, ...) Flex array allocations: kmalloc(struct_size(PTR, FAM, COUNT), ...) are replaced with: kmalloc_flex(*PTR, FAM, COUNT, ...) (where TYPE may also be *VAR) The resulting allocations no longer return "void *", instead returning "TYPE *". Signed-off-by: Kees Cook <kees@kernel.org>
2026-02-12Merge tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdmaLinus Torvalds1-43/+54
Pull rdma updates from Jason Gunthorpe: "Usual smallish cycle. The NFS biovec work to push it down into RDMA instead of indirecting through a scatterlist is pretty nice to see, been talked about for a long time now. - Various code improvements in irdma, rtrs, qedr, ocrdma, irdma, rxe - Small driver improvements and minor bug fixes to hns, mlx5, rxe, mana, mlx5, irdma - Robusness improvements in completion processing for EFA - New query_port_speed() verb to move past limited IBA defined speed steps - Support for SG_GAPS in rts and many other small improvements - Rare list corruption fix in iwcm - Better support different page sizes in rxe - Device memory support for mana - Direct bio vec to kernel MR for use by NFS-RDMA - QP rate limiting for bnxt_re - Remote triggerable NULL pointer crash in siw - DMA-buf exporter support for RDMA mmaps like doorbells" * tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma: (66 commits) RDMA/mlx5: Implement DMABUF export ops RDMA/uverbs: Add DMABUF object type and operations RDMA/uverbs: Support external FD uobjects RDMA/siw: Fix potential NULL pointer dereference in header processing RDMA/umad: Reject negative data_len in ib_umad_write IB/core: Extend rate limit support for RC QPs RDMA/mlx5: Support rate limit only for Raw Packet QP RDMA/bnxt_re: Report QP rate limit in debugfs RDMA/bnxt_re: Report packet pacing capabilities when querying device RDMA/bnxt_re: Add support for QP rate limiting MAINTAINERS: Drop RDMA files from Hyper-V section RDMA/uverbs: Add __GFP_NOWARN to ib_uverbs_unmarshall_recv() kmalloc svcrdma: use bvec-based RDMA read/write API RDMA/core: add rdma_rw_max_sge() helper for SQ sizing RDMA/core: add MR support for bvec-based RDMA operations RDMA/core: use IOVA-based DMA mapping for bvec RDMA operations RDMA/core: add bio_vec based RDMA read/write API RDMA/irdma: Use kvzalloc for paged memory DMA address array RDMA/rxe: Fix race condition in QP timer handlers RDMA/mana_ib: Add device‑memory support ...
2025-12-18RDMA/efa: Improve admin completion context state machineYonatan Nachum1-41/+50
Add a new unused state to the admin completion contexts state machine instead of the occupied field. This improves the completion validity check because it now enforce the context to be in submitted state prior to completing it. Also add allocated state as a intermediate state between unused and submitted. Reviewed-by: Daniel Kranzdorf <dkkranzd@amazon.com> Reviewed-by: Michael Margolin <mrgolin@amazon.com> Signed-off-by: Yonatan Nachum <ynachum@amazon.com> Link: https://patch.msgid.link/20251210130614.36460-3-ynachum@amazon.com Signed-off-by: Leon Romanovsky <leon@kernel.org>
2025-12-18RDMA/efa: Check stored completion CTX command ID with received oneYonatan Nachum1-2/+4
In admin command completion, we receive a CQE with the command ID which is constructed from context index and entropy bits from the admin queue producer counter. To try to detect memory corruptions in the received CQE, validate the full command ID of the fetched context with the CQE command ID. If there is a mismatch, complete the CQE with error. Also use LSBs of the admin queue producer counter to better detect entropy mismatch between smaller number of commands. Reviewed-by: Daniel Kranzdorf <dkkranzd@amazon.com> Reviewed-by: Michael Margolin <mrgolin@amazon.com> Signed-off-by: Yonatan Nachum <ynachum@amazon.com> Link: https://patch.msgid.link/20251210130614.36460-2-ynachum@amazon.com Signed-off-by: Leon Romanovsky <leon@kernel.org>
2025-12-17RDMA/efa: Remove possible negative shiftMichael Margolin1-4/+0
The page size used for device might in some cases be smaller than PAGE_SIZE what results in a negative shift when calculating the number of host pages in PAGE_SIZE for a debug log. Remove the debug line together with the calculation. Fixes: 40909f664d27 ("RDMA/efa: Add EFA verbs implementation") Link: https://patch.msgid.link/r/20251210173656.8180-1-mrgolin@amazon.com Reviewed-by: Tom Sela <tomsela@amazon.com> Reviewed-by: Yonatan Nachum <ynachum@amazon.com> Signed-off-by: Michael Margolin <mrgolin@amazon.com> Reviewed-by: Gal Pressman <gal.pressman@linux.dev> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2025-10-19RDMA/uverbs: Fix umem release in UVERBS_METHOD_CQ_CREATEShuhao Fu1-9/+7
In `UVERBS_METHOD_CQ_CREATE`, umem should be released if anything goes wrong. Currently, if `create_cq_umem` fails, umem would not be released or referenced, causing a possible leak. In this patch, we release umem at `UVERBS_METHOD_CQ_CREATE`, the driver should not release umem if it returns an error code. Fixes: 1a40c362ae26 ("RDMA/uverbs: Add a common way to create CQ with umem") Signed-off-by: Shuhao Fu <sfual@cse.ust.hk> Link: https://patch.msgid.link/aOh1le4YqtYwj-hH@osx.local Signed-off-by: Leon Romanovsky <leon@kernel.org>
2025-09-21RDMA: Use %pe format specifier for error pointersLeon Romanovsky2-4/+6
Convert error logging throughout the RDMA subsystem to use the %pe format specifier instead of PTR_ERR() with integer format specifiers. Link: https://patch.msgid.link/e81ec02df1e474be20417fb62e779776e3f47a50.1758217936.git.leon@kernel.org Reviewed-by: Zhu Yanjun <yanjun.zhu@linux.dev> Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Copyright © 1996 – 2026 Jason A. Donenfeld. All Rights Reverse Engineered.