aboutsummaryrefslogtreecommitdiffstats
path: root/drivers/crypto/qat
AgeCommit message (Collapse)AuthorFilesLines
2021-11-26crypto: qat - refactor PF top half for PFVFMarco Chiappero1-36/+40
Move logic associated to handling VF2PF interrupt to its own function. This will simplify the handling of multiple interrupt sources in the function adf_msix_isr_ae() in the future. Signed-off-by: Marco Chiappero <marco.chiappero@intel.com> Co-developed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-11-26crypto: qat - fix undetected PFVF timeout in ACK loopGiovanni Cabiddu1-6/+6
If the remote function did not ACK the reception of a message, the function __adf_iov_putmsg() could detect it as a collision. This was due to the fact that the collision and the timeout checks after the ACK loop were in the wrong order. The timeout must be checked at the end of the loop, so fix by swapping the order of the two checks. Fixes: 9b768e8a3909 ("crypto: qat - detect PFVF collision after ACK") Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Co-developed-by: Marco Chiappero <marco.chiappero@intel.com> Signed-off-by: Marco Chiappero <marco.chiappero@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-11-26crypto: qat - do not handle PFVF sources for qat_4xxxGiovanni Cabiddu1-0/+7
The QAT driver does not have support for PFVF interrupts for GEN4 devices, therefore report the vf2pf sources as 0. This prevents a NULL pointer dereference in the function adf_msix_isr_ae() if the device triggers a spurious interrupt. Fixes: 993161d36ab5 ("crypto: qat - fix handling of VF to PF interrupts") Reported-by: Adam Guerin <adam.guerin@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-11-06Merge tag 'pci-v5.16-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pciLinus Torvalds6-28/+13
Pull pci updates from Bjorn Helgaas: "Enumeration: - Conserve IRQs by setting up portdrv IRQs only when there are users (Jan Kiszka) - Rework and simplify _OSC negotiation for control of PCIe features (Joerg Roedel) - Remove struct pci_dev.driver pointer since it's redundant with the struct device.driver pointer (Uwe Kleine-König) Resource management: - Coalesce contiguous host bridge apertures from _CRS to accommodate BARs that cover more than one aperture (Kai-Heng Feng) Sysfs: - Check CAP_SYS_ADMIN before parsing user input (Krzysztof Wilczyński) - Return -EINVAL consistently from "store" functions (Krzysztof Wilczyński) - Use sysfs_emit() in endpoint "show" functions to avoid buffer overruns (Kunihiko Hayashi) PCIe native device hotplug: - Ignore Link Down/Up caused by resets during error recovery so endpoint drivers can remain bound to the device (Lukas Wunner) Virtualization: - Avoid bus resets on Atheros QCA6174, where they hang the device (Ingmar Klein) - Work around Pericom PI7C9X2G switch packet drop erratum by using store and forward mode instead of cut-through (Nathan Rossi) - Avoid trying to enable AtomicOps on VFs; the PF setting applies to all VFs (Selvin Xavier) MSI: - Document that /sys/bus/pci/devices/.../irq contains the legacy INTx interrupt or the IRQ of the first MSI (not MSI-X) vector (Barry Song) VPD: - Add pci_read_vpd_any() and pci_write_vpd_any() to access anywhere in the possible VPD space; use these to simplify the cxgb3 driver (Heiner Kallweit) Peer-to-peer DMA: - Add (not subtract) the bus offset when calculating DMA address (Wang Lu) ASPM: - Re-enable LTR at Downstream Ports so they don't report Unsupported Requests when reset or hot-added devices send LTR messages (Mingchuang Qiao) Apple PCIe controller driver: - Add driver for Apple M1 PCIe controller (Alyssa Rosenzweig, Marc Zyngier) Cadence PCIe controller driver: - Return success when probe succeeds instead of falling into error path (Li Chen) HiSilicon Kirin PCIe controller driver: - Reorganize PHY logic and add support for external PHY drivers (Mauro Carvalho Chehab) - Support PERST# GPIOs for HiKey970 external PEX 8606 bridge (Mauro Carvalho Chehab) - Add Kirin 970 support (Mauro Carvalho Chehab) - Make driver removable (Mauro Carvalho Chehab) Intel VMD host bridge driver: - If IOMMU supports interrupt remapping, leave VMD MSI-X remapping enabled (Adrian Huang) - Number each controller so we can tell them apart in /proc/interrupts (Chunguang Xu) - Avoid building on UML because VMD depends on x86 bare metal APIs (Johannes Berg) Marvell Aardvark PCIe controller driver: - Define macros for PCI_EXP_DEVCTL_PAYLOAD_* (Pali Rohár) - Set Max Payload Size to 512 bytes per Marvell spec (Pali Rohár) - Downgrade PIO Response Status messages to debug level (Marek Behún) - Preserve CRS SV (Config Request Retry Software Visibility) bit in emulated Root Control register (Pali Rohár) - Fix issue in configuring reference clock (Pali Rohár) - Don't clear status bits for masked interrupts (Pali Rohár) - Don't mask unused interrupts (Pali Rohár) - Avoid code repetition in advk_pcie_rd_conf() (Marek Behún) - Retry config accesses on CRS response (Pali Rohár) - Simplify emulated Root Capabilities initialization (Pali Rohár) - Fix several link training issues (Pali Rohár) - Fix link-up checking via LTSSM (Pali Rohár) - Fix reporting of Data Link Layer Link Active (Pali Rohár) - Fix emulation of W1C bits (Marek Behún) - Fix MSI domain .alloc() method to return zero on success (Marek Behún) - Read entire 16-bit MSI vector in MSI handler, not just low 8 bits (Marek Behún) - Clear Root Port I/O Space, Memory Space, and Bus Master Enable bits at startup; PCI core will set those as necessary (Pali Rohár) - When operating as a Root Port, set class code to "PCI Bridge" instead of the default "Mass Storage Controller" (Pali Rohár) - Add emulation for PCI_BRIDGE_CTL_BUS_RESET since aardvark doesn't implement this per spec (Pali Rohár) - Add emulation of option ROM BAR since aardvark doesn't implement this per spec (Pali Rohár) MediaTek MT7621 PCIe controller driver: - Add MediaTek MT7621 PCIe host controller driver and DT binding (Sergio Paracuellos) Qualcomm PCIe controller driver: - Add SC8180x compatible string (Bjorn Andersson) - Add endpoint controller driver and DT binding (Manivannan Sadhasivam) - Restructure to use of_device_get_match_data() (Prasad Malisetty) - Add SC7280-specific pcie_1_pipe_clk_src handling (Prasad Malisetty) Renesas R-Car PCIe controller driver: - Remove unnecessary includes (Geert Uytterhoeven) Rockchip DesignWare PCIe controller driver: - Add DT binding (Simon Xue) Socionext UniPhier Pro5 controller driver: - Serialize INTx masking/unmasking (Kunihiko Hayashi) Synopsys DesignWare PCIe controller driver: - Run dwc .host_init() method before registering MSI interrupt handler so we can deal with pending interrupts left by bootloader (Bjorn Andersson) - Clean up Kconfig dependencies (Andy Shevchenko) - Export symbols to allow more modular drivers (Luca Ceresoli) TI DRA7xx PCIe controller driver: - Allow host and endpoint drivers to be modules (Luca Ceresoli) - Enable external clock if present (Luca Ceresoli) TI J721E PCIe driver: - Disable PHY when probe fails after initializing it (Christophe JAILLET) MicroSemi Switchtec management driver: - Return error to application when command execution fails because an out-of-band reset has cleared the device BARs, Memory Space Enable, etc (Kelvin Cao) - Fix MRPC error status handling issue (Kelvin Cao) - Mask out other bits when reading of management VEP instance ID (Kelvin Cao) - Return EOPNOTSUPP instead of ENOTSUPP from sysfs show functions (Kelvin Cao) - Add check of event support (Logan Gunthorpe) Miscellaneous: - Remove unused pci_pool wrappers, which have been replaced by dma_pool (Cai Huoqing) - Use 'unsigned int' instead of bare 'unsigned' (Krzysztof Wilczyński) - Use kstrtobool() directly, sans strtobool() wrapper (Krzysztof Wilczyński) - Fix some sscanf(), sprintf() format mismatches (Krzysztof Wilczyński) - Update PCI subsystem information in MAINTAINERS (Krzysztof Wilczyński) - Correct some misspellings (Krzysztof Wilczyński)" * tag 'pci-v5.16-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci: (137 commits) PCI: Add ACS quirk for Pericom PI7C9X2G switches PCI: apple: Configure RID to SID mapper on device addition iommu/dart: Exclude MSI doorbell from PCIe device IOVA range PCI: apple: Implement MSI support PCI: apple: Add INTx and per-port interrupt support PCI: kirin: Allow removing the driver PCI: kirin: De-init the dwc driver PCI: kirin: Disable clkreq during poweroff sequence PCI: kirin: Move the power-off code to a common routine PCI: kirin: Add power_off support for Kirin 960 PHY PCI: kirin: Allow building it as a module PCI: kirin: Add MODULE_* macros PCI: kirin: Add Kirin 970 compatible PCI: kirin: Support PERST# GPIOs for HiKey970 external PEX 8606 bridge PCI: apple: Set up reference clocks when probing PCI: apple: Add initial hardware bring-up PCI: of: Allow matching of an interrupt-map local to a PCI device of/irq: Allow matching of an interrupt-map local to an interrupt controller irqdomain: Make of_phandle_args_to_fwspec() generally available PCI: Do not enable AtomicOps on VFs ...
2021-10-12crypto: qat - simplify adf_enable_aer()Uwe Kleine-König6-28/+13
A struct pci_driver is shared across all device instances, so assigning pci_driver.err_handler once per device isn't really sensible. Set adf_driver.err_handler statically instead of in adf_enable_aer(). This removes a use of pci_dev->driver, which is a step toward removing pci_dev->driver altogether. Since adf_enable_aer() returns zero unconditionally, make it a void function. Link: https://lore.kernel.org/r/20211004125935.2300113-10-u.kleine-koenig@pengutronix.de Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2021-10-08crypto: qat - share adf_enable_pf2vf_comms() from adf_pf2vf_msg.cMarco Chiappero6-23/+26
The PFVF protocol "enable" functions are direction specific but not device specific. Move the protocol enable function for the PF into the PF specific protocol file for better file organization and duplicated code reduction. NOTE: the patch keeps gen4 disabled as it doesn't have full PFVF support yet. Signed-off-by: Marco Chiappero <marco.chiappero@intel.com> Co-developed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-10-08crypto: qat - extract send and wait from adf_vf2pf_request_version()Marco Chiappero1-13/+37
In the function adf_vf2pf_request_version(), the VF sends a request to the PF and waits for a response before parsing and handling it. Since this pattern will be used by other requests, define a new function, adf_send_vf2pf_req(), that only deals with sending a VF2PF request and waiting for a response. Signed-off-by: Marco Chiappero <marco.chiappero@intel.com> Co-developed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-10-08crypto: qat - add VF and PF wrappers to common send functionMarco Chiappero3-8/+36
The send function, adf_iov_putmsg(), is shared by both PF and VF. This commit provides two direction specific APIs, adf_send_pf2vf_msg() and adf_send_vf2pf_msg() which decouple the implementation, which can change and evolve over time, from the user. With this change, the adf_iov_putmsg() is now isolated inside the file adf_pf2vf_msg.c and has been marked as static. Signed-off-by: Marco Chiappero <marco.chiappero@intel.com> Co-developed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-10-08crypto: qat - rename pfvf collision constantsMarco Chiappero1-13/+13
Replace any reference of "IOV" with PFVF in the collision constants. Signed-off-by: Marco Chiappero <marco.chiappero@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-10-08crypto: qat - move pfvf collision detection valuesMarco Chiappero2-9/+9
Keep adf_pf2vf_msg.h as much as possible focused on the protocol definition. Instead, collision parameters are an implementation detail which should stay close to the code consuming them, therefore move them to adf_pf2vf_msg.c. Signed-off-by: Marco Chiappero <marco.chiappero@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-10-08crypto: qat - make pfvf send message direction agnosticMarco Chiappero1-7/+8
The functions adf_iov_putmsg() and __adf_iov_putmsg() are shared by both PF and VF. Any logging or documentation should not refer to any specific direction. Make comments and log messages direction agnostic by replacing PF2VF with PFVF. Also fix the wording for some related comments. Signed-off-by: Marco Chiappero <marco.chiappero@intel.com> Co-developed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-10-08crypto: qat - use hweight for bit countingGiovanni Cabiddu1-14/+2
Replace homegrown bit counting logic in adf_gen2_get_num_accels() and adf_gen2_get_num_aes() with the functions hweight16() and hweight32(), respectively. Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Marco Chiappero <marco.chiappero@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-10-08crypto: qat - remove duplicated logic across GEN2 driversMarco Chiappero8-245/+107
QAT GEN2 devices share most of the behavior which means a number of device specific functions can be shared too and some differences abstracted away by simple parameters. The functions adf_enable_error_correction(), get_num_accels(), get_num_aes() and get_pf2vf_offset() for c3xxx, c62x and dh895xx have been reworked and moved to the GEN2 file, adf_gen2_hw_data.c. The definitions of tx_rx_gap and tx_rings_mask have been moved to adf_gen2_hw_data.h. Signed-off-by: Marco Chiappero <marco.chiappero@intel.com> Co-developed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-10-08crypto: qat - fix handling of VF to PF interruptsMarco Chiappero9-76/+133
Currently, VF to PF interrupt handling is based on the DH895XCC device behavior, which is not entirely common to all devices. In order to make interrupt detection and handling correct for all of the supported devices, make the interrupt handling device specific by: - introducing get_vf2pf_sources() for getting a 32 bits long value where each bit represents a vf2pf interrupt; - adding the device [enable|disable]_vf2pf_interrupts to hw_data; - defining [enable|disable]_vf2pf_interrupts for all the devices that are currently supported, using only their required and specific ERRSOU|ERRMASK registers (DH895XCC has 32 interrupts spread across ERRSOU3 and ERRSOU5, C62X/C3XXX has 16 in ERRSOU3 only, etc). Code has been shared by different devices wherever possible. This patch is based on earlier work done by Salvatore Benedetto. Signed-off-by: Marco Chiappero <marco.chiappero@intel.com> Co-developed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-10-08crypto: qat - remove unnecessary collision prevention step in PFVFMarco Chiappero1-19/+1
The initial version of the PFVF protocol included an initial "carrier sensing" to get ownership of the channel. Collisions can happen anyway, the extra wait and test does not prevent collisions, it instead slows the communication down, so remove it. Signed-off-by: Marco Chiappero <marco.chiappero@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-10-08crypto: qat - disregard spurious PFVF interruptsGiovanni Cabiddu2-0/+12
Upon receiving a PFVF message, check if the interrupt bit is set in the message. If it is not, that means that the interrupt was probably triggered by a collision. In this case, disregard the message and re-enable the interrupts. Fixes: ed8ccaef52fa ("crypto: qat - Add support for SRIOV") Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Marco Chiappero <marco.chiappero@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-10-08crypto: qat - detect PFVF collision after ACKGiovanni Cabiddu1-0/+7
Detect a PFVF collision between the local and the remote function by checking if the message on the PFVF CSR has been overwritten. This is done after the remote function confirms that the message has been received, by clearing the interrupt bit, or the maximum number of attempts (ADF_IOV_MSG_ACK_MAX_RETRY) to check the CSR has been exceeded. Fixes: ed8ccaef52fa ("crypto: qat - Add support for SRIOV") Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Co-developed-by: Marco Chiappero <marco.chiappero@intel.com> Signed-off-by: Marco Chiappero <marco.chiappero@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-09-24crypto: qat - power up 4xxx deviceGiovanni Cabiddu4-0/+47
After reset or boot, QAT 4xxx devices are inactive and require to be explicitly activated. This is done by writing the DRV_ACTIVE bit in the PM_INTERRUPT register and polling the PM_INIT_STATE to make sure that the transaction has completed properly. If this is not done, the driver will fail the initialization sequence reporting the following message: [ 22.081193] 4xxx 0000:f7:00.0: enabling device (0140 -> 0142) [ 22.720285] QAT: AE0 is inactive!! [ 22.720287] QAT: failed to get device out of reset [ 22.720288] 4xxx 0000:f7:00.0: qat_hal_clr_reset error [ 22.720290] 4xxx 0000:f7:00.0: Failed to init the AEs [ 22.720290] 4xxx 0000:f7:00.0: Failed to initialise Acceleration Engine [ 22.720789] 4xxx 0000:f7:00.0: Resetting device qat_dev0 [ 22.825099] 4xxx: probe of 0000:f7:00.0 failed with error -14 The patch also temporarily disables the power management source of interrupt, to avoid possible spurious interrupts as the power management feature is not fully supported. The device init function has been added to adf_dev_init(), and not in the probe of 4xxx to make sure that the device is re-enabled in case of reset. Note that the error code reported by hw_data->init_device() in adf_dev_init() has been shadowed for consistency with the other calls in the same function. Fixes: 8c8268166e83 ("crypto: qat - add qat_4xxx driver") Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Wojciech Ziemba <wojciech.ziemba@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-09-17crypto: qat - remove unneeded packed attributeGiovanni Cabiddu1-6/+6
Remove packed attribute from structures that do not need to be packed. These are just used internally and not shared with firmware. This also fixes a series of warning when compiling the driver with the flag -Waddress-of-packed-member, similar to the following: drivers/crypto/qat/qat_c62xvf/adf_c62xvf_hw_data.c:102:28: warning: taking address of packed member 'csr_ops' of class or structure 'adf_hw_device_data' may result in an unaligned pointer value Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reported-by: kernel test robot <lkp@intel.com> Reviewed-by: Wojciech Ziemba <wojciech.ziemba@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-09-17crypto: qat - free irq in case of failureWojciech Ziemba1-29/+34
If devm_request_irq() fails inside adf_request_irqs(), unwind properly by freeing the allocated irqs. Signed-off-by: Wojciech Ziemba <wojciech.ziemba@intel.com> Co-developed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-09-17crypto: qat - free irqs only if allocatedMaksim Lukoshkov3-37/+35
Change the irq allocation logic so that it is possible to free only the allocated irqs in case of error. A new flag is introduced for every PF/VF interrupt. This flag is set to "true" only when the interrupt is requested. During clean up, devm_free_irq() is only called if this flag is set. Signed-off-by: Maksim Lukoshkov <maksim.lukoshkov@intel.com> Co-developed-by: Wojciech Ziemba <wojciech.ziemba@intel.com> Signed-off-by: Wojciech Ziemba <wojciech.ziemba@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-09-17crypto: qat - remove unmatched CPU affinity to cluster IRQWojciech Ziemba1-1/+0
This patch removes redundant CPU affinity to AE cluster IRQ. Signed-off-by: Wojciech Ziemba <wojciech.ziemba@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-09-17crypto: qat - replace deprecated MSI APIWojciech Ziemba3-60/+55
Replace deprecated MSI enable and disable respectively and update handling of return values. Signed-off-by: Wojciech Ziemba <wojciech.ziemba@intel.com> Co-developed-by: Gupta Shashank <shashank.gupta@intel.com> Signed-off-by: Gupta Shashank <shashank.gupta@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-08-21crypto: qat - store vf.compatible flagGiovanni Cabiddu1-1/+3
If the VF is newer than the PF, it decides whether it is compatible or not. In case it is compatible, store that information in the vf.compatible flag in the accel_dev structure. Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Suggested-by: Fiona Trahe <fiona.trahe@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-08-21crypto: qat - do not export adf_iov_putmsg()Giovanni Cabiddu1-1/+0
The function adf_iov_putmsg() is only used inside the intel_qat module therefore should not be exported. Remove EXPORT_SYMBOL for the function adf_iov_putmsg(). Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Fiona Trahe <fiona.trahe@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-08-21crypto: qat - flush vf workqueue at driver removalAhsan Atta5-0/+33
There is a race condition during shutdown in adf_disable_sriov() where both the PF and the VF drivers are loaded on the host system. The PF notifies a VF with a "RESTARTING" message due to which the VF starts an asynchronous worker to stop and shutdown itself. At the same time the PF calls pci_disable_sriov() which invokes the remove() routine on the VF device driver triggering the shutdown flow again. This change fixes the problem by ensuring that the VF flushes the worker that performs stop()/shutdown() before these two functions are called in the remove(). To make sure that no additional PV/VF messages are processed by the VF, interrupts are disabled before flushing the workqueue. Signed-off-by: Ahsan Atta <ahsan.atta@intel.com> Co-developed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Fiona Trahe <fiona.trahe@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-08-21crypto: qat - remove the unnecessary get_vintmsk_offset()Marco Chiappero14-44/+2
All QAT GEN2 devices share the same register offset for masking interrupts, so they don't need any complex device specific infrastructure. Remove this function in favor of a constant in order to simplify the code. Also, future generations may require a more complex device specific handling, making the current approach obsolete anyway. Signed-off-by: Marco Chiappero <marco.chiappero@intel.com> Co-developed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Fiona Trahe <fiona.trahe@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-08-21crypto: qat - fix naming of PF/VF enable functionsMarco Chiappero9-13/+13
Currently all the functions related to the activation of the PFVF protocol, both on PF and VF, include the direction specific "vf2pf" name. Replace the existing naming schema with: - a direction agnostic naming, that applies to both PF and VF, for the function pointer ("pfvf") - a direction specific naming schema for the implementations ("pf2vf" or "vf2pf") In particular this patch renames: - adf_pf_enable_vf2pf_comms() in adf_enable_pf2vf_comms() - enable_vf2pf_comms() in enable_pfvf_comms() Signed-off-by: Marco Chiappero <marco.chiappero@intel.com> Co-developed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Fiona Trahe <fiona.trahe@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-08-21crypto: qat - complete all the init steps before service notificationMarco Chiappero1-4/+6
Make sure all the steps in the initialization sequence are complete before any completion event notification. Signed-off-by: Marco Chiappero <marco.chiappero@intel.com> Co-developed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Fiona Trahe <fiona.trahe@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-08-21crypto: qat - move IO virtualization functionsGiovanni Cabiddu4-15/+17
Move IOV functions at the end of hw_data so that PFVF functions related functions are group together. Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Fiona Trahe <fiona.trahe@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-08-21crypto: qat - fix naming for init/shutdown VF to PF notificationsMarco Chiappero5-16/+16
At start and shutdown, VFs notify the PF about their state. These notifications are carried out through a message exchange using the PFVF protocol. Function names lead to believe they do perform init or shutdown logic. This is to fix the naming to better reflect their purpose. Signed-off-by: Marco Chiappero <marco.chiappero@intel.com> Co-developed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Fiona Trahe <fiona.trahe@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-08-21crypto: qat - protect interrupt mask CSRs with a spinlockKanchana Velusamy7-4/+41
In the PF interrupt handler, the interrupt is disabled for a set of VFs by writing to the interrupt source mask register, ERRMSK. The interrupt is re-enabled in the bottom half handler by writing to the same CSR. This is done through the functions enable_vf2pf_interrupts() and disable_vf2pf_interrupts() which perform a read-modify-write operation on the ERRMSK registers to mask and unmask the source of interrupt. There can be a race condition where the top half handler for one VF interrupt runs just as the bottom half for another VF is about to re-enable the interrupt. Depending on whether the top or bottom half updates the CSR first, this would result either in a spurious interrupt or in the interrupt not being re-enabled. This patch protects the access of ERRMSK with a spinlock. The functions adf_enable_vf2pf_interrupts() and adf_disable_vf2pf_interrupts() have been changed to acquire a spin lock before accessing and modifying the ERRMSK registers. These functions use spin_lock_irqsave() to disable IRQs and avoid potential deadlocks. In addition, the function adf_disable_vf2pf_interrupts_irq() has been added. This uses spin_lock() and it is meant to be used in the top half only. Signed-off-by: Kanchana Velusamy <kanchanax.velusamy@intel.com> Co-developed-by: Marco Chiappero <marco.chiappero@intel.com> Signed-off-by: Marco Chiappero <marco.chiappero@intel.com> Co-developed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Fiona Trahe <fiona.trahe@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-08-21crypto: qat - move pf2vf interrupt [en|dis]able to adf_vf_isr.cMarco Chiappero2-20/+20
Interrupt code to enable interrupts from PF does not belong to the protocol code, so move it to the interrupt handling specific file for better code organization. Signed-off-by: Marco Chiappero <marco.chiappero@intel.com> Co-developed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Fiona Trahe <fiona.trahe@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-08-21crypto: qat - fix reuse of completion variableMarco Chiappero1-0/+2
Use reinit_completion() to set to a clean state a completion variable, used to coordinate the VF to PF request-response flow, before every new VF request. Signed-off-by: Marco Chiappero <marco.chiappero@intel.com> Co-developed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Fiona Trahe <fiona.trahe@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-08-21crypto: qat - remove intermediate tasklet for vf2pfSvyatoslav Pankratov4-13/+5
The PF driver uses the tasklet vf2pf_bh_tasklet to schedule a workqueue to handle the vf2vf protocol (pf2vf_resp_wq). Since the tasklet is only used to schedule the workqueue, this patch removes it and schedules the pf2vf_resp_wq workqueue directly for the top half. Signed-off-by: Svyatoslav Pankratov <svyatoslav.pankratov@intel.com> Co-developed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Marco Chiappero <marco.chiappero@intel.com> Reviewed-by: Fiona Trahe <fiona.trahe@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-08-21crypto: qat - rename compatibility version definitionMarco Chiappero9-17/+17
Rename ADF_PFVF_COMPATIBILITY_VERSION in ADF_PFVF_COMPAT_THIS_VERSION since it is used to indicate the current version of the PFVF protocol. Signed-off-by: Marco Chiappero <marco.chiappero@intel.com> Co-developed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Fiona Trahe <fiona.trahe@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-08-21crypto: qat - prevent spurious MSI interrupt in PFMarco Chiappero2-7/+20
There is a chance that the PFVF handler, adf_vf2pf_req_hndl(), runs twice for the same request when multiple interrupts come simultaneously from different VFs. Since the source VF is identified by a positional bit set in the ERRSOU registers and that is not cleared until the bottom half completes, new top halves from other VFs may reschedule a second bottom half for previous interrupts. This patch solves the problem in the ISR handler by not considering sources with already disabled interrupts (and processing pending), as set in the ERRMSK registers. Also, move some definitions where actually needed. Signed-off-by: Marco Chiappero <marco.chiappero@intel.com> Co-developed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Fiona Trahe <fiona.trahe@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-08-21crypto: qat - prevent spurious MSI interrupt in VFGiovanni Cabiddu1-1/+11
QAT GEN2 devices suffer from a defect where the MSI interrupt can be sent multiple times. If the second (spurious) interrupt is handled before the bottom half handler runs, then the extra interrupt is effectively ignored because the bottom half is only scheduled once. However, if the top half runs again after the bottom half runs, this will appear as a spurious PF to VF interrupt. This can be avoided by checking the interrupt mask register in addition to the interrupt source register in the interrupt handler. This patch is based on earlier work done by Conor McLoughlin. Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Co-developed-by: Marco Chiappero <marco.chiappero@intel.com> Signed-off-by: Marco Chiappero <marco.chiappero@intel.com> Reviewed-by: Fiona Trahe <fiona.trahe@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-08-21crypto: qat - handle both source of interrupt in VF ISRGiovanni Cabiddu1-3/+4
The top half of the VF drivers handled only a source at the time. If an interrupt for PF2VF and bundle occurred at the same time, the ISR scheduled only the bottom half for PF2VF. This patch fixes the VF top half so that if both sources of interrupt trigger at the same time, both bottom halves are scheduled. This patch is based on earlier work done by Conor McLoughlin. Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Marco Chiappero <marco.chiappero@intel.com> Reviewed-by: Fiona Trahe <fiona.trahe@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-08-21crypto: qat - do not ignore errors from enable_vf2pf_comms()Giovanni Cabiddu1-2/+3
The function adf_dev_init() ignores the error code reported by enable_vf2pf_comms(). If the latter fails, e.g. the VF is not compatible with the pf, then the load of the VF driver progresses. This patch changes adf_dev_init() so that the error code from enable_vf2pf_comms() is returned to the caller. Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Marco Chiappero <marco.chiappero@intel.com> Reviewed-by: Fiona Trahe <fiona.trahe@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-08-21crypto: qat - enable interrupts only after ISR allocationMarco Chiappero1-2/+2
Enable device interrupts after the setup of the interrupt handlers. Signed-off-by: Marco Chiappero <marco.chiappero@intel.com> Co-developed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Fiona Trahe <fiona.trahe@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-08-21crypto: qat - remove empty sriov_configure()Marco Chiappero1-4/+1
Remove the empty implementation of sriov_configure() and set the sriov_configure member of the pci_driver structure to NULL. This way, if a user tries to enable VFs on a device, when kernel and driver are built with CONFIG_PCI_IOV=n, the kernel reports an error message saying that the driver does not support SRIOV configuration via sysfs. Signed-off-by: Marco Chiappero <marco.chiappero@intel.com> Co-developed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Fiona Trahe <fiona.trahe@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-08-21crypto: qat - use proper type for vf_maskGiovanni Cabiddu1-3/+4
Replace vf_mask type with unsigned long to avoid a stack-out-of-bound. This is to fix the following warning reported by KASAN the first time adf_msix_isr_ae() gets called. [ 692.091987] BUG: KASAN: stack-out-of-bounds in find_first_bit+0x28/0x50 [ 692.092017] Read of size 8 at addr ffff88afdf789e60 by task swapper/32/0 [ 692.092076] Call Trace: [ 692.092089] <IRQ> [ 692.092101] dump_stack+0x9c/0xcf [ 692.092132] print_address_description.constprop.0+0x18/0x130 [ 692.092164] ? find_first_bit+0x28/0x50 [ 692.092185] kasan_report.cold+0x7f/0x111 [ 692.092213] ? static_obj+0x10/0x80 [ 692.092234] ? find_first_bit+0x28/0x50 [ 692.092262] find_first_bit+0x28/0x50 [ 692.092288] adf_msix_isr_ae+0x16e/0x230 [intel_qat] Fixes: ed8ccaef52fa ("crypto: qat - Add support for SRIOV") Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Marco Chiappero <marco.chiappero@intel.com> Reviewed-by: Fiona Trahe <fiona.trahe@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-08-21crypto: qat - fix a typo in a commentChristophe JAILLET1-1/+1
s/Enable/Disable/ when describing 'adf_disable_aer()' Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr> Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-08-21crypto: qat - disable AER if an error occurs in probe functionsChristophe JAILLET3-6/+12
If an error occurs after a 'adf_enable_aer()' call, it must be undone by a corresponding 'adf_disable_aer()' call, as already done in the remove function. Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr> Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-08-21crypto: qat - set DMA mask to 48 bits for Gen2Giovanni Cabiddu6-6/+6
Change the DMA mask from 64 to 48 for Gen2 devices as they cannot handle addresses greater than 48 bits. Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-08-21crypto: qat - simplify code and axe the use of a deprecated APIChristophe JAILLET7-76/+28
The wrappers in include/linux/pci-dma-compat.h should go away. Replace 'pci_set_dma_mask/pci_set_consistent_dma_mask' by an equivalent and less verbose 'dma_set_mask_and_coherent()' call. Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr> Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Co-developed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-06-28crypto: qat - ratelimit invalid ioctl message and print the invalid cmdColin Ian King1-1/+1
Currently incorrect QAT ioctls can spam the kernel log with error messages of the form "QAT: Invalid ioctl" if a userspace program uses the wrong ioctl command. Quench the messages by ratelimiting them and also print the invalid command being used as that is useful to know. Signed-off-by: Colin Ian King <colin.king@canonical.com> Acked-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-05-28crypto: qat - remove unused macro in FW loaderJack Xu1-1/+0
Remove the unused macro ICP_DH895XCC_PESRAM_BAR_SIZE in the firmware loader. This is to fix the following warning when compiling the driver using the clang compiler with CC=clang W=2: drivers/crypto/qat/qat_common/qat_uclo.c:345:9: warning: macro is not used [-Wunused-macros] Signed-off-by: Jack Xu <jack.xu@intel.com> Co-developed-by: Zhehui Xiang <zhehui.xiang@intel.com> Signed-off-by: Zhehui Xiang <zhehui.xiang@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2021-05-28crypto: qat - check return code of qat_hal_rd_rel_reg()Jack Xu1-1/+5
Check the return code of the function qat_hal_rd_rel_reg() and return it to the caller. This is to fix the following warning when compiling the driver with clang scan-build: drivers/crypto/qat/qat_common/qat_hal.c:1436:2: warning: 6th function call argument is an uninitialized value Signed-off-by: Jack Xu <jack.xu@intel.com> Co-developed-by: Zhehui Xiang <zhehui.xiang@intel.com> Signed-off-by: Zhehui Xiang <zhehui.xiang@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Copyright © 1996 – 2026 Jason A. Donenfeld. All Rights Reverse Engineered.