aboutsummaryrefslogtreecommitdiffstatshomepage
path: root/Documentation/fpga/dfl.rst
diff options
context:
space:
mode:
Diffstat (limited to 'Documentation/fpga/dfl.rst')
-rw-r--r--Documentation/fpga/dfl.rst207
1 files changed, 202 insertions, 5 deletions
diff --git a/Documentation/fpga/dfl.rst b/Documentation/fpga/dfl.rst
index 978c4af416a4..80255e2dc3e6 100644
--- a/Documentation/fpga/dfl.rst
+++ b/Documentation/fpga/dfl.rst
@@ -7,9 +7,10 @@ Authors:
- Enno Luebbers <enno.luebbers@intel.com>
- Xiao Guangrong <guangrong.xiao@linux.intel.com>
- Wu Hao <hao.wu@intel.com>
+- Xu Yilun <yilun.xu@intel.com>
-The Device Feature List (DFL) FPGA framework (and drivers according to this
-this framework) hides the very details of low layer hardwares and provides
+The Device Feature List (DFL) FPGA framework (and drivers according to
+this framework) hides the very details of low layer hardware and provides
unified interfaces to userspace. Applications could use these interfaces to
configure, enumerate, open and access FPGA accelerators on platforms which
implement the DFL in the device memory. Besides this, the DFL framework
@@ -56,7 +57,7 @@ FPGA Interface Unit (FIU) represents a standalone functional unit for the
interface to FPGA, e.g. the FPGA Management Engine (FME) and Port (more
descriptions on FME and Port in later sections).
-Accelerated Function Unit (AFU) represents a FPGA programmable region and
+Accelerated Function Unit (AFU) represents an FPGA programmable region and
always connects to a FIU (e.g. a Port) as its child as illustrated above.
Private Features represent sub features of the FIU and AFU. They could be
@@ -74,6 +75,125 @@ convenient for software to locate each feature by walking through this list,
and can be implemented in register regions of any FPGA device.
+Device Feature Header - Version 0
+=================================
+Version 0 (DFHv0) is the original version of the Device Feature Header.
+All multi-byte quantities in DFHv0 are little-endian.
+The format of DFHv0 is shown below::
+
+ +-----------------------------------------------------------------------+
+ |63 Type 60|59 DFH VER 52|51 Rsvd 41|40 EOL|39 Next 16|15 REV 12|11 ID 0| 0x00
+ +-----------------------------------------------------------------------+
+ |63 GUID_L 0| 0x08
+ +-----------------------------------------------------------------------+
+ |63 GUID_H 0| 0x10
+ +-----------------------------------------------------------------------+
+
+- Offset 0x00
+
+ * Type - The type of DFH (e.g. FME, AFU, or private feature).
+ * DFH VER - The version of the DFH.
+ * Rsvd - Currently unused.
+ * EOL - Set if the DFH is the end of the Device Feature List (DFL).
+ * Next - The offset in bytes of the next DFH in the DFL from the DFH start,
+ and the start of a DFH must be aligned to an 8 byte boundary.
+ If EOL is set, Next is the size of MMIO of the last feature in the list.
+ * REV - The revision of the feature associated with this header.
+ * ID - The feature ID if Type is private feature.
+
+- Offset 0x08
+
+ * GUID_L - Least significant 64 bits of a 128-bit Globally Unique Identifier
+ (present only if Type is FME or AFU).
+
+- Offset 0x10
+
+ * GUID_H - Most significant 64 bits of a 128-bit Globally Unique Identifier
+ (present only if Type is FME or AFU).
+
+
+Device Feature Header - Version 1
+=================================
+Version 1 (DFHv1) of the Device Feature Header adds the following functionality:
+
+* Provides a standardized mechanism for features to describe
+ parameters/capabilities to software.
+* Standardize the use of a GUID for all DFHv1 types.
+* Decouples the DFH location from the register space of the feature itself.
+
+All multi-byte quantities in DFHv1 are little-endian.
+The format of Version 1 of the Device Feature Header (DFH) is shown below::
+
+ +-----------------------------------------------------------------------+
+ |63 Type 60|59 DFH VER 52|51 Rsvd 41|40 EOL|39 Next 16|15 REV 12|11 ID 0| 0x00
+ +-----------------------------------------------------------------------+
+ |63 GUID_L 0| 0x08
+ +-----------------------------------------------------------------------+
+ |63 GUID_H 0| 0x10
+ +-----------------------------------------------------------------------+
+ |63 Reg Address/Offset 1| Rel 0| 0x18
+ +-----------------------------------------------------------------------+
+ |63 Reg Size 32|Params 31|30 Group 16|15 Instance 0| 0x20
+ +-----------------------------------------------------------------------+
+ |63 Next 35|34RSV33|EOP32|31 Param Version 16|15 Param ID 0| 0x28
+ +-----------------------------------------------------------------------+
+ |63 Parameter Data 0| 0x30
+ +-----------------------------------------------------------------------+
+
+ ...
+
+ +-----------------------------------------------------------------------+
+ |63 Next 35|34RSV33|EOP32|31 Param Version 16|15 Param ID 0|
+ +-----------------------------------------------------------------------+
+ |63 Parameter Data 0|
+ +-----------------------------------------------------------------------+
+
+- Offset 0x00
+
+ * Type - The type of DFH (e.g. FME, AFU, or private feature).
+ * DFH VER - The version of the DFH.
+ * Rsvd - Currently unused.
+ * EOL - Set if the DFH is the end of the Device Feature List (DFL).
+ * Next - The offset in bytes of the next DFH in the DFL from the DFH start,
+ and the start of a DFH must be aligned to an 8 byte boundary.
+ If EOL is set, Next is the size of MMIO of the last feature in the list.
+ * REV - The revision of the feature associated with this header.
+ * ID - The feature ID if Type is private feature.
+
+- Offset 0x08
+
+ * GUID_L - Least significant 64 bits of a 128-bit Globally Unique Identifier.
+
+- Offset 0x10
+
+ * GUID_H - Most significant 64 bits of a 128-bit Globally Unique Identifier.
+
+- Offset 0x18
+
+ * Reg Address/Offset - If Rel bit is set, then the value is the high 63 bits
+ of a 16-bit aligned absolute address of the feature's registers. Otherwise
+ the value is the offset from the start of the DFH of the feature's registers.
+
+- Offset 0x20
+
+ * Reg Size - Size of feature's register set in bytes.
+ * Params - Set if DFH has a list of parameter blocks.
+ * Group - Id of group if feature is part of a group.
+ * Instance - Id of feature instance within a group.
+
+- Offset 0x28 if feature has parameters
+
+ * Next - Offset to the next parameter block in 8 byte words. If EOP set,
+ size in 8 byte words of last parameter.
+ * Param Version - Version of Param ID.
+ * Param ID - ID of parameter.
+
+- Offset 0x30
+
+ * Parameter Data - Parameter data whose size and format is defined by
+ version and ID of the parameter.
+
+
FIU - FME (FPGA Management Engine)
==================================
The FPGA Management Engine performs reconfiguration and other infrastructure
@@ -89,6 +209,8 @@ The following functions are exposed through ioctls:
- Program bitstream (DFL_FPGA_FME_PORT_PR)
- Assign port to PF (DFL_FPGA_FME_PORT_ASSIGN)
- Release port from PF (DFL_FPGA_FME_PORT_RELEASE)
+- Get number of irqs of FME global error (DFL_FPGA_FME_ERR_GET_IRQ_NUM)
+- Set interrupt trigger for FME error (DFL_FPGA_FME_ERR_SET_IRQ)
More functions are exposed through sysfs
(/sys/class/fpga_region/regionX/dfl-fme.n/):
@@ -149,6 +271,10 @@ The following functions are exposed through ioctls:
- Map DMA buffer (DFL_FPGA_PORT_DMA_MAP)
- Unmap DMA buffer (DFL_FPGA_PORT_DMA_UNMAP)
- Reset AFU (DFL_FPGA_PORT_RESET)
+- Get number of irqs of port error (DFL_FPGA_PORT_ERR_GET_IRQ_NUM)
+- Set interrupt trigger for port error (DFL_FPGA_PORT_ERR_SET_IRQ)
+- Get number of irqs of UINT (DFL_FPGA_PORT_UINT_GET_IRQ_NUM)
+- Set interrupt trigger for UINT (DFL_FPGA_PORT_UINT_SET_IRQ)
DFL_FPGA_PORT_RESET:
reset the FPGA Port and its AFU. Userspace can do Port
@@ -198,7 +324,7 @@ given Device Feature Lists and create platform devices for feature devices
also abstracts operations for the private features and exposes common ops to
feature device drivers.
-The FPGA DFL Device could be different hardwares, e.g. PCIe device, platform
+The FPGA DFL Device could be different hardware, e.g. PCIe device, platform
device and etc. Its driver module is always loaded first once the device is
created by the system. This driver plays an infrastructural role in the
driver architecture. It locates the DFLs in the device memory, handles them
@@ -304,7 +430,7 @@ The driver organization in virtualization case is illustrated below:
| PCI PF Device | | | PCI VF Device |
+---------------+ | +---------------+
-FPGA PCIe device driver is always loaded first once a FPGA PCIe PF or VF device
+FPGA PCIe device driver is always loaded first once an FPGA PCIe PF or VF device
is detected. It:
* Finishes enumeration on both FPGA PCIe PF and VF device using common
@@ -462,6 +588,19 @@ since they are system-wide counters on FPGA device.
The current driver does not support sampling. So "perf record" is unsupported.
+Interrupt support
+=================
+Some FME and AFU private features are able to generate interrupts. As mentioned
+above, users could call ioctl (DFL_FPGA_*_GET_IRQ_NUM) to know whether or how
+many interrupts are supported for this private feature. Drivers also implement
+an eventfd based interrupt handling mechanism for users to get notified when
+interrupt happens. Users could set eventfds to driver via
+ioctl (DFL_FPGA_*_SET_IRQ), and then poll/select on these eventfds waiting for
+notification.
+In Current DFL, 3 sub features (Port error, FME global error and AFU interrupt)
+support interrupts.
+
+
Add new FIUs support
====================
It's possible that developers made some new function blocks (FIUs) under this
@@ -482,6 +621,64 @@ Developer only needs to provide a sub feature driver with matched feature id.
FME Partial Reconfiguration Sub Feature driver (see drivers/fpga/dfl-fme-pr.c)
could be a reference.
+Please refer to below link to existing feature id table and guide for new feature
+ids application.
+https://github.com/OPAE/dfl-feature-id
+
+
+Location of DFLs on a PCI Device
+================================
+The original method for finding a DFL on a PCI device assumed the start of the
+first DFL to offset 0 of bar 0. If the first node of the DFL is an FME,
+then further DFLs in the port(s) are specified in FME header registers.
+Alternatively, a PCIe vendor specific capability structure can be used to
+specify the location of all the DFLs on the device, providing flexibility
+for the type of starting node in the DFL. Intel has reserved the
+VSEC ID of 0x43 for this purpose. The vendor specific
+data begins with a 4 byte vendor specific register for the number of DFLs followed 4 byte
+Offset/BIR vendor specific registers for each DFL. Bits 2:0 of Offset/BIR register
+indicates the BAR, and bits 31:3 form the 8 byte aligned offset where bits 2:0 are
+zero.
+::
+
+ +----------------------------+
+ |31 Number of DFLS 0|
+ +----------------------------+
+ |31 Offset 3|2 BIR 0|
+ +----------------------------+
+ . . .
+ +----------------------------+
+ |31 Offset 3|2 BIR 0|
+ +----------------------------+
+
+Being able to specify more than one DFL per BAR has been considered, but it
+was determined the use case did not provide value. Specifying a single DFL
+per BAR simplifies the implementation and allows for extra error checking.
+
+
+Userspace driver support for DFL devices
+========================================
+The purpose of an FPGA is to be reprogrammed with newly developed hardware
+components. New hardware can instantiate a new private feature in the DFL, and
+then present a DFL device in the system. In some cases users may need a
+userspace driver for the DFL device:
+
+* Users may need to run some diagnostic test for their hardware.
+* Users may prototype the kernel driver in user space.
+* Some hardware is designed for specific purposes and does not fit into one of
+ the standard kernel subsystems.
+
+This requires direct access to MMIO space and interrupt handling from
+userspace. The uio_dfl module exposes the UIO device interfaces for this
+purpose.
+
+Currently the uio_dfl driver only supports the Ether Group sub feature, which
+has no irq in hardware. So the interrupt handling is not added in this driver.
+
+UIO_DFL should be selected to enable the uio_dfl module driver. To support a
+new DFL feature via UIO direct access, its feature id should be added to the
+driver's id_table.
+
Open discussion
===============