diff options
author | Majd Dibbiny <majd@mellanox.com> | 2018-08-28 14:29:05 +0300 |
---|---|---|
committer | Jason Gunthorpe <jgg@mellanox.com> | 2018-09-04 16:26:14 -0600 |
commit | c6a21c3864fc7f5febae7d096cd136f397c791f2 (patch) | |
tree | fb1ca013b9bf978deeb65e749b6f05d7d4143204 /drivers/infiniband/hw/mlx5/mlx5_ib.h | |
parent | IB/hfi1: Move URGENT IRQ enable to hfi1_rcvctrl() (diff) | |
download | linux-dev-c6a21c3864fc7f5febae7d096cd136f397c791f2.tar.xz linux-dev-c6a21c3864fc7f5febae7d096cd136f397c791f2.zip |
IB/mlx5: Change TX affinity assignment in RoCE LAG mode
In the current code, the TX affinity is per RoCE device, which can cause
unfairness between different contexts. e.g. if we open two contexts, and
each open 10 QPs concurrently, all of the QPs of the first context might
end up on the first port instead of distributed on the two ports as
expected
To overcome this unfairness between processes, we maintain per device TX
affinity, and per process TX affinity.
The allocation algorithm is as follow:
1. Hold two tx_port_affinity atomic variables, one per RoCE device and one
per ucontext. Both initialized to 0.
2. In mlx5_ib_alloc_ucontext do:
2.1. ucontext.tx_port_affinity = device.tx_port_affinity
2.2. device.tx_port_affinity += 1
3. In modify QP INIT2RST:
3.1. qp.tx_port_affinity = ucontext.tx_port_affinity % MLX5_PORT_NUM
3.2. ucontext.tx_port_affinity += 1
Signed-off-by: Majd Dibbiny <majd@mellanox.com>
Reviewed-by: Moni Shoua <monis@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
Diffstat (limited to 'drivers/infiniband/hw/mlx5/mlx5_ib.h')
-rw-r--r-- | drivers/infiniband/hw/mlx5/mlx5_ib.h | 4 |
1 files changed, 3 insertions, 1 deletions
diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h index 320d4dfe8c2f..387a6ec1ce3b 100644 --- a/drivers/infiniband/hw/mlx5/mlx5_ib.h +++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h @@ -139,6 +139,8 @@ struct mlx5_ib_ucontext { u64 lib_caps; DECLARE_BITMAP(dm_pages, MLX5_MAX_MEMIC_PAGES); u16 devx_uid; + /* For RoCE LAG TX affinity */ + atomic_t tx_port_affinity; }; static inline struct mlx5_ib_ucontext *to_mucontext(struct ib_ucontext *ibucontext) @@ -699,7 +701,7 @@ struct mlx5_roce { rwlock_t netdev_lock; struct net_device *netdev; struct notifier_block nb; - atomic_t next_port; + atomic_t tx_port_affinity; enum ib_port_state last_port_state; struct mlx5_ib_dev *dev; u8 native_port_num; |