qemu/include/hw/virtio, branch master

virtio-net: Fix network stall at the host side waiting for kick

2024-08-02T03:09:52Z

Patch 06b12970174 ("virtio-net: fix network stall under load") added double-check to test whether the available buffer size can satisfy the request or not, in case the guest has added some buffers to the avail ring simultaneously after the first check. It will be lucky if the available buffer size becomes okay after the double-check, then the host can send the packet to the guest. If the buffer size still can't satisfy the request, even if the guest has added some buffers, viritio-net would stall at the host side forever. The patch enables notification and checks whether the guest has added some buffers since last check of available buffers when the available buffers are insufficient. If no buffer is added, return false, else recheck the available buffers in the loop. If the available buffers are sufficient, disable notification and return true. Changes: 1. Change the return type of virtqueue_get_avail_bytes() from void to int, it returns an opaque that represents the shadow_avail_idx of the virtqueue on success, else -1 on error. 2. Add a new API: virtio_queue_enable_notification_and_check(), it takes an opaque as input arg which is returned from virtqueue_get_avail_bytes(). It enables notification firstly, then checks whether the guest has added some buffers since last check of available buffers or not by virtio_queue_poll(), return ture if yes. The patch also reverts patch "06b12970174". The case below can reproduce the stall. Guest 0 +--------+ | iperf | ---------------> | server | Host | +--------+ +--------+ | ... | iperf |---- | client |---- Guest n +--------+ | +--------+ | | iperf | ---------------> | server | +--------+ Boot many guests from qemu with virtio network: qemu ... -netdev tap,id=net_x \ -device virtio-net-pci-non-transitional,\ iommu_platform=on,mac=xx:xx:xx:xx:xx:xx,netdev=net_x Each guest acts as iperf server with commands below: iperf3 -s -D -i 10 -p 8001 iperf3 -s -D -i 10 -p 8002 The host as iperf client: iperf3 -c guest_IP -p 8001 -i 30 -w 256k -P 20 -t 40000 iperf3 -c guest_IP -p 8002 -i 30 -w 256k -P 20 -t 40000 After some time, the host loses connection to the guest, the guest can send packet to the host, but can't receive packet from the host. It's more likely to happen if SWIOTLB is enabled in the guest, allocating and freeing bounce buffer takes some CPU ticks, copying from/to bounce buffer takes more CPU ticks, compared with that there is no bounce buffer in the guest. Once the rate of producing packets from the host approximates the rate of receiveing packets in the guest, the guest would loop in NAPI. receive packets --- | | v | free buf virtnet_poll | | v | add buf to avail ring --- | | need kick the host? | NAPI continues v receive packets --- | | v | free buf virtnet_poll | | v | add buf to avail ring --- | v ... ... On the other hand, the host fetches free buf from avail ring, if the buf in the avail ring is not enough, the host notifies the guest the event by writing the avail idx read from avail ring to the event idx of used ring, then the host goes to sleep, waiting for the kick signal from the guest. Once the guest finds the host is waiting for kick singal (in virtqueue_kick_prepare_split()), it kicks the host. The host may stall forever at the sequences below: Host Guest ------------ ----------- fetch buf, send packet receive packet --- ... ... | fetch buf, send packet add buf | ... add buf virtnet_poll buf not enough avail idx-> add buf | read avail idx add buf | add buf --- receive packet --- write event idx ... | wait for kick add buf virtnet_poll ... | --- no more packet, exit NAPI In the first loop of NAPI above, indicated in the range of virtnet_poll above, the host is sending packets while the guest is receiving packets and adding buffers. step 1: The buf is not enough, for example, a big packet needs 5 buf, but the available buf count is 3. The host read current avail idx. step 2: The guest adds some buf, then checks whether the host is waiting for kick signal, not at this time. The used ring is not empty, the guest continues the second loop of NAPI. step 3: The host writes the avail idx read from avail ring to used ring as event idx via virtio_queue_set_notification(q->rx_vq, 1). step 4: At the end of the second loop of NAPI, recheck whether kick is needed, as the event idx in the used ring written by the host is beyound the range of kick condition, the guest will not send kick signal to the host. Fixes: 06b12970174 ("virtio-net: fix network stall under load") Cc: qemu-stable@nongnu.org Signed-off-by: Wencheng Yang Reviewed-by: Michael S. Tsirkin Signed-off-by: Jason Wang

Revert "virtio-pci: Implement SR-IOV PF"

2024-08-01T08:32:00Z

This reverts commit 3f868ffb0bae0c4feafabe34a371cded57fe3806. Signed-off-by: Michael S. Tsirkin

virtio-iommu: Remove probe_done

2024-07-23T00:15:42Z

Now we have switched to PCIIOMMUOps to convey host IOMMU information, the host reserved regions are transmitted when the PCIe topology is built. This happens way before the virtio-iommu driver calls the probe request. So let's remove the probe_done flag that allowed to check the probe was not done before the IOMMU MR got enabled. Besides this probe_done flag had a flaw wrt migration since it was not saved/restored. The only case at risk is if 2 devices were plugged to a PCIe to PCI bridge and thus aliased. First of all we discovered in the past this case was not properly supported for neither SMMU nor virtio-iommu on guest kernel side: see [RFC] virtio-iommu: Take into account possible aliasing in virtio_iommu_mr() https://lore.kernel.org/all/20230116124709.793084-1-eric.auger@redhat.com/ If this were supported by the guest kernel, it is unclear what the call sequence would be from a virtio-iommu driver point of view. Signed-off-by: Eric Auger Message-Id: <20240716094619.1713905-3-eric.auger@redhat.com> Tested-by: Cédric Le Goater Reviewed-by: Cédric Le Goater Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin

virtio-pci: Implement SR-IOV PF

2024-07-23T00:15:41Z

Allow user to attach SR-IOV VF to a virtio-pci PF. Signed-off-by: Akihiko Odaki Message-Id: <20240715-sriov-v5-6-3f5539093ffc@daynix.com> Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin

virtio: Add VIRTIO_F_IN_ORDER property definition

2024-07-21T18:45:56Z

Extend the virtio device property definitions to include the VIRTIO_F_IN_ORDER feature. The default state of this feature is disabled, allowing it to be explicitly enabled where it's supported. Acked-by: Eugenio Pérez Signed-off-by: Jonah Palmer Message-Id: <20240710125522.4168043-7-jonah.palmer@oracle.com> Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin

virtio: Add bool to VirtQueueElement

2024-07-21T18:45:56Z

Add the boolean 'in_order_filled' member to the VirtQueueElement structure. The use of this boolean will signify whether the element has been processed and is ready to be flushed (so long as the element is in-order). This boolean is used to support the VIRTIO_F_IN_ORDER feature. Reviewed-by: Eugenio Pérez Signed-off-by: Jonah Palmer Message-Id: <20240710125522.4168043-2-jonah.palmer@oracle.com> Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin

hw/virtio: Fix the de-initialization of vhost-user devices

2024-07-03T22:14:06Z

The unrealize functions of the various vhost-user devices are calling the corresponding vhost_*_set_status() functions with a status of 0 to shut down the device correctly. Now these vhost_*_set_status() functions all follow this scheme: bool should_start = virtio_device_should_start(vdev, status); if (vhost_dev_is_started(&vvc->vhost_dev) == should_start) { return; } if (should_start) { /* ... do the initialization stuff ... */ } else { /* ... do the cleanup stuff ... */ } The problem here is virtio_device_should_start(vdev, 0) currently always returns "true" since it internally only looks at vdev->started instead of looking at the "status" parameter. Thus once the device got started once, virtio_device_should_start() always returns true and thus the vhost_*_set_status() functions return early, without ever doing any clean-up when being called with status == 0. This causes e.g. problems when trying to hot-plug and hot-unplug a vhost user devices multiple times since the de-initialization step is completely skipped during the unplug operation. This bug has been introduced in commit 9f6bcfd99f ("hw/virtio: move vm_running check to virtio_device_started") which replaced should_start = status & VIRTIO_CONFIG_S_DRIVER_OK; with should_start = virtio_device_started(vdev, status); which later got replaced by virtio_device_should_start(). This blocked the possibility to set should_start to false in case the status flag VIRTIO_CONFIG_S_DRIVER_OK was not set. Fix it by adjusting the virtio_device_should_start() function to only consider the status flag instead of vdev->started. Since this function is only used in the various vhost_*_set_status() functions for exactly the same purpose, it should be fine to fix it in this central place there without any risk to change the behavior of other code. Fixes: 9f6bcfd99f ("hw/virtio: move vm_running check to virtio_device_started") Buglink: https://issues.redhat.com/browse/RHEL-40708 Signed-off-by: Thomas Huth Message-Id: <20240618121958.88673-1-thuth@redhat.com> Reviewed-by: Manos Pitsidianakis Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin

virtio-pci: implement No_Soft_Reset bit

2024-07-01T21:16:05Z

In current code, when guest does S3, virtio-gpu are reset due to the bit No_Soft_Reset is not set. After resetting, the display resources of virtio-gpu are destroyed, then the display can't come back and only show blank after resuming. Implement No_Soft_Reset bit of PCI_PM_CTRL register, then guest can check this bit, if this bit is set, the devices resetting will not be done, and then the display can work after resuming. No_Soft_Reset bit is implemented for all virtio devices, and was tested only on virtio-gpu device. Set it false by default for safety. Signed-off-by: Jiqian Chen Message-Id: <20240606102205.114671-3-Jiqian.Chen@amd.com> Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin

Revert "vhost-user: fix lost reconnect"

2024-07-01T21:16:04Z

This reverts commit f02a4b8e6431598612466f76aac64ab492849abf. Since the current patch cannot completely fix the lost reconnect problem, there is a scenario that is not considered: - When the virtio-blk driver is removed from the guest os, s->connected has no chance to be set to false, resulting in subsequent reconnection not being executed. The next patch will completely fix this issue with a better approach. Signed-off-by: Li Feng Message-Id: <20240516025753.130171-2-fengli@smartx.com> Reviewed-by: Raphael Norwitz Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin

virtio/virtio-pci: Handle extra notification data

2024-07-01T18:56:23Z

Add support to virtio-pci devices for handling the extra data sent from the driver to the device when the VIRTIO_F_NOTIFICATION_DATA transport feature has been negotiated. The extra data that's passed to the virtio-pci device when this feature is enabled varies depending on the device's virtqueue layout. In a split virtqueue layout, this data includes: - upper 16 bits: shadow_avail_idx - lower 16 bits: virtqueue index In a packed virtqueue layout, this data includes: - upper 16 bits: 1-bit wrap counter & 15-bit shadow_avail_idx - lower 16 bits: virtqueue index Signed-off-by: Jonah Palmer Message-Id: <20240315165557.26942-2-jonah.palmer@oracle.com> Reviewed-by: Eugenio Pérez Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin