aboutsummaryrefslogtreecommitdiffstats
path: root/drivers/net/ethernet/sfc/rx.c (follow)
AgeCommit message (Collapse)AuthorFilesLines
2016-05-31sfc: Track RPS flow IDs per channel instead of per functionJon Cooper1-8/+21
Otherwise we get confused when two flows on different channels get the same flow ID. Signed-off-by: Edward Cree <ecree@solarflare.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-05-29sfc: use flow dissector helpers for aRFSEdward Cree1-54/+21
Signed-off-by: Edward Cree <ecree@solarflare.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2015-11-18net: move skb_mark_napi_id() into core networking stackEric Dumazet1-1/+0
We would like to automatically provide busy polling support to all NAPI drivers, without them having to implement anything. skb_mark_napi_id() can be called from napi_gro_receive() and napi_get_frags(). Few drivers are still calling skb_mark_napi_id() because they use netif_receive_skb(). They should eventually call napi_gro_receive() instead. I will leave this to drivers maintainers. Signed-off-by: Eric Dumazet <edumazet@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2015-05-31sfc: free multiple Rx buffers when requiredDaniel Pieczko1-17/+25
When Rx packet data must be dropped, all the buffers associated with that Rx packet must be freed. Extend and rename efx_free_rx_buffer() to efx_free_rx_buffers() and loop through all the fragments. By doing so this patch fixes a possible memory leak. Signed-off-by: Shradha Shah <sshah@solarflare.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2014-07-22sfc: Add support for busy pollingAlexandre Rames1-1/+5
This patch adds the sfc driver code for implementing busy polling. It adds ndo_busy_poll method and locking between it and napi poll. It also adds each napi to the napi_hash right after netif_napi_add(). Uses efx_start_eventq and efx_stop_eventq in the self tests. Signed-off-by: Shradha Shah <sshah@solarflare.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2014-07-17sfc: Add per-queue statistics in ethtoolAndrew Rybchenko1-0/+2
Implement per channel software TX and RX packet counters accessed as ethtool statistics. This allows confirmation with MAC statistics. Signed-off-by: Shradha Shah <sshah@solarflare.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2014-07-15sfc: add extra RX drop counters for nodesc_trunc and noskb_dropEdward Cree1-1/+3
Added a counter rx_noskb_drop for failure to allocate an skb. Summed the per-channel rx_nodesc_trunc counters earlier so that they can be included in rx_dropped. Signed-off-by: Edward Cree <ecree@solarflare.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2013-12-18net: sfc calls skb_set_hashTom Herbert1-1/+2
Drivers should call skb_set_hash to set the hash and its type in an skbuff. Signed-off-by: Tom Herbert <therbert@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2013-12-12sfc: Add RX packet timestamping for EF10Jon Cooper1-0/+2
The EF10 firmware can optionally insert RX timestamps in the packet prefix. These only include the clock minor value. We must also enable periodic time sync events on each event queue which provide the high bits of the clock value. [bwh: Combined and rebased several changes. Added the above description and some sanity checks for inline vs separate timestamps. Changed efx_rx_skb_attach_timestamp() to read the packet prefix from the skb head area.] Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-12-12sfc: Copy RX prefix into skb head area in efx_rx_mk_skb()Ben Hutchings1-3/+7
We can potentially pull the entire packet contents into the head area and then free the page it was in. In order to read an inline timestamp safely, we need to copy the prefix into the head area as well. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-12-12sfc: Make initial fill of RX descriptors synchronousJon Cooper1-4/+5
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-12-06sfc: RX buffer allocation takes prefix size into account in IP header alignmentAndrew Rybchenko1-3/+3
rx_prefix_size is 4-bytes aligned on Falcon/Siena (16 bytes), but it is equal to 14 on EF10. So, it should be taken into account if arch requires IP header to be 4-bytes aligned (via NET_IP_ALIGN). Fixes: 8127d661e77f ('sfc: Add support for Solarflare SFC9100 family') Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-09-20sfc: Support ARFS for IPv6 flowsBen Hutchings1-28/+62
Extend efx_filter_rfs() to map TCP/IPv6 and UDP/IPv6 flows into efx_filter_spec. These are only supported on EF10; on Falcon and Siena they will be rejected by efx_farch_filter_from_gen_spec(). Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-08-29sfc: Update copyright bannersBen Hutchings1-2/+2
Update the dates for files that have been added to in 2012-2013. Drop the 'Solarstorm' brand name that's still lingering here. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-08-29sfc: Prepare for RX scatter on EF10Jon Cooper1-4/+4
RX DMA scatter is always enabled on EF10. Adjust the common RX completion handling to allow for this. RX completion events on EF10 include the length used from a single descriptor, not the cumulative length used. Add a field to struct efx_rx_queue to hold the cumulative length. [bwh: Also fix a related comment] Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-08-29sfc: Document conditions for multicast replication vs filter replacementBen Hutchings1-0/+34
Add the efx_filter_is_mc_recip() function to decide whether a filter is for a multicast recipient and can coexist with other filters with the same match values. Update efx_filter_insert_filter() kernel-doc to explain the conditions for this. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-08-27sfc: Add support for reading packet length from prefixBen Hutchings1-2/+10
Define a flag for struct efx_rx_buffer and efx_rx_packet() that indicates packet length must be read from the prefix. If this is set, read the length in __efx_rx_packet() (when the prefix should have arrived in cache). Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-08-27sfc: Generalise packet hash lookup to support EF10 RX prefixJon Cooper1-8/+7
EF10 uses an entirely different RX prefix format from Falcon-arch. Extend struct efx_nic_type to describe this. [bwh: Also replace the magic numbers used for the Falcon-arch RX prefix] Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-08-22sfc: Make most filter operations NIC-type-specificBen Hutchings1-0/+94
Aside from accelerated RFS, there is almost nothing that can be shared between the filter table implementations for the Falcon architecture and EF10. Move the few shared functions into efx.c and rx.c and the rest into farch.c. Introduce efx_nic_type operations for the implementation and inline wrapper functions that call these. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-08-21sfc: Refactor queue teardown sequence to allow for EF10 flush behaviourBen Hutchings1-1/+0
Currently efx_stop_datapath() will try to flush our DMA queues (if DMA is enabled), then finalise software and hardware state for each queue. However, for EF10 we must ask the MC to finalise each queue, which implicitly starts flushing it, and then wait for the flush events. We therefore need to delegate more of this to the NIC type. Combine all the hardware operations into a new NIC-type operation efx_nic_type::fini_dmaq, and call this before tearing down the software state and buffers for all the DMA queues. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-08-21sfc: Stop RX refill before flushing RX queuesBen Hutchings1-4/+4
rx_queue::enabled guards refill, so rename it to reflect that. Clear it at the start of the queue teardown process rather than waiting for the RX queue to be flushed. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-07-05sfc: Fix memory leak when discarding scattered packetsBen Hutchings1-7/+20
Commit 2768935a4660 ('sfc: reuse pages to avoid DMA mapping/unmapping costs') did not fully take account of DMA scattering which was introduced immediately before. If a received packet is invalid and must be discarded, we only drop a reference to the first buffer's page, but we need to drop a reference for each buffer the packet used. I think this bug was missed partly because efx_recycle_rx_buffers() was not renamed and so no longer does what its name says. It does not change the state of buffers, but only prepares the underlying pages for recycling. Rename it accordingly. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2013-06-24sfc: Improve test for IOMMU in useBen Hutchings1-1/+1
The device::iommu_group field may be set even if no IOMMU is in use. iommu_present() is still a better indicator, although it doesn't tell us whether *our* device is affected. Reported-by: Alex Williamson <alex.williamson@redhat.com> Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-06-24sfc: Do not pass non-TCP packets into GRO codeBen Hutchings1-1/+1
GRO can handle non-TCP packets and pass them up without coalescing, but it has to do some extra work to parse the packet which we can bypass using the hardware parse result. (This condition yields a false negative for TCP/IPv6 packets received by Falcon, but its performance is already poor in that case due to lack of checksum offload.) Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-06-24sfc: Increase size of RX SKB header areaJon Cooper1-1/+1
This allows the SKB to hold the headers without reallocation more often. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-06-24sfc: Enable RX checksum offload for packets not handled by GROJon Cooper1-0/+2
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-05-14sfc: Reduce RX scatter buffer size, and reduce alignment if appropriateBen Hutchings1-1/+1
efx_start_datapath() asserts that we can fit 2 RX scatter buffers plus a software structure, each appropriately aligned, into a single page. Where L1_CACHE_BYTES == 256 and PAGE_SIZE == 4096, which is the case on s390, this assertion fails. The current scatter buffer size is also not a multiple of 64 or 128, which are more common cache line sizes. If we can make both the start and end of a scatter buffer cache-aligned, this will reduce the need for read-modify-write operations on inter- processor links. Fix the alignment by reducing EFX_RX_USR_BUF_SIZE to 2048 - 256 == 1792. (We could use 2048 - L1_CACHE_BYTES, but EFX_RX_USR_BUF_SIZE also affects user-level networking where a larger amount of housekeeping data may be needed. Although this version of the driver does not support user-level networking, I prefer to keep scattering behaviour consistent with the out-of-tree version.) This still doesn't fix the s390 build because like most architectures it has NET_IP_ALIGN == 2. When NET_IP_ALIGN != 0 we cannot achieve cache line alignment at either the start or end of a scatter buffer, so there is actually no point in padding the buffers to a multiple of the cache line size. All we need is 4-byte alignment of the network header, so do that. Adjust the assertions accordingly. Reported-by: Geert Uytterhoeven <geert@linux-m68k.org> Reported-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Ben Hutchings <bhutchings@solarflare.com> Acked-by: Geert Uytterhoeven <geert@linux-m68k.org> Signed-off-by: David S. Miller <davem@davemloft.net>
2013-05-14sfc: Delete EFX_PAGE_IP_ALIGN, equivalent to NET_IP_ALIGNBen Hutchings1-3/+3
The two architectures that define CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS (powerpc and x86) now both define NET_IP_ALIGN as 0, so there is no need for this optimisation any more. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2013-03-17sfc: make local functions staticstephen hemminger1-2/+2
Trivial sparse detected functions that should be static. Signed-off-by: Stephen Hemminger <stephen@networkplumber.org> Signed-off-by: David S. Miller <davem@davemloft.net>
2013-03-07sfc: allocate more RX buffers per pageDaniel Pieczko1-38/+42
Allocating 2 buffers per page is insanely inefficient when MTU is 1500 and PAGE_SIZE is 64K (as it usually is on POWER). Allocate as many as we can fit, and choose the refill batch size at run-time so that we still always use a whole page at once. [bwh: Fix loop condition to allow for compound pages; rebase] Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-03-07sfc: Replace efx_rx_is_last_buffer() with a flagBen Hutchings1-11/+6
This condition is brittle and we have lots of flags to spare. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-03-07sfc: reuse pages to avoid DMA mapping/unmapping costsDaniel Pieczko1-94/+205
On POWER systems, DMA mapping/unmapping operations are very expensive. These changes reduce these costs by trying to reuse DMA mapped pages. After all the buffers associated with a page have been processed and passed up, the page is placed into a ring (if there is room). For each page that is required for a refill operation, a page in the ring is examined to determine if its page count has fallen to 1, ie. the kernel has released its reference to these packets. If this is the case, the page can be immediately added back into the RX descriptor ring, without having to re-map it for DMA. If the kernel is still holding a reference to this page, it is removed from the ring and unmapped for DMA. Then a new page, which can immediately be used by RX buffers in the descriptor ring, is allocated and DMA mapped. The time a page needs to spend in the recycle ring before the kernel has released its page references is based on the number of buffers that use this page. As large pages can hold more RX buffers, the RX recycle ring can be shorter. This reduces memory usage on POWER systems, while maintaining the performance gain achieved by recycling pages, following the driver change to pack more than two RX buffers into large pages. When an IOMMU is not present, the recycle ring can be small to reduce memory usage, since DMA mapping operations are inexpensive. With a small recycle ring, attempting to refill the descriptor queue with more buffers than the equivalent size of the recycle ring could ultimately lead to memory leaks if page entries in the recycle ring were overwritten. To prevent this, the check to see if the recycle ring is full is changed to check if the next entry to be written is NULL. [bwh: Combine and rebase several commits so this is complete before the following buffer-packing changes. Remove module parameter.] Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-03-07sfc: Enable RX DMA scattering where possibleBen Hutchings1-73/+138
Enable RX DMA scattering iff an RX buffer large enough for the current MTU will not fit into a single page and the NIC supports DMA scattering for kernel-mode RX queues. On Falcon and Siena, the RX_USR_BUF_SIZE field is used as the DMA limit for both all RX queues with scatter enabled. Set it to 1824, matching what Onload uses now. Maintain a statistic for frames truncated due to lack of descriptors (rx_nodesc_trunc). This is distinct from rx_frm_trunc which may be incremented when scattering is disabled and implies an over-length frame. Whenever an MTU change causes scattering to be turned on or off, update filters that point to the PF queues, but leave others unchanged, as VF drivers assume scattering is off. Add n_frags parameters to various functions, and make them iterate: - efx_rx_packet() - efx_recycle_rx_buffers() - efx_rx_mk_skb() - efx_rx_deliver() Make efx_handle_rx_event() responsible for updating efx_rx_queue::removed_count. Change the RX pipeline state to a starting ring index and number of fragments, and make __efx_rx_packet() responsible for clearing it. Based on earlier versions by David Riddoch and Jon Cooper. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-03-07sfc: Update RX buffer address together with lengthBen Hutchings1-12/+6
Adjust rx_buf->page_offset when we eat the RX hash prefix. Remove efx_rx_buf_offset(), which is now redundant. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-03-07sfc: Explicitly prefetch RX hash prefix, not just Ethernet headeBen Hutchings1-1/+1
Currently we prefetch from the Ethernet header, but we will also read the hash prefix. In practice they should be in the same cache line and this won't hurt, but it is still pointless to add on the hash prefix size. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-03-07sfc: Replace efx_rx_buf_eh() with simpler efx_rx_buf_va()Ben Hutchings1-4/+4
efx_rx_buf_va() returns the virtual address of the current start of the buffer. The callers must add the hash prefix size themselves. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-03-07sfc: Wrap __efx_rx_packet() with efx_rx_flush_packet()Ben Hutchings1-2/+1
The pipeline mechanism will need to change a bit for scattered packets. Add a wrapper to insulate efx_process_channel() from this. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-03-07sfc: Properly distinguish RX buffer and DMA lengthsBen Hutchings1-11/+8
Replace efx_nic::rx_buffer_len with efx_nic::rx_dma_len, the maximum RX DMA length. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-03-07sfc: Remove rx_alloc_method SKBAlexandre Rames1-235/+95
[bwh: Remove more dead code, and make efx_ptp_rx() pull the data it needs into the header area.] Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-03-07sfc: Allow efx_channel_type::receive_skb() to reject a packetBen Hutchings1-4/+6
Instead of having efx_ptp_rx() call netif_receive_skb() for an invalid PTP packet, make it return false for rejected packets and have efx_rx_deliver() pass them up. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-03-06sfc: Correct efx_rx_buffer::page_offset when EFX_PAGE_IP_ALIGN != 0Ben Hutchings1-1/+1
RX DMA buffers start at an offset of EFX_PAGE_IP_ALIGN bytes from the start of a cache line. This offset obviously needs to be included in the virtual address, but this was missed in commit b590ace09d51 ('sfc: Fix efx_rx_buf_offset() in the presence of swiotlb') since EFX_PAGE_IP_ALIGN is equal to 0 on both x86 and powerpc. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-02-26sfc: Fix efx_rx_buf_offset() in the presence of swiotlbBen Hutchings1-5/+5
We assume that the mapping between DMA and virtual addresses is done on whole pages, so we can find the page offset of an RX buffer using the lower bits of the DMA address. However, swiotlb maps in units of 2K, breaking this assumption. Add an explicit page_offset field to struct efx_rx_buffer. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2013-02-26sfc: Properly sync RX DMA buffer when it is not the last in the pageBen Hutchings1-5/+10
We may currently allocate two RX DMA buffers to a page, and only unmap the page when the second is completed. We do not sync the first RX buffer to be completed; this can result in packet loss or corruption if the last RX buffer completed in a NAPI poll is the first in a page and is not DMA-coherent. (In the middle of a NAPI poll, we will handle the following RX completion and unmap the page *before* looking at the content of the first buffer.) Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2012-12-01sfc: Delete redundant page_addr variable from efx_init_rx_buffers_page()Ben Hutchings1-5/+1
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2012-09-07sfc: Add channel specific receive_skb handler and post_remove callbackStuart Hodgson1-2/+11
Allows an extra channel to override the standard receive_skb handler and also for extra non generic operations to be performed on remove. Also set default rx strategy so only skbs can be delivered to the PTP receive function. Signed-off-by: Stuart Hodgson <smhodgson@solarflare.com> Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2012-09-07sfc: Add explicit RX queue flag to channelStuart Hodgson1-2/+5
The PTP channel will have its own RX queue even though it's not a regular traffic channel. Original work by Ben Hutchings <bhutchings@solarflare.com> Signed-off-by: Stuart Hodgson <smhodgson@solarflare.com> Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2012-07-18Merge branch 'for-davem' of git://git.kernel.org/pub/scm/linux/kernel/git/bwh/sfc-nextDavid S. Miller1-11/+11
Ben Hutchings says: ==================== 1. Fix potential badness when running a self-test with SR-IOV enabled. 2. Fix calculation of some interface statistics that could run backward. 3. Miscellaneous cleanup. ==================== Signed-off-by: David S. Miller <davem@davemloft.net>
2012-07-17sfc: Use generic DMA API, not PCI-DMA APIBen Hutchings1-11/+11
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
2012-07-10drivers/net/ethernet: Fix (nearly-)kernel-doc comments for various functionsBen Hutchings1-0/+1
Fix incorrect start markers, wrapped summary lines, missing section breaks, incorrect separators, and some name mismatches. Delete a few that are content-free. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com> Acked-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2012-05-10sfc: By default refill RX rings as soon as space for a batchDavid Riddoch1-6/+12
Previously we refilled with much larger batches, which caused large latency spikes. We now have many more much much smaller spikes! Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
Copyright © 1996 – 2023 Jason A. Donenfeld. All Rights Reverse Engineered.