| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
|
|
|
|
|
|
|
|
|
| |
Nobody uses this and it's impossible to maintain given the current CI
situation.
RHEL 7 and 8 release remain for now, though that might not always be the
case. See the link for details.
Link: https://lists.zx2c4.com/pipermail/wireguard/2022-June/007664.html
Suggested-by: Philip J. Perry <phil@elrepo.org>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
|
|
|
| |
Also bump the c8s version stamp.
Reported-by: Vladimír Beneš <vbenes@redhat.com>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
| |
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When we try to transmit an skb with md_dst attached through wireguard
we hit a null pointer dereference in wg_xmit() due to the use of
dst_mtu() which calls into dst_blackhole_mtu() which in turn tries to
dereference dst->dev.
Since wireguard doesn't use md_dsts we should use skb_valid_dst(), which
checks for DST_METADATA flag, and if it's set, then falls back to
wireguard's device mtu. That gives us the best chance of transmitting
the packet; otherwise if the blackhole netdev is used we'd get
ETH_MIN_MTU.
[ 263.693506] BUG: kernel NULL pointer dereference, address: 00000000000000e0
[ 263.693908] #PF: supervisor read access in kernel mode
[ 263.694174] #PF: error_code(0x0000) - not-present page
[ 263.694424] PGD 0 P4D 0
[ 263.694653] Oops: 0000 [#1] PREEMPT SMP NOPTI
[ 263.694876] CPU: 5 PID: 951 Comm: mausezahn Kdump: loaded Not tainted 5.18.0-rc1+ #522
[ 263.695190] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1.fc35 04/01/2014
[ 263.695529] RIP: 0010:dst_blackhole_mtu+0x17/0x20
[ 263.695770] Code: 00 00 00 0f 1f 44 00 00 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 8b 47 10 48 83 e0 fc 8b 40 04 85 c0 75 09 48 8b 07 <8b> 80 e0 00 00 00 c3 66 90 0f 1f 44 00 00 48 89 d7 be 01 00 00 00
[ 263.696339] RSP: 0018:ffffa4a4422fbb28 EFLAGS: 00010246
[ 263.696600] RAX: 0000000000000000 RBX: ffff8ac9c3553000 RCX: 0000000000000000
[ 263.696891] RDX: 0000000000000401 RSI: 00000000fffffe01 RDI: ffffc4a43fb48900
[ 263.697178] RBP: ffffa4a4422fbb90 R08: ffffffff9622635e R09: 0000000000000002
[ 263.697469] R10: ffffffff9b69a6c0 R11: ffffa4a4422fbd0c R12: ffff8ac9d18b1a00
[ 263.697766] R13: ffff8ac9d0ce1840 R14: ffff8ac9d18b1a00 R15: ffff8ac9c3553000
[ 263.698054] FS: 00007f3704c337c0(0000) GS:ffff8acaebf40000(0000) knlGS:0000000000000000
[ 263.698470] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 263.698826] CR2: 00000000000000e0 CR3: 0000000117a5c000 CR4: 00000000000006e0
[ 263.699214] Call Trace:
[ 263.699505] <TASK>
[ 263.699759] wg_xmit+0x411/0x450
[ 263.700059] ? bpf_skb_set_tunnel_key+0x46/0x2d0
[ 263.700382] ? dev_queue_xmit_nit+0x31/0x2b0
[ 263.700719] dev_hard_start_xmit+0xd9/0x220
[ 263.701047] __dev_queue_xmit+0x8b9/0xd30
[ 263.701344] __bpf_redirect+0x1a4/0x380
[ 263.701664] __dev_queue_xmit+0x83b/0xd30
[ 263.701961] ? packet_parse_headers+0xb4/0xf0
[ 263.702275] packet_sendmsg+0x9a8/0x16a0
[ 263.702596] ? _raw_spin_unlock_irqrestore+0x23/0x40
[ 263.702933] sock_sendmsg+0x5e/0x60
[ 263.703239] __sys_sendto+0xf0/0x160
[ 263.703549] __x64_sys_sendto+0x20/0x30
[ 263.703853] do_syscall_64+0x3b/0x90
[ 263.704162] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 263.704494] RIP: 0033:0x7f3704d50506
[ 263.704789] Code: 48 c7 c0 ff ff ff ff eb b7 66 2e 0f 1f 84 00 00 00 00 00 90 41 89 ca 64 8b 04 25 18 00 00 00 85 c0 75 11 b8 2c 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 72 c3 90 55 48 83 ec 30 44 89 4c 24 2c 4c 89
[ 263.705652] RSP: 002b:00007ffe954b0b88 EFLAGS: 00000246 ORIG_RAX: 000000000000002c
[ 263.706141] RAX: ffffffffffffffda RBX: 0000558bb259b490 RCX: 00007f3704d50506
[ 263.706544] RDX: 000000000000004a RSI: 0000558bb259b7b2 RDI: 0000000000000003
[ 263.706952] RBP: 0000000000000000 R08: 00007ffe954b0b90 R09: 0000000000000014
[ 263.707339] R10: 0000000000000000 R11: 0000000000000246 R12: 00007ffe954b0b90
[ 263.707735] R13: 000000000000004a R14: 0000558bb259b7b2 R15: 0000000000000001
[ 263.708132] </TASK>
[ 263.708398] Modules linked in: bridge netconsole bonding [last unloaded: bridge]
[ 263.708942] CR2: 00000000000000e0
Link: https://github.com/cilium/cilium/issues/19428
Reported-by: Martynas Pumputis <m@lambda.lt>
Signed-off-by: Nikolay Aleksandrov <razor@blackwall.org>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
[Jason: polyfilled for < 4.3]
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
|
|
|
|
|
|
|
|
| |
We make too nuanced use of ptr_ring to entirely move to the skb_array
wrappers, but we at least should avoid the naughty function pointer cast
when cleaning up skbs. Otherwise RAP/CFI will honk at us. This patch
uses the __skb_array_destroy_skb wrapper for the cleanup, rather than
directly providing kfree_skb, which is what other drivers in the same
situation do too.
Reported-by: PaX Team <pageexec@freemail.hu>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
|
|
|
| |
It's been over a year since we announced sunsetting this.
Link: https://lore.kernel.org/wireguard/CAHmME9rckipsdZYW+LA=x6wCMybdFFA+VqoogFXnR=kHYiCteg@mail.gmail.com/T
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The comment to sk_change_net is instructive:
Kernel sockets, f.e. rtnl or icmp_socket, are a part of a namespace.
They should not hold a reference to a namespace in order to allow
to stop it.
Sockets after sk_change_net should be released using sk_release_kernel
We weren't following these rules before, and were instead using
__sock_create, which means we kept a reference to the namespace, which
in turn meant that interfaces were not cleaned up on namespace
exit.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
On ARM v6 and later, we define CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
because the ordinary load/store instructions (ldr, ldrh, ldrb) can
tolerate any misalignment of the memory address. However, load/store
double and load/store multiple instructions (ldrd, ldm) may still only
be used on memory addresses that are 32-bit aligned, and so we have to
use the CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS macro with care, or we
may end up with a severe performance hit due to alignment traps that
require fixups by the kernel. Testing shows that this currently happens
with clang-13 but not gcc-11. In theory, any compiler version can
produce this bug or other problems, as we are dealing with undefined
behavior in C99 even on architectures that support this in hardware,
see also https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100363.
Fortunately, the get_unaligned() accessors do the right thing: when
building for ARMv6 or later, the compiler will emit unaligned accesses
using the ordinary load/store instructions (but avoid the ones that
require 32-bit alignment). When building for older ARM, those accessors
will emit the appropriate sequence of ldrb/mov/orr instructions. And on
architectures that can truly tolerate any kind of misalignment, the
get_unaligned() accessors resolve to the leXX_to_cpup accessors that
operate on aligned addresses.
Since the compiler will in fact emit ldrd or ldm instructions when
building this code for ARM v6 or later, the solution is to use the
unaligned accessors unconditionally on architectures where this is
known to be fast. The _aligned version of the hash function is
however still needed to get the best performance on architectures
that cannot do any unaligned access in hardware.
This new version avoids the undefined behavior and should produce
the fastest hash on all architectures we support.
Reported-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com>
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
|
|
|
| |
Use 2-factor argument form kvcalloc() instead of kvzalloc().
Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Each peer's endpoint contains a dst_cache entry that takes a reference
to another netdev. When the containing namespace exits, we take down the
socket and prevent future sockets from being created (by setting
creating_net to NULL), which removes that potential reference on the
netns. However, it doesn't release references to the netns that a netdev
cached in dst_cache might be taking, so the netns still might fail to
exit. Since the socket is gimped anyway, we can simply clear all the
dst_caches (by way of clearing the endpoint src), which will release all
references.
However, the current dst_cache_reset function only releases those
references lazily. But it turns out that all of our usages of
wg_socket_clear_peer_endpoint_src are called from contexts that are not
exactly high-speed or bottle-necked. For example, when there's
connection difficulty, or when userspace is reconfiguring the interface.
And in particular for this patch, when the netns is exiting. So for
those cases, it makes more sense to call dst_release immediately. For
that, we add a small helper function to dst_cache.
This patch also adds a test to netns.sh from Hangbin Liu to ensure this
doesn't regress.
Test-by: Hangbin Liu <liuhangbin@gmail.com>
Reported-by: Xiumei Mu <xmu@redhat.com>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
|
|
|
|
| |
RHEL 8.5 has been released. Replace all ISCENTOS8S checks with ISRHEL8.
Increase RHEL_MINOR for CentOS 8 Stream detection to 6.
Signed-off-by: Peter Georg <peter.georg@physik.uni-regensburg.de>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
grsecurity kernels tend to carry additional backports and changes, like
commit b60b87fc2996 ("netlink: add ethernet address policy types") or
the SYM_FUNC_* changes. RAP nowadays hooks the latter, therefore no
diversion to RAP_ENTRY is needed any more.
Instead of relying on the kernel version test, also test for the macros
we're about to define to not already be defined to account for these
additional changes in the grsecurity patch without breaking
compatibility to the older public ones.
Also test for CONFIG_PAX instead of RAP_PLUGIN for the timer API related
changes as these don't depend on the RAP plugin to be enabled but just a
PaX/grsecurity patch to be applied. While there is no preprocessor knob
for the latter, use CONFIG_PAX as this will likely be enabled in every
kernel that uses the patch.
Signed-off-by: Mathias Krause <minipli@grsecurity.net>
[zx2c4: small changes to include a header nearby a macro def test]
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
| |
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This reverts commit cad80597c7947f0def83caf8cb56aff0149c83a8.
Because this commit has not been backported so far, due to the implications
of building Ubuntu's backport of wireguard in a timely manner.
For now, reverting this fix would allow wireguard-linux-compat CI to work
on Ubuntu 18.04.
A different fix or the same one can be applied again when the time is
right.
Signed-off-by: Thadeu Lima de Souza Cascardo <cascardo@canonical.com>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
|
|
|
|
|
|
| |
CentOS Stream 8 by now (4.18.0-301.1.el8) reports RHEL_MINOR=5. The
current RHEL 8 minor release is still 3. RHEL 8.4 is in beta. Replace
equal comparison by greater equal to (hopefully) be a little bit more
future proof.
Signed-off-by: Peter Georg <peter.georg@physik.uni-regensburg.de>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
| |
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
|
|
|
| |
This corresponds to the fancier upstream commit that's still on lkml,
which passes a zeroed ip_options struct to __icmp_send.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
|
|
|
|
|
| |
linux commit 22f6bbb7bcfcef0b373b0502a7ff390275c575dd ("net: use
skb_list_del_init() to remove from RX sublists") will be backported to Ubuntu
18.04 default kernel, which is based on linux 4.15.
Signed-off-by: Thadeu Lima de Souza Cascardo <cascardo@canonical.com>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Having two ring buffers per-peer means that every peer results in two
massive ring allocations. On an 8-core x86_64 machine, this commit
reduces the per-peer allocation from 18,688 bytes to 1,856 bytes, which
is an 90% reduction. Ninety percent! With some single-machine
deployments approaching 500,000 peers, we're talking about a reduction
from 7 gigs of memory down to 700 megs of memory.
In order to get rid of these per-peer allocations, this commit switches
to using a list-based queueing approach. Currently GSO fragments are
chained together using the skb->next pointer (the skb_list_* singly
linked list approach), so we form the per-peer queue around the unused
skb->prev pointer (which sort of makes sense because the links are
pointing backwards). Use of skb_queue_* is not possible here, because
that is based on doubly linked lists and spinlocks. Multiple cores can
write into the queue at any given time, because its writes occur in the
start_xmit path or in the udp_recv path. But reads happen in a single
workqueue item per-peer, amounting to a multi-producer, single-consumer
paradigm.
The MPSC queue is implemented locklessly and never blocks. However, it
is not linearizable (though it is serializable), with a very tight and
unlikely race on writes, which, when hit (some tiny fraction of the
0.15% of partial adds on a fully loaded 16-core x86_64 system), causes
the queue reader to terminate early. However, because every packet sent
queues up the same workqueue item after it is fully added, the worker
resumes again, and stopping early isn't actually a problem, since at
that point the packet wouldn't have yet been added to the encryption
queue. These properties allow us to avoid disabling interrupts or
spinning. The design is based on Dmitry Vyukov's algorithm [1].
Performance-wise, ordinarily list-based queues aren't preferable to
ringbuffers, because of cache misses when following pointers around.
However, we *already* have to follow the adjacent pointers when working
through fragments, so there shouldn't actually be any change there. A
potential downside is that dequeueing is a bit more complicated, but the
ptr_ring structure used prior had a spinlock when dequeueing, so all and
all the difference appears to be a wash.
Actually, from profiling, the biggest performance hit, by far, of this
commit winds up being atomic_add_unless(count, 1, max) and atomic_
dec(count), which account for the majority of CPU time, according to
perf. In that sense, the previous ring buffer was superior in that it
could check if it was full by head==tail, which the list-based approach
cannot do.
But all and all, this enables us to get massive memory savings, allowing
WireGuard to scale for real world deployments, without taking much of a
performance hit.
[1] http://www.1024cores.net/home/lock-free-algorithms/queues/intrusive-mpsc-node-based-queue
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
|
|
|
|
| |
With the 4.4.256 and 4.9.256 kernels, the previous calculation for
integer comparison overflowed. This commit redefines the broken
constants to have more space for the sublevel.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
|
|
| |
We don't need this in all files, and it just complicates things.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
| |
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
| |
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
|
|
|
|
|
| |
The 5.4 series of -rt kernels moved from PREEMPT_RT_BASE/PREEMPT_RT_FULL
to PREEMPT_RT, so we have to account for it here. Otherwise users get
scheduling-while-atomic splats.
Reported-by: Erik Schuitema <erik@essd.nl>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
| |
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
| |
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
| |
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
| |
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
|
|
|
|
| |
Now that WireGuard is properly supported by 15.2 and people have had
sufficient time to upgrade, we can drop support for 15.1 in this compat
module.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
| |
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
| |
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
|
| |
Reported-by: Vladimir Benes <vbenes@redhat.com>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
|
|
|
| |
These are required for moving wg_examine_packet_protocol out of
wireguard and into upstream.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
|
|
| |
>=15.2 is in SUSE's kernel now.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
|
| |
Reported-by: Vladimir Benes <vbenes@redhat.com>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Before, we took a reference to the creating netns if the new netns was
different. This caused issues with circular references, with two
wireguard interfaces swapping namespaces. The solution is to rather not
take any extra references at all, but instead simply invalidate the
creating netns pointer when that netns is deleted.
In order to prevent this from happening again, this commit improves the
rough object leak tracking by allowing it to account for created and
destroyed interfaces, aside from just peers and keys. That then makes it
possible to check for the object leak when having two interfaces take a
reference to each others' namespaces.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
| |
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
|
|
|
|
| |
The 42.x series is no longer supported, and the 15.2 kernel is getting
a proper backport, so at the moment, we only care about supporting 15.1.
Eventually we'll drop that too.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
| |
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
| |
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
| |
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
| |
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
| |
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
| |
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
| |
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
| |
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
| |
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
| |
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
|
|
|
| |
Also remove the confusing 119/118 distinction from the Debian clause,
which is no longer as important.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
| |
|
|
| |
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|