if_wg: ensure peer lifetime

The peer (and keypair and local) lifecycle are managed through EPOCH and
refcounts. Primarily this is used in wg_noise to keep track of active
keypairs, however we can also use it to be sure no more peer references
exist. The structures are linked as such, so noise_remote cannot be
freed until all noise_keypairs are freed, and noise_local cannot be
freed until all noise_remotes are freed.

noise_keypair -> noise_remote -> noise_local

Therefore, if you hold a keypair reference you can be sure that remote
and local will still be around.

There are three main ways peers are referenced:

1) Incoming packets
1.a) Incoming handshake packets are passed to noise_consume_*,
     which will (on success) return a refcounted remote which is dropped
     at the end of wg_handshake.
1.b) Incoming cookie packets will have their index looked which will (on
     success) return a refcounted remote, which is also dropped at the
     end of wg_handshake.
1.c) Incoming data packets will have their index looked up which will
     (on success) return a refcounted keypair. This keypair will be
     dropped after the packet has been passed up the network stack, or
     otherwise freed.

2) Outgoing data packets
2.a) Outgoing data packets are first looked up by wg_aip_lookup, which
     returns a peer pointer, with an incremented remote refcount. This
     is then dropped in wg_transmit after adding the packet to the
     staged queue and sending the staged queue.
2.b) Packets in the staged queue do not hold any refcount for the remote
     or keypair, because they do not reference the peer in any way, they
     are just in the queue.
2.c) Packets finally get a refcoutned keypair in wg_peer_send_staged,
     which is dropped after the packet is sent out the UDP socket, or
     otherwise freed.

3) wg_timers system
3.a) The wg_timers system holds a reference to the peer whenever a
     callout is scheduled. Instead of holding a refcount, we instead
     disable the peer's timers, such that no callouts can be scheduled.

Some rationale for changes here:

We move the p_{send,recv} taskqgroup_detach into peer_free_deferred as
they will NULL fields in p_{send,recv}. If there are packets being
processed in wg_{en,de}crypt, then a call tou GROUPTASK_ENQUEUE will
dereference a NULL pointer. In general, we remove all references to the
peer in wg_peer_destroy, and free/deinit all the peer members once no
more references to the remote exist, in wg_peer_free_deferred.

Currently we take a refcount in wg_aip_lookup, which is to be sure that
the peer reference is valid for the entirety of wg_transmit. We do not
care about the refcount in wg_decrypt. It might be worth considering
storing the remote pointer in the allowedip entry, but it could be
argued both ways. For the time being, this is still correct. We don't
have a refcount for the peer stored in the allowedip table, as it is
protected by the table lock. One note here is the NULL p_remote check is
necessary to support selftest/allowedips.c, which does not specify a
p_remote. If we update the tests, then we may remove this check.

There are two added p_enabled checks, in run_retry_handshake and
run_send_keepalive. This is to align them with the other callout_reset
calls. In the case of p_zero_key_material, if we have set p_enabled =
false, then we subsequently clear keypairs and handshakes (on wg_down),
or we free the peer which will clear the keypairs for us.

We want to hold a refcount of remote in wg_{en,de}crypt to ensure that
the peer is still valid in the call to GROUPTASK_ENQUEUE. If we don't
then peer may become invalid after setting p_state. Another thread may
take the packet, put the keypair refcount and free the peer prior to the
call to GROUPTASK_ENQUEUE.

We no longer need to hold (haven't for a while) the EPOCH in
wg_send_initiation and wg_send_response, as we hold valid references for
the duration. This could be either a refcount of a remote or through the
wg_timers system as described above.

We also fix some refcount leaks in wgc_set.

Notes:

We may want to pull NET_EPOCH_WAIT out of wg_timers_disable, to improve
performance. However, we can destroy 20000 peers in less than 20ms so
the performance is not critical for this snapshot and can be addressed
later.

Finally, there is the special case of noise_remote_arg, which stores the
corresponding peer pointer. The peer is not refcounted however it will
have the same scope as the remote. In otherwords it is valid until we
call noise_remote_put on the remote.

Signed-off-by: Matt Dunwoodie <ncon@noconroy.net>

1 files changed, 0 insertions, 6 deletions

diff --git a/src/wg_noise.c b/src/wg_noise.c
index 1e18e61..5ef7a58 100644
--- a/src/wg_noise.c
+++ b/src/wg_noise.c
@@ -777,12 +777,6 @@ noise_keypair_remote(struct noise_keypair *kp)
 	return (noise_remote_ref(kp->kp_remote));
 }
 
-void *
-noise_keypair_remote_arg(struct noise_keypair *kp)
-{
-	return kp->kp_remote->r_arg;
-}
-
 int
 noise_keypair_nonce_next(struct noise_keypair *kp, uint64_t *send)
 {