try to notify the partner when the port is going away or down.

by notify i mean we send an lacp packet with our collecting and
distributing flags cleared, which should tell the remote system
that it should no longer handle packets on their port as part of
their aggregation. this is implemented by "unselecting" a port.

if an active port is going away, ie, being removed from an aggr via
"ifconfig aggr0 -trunkport port0", all that happens is software
state on our side changes and we stop considering the interface as
part of the aggr interface. the partner system is otherwise oblivious
and can continue to send us packets until its expiry timeout fires
because it doesn't know any better.

we already intercept a ports ioctl handling, so if someone goes
"ifconfig portX down" while it is attached to an aggr, we can catch
that before the underlying driver actually tears the rings down, and
we still have a chance to try and send a packet to the peer. this
is useful because our drivers generally do not drop the physical
link, so again, the partner system is oblivious to the change on
our side until its expiry timer fires.

expiry timeouts can be up to 90 seconds away, which is a lot of
traffic to blackhole. sending the notification to the parnter means
they withdraw this link at the same time the local system is pulling
the port out of the aggregation. hopefully. it is possible the
packet is lost, but this is a good start.

the only caveat to this is is my implementation ignores the transmit
state machine from the lacp spec, and may cause more than 3 lacp
packets per second to be transmitted to the partner system. oh
well.

i should look at the marker protocol too.

1 files changed, 16 insertions, 1 deletions

diff --git a/sys/net/if_aggr.c b/sys/net/if_aggr.c
index 6cd96632981..f30c73b7785 100644
--- a/sys/net/if_aggr.c
+++ b/sys/net/if_aggr.c
@@ -1,4 +1,4 @@
-/*	$OpenBSD: if_aggr.c,v 1.15 2019/07/19 04:35:01 dlg Exp $ */
+/*	$OpenBSD: if_aggr.c,v 1.16 2019/07/19 05:21:25 dlg Exp $ */
 
 /*
  * Copyright (c) 2019 The University of Queensland
@@ -474,6 +474,7 @@ static void	aggr_ptm_tx(void *);
 
 static void	aggr_transmit_machine(void *);
 static void	aggr_ntt(struct aggr_port *);
+static void	aggr_ntt_transmit(struct aggr_port *);
 
 static void	aggr_set_selected(struct aggr_port *, enum aggr_port_selected,
 		    enum lacp_mux_event);
@@ -1307,6 +1308,16 @@ aggr_p_ioctl(struct ifnet *ifp0, u_long cmd, caddr_t data)
 		error = EBUSY;
 		break;
 
+	case SIOCSIFFLAGS:
+		if (!ISSET(ifp0->if_flags, IFF_UP) &&
+		    ISSET(ifp0->if_flags, IFF_RUNNING)) {
+			/* port is going down */
+			if (p->p_selected == AGGR_PORT_SELECTED) {
+				aggr_unselected(p);
+				aggr_ntt_transmit(p); /* XXX */
+			}
+		}
+		/* FALLTHROUGH */
 	default:
 		error = (*p->p_ioctl)(ifp0, cmd, data);
 		break;
@@ -1338,12 +1349,16 @@ aggr_p_dtor(struct aggr_softc *sc, struct aggr_port *p, const char *op)
 	struct ifnet *ifp0 = p->p_ifp0;
 	struct arpcom *ac0 = (struct arpcom *)ifp0;
 	struct aggr_multiaddr *ma;
+	enum aggr_port_selected selected;
 
 	DPRINTF(sc, "%s %s %s: destroying port\n",
 	    ifp->if_xname, ifp0->if_xname, op);
 
+	selected = p->p_selected;
 	aggr_rxm(sc, p, LACP_RXM_E_NOT_PORT_ENABLED);
 	aggr_unselected(p);
+	if (aggr_port_enabled(p) && selected == AGGR_PORT_SELECTED)
+		aggr_ntt_transmit(p);
 
 	timeout_del(&p->p_ptm_tx);
 	timeout_del_barrier(&p->p_txm_ntt); /* XXX */