diff options
authorJason A. Donenfeld <Jason@zx2c4.com>2018-09-16 02:40:08 +0200
committerJason A. Donenfeld <Jason@zx2c4.com>2019-03-25 13:09:52 +0100
commit12ffb5959788767eb4d27f13697e6f19b8988419 (patch)
parentnet: WireGuard secure network tunnel (diff)
[DO NOT UPSTREAM] integration tree maintainer scriptsjd/wireguard
People have been asking me how I'm keeping track of the 00/XX cover letter and syncing changes between the out-of-tree module repo and this repo, and how I deal with so many rebases. So this commit shows the scripts to do it. It obviously shouldn't find its way upstream. Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
13 files changed, 279 insertions, 0 deletions
diff --git a/.gitignore b/.gitignore
index a20ac26aa2f5..ee71040457ee 100644
--- a/.gitignore
+++ b/.gitignore
@@ -133,3 +133,5 @@ all.config
# Kdevelop4
diff --git a/changelog.txt b/changelog.txt
new file mode 100644
index 000000000000..6c41670bdf2a
--- /dev/null
+++ b/changelog.txt
@@ -0,0 +1,65 @@
+Changes v8->v9, along with who suggested it.
+- [EVERYBODY] Zinc no longer ships generated assembly code. Rather, we now
+ bundle in the original perlasm generator for it. This is ongoing joint work
+ with Andy Polyakov upstream, so that the same .pl files can live in our tree
+ as well as in the CRYPTOGAMS tree. I personally find that the code required
+ to share this in both repositories to be a tiny bit ugly. I think there would
+ be some degree of an advantage to removing that and making the .pl
+ kernel-only, and then carefully tracking Andy's changes (as we already
+ do). Previous opinions on the list, though, were that there's also
+ significant advantage to being able to share the exact same code in both.
+ And I think there's a decent amount of wisdom in that too. Since that
+ appeared to be the prevailing view, and since it also has good reasons
+ arguments, we'll go with that for now.
+ Meanwhile, Andy is recently back from some time away, and so we'll be
+ improving these even further in the months to come. Already there are a few
+ things ready that haven't been integrated here just yet.
+- [Eric Biggers] In Zinc introductory commit, add more details on what Zinc is
+ for and what the inclusion criteria are, as well as some notes on API.
+- Clarify the peer removal logic and make lifetimes more precise.
+- [Jann Horn] Use READ_ONCE for is_valid and is_dead.
+- [Jann Horn] No need to use atomic when the recounter is mutex protected.
+- [Andrew Lunn] Fix up macros and annotations in allowedips.
+- [Andrew Lunn] Increment drop counter when staged packets are dropped.
+- Use static constants instead of enums for 64-bit values in selftest.
+- Mark large constants as ULL in poly1305-donna64.
+- Fix sparse warnings in allowedips debugging code.
+- Do not use wg_peer_get_maybe_zero in timer callbacks, since we now can
+ carefully control the lifetime of these functions and ensure they never
+ execute after dropping the last reference.
+- Cleanup hashing in ratelimiter.
+- Do not guard timer removals, since del_timer is always okay.
+- [Theodore Ts'o, Andrew Lunn, Sultan Alsawaf] We now check for PM_AUTOSLEEP,
+ which makes the clear-on-suspend decision a bit more general.
+- Set csum_level to ~0, since the poly1305 authenticator certainly means
+ that no data was modified in transit.
+- [Andrejs Hanins] Use CHECKSUM_PARTIAL check for skb_checksum_help instead
+ of skb_checksum_setup check.
+- [Ard Biesheuvel] Enable the selftests by default, and leave it to
+ small-system builders to disable this.
+- [Sultan Alsawaf] We no longer allocate large hashtables as part of the
+ net_device structure.
+- Iteration of allowed IPs per peer now uses a normal boring fast linked
+ list instead of needing to traverse a giant tree. This makes retrieving
+ peer properties over netlink several orders of magnitude faster.
+- The removal of all peers now includes two iterations through the peer
+ list with a single synchronize_rcu_bh() in between iterations, as opposed
+ to having one iteration, but a synchronize_rcu_bh() for every single peer.
+- [Arnd Bergmann] 64-bit timestamps.
+- The test suite now checks the validity of timestamps and checks that private
+ keys are being properly clamped.
+- Numerous API changes made in-tree since v8, such as shedding _bh() on a few
+ functions and networking API changes and such.
+- Following the Adiantum merge, the two commits that port the old crypto API
+ over to use Zinc have been removed from this series. We can certainly add
+ them back in at some point, but I thought it'd be favorable to at least
+ begin to receive some sign-offs on the Zinc-specific commits, now that
+ (hopefully all of) the previous feedback has been taken care of. The two
+ commits porting it over are fairly standalone as well, so that shouldn't
+ impact the ability to review this. For now those are living in the
+ jd/with-cryptoapi-port branch of kernel.org's zx2c4/linux.git tree. This
+ also allows us to move this all forward a little bit.
diff --git a/diff-all.sh b/diff-all.sh
new file mode 100755
index 000000000000..a986dcd6dbb7
--- /dev/null
+++ b/diff-all.sh
@@ -0,0 +1,27 @@
+SELF="$(readlink -f "${BASH_SOURCE[0]}")"
+diff_it() {
+ local base="$1"
+ local foreign="$2"
+ local file="$3"
+ file="${file#"$base"}"
+ [[ -f $base/$file && -f $foreign/$file ]] && git --no-pager diff --color=always --no-index "$base/$file" "$foreign/$file"
+for i in "$WG/src"/*.c "$WG/src"/*.h; do
+ diff_it "$WG/src" "$IT/drivers/net/wireguard" "$i"
+for i in $(find "$WG/src/selftest" -type f); do
+ diff_it "$WG/src/selftest" "$IT/drivers/net/wireguard/selftest" "$i"
+for i in $(find "$WG/src/crypto/zinc" -type f); do
+ diff_it "$WG/src/crypto/zinc" "$IT/lib/zinc" "$i"
+for i in $(find "$WG/src/crypto/include/zinc" -type f); do
+ diff_it "$WG/src/crypto/include/zinc" "$IT/include/zinc" "$i"
+diff_it "$WG/src/tests" "$IT/tools/testing/selftests/wireguard" "netns.sh"
diff --git a/make-series.sh b/make-series.sh
new file mode 100755
index 000000000000..659fcaf953f0
--- /dev/null
+++ b/make-series.sh
@@ -0,0 +1,12 @@
+SELF="$(readlink -f "${BASH_SOURCE[0]}")"
+cd "${SELF%/*}"
+set -ex
+[[ $1 =~ ^v[0-9]+$ ]] || { echo "ERROR: pass the version directory as an argument" >&2; exit 1; }
+mkdir -p "$1"
+rm -fv "$1"/*.patch
+git format-patch -o "$1" --notes --cover-letter --subject-prefix="PATCH net-next $1" net-next/master..master~
+sed 's/\*\*\* SUBJECT HERE \*\*\*/WireGuard: Secure Network Tunnel/' "$1/0000-cover-letter.patch" | head -n 8 > "$1/0000-cover-letter.patch.tmp"
+cat "$1/0000-cover-letter.patch.tmp" "changelog.txt" "zero-zero-text.txt" > "$1/0000-cover-letter.patch"
+rm -f "$1/0000-cover-letter.patch.tmp"
diff --git a/merge-latest.sh b/merge-latest.sh
new file mode 100755
index 000000000000..239da02c13c8
--- /dev/null
+++ b/merge-latest.sh
@@ -0,0 +1,13 @@
+SELF="$(readlink -f "${BASH_SOURCE[0]}")"
+set -x
+git -C "$WG" diff where-we-left-off..master "$WG/src"/*.c "$WG/src"/*.h "$WG/src/selftest/" | patch -d "$IT/drivers/net/wireguard" -p2
+git -C "$WG" diff where-we-left-off..master "$WG/src/uapi" | patch -d "$IT/include/uapi/linux" -p3
+git -C "$WG" diff where-we-left-off..master "$WG/src/crypto/zinc" | patch -d "$IT/lib/zinc" -p4
+git -C "$WG" diff where-we-left-off..master "$WG/src/crypto/include/zinc" | patch -d "$IT/include/zinc" -p5
+cp "$WG/src/tests/netns.sh" "$IT/tools/testing/selftests/wireguard/netns.sh"
+git -C "$WG" tag -f where-we-left-off
diff --git a/rebase-notes.sh b/rebase-notes.sh
new file mode 100755
index 000000000000..c9ac23cc4f05
--- /dev/null
+++ b/rebase-notes.sh
@@ -0,0 +1,10 @@
+while read -r note_ref obj_ref; do
+ subject="$(git log --pretty=format:%s --max-count=1 "$obj_ref")"
+ new_obj_ref="$(git log --pretty=format:%H --max-count=1 --grep="$subject")"
+ [[ -n $new_obj_ref ]] || continue
+ [[ $new_obj_ref != "$obj_ref" ]] || continue
+ git notes copy "$obj_ref" "$new_obj_ref"
+ git notes remove "$obj_ref"
+done < <(git notes)
diff --git a/rebase-on-net-next.sh b/rebase-on-net-next.sh
new file mode 100755
index 000000000000..a730a2bed229
--- /dev/null
+++ b/rebase-on-net-next.sh
@@ -0,0 +1,6 @@
+SELF="$(readlink -f "${BASH_SOURCE[0]}")"
+cd "${SELF%/*}"
+set -ex
+git fetch net-next
+git rebase FETCH_HEAD
diff --git a/retag-and-push.sh b/retag-and-push.sh
new file mode 100755
index 000000000000..3db86dfc245a
--- /dev/null
+++ b/retag-and-push.sh
@@ -0,0 +1,9 @@
+SELF="$(readlink -f "${BASH_SOURCE[0]}")"
+cd "${SELF%/*}"
+set -ex
+git tag -f wireguard $(git log --grep="net: WireGuard" --max-count=1 --pretty=format:%H)
+git tag -f zinc $(git log --grep="zinc: introduce" --max-count=1 --pretty=format:%H)
+git tag -f simd $(git log --grep="asm: simd" --max-count=1 --pretty=format:%H)
+git tag -f big_key_rewrite $(git log --grep="security/keys: rewrite" --max-count=1 --pretty=format:%H)
+git push -f origin master:jd/wireguard wireguard zinc simd big_key_rewrite
diff --git a/run-in-harness.sh b/run-in-harness.sh
new file mode 100755
index 000000000000..6c6094bea100
--- /dev/null
+++ b/run-in-harness.sh
@@ -0,0 +1,9 @@
+SELF="$(readlink -f "${BASH_SOURCE[0]}")"
+set -ex
+export GIT_URI_integration="$IT"
+export KERNEL_VERSION=integration-git-debug
+make -C "$WG/src/tests/qemu" -j$(nproc)
diff --git a/run-me/run.sh b/run-me/run.sh
new file mode 100755
index 000000000000..d0d7c4ae2ee0
--- /dev/null
+++ b/run-me/run.sh
@@ -0,0 +1,6 @@
+SELF="$(readlink -f "${BASH_SOURCE[0]}")"
+cd "${SELF%/*}"
+set -ex
+make -C .. O=$(pwd) -j9 "$@"
+exec qemu-system-x86_64 -nodefaults -nographic -smp 4 -m 128M -serial stdio -no-reboot -monitor none -cpu host -machine q35,accel=kvm -kernel arch/x86/boot/bzImage
diff --git a/todo-for-next.txt b/todo-for-next.txt
new file mode 100644
index 000000000000..03374873178b
--- /dev/null
+++ b/todo-for-next.txt
@@ -0,0 +1,2 @@
+- Maybe add netdev_dbg_ratelimited function. [Andrew]
+- More documentation on what zinc is for and isnt for.
diff --git a/zero-zero-text.txt b/zero-zero-text.txt
new file mode 100644
index 000000000000..22a270ef4595
--- /dev/null
+++ b/zero-zero-text.txt
@@ -0,0 +1,93 @@
+This patchset is available on git.kernel.org in this branch, where it may be
+pulled directly for inclusion into net-next:
+ * https://git.kernel.org/pub/scm/linux/kernel/git/zx2c4/linux.git/log/?h=jd/wireguard
+WireGuard is a secure network tunnel written especially for Linux, which
+has faced around three years of serious development, deployment, and
+scrutiny. It delivers excellent performance and is extremely easy to
+use and configure. It has been designed with the primary goal of being
+both easy to audit by virtue of being small and highly secure from a
+cryptography and systems security perspective. WireGuard is used by some
+massive companies pushing enormous amounts of traffic, and likely
+already today you've consumed bytes that at some point transited through
+a WireGuard tunnel. Even as an out-of-tree module, WireGuard has been
+integrated into various userspace tools, Linux distributions, mobile
+phones, and data centers. There are ports in several languages to
+several operating systems, and even commercial hardware and services
+sold integrating WireGuard. It is time, therefore, for WireGuard to be
+properly integrated into Linux.
+Ample information, including documentation, installation instructions,
+and project details, is available at:
+ * https://www.wireguard.com/
+ * https://www.wireguard.com/papers/wireguard.pdf
+As it is currently an out-of-tree module, it lives in its own git repo
+and has its own mailing list, and every commit for the module is tested
+against every stable kernel since 3.10 on a variety of architectures
+using an extensive test suite:
+ * https://git.zx2c4.com/WireGuard
+ https://git.kernel.org/pub/scm/linux/kernel/git/zx2c4/WireGuard.git/
+ * https://lists.zx2c4.com/mailman/listinfo/wireguard
+ * https://www.wireguard.com/build-status/
+The project has been broadly discussed at conferences, and was presented
+to the Netdev developers in Seoul last November, where a paper was
+released detailing some interesting aspects of the project. Dave asked
+me after the talk if I would consider sending in a v1 "sooner rather
+than later", hence this patchset. Zinc was presented at Kernel Recipes
+in September, and a video is available online. Both Zinc and WireGuard
+were presented at Plumbers in Vancouver in November.
+ * https://www.wireguard.com/presentations/
+ * https://www.wireguard.com/papers/wireguard-netdev22.pdf
+ * Zinc talk: https://www.youtube.com/watch?v=bFhdln8aJ_U
+ * Netdev talk: https://www.youtube.com/watch?v=54orFwtQ1XY
+The cryptography in the protocol itself has been formally verified by
+several independent academic teams with positive results, and I know of
+two additional efforts on their way to further corroborate those
+findings. The version 1 protocol is "complete", and so the purpose of
+this review is to assess the implementation of the protocol. However, it
+still may be of interest to know that the thing you're reviewing uses a
+protocol with various nice security properties:
+ * https://www.wireguard.com/formal-verification/
+This patchset is divided into four segments. The first introduces a very
+simple helper for working with the FPU state for the purposes of amortizing
+SIMD operations. The second segment is a small collection of cryptographic
+primitives, split up into several commits by primitive and by hardware. The
+third shows a non-WireGuard use case for Zinc. The last is WireGuard itself,
+presented as an unintrusive and self-contained virtual network driver.
+It is intended that this entire patch series enter the kernel through
+DaveM's net-next tree. Subsequently, WireGuard patches will go through
+DaveM's net-next tree, while Zinc patches will go through Greg KH's tree in
+cases when an entire development cycle has no relationships with existing code
+in crypto/; however, if there are any relationships with code in crypto/, then
+pull requests will be sent to Herbert instead in case there are merge
+Cc: Linus Torvalds <torvalds@linux-foundation.org>
+Cc: "David S. Miller" <davem@davemloft.net>
+Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
+Cc: Eric Biggers <ebiggers@kernel.org>
+Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
+Cc: Samuel Neves <samuel.c.p.neves@gmail.com>
+Cc: Herbert Xu <herbert@gondor.apana.org.au>
+Cc: linux-crypto@vger.kernel.org
+Cc: linux-kernel@vger.kernel.org
+Cc: netdev@vger.kernel.org
diff --git a/zip-up-changed-files.sh b/zip-up-changed-files.sh
new file mode 100755
index 000000000000..d2461fc9b77a
--- /dev/null
+++ b/zip-up-changed-files.sh
@@ -0,0 +1,25 @@
+SELF="$(readlink -f "${BASH_SOURCE[0]}")"
+cd "${SELF%/*}"
+set -ex
+COMMITS=( $(git log net-next/master..master --pretty=format:%H) )
+declare -A FILES
+for commit in "${COMMITS[@]}"; do
+ for file in $(git diff-tree --no-commit-id --name-only -r "$commit"); do
+ if [[ -n ${FILES["$file"]} ]]; then
+ FILES["$file"]="_"
+ else
+ FILES["$file"]="$commit"
+ fi
+ done
+while read -r status file; do
+ [[ $status == M || $status == D || $status == A ]] || continue
+ commit="${FILES["$file"]}"
+ [[ -n $commit && $commit != _ ]] || continue
+ git commit --fixup="$commit" "$file"
+done < <(git status --porcelain -uno)
+GIT_SEQUENCE_EDITOR=true git rebase -i --autosquash net-next/master