Make kcov MP-safe. Calling the injected tracing function

__sanitizer_cov_trace_pc() early in the boot process caused a subtle
crash while booting the secondary CPU(s). On amd64, accessing curcpu
during this period is not safe since its GSBASE register is yet not
written. After the CPU has been booted curproc can also be NULL for a
brief period of time before the idle thread tied to the same CPU has
started. The two problems can simply be avoided by postponing access to
curcpu and curproc until /dev/kcov has been opened at least once.

The end goal here is to allow fuzzing of MP kernels, which already is in
full swing.

This work has gone through many iterations before settling on the least
intrusive change; many thanks for visa@ for reviewing and providing
valuable input.

Issue originally reported by Greg Steuck on tech@ who also took the time
to test all iterations and providing me access to a virtualised OpenBSD
machine for easier testing.

ok mpi@ visa@

1 files changed, 12 insertions, 4 deletions

diff --git a/sys/dev/kcov.c b/sys/dev/kcov.c
index 8e36bc8b8ef..6a8dc3ebfbc 100644
--- a/sys/dev/kcov.c
+++ b/sys/dev/kcov.c
@@ -1,4 +1,4 @@
-/*	$OpenBSD: kcov.c,v 1.4 2018/08/27 15:57:39 anton Exp $	*/
+/*	$OpenBSD: kcov.c,v 1.5 2018/12/12 07:29:38 anton Exp $	*/
 
 /*
  * Copyright (c) 2018 Anton Lindqvist <anton@openbsd.org>
@@ -58,6 +58,8 @@ static inline int inintr(void);
 
 TAILQ_HEAD(, kcov_dev) kd_list = TAILQ_HEAD_INITIALIZER(kd_list);
 
+int kcov_cold = 1;
+
 #ifdef KCOV_DEBUG
 int kcov_debug = 1;
 #endif
@@ -76,12 +78,15 @@ int kcov_debug = 1;
 void
 __sanitizer_cov_trace_pc(void)
 {
-	extern int cold;
 	struct kcov_dev *kd;
 	uint64_t idx;
 
-	/* Do not trace during boot. */
-	if (cold)
+	/*
+	 * Do not trace before kcovopen() has been called at least once.
+	 * At this point, all secondary CPUs have booted and accessing curcpu()
+	 * is safe.
+	 */
+	if (kcov_cold)
 		return;
 
 	/* Do not trace in interrupts to prevent noisy coverage. */
@@ -112,6 +117,9 @@ kcovopen(dev_t dev, int flag, int mode, struct proc *p)
 	if (kd_lookup(minor(dev)) != NULL)
 		return (EBUSY);
 
+	if (kcov_cold)
+		kcov_cold = 0;
+
 	DPRINTF("%s: unit=%d\n", __func__, minor(dev));
 
 	kd = malloc(sizeof(*kd), M_SUBPROC, M_WAITOK | M_ZERO);