Add hw.ncpuonline to count the number of online CPUs.

The introduction of hw.smt means that logical CPUs can be disabled
after boot and prior to suspend/resume.  If hw.smt=0 (the default),
there needs to be a way to count the number of hardware threads
available on the system at any given time.

So, import HW_NCPUONLINE/hw.ncpuonline from NetBSD and document it.
hw.ncpu becomes equal to the number of CPUs given to sched_init_cpu()
during boot, while hw.ncpuonline is equal to the number of CPUs available
to the scheduler in the cpuset "sched_all_cpus". Set_SC_NPROCESSORS_ONLN
equal to this new sysctl and keep _SC_NPROCESSORS_CONF equal to hw.ncpu.

This is preferable to adding a new sysctl to count the number of
configured CPUs and keeping hw.ncpu equal to the number of online
CPUs because such a change would break software in the ecosystem
that relies on HW_NCPU/hw.ncpu to measure CPU usage and the like.
Such software in base includes top(1), systat(1), and snmpd(8),
and perhaps others.

We don't need additional locking to count the cardinality of a cpuset
in this case because the only interfaces that can modify said cardinality
are sysctl(2) and ioctl(2), both of which are under the KERNEL_LOCK.

Software using HW_NCPU/hw.ncpu to determine optimal parallism will need
to be updated to use HW_NCPUONLINE/hw.ncpuonline.  Until then, such software
may perform suboptimally.  However, most changes will be similar to the
change included here for libcxx's std::thread:hardware_concurrency():
using HW_NCPUONLINE in lieu of HW_NCPU should be sufficient for determining
optimal parallelism for most software if the change to _SC_NPROCESSORS_ONLN
is insufficient.

Prompted by deraadt. Discussed at length with kettenis, deraadt, and sthen.
Lots of patch tweaks from kettenis.

ok kettenis, "proceed" deraadt

1 files changed, 21 insertions, 1 deletions

diff --git a/sys/kern/kern_sched.c b/sys/kern/kern_sched.c
index 4309255ca9d..ce9e80066a3 100644
--- a/sys/kern/kern_sched.c
+++ b/sys/kern/kern_sched.c
@@ -1,4 +1,4 @@
-/*	$OpenBSD: kern_sched.c,v 1.50 2018/07/07 15:19:25 visa Exp $	*/
+/*	$OpenBSD: kern_sched.c,v 1.51 2018/07/12 01:23:38 cheloha Exp $	*/
 /*
  * Copyright (c) 2007, 2008 Artur Grabowski <art@openbsd.org>
  *
@@ -812,6 +812,26 @@ cpuset_complement(struct cpuset *to, struct cpuset *a, struct cpuset *b)
 		to->cs_set[i] = b->cs_set[i] & ~a->cs_set[i];
 }
 
+int
+cpuset_cardinality(struct cpuset *cs)
+{
+	int cardinality, i, n;
+
+	cardinality = 0;
+
+	for (i = 0; i < CPUSET_ASIZE(ncpus); i++)
+		for (n = cs->cs_set[i]; n != 0; n &= n - 1)
+			cardinality++;
+
+	return (cardinality);
+}
+
+int
+sysctl_hwncpuonline(void)
+{
+	return cpuset_cardinality(&sched_all_cpus);
+}
+
 #ifdef __HAVE_CPU_TOPOLOGY
 
 #include <sys/sysctl.h>