diff options
| author |   Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com> | 2022-08-18 18:40:42 +0530 |
|---|---|---|
| committer |   Andrew Morton <akpm@linux-foundation.org> | 2022-09-26 19:46:13 -0700 |
| commit | 3e061d924fe9c7b487ca45935f92fcd414ce6f1a (patch) | |
| tree | b129578bb8d90d77710fc9ce6495f10dbf78f2f1 | |
| parent | mm/demotion: update node_is_toptier to work with memory tiers (diff) | |
| download | linux-dev-3e061d924fe9c7b487ca45935f92fcd414ce6f1a.tar.xz linux-dev-3e061d924fe9c7b487ca45935f92fcd414ce6f1a.zip | |
lib/nodemask: optimize node_random for nodemask with single NUMA node
The most common case for certain node_random usage (demotion nodemask) is
with nodemask weight 1. We can avoid calling get_random_init() in that
case and always return the only node set in the nodemask.
A simple test as below
before = rdtsc_ordered();
for (i= 0; i < 100; i++) {
rand = node_random(&nmask);
}
after = rdtsc_ordered();
Without fix after - before : 16438
With fix after - before : 816
Link: https://lkml.kernel.org/r/20220818131042.113280-11-aneesh.kumar@linux.ibm.com
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
Reviewed-by: "Huang, Ying" <ying.huang@intel.com>
Acked-by: Wei Xu <weixugc@google.com>
Cc: Alistair Popple <apopple@nvidia.com>
Cc: Bharata B Rao <bharata@amd.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Cc: Hesham Almatary <hesham.almatary@huawei.com>
Cc: Jagdish Gediya <jvgediya.oss@gmail.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Tim Chen <tim.c.chen@intel.com>
Cc: Yang Shi <shy828301@gmail.com>
Cc: SeongJae Park <sj@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
| -rw-r--r-- | include/linux/nodemask.h | 15 |
1 files changed, 12 insertions, 3 deletions
diff --git a/include/linux/nodemask.h b/include/linux/nodemask.h index 3a0eec9f2faa..e66742db741c 100644 --- a/include/linux/nodemask.h +++ b/include/linux/nodemask.h @@ -505,12 +505,21 @@ static inline int num_node_state(enum node_states state) static inline int node_random(const nodemask_t *maskp) { #if defined(CONFIG_NUMA) && (MAX_NUMNODES > 1) - int w, bit = NUMA_NO_NODE; + int w, bit; w = nodes_weight(*maskp); - if (w) + switch (w) { + case 0: + bit = NUMA_NO_NODE; + break; + case 1: + bit = first_node(*maskp); + break; + default: bit = bitmap_ord_to_pos(maskp->bits, - get_random_int() % w, MAX_NUMNODES); + get_random_int() % w, MAX_NUMNODES); + break; + } return bit; #else return 0; |