aboutsummaryrefslogtreecommitdiffstatshomepage
path: root/Documentation/this_cpu_ops.txt
diff options
context:
space:
mode:
authorMauro Carvalho Chehab <mchehab+huawei@kernel.org>2020-06-23 15:31:38 +0200
committerJonathan Corbet <corbet@lwn.net>2020-06-26 11:33:42 -0600
commitc9b54d6f362c0846a11fedea20ec8b8da9b4c93d (patch)
tree345212980b4b2f375e88016e59c27261fd6e6a8e /Documentation/this_cpu_ops.txt
parentdocs: move mailbox.txt to driver-api and rename it (diff)
downloadwireguard-linux-c9b54d6f362c0846a11fedea20ec8b8da9b4c93d.tar.xz
wireguard-linux-c9b54d6f362c0846a11fedea20ec8b8da9b4c93d.zip
docs: move other kAPI documents to core-api
There are a number of random documents that seem to be describing some aspects of the core-api. Move them to such directory, adding them at the core-api/index.rst file. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Link: https://lore.kernel.org/r/86d979ed183adb76af93a92f20189bccf97f0055.1592918949.git.mchehab+huawei@kernel.org Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Diffstat (limited to 'Documentation/this_cpu_ops.txt')
-rw-r--r--Documentation/this_cpu_ops.txt339
1 files changed, 0 insertions, 339 deletions
diff --git a/Documentation/this_cpu_ops.txt b/Documentation/this_cpu_ops.txt
deleted file mode 100644
index 5cb8b883ae83..000000000000
--- a/Documentation/this_cpu_ops.txt
+++ /dev/null
@@ -1,339 +0,0 @@
-===================
-this_cpu operations
-===================
-
-:Author: Christoph Lameter, August 4th, 2014
-:Author: Pranith Kumar, Aug 2nd, 2014
-
-this_cpu operations are a way of optimizing access to per cpu
-variables associated with the *currently* executing processor. This is
-done through the use of segment registers (or a dedicated register where
-the cpu permanently stored the beginning of the per cpu area for a
-specific processor).
-
-this_cpu operations add a per cpu variable offset to the processor
-specific per cpu base and encode that operation in the instruction
-operating on the per cpu variable.
-
-This means that there are no atomicity issues between the calculation of
-the offset and the operation on the data. Therefore it is not
-necessary to disable preemption or interrupts to ensure that the
-processor is not changed between the calculation of the address and
-the operation on the data.
-
-Read-modify-write operations are of particular interest. Frequently
-processors have special lower latency instructions that can operate
-without the typical synchronization overhead, but still provide some
-sort of relaxed atomicity guarantees. The x86, for example, can execute
-RMW (Read Modify Write) instructions like inc/dec/cmpxchg without the
-lock prefix and the associated latency penalty.
-
-Access to the variable without the lock prefix is not synchronized but
-synchronization is not necessary since we are dealing with per cpu
-data specific to the currently executing processor. Only the current
-processor should be accessing that variable and therefore there are no
-concurrency issues with other processors in the system.
-
-Please note that accesses by remote processors to a per cpu area are
-exceptional situations and may impact performance and/or correctness
-(remote write operations) of local RMW operations via this_cpu_*.
-
-The main use of the this_cpu operations has been to optimize counter
-operations.
-
-The following this_cpu() operations with implied preemption protection
-are defined. These operations can be used without worrying about
-preemption and interrupts::
-
- this_cpu_read(pcp)
- this_cpu_write(pcp, val)
- this_cpu_add(pcp, val)
- this_cpu_and(pcp, val)
- this_cpu_or(pcp, val)
- this_cpu_add_return(pcp, val)
- this_cpu_xchg(pcp, nval)
- this_cpu_cmpxchg(pcp, oval, nval)
- this_cpu_cmpxchg_double(pcp1, pcp2, oval1, oval2, nval1, nval2)
- this_cpu_sub(pcp, val)
- this_cpu_inc(pcp)
- this_cpu_dec(pcp)
- this_cpu_sub_return(pcp, val)
- this_cpu_inc_return(pcp)
- this_cpu_dec_return(pcp)
-
-
-Inner working of this_cpu operations
-------------------------------------
-
-On x86 the fs: or the gs: segment registers contain the base of the
-per cpu area. It is then possible to simply use the segment override
-to relocate a per cpu relative address to the proper per cpu area for
-the processor. So the relocation to the per cpu base is encoded in the
-instruction via a segment register prefix.
-
-For example::
-
- DEFINE_PER_CPU(int, x);
- int z;
-
- z = this_cpu_read(x);
-
-results in a single instruction::
-
- mov ax, gs:[x]
-
-instead of a sequence of calculation of the address and then a fetch
-from that address which occurs with the per cpu operations. Before
-this_cpu_ops such sequence also required preempt disable/enable to
-prevent the kernel from moving the thread to a different processor
-while the calculation is performed.
-
-Consider the following this_cpu operation::
-
- this_cpu_inc(x)
-
-The above results in the following single instruction (no lock prefix!)::
-
- inc gs:[x]
-
-instead of the following operations required if there is no segment
-register::
-
- int *y;
- int cpu;
-
- cpu = get_cpu();
- y = per_cpu_ptr(&x, cpu);
- (*y)++;
- put_cpu();
-
-Note that these operations can only be used on per cpu data that is
-reserved for a specific processor. Without disabling preemption in the
-surrounding code this_cpu_inc() will only guarantee that one of the
-per cpu counters is correctly incremented. However, there is no
-guarantee that the OS will not move the process directly before or
-after the this_cpu instruction is executed. In general this means that
-the value of the individual counters for each processor are
-meaningless. The sum of all the per cpu counters is the only value
-that is of interest.
-
-Per cpu variables are used for performance reasons. Bouncing cache
-lines can be avoided if multiple processors concurrently go through
-the same code paths. Since each processor has its own per cpu
-variables no concurrent cache line updates take place. The price that
-has to be paid for this optimization is the need to add up the per cpu
-counters when the value of a counter is needed.
-
-
-Special operations
-------------------
-
-::
-
- y = this_cpu_ptr(&x)
-
-Takes the offset of a per cpu variable (&x !) and returns the address
-of the per cpu variable that belongs to the currently executing
-processor. this_cpu_ptr avoids multiple steps that the common
-get_cpu/put_cpu sequence requires. No processor number is
-available. Instead, the offset of the local per cpu area is simply
-added to the per cpu offset.
-
-Note that this operation is usually used in a code segment when
-preemption has been disabled. The pointer is then used to
-access local per cpu data in a critical section. When preemption
-is re-enabled this pointer is usually no longer useful since it may
-no longer point to per cpu data of the current processor.
-
-
-Per cpu variables and offsets
------------------------------
-
-Per cpu variables have *offsets* to the beginning of the per cpu
-area. They do not have addresses although they look like that in the
-code. Offsets cannot be directly dereferenced. The offset must be
-added to a base pointer of a per cpu area of a processor in order to
-form a valid address.
-
-Therefore the use of x or &x outside of the context of per cpu
-operations is invalid and will generally be treated like a NULL
-pointer dereference.
-
-::
-
- DEFINE_PER_CPU(int, x);
-
-In the context of per cpu operations the above implies that x is a per
-cpu variable. Most this_cpu operations take a cpu variable.
-
-::
-
- int __percpu *p = &x;
-
-&x and hence p is the *offset* of a per cpu variable. this_cpu_ptr()
-takes the offset of a per cpu variable which makes this look a bit
-strange.
-
-
-Operations on a field of a per cpu structure
---------------------------------------------
-
-Let's say we have a percpu structure::
-
- struct s {
- int n,m;
- };
-
- DEFINE_PER_CPU(struct s, p);
-
-
-Operations on these fields are straightforward::
-
- this_cpu_inc(p.m)
-
- z = this_cpu_cmpxchg(p.m, 0, 1);
-
-
-If we have an offset to struct s::
-
- struct s __percpu *ps = &p;
-
- this_cpu_dec(ps->m);
-
- z = this_cpu_inc_return(ps->n);
-
-
-The calculation of the pointer may require the use of this_cpu_ptr()
-if we do not make use of this_cpu ops later to manipulate fields::
-
- struct s *pp;
-
- pp = this_cpu_ptr(&p);
-
- pp->m--;
-
- z = pp->n++;
-
-
-Variants of this_cpu ops
-------------------------
-
-this_cpu ops are interrupt safe. Some architectures do not support
-these per cpu local operations. In that case the operation must be
-replaced by code that disables interrupts, then does the operations
-that are guaranteed to be atomic and then re-enable interrupts. Doing
-so is expensive. If there are other reasons why the scheduler cannot
-change the processor we are executing on then there is no reason to
-disable interrupts. For that purpose the following __this_cpu operations
-are provided.
-
-These operations have no guarantee against concurrent interrupts or
-preemption. If a per cpu variable is not used in an interrupt context
-and the scheduler cannot preempt, then they are safe. If any interrupts
-still occur while an operation is in progress and if the interrupt too
-modifies the variable, then RMW actions can not be guaranteed to be
-safe::
-
- __this_cpu_read(pcp)
- __this_cpu_write(pcp, val)
- __this_cpu_add(pcp, val)
- __this_cpu_and(pcp, val)
- __this_cpu_or(pcp, val)
- __this_cpu_add_return(pcp, val)
- __this_cpu_xchg(pcp, nval)
- __this_cpu_cmpxchg(pcp, oval, nval)
- __this_cpu_cmpxchg_double(pcp1, pcp2, oval1, oval2, nval1, nval2)
- __this_cpu_sub(pcp, val)
- __this_cpu_inc(pcp)
- __this_cpu_dec(pcp)
- __this_cpu_sub_return(pcp, val)
- __this_cpu_inc_return(pcp)
- __this_cpu_dec_return(pcp)
-
-
-Will increment x and will not fall-back to code that disables
-interrupts on platforms that cannot accomplish atomicity through
-address relocation and a Read-Modify-Write operation in the same
-instruction.
-
-
-&this_cpu_ptr(pp)->n vs this_cpu_ptr(&pp->n)
---------------------------------------------
-
-The first operation takes the offset and forms an address and then
-adds the offset of the n field. This may result in two add
-instructions emitted by the compiler.
-
-The second one first adds the two offsets and then does the
-relocation. IMHO the second form looks cleaner and has an easier time
-with (). The second form also is consistent with the way
-this_cpu_read() and friends are used.
-
-
-Remote access to per cpu data
-------------------------------
-
-Per cpu data structures are designed to be used by one cpu exclusively.
-If you use the variables as intended, this_cpu_ops() are guaranteed to
-be "atomic" as no other CPU has access to these data structures.
-
-There are special cases where you might need to access per cpu data
-structures remotely. It is usually safe to do a remote read access
-and that is frequently done to summarize counters. Remote write access
-something which could be problematic because this_cpu ops do not
-have lock semantics. A remote write may interfere with a this_cpu
-RMW operation.
-
-Remote write accesses to percpu data structures are highly discouraged
-unless absolutely necessary. Please consider using an IPI to wake up
-the remote CPU and perform the update to its per cpu area.
-
-To access per-cpu data structure remotely, typically the per_cpu_ptr()
-function is used::
-
-
- DEFINE_PER_CPU(struct data, datap);
-
- struct data *p = per_cpu_ptr(&datap, cpu);
-
-This makes it explicit that we are getting ready to access a percpu
-area remotely.
-
-You can also do the following to convert the datap offset to an address::
-
- struct data *p = this_cpu_ptr(&datap);
-
-but, passing of pointers calculated via this_cpu_ptr to other cpus is
-unusual and should be avoided.
-
-Remote access are typically only for reading the status of another cpus
-per cpu data. Write accesses can cause unique problems due to the
-relaxed synchronization requirements for this_cpu operations.
-
-One example that illustrates some concerns with write operations is
-the following scenario that occurs because two per cpu variables
-share a cache-line but the relaxed synchronization is applied to
-only one process updating the cache-line.
-
-Consider the following example::
-
-
- struct test {
- atomic_t a;
- int b;
- };
-
- DEFINE_PER_CPU(struct test, onecacheline);
-
-There is some concern about what would happen if the field 'a' is updated
-remotely from one processor and the local processor would use this_cpu ops
-to update field b. Care should be taken that such simultaneous accesses to
-data within the same cache line are avoided. Also costly synchronization
-may be necessary. IPIs are generally recommended in such scenarios instead
-of a remote write to the per cpu area of another processor.
-
-Even in cases where the remote writes are rare, please bear in
-mind that a remote write will evict the cache line from the processor
-that most likely will access it. If the processor wakes up and finds a
-missing local cache line of a per cpu area, its performance and hence
-the wake up times will be affected.