diff options
author | Mauro Carvalho Chehab <mchehab+huawei@kernel.org> | 2020-06-23 15:31:38 +0200 |
---|---|---|
committer | Jonathan Corbet <corbet@lwn.net> | 2020-06-26 11:33:42 -0600 |
commit | c9b54d6f362c0846a11fedea20ec8b8da9b4c93d (patch) | |
tree | 345212980b4b2f375e88016e59c27261fd6e6a8e /Documentation/this_cpu_ops.txt | |
parent | docs: move mailbox.txt to driver-api and rename it (diff) | |
download | wireguard-linux-c9b54d6f362c0846a11fedea20ec8b8da9b4c93d.tar.xz wireguard-linux-c9b54d6f362c0846a11fedea20ec8b8da9b4c93d.zip |
docs: move other kAPI documents to core-api
There are a number of random documents that seem to be
describing some aspects of the core-api. Move them to such
directory, adding them at the core-api/index.rst file.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/86d979ed183adb76af93a92f20189bccf97f0055.1592918949.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Diffstat (limited to 'Documentation/this_cpu_ops.txt')
-rw-r--r-- | Documentation/this_cpu_ops.txt | 339 |
1 files changed, 0 insertions, 339 deletions
diff --git a/Documentation/this_cpu_ops.txt b/Documentation/this_cpu_ops.txt deleted file mode 100644 index 5cb8b883ae83..000000000000 --- a/Documentation/this_cpu_ops.txt +++ /dev/null @@ -1,339 +0,0 @@ -=================== -this_cpu operations -=================== - -:Author: Christoph Lameter, August 4th, 2014 -:Author: Pranith Kumar, Aug 2nd, 2014 - -this_cpu operations are a way of optimizing access to per cpu -variables associated with the *currently* executing processor. This is -done through the use of segment registers (or a dedicated register where -the cpu permanently stored the beginning of the per cpu area for a -specific processor). - -this_cpu operations add a per cpu variable offset to the processor -specific per cpu base and encode that operation in the instruction -operating on the per cpu variable. - -This means that there are no atomicity issues between the calculation of -the offset and the operation on the data. Therefore it is not -necessary to disable preemption or interrupts to ensure that the -processor is not changed between the calculation of the address and -the operation on the data. - -Read-modify-write operations are of particular interest. Frequently -processors have special lower latency instructions that can operate -without the typical synchronization overhead, but still provide some -sort of relaxed atomicity guarantees. The x86, for example, can execute -RMW (Read Modify Write) instructions like inc/dec/cmpxchg without the -lock prefix and the associated latency penalty. - -Access to the variable without the lock prefix is not synchronized but -synchronization is not necessary since we are dealing with per cpu -data specific to the currently executing processor. Only the current -processor should be accessing that variable and therefore there are no -concurrency issues with other processors in the system. - -Please note that accesses by remote processors to a per cpu area are -exceptional situations and may impact performance and/or correctness -(remote write operations) of local RMW operations via this_cpu_*. - -The main use of the this_cpu operations has been to optimize counter -operations. - -The following this_cpu() operations with implied preemption protection -are defined. These operations can be used without worrying about -preemption and interrupts:: - - this_cpu_read(pcp) - this_cpu_write(pcp, val) - this_cpu_add(pcp, val) - this_cpu_and(pcp, val) - this_cpu_or(pcp, val) - this_cpu_add_return(pcp, val) - this_cpu_xchg(pcp, nval) - this_cpu_cmpxchg(pcp, oval, nval) - this_cpu_cmpxchg_double(pcp1, pcp2, oval1, oval2, nval1, nval2) - this_cpu_sub(pcp, val) - this_cpu_inc(pcp) - this_cpu_dec(pcp) - this_cpu_sub_return(pcp, val) - this_cpu_inc_return(pcp) - this_cpu_dec_return(pcp) - - -Inner working of this_cpu operations ------------------------------------- - -On x86 the fs: or the gs: segment registers contain the base of the -per cpu area. It is then possible to simply use the segment override -to relocate a per cpu relative address to the proper per cpu area for -the processor. So the relocation to the per cpu base is encoded in the -instruction via a segment register prefix. - -For example:: - - DEFINE_PER_CPU(int, x); - int z; - - z = this_cpu_read(x); - -results in a single instruction:: - - mov ax, gs:[x] - -instead of a sequence of calculation of the address and then a fetch -from that address which occurs with the per cpu operations. Before -this_cpu_ops such sequence also required preempt disable/enable to -prevent the kernel from moving the thread to a different processor -while the calculation is performed. - -Consider the following this_cpu operation:: - - this_cpu_inc(x) - -The above results in the following single instruction (no lock prefix!):: - - inc gs:[x] - -instead of the following operations required if there is no segment -register:: - - int *y; - int cpu; - - cpu = get_cpu(); - y = per_cpu_ptr(&x, cpu); - (*y)++; - put_cpu(); - -Note that these operations can only be used on per cpu data that is -reserved for a specific processor. Without disabling preemption in the -surrounding code this_cpu_inc() will only guarantee that one of the -per cpu counters is correctly incremented. However, there is no -guarantee that the OS will not move the process directly before or -after the this_cpu instruction is executed. In general this means that -the value of the individual counters for each processor are -meaningless. The sum of all the per cpu counters is the only value -that is of interest. - -Per cpu variables are used for performance reasons. Bouncing cache -lines can be avoided if multiple processors concurrently go through -the same code paths. Since each processor has its own per cpu -variables no concurrent cache line updates take place. The price that -has to be paid for this optimization is the need to add up the per cpu -counters when the value of a counter is needed. - - -Special operations ------------------- - -:: - - y = this_cpu_ptr(&x) - -Takes the offset of a per cpu variable (&x !) and returns the address -of the per cpu variable that belongs to the currently executing -processor. this_cpu_ptr avoids multiple steps that the common -get_cpu/put_cpu sequence requires. No processor number is -available. Instead, the offset of the local per cpu area is simply -added to the per cpu offset. - -Note that this operation is usually used in a code segment when -preemption has been disabled. The pointer is then used to -access local per cpu data in a critical section. When preemption -is re-enabled this pointer is usually no longer useful since it may -no longer point to per cpu data of the current processor. - - -Per cpu variables and offsets ------------------------------ - -Per cpu variables have *offsets* to the beginning of the per cpu -area. They do not have addresses although they look like that in the -code. Offsets cannot be directly dereferenced. The offset must be -added to a base pointer of a per cpu area of a processor in order to -form a valid address. - -Therefore the use of x or &x outside of the context of per cpu -operations is invalid and will generally be treated like a NULL -pointer dereference. - -:: - - DEFINE_PER_CPU(int, x); - -In the context of per cpu operations the above implies that x is a per -cpu variable. Most this_cpu operations take a cpu variable. - -:: - - int __percpu *p = &x; - -&x and hence p is the *offset* of a per cpu variable. this_cpu_ptr() -takes the offset of a per cpu variable which makes this look a bit -strange. - - -Operations on a field of a per cpu structure --------------------------------------------- - -Let's say we have a percpu structure:: - - struct s { - int n,m; - }; - - DEFINE_PER_CPU(struct s, p); - - -Operations on these fields are straightforward:: - - this_cpu_inc(p.m) - - z = this_cpu_cmpxchg(p.m, 0, 1); - - -If we have an offset to struct s:: - - struct s __percpu *ps = &p; - - this_cpu_dec(ps->m); - - z = this_cpu_inc_return(ps->n); - - -The calculation of the pointer may require the use of this_cpu_ptr() -if we do not make use of this_cpu ops later to manipulate fields:: - - struct s *pp; - - pp = this_cpu_ptr(&p); - - pp->m--; - - z = pp->n++; - - -Variants of this_cpu ops ------------------------- - -this_cpu ops are interrupt safe. Some architectures do not support -these per cpu local operations. In that case the operation must be -replaced by code that disables interrupts, then does the operations -that are guaranteed to be atomic and then re-enable interrupts. Doing -so is expensive. If there are other reasons why the scheduler cannot -change the processor we are executing on then there is no reason to -disable interrupts. For that purpose the following __this_cpu operations -are provided. - -These operations have no guarantee against concurrent interrupts or -preemption. If a per cpu variable is not used in an interrupt context -and the scheduler cannot preempt, then they are safe. If any interrupts -still occur while an operation is in progress and if the interrupt too -modifies the variable, then RMW actions can not be guaranteed to be -safe:: - - __this_cpu_read(pcp) - __this_cpu_write(pcp, val) - __this_cpu_add(pcp, val) - __this_cpu_and(pcp, val) - __this_cpu_or(pcp, val) - __this_cpu_add_return(pcp, val) - __this_cpu_xchg(pcp, nval) - __this_cpu_cmpxchg(pcp, oval, nval) - __this_cpu_cmpxchg_double(pcp1, pcp2, oval1, oval2, nval1, nval2) - __this_cpu_sub(pcp, val) - __this_cpu_inc(pcp) - __this_cpu_dec(pcp) - __this_cpu_sub_return(pcp, val) - __this_cpu_inc_return(pcp) - __this_cpu_dec_return(pcp) - - -Will increment x and will not fall-back to code that disables -interrupts on platforms that cannot accomplish atomicity through -address relocation and a Read-Modify-Write operation in the same -instruction. - - -&this_cpu_ptr(pp)->n vs this_cpu_ptr(&pp->n) --------------------------------------------- - -The first operation takes the offset and forms an address and then -adds the offset of the n field. This may result in two add -instructions emitted by the compiler. - -The second one first adds the two offsets and then does the -relocation. IMHO the second form looks cleaner and has an easier time -with (). The second form also is consistent with the way -this_cpu_read() and friends are used. - - -Remote access to per cpu data ------------------------------- - -Per cpu data structures are designed to be used by one cpu exclusively. -If you use the variables as intended, this_cpu_ops() are guaranteed to -be "atomic" as no other CPU has access to these data structures. - -There are special cases where you might need to access per cpu data -structures remotely. It is usually safe to do a remote read access -and that is frequently done to summarize counters. Remote write access -something which could be problematic because this_cpu ops do not -have lock semantics. A remote write may interfere with a this_cpu -RMW operation. - -Remote write accesses to percpu data structures are highly discouraged -unless absolutely necessary. Please consider using an IPI to wake up -the remote CPU and perform the update to its per cpu area. - -To access per-cpu data structure remotely, typically the per_cpu_ptr() -function is used:: - - - DEFINE_PER_CPU(struct data, datap); - - struct data *p = per_cpu_ptr(&datap, cpu); - -This makes it explicit that we are getting ready to access a percpu -area remotely. - -You can also do the following to convert the datap offset to an address:: - - struct data *p = this_cpu_ptr(&datap); - -but, passing of pointers calculated via this_cpu_ptr to other cpus is -unusual and should be avoided. - -Remote access are typically only for reading the status of another cpus -per cpu data. Write accesses can cause unique problems due to the -relaxed synchronization requirements for this_cpu operations. - -One example that illustrates some concerns with write operations is -the following scenario that occurs because two per cpu variables -share a cache-line but the relaxed synchronization is applied to -only one process updating the cache-line. - -Consider the following example:: - - - struct test { - atomic_t a; - int b; - }; - - DEFINE_PER_CPU(struct test, onecacheline); - -There is some concern about what would happen if the field 'a' is updated -remotely from one processor and the local processor would use this_cpu ops -to update field b. Care should be taken that such simultaneous accesses to -data within the same cache line are avoided. Also costly synchronization -may be necessary. IPIs are generally recommended in such scenarios instead -of a remote write to the per cpu area of another processor. - -Even in cases where the remote writes are rare, please bear in -mind that a remote write will evict the cache line from the processor -that most likely will access it. If the processor wakes up and finds a -missing local cache line of a per cpu area, its performance and hence -the wake up times will be affected. |