summaryrefslogtreecommitdiffstats
path: root/sys/uvm/uvm_amap.c (follow)
Commit message (Collapse)AuthorAgeFilesLines
* (re)Introduce locking for amaps & anons.mpi2021-01-191-35/+134
| | | | | | | | | | | | | | | A rwlock is attached to every amap and is shared with all its anon. The same lock will be used by multiple amaps if they have anons in common. This should be enough to get the upper part of the fault handler out of the KERNEL_LOCK() which seems to bring up to 20% improvements in builds. This is based/copied/adapted from the most recent work done in NetBSD which is an evolution of the precendent simple_lock scheme. Tested by many, thanks! ok kettenis@, mvs@
* Introduce amap_adjref_anons() an helper to reference count amaps.mpi2020-11-131-33/+60
| | | | | | | Reduce code duplication, reduce differences with NetBSD and simplify upcoming locking diff. ok jmatthew@
* Use KASSERT() instead of if(x) panic() for NULL dereference checks.mpi2020-10-121-20/+17
| | | | | | | | | | Improves readability and reduces the difference with NetBSD without compromising debuggability on RAMDISK. While here also use local variables to help with future locking and reference counting. ok semarie@
* Use KASSERT() instead of if(x) panic() for sanity checks.mpi2020-09-251-14/+6
| | | | | | Reduce the diff with NetBSD. ok kettenis@, deraadt@
* Spell inline correctly.mpi2020-09-221-10/+7
| | | | | | Reduce differences with NetBSD. ok mvs@, kettenis@
* Add uvm_anfree_list() to free anons as a list of pages. Use this inbeck2020-01-041-2/+8
| | | | | | | | | | | the amap code to free pages as a list instead of one at a time to allow for more efficient freeing. Most of the work done at elk lakes, with testing by me and mlarkin and kettenis. Speeds up a test program which zeros a big pile of memory and then exits considerably. ok kettenis@
* R.I.P. UVM_WAIT(). Use tsleep_nsec(9) directly.cheloha2019-07-181-2/+2
| | | | | | | | | | | UVM_WAIT() doesn't provide much of a useful abstraction. All callers tsleep forever and no callers set PCATCH, so only 2 of 4 parameters are actually used. Might as well just use tsleep_nsec(9) directly and make the uvm code a bit less specialized. Suggested by mpi@. ok mpi@ visa@ millert@
* free size for amap; ok visa@anton2019-05-151-2/+4
|
* Sprinkle some free sizes in uvm/dhill2017-01-311-3/+3
| | | | ok stefan@ visa@
* Prevent infinite loops for amap allocations with >= 2^17 slotsstefan2016-10-081-3/+6
| | | | | This was caused by an integer overflow in a loop. mlarkin@ noticed the hang when trying to run a vmm(4) guest with lots of RAM.
* all pools have their ipl set via pool_setipl, so fold it into pool_init.dlg2016-09-151-12/+8
| | | | | | | | | | | | | | | | | | | | | | the ioff argument to pool_init() is unused and has been for many years, so this replaces it with an ipl argument. because the ipl will be set on init we no longer need pool_setipl. most of these changes have been done with coccinelle using the spatch below. cocci sucks at formatting code though, so i fixed that by hand. the manpage and subr_pool.c bits i did myself. ok tedu@ jmatthew@ @ipl@ expression pp; expression ipl; expression s, a, o, f, m, p; @@ -pool_init(pp, s, a, o, f, m, p); -pool_setipl(pp, ipl); +pool_init(pp, s, a, ipl, f, m, p);
* check flags with mask instead of equality, in case we decide to mixtedu2016-07-271-2/+2
| | | | another flag in at some point. ok stefan
* Make sure that amap slot calculation does not overflowstefan2016-07-141-5/+11
| | | | | | | | | This prevents from too small amaps being allocated by forcing the allocation of a large number of slots. Based on an analysis from Jesse Hertz and Tim Newsham. ok kettenis@
* Make sure variables are used initialized in amap_wiperangestefan2016-07-111-4/+4
| | | | | | | Uninitialized variables used in an if/else could cause a slower codepath to be taken, but the end effect of both paths is the same. Found by jsg@
* Fix bugs introduced with the amap reworkstefan2016-07-091-68/+89
| | | | | | | | | | | | | | | | | | - The number of slots must be initialized in the chunk of a small amap, otherwise unmapping() part of a mmap()'d range would delay freeing of vm_anons for small amaps - If the first chunk of a bucket is freed, check if the next chunk in the list has to become the new first chunk - Use a separate loop for each type of traversal (small amap, by bucket by list) in amap_wiperange(). This makes the code easier to follow and also fixes a bug where too many chunks were wiped out when traversing by list However, the last two bugs should happen only when turning a previously private mapping into a shared one, then forking, and then having both processes unmap a part of the mapping. snap and ports build tested by krw@, review by kettenis@
* pool_setipl on all uvm pools.dlg2016-06-171-1/+4
| | | | ok kettenis@ visa@
* Make amaps use less kernel memory (2nd try)stefan2016-05-261-157/+398
| | | | | | | | | | | | | | | | | | | | | The original diff would crash at least i386 and powerpc, as spotted by guenther@ The reason was an incorrect use of sizeof in amap_lookups(). Confirmation that powerpc works by mpi@ and mglocker@ "throw it in" deraadt@ Original commit message: This is achieved by grouping amap slots into chunks that are allocated on-demand by pool(9). Endless "fltamapcopy" loops because of kmem shortage should be solved now. The kmem savings are also important to later enable vmm(4) to use larged shared memory mappings for guest VM RAM. This adapts libkvm also because the amap structure layout has changed. Testing and fix of libkvm glitch in initial diff by tb@ Feedback and "time to get this in" kettenis@
* Revert previous: breaks i386 and powerpc, probably all non-PMAP_DIRECT archsguenther2016-05-221-398/+155
|
* Make amaps use less kernel memorystefan2016-05-221-155/+398
| | | | | | | | | | | | This is achieved by grouping amap slots into chunks that are allocated on-demand by pool(9). Endless "fltamapcopy" loops because of kmem shortage should be solved now. The kmem savings are also important to later enable vmm(4) to use larged shared memory mappings for guest VM RAM. This adapts libkvm also because the amap structure layout has changed. Testing and fix of libkvm glitch in initial diff by tb@ Feedback and "time to get this in" kettenis@
* Additional parameter for amap_alloc().stefan2016-05-081-18/+24
| | | | | | | | It is supposed to control whether an amap should allocate memory to store anon pointers lazily or upfront. Needed for upcoming amap changes. ok kettenis@
* Wait for RAM in uvm_fault when allocating uvm structures failsstefan2016-05-081-3/+15
| | | | | | | | | | | Only fail hard when running out of swap space also, as suggested by kettenis@ While there, let amap_add() return a success status and handle amap_add() errors in uvm_fault() similar to other out of RAM situations. These bits are needed for further amap reorganization diffs. lots of feedback and ok kettenis@
* Remove am_maxslot from amap.stefan2016-04-161-20/+12
| | | | | | | | | | am_maxslot represents the total number of slots an amap can be extended to. Since we do not extend amaps, this field as well as rounding the number of slots to the next malloc bucket is not useful. This also removes the corresponding output from procmap(1). ok kettenis@
* Simplify amap traversal in amap_swap_off.stefan2016-04-121-24/+5
| | | | | | | | | | | There's no need to insert marker elements to find the next item in the amap list. The next amap can be determined by looking at the currently examined amap. Care must be taken to get the next element before the current amap is possibly deleted, and after all the current amap's pages were read in from swap (because the page-in may sleep and remove items from the amap list).
* UVM_FLAG_AMAPPAD has no effect anymore, nuke it.stefan2016-04-041-10/+9
| | | | | | | | | | | | | | This flag caused amaps to be allocated with additional spare slots, to make extending them cheaper. However, the kernel never extends amaps, so allocating spare slots is pointless. Also UVM_FLAG_AMAPPAD only has an effect in combination with UVM_FLAG_OVERLAY. The only function that used both flags was sys_obreak, but that function had the use of UVM_FLAG_OVERLAY removed recently. While there, kill the unused prototypes amap_flags and amap_refs. They're defined as macros already. ok mlarkin@ kettenis@ mpi@
* amap_extend is never called, remove it.stefan2016-03-271-169/+1
| | | | | | | | In the code, this function is called when vm_map_entries are merged. However, only kernel map entries are merged, and these do not use amaps. Therefore amap_extend() is never called at runtime. ok millert@, KASSERT suggestion and ok mpi@
* Remove redundant check.stefan2016-03-161-6/+1
| | | | | | | | The compiler is also smart enough to recognize that this is redundant. The resulting code on amd64 is basically equivalent (slightly different register allocation and instruction scheduling). ok mpi@ deraadt@
* For amaps with only a few slots, allocate the slots via pool(9)stefan2016-03-151-13/+57
| | | | | | | | | | This saves some memory compared to using malloc, because there's no roundup to the next bucket size. And it reduces kmem pressure at least for some architectures (e.g. amd64). Testing by sthen@ and tobiasu@, thanks! ok sthen@ deraadt@
* Remove unused amap_share_protect().stefan2016-03-061-44/+1
| | | | ok mpi@ visa@
* Remove the unused loan_count field and the related uvm logic. Most ofvisa2015-08-211-11/+4
| | | | | | the page loaning code is already in the Attic. ok kettenis@, beck@
* convert pool_init nointr to waitoktedu2014-12-231-3/+3
|
* remove lock.h from uvm_extern.h. another holdover from the simpletonlocktedu2014-12-171-1/+2
| | | | | era. fix uvm including c files to include lock.h or atomic.h as necessary. ok deraadt
* Sprinkle in a little more mallocarray().doug2014-12-091-3/+3
| | | | ok deraadt@ tedu@
* Replace a plethora of historical protection options with justderaadt2014-11-161-2/+2
| | | | | | | PROT_NONE, PROT_READ, PROT_WRITE, and PROT_EXEC from mman.h. PROT_MASK is introduced as the one true way of extracting those bits. Remove UVM_ADV_* wrapper, using the standard names. ok doug guenther kettenis
* remove uneeded proc.h includesjsg2014-09-141-2/+1
| | | | ok mpi@ kspillner@
* add a size argument to free. will be used soon, but for now default to 0.tedu2014-07-121-7/+7
| | | | after discussions with beck deraadt kettenis.
* Chuck Cranor rescinded clauses in his licensejsg2014-07-111-8/+1
| | | | | | | | | | | | | on the 2nd of February 2011 in NetBSD. http://marc.info/?l=netbsd-source-changes&m=129658899212732&w=2 http://marc.info/?l=netbsd-source-changes&m=129659095515558&w=2 http://marc.info/?l=netbsd-source-changes&m=129659157916514&w=2 http://marc.info/?l=netbsd-source-changes&m=129665962324372&w=2 http://marc.info/?l=netbsd-source-changes&m=129666033625342&w=2 http://marc.info/?l=netbsd-source-changes&m=129666052825545&w=2 http://marc.info/?l=netbsd-source-changes&m=129666922906480&w=2 http://marc.info/?l=netbsd-source-changes&m=129667725518082&w=2
* compress code by turning four line comments into one line comments.tedu2014-04-131-100/+25
| | | | emphatic ok usual suspects, grudging ok miod
* UVM_UNLOCK_AND_WAIT no longer unlocks, so rename it to UVM_WAIT.tedu2013-05-301-3/+2
|
* remove lots of comments about locking per beck's requesttedu2013-05-301-57/+15
|
* remove simple_locks from uvm code. ok beck deraadttedu2013-05-301-14/+1
|
* the simplelock is a lietedu2013-05-231-4/+3
|
* Rip out and burn support for UVM_HIST.oga2011-07-031-64/+2
| | | | | | | | The vm hackers don't use it, don't maintain it and have to look at it all the time. About time this 800 lines of code hit /dev/null. ``never liked it'' tedu@. ariane@ was very happy when i told her i wrote this diff.
* explicitly specify flags to malloc and pool_get instead of relying on 0.tedu2010-07-031-2/+3
| | | | | This is more clear, and as thib pointed out, the default in softraid was wrong. ok thib.
* Move all of the pseudo-inline functions in uvm into C files.oga2009-03-251-4/+208
| | | | | | | | | By pseudo-inline, I mean that if a certain macro was defined, they would be inlined. However, no architecture defines that, and none has for a very very long time. Therefore mainly this just makes the code a damned sight easier to read. Some k&r -> ansi declarations while I'm in there. "just commit it" art@. ok weingart@.
* Don't extend amaps beyond what their supposed maximum. This code path isart2008-10-081-1/+4
| | | | | | | not taken anymore, but it doesn't hurt to be correct. from NetBSD, through mickey in pr 5812 prodded by otto@
* less waste for amaps in the common case:otto2008-09-121-36/+17
| | | | | allocate a single malloc chunk instead of three and allocate a single slot for a single page instead of four slots. ok miod@ tedu@ @deraadt
* Plug potential memory leak.kettenis2008-08-261-1/+6
| | | | "looks sane to me" otto@, ok miod@
* Use M_ZERO in a few more places to shave bytes from the kernel.art2007-09-071-3/+2
| | | | eyeballed and ok dlg@
* Bring back Mickey's UVM anon change. Testing by thib@, beck@ andpedro2007-06-181-9/+116
| | | | ckuethe@ for a while. Okay beck@, "it is good timing" deraadt@.
* set hiwat mark for some of the more popular pools to reduce bouncingtedu2007-06-011-1/+2
| | | | ok art bob
Copyright © 1996 – 2023 Jason A. Donenfeld. All Rights Reverse Engineered.