summaryrefslogtreecommitdiffstats
path: root/sys/crypto/crypto.c (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Fix the IPL and flags of the MP-safe crypto taskq. Now a sane IPLvisa2017-11-301-2/+2
| | | | | | | | is passed to the mutex implementation, and the queue actually runs without the kernel lock. Tested by dhill@ OK mikeb@, dhill@, kettenis@
* Reduce the per-packet allocation costs for crypto operations (cryptop)patrick2017-02-071-24/+18
| | | | | | | | | | | | | | by pre-allocating two cryptodesc objects and storing them in an array instead of a linked list. If more than two cryptodesc objects are required use mallocarray to fetch them. Adapt the drivers to the new API. This change results in one pool-get per ESP packet instead of three. It also simplifies softraid crypto where more cryptodesc objects are allocated than used. From, with and ok markus@, ok bluhm@ "looks sane" mpi@
* convert bcopy to memcpy. from david hilltedu2016-09-191-2/+2
|
* all pools have their ipl set via pool_setipl, so fold it into pool_init.dlg2016-09-151-7/+5
| | | | | | | | | | | | | | | | | | | | | | the ioff argument to pool_init() is unused and has been for many years, so this replaces it with an ipl argument. because the ipl will be set on init we no longer need pool_setipl. most of these changes have been done with coccinelle using the spatch below. cocci sucks at formatting code though, so i fixed that by hand. the manpage and subr_pool.c bits i did myself. ok tedu@ jmatthew@ @ipl@ expression pp; expression ipl; expression s, a, o, f, m, p; @@ -pool_init(pp, s, a, o, f, m, p); -pool_setipl(pp, ipl); +pool_init(pp, s, a, ipl, f, m, p);
* Add a mechanism for dispatching mpsafe crypto operations. This adds a newkettenis2016-04-181-3/+17
| | | | | | | | CRYPTOCAP_F_MPSAFE flag that crypto implementations can set to indicate that their cc_process() implementation can safely run without holding the kernel lock. ok mikeb@
* fairly simple sizes for free(); ok teduderaadt2015-08-281-3/+5
|
* we want to defer work traditionally (in openbsd) handled in andlg2015-02-091-2/+2
| | | | | | | | | | | | | | | | | | | | | interrupt context to a taskq running in a thread. however, there is a concern that if we do that then we allow accidental use of sleeping APIs in this work, which will make it harder to move the work back to interrupts in the future. guenther and kettenis came up with the idea of marking a proc with CANTSLEEP which the sleep paths can check and panic on. this builds on that so you create taskqs that run with CANTSLEEP set except when they need to sleep for more tasks to run. the taskq_create api is changed to take a flags argument so users can specify CANTSLEEP. MPSAFE is also passed via this flags field now. this means archs that defined IPL_MPSAFE to 0 can now create mpsafe taskqs too. lots of discussion at s2k15 ok guenther@ miod@ mpi@ tedu@ pelikan@
* remove the second void * argument on tasks.dlg2015-01-271-4/+3
| | | | | | | | | | | | | | | | | | | | | when workqs were introduced, we provided a second argument so you could pass a thing and some context to work on it in. there were very few things that took advantage of the second argument, so when i introduced pools i suggested removing it. since tasks were meant to replace workqs, it was requested that we keep the second argument to make porting from workqs to tasks easier. now that workqs are gone, i had a look at the use of the second argument again and found only one good use of it (vdsp(4) on sparc64 if you're interested) and a tiny handful of questionable uses. the vast majority of tasks only used a single argument. i have since modified all tasks that used two args to only use one, so now we can remove the second argument. so this is a mechanical change. all tasks only passed NULL as their second argument, so we can just remove it. ok krw@
* pools lock themselves now, we just have to tell them what IPL theydlg2014-10-231-13/+4
| | | | | | | | | | will be used from. this adds pool_setipl at IPL_VM to the crypto descriptor pools, and removes all the splvm handling around the use of those pools. tested by many via tech@ ok kettenis@ deraadt@
* apply only the bit of r1.69 that should have been committed:dlg2014-10-231-2/+2
| | | | | | | | | | | | | make the crypto taskq protect things at IPL_VM instead of IPL_HIGH. everything else in crypto.c uses splvm/IPL_VM. it seems this IPL_HIGH came about because the hand rolled task list and thread that crypto used to use was converted to workqs, which unconditionally used IPL_HIGH internally. when it was converted from workqs to tasks it blindly ported the protection workqs gave. tested by many via tech@ and snapshots ok kettenis@
* revert previous. it did more than the commit message said it did.dlg2014-10-231-5/+14
|
* make the crypto taskq protect things at IPL_VM instead of IPL_HIGH.dlg2014-10-221-14/+5
| | | | | | | | | | | everything else in crypto.c uses splvm/IPL_VM. it seems this IPL_HIGH came about because the hand rolled task list and thread that crypto used to use was converted to workqs, which unconditionally used IPL_HIGH internally. when it was converted from workqs to tasks it blindly ported the protection workqs gave. tested by many via tech@ and snapshots ok kettenis@
* replace bzeros after allocations with M_ZERO and PR_ZERO as appropriate.dlg2014-10-201-9/+4
| | | | ok deraadt@
* remove uneeded proc.h includesjsg2014-09-141-2/+1
| | | | ok mpi@ kspillner@
* Bye bye /dev/cryptomikeb2014-08-201-126/+1
| | | | | | | The interface has been disabled by default for about 4 years and currently there's not much value in having it around at all. ok deraadt
* use mallocarray()deraadt2014-07-131-4/+4
|
* add a size argument to free. will be used soon, but for now default to 0.tedu2014-07-121-2/+2
| | | | after discussions with beck deraadt kettenis.
* cc_queued is not used for anything atm, remove it; ok jsing, markusmikeb2014-01-211-17/+1
|
* Respect CRYPTO_F_NOQUEUE flag when dispatching a crypto operationmikeb2014-01-211-2/+2
| | | | ok jsing, markus
* convert crypto work queue to the task_add(9) api; ok dlgmikeb2013-10-311-13/+14
|
* institute a hard cap on crypto devs instead of a useless wraparound checktedu2013-03-271-3/+2
| | | | ok beck
* for key material that is being being discarded, convert bzero() toderaadt2011-01-111-2/+2
| | | | | explicit_bzero() where required ok markus mikeb
* Reintroduce most crypto/crypto.c r1.55:jsing2010-09-081-12/+8
| | | | | | | | | | | Move pool initialization to init_crypto and zap the crypto_pool_initialized variable. This way we don't have to check if the pool are initialized every time we do a crypto_getreq(). However, also perform the crypto initialisation earlier in init_main so that the crypto pools are initialised before they are used. ok mikeb@ thib@ deraadt@
* Backout r1.55 since this breaks anything which does crypto ops prior tojsing2010-08-081-6/+11
| | | | | | | | | init_crypto() being called from late in init_main(). In particular, this breaks softraid crypto volumes that are assembled at boot. No cookies for thib/mikeb! "Back it out, right now" deraadt@
* Revert part of previous.thib2010-07-081-4/+6
| | | | | | | The splvm protection is needed after all, as we are walking the list of registered crypto drivers and doing that unprotected is unwise. Pointed out by kettenis@
* Move pool initialization to init_crypto and zap the crypto_pool_initializedthib2010-07-081-16/+9
| | | | | | | | | | | | | variable. This way we don't have to check if the pool are initialized every time we do a crypto_getreq(). Move splvm lower as it isnt need all through crypto_newsession(). tiny KNF nit. From mikeb OK deraadt@
* Remove the CRYPTO_ALGORITHM_ALL define, fixup accordinglythib2010-06-091-9/+9
| | | | | | | | and make the loop invartiants <= CRYPTO_ALGORITHM_MAX Do this also for the CRK_ALGORITHM_MAX this also fixes the a bug that caused us to skip CRK_DH_COMPUTE_KEY. ok deraadt@
* crypto hardware (eg, hifn) establishes its interrupt handler atdlg2009-09-031-7/+12
| | | | | | | | | | | | | | | | | | | | IPL_NET. when the hardware finishes some work for the crypto subsystem and therefore something in the kernel that wanted crypto done, it calls crypto_done from that interrupt handler. one of the things that uses crypto is ipsec. when crypto is done for ipsec it then pushes the packet along the network stack. the problem is that all the structures inside the network stack are only protected at splsoftnet. we could be in the middle of modifications to the pf state table or the pfsync queues when we get a hifn interrupt and then go stomp on the same structures. the solution is to defer the completions so they can do the right spl protections. this basically reverts r1.46 of src/sys/crypto/crypto.c. found by naddy@
* reintroduce mutexes to workqs for locking.dlg2008-10-301-2/+2
| | | | | | tested by many on many archs including several alpha test. ok tedu@ go for it deraadt@
* finish conversion to workq. remove list remnants, and put spl in the righttedu2007-11-281-10/+23
| | | | places. handle the no workq case here. ok deraadt
* convert crypto thread to workq. add WQ_DIRECTOK flag to workq.tedu2007-11-251-55/+14
| | | | | | combined, this lets us use crypto before the thread is running and therefore cryptoraid can attach nice and early. ok/testing deraadt mbalmer marco
* do not call crypto_done() on errors, since the drivers already do this.markus2007-11-141-2/+1
| | | | | otherwise we call the callback twice; fixes panics on crypto errors as seen on reboot; ok hshoexer
* remove some silly casts. put spl calls after all declarations.tedu2006-05-311-12/+21
| | | | | | | put one splx in a better spot. make a variable size MALLOC use malloc. remove null test after malloc(M_WAITOK). add PR_NOWAIT flag to pool_get instead of 0. change callbacks to correct type. ok brad deraadt markus mickey
* splimp -> splvmbrad2006-03-041-12/+12
| | | | ok miod@
* Don't use crypto thread for callbacks.mpf2004-12-211-67/+7
| | | | | | | This primarily improves IPsec performance when using crypto accelerators. With help from markus@, tested by wvdputte@. ok deraadt@, markus@
* In crypto_thread(), always save return value from splimp(). We were onlyaaron2004-06-201-2/+2
| | | | | | storing it once on kernel startup. Scary. "holy crap" --deraadt. art@ ok Unclear if this was actually a problem in practice, but this doesn't hurt.
* Fastpath for userland crypto requests. This change makes userlandbeck2003-06-031-11/+21
| | | | | | | | | | | crypto requests attempt to call the crypto driver directly to process crypto layer requests, as opposed to queueing them in the kernel crypto thread. If we can't use the crypto devices (i.e. they're busy) we fall back to queueing the request up in the crypto thread as before. This does allow for faster performance in some cases (smaller requests, how small seems to be dependent on the card/cpu combination) where context switching is a major issue in performance. ok deraadt@ jason@
* Copy the ENTIRE table into the supported algorithms (how the hell did thisjason2003-02-191-2/+2
| | | | work before?!)
* From Angelos:jason2002-11-211-106/+238
| | | | | | - simplistic load balancing across multiple cards - simplified registration process - a few style nits.
* I don't know why this breaks things for me when sshd starts on sparc64.art2002-07-171-88/+63
| | | | | But after wasting the whole day trying to just locate the problem I don't care. Back out since this wasn't tested and showed to anyone else.
* Double-pass over drivers, first hardware only, then software (if weangelos2002-07-161-26/+45
| | | | are interested in software).
* Fix a typo, cleanup on session migration code in crypto_invoke(), andangelos2002-07-161-38/+44
| | | | | | | add a convention that if the driver returns ERESTART as an error message of its process method, the crypto framework will unregister the driver and migrate all its sessions. After discussion with Sam Leffler and Jason Wright.
* kernel changes to make asymmetric crypto work in userlandbeck2002-06-111-14/+7
| | | | | | | | | | - modify getfeat to return something more useful to us on devices (like lofn and everything else until jason fixes it) that can't do rsa stuff, etc and can only do mod_exp.. - error handling fixes so we correctly fail to software when we can't deal with a particular key size - add sysctl kern.userasymcrypto to turn on/off userland asymmetric crypto via /dev/crypto - 1 == on, 0 == off, default is off
* fix ivory tower greek fix. ok angelos@beck2002-06-101-5/+6
|
* Don't use an int for the flags, when the structure usesangelos2002-06-091-2/+4
| | | | u_int8_t. Also, make sure the logic is correct (bad theo!)
* initial hack at a CIOCSYMFEAT ioctlderaadt2002-04-231-1/+30
|
* driver queueing & callback code for keying operationsderaadt2002-04-231-8/+123
|
* crypto_check_alg() is not neededderaadt2002-03-041-32/+1
|
* It looks like there has been one crack smoking and a few cut and pastes.art2002-01-231-3/+3
| | | | | PR_FREEHEADER should not be set in pool_init by the caller. It shouldn't be set in pool_init at all. Besides, it's going away soon anyway.
* Pool deals fairly well with physical memory shortage, but it doesn't dealart2002-01-231-3/+3
| | | | | | | | | | | | | | | | | | well (not at all) with shortages of the vm_map where the pages are mapped (usually kmem_map). Try to deal with it: - group all information the backend allocator for a pool in a separate struct. The pool will only have a pointer to that struct. - change the pool_init API to reflect that. - link all pools allocating from the same allocator on a linked list. - Since an allocator is responsible to wait for physical memory it will only fail (waitok) when it runs out of its backing vm_map, carefully drain pools using the same allocator so that va space is freed. (see comments in code for caveats and details). - change pool_reclaim to return if it actually succeeded to free some memory, use that information to make draining easier and more efficient. - get rid of PR_URGENT, noone uses it.
Copyright © 1996 – 2023 Jason A. Donenfeld. All Rights Reverse Engineered.