diff options
| author |   Pekka Enberg <penberg@cs.helsinki.fi> | 2009-06-12 14:03:06 +0300 |
|---|---|---|
| committer | Pekka Enberg <penberg@cs.helsinki.fi> | 2009-06-12 18:53:33 +0300 |
| commit | 7e85ee0c1d15ca5f8bff0f514f158eba1742dd87 (patch) | |
| tree | 8f9c21f0df6bea88740d7dd48834ac9ffc238e93 /mm/slab.c | |
| parent | slab: fix gfp flag in setup_cpu_cache() (diff) | |
| download | linux-dev-7e85ee0c1d15ca5f8bff0f514f158eba1742dd87.tar.xz linux-dev-7e85ee0c1d15ca5f8bff0f514f158eba1742dd87.zip | |
slab,slub: don't enable interrupts during early boot
As explained by Benjamin Herrenschmidt:
Oh and btw, your patch alone doesn't fix powerpc, because it's missing
a whole bunch of GFP_KERNEL's in the arch code... You would have to
grep the entire kernel for things that check slab_is_available() and
even then you'll be missing some.
For example, slab_is_available() didn't always exist, and so in the
early days on powerpc, we used a mem_init_done global that is set form
mem_init() (not perfect but works in practice). And we still have code
using that to do the test.
Therefore, mask out __GFP_WAIT, __GFP_IO, and __GFP_FS in the slab allocators
in early boot code to avoid enabling interrupts.
Signed-off-by: Pekka Enberg <penberg@cs.helsinki.fi>
Diffstat (limited to 'mm/slab.c')
| -rw-r--r-- | mm/slab.c | 18 |
1 files changed, 18 insertions, 0 deletions
diff --git a/mm/slab.c b/mm/slab.c index cd76964b53bc..453efcb1c980 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -304,6 +304,12 @@ struct kmem_list3 { }; /* + * The slab allocator is initialized with interrupts disabled. Therefore, make + * sure early boot allocations don't accidentally enable interrupts. + */ +static gfp_t slab_gfp_mask __read_mostly = SLAB_GFP_BOOT_MASK; + +/* * Need this for bootstrapping a per node allocator. */ #define NUM_INIT_LISTS (3 * MAX_NUMNODES) @@ -1654,6 +1660,14 @@ void __init kmem_cache_init(void) */ } +void __init kmem_cache_init_late(void) +{ + /* + * Interrupts are enabled now so all GFP allocations are safe. + */ + slab_gfp_mask = __GFP_BITS_MASK; +} + static int __init cpucache_init(void) { int cpu; @@ -3354,6 +3368,8 @@ __cache_alloc_node(struct kmem_cache *cachep, gfp_t flags, int nodeid, unsigned long save_flags; void *ptr; + flags &= slab_gfp_mask; + lockdep_trace_alloc(flags); if (slab_should_failslab(cachep, flags)) @@ -3434,6 +3450,8 @@ __cache_alloc(struct kmem_cache *cachep, gfp_t flags, void *caller) unsigned long save_flags; void *objp; + flags &= slab_gfp_mask; + lockdep_trace_alloc(flags); if (slab_should_failslab(cachep, flags)) |