| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
This is much the same as for SPARC except that we can do the find_zero()
function more efficiently using the count-leading-zeroes instructions.
Tested on 32-bit and 64-bit PowerPC.
Signed-off-by: Paul Mackerras <paulus@samba.org>
Acked-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
|
|
The aligned_byte_mask() definition is wrong for 32-bit big-endian
machines: the "7-(n)" part of the definition assumes a long is 8
bytes. This fixes it by using BITS_PER_LONG - 8 instead of 8*7.
Tested on 32-bit and 64-bit PowerPC.
Signed-off-by: Paul Mackerras <paulus@samba.org>
Acked-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
|
|
After 303395ac3bf3e2cb488435537d416bc840438fcb, some headers are
autogenerated. Include these autogenerated headers (mainly
unistd_32_ia32.h) in out-of-tree builds to allow DKMS modules to be
built succesfully.
Signed-off-by: Peter Lekensteyn <lekensteyn@gmail.com>
Signed-off-by: Michal Marek <mmarek@suse.cz>
|
|
This throws away the sparc-specific functions in favor of the generic
optimized version.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
|
|
This throws away the old x86-specific functions in favor of the generic
optimized version.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
|
|
This adds a new generic optimized strnlen_user() function that uses the
<asm/word-at-a-time.h> infrastructure to portably do efficient string
handling.
In many ways, strnlen is much simpler than strncpy, and in particular we
can always pre-align the words we load from memory. That means that all
the worries about alignment etc are a non-issue, so this one can easily
be used on any architecture. You obviously do have to do the
appropriate word-at-a-time.h macros.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
|
|
This changes the interfaces in <asm/word-at-a-time.h> to be a bit more
complicated, but a lot more generic.
In particular, it allows us to really do the operations efficiently on
both little-endian and big-endian machines, pretty much regardless of
machine details. For example, if you can rely on a fast population
count instruction on your architecture, this will allow you to make your
optimized <asm/word-at-a-time.h> file with that.
NOTE! The "generic" version in include/asm-generic/word-at-a-time.h is
not truly generic, it actually only works on big-endian. Why? Because
on little-endian the generic algorithms are wasteful, since you can
inevitably do better. The x86 implementation is an example of that.
(The only truly non-generic part of the asm-generic implementation is
the "find_zero()" function, and you could make a little-endian version
of it. And if the Kbuild infrastructure allowed us to pick a particular
header file, that would be lovely)
The <asm/word-at-a-time.h> functions are as follows:
- WORD_AT_A_TIME_CONSTANTS: specific constants that the algorithm
uses.
- has_zero(): take a word, and determine if it has a zero byte in it.
It gets the word, the pointer to the constant pool, and a pointer to
an intermediate "data" field it can set.
This is the "quick-and-dirty" zero tester: it's what is run inside
the hot loops.
- "prep_zero_mask()": take the word, the data that has_zero() produced,
and the constant pool, and generate an *exact* mask of which byte had
the first zero. This is run directly *outside* the loop, and allows
the "has_zero()" function to answer the "is there a zero byte"
question without necessarily getting exactly *which* byte is the
first one to contain a zero.
If you do multiple byte lookups concurrently (eg "hash_name()", which
looks for both NUL and '/' bytes), after you've done the prep_zero_mask()
phase, the result of those can be or'ed together to get the "either
or" case.
- The result from "prep_zero_mask()" can then be fed into "find_zero()"
(to find the byte offset of the first byte that was zero) or into
"zero_bytemask()" (to find the bytemask of the bytes preceding the
zero byte).
The existence of zero_bytemask() is optional, and is not necessary
for the normal string routines. But dentry name hashing needs it, so
if you enable DENTRY_WORD_AT_A_TIME you need to expose it.
This changes the generic strncpy_from_user() function and the dentry
hashing functions to use these modified word-at-a-time interfaces. This
gets us back to the optimized state of the x86 strncpy that we lost in
the previous commit when moving over to the generic version.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
|
|
The generic strncpy_from_user() is not really optimal, since it is
designed to work on both little-endian and big-endian. And on
little-endian you can simplify much of the logic to find the first zero
byte, since little-endian arithmetic doesn't have to worry about the
carry bit propagating into earlier bytes (only later bytes, which we
don't care about).
But I have patches to make the generic routines use the architecture-
specific <asm/word-at-a-time.h> infrastructure, so that we can regain
the little-endian optimizations. But before we do that, switch over to
the generic routines to make the patches each do just one well-defined
thing.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
|
|
Now we have four copies of this code, Linus "suggested" it was about time
we stopped copying it and turned it into a helper.
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
|
|
Add device info into list before doing context mapping, because device
info will be used by iommu_enable_dev_iotlb(). Without it, ATS won't get
enabled as it should be.
ATS, while a dubious decision from a security point of view, can be very
important for performance.
Signed-off-by: Xudong Hao <xudong.hao@intel.com>
Signed-off-by: Xiantao Zhang <xiantao.zhang@intel.com>
Acked-by: Chris Wright <chrisw@sous-sol.org>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
Cc: stable@kernel.org
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
|
|
There is no "ARCH=tile" (just like there is no "ARCH=x86") so we need
to pick a default configuration, either tilepro or tilegx, when users
specify ARCH=tile. We'll use tilegx, since that's our current chip.
Reported-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
|
|
This is because __builtin_clz(0) returns 64 for the "undefined" case
of 0, since the builtin just does a right-shift 32 and "clz" instruction.
So, use the alpha approach of casting to u32 and using __builtin_clzll().
Cc: stable@vger.kernel.org
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
|
|
Also create a TILEPRO config setting to use for #ifdefs where it
is cleaner to do so, and make the 64BIT setting depend directly
on the setting of TILEGX.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
|
|
Commit d065bd810b6deb67d4897a14bfe21f8eb526ba99
(mm: retry page fault when blocking on disk transfer) and
commit 37b23e0525d393d48a7d59f870b3bc061a30ccdb
(x86,mm: make pagefault killable)
The above commits introduced changes into the x86 pagefault handler
for making the page fault handler retryable as well as killable.
These changes reduce the mmap_sem hold time, which is crucial
during OOM killer invocation.
Port these changes to tile.
Signed-off-by: Kautuk Consul <consul.kautuk@gmail.com>
[cmetcalf@tilera.com: initialize "flags" after "write" updated.]
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
|
|
If the kernel unexpectedly takes a bad trap, it's convenient to
have it report the type of trap as part of the error. This gives
customers a bit more context before they call up customer support.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
|
|
This just adds a few more attributes to the information Linux
can query from the hypervisor for the /sys/hypervisor/board/ directory,
providing part, serial#, revision#, and description for cpu modules
(as opposed to the board itself, or any mezzanine boards).
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
|
|
The hardwall drain code was not properly implemented for tilegx,
just tilepro, so you couldn't reliably restart an application that
made use of the udn.
In addition, the code was only applicable to the udn (user dynamic
network). On tilegx there is a second user network that is available
(the "idn"), and there is support for having I/O shims deliver
user-level interrupts to applications ("ipi") which functions in a
very similar way to the inter-core permissions used for udn/idn.
So this change also generalizes the code from supporting just the udn
to supports udn/idn/ipi on tilegx.
By default we now use /dev/hardwall/{udn,idn,ipi} with separate
minor numbers for the three devices.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
|
|
This change adds support for a new "super" bit in the PTE, using the new
arch_make_huge_pte() method. The Tilera hypervisor sees the bit set at a
given level of the page table and gangs together 4, 16, or 64 consecutive
pages from that level of the hierarchy to create a larger TLB entry.
One extra "super" page size can be specified at each of the three levels
of the page table hierarchy on tilegx, using the "hugepagesz" argument
on the boot command line. A new hypervisor API is added to allow Linux
to tell the hypervisor how many PTEs to gang together at each level of
the page table.
To allow pre-allocating huge pages larger than the buddy allocator can
handle, this change modifies the Tilera bootmem support to put all of
memory on tilegx platforms into bootmem.
As part of this change I eliminate the vestigial CONFIG_HIGHPTE support,
which never worked anyway, and eliminate the hv_page_size() API in favor
of the standard vma_kernel_pagesize() API.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
|
|
The tile support for multiple-size huge pages requires tagging
the hugetlb PTE with a "super" bit for PTEs that are multiples of
the basic size of a pagetable span. To set that bit properly
we need to tweak the PTe in make_huge_pte() based on the vma.
This change provides the API for a subsequent tile-specific
change to use.
Reviewed-by: Hillf Danton <dhillf@gmail.com>
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
|
|
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
|
|
We already had a syscall that did some dcache flushing, but it was
not used in practice. Make it MIPS compatible instead so it can
do both the DCACHE and ICACHE actions. We have code that wants to
be able to use the ICACHE flush mode from userspace so this change
enables that.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
|
|
This change introduces new flags for the hv_install_context()
API that passes a page table pointer to the hypervisor. Clients
can explicitly request 4K, 16K, or 64K small pages when they
install a new context. In practice, the page size is fixed at
kernel compile time and the same size is always requested every
time a new page table is installed.
The <hv/hypervisor.h> header changes so that it provides more abstract
macros for managing "page" things like PFNs and page tables. For
example there is now a HV_DEFAULT_PAGE_SIZE_SMALL instead of the old
HV_PAGE_SIZE_SMALL. The various PFN routines have been eliminated and
only PA- or PTFN-based ones remain (since PTFNs are always expressed
in fixed 2KB "page" size). The page-table management macros are
renamed with a leading underscore and take page-size arguments with
the presumption that clients will use those macros in some single
place to provide the "real" macros they will use themselves.
I happened to notice the old hv_set_caching() API was totally broken
(it assumed 4KB pages) so I changed it so it would nominally work
correctly with other page sizes.
Tag modules with the page size so you can't load a module built with
a conflicting page size. (And add a test for SMP while we're at it.)
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
|
|
Use direct load/store for the get_user/put_user.
Previously, we would call out to a helper routine that would do the
appropriate thing and then return, handling the possible exception
internally. Now we inline the load or store, along with a "we succeeded"
indication in a register; if the load or store faults, we write a
"we failed" indication into the same register and then return to the
following instruction. This is more efficient and gives us more compact
code, as well as being more in line with what other architectures do.
The special futex assembly source file for TILE-Gx also disappears in
this change; we just use the same inlining idiom there as well, putting
the appropriate atomic operations directly into futex_atomic_op_inuser()
(and thus into the FUTEX_WAIT function).
The underlying atomic copy_from_user, copy_to_user functions were
renamed using the (cryptic) x86 convention as copy_from_user_ll and
copy_to_user_ll.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
|
|
The toolchain supports big-endian mode now, so add support for building
the kernel to run big-endian as well.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
|
|
The change adds some infrastructure for managing tile pmd's more generally,
using pte_pmd() and pmd_pte() methods to translate pmd values to and
from ptes, since on TILEPro a pmd is really just a nested structure
holding a pgd (aka pte). Several existing pmd methods are moved into
this framework, and a whole raft of additional pmd accessors are defined
that are used by the transparent hugepage framework.
The tile PTE now has a "client2" bit. The bit is used to indicate a
transparent huge page is in the process of being split into subpages.
This change also fixes a generic bug where the return value of the
generic pmdp_splitting_flush() was incorrect.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
|
|
In general we want to avoid ever touching memory while within an
interrupt critical section, since the page fault path goes through
a different path from the hypervisor when in an interrupt critical
section, and we carefully decided with tilegx that we didn't need
to support this path in the kernel. (On tilepro we did implement
that path as part of supporting atomic instructions in software.)
In practice we always need to touch the kernel stack, since that's
where we store the interrupt state before releasing the critical
section, but this change cleans up a few things. The IRQ_ENABLE
macro is split up so that when we want to enable interrupts in a
deferred way (e.g. for cpu_idle or for interrupt return) we can
read the per-cpu enable mask before entering the critical section.
The cache-migration code is changed to use interrupt masking instead
of interrupt critical sections. And, the interrupt-entry code is
changed so that we defer loading "tp" from per-cpu data until after
we have released the interrupt critical section.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
|
|
As per commits 2922585b9329 ("lib: Sparc's strncpy_from_user is generic
enough, move under lib/") and 92ae03f2ef99 ("x86: merge 32/64-bit
versions of 'strncpy_from_user()' and speed it up"), and corresponding
discussion on linux-arch.
Signed-off-by: Jonas Bonn <jonas@southpole.se>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
|
Paul Mackerras
Lekensteyn
David Miller
Linus Torvalds
David Woodhouse
Chris Metcalf
Kautuk Consul
Jonas Bonn