diff options
Diffstat (limited to 'Documentation/filesystems')
| -rw-r--r-- | Documentation/filesystems/Locking | 5 | ||||
| -rw-r--r-- | Documentation/filesystems/ext3.txt | 6 | ||||
| -rw-r--r-- | Documentation/filesystems/gfs2-glocks.txt | 119 | ||||
| -rw-r--r-- | Documentation/filesystems/gfs2.txt | 9 | ||||
| -rw-r--r-- | Documentation/filesystems/nfs/pnfs.txt | 2 | ||||
| -rw-r--r-- | Documentation/filesystems/porting | 16 | ||||
| -rw-r--r-- | Documentation/filesystems/proc.txt | 26 | ||||
| -rw-r--r-- | Documentation/filesystems/qnx6.txt | 28 | ||||
| -rw-r--r-- | Documentation/filesystems/vfs.txt | 17 |
9 files changed, 187 insertions, 41 deletions
diff --git a/Documentation/filesystems/Locking b/Documentation/filesystems/Locking index 4fca82e5276e..8e2da1e06e3b 100644 --- a/Documentation/filesystems/Locking +++ b/Documentation/filesystems/Locking @@ -60,8 +60,8 @@ ata *); ssize_t (*getxattr) (struct dentry *, const char *, void *, size_t); ssize_t (*listxattr) (struct dentry *, char *, size_t); int (*removexattr) (struct dentry *, const char *); - void (*truncate_range)(struct inode *, loff_t, loff_t); int (*fiemap)(struct inode *, struct fiemap_extent_info *, u64 start, u64 len); + void (*update_time)(struct inode *, struct timespec *, int); locking rules: all may block @@ -87,8 +87,9 @@ setxattr: yes getxattr: no listxattr: no removexattr: yes -truncate_range: yes fiemap: no +update_time: no + Additionally, ->rmdir(), ->unlink() and ->rename() have ->i_mutex on victim. cross-directory ->rename() has (per-superblock) ->s_vfs_rename_sem. diff --git a/Documentation/filesystems/ext3.txt b/Documentation/filesystems/ext3.txt index b100adc38adb..293855e95000 100644 --- a/Documentation/filesystems/ext3.txt +++ b/Documentation/filesystems/ext3.txt @@ -59,9 +59,9 @@ commit=nrsec (*) Ext3 can be told to sync all its data and metadata Setting it to very large values will improve performance. -barrier=<0(*)|1> This enables/disables the use of write barriers in -barrier the jbd code. barrier=0 disables, barrier=1 enables. -nobarrier (*) This also requires an IO stack which can support +barrier=<0|1(*)> This enables/disables the use of write barriers in +barrier (*) the jbd code. barrier=0 disables, barrier=1 enables. +nobarrier This also requires an IO stack which can support barriers, and if jbd gets an error on a barrier write, it will disable again with a warning. Write barriers enforce proper on-disk ordering diff --git a/Documentation/filesystems/gfs2-glocks.txt b/Documentation/filesystems/gfs2-glocks.txt index 0494f78d87e4..fcc79957be63 100644 --- a/Documentation/filesystems/gfs2-glocks.txt +++ b/Documentation/filesystems/gfs2-glocks.txt @@ -61,7 +61,9 @@ go_unlock | Called on the final local unlock of a lock go_dump | Called to print content of object for debugfs file, or on | error to dump glock to the log. go_type | The type of the glock, LM_TYPE_..... -go_min_hold_time | The minimum hold time +go_callback | Called if the DLM sends a callback to drop this lock +go_flags | GLOF_ASPACE is set, if the glock has an address space + | associated with it The minimum hold time for each lock is the time after a remote lock grant for which we ignore remote demote requests. This is in order to @@ -89,6 +91,7 @@ go_demote_ok | Sometimes | Yes go_lock | Yes | No go_unlock | Yes | No go_dump | Sometimes | Yes +go_callback | Sometimes (N/A) | Yes N.B. Operations must not drop either the bit lock or the spinlock if its held on entry. go_dump and do_demote_ok must never block. @@ -111,4 +114,118 @@ itself (locking order as above), and the other, known as the iopen glock is used in conjunction with the i_nlink field in the inode to determine the lifetime of the inode in question. Locking of inodes is on a per-inode basis. Locking of rgrps is on a per rgrp basis. +In general we prefer to lock local locks prior to cluster locks. + + Glock Statistics + ------------------ + +The stats are divided into two sets: those relating to the +super block and those relating to an individual glock. The +super block stats are done on a per cpu basis in order to +try and reduce the overhead of gathering them. They are also +further divided by glock type. All timings are in nanoseconds. + +In the case of both the super block and glock statistics, +the same information is gathered in each case. The super +block timing statistics are used to provide default values for +the glock timing statistics, so that newly created glocks +should have, as far as possible, a sensible starting point. +The per-glock counters are initialised to zero when the +glock is created. The per-glock statistics are lost when +the glock is ejected from memory. + +The statistics are divided into three pairs of mean and +variance, plus two counters. The mean/variance pairs are +smoothed exponential estimates and the algorithm used is +one which will be very familiar to those used to calculation +of round trip times in network code. See "TCP/IP Illustrated, +Volume 1", W. Richard Stevens, sect 21.3, "Round-Trip Time Measurement", +p. 299 and onwards. Also, Volume 2, Sect. 25.10, p. 838 and onwards. +Unlike the TCP/IP Illustrated case, the mean and variance are +not scaled, but are in units of integer nanoseconds. + +The three pairs of mean/variance measure the following +things: + + 1. DLM lock time (non-blocking requests) + 2. DLM lock time (blocking requests) + 3. Inter-request time (again to the DLM) + +A non-blocking request is one which will complete right +away, whatever the state of the DLM lock in question. That +currently means any requests when (a) the current state of +the lock is exclusive, i.e. a lock demotion (b) the requested +state is either null or unlocked (again, a demotion) or (c) the +"try lock" flag is set. A blocking request covers all the other +lock requests. + +There are two counters. The first is there primarily to show +how many lock requests have been made, and thus how much data +has gone into the mean/variance calculations. The other counter +is counting queuing of holders at the top layer of the glock +code. Hopefully that number will be a lot larger than the number +of dlm lock requests issued. + +So why gather these statistics? There are several reasons +we'd like to get a better idea of these timings: + +1. To be able to better set the glock "min hold time" +2. To spot performance issues more easily +3. To improve the algorithm for selecting resource groups for +allocation (to base it on lock wait time, rather than blindly +using a "try lock") + +Due to the smoothing action of the updates, a step change in +some input quantity being sampled will only fully be taken +into account after 8 samples (or 4 for the variance) and this +needs to be carefully considered when interpreting the +results. + +Knowing both the time it takes a lock request to complete and +the average time between lock requests for a glock means we +can compute the total percentage of the time for which the +node is able to use a glock vs. time that the rest of the +cluster has its share. That will be very useful when setting +the lock min hold time. + +Great care has been taken to ensure that we +measure exactly the quantities that we want, as accurately +as possible. There are always inaccuracies in any +measuring system, but I hope this is as accurate as we +can reasonably make it. + +Per sb stats can be found here: +/sys/kernel/debug/gfs2/<fsname>/sbstats +Per glock stats can be found here: +/sys/kernel/debug/gfs2/<fsname>/glstats + +Assuming that debugfs is mounted on /sys/kernel/debug and also +that <fsname> is replaced with the name of the gfs2 filesystem +in question. + +The abbreviations used in the output as are follows: + +srtt - Smoothed round trip time for non-blocking dlm requests +srttvar - Variance estimate for srtt +srttb - Smoothed round trip time for (potentially) blocking dlm requests +srttvarb - Variance estimate for srttb +sirt - Smoothed inter-request time (for dlm requests) +sirtvar - Variance estimate for sirt +dlm - Number of dlm requests made (dcnt in glstats file) +queue - Number of glock requests queued (qcnt in glstats file) + +The sbstats file contains a set of these stats for each glock type (so 8 lines +for each type) and for each cpu (one column per cpu). The glstats file contains +a set of these stats for each glock in a similar format to the glocks file, but +using the format mean/variance for each of the timing stats. + +The gfs2_glock_lock_time tracepoint prints out the current values of the stats +for the glock in question, along with some addition information on each dlm +reply that is received: + +status - The status of the dlm request +flags - The dlm request flags +tdiff - The time taken by this specific request +(remaining fields as per above list) + diff --git a/Documentation/filesystems/gfs2.txt b/Documentation/filesystems/gfs2.txt index 4cda926628aa..cc4f2306609e 100644 --- a/Documentation/filesystems/gfs2.txt +++ b/Documentation/filesystems/gfs2.txt @@ -1,7 +1,7 @@ Global File System ------------------ -http://sources.redhat.com/cluster/wiki/ +https://fedorahosted.org/cluster/wiki/HomePage GFS is a cluster file system. It allows a cluster of computers to simultaneously use a block device that is shared between them (with FC, @@ -30,7 +30,8 @@ needed, simply: If you are using Fedora, you need to install the gfs2-utils package and, for lock_dlm, you will also need to install the cman package -and write a cluster.conf as per the documentation. +and write a cluster.conf as per the documentation. For F17 and above +cman has been replaced by the dlm package. GFS2 is not on-disk compatible with previous versions of GFS, but it is pretty close. @@ -39,8 +40,6 @@ The following man pages can be found at the URL above: fsck.gfs2 to repair a filesystem gfs2_grow to expand a filesystem online gfs2_jadd to add journals to a filesystem online - gfs2_tool to manipulate, examine and tune a filesystem - gfs2_quota to examine and change quota values in a filesystem + tunegfs2 to manipulate, examine and tune a filesystem gfs2_convert to convert a gfs filesystem to gfs2 in-place - mount.gfs2 to help mount(8) mount a filesystem mkfs.gfs2 to make a filesystem diff --git a/Documentation/filesystems/nfs/pnfs.txt b/Documentation/filesystems/nfs/pnfs.txt index c7919c6e3bea..52ae07f5f578 100644 --- a/Documentation/filesystems/nfs/pnfs.txt +++ b/Documentation/filesystems/nfs/pnfs.txt @@ -93,7 +93,7 @@ The API to the login script is as follows: (allways exists) (More protocols can be defined in the future. The client does not interpret this string it is - passed unchanged as recieved from the Server) + passed unchanged as received from the Server) -o osdname of the requested target OSD (Might be empty) (A string which denotes the OSD name, there is a diff --git a/Documentation/filesystems/porting b/Documentation/filesystems/porting index 74acd9618819..8c91d1057d9a 100644 --- a/Documentation/filesystems/porting +++ b/Documentation/filesystems/porting @@ -297,7 +297,8 @@ in the beginning of ->setattr unconditionally. be used instead. It gets called whenever the inode is evicted, whether it has remaining links or not. Caller does *not* evict the pagecache or inode-associated metadata buffers; getting rid of those is responsibility of method, as it had -been for ->delete_inode(). +been for ->delete_inode(). Caller makes sure async writeback cannot be running +for the inode while (or after) ->evict_inode() is called. ->drop_inode() returns int now; it's called on final iput() with inode->i_lock held and it returns true if filesystems wants the inode to be @@ -306,14 +307,11 @@ updated appropriately. generic_delete_inode() is also alive and it consists simply of return 1. Note that all actual eviction work is done by caller after ->drop_inode() returns. - clear_inode() is gone; use end_writeback() instead. As before, it must -be called exactly once on each call of ->evict_inode() (as it used to be for -each call of ->delete_inode()). Unlike before, if you are using inode-associated -metadata buffers (i.e. mark_buffer_dirty_inode()), it's your responsibility to -call invalidate_inode_buffers() before end_writeback(). - No async writeback (and thus no calls of ->write_inode()) will happen -after end_writeback() returns, so actions that should not overlap with ->write_inode() -(e.g. freeing on-disk inode if i_nlink is 0) ought to be done after that call. + As before, clear_inode() must be called exactly once on each call of +->evict_inode() (as it used to be for each call of ->delete_inode()). Unlike +before, if you are using inode-associated metadata buffers (i.e. +mark_buffer_dirty_inode()), it's your responsibility to call +invalidate_inode_buffers() before clear_inode(). NOTE: checking i_nlink in the beginning of ->write_inode() and bailing out if it's zero is not *and* *never* *had* *been* enough. Final unlink() and iput() diff --git a/Documentation/filesystems/proc.txt b/Documentation/filesystems/proc.txt index b7413cb46dcb..fb0a6aeb936c 100644 --- a/Documentation/filesystems/proc.txt +++ b/Documentation/filesystems/proc.txt @@ -40,6 +40,7 @@ Table of Contents 3.4 /proc/<pid>/coredump_filter - Core dump filtering settings 3.5 /proc/<pid>/mountinfo - Information about mounts 3.6 /proc/<pid>/comm & /proc/<pid>/task/<tid>/comm + 3.7 /proc/<pid>/task/<tid>/children - Information about task children 4 Configuring procfs 4.1 Mount options @@ -310,6 +311,11 @@ Table 1-4: Contents of the stat files (as of 2.6.30-rc7) start_data address above which program data+bss is placed end_data address below which program data+bss is placed start_brk address above which program heap can be expanded with brk() + arg_start address above which program command line is placed + arg_end address below which program command line is placed + env_start address above which program environment is placed + env_end address below which program environment is placed + exit_code the thread's exit_code in the form reported by the waitpid system call .............................................................................. The /proc/PID/maps file containing the currently mapped memory regions and @@ -743,6 +749,7 @@ Committed_AS: 100056 kB VmallocTotal: 112216 kB VmallocUsed: 428 kB VmallocChunk: 111088 kB +AnonHugePages: 49152 kB MemTotal: Total usable ram (i.e. physical ram minus a few reserved bits and the kernel binary code) @@ -776,6 +783,7 @@ VmallocChunk: 111088 kB Dirty: Memory which is waiting to get written back to the disk Writeback: Memory which is actively being written back to the disk AnonPages: Non-file backed pages mapped into userspace page tables +AnonHugePages: Non-file backed huge pages mapped into userspace page tables Mapped: files which have been mmaped, such as libraries Slab: in-kernel data structures cache SReclaimable: Part of Slab, that might be reclaimed, such as caches @@ -996,7 +1004,6 @@ Table 1-9: Network info in /proc/net snmp SNMP data sockstat Socket statistics tcp TCP sockets - tr_rif Token ring RIF routing table udp UDP sockets unix UNIX domain sockets wireless Wireless interface data (Wavelan etc) @@ -1577,6 +1584,23 @@ then the kernel's TASK_COMM_LEN (currently 16 chars) will result in a truncated comm value. +3.7 /proc/<pid>/task/<tid>/children - Information about task children +------------------------------------------------------------------------- +This file provides a fast way to retrieve first level children pids +of a task pointed by <pid>/<tid> pair. The format is a space separated +stream of pids. + +Note the "first level" here -- if a child has own children they will +not be listed here, one needs to read /proc/<children-pid>/task/<tid>/children +to obtain the descendants. + +Since this interface is intended to be fast and cheap it doesn't +guarantee to provide precise results and some children might be +skipped, especially if they've exited right after we printed their +pids, so one need to either stop or freeze processes being inspected +if precise results are needed. + + ------------------------------------------------------------------------------ Configuring procfs ------------------------------------------------------------------------------ diff --git a/Documentation/filesystems/qnx6.txt b/Documentation/filesystems/qnx6.txt index 050223ea03c7..e59f2f09f56e 100644 --- a/Documentation/filesystems/qnx6.txt +++ b/Documentation/filesystems/qnx6.txt @@ -17,7 +17,7 @@ concepts of blocks, inodes and directories. On QNX it is possible to create little endian and big endian qnx6 filesystems. This feature makes it possible to create and use a different endianness fs for the target (QNX is used on quite a range of embedded systems) plattform -running on a different endianess. +running on a different endianness. The Linux driver handles endianness transparently. (LE and BE) Blocks @@ -26,7 +26,7 @@ Blocks The space in the device or file is split up into blocks. These are a fixed size of 512, 1024, 2048 or 4096, which is decided when the filesystem is created. -Blockpointers are 32bit, so the maximum space that can be adressed is +Blockpointers are 32bit, so the maximum space that can be addressed is 2^32 * 4096 bytes or 16TB The superblocks @@ -47,16 +47,16 @@ inactive superblock. Each superblock holds a set of root inodes for the different filesystem parts. (Inode, Bitmap and Longfilenames) Each of these root nodes holds information like total size of the stored -data and the adressing levels in that specific tree. -If the level value is 0, up to 16 direct blocks can be adressed by each +data and the addressing levels in that specific tree. +If the level value is 0, up to 16 direct blocks can be addressed by each node. -Level 1 adds an additional indirect adressing level where each indirect -adressing block holds up to blocksize / 4 bytes pointers to data blocks. -Level 2 adds an additional indirect adressig block level (so, already up -to 16 * 256 * 256 = 1048576 blocks that can be adressed by such a tree)a +Level 1 adds an additional indirect addressing level where each indirect +addressing block holds up to blocksize / 4 bytes pointers to data blocks. +Level 2 adds an additional indirect addressing block level (so, already up +to 16 * 256 * 256 = 1048576 blocks that can be addressed by such a tree). Unused block pointers are always set to ~0 - regardless of root node, -indirect adressing blocks or inodes. +indirect addressing blocks or inodes. Data leaves are always on the lowest level. So no data is stored on upper tree levels. @@ -64,7 +64,7 @@ The first Superblock is located at 0x2000. (0x2000 is the bootblock size) The Audi MMI 3G first superblock directly starts at byte 0. Second superblock position can either be calculated from the superblock information (total number of filesystem blocks) or by taking the highest -device address, zeroing the last 3 bytes and then substracting 0x1000 from +device address, zeroing the last 3 bytes and then subtracting 0x1000 from that address. 0x1000 is the size reserved for each superblock - regardless of the @@ -83,8 +83,8 @@ size, number of blocks used, access time, change time and modification time. Object mode field is POSIX format. (which makes things easier) There are also pointers to the first 16 blocks, if the object data can be -adressed with 16 direct blocks. -For more than 16 blocks an indirect adressing in form of another tree is +addressed with 16 direct blocks. +For more than 16 blocks an indirect addressing in form of another tree is used. (scheme is the same as the one used for the superblock root nodes) The filesize is stored 64bit. Inode counting starts with 1. (whilst long @@ -118,13 +118,13 @@ no block pointers and the directory file record pointing to the target file inode. Character and block special devices do not exist in QNX as those files -are handled by the QNX kernel/drivers and created in /dev independant of the +are handled by the QNX kernel/drivers and created in /dev independent of the underlaying filesystem. Long filenames -------------- -Long filenames are stored in a seperate adressing tree. The staring point +Long filenames are stored in a separate addressing tree. The staring point is the longfilename root node in the active superblock. Each data block (tree leaves) holds one long filename. That filename is limited to 510 bytes. The first two starting bytes are used as length field diff --git a/Documentation/filesystems/vfs.txt b/Documentation/filesystems/vfs.txt index 0d0492028082..efd23f481704 100644 --- a/Documentation/filesystems/vfs.txt +++ b/Documentation/filesystems/vfs.txt @@ -363,7 +363,7 @@ struct inode_operations { ssize_t (*getxattr) (struct dentry *, const char *, void *, size_t); ssize_t (*listxattr) (struct dentry *, char *, size_t); int (*removexattr) (struct dentry *, const char *); - void (*truncate_range)(struct inode *, loff_t, loff_t); + void (*update_time)(struct inode *, struct timespec *, int); }; Again, all methods are called without any locks being held, unless @@ -472,9 +472,9 @@ otherwise noted. removexattr: called by the VFS to remove an extended attribute from a file. This method is called by removexattr(2) system call. - truncate_range: a method provided by the underlying filesystem to truncate a - range of blocks , i.e. punch a hole somewhere in a file. - + update_time: called by the VFS to update a specific time or the i_version of + an inode. If this is not defined the VFS will update the inode itself + and call mark_inode_dirty_sync. The Address Space Object ======================== @@ -760,7 +760,7 @@ struct file_operations ---------------------- This describes how the VFS can manipulate an open file. As of kernel -2.6.22, the following members are defined: +3.5, the following members are defined: struct file_operations { struct module *owner; @@ -790,6 +790,8 @@ struct file_operations { int (*flock) (struct file *, int, struct file_lock *); ssize_t (*splice_write)(struct pipe_inode_info *, struct file *, size_t, unsigned int); ssize_t (*splice_read)(struct file *, struct pipe_inode_info *, size_t, unsigned int); + int (*setlease)(struct file *, long arg, struct file_lock **); + long (*fallocate)(struct file *, int mode, loff_t offset, loff_t len); }; Again, all methods are called without any locks being held, unless @@ -858,6 +860,11 @@ otherwise noted. splice_read: called by the VFS to splice data from file to a pipe. This method is used by the splice(2) system call + setlease: called by the VFS to set or release a file lock lease. + setlease has the file_lock_lock held and must not sleep. + + fallocate: called by the VFS to preallocate blocks or punch a hole. + Note that the file operations are implemented by the specific filesystem in which the inode resides. When opening a device node (character or block special) most filesystems will call special |