diff options
Diffstat (limited to 'Documentation/admin-guide/aoe')
| -rw-r--r-- | Documentation/admin-guide/aoe/aoe.rst | 150 | ||||
| -rw-r--r-- | Documentation/admin-guide/aoe/autoload.sh | 17 | ||||
| -rw-r--r-- | Documentation/admin-guide/aoe/examples.rst | 23 | ||||
| -rw-r--r-- | Documentation/admin-guide/aoe/index.rst | 17 | ||||
| -rw-r--r-- | Documentation/admin-guide/aoe/status.sh | 30 | ||||
| -rw-r--r-- | Documentation/admin-guide/aoe/todo.rst | 17 | ||||
| -rw-r--r-- | Documentation/admin-guide/aoe/udev-install.sh | 33 | ||||
| -rw-r--r-- | Documentation/admin-guide/aoe/udev.txt | 26 |
8 files changed, 313 insertions, 0 deletions
diff --git a/Documentation/admin-guide/aoe/aoe.rst b/Documentation/admin-guide/aoe/aoe.rst new file mode 100644 index 000000000000..a05e751363a0 --- /dev/null +++ b/Documentation/admin-guide/aoe/aoe.rst @@ -0,0 +1,150 @@ +Introduction +============ + +ATA over Ethernet is a network protocol that provides simple access to +block storage on the LAN. + + http://support.coraid.com/documents/AoEr11.txt + +The EtherDrive (R) HOWTO for 2.6 and 3.x kernels is found at ... + + http://support.coraid.com/support/linux/EtherDrive-2.6-HOWTO.html + +It has many tips and hints! Please see, especially, recommended +tunings for virtual memory: + + http://support.coraid.com/support/linux/EtherDrive-2.6-HOWTO-5.html#ss5.19 + +The aoetools are userland programs that are designed to work with this +driver. The aoetools are on sourceforge. + + http://aoetools.sourceforge.net/ + +The scripts in this Documentation/admin-guide/aoe directory are intended to +document the use of the driver and are not necessary if you install +the aoetools. + + +Creating Device Nodes +===================== + + Users of udev should find the block device nodes created + automatically, but to create all the necessary device nodes, use the + udev configuration rules provided in udev.txt (in this directory). + + There is a udev-install.sh script that shows how to install these + rules on your system. + + There is also an autoload script that shows how to edit + /etc/modprobe.d/aoe.conf to ensure that the aoe module is loaded when + necessary. Preloading the aoe module is preferable to autoloading, + however, because AoE discovery takes a few seconds. It can be + confusing when an AoE device is not present the first time the a + command is run but appears a second later. + +Using Device Nodes +================== + + "cat /dev/etherd/err" blocks, waiting for error diagnostic output, + like any retransmitted packets. + + "echo eth2 eth4 > /dev/etherd/interfaces" tells the aoe driver to + limit ATA over Ethernet traffic to eth2 and eth4. AoE traffic from + untrusted networks should be ignored as a matter of security. See + also the aoe_iflist driver option described below. + + "echo > /dev/etherd/discover" tells the driver to find out what AoE + devices are available. + + In the future these character devices may disappear and be replaced + by sysfs counterparts. Using the commands in aoetools insulates + users from these implementation details. + + The block devices are named like this:: + + e{shelf}.{slot} + e{shelf}.{slot}p{part} + + ... so that "e0.2" is the third blade from the left (slot 2) in the + first shelf (shelf address zero). That's the whole disk. The first + partition on that disk would be "e0.2p1". + +Using sysfs +=========== + + Each aoe block device in /sys/block has the extra attributes of + state, mac, and netif. The state attribute is "up" when the device + is ready for I/O and "down" if detected but unusable. The + "down,closewait" state shows that the device is still open and + cannot come up again until it has been closed. + + The mac attribute is the ethernet address of the remote AoE device. + The netif attribute is the network interface on the localhost + through which we are communicating with the remote AoE device. + + There is a script in this directory that formats this information in + a convenient way. Users with aoetools should use the aoe-stat + command:: + + root@makki root# sh Documentation/admin-guide/aoe/status.sh + e10.0 eth3 up + e10.1 eth3 up + e10.2 eth3 up + e10.3 eth3 up + e10.4 eth3 up + e10.5 eth3 up + e10.6 eth3 up + e10.7 eth3 up + e10.8 eth3 up + e10.9 eth3 up + e4.0 eth1 up + e4.1 eth1 up + e4.2 eth1 up + e4.3 eth1 up + e4.4 eth1 up + e4.5 eth1 up + e4.6 eth1 up + e4.7 eth1 up + e4.8 eth1 up + e4.9 eth1 up + + Use /sys/module/aoe/parameters/aoe_iflist (or better, the driver + option discussed below) instead of /dev/etherd/interfaces to limit + AoE traffic to the network interfaces in the given + whitespace-separated list. Unlike the old character device, the + sysfs entry can be read from as well as written to. + + It's helpful to trigger discovery after setting the list of allowed + interfaces. The aoetools package provides an aoe-discover script + for this purpose. You can also directly use the + /dev/etherd/discover special file described above. + +Driver Options +============== + + There is a boot option for the built-in aoe driver and a + corresponding module parameter, aoe_iflist. Without this option, + all network interfaces may be used for ATA over Ethernet. Here is a + usage example for the module parameter:: + + modprobe aoe_iflist="eth1 eth3" + + The aoe_deadsecs module parameter determines the maximum number of + seconds that the driver will wait for an AoE device to provide a + response to an AoE command. After aoe_deadsecs seconds have + elapsed, the AoE device will be marked as "down". A value of zero + is supported for testing purposes and makes the aoe driver keep + trying AoE commands forever. + + The aoe_maxout module parameter has a default of 128. This is the + maximum number of unresponded packets that will be sent to an AoE + target at one time. + + The aoe_dyndevs module parameter defaults to 1, meaning that the + driver will assign a block device minor number to a discovered AoE + target based on the order of its discovery. With dynamic minor + device numbers in use, a greater range of AoE shelf and slot + addresses can be supported. Users with udev will never have to + think about minor numbers. Using aoe_dyndevs=0 allows device nodes + to be pre-created using a static minor-number scheme with the + aoe-mkshelf script in the aoetools. diff --git a/Documentation/admin-guide/aoe/autoload.sh b/Documentation/admin-guide/aoe/autoload.sh new file mode 100644 index 000000000000..815dff4691c9 --- /dev/null +++ b/Documentation/admin-guide/aoe/autoload.sh @@ -0,0 +1,17 @@ +#!/bin/sh +# set aoe to autoload by installing the +# aliases in /etc/modprobe.d/ + +f=/etc/modprobe.d/aoe.conf + +if test ! -r $f || test ! -w $f; then + echo "cannot configure $f for module autoloading" 1>&2 + exit 1 +fi + +grep major-152 $f >/dev/null +if [ $? = 1 ]; then + echo alias block-major-152 aoe >> $f + echo alias char-major-152 aoe >> $f +fi + diff --git a/Documentation/admin-guide/aoe/examples.rst b/Documentation/admin-guide/aoe/examples.rst new file mode 100644 index 000000000000..91f3198e52c1 --- /dev/null +++ b/Documentation/admin-guide/aoe/examples.rst @@ -0,0 +1,23 @@ +Example of udev rules +--------------------- + + .. include:: udev.txt + :literal: + +Example of udev install rules script +------------------------------------ + + .. literalinclude:: udev-install.sh + :language: shell + +Example script to get status +---------------------------- + + .. literalinclude:: status.sh + :language: shell + +Example of AoE autoload script +------------------------------ + + .. literalinclude:: autoload.sh + :language: shell diff --git a/Documentation/admin-guide/aoe/index.rst b/Documentation/admin-guide/aoe/index.rst new file mode 100644 index 000000000000..d71c5df15922 --- /dev/null +++ b/Documentation/admin-guide/aoe/index.rst @@ -0,0 +1,17 @@ +======================= +ATA over Ethernet (AoE) +======================= + +.. toctree:: + :maxdepth: 1 + + aoe + todo + examples + +.. only:: subproject and html + + Indices + ======= + + * :ref:`genindex` diff --git a/Documentation/admin-guide/aoe/status.sh b/Documentation/admin-guide/aoe/status.sh new file mode 100644 index 000000000000..eeec7baae57a --- /dev/null +++ b/Documentation/admin-guide/aoe/status.sh @@ -0,0 +1,30 @@ +#! /bin/sh +# collate and present sysfs information about AoE storage +# +# A more complete version of this script is aoe-stat, in the +# aoetools. + +set -e +format="%8s\t%8s\t%8s\n" +me=`basename $0` +sysd=${sysfs_dir:-/sys} + +# printf "$format" device mac netif state + +# Suse 9.1 Pro doesn't put /sys in /etc/mtab +#test -z "`mount | grep sysfs`" && { +test ! -d "$sysd/block" && { + echo "$me Error: sysfs is not mounted" 1>&2 + exit 1 +} + +for d in `ls -d $sysd/block/etherd* 2>/dev/null | grep -v p` end; do + # maybe ls comes up empty, so we use "end" + test $d = end && continue + + dev=`echo "$d" | sed 's/.*!//'` + printf "$format" \ + "$dev" \ + "`cat \"$d/netif\"`" \ + "`cat \"$d/state\"`" +done | sort diff --git a/Documentation/admin-guide/aoe/todo.rst b/Documentation/admin-guide/aoe/todo.rst new file mode 100644 index 000000000000..dea8db5a33e1 --- /dev/null +++ b/Documentation/admin-guide/aoe/todo.rst @@ -0,0 +1,17 @@ +TODO +==== + +There is a potential for deadlock when allocating a struct sk_buff for +data that needs to be written out to aoe storage. If the data is +being written from a dirty page in order to free that page, and if +there are no other pages available, then deadlock may occur when a +free page is needed for the sk_buff allocation. This situation has +not been observed, but it would be nice to eliminate any potential for +deadlock under memory pressure. + +Because ATA over Ethernet is not fragmented by the kernel's IP code, +the destructor member of the struct sk_buff is available to the aoe +driver. By using a mempool for allocating all but the first few +sk_buffs, and by registering a destructor, we should be able to +efficiently allocate sk_buffs without introducing any potential for +deadlock. diff --git a/Documentation/admin-guide/aoe/udev-install.sh b/Documentation/admin-guide/aoe/udev-install.sh new file mode 100644 index 000000000000..15e86f58c036 --- /dev/null +++ b/Documentation/admin-guide/aoe/udev-install.sh @@ -0,0 +1,33 @@ +# install the aoe-specific udev rules from udev.txt into +# the system's udev configuration +# + +me="`basename $0`" + +# find udev.conf, often /etc/udev/udev.conf +# (or environment can specify where to find udev.conf) +# +if test -z "$conf"; then + if test -r /etc/udev/udev.conf; then + conf=/etc/udev/udev.conf + else + conf="`find /etc -type f -name udev.conf 2> /dev/null`" + if test -z "$conf" || test ! -r "$conf"; then + echo "$me Error: no udev.conf found" 1>&2 + exit 1 + fi + fi +fi + +# find the directory where udev rules are stored, often +# /etc/udev/rules.d +# +rules_d="`sed -n '/^udev_rules=/{ s!udev_rules=!!; s!\"!!g; p; }' $conf`" +if test -z "$rules_d" ; then + rules_d=/etc/udev/rules.d +fi +if test ! -d "$rules_d"; then + echo "$me Error: cannot find udev rules directory" 1>&2 + exit 1 +fi +sh -xc "cp `dirname $0`/udev.txt $rules_d/60-aoe.rules" diff --git a/Documentation/admin-guide/aoe/udev.txt b/Documentation/admin-guide/aoe/udev.txt new file mode 100644 index 000000000000..5fb756466bc7 --- /dev/null +++ b/Documentation/admin-guide/aoe/udev.txt @@ -0,0 +1,26 @@ +# These rules tell udev what device nodes to create for aoe support. +# They may be installed along the following lines. Check the section +# 8 udev manpage to see whether your udev supports SUBSYSTEM, and +# whether it uses one or two equal signs for SUBSYSTEM and KERNEL. +# +# ecashin@makki ~$ su +# Password: +# bash# find /etc -type f -name udev.conf +# /etc/udev/udev.conf +# bash# grep udev_rules= /etc/udev/udev.conf +# udev_rules="/etc/udev/rules.d/" +# bash# ls /etc/udev/rules.d/ +# 10-wacom.rules 50-udev.rules +# bash# cp /path/to/linux/Documentation/admin-guide/aoe/udev.txt \ +# /etc/udev/rules.d/60-aoe.rules +# + +# aoe char devices +SUBSYSTEM=="aoe", KERNEL=="discover", NAME="etherd/%k", GROUP="disk", MODE="0220" +SUBSYSTEM=="aoe", KERNEL=="err", NAME="etherd/%k", GROUP="disk", MODE="0440" +SUBSYSTEM=="aoe", KERNEL=="interfaces", NAME="etherd/%k", GROUP="disk", MODE="0220" +SUBSYSTEM=="aoe", KERNEL=="revalidate", NAME="etherd/%k", GROUP="disk", MODE="0220" +SUBSYSTEM=="aoe", KERNEL=="flush", NAME="etherd/%k", GROUP="disk", MODE="0220" + +# aoe block devices +KERNEL=="etherd*", GROUP="disk" |