glibc/io/Versions, branch master

io: Add closefrom [BZ #10353]

2021-07-08T17:08:14Z

The function closes all open file descriptors greater than or equal to input argument. Negative values are clamped to 0, i.e, it will close all file descriptors. As indicated by the bug report, this is a common symbol provided by different systems (Solaris, OpenBSD, NetBSD, FreeBSD) and, although its has inherent issues with not taking in consideration internal libc file descriptors (such as syslog), this is also a common feature used in multiple projects [1][2][3][4][5]. The Linux fallback implementation iterates over /proc and close all file descriptors sequentially. Although it was raised the questioning whether getdents on /proc/self/fd might return disjointed entries when file descriptor are closed; it does not seems the case on my testing on multiple kernel (v4.18, v5.4, v5.9) and the same strategy is used on different projects [1][2][3][5]. Also, the interface is set a fail-safe meaning that a failure in the fallback results in a process abort. Checked on x86_64-linux-gnu and i686-linux-gnu on kernel 5.11 and 4.15. [1] https://github.com/systemd/systemd/blob/5238e9575906297608ff802a27e2ff9effa3b338/src/basic/fd-util.c#L217 [2] https://github.com/lxc/lxc/blob/ddf4b77e11a4d08f09b7b9cd13e593f8c047edc5/src/lxc/start.c#L236 [3] https://github.com/python/cpython/blob/9e4f2f3a6b8ee995c365e86d976937c141d867f8/Modules/_posixsubprocess.c#L220 [4] https://github.com/rust-lang/rust/blob/5f47c0613ed4eb46fca3633c1297364c09e5e451/src/libstd/sys/unix/process2.rs#L303-L308 [5] https://github.com/openjdk/jdk/blob/master/src/java.base/unix/native/libjava/childproc.c#L82

Add missing symbols to Version files

2021-06-02T05:32:19Z

Some symbols have explicit versioned_symbol or compat_symbol markers in the sources, but no corresponding entry in the Versions files. This presently works because the local: * directive is only applied to the base version. Reviewed-by: Adhemerval Zanella

Remove mknod wrapper functions, move them to symbols

2020-10-09T20:02:06Z

This patch removes the mknod and mknodat static wrapper and add the symbols on the libc with the expected names. Both the prototypes of the internal symbol linked by the static wrappers and the inline redirectors are also removed from the installed sys/stat.h header file. The wrapper implementation license LGPL exception is also removed since it is no longer statically linked to binaries. Internally the _STAT_VER* definitions are moved to the arch-specific xstatver.h file. Checked with a build for all affected ABIs. I also checked on x86_64, i686, powerpc, powerpc64le, sparcv9, sparc64, s390, and s390x. Reviewed-by: Lukasz Majewski

Remove stat wrapper functions, move them to exported symbols

2020-10-09T20:02:06Z

This patch removes the stat, stat64, lstat, lstat64, fstat, fstat64, fstatat, and fstatat64 static wrapper and add the symbol on the libc with the expected names. Both the prototypes of the internal symbol linked by the static wrappers and the inline redirectors are also removed from the installed sys/stat.h header file. The wrapper implementation license LGPL exception is also removed since it is no longer statically linked to binaries. Internally the _STAT_VER* definitions are moved to a arch-specific xstatver.h file. The internal defines that redirects internals {f}stat{at} to their {f}xstat{at} counterparts are removed for Linux (!NO_RTLD_HIDDEN). Hurd still requires them since {f}stat{at} pulls extra objects that makes the loader build fail otherwise (I haven't dig into why exactly). Checked with a build for all affected ABIs. I also checked on x86_64, i686, powerpc, powerpc64le, sparcv9, sparc64, s390, and s390x. Reviewed-by: Lukasz Majewski

Move implementation of into a C file

2020-02-18T12:44:48Z

file_change_detection_for_stat partially initialize struct file_change_detection in some cases, when the size member alone determines the outcome of all comparisons. This results in maybe-uninitialized compiler warnings in case of sufficiently aggressive inlining. Once the implementation is moved into a separate C file, this kind of inlining is no longer possible, so the compiler warnings are gone.

Add the statx function

2018-07-10T14:14:45Z

Fix Linux fcntl OFD locks for non-LFS architectures (BZ#20251)

2018-06-26T16:22:53Z

This patch fixes the OFD ("file private") locks for architectures that support non-LFS flock definition (__USE_FILE_OFFSET64 not defined). The issue in this case is both F_OFD_{GETLK,SETLK,SETLKW} and F_{SET,GET}L{W}K64 expects a flock64 argument and when using old F_OFD_* flags with a non LFS flock argument the kernel might interpret the underlying data wrongly. Kernel idea originally was to avoid using such flags in non-LFS syscall, but since GLIBC uses fcntl with LFS semantic as default it is possible to provide the functionality and avoid the bogus struct kernel passing by adjusting the struct manually for the required flags. The idea follows other LFS interfaces that provide two symbols: 1. A new LFS fcntl64 is added on default ABI with the usual macros to select it for FILE_OFFSET_BITS=64. 2. The Linux non-LFS fcntl use a stack allocated struct flock64 for F_OFD_{GETLK,SETLK,SETLKW} copy the results on the user provided struct. 3. Keep a compat symbol with old broken semantic for architectures that do not define __OFF_T_MATCHES_OFF64_T. So for architectures which defines __USE_FILE_OFFSET64, fcntl64 will aliased to fcntl and no adjustment would be required. So to actually use F_OFD_* with LFS support the source must be built with LFS support (_FILE_OFFSET_BITS=64). Also F_OFD_SETLKW command is handled a cancellation point, as for F_SETLKW{64}. Checked on x86_64-linux-gnu and i686-linux-gnu. [BZ #20251] * NEWS: Mention fcntl64 addition. * csu/check_fds.c: Replace __fcntl_nocancel by __fcntl64_nocancel. * login/utmp_file.c: Likewise. * sysdeps/posix/fdopendir.c: Likewise. * sysdeps/posix/opendir.c: Likewise. * sysdeps/unix/pt-fcntl.c: Likewise. * include/fcntl.h (__libc_fcntl64, __fcntl64, __fcntl64_nocancel_adjusted): New prototype. (__fcntl_nocancel_adjusted): Remove prototype. * io/Makefile (routines): Add fcntl64. (CFLAGS-fcntl64.c): New rule. * io/Versions [GLIBC_2.28] (fcntl64): New symbol. [GLIBC_PRIVATE] (__libc_fcntl): Rename to __libc_fcntl64. * io/fcntl.h (fcntl64): Add prototype and redirect if __USE_FILE_OFFSET64 is defined. * io/fcntl64.c: New file. * manual/llio.text: Add a note for which commands fcntl acts a cancellation point. * nptl/Makefile (CFLAGS-fcntl64.c): New rule. * sysdeps/mach/hurd/fcntl.c: Alias fcntl to fcntl64 symbols. * sysdeps/mach/hurd/i386/libc.abilist [GLIBC_2.28] (fcntl, fcntl64): New symbols. * sysdeps/unix/sysv/linux/fcntl.c (__libc_fcntl): Fix F_GETLK64, F_OFD_GETLK, F_SETLK64, F_SETLKW64, F_OFD_SETLK, and F_OFD_SETLKW for non-LFS case. * sysdeps/unix/sysv/linux/fcntl64.c: New file. * sysdeps/unix/sysv/linux/fcntl_nocancel.c (__fcntl_nocancel): Rename to __fcntl64_nocancel. (__fcntl_nocancel_adjusted): Rename to __fcntl64_nocancel_adjusted. * sysdeps/unix/sysv/linux/not-cancel.h (__fcntl_nocancel): Rename to __fcntl64_nocancel. * sysdeps/unix/sysv/linux/tst-ofdlocks.c: New file. * sysdeps/unix/sysv/linux/tst-ofdlocks-compat.c: Likewise. * sysdeps/unix/sysv/linux/Makefile (tests): Add tst-ofdlocks. (tests-internal): Add tst-ofdlocks-compat. * sysdeps/unix/sysv/linux/aarch64/libc.abilist [GLIBC_2.28] (fcntl64): New symbol. * sysdeps/unix/sysv/linux/alpha/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/ia64/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/mips/mips64/n64/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/powerpc/powerpc64/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/powerpc/powerpc64/libc-le.abilist: Likewise. * sysdeps/unix/sysv/linux/riscv/rv64/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/s390/s390-64/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/sparc/sparc64/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/x86_64/64/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/x86_64/x32/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/arm/libc.abilist [GLIBC_2.28] (fcntl, fcntl64): Likewise. * sysdeps/unix/sysv/linux/hppa/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/i386/libc.abilis: Likewise. * sysdeps/unix/sysv/linux/m68k/coldfire/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/m68k/m680x0/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/microblaze/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/mips/mips32/fpu/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/mips/mips32/nofpu/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/mips/mips64/n32/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/nios2/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/powerpc/powerpc32/fpu/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/powerpc/powerpc32/nofpu/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/s390/s390-32/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/sh/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/sparc/sparc32/libc.abilist: Likewise.

Avoid cancellable I/O primitives in ld.so.

2018-06-12T13:53:04Z

Neither the entry points, nor lazy symbol resolution, nor initial shared library load-up, are cancellation points, so ld.so should exclusively use I/O primitives that are not cancellable. We currently achieve this by having the cancellation hooks compile as no-ops when IS_IN(rtld); this patch changes to using exclusively _nocancel primitives in the source code instead, which makes the intent clearer and significantly reduces the amount of code compiled under IS_IN(rtld) as well as IS_IN(libc) -- in particular, elf/Makefile no longer thinks we require a copy of unwind.c in rtld-libc.a. (The older mechanism is preserved as a backstop.) The bulk of the change is splitting up the files that define the _nocancel I/O functions, so they don't also define the variants that *are* cancellation points; after which, the existing logic for picking out the bits of libc that need to be recompiled as part of ld.so Just Works. I did this for all of the _nocancel functions, not just the ones used by ld.so, for consistency. fcntl was a little tricky because it's only a cancellation point for certain opcodes (F_SETLKW(64), which can block), and the existing __fcntl_nocancel wasn't applying the FCNTL_ADJUST_CMD hook, which strikes me as asking for trouble, especially as the only nontrivial definition of FCNTL_ADJUST_CMD (for powerpc64) changes F_*LK* opcodes. To fix this, fcntl_common moves to fcntl_nocancel.c along with __fcntl_nocancel, and changes its name to the extern (but hidden) symbol __fcntl_nocancel_adjusted, so that regular fcntl can continue calling it. __fcntl_nocancel now applies FCNTL_ADJUST_CMD; so that both both fcntl.c and fcntl_nocancel.c can see it, the only nontrivial definition moves from sysdeps/u/s/l/powerpc/powerpc64/fcntl.c to .../powerpc64/sysdep.h and becomes entirely a macro, instead of a macro that calls an inline function. The nptl version of libpthread also changes a little, because its "compat-routines" formerly included files that defined all the _nocancel functions it uses; instead of continuing to duplicate them, I exported the relevant ones from libc.so as GLIBC_PRIVATE. Since the Linux fcntl.c calls a function defined by fcntl_nocancel.c, it can no longer be used from libpthread.so; instead, introduce a custom forwarder, pt-fcntl.c, and export __libc_fcntl from libc.so as GLIBC_PRIVATE. The nios2-linux ABI doesn't include a copy of vfork() in libpthread, and it was handling that by manipulating libpthread-routines in .../linux/nios2/Makefile; it is cleaner to do what other such ports do, and have a pt-vfork.S that defines no symbols. Right now, it appears that Hurd does not implement _nocancel I/O, so sysdeps/generic/not-cancel.h will forward everything back to the regular functions. This changed the names of some of the functions that sysdeps/mach/hurd/dl-sysdep.c needs to interpose. * elf/dl-load.c, elf/dl-misc.c, elf/dl-profile.c, elf/rtld.c * sysdeps/unix/sysv/linux/dl-sysdep.c Include not-cancel.h. Use __close_nocancel instead of __close, __open64_nocancel instead of __open, __read_nocancel instead of __libc_read, and __write_nocancel instead of __libc_write. * csu/check_fds.c (check_one_fd) * sysdeps/posix/fdopendir.c (__fdopendir) * sysdeps/posix/opendir.c (__alloc_dir): Use __fcntl_nocancel instead of __fcntl and/or __libc_fcntl. * sysdeps/unix/sysv/linux/pthread_setname.c (pthread_setname_np) * sysdeps/unix/sysv/linux/pthread_getname.c (pthread_getname_np) * sysdeps/unix/sysv/linux/i386/smp.h (is_smp_system): Use __open64_nocancel instead of __open_nocancel. * sysdeps/unix/sysv/linux/not-cancel.h: Move all of the hidden_proto declarations to the end and issue them if either IS_IN(libc) or IS_IN(rtld). * sysdeps/unix/sysv/linux/Makefile [subdir=io] (sysdep_routines): Add close_nocancel, fcntl_nocancel, nanosleep_nocancel, open_nocancel, open64_nocancel, openat_nocancel, pause_nocancel, read_nocancel, waitpid_nocancel, write_nocancel. * io/Versions [GLIBC_PRIVATE]: Add __libc_fcntl, __fcntl_nocancel, __open64_nocancel, __write_nocancel. * posix/Versions: Add __nanosleep_nocancel, __pause_nocancel. * nptl/pt-fcntl.c: New file. * nptl/Makefile (pthread-compat-wrappers): Remove fcntl. (libpthread-routines): Add pt-fcntl. * include/fcntl.h (__fcntl_nocancel_adjusted): New function. (__libc_fcntl): Remove attribute_hidden. * sysdeps/unix/sysv/linux/fcntl.c (__libc_fcntl): Call __fcntl_nocancel_adjusted, not fcntl_common. (__fcntl_nocancel): Move to new file fcntl_nocancel.c. (fcntl_common): Rename to __fcntl_nocancel_adjusted; also move to fcntl_nocancel.c. * sysdeps/unix/sysv/linux/fcntl_nocancel.c: New file. * sysdeps/unix/sysv/linux/powerpc/powerpc64/fcntl.c: Remove file. * sysdeps/unix/sysv/linux/powerpc/powerpc64/sysdep.h: Define FCNTL_ADJUST_CMD here, as a self-contained macro. * sysdeps/unix/sysv/linux/close.c: Move __close_nocancel to... * sysdeps/unix/sysv/linux/close_nocancel.c: ...this new file. * sysdeps/unix/sysv/linux/nanosleep.c: Move __nanosleep_nocancel to... * sysdeps/unix/sysv/linux/nanosleep_nocancel.c: ...this new file. * sysdeps/unix/sysv/linux/open.c: Move __open_nocancel to... * sysdeps/unix/sysv/linux/open_nocancel.c: ...this new file. * sysdeps/unix/sysv/linux/open64.c: Move __open64_nocancel to... * sysdeps/unix/sysv/linux/open64_nocancel.c: ...this new file. * sysdeps/unix/sysv/linux/openat.c: Move __openat_nocancel to... * sysdeps/unix/sysv/linux/openat_nocancel.c: ...this new file. * sysdeps/unix/sysv/linux/openat64.c: Move __openat64_nocancel to... * sysdeps/unix/sysv/linux/openat64_nocancel.c: ...this new file. * sysdeps/unix/sysv/linux/pause.c: Move __pause_nocancel to... * sysdeps/unix/sysv/linux/pause_nocancel.c: ...this new file. * sysdeps/unix/sysv/linux/read.c: Move __read_nocancel to... * sysdeps/unix/sysv/linux/read_nocancel.c: ...this new file. * sysdeps/unix/sysv/linux/waitpid.c: Move __waitpid_nocancel to... * sysdeps/unix/sysv/linux/waitpid_nocancel.c: ...this new file. * sysdeps/unix/sysv/linux/write.c: Move __write_nocancel to... * sysdeps/unix/sysv/linux/write_nocancel.c: ...this new file. * sysdeps/unix/sysv/linux/nios2/Makefile: Don't override libpthread-routines. * sysdeps/unix/sysv/linux/nios2/pt-vfork.S: New file which defines nothing. * sysdeps/mach/hurd/dl-sysdep.c: Define __read instead of __libc_read, and __write instead of __libc_write. Define __open64 in addition to __open.

copy_file_range: New function to copy file data

2017-12-22T09:55:40Z

The semantics are based on the Linux system call, but a very close emulation in user space is provided.

Add LFS support for fts functions (bug 11460)

2015-11-06T10:24:16Z

fts didn't have large-file support yet and fts.h had an #error preventing usage when _FILE_OFFSET_BITS was set. This required nasty workarounds for programs using fts with LFS. This patch implements LFS support for fts by adding FTS64 and FTENT64 variants plus fts64 functions. Which are simple aliases for 64bit off_t arches. Also includes a simple testcase for some of the fts functions with or without LFS enabled. [BZ #11460] * io/Makefile (routines): Add fts64. (tests): Add tst-fts and tst-fts-lfs. (CFLAGS-fts64.c): New. * io/Versions (GLIBC_2.23): New. * io/fts.c: Replace FTS with FTSOBJ, FTSENT with FTSENTRY. Use function defines FTS_OPEN, FTS_CLOSE, FTS_READ, FTS_SET and FTS_CHILDREN. Define FTSOBJ, FTSENTRY, FTS_OPEN, FTS_CLOSE, FTS_READ, FTS_SET, FTS_CHILDREN, INO_T, STAT and LSTAT if necessary. * io/fts.h (FTS64): New if _USE_LARGEFILE64. (FTSENT64): Likewise. (fts64_children): Likewise. (fts64_close): Likewise. (fts64_open): Likewise. (fts64_read): Likewise. (fts64_set): Likewise. * io/fts64.c: New file. * io/tst-fts.c: New test. * io/tst-fts-lfs.c: Likewise. * sysdeps/unix/sysv/linux/aarch64/libc.abilist (GLIBC_2.23): Add GLIBC_2.23, fts64_children, fts64_close, fts64_open, fts64_read and fts64_set. * sysdeps/unix/sysv/linux/alpha/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/arm/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/hppa/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/i386/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/ia64/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/m68k/coldfire/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/m68k/m680x0/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/microblaze/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/mips/mips32/fpu/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/mips/mips32/nofpu/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/mips/mips64/n32/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/mips/mips64/n64/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/nios2/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/powerpc/powerpc32/fpu/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/powerpc/powerpc32/nofpu/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/powerpc/powerpc64/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/powerpc/powerpc64/libc-le.abilist: Likewise. * sysdeps/unix/sysv/linux/s390/s390-32/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/s390/s390-64/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/sh/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/sparc/sparc32/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/sparc/sparc64/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/tile/tilegx/tilegx32/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/tile/tilegx/tilegx64/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/tile/tilepro/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/x86_64/64/libc.abilist: Likewise. * sysdeps/unix/sysv/linux/x86_64/x32/libc.abilist: Likewise. * sysdeps/wordsize-64/fts.c: New file. * sysdeps/wordsize-64/fts64.c: Likewise. * sysdeps/unix/sysv/linux/mips/mips64/n64/fts.c: Likewise. * sysdeps/unix/sysv/linux/mips/mips64/n64/fts64.c: Likewise. * sysdeps/unix/sysv/linux/x86_64/x32/fts.c: likewise. * sysdeps/unix/sysv/linux/x86_64/x32/fts64.c: likewise.