Skip to content
Snippets Groups Projects
Commit c13baa4b authored by Emanuele Aina's avatar Emanuele Aina
Browse files

Backport fixes for tests on kernels >= 5.1

In Debian, glibc 2.8 is shipped in Buster with the 4.19 kernel, while
Bullseye ships newer versions of both.

In Apertis we also ship the Buster base with newer kernels, 5.4 in v2020
and 5.8 in v2021. This means that we can hit issues that do not manifest
in Debian since those are combinations that do not get tested there.

This is one of those cases, where some changes in the kernel headers
broke the glibc testsuite. Upstream ships the fixes in the 2.9 branch,
so here we backport a few patches to make the tests happy again.

In particular, with the 5.4 and 5.7 kernels we hit these failures:

    FAIL: conform/POSIX2008/arpa/inet.h/conform
    FAIL: conform/POSIX2008/netdb.h/conform
    FAIL: conform/POSIX2008/netinet/in.h/conform
    FAIL: conform/POSIX2008/sys/socket.h/conform
    FAIL: conform/UNIX98/arpa/inet.h/conform
    FAIL: conform/UNIX98/netdb.h/conform
    FAIL: conform/UNIX98/netinet/in.h/conform
    FAIL: conform/UNIX98/sys/socket.h/conform
    FAIL: conform/XOPEN2K/arpa/inet.h/conform
    FAIL: conform/XOPEN2K/netdb.h/conform
    FAIL: conform/XOPEN2K/netinet/in.h/conform
    FAIL: conform/XOPEN2K/sys/socket.h/conform
    FAIL: conform/XOPEN2K8/arpa/inet.h/conform
    FAIL: conform/XOPEN2K8/netdb.h/conform
    FAIL: conform/XOPEN2K8/netinet/in.h/conform
    FAIL: conform/XOPEN2K8/sys/socket.h/conform
    FAIL: conform/XPG42/arpa/inet.h/conform
    FAIL: conform/XPG42/netdb.h/conform
    FAIL: conform/XPG42/netinet/in.h/conform
    FAIL: conform/XPG42/sys/socket.h/conform
    FAIL: io/tst-copy_file_range
    FAIL: misc/check-installed-headers-c

The backported patch address the issues below:

* https://sourceware.org/bugzilla/show_bug.cgi?id=24532
* https://bugzilla.redhat.com/show_bug.cgi?id=1769304



Signed-off-by: Emanuele Aina's avatarEmanuele Aina <emanuele.aina@collabora.com>
parent 745dc132
No related branches found
No related tags found
5 merge requests!11Merge changes from apertis/v2020-updates into apertis/v2020,!7Merge changes from apertis/v2019-updates into apertis/v2019,!6Sync apertis updates,!5v2020 backport: Backport fixes for tests on kernels >= 5.1,!4Backport fixes for tests on kernels >= 5.1
From df648905e7d8340bb3e78813fd25e2077b9685d9 Mon Sep 17 00:00:00 2001
From: Joseph Myers <joseph@codesourcery.com>
Date: Mon, 17 Dec 2018 18:29:36 +0000
Subject: [PATCH] Add test that MAP_* constants agree with kernel.
Continuing the process of building up and using Python infrastructure
for extracting and using values in headers, this patch adds a test
that MAP_* constants from sys/mman.h agree with those in the Linux
kernel headers. (Other sys/mman.h constants could be added to the
test separately.)
This set of constants has grown over time, so the generic code is
enhanced to allow saying extra constants are OK on either side of the
comparison (where the caller sets those parameters based on the Linux
kernel headers version, compared with the version the headers were
last updated from). Although the test is a custom Python file, my
intention is to move in future to a single Python script for such
tests and text files it takes as inputs, once there are enough
examples to provide a guide to the common cases in such tests (I'd
like to end up with most or all such sets of constants copied from
kernel headers having such tests, and likewise for structure layouts
from the kernel).
The Makefile code is essentially the same as for tst-signal-numbers,
but I didn't try to find an object file to depend on to represent the
dependency on the headers used by the test (the conform/ tests don't
try to represent such header dependencies at all, for example).
Tested with build-many-glibcs.py, and also for x86_64 with older
kernel headers.
* scripts/glibcextract.py (compare_macro_consts): Take parameters
to allow extra macros from first or second sources.
* sysdeps/unix/sysv/linux/tst-mman-consts.py: New file.
* sysdeps/unix/sysv/linux/Makefile [$(subdir) = misc]
(tests-special): Add $(objpfx)tst-mman-consts.out.
($(objpfx)tst-mman-consts.out): New makefile target.
---
ChangeLog | 9 +++
scripts/glibcextract.py | 21 +++++--
sysdeps/unix/sysv/linux/Makefile | 9 +++
sysdeps/unix/sysv/linux/tst-mman-consts.py | 65 ++++++++++++++++++++++
4 files changed, 100 insertions(+), 4 deletions(-)
create mode 100644 sysdeps/unix/sysv/linux/tst-mman-consts.py
--- a/scripts/glibcextract.py
+++ b/scripts/glibcextract.py
@@ -136,12 +136,19 @@ def compute_macro_consts(source_text, cc
return compute_c_consts(sym_data, cc)
-def compare_macro_consts(source_1, source_2, cc, macro_re, exclude_re=None):
+def compare_macro_consts(source_1, source_2, cc, macro_re, exclude_re=None,
+ allow_extra_1=False, allow_extra_2=False):
"""Compare the values of macros defined by two different sources.
The sources would typically be includes of a glibc header and a
- kernel header. Return 1 if there were any differences, 0 if the
- macro values were the same.
+ kernel header. If allow_extra_1, the first source may define
+ extra macros (typically if the kernel headers are older than the
+ version glibc has taken definitions from); if allow_extra_2, the
+ second source may define extra macros (typically if the kernel
+ headers are newer than the version glibc has taken definitions
+ from). Return 1 if there were any differences other than those
+ allowed, 0 if the macro values were the same apart from any
+ allowed differences.
"""
macros_1 = compute_macro_consts(source_1, cc, macro_re, exclude_re)
@@ -150,13 +157,19 @@ def compare_macro_consts(source_1, sourc
return 0
print('First source:\n%s\n' % source_1)
print('Second source:\n%s\n' % source_2)
+ ret = 0
for name, value in sorted(macros_1.items()):
if name not in macros_2:
print('Only in first source: %s' % name)
+ if not allow_extra_1:
+ ret = 1
elif macros_1[name] != macros_2[name]:
print('Different values for %s: %s != %s'
% (name, macros_1[name], macros_2[name]))
+ ret = 1
for name in sorted(macros_2.keys()):
if name not in macros_1:
print('Only in second source: %s' % name)
- return 1
+ if not allow_extra_2:
+ ret = 1
+ return ret
--- a/sysdeps/unix/sysv/linux/Makefile
+++ b/sysdeps/unix/sysv/linux/Makefile
@@ -97,6 +97,15 @@ $(objpfx)tst-sysconf-iov_max: $(objpfx)t
$(objpfx)tst-pkey: $(shared-thread-library)
+tests-special += $(objpfx)tst-mman-consts.out
+$(objpfx)tst-mman-consts.out: ../sysdeps/unix/sysv/linux/tst-mman-consts.py
+ PYTHONPATH=../scripts \
+ $(PYTHON) ../sysdeps/unix/sysv/linux/tst-mman-consts.py \
+ --cc="$(CC) $(patsubst -DMODULE_NAME=%, \
+ -DMODULE_NAME=testsuite, \
+ $(CPPFLAGS))" \
+ < /dev/null > $@ 2>&1; $(evaluate-test)
+
endif # $(subdir) == misc
ifeq ($(subdir),time)
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/tst-mman-consts.py
@@ -0,0 +1,65 @@
+#!/usr/bin/python3
+# Test that glibc's sys/mman.h constants match the kernel's.
+# Copyright (C) 2018 Free Software Foundation, Inc.
+# This file is part of the GNU C Library.
+#
+# The GNU C Library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# The GNU C Library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with the GNU C Library; if not, see
+# <http://www.gnu.org/licenses/>.
+
+import argparse
+import sys
+
+import glibcextract
+
+
+def linux_kernel_version(cc):
+ """Return the (major, minor) version of the Linux kernel headers."""
+ sym_data = ['#include <linux/version.h>', 'START',
+ ('LINUX_VERSION_CODE', 'LINUX_VERSION_CODE')]
+ val = glibcextract.compute_c_consts(sym_data, cc)['LINUX_VERSION_CODE']
+ val = int(val)
+ return ((val & 0xff0000) >> 16, (val & 0xff00) >> 8)
+
+
+def main():
+ """The main entry point."""
+ parser = argparse.ArgumentParser(
+ description="Test that glibc's sys/mman.h constants "
+ "match the kernel's.")
+ parser.add_argument('--cc', metavar='CC',
+ help='C compiler (including options) to use')
+ args = parser.parse_args()
+ linux_version_headers = linux_kernel_version(args.cc)
+ linux_version_glibc = (4, 19)
+ sys.exit(glibcextract.compare_macro_consts(
+ '#define _GNU_SOURCE 1\n'
+ '#include <sys/mman.h>\n',
+ '#define _GNU_SOURCE 1\n'
+ '#include <linux/mman.h>\n',
+ args.cc,
+ 'MAP_.*',
+ # A series of MAP_HUGE_<size> macros are defined by the kernel
+ # but not by glibc. MAP_UNINITIALIZED is kernel-only.
+ # MAP_FAILED is not a MAP_* flag and is glibc-only, as is the
+ # MAP_ANON alias for MAP_ANONYMOUS. MAP_RENAME, MAP_AUTOGROW,
+ # MAP_LOCAL and MAP_AUTORSRV are in the kernel header for
+ # MIPS, marked as "not used by linux"; SPARC has MAP_INHERIT
+ # in the kernel header, but does not use it.
+ 'MAP_HUGE_[0-9].*|MAP_UNINITIALIZED|MAP_FAILED|MAP_ANON'
+ '|MAP_RENAME|MAP_AUTOGROW|MAP_LOCAL|MAP_AUTORSRV|MAP_INHERIT',
+ linux_version_glibc > linux_version_headers,
+ linux_version_headers > linux_version_glibc))
+
+if __name__ == '__main__':
+ main()
From 7b36d26b22d147ffc347f427f9fd584700578a94 Mon Sep 17 00:00:00 2001
From: Samuel Thibault <samuel.thibault@ens-lyon.org>
Date: Mon, 3 Dec 2018 14:40:48 +0100
Subject: [PATCH] Fix test-as-const-jmp_buf-ssp.c generation on gnu-i386
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
hurd's jmp_buf-ssp.sym does not define any symbol.
scripts/gen-as-const.py currently was emitting an empty line in that
case, and the gawk invocation was prepending "asconst_" to it, ending up
with:
.../build/glibc/setjmp/test-as-const-jmp_buf-ssp.c:1:2: error: expected « = », « , », « ; », « asm » or
« __attribute__ » at end of input
1 | asconst_
| ^~~~~~~~
* scripts/gen-as-const.py (main): Avoid emitting empty line when
there is no element in `consts'.
---
ChangeLog | 5 +++++
scripts/gen-as-const.py | 2 +-
2 files changed, 6 insertions(+), 1 deletion(-)
--- a/scripts/gen-as-const.py
+++ b/scripts/gen-as-const.py
@@ -153,7 +153,7 @@ def main():
print(gen_test(sym_data))
else:
consts = compute_c_consts(sym_data, args.cc)
- print('\n'.join('#define %s %s' % c for c in sorted(consts.items())))
+ print(''.join('#define %s %s\n' % c for c in sorted(consts.items())), end='')
if __name__ == '__main__':
main()
From 845278f2c6f93b1bb72c1e61529785740ea05f62 Mon Sep 17 00:00:00 2001
From: Florian Weimer <fweimer@redhat.com>
Date: Wed, 24 Jul 2019 10:59:34 +0200
Subject: [PATCH] Linux: Use in-tree copy of SO_ constants for !__USE_MISC [BZ
#24532]
The kernel changes for a 64-bit time_t on 32-bit architectures
resulted in <asm/socket.h> indirectly including <linux/posix_types.h>.
The latter is not namespace-clean for the POSIX version of
<sys/socket.h>.
This issue has persisted across several Linux releases, so this commit
creates our own copy of the SO_* definitions for !__USE_MISC mode.
The new test socket/tst-socket-consts ensures that the copy is
consistent with the kernel definitions (which vary across
architectures). The test is tricky to get right because CPPFLAGS
includes include/libc-symbols.h, which in turn defines _GNU_SOURCE
unconditionally.
Tested with build-many-glibcs.py. I verified that a discrepancy in
the definitions actually results in a failure of the
socket/tst-socket-consts test.
(cherry picked from commit 7854ebf8ed18180189c335f6f499fe9322458f0b)
---
ChangeLog | 22 +++++
sysdeps/unix/sysv/linux/Makefile | 14 ++-
.../sysv/linux/alpha/bits/socket-constants.h | 38 +++++++
.../unix/sysv/linux/bits/socket-constants.h | 38 +++++++
sysdeps/unix/sysv/linux/bits/socket.h | 98 ++-----------------
.../sysv/linux/hppa/bits/socket-constants.h | 38 +++++++
.../sysv/linux/mips/bits/socket-constants.h | 38 +++++++
.../linux/powerpc/bits/socket-constants.h | 38 +++++++
.../sysv/linux/sparc/bits/socket-constants.h | 38 +++++++
sysdeps/unix/sysv/linux/tst-socket-consts.py | 65 ++++++++++++
10 files changed, 333 insertions(+), 94 deletions(-)
create mode 100644 sysdeps/unix/sysv/linux/alpha/bits/socket-constants.h
create mode 100644 sysdeps/unix/sysv/linux/bits/socket-constants.h
create mode 100644 sysdeps/unix/sysv/linux/hppa/bits/socket-constants.h
create mode 100644 sysdeps/unix/sysv/linux/mips/bits/socket-constants.h
create mode 100644 sysdeps/unix/sysv/linux/powerpc/bits/socket-constants.h
create mode 100644 sysdeps/unix/sysv/linux/sparc/bits/socket-constants.h
create mode 100644 sysdeps/unix/sysv/linux/tst-socket-consts.py
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,25 @@
+2019-07-24 Florian Weimer <fweimer@redhat.com>
+
+ [BZ #24532]
+ Linux: Use in-tree copy of SO_ constants for !__USE_MISC.
+ * sysdeps/unix/sysv/linux/Makefile [$(subdir) == socket]
+ (sysdep_headers): Add bits/socket-constants.h.
+ (tests-special): Add tst-socket-consts.out.
+ (tst-socket-consts.out): New target.
+ * sysdeps/unix/sysv/linux/bits/socket.h: Remove macro tracking
+ around <asm/unistd.h>.
+ [__USE_MISC]: Include <bits/types/time_t.h> before <asm/socket.h>.
+ [!__USE_MISC]: Include <sys/socket-constants.h> instead of
+ <asm/socket.h>.
+ * sysdeps/unix/sysv/linux/bits/tst-socket-consts.py: New file.
+ * sysdeps/unix/sysv/linux/bits/socket-constants.h: Likewise.
+ * sysdeps/unix/sysv/linux/alpha/bits/socket-constants.h: Likewise.
+ * sysdeps/unix/sysv/linux/hppa/bits/socket-constants.h: Likewise.
+ * sysdeps/unix/sysv/linux/mips/bits/socket-constants.h: Likewise.
+ * sysdeps/unix/sysv/linux/powerpc/bits/socket-constants.h:
+ Likewise.
+ * sysdeps/unix/sysv/linux/sparc/bits/socket-constants.h: Likewise.
+
2019-04-23 Adhemerval Zanella <adhemerval.zanella@linaro.org>
[BZ #18035]
--- a/sysdeps/unix/sysv/linux/Makefile
+++ b/sysdeps/unix/sysv/linux/Makefile
@@ -134,11 +134,21 @@ endif
ifeq ($(subdir),socket)
sysdep_headers += net/if_ppp.h net/ppp-comp.h \
net/ppp_defs.h net/if_arp.h net/route.h net/ethernet.h \
- net/if_slip.h net/if_packet.h net/if_shaper.h
+ net/if_slip.h net/if_packet.h net/if_shaper.h \
+ bits/socket-constants.h
sysdep_routines += cmsg_nxthdr
CFLAGS-recvmmsg.c = -fexceptions -fasynchronous-unwind-tables
CFLAGS-sendmmsg.c = -fexceptions -fasynchronous-unwind-tables
-endif
+
+tests-special += $(objpfx)tst-socket-consts.out
+$(objpfx)tst-socket-consts.out: ../sysdeps/unix/sysv/linux/tst-socket-consts.py
+ PYTHONPATH=../scripts \
+ $(PYTHON) ../sysdeps/unix/sysv/linux/tst-socket-consts.py \
+ --cc="$(CC) $(patsubst -DMODULE_NAME=%, \
+ -DMODULE_NAME=testsuite, \
+ $(CPPFLAGS)) -D_ISOMAC" \
+ < /dev/null > $@ 2>&1; $(evaluate-test)
+endif # $(subdir) == socket
ifeq ($(subdir),sunrpc)
sysdep_headers += nfs/nfs.h
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/alpha/bits/socket-constants.h
@@ -0,0 +1,38 @@
+/* Socket constants which vary among Linux architectures. Version for alpha.
+ Copyright (C) 2019 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#ifndef _SYS_SOCKET_H
+# error "Never include <bits/socket-constants.h> directly; use <sys/socket.h> instead."
+#endif
+
+#define SOL_SOCKET 65535
+#define SO_ACCEPTCONN 4116
+#define SO_BROADCAST 32
+#define SO_DONTROUTE 16
+#define SO_ERROR 4103
+#define SO_KEEPALIVE 8
+#define SO_LINGER 128
+#define SO_OOBINLINE 256
+#define SO_RCVBUF 4098
+#define SO_RCVLOWAT 4112
+#define SO_RCVTIMEO 4114
+#define SO_REUSEADDR 4
+#define SO_SNDBUF 4097
+#define SO_SNDLOWAT 4113
+#define SO_SNDTIMEO 4115
+#define SO_TYPE 4104
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/bits/socket-constants.h
@@ -0,0 +1,38 @@
+/* Socket constants which vary among Linux architectures.
+ Copyright (C) 2019 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#ifndef _SYS_SOCKET_H
+# error "Never include <bits/socket-constants.h> directly; use <sys/socket.h> instead."
+#endif
+
+#define SOL_SOCKET 1
+#define SO_ACCEPTCONN 30
+#define SO_BROADCAST 6
+#define SO_DONTROUTE 5
+#define SO_ERROR 4
+#define SO_KEEPALIVE 9
+#define SO_LINGER 13
+#define SO_OOBINLINE 10
+#define SO_RCVBUF 8
+#define SO_RCVLOWAT 18
+#define SO_RCVTIMEO 20
+#define SO_REUSEADDR 2
+#define SO_SNDBUF 7
+#define SO_SNDLOWAT 19
+#define SO_SNDTIMEO 21
+#define SO_TYPE 3
--- a/sysdeps/unix/sysv/linux/bits/socket.h
+++ b/sysdeps/unix/sysv/linux/bits/socket.h
@@ -346,98 +346,12 @@ struct ucred
};
#endif
-/* Ugly workaround for unclean kernel headers. */
-#ifndef __USE_MISC
-# ifndef FIOGETOWN
-# define __SYS_SOCKET_H_undef_FIOGETOWN
-# endif
-# ifndef FIOSETOWN
-# define __SYS_SOCKET_H_undef_FIOSETOWN
-# endif
-# ifndef SIOCATMARK
-# define __SYS_SOCKET_H_undef_SIOCATMARK
-# endif
-# ifndef SIOCGPGRP
-# define __SYS_SOCKET_H_undef_SIOCGPGRP
-# endif
-# ifndef SIOCGSTAMP
-# define __SYS_SOCKET_H_undef_SIOCGSTAMP
-# endif
-# ifndef SIOCGSTAMPNS
-# define __SYS_SOCKET_H_undef_SIOCGSTAMPNS
-# endif
-# ifndef SIOCSPGRP
-# define __SYS_SOCKET_H_undef_SIOCSPGRP
-# endif
-#endif
-#ifndef IOCSIZE_MASK
-# define __SYS_SOCKET_H_undef_IOCSIZE_MASK
-#endif
-#ifndef IOCSIZE_SHIFT
-# define __SYS_SOCKET_H_undef_IOCSIZE_SHIFT
-#endif
-#ifndef IOC_IN
-# define __SYS_SOCKET_H_undef_IOC_IN
-#endif
-#ifndef IOC_INOUT
-# define __SYS_SOCKET_H_undef_IOC_INOUT
-#endif
-#ifndef IOC_OUT
-# define __SYS_SOCKET_H_undef_IOC_OUT
-#endif
-
-/* Get socket manipulation related informations from kernel headers. */
-#include <asm/socket.h>
-
-#ifndef __USE_MISC
-# ifdef __SYS_SOCKET_H_undef_FIOGETOWN
-# undef __SYS_SOCKET_H_undef_FIOGETOWN
-# undef FIOGETOWN
-# endif
-# ifdef __SYS_SOCKET_H_undef_FIOSETOWN
-# undef __SYS_SOCKET_H_undef_FIOSETOWN
-# undef FIOSETOWN
-# endif
-# ifdef __SYS_SOCKET_H_undef_SIOCATMARK
-# undef __SYS_SOCKET_H_undef_SIOCATMARK
-# undef SIOCATMARK
-# endif
-# ifdef __SYS_SOCKET_H_undef_SIOCGPGRP
-# undef __SYS_SOCKET_H_undef_SIOCGPGRP
-# undef SIOCGPGRP
-# endif
-# ifdef __SYS_SOCKET_H_undef_SIOCGSTAMP
-# undef __SYS_SOCKET_H_undef_SIOCGSTAMP
-# undef SIOCGSTAMP
-# endif
-# ifdef __SYS_SOCKET_H_undef_SIOCGSTAMPNS
-# undef __SYS_SOCKET_H_undef_SIOCGSTAMPNS
-# undef SIOCGSTAMPNS
-# endif
-# ifdef __SYS_SOCKET_H_undef_SIOCSPGRP
-# undef __SYS_SOCKET_H_undef_SIOCSPGRP
-# undef SIOCSPGRP
-# endif
-#endif
-#ifdef __SYS_SOCKET_H_undef_IOCSIZE_MASK
-# undef __SYS_SOCKET_H_undef_IOCSIZE_MASK
-# undef IOCSIZE_MASK
-#endif
-#ifdef __SYS_SOCKET_H_undef_IOCSIZE_SHIFT
-# undef __SYS_SOCKET_H_undef_IOCSIZE_SHIFT
-# undef IOCSIZE_SHIFT
-#endif
-#ifdef __SYS_SOCKET_H_undef_IOC_IN
-# undef __SYS_SOCKET_H_undef_IOC_IN
-# undef IOC_IN
-#endif
-#ifdef __SYS_SOCKET_H_undef_IOC_INOUT
-# undef __SYS_SOCKET_H_undef_IOC_INOUT
-# undef IOC_INOUT
-#endif
-#ifdef __SYS_SOCKET_H_undef_IOC_OUT
-# undef __SYS_SOCKET_H_undef_IOC_OUT
-# undef IOC_OUT
+#ifdef __USE_MISC
+# include <bits/types/time_t.h>
+# include <asm/socket.h>
+#else
+# define SO_DEBUG 1
+# include <bits/socket-constants.h>
#endif
/* Structure used to manipulate the SO_LINGER option. */
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/hppa/bits/socket-constants.h
@@ -0,0 +1,38 @@
+/* Socket constants which vary among Linux architectures. Version for hppa.
+ Copyright (C) 2019 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#ifndef _SYS_SOCKET_H
+# error "Never include <bits/socket-constants.h> directly; use <sys/socket.h> instead."
+#endif
+
+#define SOL_SOCKET 65535
+#define SO_ACCEPTCONN 16412
+#define SO_BROADCAST 32
+#define SO_DONTROUTE 16
+#define SO_ERROR 4103
+#define SO_KEEPALIVE 8
+#define SO_LINGER 128
+#define SO_OOBINLINE 256
+#define SO_RCVBUF 4098
+#define SO_RCVLOWAT 4100
+#define SO_RCVTIMEO 4102
+#define SO_REUSEADDR 4
+#define SO_SNDBUF 4097
+#define SO_SNDLOWAT 4099
+#define SO_SNDTIMEO 4101
+#define SO_TYPE 4104
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/mips/bits/socket-constants.h
@@ -0,0 +1,38 @@
+/* Socket constants which vary among Linux architectures. Version for MIPS.
+ Copyright (C) 2019 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#ifndef _SYS_SOCKET_H
+# error "Never include <bits/socket-constants.h> directly; use <sys/socket.h> instead."
+#endif
+
+#define SOL_SOCKET 65535
+#define SO_ACCEPTCONN 4105
+#define SO_BROADCAST 32
+#define SO_DONTROUTE 16
+#define SO_ERROR 4103
+#define SO_KEEPALIVE 8
+#define SO_LINGER 128
+#define SO_OOBINLINE 256
+#define SO_RCVBUF 4098
+#define SO_RCVLOWAT 4100
+#define SO_RCVTIMEO 4102
+#define SO_REUSEADDR 4
+#define SO_SNDBUF 4097
+#define SO_SNDLOWAT 4099
+#define SO_SNDTIMEO 4101
+#define SO_TYPE 4104
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/powerpc/bits/socket-constants.h
@@ -0,0 +1,38 @@
+/* Socket constants which vary among Linux architectures. Version for POWER.
+ Copyright (C) 2019 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#ifndef _SYS_SOCKET_H
+# error "Never include <bits/socket-constants.h> directly; use <sys/socket.h> instead."
+#endif
+
+#define SOL_SOCKET 1
+#define SO_ACCEPTCONN 30
+#define SO_BROADCAST 6
+#define SO_DONTROUTE 5
+#define SO_ERROR 4
+#define SO_KEEPALIVE 9
+#define SO_LINGER 13
+#define SO_OOBINLINE 10
+#define SO_RCVBUF 8
+#define SO_RCVLOWAT 16
+#define SO_RCVTIMEO 18
+#define SO_REUSEADDR 2
+#define SO_SNDBUF 7
+#define SO_SNDLOWAT 17
+#define SO_SNDTIMEO 19
+#define SO_TYPE 3
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/sparc/bits/socket-constants.h
@@ -0,0 +1,38 @@
+/* Socket constants which vary among Linux architectures. Version for SPARC.
+ Copyright (C) 2019 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#ifndef _SYS_SOCKET_H
+# error "Never include <bits/socket-constants.h> directly; use <sys/socket.h> instead."
+#endif
+
+#define SOL_SOCKET 65535
+#define SO_ACCEPTCONN 32768
+#define SO_BROADCAST 32
+#define SO_DONTROUTE 16
+#define SO_ERROR 4103
+#define SO_KEEPALIVE 8
+#define SO_LINGER 128
+#define SO_OOBINLINE 256
+#define SO_RCVBUF 4098
+#define SO_RCVLOWAT 2048
+#define SO_RCVTIMEO 8192
+#define SO_REUSEADDR 4
+#define SO_SNDBUF 4097
+#define SO_SNDLOWAT 4096
+#define SO_SNDTIMEO 16384
+#define SO_TYPE 4104
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/tst-socket-consts.py
@@ -0,0 +1,65 @@
+#!/usr/bin/python3
+# Test that glibc's sys/socket.h SO_* constants match the kernel's.
+# Copyright (C) 2018-2019 Free Software Foundation, Inc.
+# This file is part of the GNU C Library.
+#
+# The GNU C Library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# The GNU C Library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with the GNU C Library; if not, see
+# <http://www.gnu.org/licenses/>.
+
+import argparse
+import sys
+
+import glibcextract
+
+def main():
+ """The main entry point."""
+ parser = argparse.ArgumentParser(
+ description="Test that glibc's sys/socket.h constants "
+ "match the kernel's.")
+ parser.add_argument('--cc', metavar='CC',
+ help='C compiler (including options) to use')
+ args = parser.parse_args()
+
+ def check(define):
+ return glibcextract.compare_macro_consts(
+ source_1=define + '#include <sys/socket.h>\n',
+ # Some constants in <asm/socket.h> may depend on the size
+ # of pid_t or time_t.
+ source_2='#include <sys/types.h>\n'
+ '#include <asm/socket.h>\n',
+ cc=args.cc,
+ # We cannot compare all macros because some macros cannot
+ # be expanded as constants, and glibcextract currently is
+ # not able to isolate errors.
+ macro_re='SOL?_.*',
+ # <sys/socket.h> and <asm/socket.h> are not a good match.
+ # Most socket-related constants are not defined in any
+ # UAPI header. Check only the intersection of the macros
+ # in both headers. Regular tests ensure that expected
+ # macros for _GNU_SOURCE are present, and the conformance
+ # tests cover most of the other modes.
+ allow_extra_1=True,
+ allow_extra_2=True)
+ # _GNU_SOURCE is defined by include/libc-symbols.h, which is
+ # included by the --cc command. Defining _ISOMAC does not prevent
+ # that.
+ status = max(
+ check(''),
+ check('#undef _GNU_SOURCE\n'),
+ check('#undef _GNU_SOURCE\n'
+ '#define _POSIX_SOURCE 1\n'))
+ sys.exit(status)
+
+if __name__ == '__main__':
+ main()
From 477a02f63751c4b759ddd9454d17f2a7ad120ee3 Mon Sep 17 00:00:00 2001
From: Joseph Myers <joseph@codesourcery.com>
Date: Mon, 3 Dec 2018 22:08:50 +0000
Subject: [PATCH] Make gen-as-const.py handle '--' consistently with awk
script.
It was reported in
<https://sourceware.org/ml/libc-alpha/2018-12/msg00045.html> that
gen-as-const.py fails to generate test code in the case where a .sym
file has no symbols in it, so resulting in a test failing to link for
Hurd.
The relevant difference from the old awk script is that the old script
treated '--' lines as indicating that the text to do at the start of
the test (or file used to compute constants) should be output at that
point if not already output, as well as treating lines with actual
entries for constants like that. This patch changes gen-as-const.py
accordingly, making it the sole responsibility of the code parsing
.sym files to determine when such text should be output and ensuring
it's always output at some point even if there are no symbols and no
'--' lines, since not outputting it means the test fails to link.
Handling '--' like that also avoids any problems that would arise if
the first entry for a symbol were inside #ifdef (since the text in
question must not be output inside #ifdef).
Tested for x86_64, and with build-many-glibcs.py for i686-gnu. Note
that there are still compilation test failures for i686-gnu
(linknamespace tests, possibly arising from recent posix_spawn-related
changes).
* scripts/gen-as-const.py (compute_c_consts): Take an argument
'START' to indicate that start text should be output.
(gen_test): Likewise.
(main): Generate 'START' for first symbol or '--' line, or at end
of input if not previously generated.
---
ChangeLog | 8 ++++++
scripts/gen-as-const.py | 64 ++++++++++++++++++++++-------------------
2 files changed, 43 insertions(+), 29 deletions(-)
--- a/scripts/gen-as-const.py
+++ b/scripts/gen-as-const.py
@@ -34,28 +34,28 @@ def compute_c_consts(sym_data, cc):
"""Compute the values of some C constants.
The first argument is a list whose elements are either strings
- (preprocessor directives) or pairs of strings (a name and a C
+ (preprocessor directives, or the special string 'START' to
+ indicate this function should insert its initial boilerplate text
+ in the output there) or pairs of strings (a name and a C
expression for the corresponding value). Preprocessor directives
in the middle of the list may be used to select which constants
end up being evaluated using which expressions.
"""
out_lines = []
- started = False
for arg in sym_data:
if isinstance(arg, str):
- out_lines.append(arg)
+ if arg == 'START':
+ out_lines.append('void\ndummy (void)\n{')
+ else:
+ out_lines.append(arg)
continue
name = arg[0]
value = arg[1]
- if not started:
- out_lines.append('void\ndummy (void)\n{')
- started = True
out_lines.append('asm ("@@@name@@@%s@@@value@@@%%0@@@end@@@" '
': : \"i\" ((long int) (%s)));'
% (name, value))
- if started:
- out_lines.append('}')
+ out_lines.append('}')
out_lines.append('')
out_text = '\n'.join(out_lines)
with tempfile.TemporaryDirectory() as temp_dir:
@@ -89,32 +89,32 @@ def gen_test(sym_data):
"""
out_lines = []
- started = False
for arg in sym_data:
if isinstance(arg, str):
- out_lines.append(arg)
+ if arg == 'START':
+ out_lines.append('#include <stdint.h>\n'
+ '#include <stdio.h>\n'
+ '#include <bits/wordsize.h>\n'
+ '#if __WORDSIZE == 64\n'
+ 'typedef uint64_t c_t;\n'
+ '# define U(n) UINT64_C (n)\n'
+ '#else\n'
+ 'typedef uint32_t c_t;\n'
+ '# define U(n) UINT32_C (n)\n'
+ '#endif\n'
+ 'static int\n'
+ 'do_test (void)\n'
+ '{\n'
+ # Compilation test only, using static
+ # assertions.
+ ' return 0;\n'
+ '}\n'
+ '#include <support/test-driver.c>')
+ else:
+ out_lines.append(arg)
continue
name = arg[0]
value = arg[1]
- if not started:
- out_lines.append('#include <stdint.h>\n'
- '#include <stdio.h>\n'
- '#include <bits/wordsize.h>\n'
- '#if __WORDSIZE == 64\n'
- 'typedef uint64_t c_t;\n'
- '# define U(n) UINT64_C (n)\n'
- '#else\n'
- 'typedef uint32_t c_t;\n'
- '# define U(n) UINT32_C (n)\n'
- '#endif\n'
- 'static int\n'
- 'do_test (void)\n'
- '{\n'
- # Compilation test only, using static assertions.
- ' return 0;\n'
- '}\n'
- '#include <support/test-driver.c>')
- started = True
out_lines.append('_Static_assert (U (asconst_%s) == (c_t) (%s), '
'"value of %s");'
% (name, value, name))
@@ -134,6 +134,7 @@ def main():
args = parser.parse_args()
sym_data = []
with open(args.sym_file, 'r') as sym_file:
+ started = False
for line in sym_file:
line = line.strip()
if line == '':
@@ -143,12 +144,17 @@ def main():
sym_data.append(line)
continue
words = line.split(maxsplit=1)
+ if not started:
+ sym_data.append('START')
+ started = True
# Separator.
if words[0] == '--':
continue
name = words[0]
value = words[1] if len(words) > 1 else words[0]
sym_data.append((name, value))
+ if not started:
+ sym_data.append('START')
if args.test:
print(gen_test(sym_data))
else:
From a8110b727e508f7ddf34f940af622e6f95435201 Mon Sep 17 00:00:00 2001
From: Joseph Myers <joseph@codesourcery.com>
Date: Mon, 10 Dec 2018 22:27:13 +0000
Subject: [PATCH] Move tst-signal-numbers to Python.
This patch converts the tst-signal-numbers test from shell + awk to
Python.
As with gen-as-const, the point is not so much that shell and awk are
problematic for this code, as that it's useful to build up general
infrastructure in Python for use of a range of code involving
extracting values from C headers. This patch moves some code from
gen-as-const.py to a new glibcextract.py, which also gains functions
relating to listing macros, and comparing the values of a set of
macros from compiling two different pieces of code.
It's not just signal numbers that should have such tests; pretty much
any case where glibc copies constants from Linux kernel headers should
have such tests that the values and sets of constants agree except
where differences are known to be OK. Much the same also applies to
structure layouts (although testing those without hardcoding lists of
fields to test will be more complicated).
Given this patch, another test for a set of macros would essentially
be just a call to glibcextract.compare_macro_consts (plus boilerplate
code - and we could move to having separate text files defining such
tests, like the .sym inputs to gen-as-const, so that only a single
Python script is needed for most such tests). Some such tests would
of course need new features, e.g. where the set of macros changes in
new kernel versions (so you need to allow new macro names on the
kernel side if the kernel headers are newer than the version known to
glibc, and extra macros on the glibc side if the kernel headers are
older). tst-syscall-list.sh could become a Python script that uses
common code to generate lists of macros but does other things with its
own custom logic.
There are a few differences from the existing shell + awk test.
Because the new test evaluates constants using the compiler, no
special handling is needed any more for one signal name being defined
to another. Because asm/signal.h now needs to pass through the
compiler, not just the preprocessor, stddef.h is included as well
(given the asm/signal.h issue that it requires an externally provided
definition of size_t). The previous code defined __ASSEMBLER__ with
asm/signal.h; this is removed (__ASSEMBLY__, a different macro,
eliminates the requirement for stddef.h on some but not all
architectures).
Tested for x86_64, and with build-many-glibcs.py.
* scripts/glibcextract.py: New file.
* scripts/gen-as-const.py: Do not import os.path, re, subprocess
or tempfile. Import glibcexctract.
(compute_c_consts): Remove. Moved to glibcextract.py.
(gen_test): Update reference to compute_c_consts.
(main): Likewise.
* sysdeps/unix/sysv/linux/tst-signal-numbers.py: New file.
* sysdeps/unix/sysv/linux/tst-signal-numbers.sh: Remove.
* sysdeps/unix/sysv/linux/Makefile
($(objpfx)tst-signal-numbers.out): Use tst-signal-numbers.py.
Redirect stderr as well as stdout.
---
ChangeLog | 14 ++
scripts/gen-as-const.py | 60 +------
scripts/glibcextract.py | 162 ++++++++++++++++++
sysdeps/unix/sysv/linux/Makefile | 11 +-
sysdeps/unix/sysv/linux/tst-signal-numbers.py | 48 ++++++
sysdeps/unix/sysv/linux/tst-signal-numbers.sh | 86 ----------
6 files changed, 234 insertions(+), 147 deletions(-)
create mode 100644 scripts/glibcextract.py
create mode 100644 sysdeps/unix/sysv/linux/tst-signal-numbers.py
delete mode 100644 sysdeps/unix/sysv/linux/tst-signal-numbers.sh
--- a/scripts/gen-as-const.py
+++ b/scripts/gen-as-const.py
@@ -24,68 +24,14 @@
# A line giving just a name implies an expression consisting of just that name.
import argparse
-import os.path
-import re
-import subprocess
-import tempfile
-
-
-def compute_c_consts(sym_data, cc):
- """Compute the values of some C constants.
-
- The first argument is a list whose elements are either strings
- (preprocessor directives, or the special string 'START' to
- indicate this function should insert its initial boilerplate text
- in the output there) or pairs of strings (a name and a C
- expression for the corresponding value). Preprocessor directives
- in the middle of the list may be used to select which constants
- end up being evaluated using which expressions.
- """
- out_lines = []
- for arg in sym_data:
- if isinstance(arg, str):
- if arg == 'START':
- out_lines.append('void\ndummy (void)\n{')
- else:
- out_lines.append(arg)
- continue
- name = arg[0]
- value = arg[1]
- out_lines.append('asm ("@@@name@@@%s@@@value@@@%%0@@@end@@@" '
- ': : \"i\" ((long int) (%s)));'
- % (name, value))
- out_lines.append('}')
- out_lines.append('')
- out_text = '\n'.join(out_lines)
- with tempfile.TemporaryDirectory() as temp_dir:
- c_file_name = os.path.join(temp_dir, 'test.c')
- s_file_name = os.path.join(temp_dir, 'test.s')
- with open(c_file_name, 'w') as c_file:
- c_file.write(out_text)
- # Compilation has to be from stdin to avoid the temporary file
- # name being written into the generated dependencies.
- cmd = ('%s -S -o %s -x c - < %s' % (cc, s_file_name, c_file_name))
- subprocess.check_call(cmd, shell=True)
- consts = {}
- with open(s_file_name, 'r') as s_file:
- for line in s_file:
- match = re.search('@@@name@@@([^@]*)'
- '@@@value@@@[^0-9Xxa-fA-F-]*'
- '([0-9Xxa-fA-F-]+).*@@@end@@@', line)
- if match:
- if (match.group(1) in consts
- and match.group(2) != consts[match.group(1)]):
- raise ValueError('duplicate constant %s'
- % match.group(1))
- consts[match.group(1)] = match.group(2)
- return consts
+import glibcextract
def gen_test(sym_data):
"""Generate a test for the values of some C constants.
- The first argument is as for compute_c_consts.
+ The first argument is as for glibcextract.compute_c_consts.
"""
out_lines = []
@@ -158,7 +104,7 @@ def main():
if args.test:
print(gen_test(sym_data))
else:
- consts = compute_c_consts(sym_data, args.cc)
+ consts = glibcextract.compute_c_consts(sym_data, args.cc)
print(''.join('#define %s %s\n' % c for c in sorted(consts.items())), end='')
if __name__ == '__main__':
--- /dev/null
+++ b/scripts/glibcextract.py
@@ -0,0 +1,162 @@
+#!/usr/bin/python3
+# Extract information from C headers.
+# Copyright (C) 2018 Free Software Foundation, Inc.
+# This file is part of the GNU C Library.
+#
+# The GNU C Library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# The GNU C Library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with the GNU C Library; if not, see
+# <http://www.gnu.org/licenses/>.
+
+import os.path
+import re
+import subprocess
+import tempfile
+
+
+def compute_c_consts(sym_data, cc):
+ """Compute the values of some C constants.
+
+ The first argument is a list whose elements are either strings
+ (preprocessor directives, or the special string 'START' to
+ indicate this function should insert its initial boilerplate text
+ in the output there) or pairs of strings (a name and a C
+ expression for the corresponding value). Preprocessor directives
+ in the middle of the list may be used to select which constants
+ end up being evaluated using which expressions.
+
+ """
+ out_lines = []
+ for arg in sym_data:
+ if isinstance(arg, str):
+ if arg == 'START':
+ out_lines.append('void\ndummy (void)\n{')
+ else:
+ out_lines.append(arg)
+ continue
+ name = arg[0]
+ value = arg[1]
+ out_lines.append('asm ("@@@name@@@%s@@@value@@@%%0@@@end@@@" '
+ ': : \"i\" ((long int) (%s)));'
+ % (name, value))
+ out_lines.append('}')
+ out_lines.append('')
+ out_text = '\n'.join(out_lines)
+ with tempfile.TemporaryDirectory() as temp_dir:
+ c_file_name = os.path.join(temp_dir, 'test.c')
+ s_file_name = os.path.join(temp_dir, 'test.s')
+ with open(c_file_name, 'w') as c_file:
+ c_file.write(out_text)
+ # Compilation has to be from stdin to avoid the temporary file
+ # name being written into the generated dependencies.
+ cmd = ('%s -S -o %s -x c - < %s' % (cc, s_file_name, c_file_name))
+ subprocess.check_call(cmd, shell=True)
+ consts = {}
+ with open(s_file_name, 'r') as s_file:
+ for line in s_file:
+ match = re.search('@@@name@@@([^@]*)'
+ '@@@value@@@[^0-9Xxa-fA-F-]*'
+ '([0-9Xxa-fA-F-]+).*@@@end@@@', line)
+ if match:
+ if (match.group(1) in consts
+ and match.group(2) != consts[match.group(1)]):
+ raise ValueError('duplicate constant %s'
+ % match.group(1))
+ consts[match.group(1)] = match.group(2)
+ return consts
+
+
+def list_macros(source_text, cc):
+ """List the preprocessor macros defined by the given source code.
+
+ The return value is a pair of dicts, the first one mapping macro
+ names to their expansions and the second one mapping macro names
+ to lists of their arguments, or to None for object-like macros.
+
+ """
+ with tempfile.TemporaryDirectory() as temp_dir:
+ c_file_name = os.path.join(temp_dir, 'test.c')
+ i_file_name = os.path.join(temp_dir, 'test.i')
+ with open(c_file_name, 'w') as c_file:
+ c_file.write(source_text)
+ cmd = ('%s -E -dM -o %s %s' % (cc, i_file_name, c_file_name))
+ subprocess.check_call(cmd, shell=True)
+ macros_exp = {}
+ macros_args = {}
+ with open(i_file_name, 'r') as i_file:
+ for line in i_file:
+ match = re.fullmatch('#define ([0-9A-Za-z_]+)(.*)\n', line)
+ if not match:
+ raise ValueError('bad -dM output line: %s' % line)
+ name = match.group(1)
+ value = match.group(2)
+ if value.startswith(' '):
+ value = value[1:]
+ args = None
+ elif value.startswith('('):
+ match = re.fullmatch(r'\((.*?)\) (.*)', value)
+ if not match:
+ raise ValueError('bad -dM output line: %s' % line)
+ args = match.group(1).split(',')
+ value = match.group(2)
+ else:
+ raise ValueError('bad -dM output line: %s' % line)
+ if name in macros_exp:
+ raise ValueError('duplicate macro: %s' % line)
+ macros_exp[name] = value
+ macros_args[name] = args
+ return macros_exp, macros_args
+
+
+def compute_macro_consts(source_text, cc, macro_re, exclude_re=None):
+ """Compute the integer constant values of macros defined by source_text.
+
+ Macros must match the regular expression macro_re, and if
+ exclude_re is defined they must not match exclude_re. Values are
+ computed with compute_c_consts.
+
+ """
+ macros_exp, macros_args = list_macros(source_text, cc)
+ macros_set = {m for m in macros_exp
+ if (macros_args[m] is None
+ and re.fullmatch(macro_re, m)
+ and (exclude_re is None
+ or not re.fullmatch(exclude_re, m)))}
+ sym_data = [source_text, 'START']
+ sym_data.extend(sorted((m, m) for m in macros_set))
+ return compute_c_consts(sym_data, cc)
+
+
+def compare_macro_consts(source_1, source_2, cc, macro_re, exclude_re=None):
+ """Compare the values of macros defined by two different sources.
+
+ The sources would typically be includes of a glibc header and a
+ kernel header. Return 1 if there were any differences, 0 if the
+ macro values were the same.
+
+ """
+ macros_1 = compute_macro_consts(source_1, cc, macro_re, exclude_re)
+ macros_2 = compute_macro_consts(source_2, cc, macro_re, exclude_re)
+ if macros_1 == macros_2:
+ return 0
+ print('First source:\n%s\n' % source_1)
+ print('Second source:\n%s\n' % source_2)
+ for name, value in sorted(macros_1.items()):
+ if name not in macros_2:
+ print('Only in first source: %s' % name)
+ elif macros_1[name] != macros_2[name]:
+ print('Different values for %s: %s != %s'
+ % (name, macros_1[name], macros_2[name]))
+ for name in sorted(macros_2.keys()):
+ if name not in macros_1:
+ print('Only in second source: %s' % name)
+ return 1
--- a/sysdeps/unix/sysv/linux/Makefile
+++ b/sysdeps/unix/sysv/linux/Makefile
@@ -112,11 +112,14 @@ tests-special += $(objpfx)tst-signal-num
# in this context, but signal.c includes signal.h and not much else so it'll
# be conservatively correct.
$(objpfx)tst-signal-numbers.out: \
- ../sysdeps/unix/sysv/linux/tst-signal-numbers.sh \
+ ../sysdeps/unix/sysv/linux/tst-signal-numbers.py \
$(objpfx)signal.o*
- AWK=$(AWK) $(SHELL) ../sysdeps/unix/sysv/linux/tst-signal-numbers.sh \
- $(CC) $(patsubst -DMODULE_NAME=%,-DMODULE_NAME=testsuite,$(CPPFLAGS)) \
- < /dev/null > $@; $(evaluate-test)
+ PYTHONPATH=../scripts \
+ $(PYTHON) ../sysdeps/unix/sysv/linux/tst-signal-numbers.py \
+ --cc="$(CC) $(patsubst -DMODULE_NAME=%, \
+ -DMODULE_NAME=testsuite, \
+ $(CPPFLAGS))" \
+ < /dev/null > $@ 2>&1; $(evaluate-test)
endif
ifeq ($(subdir),socket)
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/tst-signal-numbers.py
@@ -0,0 +1,48 @@
+#!/usr/bin/python3
+# Test that glibc's signal numbers match the kernel's.
+# Copyright (C) 2018 Free Software Foundation, Inc.
+# This file is part of the GNU C Library.
+#
+# The GNU C Library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# The GNU C Library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with the GNU C Library; if not, see
+# <http://www.gnu.org/licenses/>.
+
+import argparse
+import sys
+
+import glibcextract
+
+
+def main():
+ """The main entry point."""
+ parser = argparse.ArgumentParser(
+ description="Test that glibc's signal numbers match the kernel's.")
+ parser.add_argument('--cc', metavar='CC',
+ help='C compiler (including options) to use')
+ args = parser.parse_args()
+ sys.exit(glibcextract.compare_macro_consts(
+ '#define _GNU_SOURCE 1\n'
+ '#include <signal.h>\n',
+ '#define _GNU_SOURCE 1\n'
+ '#include <stddef.h>\n'
+ '#include <asm/signal.h>\n',
+ args.cc,
+ # Filter out constants that aren't signal numbers.
+ 'SIG[A-Z]+',
+ # Discard obsolete signal numbers and unrelated constants:
+ # SIGCLD, SIGIOT, SIGSWI, SIGUNUSED.
+ # SIGSTKSZ, SIGRTMIN, SIGRTMAX.
+ 'SIG(CLD|IOT|RT(MIN|MAX)|STKSZ|SWI|UNUSED)'))
+
+if __name__ == '__main__':
+ main()
--- a/sysdeps/unix/sysv/linux/tst-signal-numbers.sh
+++ /dev/null
@@ -1,86 +0,0 @@
-#! /bin/sh
-# Test that glibc's signal numbers match the kernel's.
-# Copyright (C) 2017-2018 Free Software Foundation, Inc.
-# This file is part of the GNU C Library.
-
-# The GNU C Library is free software; you can redistribute it and/or
-# modify it under the terms of the GNU Lesser General Public
-# License as published by the Free Software Foundation; either
-# version 2.1 of the License, or (at your option) any later version.
-
-# The GNU C Library is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-# Lesser General Public License for more details.
-
-# You should have received a copy of the GNU Lesser General Public
-# License along with the GNU C Library; if not, see
-# <http://www.gnu.org/licenses/>.
-
-set -e
-if [ -n "$BASH_VERSION" ]; then set -o pipefail; fi
-LC_ALL=C; export LC_ALL
-
-# We cannot use Linux's asm/signal.h to define signal numbers, because
-# it isn't sufficiently namespace-clean. Instead, this test checks
-# that our signal numbers match the kernel's. This script expects
-# "$@" to be $(CC) $(CPPFLAGS) as set by glibc's Makefiles, and $AWK
-# to be set in the environment.
-
-# Before doing anything else, fail if the compiler doesn't work.
-"$@" -E -xc -dM - < /dev/null > /dev/null
-
-tmpG=`mktemp -t signums_glibc.XXXXXXXXX`
-tmpK=`mktemp -t signums_kernel.XXXXXXXXX`
-trap "rm -f '$tmpG' '$tmpK'" 0
-
-# Filter out constants that aren't signal numbers.
-# If SIGPOLL is defined as SIGIO, swap it around so SIGIO is defined as
-# SIGPOLL. Similarly for SIGABRT and SIGIOT.
-# Discard obsolete signal numbers and unrelated constants:
-# SIGCLD, SIGIOT, SIGSWI, SIGUNUSED.
-# SIGSTKSZ, SIGRTMIN, SIGRTMAX.
-# Then sort the list.
-filter_defines ()
-{
- $AWK '
-/^#define SIG[A-Z]+ ([0-9]+|SIG[A-Z0-9]+)$/ { signals[$2] = $3 }
-END {
- if ("SIGPOLL" in signals && "SIGIO" in signals &&
- signals["SIGPOLL"] == "SIGIO") {
- signals["SIGPOLL"] = signals["SIGIO"]
- signals["SIGIO"] = "SIGPOLL"
- }
- if ("SIGABRT" in signals && "SIGIOT" in signals &&
- signals["SIGABRT"] == "SIGIOT") {
- signals["SIGABRT"] = signals["SIGIOT"]
- signals["SIGIOT"] = "SIGABRT"
- }
- for (sig in signals) {
- if (sig !~ /^SIG(CLD|IOT|RT(MIN|MAX)|STKSZ|SWI|UNUSED)$/) {
- printf("#define %s %s\n", sig, signals[sig])
- }
- }
-}' | sort
-}
-
-# $CC may contain command-line switches, so it should be word-split.
-printf '%s' '#define _GNU_SOURCE 1
-#include <signal.h>
-' |
- "$@" -E -xc -dM - |
- filter_defines > "$tmpG"
-
-printf '%s' '#define _GNU_SOURCE 1
-#define __ASSEMBLER__ 1
-#include <asm/signal.h>
-' |
- "$@" -E -xc -dM - |
- filter_defines > "$tmpK"
-
-if cmp -s "$tmpG" "$tmpK"; then
- exit 0
-else
- diff -u "$tmpG" "$tmpK"
- exit 1
-fi
From 7e1d42400c1b8f03316fe14176133c8853cd3bbe Mon Sep 17 00:00:00 2001
From: Joseph Myers <joseph@codesourcery.com>
Date: Fri, 30 Nov 2018 15:20:41 +0000
Subject: [PATCH] Replace gen-as-const.awk by gen-as-const.py.
This patch replaces gen-as-const.awk, and some fragments of the
Makefile code that used it, by a Python script. The point is not such
much that awk is problematic for this particular script, as that I'd
like to build up a general Python infrastructure for extracting
information from C headers, for use in writing tests of such headers.
Thus, although this patch does not set up such infrastructure, the
compute_c_consts function in gen-as-const.py might be moved to a
separate Python module in a subsequent patch as a starting point for
such infrastructure.
The general idea of the code is the same as in the awk version, but no
attempt is made to make the output files textually identical. When
generating a header, a dict of constant names and values is generated
internally then defines are printed in sorted order (rather than the
order in the .sym file, which would have been used before). When
generating a test that the values computed match those from a normal
header inclusion, the test code is made into a compilation test using
_Static_assert, where previously the comparisons were done only when
the test was executed. One fragment of test generation (converting
the previously generated header to use asconst_* prefixes on its macro
names) is still in awk code in the makefiles; only the .sym processing
and subsequent execution of the compiler to extract constants have
moved to the Python script.
Tested for x86_64, and with build-many-glibcs.py.
* scripts/gen-as-const.py: New file.
* scripts/gen-as-const.awk: Remove.
* Makerules ($(common-objpfx)%.h $(common-objpfx)%.h.d): Use
gen-as-const.py.
($(objpfx)test-as-const-%.c): Likewise.
---
ChangeLog | 8 ++
Makerules | 18 ++---
scripts/gen-as-const.awk | 63 ----------------
scripts/gen-as-const.py | 159 +++++++++++++++++++++++++++++++++++++++
4 files changed, 174 insertions(+), 74 deletions(-)
delete mode 100644 scripts/gen-as-const.awk
create mode 100644 scripts/gen-as-const.py
--- a/Makerules
+++ b/Makerules
@@ -282,15 +282,12 @@ ifdef gen-as-const-headers
# may include <tcb-offsets.h>. Target header files can check if
# GEN_AS_CONST_HEADERS is defined to avoid circular dependency which
# may lead to build hang on a many-core machine.
-$(common-objpfx)%.h $(common-objpfx)%.h.d: $(..)scripts/gen-as-const.awk \
+$(common-objpfx)%.h $(common-objpfx)%.h.d: $(..)scripts/gen-as-const.py \
%.sym $(common-before-compile)
- $(AWK) -f $< $(filter %.sym,$^) \
- | $(CC) -S -o $(@:.h.d=.h)T3 $(CFLAGS) $(CPPFLAGS) \
- -DGEN_AS_CONST_HEADERS -x c - \
- -MD -MP -MF $(@:.h=.h.d)T -MT '$(@:.h=.h.d) $(@:.h.d=.h)'
- sed -n 's/^.*@@@name@@@\([^@]*\)@@@value@@@[^0-9Xxa-fA-F-]*\([0-9Xxa-fA-F-][0-9Xxa-fA-F-]*\).*@@@end@@@.*$$/#define \1 \2/p' \
- $(@:.h.d=.h)T3 > $(@:.h.d=.h)T
- rm -f $(@:.h.d=.h)T3
+ $(PYTHON) $< --cc="$(CC) $(CFLAGS) $(CPPFLAGS) -DGEN_AS_CONST_HEADERS \
+ -MD -MP -MF $(@:.h=.h.d)T \
+ -MT '$(@:.h=.h.d) $(@:.h.d=.h)'" \
+ $(filter %.sym,$^) > $(@:.h.d=.h)T
sed $(sed-remove-objpfx) $(sed-remove-dotdot) \
$(@:.h=.h.d)T > $(@:.h=.h.d)T2
rm -f $(@:.h=.h.d)T
@@ -301,11 +298,10 @@ before-compile += $(gen-as-const-headers
tests-internal += $(gen-as-const-headers:%.sym=test-as-const-%)
generated += $(gen-as-const-headers:%.sym=test-as-const-%.c)
-$(objpfx)test-as-const-%.c: $(..)scripts/gen-as-const.awk $(..)Makerules \
+$(objpfx)test-as-const-%.c: $(..)scripts/gen-as-const.py $(..)Makerules \
%.sym $(common-objpfx)%.h
($(AWK) '{ sub(/^/, "asconst_", $$2); print; }' $(filter %.h,$^); \
- $(AWK) -v test=1 -f $< $(filter %.sym,$^); \
- echo '#include "$(..)test-skeleton.c"') > $@T
+ $(PYTHON) $< --test $(filter %.sym,$^)) > $@T
mv -f $@T $@
endif
--- a/scripts/gen-as-const.awk
+++ /dev/null
@@ -1,63 +0,0 @@
-# Script used in producing headers of assembly constants from C expressions.
-# The input to this script looks like:
-# #cpp-directive ...
-# NAME1
-# NAME2 expression ...
-# The output of this script is C code to be run through gcc -S and then
-# massaged to extract the integer constant values of the given C expressions.
-# A line giving just a name implies an expression consisting of just that name.
-
-BEGIN { started = 0 }
-
-# cpp directives go straight through.
-/^#/ { print; next }
-
-NF >= 1 && !started {
- if (test) {
- print "\n#include <inttypes.h>";
- print "\n#include <stdio.h>";
- print "\n#include <bits/wordsize.h>";
- print "\n#if __WORDSIZE == 64";
- print "\ntypedef uint64_t c_t;";
- print "\n#define U(n) UINT64_C (n)";
- print "\n#define PRI PRId64";
- print "\n#else";
- print "\ntypedef uint32_t c_t;";
- print "\n#define U(n) UINT32_C (n)";
- print "\n#define PRI PRId32";
- print "\n#endif";
- print "\nstatic int do_test (void)\n{\n int bad = 0, good = 0;\n";
- print "#define TEST(name, source, expr) \\\n" \
- " if (U (asconst_##name) != (c_t) (expr)) { ++bad;" \
- " fprintf (stderr, \"%s: %s is %\" PRI \" but %s is %\"PRI \"\\n\"," \
- " source, #name, U (asconst_##name), #expr, (c_t) (expr));" \
- " } else ++good;\n";
- }
- else
- print "void dummy(void) {";
- started = 1;
-}
-
-# Separator.
-$1 == "--" { next }
-
-NF == 1 { sub(/^.*$/, "& &"); }
-
-NF > 1 {
- name = $1;
- sub(/^[^ ]+[ ]+/, "");
- if (test)
- print " TEST (" name ", \"" FILENAME ":" FNR "\", " $0 ")";
- else
- printf "asm (\"@@@name@@@%s@@@value@@@%%0@@@end@@@\" : : \"i\" ((long) %s));\n",
- name, $0;
-}
-
-END {
- if (test) {
- print " printf (\"%d errors in %d tests\\n\", bad, good + bad);"
- print " return bad != 0 || good == 0;\n}\n";
- print "#define TEST_FUNCTION do_test ()";
- }
- else if (started) print "}";
-}
--- /dev/null
+++ b/scripts/gen-as-const.py
@@ -0,0 +1,159 @@
+#!/usr/bin/python3
+# Produce headers of assembly constants from C expressions.
+# Copyright (C) 2018 Free Software Foundation, Inc.
+# This file is part of the GNU C Library.
+#
+# The GNU C Library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# The GNU C Library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with the GNU C Library; if not, see
+# <http://www.gnu.org/licenses/>.
+
+# The input to this script looks like:
+# #cpp-directive ...
+# NAME1
+# NAME2 expression ...
+# A line giving just a name implies an expression consisting of just that name.
+
+import argparse
+import os.path
+import re
+import subprocess
+import tempfile
+
+
+def compute_c_consts(sym_data, cc):
+ """Compute the values of some C constants.
+
+ The first argument is a list whose elements are either strings
+ (preprocessor directives) or pairs of strings (a name and a C
+ expression for the corresponding value). Preprocessor directives
+ in the middle of the list may be used to select which constants
+ end up being evaluated using which expressions.
+
+ """
+ out_lines = []
+ started = False
+ for arg in sym_data:
+ if isinstance(arg, str):
+ out_lines.append(arg)
+ continue
+ name = arg[0]
+ value = arg[1]
+ if not started:
+ out_lines.append('void\ndummy (void)\n{')
+ started = True
+ out_lines.append('asm ("@@@name@@@%s@@@value@@@%%0@@@end@@@" '
+ ': : \"i\" ((long int) (%s)));'
+ % (name, value))
+ if started:
+ out_lines.append('}')
+ out_lines.append('')
+ out_text = '\n'.join(out_lines)
+ with tempfile.TemporaryDirectory() as temp_dir:
+ c_file_name = os.path.join(temp_dir, 'test.c')
+ s_file_name = os.path.join(temp_dir, 'test.s')
+ with open(c_file_name, 'w') as c_file:
+ c_file.write(out_text)
+ # Compilation has to be from stdin to avoid the temporary file
+ # name being written into the generated dependencies.
+ cmd = ('%s -S -o %s -x c - < %s' % (cc, s_file_name, c_file_name))
+ subprocess.check_call(cmd, shell=True)
+ consts = {}
+ with open(s_file_name, 'r') as s_file:
+ for line in s_file:
+ match = re.search('@@@name@@@([^@]*)'
+ '@@@value@@@[^0-9Xxa-fA-F-]*'
+ '([0-9Xxa-fA-F-]+).*@@@end@@@', line)
+ if match:
+ if (match.group(1) in consts
+ and match.group(2) != consts[match.group(1)]):
+ raise ValueError('duplicate constant %s'
+ % match.group(1))
+ consts[match.group(1)] = match.group(2)
+ return consts
+
+
+def gen_test(sym_data):
+ """Generate a test for the values of some C constants.
+
+ The first argument is as for compute_c_consts.
+
+ """
+ out_lines = []
+ started = False
+ for arg in sym_data:
+ if isinstance(arg, str):
+ out_lines.append(arg)
+ continue
+ name = arg[0]
+ value = arg[1]
+ if not started:
+ out_lines.append('#include <stdint.h>\n'
+ '#include <stdio.h>\n'
+ '#include <bits/wordsize.h>\n'
+ '#if __WORDSIZE == 64\n'
+ 'typedef uint64_t c_t;\n'
+ '# define U(n) UINT64_C (n)\n'
+ '#else\n'
+ 'typedef uint32_t c_t;\n'
+ '# define U(n) UINT32_C (n)\n'
+ '#endif\n'
+ 'static int\n'
+ 'do_test (void)\n'
+ '{\n'
+ # Compilation test only, using static assertions.
+ ' return 0;\n'
+ '}\n'
+ '#include <support/test-driver.c>')
+ started = True
+ out_lines.append('_Static_assert (U (asconst_%s) == (c_t) (%s), '
+ '"value of %s");'
+ % (name, value, name))
+ return '\n'.join(out_lines)
+
+
+def main():
+ """The main entry point."""
+ parser = argparse.ArgumentParser(
+ description='Produce headers of assembly constants.')
+ parser.add_argument('--cc', metavar='CC',
+ help='C compiler (including options) to use')
+ parser.add_argument('--test', action='store_true',
+ help='Generate test case instead of header')
+ parser.add_argument('sym_file',
+ help='.sym file to process')
+ args = parser.parse_args()
+ sym_data = []
+ with open(args.sym_file, 'r') as sym_file:
+ for line in sym_file:
+ line = line.strip()
+ if line == '':
+ continue
+ # Pass preprocessor directives through.
+ if line.startswith('#'):
+ sym_data.append(line)
+ continue
+ words = line.split(maxsplit=1)
+ # Separator.
+ if words[0] == '--':
+ continue
+ name = words[0]
+ value = words[1] if len(words) > 1 else words[0]
+ sym_data.append((name, value))
+ if args.test:
+ print(gen_test(sym_data))
+ else:
+ consts = compute_c_consts(sym_data, args.cc)
+ print('\n'.join('#define %s %s' % c for c in sorted(consts.items())))
+
+if __name__ == '__main__':
+ main()
......@@ -161,3 +161,11 @@ any/submitted-resolv-unaligned.diff
any/local-cudacc-float128.diff
any/git-libio-stdout-putc.diff
remove-bashisms
apertis/backport-Replace-gen-as-const.awk-by-gen-as-const.py.patch
apertis/backport-Fix-test-as-const-jmp_buf-ssp.c-generation-on-gnu-i3.patch
apertis/backport-Make-gen-as-const.py-handle-consistently-with-awk-sc.patch
apertis/backport-Move-tst-signal-numbers-to-Python.patch
apertis/backport-Add-test-that-MAP_-constants-agree-with-kernel.patch
apertis/backport-Use-a-proper-C-tokenizer-to-implement-the-obsolete-t.patch
apertis/backport-Linux-Use-in-tree-copy-of-SO_-constants-for-__USE_MI.patch
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment