This function no longer waits for threads to leave g1, so rename it to
__condvar_switch_g1
Signed-off-by: Malte Skarupke <malteskarupke@fastmail.fm>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
In my previous change I turned a nested loop into a simple loop. I'm doing
the resulting indentation changes in a separate commit to make the diff on
the previous commit easier to review.
Signed-off-by: Malte Skarupke <malteskarupke@fastmail.fm>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
The loop was a little more complicated than necessary. There was only one
break statement out of the inner loop, and the outer loop was nearly empty.
So just remove the outer loop, moving its code to the one break statement in
the inner loop. This allows us to replace all gotos with break statements.
Signed-off-by: Malte Skarupke <malteskarupke@fastmail.fm>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
This variable used to be needed to wait in group switching until all sleepers
have confirmed that they have woken. This is no longer needed. Nothing waits
on this variable so there is no need to track how many threads are currently
asleep in each group.
Signed-off-by: Malte Skarupke <malteskarupke@fastmail.fm>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
pthread_cond_wait was checking whether it was in a closed group no less than
four times. Checking once is enough. Here are the four checks:
1. While spin-waiting. This was dead code: maxspin is set to 0 and has been
for years.
2. Before deciding to go to sleep, and before incrementing grefs: I kept this
3. After incrementing grefs. There is no reason to think that the group would
close while we do an atomic increment. Obviously it could close at any
point, but that doesn't mean we have to recheck after every step. This
check was equally good as check 2, except it has to do more work.
4. When we find ourselves in a group that has a signal. We only get here after
we check that we're not in a closed group. There is no need to check again.
The check would only have helped in cases where the compare_exchange in the
next line would also have failed. Relying on the compare_exchange is fine.
Removing the duplicate checks clarifies the code.
Signed-off-by: Malte Skarupke <malteskarupke@fastmail.fm>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
This wake is unnecessary. We only switch groups after every sleeper in a group
has been woken. Sure, they may take a while to actually wake up and may still
hold a reference, but waking them a second time doesn't speed that up. Instead
this just makes the code more complicated and may hide problems.
In particular this safety wake wouldn't even have helped with the bug that was
fixed by Barrus' patch: The bug there was that pthread_cond_signal would not
switch g1 when it should, so we wouldn't even have entered this code path.
Signed-off-by: Malte Skarupke <malteskarupke@fastmail.fm>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Some comments were wrong after the most recent commit. This fixes that.
Also fixing indentation where it was using spaces instead of tabs.
Signed-off-by: Malte Skarupke <malteskarupke@fastmail.fm>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
This fixes the lost wakeup (from a bug in signal stealing) with a change
in the usage of g_signals[] in the condition variable internal state.
It also completely eliminates the concept and handling of signal stealing,
as well as the need for signalers to block to wait for waiters to wake
up every time there is a G1/G2 switch. This greatly reduces the average
and maximum latency for pthread_cond_signal.
The g_signals[] field now contains a signal count that is relative to
the current g1_start value. Since it is a 32-bit field, and the LSB is
still reserved (though not currently used anymore), it has a 31-bit value
that corresponds to the low 31 bits of the sequence number in g1_start.
(since g1_start also has an LSB flag, this means bits 31:1 in g_signals
correspond to bits 31:1 in g1_start, plus the current signal count)
By making the signal count relative to g1_start, there is no longer
any ambiguity or A/B/A issue, and thus any checks before blocking,
including the futex call itself, are guaranteed not to block if the G1/G2
switch occurs, even if the signal count remains the same. This allows
initially safely blocking in G2 until the switch to G1 occurs, and
then transitioning from G1 to a new G1 or G2, and always being able to
distinguish the state change. This removes the race condition and A/B/A
problems that otherwise ocurred if a late (pre-empted) waiter were to
resume just as the futex call attempted to block on g_signal since
otherwise there was no last opportunity to re-check things like whether
the current G1 group was already closed.
By fixing these issues, the signal stealing code can be eliminated,
since there is no concept of signal stealing anymore. The code to block
for all waiters to exit g_refs can also be removed, since any waiters
that are still in the g_refs region can be guaranteed to safely wake
up and exit. If there are still any left at this time, they are all
sent one final futex wakeup to ensure that they are not blocked any
longer, but there is no need for the signaller to block and wait for
them to wake up and exit the g_refs region.
The signal count is then effectively "zeroed" but since it is now
relative to g1_start, this is done by advancing it to a new value that
can be observed by any pending blocking waiters. Any late waiters can
always tell the difference, and can thus just cleanly exit if they are
in a stale G1 or G2. They can never steal a signal from the current
G1 if they are not in the current G1, since the signal value that has
to match in the cmpxchg has the low 31 bits of the g1_start value
contained in it, and that's first checked, and then it won't match if
there's a G1/G2 change.
Note: the 31-bit sequence number used in g_signals is designed to
handle wrap-around when checking the signal count, but if the entire
31-bit wraparound (2 billion signals) occurs while there is still a
late waiter that has not yet resumed, and it happens to then match
the current g1_start low bits, and the pre-emption occurs after the
normal "closed group" checks (which are 64-bit) but then hits the
futex syscall and signal consuming code, then an A/B/A issue could
still result and cause an incorrect assumption about whether it
should block. This particular scenario seems unlikely in practice.
Note that once awake from the futex, the waiter would notice the
closed group before consuming the signal (since that's still a 64-bit
check that would not be aliased in the wrap-around in g_signals),
so the biggest impact would be blocking on the futex until the next
full wakeup from a G1/G2 switch.
Signed-off-by: Frank Barrus <frankbarrus_sw@shaggy.cc>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
The new test elf/tst-rseq-tls-range-4096-static reliably detected
the extra TLS allocation problem (tcb_offset was dropped from
the allocation size) on aarch64. It also failed with a crash
in dlopen *before* the extra TLS changes, so TLS alignment with
static dlopen was already broken.
Reviewed-by: Michael Jeanson <mjeanson@efficios.com>
Use the same code to compute the TLS block size and its alignment.
The code in elf/dl-tls.c is linked in anyway for all binaries
due to the reference to _dl_tls_static_surplus_init.
It is not possible to call _dl_allocate_tls_storage directly
because malloc is not available in the static case. (The
dynamic linker uses the minimal malloc at this stage.) Therefore,
split _dl_tls_block_size_with_pre and _dl_tls_block_align from
_dl_allocate_tls_storage, and call those new functions from
__libc_setup_tls.
This fixes extra TLS allocation for the static case, and apparently
some pre-existing bugs as well (the independent recomputation of
TLS block sizes in init_static_tls looks rather suspect).
Fixes commit 0e411c5d30 ("Add generic
'extra TLS'").
The old code used the slotinfo array as a scratch area to pass the
list of TLS-using objects to _dl_determine_tlsoffset. All array
entries are subsequently overwritten by _dl_add_to_slotinfo,
except the first one. The link maps are usually not at their
right position for their module ID in the slotinfo array, so
the initial use of the slotinfo array would be incorrect if not
for scratch purposes only.
In _dl_tls_initial_modid_limit_setup, the old code relied that
some link map was written to the first slotinfo entry. After the
change, this no longer happens because TLS module ID zero is unused.
It's also necessary to move the call after the real initialization
of the slotinfo array.
This fixes an AArch64 build failure:
python3 -B ../sysdeps/aarch64/fpu/scripts/bench_libmvec_advsimd.py bench-float-advsimd-cospi > …/benchtests/bench-float-advsimd-cospi.c
Traceback (most recent call last):
File "…/sysdeps/aarch64/fpu/scripts/bench_libmvec_advsimd.py", line 106, in <module>
main(sys.argv[1])
~~~~^^^^^^^^^^^^^
File "…/sysdeps/aarch64/fpu/scripts/bench_libmvec_advsimd.py", line 81, in main
with open(f"../benchtests/libmvec/{input_filename}") as f:
~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '../benchtests/libmvec/cospif-inputs'
Careful updates of grnd_alloc.len are required to ensure that
after fork, grnd_alloc.states does not contain entries that
are also encountered by __getrandom_reset_state in TCBs.
For the same reason, it is necessary to overwrite the TCB state
pointer with NULL before updating grnd_alloc.states in
__getrandom_vdso_release.
Before this change, different TCBs could share the same getrandom
state after multi-threaded fork. This would be a critical security
bug (predictable randomness) if not caught during development.
The additional check in stdlib/tst-arc4random-thread makes it more
likely that the test fails due to the bugs mentioned above.
Both __getrandom_reset_state and __getrandom_vdso_release could
put reserved NULL pointers into the states array. This is also
fixed with this commit. After these changes, no null pointers were
observed in the states array during testing.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Some kernels on S390 appear to return a CPU affinity mask based on
configured processors rather than the ones online. Overallocate the CPU
set to match that, but operate only on the ones online.
Signed-off-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Co-authored-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
If the bit is not 0, the operations FRCHG and FSCHG are
undefined and cause a trap; qemu now checks for this as
well, so we set it to 0 temporarily and restore the old
value in getcontext afterwards (setcontext/swapcontext
already do so).
From the discussion in the bugreport, this can probably
be optimised in one place but none of the people involved
are SH4 assembly experts, this patch is field-tested, and
it’s not a code path run often. The other question, what
happens if a signal occurs while the bit is temporarily 0,
is also still unsolved, but to fix that a kernel change is
most likely needed; this patch changes a certain trap on
many CPUs for a hard-to-get trap in a signal handler if a
signal is delivered during the few instructions the PR bit
is temporarily set to 0, so it’s not a regression for most
users.
See BZ and https://bugs.launchpad.net/qemu/+bug/1796520 for
related discussion, references and review comments.
Signed-off-by: mirabilos <tg@debian.org>
Reviewed-by: Oleg Endo <olegendo@gcc.gnu.org>
Tested-by: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Adds commonly used IPv6 packet header macros similar to what is available
on NetBSD and FreeBSD in sys/netinet/ip6.h and Android in
libc/include/netinet/ip6.h
Usage example IPV6_VERSION_MASK and IPV6_VERSION:
if ((ip6->ip6_vfc & IPV6_VERSION_MASK) == IPV6_VERSION)
return true;
Usage example IPV6_FLOWINFO_MASK:
ip6->ip6_flow = (flow & IPV6_FLOWINFO_MASK);
The relevant standard is RFC2460 (Internet Protocol, Version 6
Specification). It defines the Internet Protocol version (IPV6_VERSION)
and reduced the size of the flow label field from 24 to 20 bits
(IPV6_FLOWLABEL_MASK). The traffic class and flow label fields together
make up the flow information (IPV6_FLOWINFO_MASK).
Tested on x86_64 GNU/Linux
Signed-off-by: Dan Luedtke <danrl@google.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
clang-19 shows:
scanf13.c:28:40: error: 'sscanf' may overflow; destination buffer in argument 4 has size 8, but the corresponding specifier may require size 11 [-Werror,-Wfortify-source]
28 | "A%ms%10ms%4m[bcd]%4mcB", &sp1, &sp2, &sp3, &sp4) != 4)
| ^
scanf13.c:94:34: error: 'sscanf' may overflow; destination buffer in argument 3 has size 8, but the corresponding specifier may require size 2049 [-Werror,-Wfortify-source]
94 | if (sscanf (buf, "%2048ms%mc", &sp3, &sp4) != 2)
| ^
scanf13.c:110:61: error: 'sscanf' may overflow; destination buffer in argument 4 has size 8, but the corresponding specifier may require size 1501 [-Werror,-Wfortify-source]
110 | if (sscanf (buf, "%4mc%1500m[dr/]%548m[abc/d]%3mc", &sp1, &sp2, &sp3, &sp4)
| ^
scanf13.c:110:67: error: 'sscanf' may overflow; destination buffer in argument 5 has size 8, but the corresponding specifier may require size 549 [-Werror,-Wfortify-source]
110 | if (sscanf (buf, "%4mc%1500m[dr/]%548m[abc/d]%3mc", &sp1, &sp2, &sp3, &sp4)
clang does have some support to handle 'm' prefix for -Wformat; but it
lacks support for -Wfortify to understand that it is up to libc to
allocate the memory, and uses the pointer size instead to calculate
validity.
The __ifunc_resolver macro expands to:
extern __typeof (__redirect_name) name __attribute__ ((ifunc ("iname_ifunc")));
static __typeof (__redirect_name) *name_ifunc (void) { [...] };
And although NAME_IFUNC is and alias for NAME, clang-18 still emits
an 'unused function 'name_ifunc' [-Werror,-Wunused-function]'
warning.
clang issues:
error: value size does not match register size specified by the
constraint and modifier [-Werror,-Wasm-operand-widths]
while tryng to use 32 bit variables with 'mrs' to get/set the
fpsr, dczid_el0, and ctr.
The Mach RPC host_get_uptime64() is implemented. It returns the elapsed time
value since bootup. See
https://git.savannah.gnu.org/cgit/hurd/gnumach.git/commit/?id=fc494bfe3fb6363e1077dc035eb119970d84a9d1
In this patch, the RPC is used to implement the monotonic clock for
mach.
* config.h.in: Add HAVE_HOST_GET_UPTIME64 config entry
* sysdeps/mach/clock_gettime.c: Add CLOCK_MONOTONIC case
* sysdeps/mach/configure: Check the existence of host_get_uptime64 RPC
* sysdeps/mach/configure.ac: Check the existence of host_get_uptime64 RPC
Message-ID: <20250106043907.1046-1-zhmingluo@163.com>
commit 494d65129e
Author: Michael Jeanson <mjeanson@efficios.com>
Date: Thu Aug 1 10:35:34 2024 -0400
nptl: Introduce <rseq-access.h> for RSEQ_* accessors
added things like
asm volatile ("movl %%fs:%P1(%q2),%0" \
: "=r" (__value) \
: "i" (offsetof (struct rseq_area, member)), \
"r" (__rseq_offset)); \
But this doesn't work for x32 when __rseq_offset is negative since the
address is computed as
FS + 32-bit to 64-bit zero extension of __rseq_offset
+ offsetof (struct rseq_area, member)
Cast __rseq_offset to long long int
"r" ((long long int) __rseq_offset)); \
to sign-extend 32-bit __rseq_offset to 64-bit. This is a no-op for x86-64
since x86-64 __rseq_offset is 64-bit. This fixes BZ #32543.
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
Reviewed-by: Florian Weimer <fweimer@redhat.com>
Sync the internal copy of '<sys/rseq.h>' with the latest Linux kernel
'include/uapi/linux/rseq.h'.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Reviewed-by: Florian Weimer <fweimer@redhat.com>
The rseq extensible ABI implementation moved the rseq area to the 'extra
TLS' block, remove the unused 'rseq_area' member of 'struct pthread'.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Reviewed-by: Florian Weimer <fweimer@redhat.com>
Move the rseq area to the newly added 'extra TLS' block, this is the
last step in adding support for the rseq extended ABI. The size of the
rseq area is now dynamic and depends on the rseq features reported by
the kernel through the elf auxiliary vector. This will allow
applications to use rseq features past the 32 bytes of the original rseq
ABI as they become available in future kernels.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Reviewed-by: Florian Weimer <fweimer@redhat.com>
In preparation to move the rseq area to the 'extra TLS' block, we need
accessors based on the thread pointer and the rseq offset. The ONCE
variant of the accessors ensures single-copy atomicity for loads and
stores which is required for all fields once the registration is active.
A separate header is required to allow including <atomic.h> which
results in an include loop when added to <tcb-access.h>.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Reviewed-by: Florian Weimer <fweimer@redhat.com>
This allows accessing the internal aliases of __rseq_size and
__rseq_offset from ld.so without ifdefs and avoids dynamic symbol
binding at run time for both variables.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Reviewed-by: Florian Weimer <fweimer@redhat.com>
Add the Linux implementation of 'extra TLS' which will allocate space
for the rseq area at the end of the TLS blocks in allocation order.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Reviewed-by: Florian Weimer <fweimer@redhat.com>
Add the logic to append an 'extra TLS' block in the TLS block allocator
with a generic stub implementation. The duplicated code in
'csu/libc-tls.c' and 'elf/dl-tls.c' is to handle both statically linked
applications and the ELF dynamic loader.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Reviewed-by: Florian Weimer <fweimer@redhat.com>
Get the rseq feature size and alignment requirement from the auxiliary
vector for use inside the dynamic loader. Use '__rseq_size' directly to
store the feature size. If the main thread registration fails or is
disabled by tunable, reset the value to 0.
This will be used in the TLS block allocator to compute the size and
alignment of the rseq area block for the extended ABI support.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Reviewed-by: Florian Weimer <fweimer@redhat.com>
Similar to a9944a52c9 and
f9493a15ea, we need to hide calloc use from
the compiler to accommodate GCC's r15-6566-g804e9d55d9e54c change.
First, include tst-malloc-aux.h, but then use `volatile` variables
for size.
The test passes without the tst-malloc-aux.h change but IMO we want
it there for consistency and to avoid future problems (possibly silent).
Reviewed-by: H.J. Lu <hjl.tools@gmail.com>
Clear libc_cv_cc_wimplicit_fallthrough if -Wimplicit-fallthrough isn't
supported. Tested with GCC 6.4.1 on x86-64.
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
Reviewed-by: Sam James <sam@gentoo.org>
