prlimit: Translate old_rlimit from RLIM64_INFINITY to RLIM_INFINITY [BZ #22678]

prlimit called without a new value fails on 32-bit machines if any of
the soft or hard limits are infinity. This is because prlimit does not
translate old_rlimit from RLIM64_INFINITY to RLIM_INFINITY, but checks
that the value returned by the prlimit64 syscall fits into a 32-bit
value, like it is done for example in getrlimit. Note that on the
other hand new_rlimit is correctly translated from RLIM_INFINITY to
RLIM64_INFINITY before calling the syscall.

This patch fixes that.

Changelog:
	[BZ #22678]
	* sysdeps/unix/sysv/linux/prlimit.c (prlimit): Translate
	old_rlimit from RLIM64_INFINITY to RLIM_INFINITY.
This commit is contained in:
Aurelien Jarno 2018-01-05 20:34:10 +01:00
parent 8e900969db
commit 24731685c8
2 changed files with 15 additions and 6 deletions

View file

@ -1,3 +1,9 @@
2018-01-05 Aurelien Jarno <aurelien@aurel32.net>
[BZ #22678]
* sysdeps/unix/sysv/linux/prlimit.c (prlimit): Translate
old_rlimit from RLIM64_INFINITY to RLIM_INFINITY.
2018-01-05 Aurelien Jarno <aurelien@aurel32.net> 2018-01-05 Aurelien Jarno <aurelien@aurel32.net>
Adhemerval Zanella <adhemerval.zanella@linaro.org> Adhemerval Zanella <adhemerval.zanella@linaro.org>

View file

@ -50,21 +50,24 @@ prlimit (__pid_t pid, enum __rlimit_resource resource,
{ {
/* The prlimit64 syscall is ill-designed for 32-bit machines. /* The prlimit64 syscall is ill-designed for 32-bit machines.
We have to provide a 32-bit variant since otherwise the LFS We have to provide a 32-bit variant since otherwise the LFS
system would not work. But what shall we do if the syscall system would not work. The infinity value can be translated,
succeeds but the old values do not fit into a rlimit but otherwise what shall we do if the syscall succeeds but the
structure? We cannot return an error because the operation old values do not fit into a rlimit structure? We cannot return
itself worked. Best is perhaps to return RLIM_INFINITY. */ an error because the operation itself worked. Best is perhaps
to return RLIM_INFINITY. */
old_rlimit->rlim_cur = old_rlimit64_mem.rlim_cur; old_rlimit->rlim_cur = old_rlimit64_mem.rlim_cur;
if (old_rlimit->rlim_cur != old_rlimit64_mem.rlim_cur) if (old_rlimit->rlim_cur != old_rlimit64_mem.rlim_cur)
{ {
if (new_rlimit == NULL) if ((new_rlimit == NULL)
&& (old_rlimit64_mem.rlim_cur != RLIM64_INFINITY))
return INLINE_SYSCALL_ERROR_RETURN_VALUE (EOVERFLOW); return INLINE_SYSCALL_ERROR_RETURN_VALUE (EOVERFLOW);
old_rlimit->rlim_cur = RLIM_INFINITY; old_rlimit->rlim_cur = RLIM_INFINITY;
} }
old_rlimit->rlim_max = old_rlimit64_mem.rlim_max; old_rlimit->rlim_max = old_rlimit64_mem.rlim_max;
if (old_rlimit->rlim_max != old_rlimit64_mem.rlim_max) if (old_rlimit->rlim_max != old_rlimit64_mem.rlim_max)
{ {
if (new_rlimit == NULL) if ((new_rlimit == NULL)
&& (old_rlimit64_mem.rlim_max != RLIM64_INFINITY))
return INLINE_SYSCALL_ERROR_RETURN_VALUE (EOVERFLOW); return INLINE_SYSCALL_ERROR_RETURN_VALUE (EOVERFLOW);
old_rlimit->rlim_max = RLIM_INFINITY; old_rlimit->rlim_max = RLIM_INFINITY;
} }