make fseek detect and produce an error for invalid whence arguments

this is a POSIX requirement. we previously relied on the underlying fd
(or other backend) seek operation to produce the error, but since
linux lseek now supports other seek modes (SEEK_DATA and SEEK_HOLE)
which do not interact well with stdio buffering, this is insufficient.
instead, explicitly check whence before performing any operations.
This commit is contained in:
Rich Felker 2022-01-09 00:33:56 -05:00
parent cbacd638e3
commit 52f0deb969

View file

@ -1,7 +1,14 @@
#include "stdio_impl.h"
#include <errno.h>
int __fseeko_unlocked(FILE *f, off_t off, int whence)
{
/* Fail immediately for invalid whence argument. */
if (whence != SEEK_CUR && whence != SEEK_SET && whence != SEEK_END) {
errno = EINVAL;
return -1;
}
/* Adjust relative offset for unread data in buffer, if any. */
if (whence == SEEK_CUR && f->rend) off -= f->rend - f->rpos;