| Message ID | c9b7162513ee169da4b9e7e73d66288c083e4a7a.camel@gmail.com |
|---|---|
| State | Accepted |
| Headers | show |
| Series | package/musl: Make scheduler functionsLinux-compatible | expand |
On 14/05/2019 19:31, stefan.nickl@gmail.com wrote: > The POSIX functions sched_getscheduler(), sched_setscheduler(), > sched_getparam(), sched_setparam() are technically not correctly > implemented by the Linux syscalls of the same name, because what the > kernel calls a PID and what POSIX calls a PID isn't truly the same, > resulting in somewhat different semantics as to what these functions > exactly apply to. > Details: https://sourceware.org/bugzilla/show_bug.cgi?id=14829 > > Since the musl developers put a high premium on POSIX compliance, they > deliberately implement these functions to return -ENOSYS instead of > relaying them to the respective Linux syscalls as glibc/uClibc do. > > Unfortunally this breaks virtually all Linux programs using these > functions under musl. For example running 'chrt -p 1' fails with > 'Function not implemented' on a musl-libc based system. > In particular, it affects embedded systems using these interfaces > for scheduling real-time processes. > > As it seems unfeasible to fix all affected programs to manually use > syscall wrappers instead of the libc functions, make musl behave the > Linux way. As Thomas wrote, the patch as it is is not upstreamable since musl considers this a feature. However, it is possible that Rich can be convinced to actually read the POSIX spec and decide that linux behaviour is correct (except for the fact that it sched_setscheduler() should return the previous scheduling policy rather than 0, but that can be approximated (in a racy way) by calling sched_getscheduler() first). So it might be worth trying to push the patch upstream. > Signed-off-by: Stefan Nickl <Stefan.Nickl@gmail.com> > --- > ...e-scheduler-functions-Linux-compatib.patch | 76 +++++++++++++++++++ > 1 file changed, 76 insertions(+) > create mode 100644 package/musl/0002-package-musl-Make-scheduler-functions-Linux-compatib.patch > > diff --git a/package/musl/0002-package-musl-Make-scheduler-functions-Linux-compatib.patch b/package/musl/0002-package-musl-Make-scheduler-functions-Linux-compatib.patch > new file mode 100644 > index 0000000000..7c3acf9f02 > --- /dev/null > +++ b/package/musl/0002-package-musl-Make-scheduler-functions-Linux-compatib.patch > @@ -0,0 +1,76 @@ > +From 407c96fc790d0d11ca9603a2a533216c745b5051 Mon Sep 17 00:00:00 2001 > +From: Stefan Nickl <Stefan.Nickl@gmail.com> > +Date: Mon, 13 May 2019 22:33:21 +0200 > +Subject: [PATCH] Make scheduler functions Linux-compatible > + > +Let sched_getscheduler(), sched_setscheduler(), sched_getparam(), > +sched_setparam() invoke the Linux syscalls of the same name instead > +of returning -ENOSYS. ... but then you'd need to refer to https://pubs.opengroup.org/onlinepubs/7908799/xsh/sched_setparam.html and explain that in Linux, a pid_t can only refer to a process (because a thread's PID can only be obtained with gettid() or clone(), neither of which is available in musl), and that POSIX specifies that in a process with multiple threads, sched_setscheduler should affect only the main thread, which is exactly what Linux does. Either way (whether you upstream it or not), this patch looks good to me. Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Regards, Arnout > + > +Signed-off-by: Stefan Nickl <Stefan.Nickl@gmail.com> > +--- > + src/sched/sched_getparam.c | 3 +-- > + src/sched/sched_getscheduler.c | 3 +-- > + src/sched/sched_setparam.c | 3 +-- > + src/sched/sched_setscheduler.c | 3 +-- > + 4 files changed, 4 insertions(+), 8 deletions(-) > + > +diff --git a/src/sched/sched_getparam.c b/src/sched/sched_getparam.c > +index 76f10e4..65be107 100644 > +--- a/src/sched/sched_getparam.c > ++++ b/src/sched/sched_getparam.c > +@@ -1,8 +1,7 @@ > + #include <sched.h> > +-#include <errno.h> > + #include "syscall.h" > + > + int sched_getparam(pid_t pid, struct sched_param *param) > + { > +- return __syscall_ret(-ENOSYS); > ++ return syscall(SYS_sched_getparam, pid, param); > + } > +diff --git a/src/sched/sched_getscheduler.c b/src/sched/sched_getscheduler.c > +index 394e508..4c922f6 100644 > +--- a/src/sched/sched_getscheduler.c > ++++ b/src/sched/sched_getscheduler.c > +@@ -1,8 +1,7 @@ > + #include <sched.h> > +-#include <errno.h> > + #include "syscall.h" > + > + int sched_getscheduler(pid_t pid) > + { > +- return __syscall_ret(-ENOSYS); > ++ return syscall(SYS_sched_getscheduler, pid); > + } > +diff --git a/src/sched/sched_setparam.c b/src/sched/sched_setparam.c > +index 18623ee..f699faf 100644 > +--- a/src/sched/sched_setparam.c > ++++ b/src/sched/sched_setparam.c > +@@ -1,8 +1,7 @@ > + #include <sched.h> > +-#include <errno.h> > + #include "syscall.h" > + > + int sched_setparam(pid_t pid, const struct sched_param *param) > + { > +- return __syscall_ret(-ENOSYS); > ++ return syscall(SYS_sched_setparam, pid, param); > + } > +diff --git a/src/sched/sched_setscheduler.c b/src/sched/sched_setscheduler.c > +index 4435f21..e678221 100644 > +--- a/src/sched/sched_setscheduler.c > ++++ b/src/sched/sched_setscheduler.c > +@@ -1,8 +1,7 @@ > + #include <sched.h> > +-#include <errno.h> > + #include "syscall.h" > + > + int sched_setscheduler(pid_t pid, int sched, const struct sched_param *param) > + { > +- return __syscall_ret(-ENOSYS); > ++ return syscall(SYS_sched_setscheduler, pid, sched, param); > + } > +-- > +2.21.0 > + >
On Tue, 2019-05-14 at 20:04 +0200, Arnout Vandecappelle wrote: > As Thomas wrote, the patch as it is is not upstreamable since musl > considers > this a feature. > > However, it is possible that Rich can be convinced to actually read > the POSIX > spec and decide that linux behaviour is correct (except for the fact > that it > sched_setscheduler() should return the previous scheduling policy > rather than 0, > but that can be approximated (in a racy way) by calling > sched_getscheduler() > first). So it might be worth trying to push the patch upstream. Hi Arnout, I think upstreaming is out of the question since Rich has actually *undone* it before https://git.musl-libc.org/cgit/musl/commit/?id=1e21e78bf7a5 Also asked on #musl a couple days ago whether they might be willing to at least #ifdef it, and only got suggestions for application side fixes. So, no dice it seems. Cheers, Stefan
On 14/05/2019 20:54, stefan.nickl@gmail.com wrote: > On Tue, 2019-05-14 at 20:04 +0200, Arnout Vandecappelle wrote: >> As Thomas wrote, the patch as it is is not upstreamable since musl >> considers >> this a feature. >> >> However, it is possible that Rich can be convinced to actually read >> the POSIX >> spec and decide that linux behaviour is correct (except for the fact >> that it >> sched_setscheduler() should return the previous scheduling policy >> rather than 0, >> but that can be approximated (in a racy way) by calling >> sched_getscheduler() >> first). So it might be worth trying to push the patch upstream. > > Hi Arnout, > > I think upstreaming is out of the question since Rich has actually > *undone* it before > https://git.musl-libc.org/cgit/musl/commit/?id=1e21e78bf7a5 > > Also asked on #musl a couple days ago whether they might be willing to > at least #ifdef it, and only got suggestions for application side > fixes. So, no dice it seems. Yes, but has the argument "Linux does in fact implement POSIX" been used already? Regards, Arnout
On Tue, 2019-05-14 at 21:38 +0200, Arnout Vandecappelle wrote: > > On 14/05/2019 20:54, stefan.nickl@gmail.com wrote: > > On Tue, 2019-05-14 at 20:04 +0200, Arnout Vandecappelle wrote: > > > As Thomas wrote, the patch as it is is not upstreamable since > > > musl > > > considers > > > this a feature. > > > > > > However, it is possible that Rich can be convinced to actually > > > read > > > the POSIX > > > spec and decide that linux behaviour is correct (except for the > > > fact > > > that it > > > sched_setscheduler() should return the previous scheduling policy > > > rather than 0, > > > but that can be approximated (in a racy way) by calling > > > sched_getscheduler() > > > first). So it might be worth trying to push the patch upstream. > > > > Hi Arnout, > > > > I think upstreaming is out of the question since Rich has actually > > *undone* it before > > https://git.musl-libc.org/cgit/musl/commit/?id=1e21e78bf7a5 > > > > Also asked on #musl a couple days ago whether they might be willing > > to > > at least #ifdef it, and only got suggestions for application side > > fixes. So, no dice it seems. > > Yes, but has the argument "Linux does in fact implement POSIX" been > used already? I haven't used that argument and I'm not aware of anyone that has. This comes pretty close though: https://github.com/MusicPlayerDaemon/MPD/issues/218 And just recently Rich managed to convince even the glibc developers of the opposite, so I certainly don't feel up to arguing that position ;-) https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=c70824b9a4645c0ecd049da8cfdb2c28ae7ada23 Cheers, Stefan
Stefan, All, On 2019-05-14 22:03 +0200, stefan.nickl@gmail.com spake thusly: > On Tue, 2019-05-14 at 21:38 +0200, Arnout Vandecappelle wrote: > > > > On 14/05/2019 20:54, stefan.nickl@gmail.com wrote: > > > On Tue, 2019-05-14 at 20:04 +0200, Arnout Vandecappelle wrote: > > > > As Thomas wrote, the patch as it is is not upstreamable since > > > > musl > > > > considers > > > > this a feature. > > > > > > > > However, it is possible that Rich can be convinced to actually > > > > read > > > > the POSIX > > > > spec and decide that linux behaviour is correct (except for the > > > > fact > > > > that it > > > > sched_setscheduler() should return the previous scheduling policy > > > > rather than 0, > > > > but that can be approximated (in a racy way) by calling > > > > sched_getscheduler() > > > > first). So it might be worth trying to push the patch upstream. > > > > > > Hi Arnout, > > > > > > I think upstreaming is out of the question since Rich has actually > > > *undone* it before > > > https://git.musl-libc.org/cgit/musl/commit/?id=1e21e78bf7a5 > > > > > > Also asked on #musl a couple days ago whether they might be willing > > > to > > > at least #ifdef it, and only got suggestions for application side > > > fixes. So, no dice it seems. > > > > Yes, but has the argument "Linux does in fact implement POSIX" been > > used already? > > I haven't used that argument and I'm not aware of anyone that has. > > This comes pretty close though: > https://github.com/MusicPlayerDaemon/MPD/issues/218 > > And just recently Rich managed to convince even the glibc developers of > the opposite, so I certainly don't feel up to arguing that position ;-) > https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=c70824b9a4645c0ecd049da8cfdb2c28ae7ada23 So, if glibc is going the musl way, because Linux does not allow to properly implement the sched_* stuff, when why do we even attempt to implement them in musl at all? Yes, I read the argument that "applications are broken without that", but they are already brioken anyway because the underlyign syscalls are unsound (from what I read in the various threads and the bugzilla). As Thomas said, if we add this patch, we'll have to carry it ad libitum, because there are absolutely zero chance this is upstreamed one day. Besides, it also does not address the case for pre-built toolchains, which leaves users out in the cold with broken applications anway. So, as painful as it might be, I'd say no to this patch. Regards, Yann E. MORIN.
On Tue, 2019-05-14 at 22:24 +0200, Yann E. MORIN wrote: > Stefan, All, > > On 2019-05-14 22:03 +0200, stefan.nickl@gmail.com spake thusly: > > On Tue, 2019-05-14 at 21:38 +0200, Arnout Vandecappelle wrote: > > > On 14/05/2019 20:54, stefan.nickl@gmail.com wrote: > > > > On Tue, 2019-05-14 at 20:04 +0200, Arnout Vandecappelle wrote: > > > > > As Thomas wrote, the patch as it is is not upstreamable > > > > > since > > > > > musl > > > > > considers > > > > > this a feature. > > > > > > > > > > However, it is possible that Rich can be convinced to > > > > > actually > > > > > read > > > > > the POSIX > > > > > spec and decide that linux behaviour is correct (except for > > > > > the > > > > > fact > > > > > that it > > > > > sched_setscheduler() should return the previous scheduling > > > > > policy > > > > > rather than 0, > > > > > but that can be approximated (in a racy way) by calling > > > > > sched_getscheduler() > > > > > first). So it might be worth trying to push the patch > > > > > upstream. > > > > > > > > Hi Arnout, > > > > > > > > I think upstreaming is out of the question since Rich has > > > > actually > > > > *undone* it before > > > > https://git.musl-libc.org/cgit/musl/commit/?id=1e21e78bf7a5 > > > > > > > > Also asked on #musl a couple days ago whether they might be > > > > willing > > > > to > > > > at least #ifdef it, and only got suggestions for application > > > > side > > > > fixes. So, no dice it seems. > > > > > > Yes, but has the argument "Linux does in fact implement POSIX" > > > been > > > used already? > > > > I haven't used that argument and I'm not aware of anyone that has. > > > > This comes pretty close though: > > https://github.com/MusicPlayerDaemon/MPD/issues/218 > > > > And just recently Rich managed to convince even the glibc > > developers of > > the opposite, so I certainly don't feel up to arguing that position > > ;-) > > https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=c70824b9a4645c0ecd049da8cfdb2c28ae7ada23 > > So, if glibc is going the musl way, because Linux does not allow to > properly implement the sched_* stuff, when why do we even attempt to > implement them in musl at all? Hi Yann, that's not what I'm seeing. The glibc folks are not going to change the library call <-> Linux syscall correspondence. They merely make note of the fact that they've screwed up in the past, and that now it is too late to change. All that happens is that the non-conformance gets documented. What makes this extra confusing is that there are source files in the glibc tree with -ENOSYS-returning implementations of these functions, but it seems the functions that really get used for Linux are generated from a syscalls.list file to use the syscall. Maybe someone more familiar with glibc can confirm this. In the uclibc-ng case, it seems rather straightforward, see e.g. libc/sysdeps/linux/common/sched_setscheduler.c Cheers, Stefan
diff --git a/package/musl/0002-package-musl-Make-scheduler-functions-Linux-compatib.patch b/package/musl/0002-package-musl-Make-scheduler-functions-Linux-compatib.patch new file mode 100644 index 0000000000..7c3acf9f02 --- /dev/null +++ b/package/musl/0002-package-musl-Make-scheduler-functions-Linux-compatib.patch @@ -0,0 +1,76 @@ +From 407c96fc790d0d11ca9603a2a533216c745b5051 Mon Sep 17 00:00:00 2001 +From: Stefan Nickl <Stefan.Nickl@gmail.com> +Date: Mon, 13 May 2019 22:33:21 +0200 +Subject: [PATCH] Make scheduler functions Linux-compatible + +Let sched_getscheduler(), sched_setscheduler(), sched_getparam(), +sched_setparam() invoke the Linux syscalls of the same name instead +of returning -ENOSYS. + +Signed-off-by: Stefan Nickl <Stefan.Nickl@gmail.com> +--- + src/sched/sched_getparam.c | 3 +-- + src/sched/sched_getscheduler.c | 3 +-- + src/sched/sched_setparam.c | 3 +-- + src/sched/sched_setscheduler.c | 3 +-- + 4 files changed, 4 insertions(+), 8 deletions(-) + +diff --git a/src/sched/sched_getparam.c b/src/sched/sched_getparam.c +index 76f10e4..65be107 100644 +--- a/src/sched/sched_getparam.c ++++ b/src/sched/sched_getparam.c +@@ -1,8 +1,7 @@ + #include <sched.h> +-#include <errno.h> + #include "syscall.h" + + int sched_getparam(pid_t pid, struct sched_param *param) + { +- return __syscall_ret(-ENOSYS); ++ return syscall(SYS_sched_getparam, pid, param); + } +diff --git a/src/sched/sched_getscheduler.c b/src/sched/sched_getscheduler.c +index 394e508..4c922f6 100644 +--- a/src/sched/sched_getscheduler.c ++++ b/src/sched/sched_getscheduler.c +@@ -1,8 +1,7 @@ + #include <sched.h> +-#include <errno.h> + #include "syscall.h" + + int sched_getscheduler(pid_t pid) + { +- return __syscall_ret(-ENOSYS); ++ return syscall(SYS_sched_getscheduler, pid); + } +diff --git a/src/sched/sched_setparam.c b/src/sched/sched_setparam.c +index 18623ee..f699faf 100644 +--- a/src/sched/sched_setparam.c ++++ b/src/sched/sched_setparam.c +@@ -1,8 +1,7 @@ + #include <sched.h> +-#include <errno.h> + #include "syscall.h" + + int sched_setparam(pid_t pid, const struct sched_param *param) + { +- return __syscall_ret(-ENOSYS); ++ return syscall(SYS_sched_setparam, pid, param); + } +diff --git a/src/sched/sched_setscheduler.c b/src/sched/sched_setscheduler.c +index 4435f21..e678221 100644 +--- a/src/sched/sched_setscheduler.c ++++ b/src/sched/sched_setscheduler.c +@@ -1,8 +1,7 @@ + #include <sched.h> +-#include <errno.h> + #include "syscall.h" + + int sched_setscheduler(pid_t pid, int sched, const struct sched_param *param) + { +- return __syscall_ret(-ENOSYS); ++ return syscall(SYS_sched_setscheduler, pid, sched, param); + } +-- +2.21.0 +
The POSIX functions sched_getscheduler(), sched_setscheduler(), sched_getparam(), sched_setparam() are technically not correctly implemented by the Linux syscalls of the same name, because what the kernel calls a PID and what POSIX calls a PID isn't truly the same, resulting in somewhat different semantics as to what these functions exactly apply to. Details: https://sourceware.org/bugzilla/show_bug.cgi?id=14829 Since the musl developers put a high premium on POSIX compliance, they deliberately implement these functions to return -ENOSYS instead of relaying them to the respective Linux syscalls as glibc/uClibc do. Unfortunally this breaks virtually all Linux programs using these functions under musl. For example running 'chrt -p 1' fails with 'Function not implemented' on a musl-libc based system. In particular, it affects embedded systems using these interfaces for scheduling real-time processes. As it seems unfeasible to fix all affected programs to manually use syscall wrappers instead of the libc functions, make musl behave the Linux way. Signed-off-by: Stefan Nickl <Stefan.Nickl@gmail.com> --- ...e-scheduler-functions-Linux-compatib.patch | 76 +++++++++++++++++++ 1 file changed, 76 insertions(+) create mode 100644 package/musl/0002-package-musl-Make-scheduler-functions-Linux-compatib.patch