| Message ID | 20161123173650.1740-1-gael.portay@savoirfairelinux.com |
|---|---|
| State | Changes Requested |
| Headers | show |
Gaël, All, On 2016-11-23 12:36 -0500, Gaël PORTAY spake thusly: > Pseudo consists of: > * a client (cli), > * a server (daemon) and > * a database managed by this daemon > which makes a relation between inode, filename, type/mode... > > Without -f/-d argument, the client forks a process, sets LD_PRELOAD, > execve's the command given as argument and waits for its completion. > The LD_PRELOAD'ed library tries to connect to the daemon and spawns it > if it isn't running. > > Neither the client itself nor the LD_PRELOAD'ed library waits for the > daemon to terminate. As a consequence, the daemon may still be alive > after the completion of the command executed through the client. > > Under some circumstances (running build in a docker), the daemon (which > may be still alive after the build) is killed with SIGKILL, thus it > does not let it the time to properly sync its database. > > The database is rendered inconsistent for the next build. In this case, > makedevs will complain about the file type/mode, see error below. > > makedevs: line xx: node (...)/output/target/dev/console exists but is of wrong type > > Note that this error was introduced by c85cd189. > > Because the database did not have the time to sync (ie. /dev/console > record is missing), makedevs tests the existing and real /dev/console > file mode (created by the previous build) which is a regular file to > what should have been reported by pseudo, i.e. a character device. > Those two modes are different and makedevs exits with error. > > To solve this issue, this patch make the wrapper control the life of > the daemon. It spawns the server before running the script inside the > pseudo context, tells it to stop with "pseudo -S", and waits for the > server to exit before the wrapper exits. > > Signed-off-by: Gaël PORTAY <gael.portay@savoirfairelinux.com> > Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> > --- > package/pseudo/pseudo-wrapper | 17 +++++++++++++++-- > 1 file changed, 15 insertions(+), 2 deletions(-) > > diff --git a/package/pseudo/pseudo-wrapper b/package/pseudo/pseudo-wrapper > index 9c8dbdb..194c88f 100644 > --- a/package/pseudo/pseudo-wrapper > +++ b/package/pseudo/pseudo-wrapper > @@ -8,7 +8,6 @@ if [ "${0##*/}" = "fakeroot" ]; then > fi > > export PSEUDO_PREFIX="$(dirname "${0%/*}")" > -export PSEUDO_OPTS="-t0" > if [ -n "${TARGET_DIR}" ]; then > export PSEUDO_PASSWD="${TARGET_DIR}" > fi > @@ -16,4 +15,18 @@ if [ -n "${BASE_DIR}" ]; then > export PSEUDO_LOCALSTATEDIR="${BASE_DIR}/build/.pseudodb" > fi > > -exec "${0%/*}/pseudo" "${@}" > +# spawn server manually... > +"${0%/*}/pseudo" -f & > +pid="${!}" There still is a little corner case I'm not too happy about... The above returns immediately, but we don;t know if the daemon is ready. It may take a bit of time for it to start compeltely (i.e. reload the DB and repair its now-internal state)... > +# ... run script/command... > +"${0%/*}/pseudo" "${@}" But then we (almost) immediately start a client, which may try to connect to the daemon while it is not yet ready, and thus the client may not detect it yet, and thus wmay spawn his own daemon. And we'd be back to square one... Of course, we could run: pseudo -d which is guaranteed to return after the daemon is ready... > +rc="${?}" > + > +# ... terminate server... > +"${0%/*}/pseudo" -S > + > +# ... and wait for its completion to make sure database is synced before > +# returning. > +wait "${pid}" But then we would not have a PID to wait on... :-/ So, the question is: are we sure the client will talk to the daemon that we spawn? If the answer is "yes, we're sure", then all is good; id the answer is "no, we're not sure", then it's a problem... Thoughts? Now, my position on all this mess is : - we revert back to using fakeroot for 2016.11; - long term, we want to definitely switch to using pseudo: - it is actively maintained,, - it has more features than fakeroot, - some of those features are very interesting for us. Regards, Yann E. MORIN. > +exit "${rc}" > -- > 2.10.2 >
Hello, On Wed, 23 Nov 2016 19:28:43 +0100, Yann E. MORIN wrote: > - long term, we want to definitely switch to using pseudo: > - it is actively maintained,, > - it has more features than fakeroot, > - some of those features are very interesting for us. Could you explain which additional features of pseudo are interesting for us? Thanks, Thomas
Thomas, All, On 2016-11-23 21:57 +0100, Thomas Petazzoni spake thusly: > On Wed, 23 Nov 2016 19:28:43 +0100, Yann E. MORIN wrote: > > - long term, we want to definitely switch to using pseudo: > > - it is actively maintained,, > > - it has more features than fakeroot, > > - some of those features are very interesting for us. > > Could you explain which additional features of pseudo are interesting > for us? pseudo can intercept and redirect acceses to /etc/paswd (and group, shadow, gshadow) to the ones in the target. With that, we could probably greatly simplify our handling of users. Regards, Yann E. MORIN.
Hello, On Wed, 23 Nov 2016 22:34:03 +0100, Yann E. MORIN wrote: > On 2016-11-23 21:57 +0100, Thomas Petazzoni spake thusly: > > On Wed, 23 Nov 2016 19:28:43 +0100, Yann E. MORIN wrote: > > > - long term, we want to definitely switch to using pseudo: > > > - it is actively maintained,, > > > - it has more features than fakeroot, > > > - some of those features are very interesting for us. > > > > Could you explain which additional features of pseudo are interesting > > for us? > > pseudo can intercept and redirect acceses to /etc/paswd (and group, > shadow, gshadow) to the ones in the target. With that, we could probably > greatly simplify our handling of users. Is that the "additional features that are very interesting for us"? If it is, then I don't think it's that interesting. At least it definitely doesn't offset the drawback of having to build host-sqlite and host-attr. Our handling of users is not complicated, it's very explicit, works fine, and is actually probably better than some black magic that intercepts accesses to /etc/passwd. Thomas
On 23-11-16 19:28, Yann E. MORIN wrote: > Gaël, All, > > On 2016-11-23 12:36 -0500, Gaël PORTAY spake thusly: [snip] >> -exec "${0%/*}/pseudo" "${@}" >> +# spawn server manually... >> +"${0%/*}/pseudo" -f & >> +pid="${!}" > > There still is a little corner case I'm not too happy about... The above > returns immediately, but we don;t know if the daemon is ready. It may > take a bit of time for it to start compeltely (i.e. reload the DB and > repair its now-internal state)... > >> +# ... run script/command... >> +"${0%/*}/pseudo" "${@}" > > But then we (almost) immediately start a client, which may try to > connect to the daemon while it is not yet ready, and thus the client may > not detect it yet, and thus wmay spawn his own daemon. > > And we'd be back to square one... > > Of course, we could run: > > pseudo -d > > which is guaranteed to return after the daemon is ready... > >> +rc="${?}" >> + >> +# ... terminate server... >> +"${0%/*}/pseudo" -S >> + >> +# ... and wait for its completion to make sure database is synced before >> +# returning. >> +wait "${pid}" > > But then we would not have a PID to wait on... :-/ PID is not a problem, we can get it from LOCALSTATEDIR. The problem is that the wait builtin only works for direct children of the shell. > > So, the question is: are we sure the client will talk to the daemon that > we spawn? > If the answer is "yes, we're sure", then all is good; id the answer is > "no, we're not sure", then it's a problem... I don't think we're really sure, but I do believe that pseudo protects against this situation, i.e. when two servers start up concurrently, only one of them will really run. The problem is that the one that commits suicide may be the one that we start from the pseudo wrapper, so we end up wait'ing for the wrong pid. > > Thoughts? > > Now, my position on all this mess is : > > - we revert back to using fakeroot for 2016.11; Yeah, I think we need to do that. Too bad for the Fedora users. Regards, Arnout > > - long term, we want to definitely switch to using pseudo: > - it is actively maintained,, > - it has more features than fakeroot, > - some of those features are very interesting for us. > > Regards, > Yann E. MORIN. > >> +exit "${rc}" >> -- >> 2.10.2 >> >
Hi all,
This patch *should* be abandonned.
The issue has been fixed and mainlined by the Seebs (the maintainer of pseudo),
see commit d6eb2df3fe63b765f35d62332add4d0e4e9c6a39.
Now we can update the wrapper script to run:
exec "${0%/*}/pseudo" -S "${@}"
The -S argument tells server to shutdown and "waits" for the server to complete.
Seebs has picked up the solution proposed by Arnout. It uses the socket to
determine when the server has terminated. The socket is closed right before
exiting.
On Wed, Nov 23, 2016 at 10:47:26PM +0100, Arnout Vandecappelle wrote:
>
> >
> > So, the question is: are we sure the client will talk to the daemon that
> > we spawn?
> > If the answer is "yes, we're sure", then all is good; id the answer is
> > "no, we're not sure", then it's a problem...
>
> I don't think we're really sure, but I do believe that pseudo protects against
> this situation, i.e. when two servers start up concurrently, only one of them
> will really run. The problem is that the one that commits suicide may be the one
> that we start from the pseudo wrapper, so we end up wait'ing for the wrong pid.
There is a lock, thus only a single instance of a server could run.
> >
> > Thoughts?
> >
> > Now, my position on all this mess is :
> >
> > - we revert back to using fakeroot for 2016.11;
>
> Yeah, I think we need to do that. Too bad for the Fedora users.
Agree. Even if this issue can be easily fixed, it remains the issue reported by
Jerome.
Regards,
Gael
On 24-11-16 00:59, Gaël PORTAY wrote:
> This patch *should* be abandonned.
In such a situation, you should mark the patch as "Changes Requested" in
patchwork. You can do that yourself for your own patches.
I've now done that.
Regards,
Arnout
diff --git a/package/pseudo/pseudo-wrapper b/package/pseudo/pseudo-wrapper index 9c8dbdb..194c88f 100644 --- a/package/pseudo/pseudo-wrapper +++ b/package/pseudo/pseudo-wrapper @@ -8,7 +8,6 @@ if [ "${0##*/}" = "fakeroot" ]; then fi export PSEUDO_PREFIX="$(dirname "${0%/*}")" -export PSEUDO_OPTS="-t0" if [ -n "${TARGET_DIR}" ]; then export PSEUDO_PASSWD="${TARGET_DIR}" fi @@ -16,4 +15,18 @@ if [ -n "${BASE_DIR}" ]; then export PSEUDO_LOCALSTATEDIR="${BASE_DIR}/build/.pseudodb" fi -exec "${0%/*}/pseudo" "${@}" +# spawn server manually... +"${0%/*}/pseudo" -f & +pid="${!}" + +# ... run script/command... +"${0%/*}/pseudo" "${@}" +rc="${?}" + +# ... terminate server... +"${0%/*}/pseudo" -S + +# ... and wait for its completion to make sure database is synced before +# returning. +wait "${pid}" +exit "${rc}"