[1/2] package/systemd: set machine id file instalation as optional choice
diff mbox series

Message ID 20191117113345.159653-1-b.bilas@grinn-global.com
State New
Headers show
Series
  • [1/2] package/systemd: set machine id file instalation as optional choice
Related show

Commit Message

Bartosz Bilas Nov. 17, 2019, 11:33 a.m. UTC
In case of e.g writable systems there is no neccesity to have
pre-installed empty machine-id file on target because systemd
generates that automatically during system boot.
Also in case of having empty machine-id file we are not able
to use a service with ConditionFirstBoot because systemd recognizes
machine-id file in system therefore we can't detect new system instance boots up.
Set this option as enable by default to keep compatibility with old builds.

Signed-off-by: Bartosz Bilas <b.bilas@grinn-global.com>
---
 package/systemd/Config.in  | 13 +++++++++++++
 package/systemd/systemd.mk |  2 ++
 2 files changed, 15 insertions(+)

Comments

Arnout Vandecappelle Nov. 17, 2019, 1:39 p.m. UTC | #1
Hi Bartosz,

On 17/11/2019 12:33, Bartosz Bilas wrote:
> In case of e.g writable systems there is no neccesity to have
                 writeable                    necessity

> pre-installed empty machine-id file on target because systemd
> generates that automatically during system boot.
> Also in case of having empty machine-id file we are not able
> to use a service with ConditionFirstBoot because systemd recognizes
> machine-id file in system therefore we can't detect new system instance boots up.
> Set this option as enable by default to keep compatibility with old builds.

 Nice that you take a look at this! I'm immediately going to follow it up with
more feature requests though :-)

- If there is a writeable persistent partition, but /etc is not writeable, can
we create a symlink and will systemd still create it correctly at first boot?

- Maybe if BR2_TARGET_GENERIC_REMOUNT_ROOTFS_RW is set, we should default it or
even force it to no.


 Regards,
 Arnout

> 
> Signed-off-by: Bartosz Bilas <b.bilas@grinn-global.com>
> ---
>  package/systemd/Config.in  | 13 +++++++++++++
>  package/systemd/systemd.mk |  2 ++
>  2 files changed, 15 insertions(+)
> 
> diff --git a/package/systemd/Config.in b/package/systemd/Config.in
> index aef39abe27..fadc1a32c8 100644
> --- a/package/systemd/Config.in
> +++ b/package/systemd/Config.in
> @@ -112,6 +112,19 @@ config BR2_PACKAGE_SYSTEMD_BOOT_EFI_ARCH
>  	default "x64"   if BR2_x86_64
>  	depends on BR2_PACKAGE_SYSTEMD_BOOT
>  
> +config BR2_PACKAGE_SYSTEMD_MACHINEID_FILE
> +	bool "Install empty machine id file"
> +	default y
> +	help
> +	  The /etc/machine-id file contains the unique machine ID
> +	  of the local system that is set during installation or
> +	  boot. The machine ID is a single newline-terminated,
> +	  hexadecimal, 32-character, lowercase ID. When decoded from
> +	  hexadecimal, this corresponds to a 16-byte/128-bit value.
> +	  This ID may not be all zeros.
> +
> +	  https://www.freedesktop.org/software/systemd/man/machine-id.html
> +
>  config BR2_PACKAGE_SYSTEMD_JOURNAL_GATEWAY
>  	bool "HTTP server for journal events"
>  	select BR2_PACKAGE_LIBMICROHTTPD
> diff --git a/package/systemd/systemd.mk b/package/systemd/systemd.mk
> index 92490eb86b..fc348fe120 100644
> --- a/package/systemd/systemd.mk
> +++ b/package/systemd/systemd.mk
> @@ -461,9 +461,11 @@ define SYSTEMD_INSTALL_INIT_HOOK
>  		$(TARGET_DIR)/etc/systemd/system/multi-user.target.wants/remote-fs.target
>  endef
>  
> +ifeq ($(BR2_PACKAGE_SYSTEMD_MACHINEID_FILE),y)
>  define SYSTEMD_INSTALL_MACHINEID_HOOK
>  	touch $(TARGET_DIR)/etc/machine-id
>  endef
> +endif
>  
>  SYSTEMD_POST_INSTALL_TARGET_HOOKS += \
>  	SYSTEMD_INSTALL_TARGET_CRYPTSETUP \
>
Jérémy ROSEN Nov. 17, 2019, 2:19 p.m. UTC | #2
hmmm, I'm kinda torn on that one..

The only difference I see betweed no machine-id and an empty machine-id is
systemd-firstboot and ConditionFirstBoot

if the file is empty or missing and systemd can write to it, systemd will
set it up (not sure what happens for Arnout's symlink, that's worth testing)
however, if the file exists but is empty we will skip systemd-firstboot

for context, systemd-firstboot does the following:
* set (unconditionnaly) a machine-id
* sets the locale (LANG= LC_MESSGES)
* sets the hostname
* sets the timezone
* sets the root password

systemd-firstboot can use values from its command line, but when launched
on the target, the point is usually to prompt the user for answers.
(as a side note, we can call it with --root=$DESTDIR once we have
host-systemd)

So I would tend to consider systemd-firstboot as "an uncommon case" because
prompting user on first boot is not something we see often in the embedded
space

The good news is: we already know when systemd-firstboot is used because
it's a config option of the systemd package.

ConditionFirstBoot is a bit more useful for us.. it allows to easily run a
service only at first boot (or skip it on first boot)

we also have a big clue if the system is meant to be read-only via the
"remount RW" option

So, putting it all together
* if we use systemd-firstboot, we don't want a machine-id
* if we have a read-only rootfs, we want an empty machine-id
* in other cases, we can deal with both, but it's better to have no
machine-id
* I don't think that the concept of first boot makes any sense with a
read-only rootfs...

Could something like that be enough ? can we trust "remount RW" ?
maybe "remount RW" should be renamed "create a RW filesystem" and enable
various tweaks related to RO vs RW

my two cents...


Le dim. 17 nov. 2019 à 14:39, Arnout Vandecappelle <arnout@mind.be> a
écrit :

>  Hi Bartosz,
>
> On 17/11/2019 12:33, Bartosz Bilas wrote:
> > In case of e.g writable systems there is no neccesity to have
>                  writeable                    necessity
>
> > pre-installed empty machine-id file on target because systemd
> > generates that automatically during system boot.
> > Also in case of having empty machine-id file we are not able
> > to use a service with ConditionFirstBoot because systemd recognizes
> > machine-id file in system therefore we can't detect new system instance
> boots up.
> > Set this option as enable by default to keep compatibility with old
> builds.
>
>  Nice that you take a look at this! I'm immediately going to follow it up
> with
> more feature requests though :-)
>
> - If there is a writeable persistent partition, but /etc is not writeable,
> can
> we create a symlink and will systemd still create it correctly at first
> boot?
>
> - Maybe if BR2_TARGET_GENERIC_REMOUNT_ROOTFS_RW is set, we should default
> it or
> even force it to no.
>
>
>  Regards,
>  Arnout
>
> >
> > Signed-off-by: Bartosz Bilas <b.bilas@grinn-global.com>
> > ---
> >  package/systemd/Config.in  | 13 +++++++++++++
> >  package/systemd/systemd.mk |  2 ++
> >  2 files changed, 15 insertions(+)
> >
> > diff --git a/package/systemd/Config.in b/package/systemd/Config.in
> > index aef39abe27..fadc1a32c8 100644
> > --- a/package/systemd/Config.in
> > +++ b/package/systemd/Config.in
> > @@ -112,6 +112,19 @@ config BR2_PACKAGE_SYSTEMD_BOOT_EFI_ARCH
> >       default "x64"   if BR2_x86_64
> >       depends on BR2_PACKAGE_SYSTEMD_BOOT
> >
> > +config BR2_PACKAGE_SYSTEMD_MACHINEID_FILE
> > +     bool "Install empty machine id file"
> > +     default y
> > +     help
> > +       The /etc/machine-id file contains the unique machine ID
> > +       of the local system that is set during installation or
> > +       boot. The machine ID is a single newline-terminated,
> > +       hexadecimal, 32-character, lowercase ID. When decoded from
> > +       hexadecimal, this corresponds to a 16-byte/128-bit value.
> > +       This ID may not be all zeros.
> > +
> > +       https://www.freedesktop.org/software/systemd/man/machine-id.html
> > +
> >  config BR2_PACKAGE_SYSTEMD_JOURNAL_GATEWAY
> >       bool "HTTP server for journal events"
> >       select BR2_PACKAGE_LIBMICROHTTPD
> > diff --git a/package/systemd/systemd.mk b/package/systemd/systemd.mk
> > index 92490eb86b..fc348fe120 100644
> > --- a/package/systemd/systemd.mk
> > +++ b/package/systemd/systemd.mk
> > @@ -461,9 +461,11 @@ define SYSTEMD_INSTALL_INIT_HOOK
> >
>  $(TARGET_DIR)/etc/systemd/system/multi-user.target.wants/remote-fs.target
> >  endef
> >
> > +ifeq ($(BR2_PACKAGE_SYSTEMD_MACHINEID_FILE),y)
> >  define SYSTEMD_INSTALL_MACHINEID_HOOK
> >       touch $(TARGET_DIR)/etc/machine-id
> >  endef
> > +endif
> >
> >  SYSTEMD_POST_INSTALL_TARGET_HOOKS += \
> >       SYSTEMD_INSTALL_TARGET_CRYPTSETUP \
> >
> _______________________________________________
> buildroot mailing list
> buildroot@busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
>
Arnout Vandecappelle Nov. 17, 2019, 4 p.m. UTC | #3
On 17/11/2019 15:19, Jérémy ROSEN wrote:
> hmmm, I'm kinda torn on that one..
> 
> The only difference I see betweed no machine-id and an empty machine-id is
> systemd-firstboot and ConditionFirstBoot

 That's exactly the point I think.

 I assumed, however, that ConditionFirstBoot was evaluated by systemd-firstboot,
but that's not the case then?


> if the file is empty or missing and systemd can write to it, systemd will set it
> up (not sure what happens for Arnout's symlink, that's worth testing)
> however, if the file exists but is empty we will skip systemd-firstboot
> 
> for context, systemd-firstboot does the following:
> * set (unconditionnaly) a machine-id

 Are you saying that if systemd-firstboot it not enabled, no machine-id will be
generated?

> * sets the locale (LANG= LC_MESSGES)
> * sets the hostname
> * sets the timezone
> * sets the root password
> 
> systemd-firstboot can use values from its command line, but when launched on the
> target, the point is usually to prompt the user for answers.
> (as a side note, we can call it with --root=$DESTDIR once we have host-systemd)
> 
> So I would tend to consider systemd-firstboot as "an uncommon case" because
> prompting user on first boot is not something we see often in the embedded space
> 
> The good news is: we already know when systemd-firstboot is used because it's a
> config option of the systemd package.
> 
> ConditionFirstBoot is a bit more useful for us.. it allows to easily run a
> service only at first boot (or skip it on first boot)

 Well, here are a few things you'd typically want to generate on first boot:

- machine id;
- ssh keys;
- various certificates;
- maybe a hashed root password based on a serial number of MAC address;
- maybe a random or hashed hostname;
- register on the phone-home server;
- resize the data partition based on actual size of the eMMC.

 None of them interactively, of course.

 So apparently systemd-firstboot can only do the first one. That's indeed not
very useful.


> we also have a big clue if the system is meant to be read-only via the "remount
> RW" option

 Yeah, but it's just a clue. You can have a writeable machine ID without a
remount RW option in a number of situations:

- custom skeleton
- fstab is overwritten by fs-overlay
- /etc is (bind)-mounted to a writeable partition
- /etc/machine-id is symlinked into a writeable partition
- root is squashfs (so it's not writeable even if RW is on)


> So, putting it all together
> * if we use systemd-firstboot, we don't want a machine-id

 ... at least, not an empty one :-)

> * if we have a read-only rootfs, we want an empty machine-id
> * in other cases, we can deal with both, but it's better to have no machine-id
> * I don't think that the concept of first boot makes any sense with a read-only
> rootfs...

 With readonly *rootfs* it certainly does, but not without any persistent
storage at all.

> 
> Could something like that be enough ? can we trust "remount RW" ?
> maybe "remount RW" should be renamed "create a RW filesystem" and enable various
> tweaks related to RO vs RW

 As written above: no.


 The problem is: we're not a distro. We leave too much freedom for the user to
integrate things in various ways to be able to make assumptions about what is
the right way to do things. So, the only thing we can do is to give a decent
out-of-the-box experience, and let the user figure out how to tweak things -
possibly adding a config option for a common situation that is easily handled in
a generic way. The other thing we can do is to provide documentation about the
proper way to integrate things in different scenarios.

 I'm starting to agree that this option is maybe not that great.

 Actually, come to think of it, maybe it's better to never create the machine-id
at all. What is the effect (with a readonly rootfs) if machine-id doesn't exist?
Does it block booting, does the boot take a little longer, or do we boot with a
different machine-id all the time? And in the latter case, is there a real
problem with that?

 Regards,
 Arnout

> 
> my two cents...
> 
> 
> Le dim. 17 nov. 2019 à 14:39, Arnout Vandecappelle <arnout@mind.be
> <mailto:arnout@mind.be>> a écrit :
> 
>      Hi Bartosz,
> 
>     On 17/11/2019 12:33, Bartosz Bilas wrote:
>     > In case of e.g writable systems there is no neccesity to have
>                      writeable                    necessity
> 
>     > pre-installed empty machine-id file on target because systemd
>     > generates that automatically during system boot.
>     > Also in case of having empty machine-id file we are not able
>     > to use a service with ConditionFirstBoot because systemd recognizes
>     > machine-id file in system therefore we can't detect new system instance
>     boots up.
>     > Set this option as enable by default to keep compatibility with old builds.
> 
>      Nice that you take a look at this! I'm immediately going to follow it up with
>     more feature requests though :-)
> 
>     - If there is a writeable persistent partition, but /etc is not writeable, can
>     we create a symlink and will systemd still create it correctly at first boot?
> 
>     - Maybe if BR2_TARGET_GENERIC_REMOUNT_ROOTFS_RW is set, we should default it or
>     even force it to no.
> 
> 
>      Regards,
>      Arnout
> 
>     >
>     > Signed-off-by: Bartosz Bilas <b.bilas@grinn-global.com
>     <mailto:b.bilas@grinn-global.com>>
>     > ---
>     >  package/systemd/Config.in  | 13 +++++++++++++
>     >  package/systemd/systemd.mk <http://systemd.mk> |  2 ++
>     >  2 files changed, 15 insertions(+)
>     >
>     > diff --git a/package/systemd/Config.in b/package/systemd/Config.in
>     > index aef39abe27..fadc1a32c8 100644
>     > --- a/package/systemd/Config.in
>     > +++ b/package/systemd/Config.in
>     > @@ -112,6 +112,19 @@ config BR2_PACKAGE_SYSTEMD_BOOT_EFI_ARCH
>     >       default "x64"   if BR2_x86_64
>     >       depends on BR2_PACKAGE_SYSTEMD_BOOT
>     > 
>     > +config BR2_PACKAGE_SYSTEMD_MACHINEID_FILE
>     > +     bool "Install empty machine id file"
>     > +     default y
>     > +     help
>     > +       The /etc/machine-id file contains the unique machine ID
>     > +       of the local system that is set during installation or
>     > +       boot. The machine ID is a single newline-terminated,
>     > +       hexadecimal, 32-character, lowercase ID. When decoded from
>     > +       hexadecimal, this corresponds to a 16-byte/128-bit value.
>     > +       This ID may not be all zeros.
>     > +
>     > +       https://www.freedesktop.org/software/systemd/man/machine-id.html
>     > +
>     >  config BR2_PACKAGE_SYSTEMD_JOURNAL_GATEWAY
>     >       bool "HTTP server for journal events"
>     >       select BR2_PACKAGE_LIBMICROHTTPD
>     > diff --git a/package/systemd/systemd.mk <http://systemd.mk>
>     b/package/systemd/systemd.mk <http://systemd.mk>
>     > index 92490eb86b..fc348fe120 100644
>     > --- a/package/systemd/systemd.mk <http://systemd.mk>
>     > +++ b/package/systemd/systemd.mk <http://systemd.mk>
>     > @@ -461,9 +461,11 @@ define SYSTEMD_INSTALL_INIT_HOOK
>     >             
>      $(TARGET_DIR)/etc/systemd/system/multi-user.target.wants/remote-fs.target
>     >  endef
>     > 
>     > +ifeq ($(BR2_PACKAGE_SYSTEMD_MACHINEID_FILE),y)
>     >  define SYSTEMD_INSTALL_MACHINEID_HOOK
>     >       touch $(TARGET_DIR)/etc/machine-id
>     >  endef
>     > +endif
>     > 
>     >  SYSTEMD_POST_INSTALL_TARGET_HOOKS += \
>     >       SYSTEMD_INSTALL_TARGET_CRYPTSETUP \
>     >
>     _______________________________________________
>     buildroot mailing list
>     buildroot@busybox.net <mailto:buildroot@busybox.net>
>     http://lists.busybox.net/mailman/listinfo/buildroot
> 
> 
> 
> -- 
> SMILE  <http://www.smile.eu/>
> 
> 20 rue des Jardins
> 92600 Asnières-sur-Seine
> 
> 	
> *Jérémy ROSEN*
> Architecte technique
> 
> email jeremy.rosen@smile.fr <mailto:jeremy.rosen@smile.fr> 
> phone  +33 6 88 25 87 42 
> url http://www.smile.eu <http://www.smile.eu/>
> 
> Twitter <https://twitter.com/GroupeSmile> Facebook
> <https://www.facebook.com/smileopensource> LinkedIn
> <https://www.linkedin.com/company/smile> Github <https://github.com/Smile-SA>
> 
> 
> Découvrez l’univers Smile, rendez-vous sur smile.eu
> <https://www.smile.eu/fr/publications/livres-blancs/yocto?utm_source=signature&utm_medium=email&utm_campaign=signature>
Jérémy ROSEN Nov. 17, 2019, 4:14 p.m. UTC | #4
Argh, each time I simplify a bit, it comes back to bite me :)

Le dim. 17 nov. 2019 à 17:00, Arnout Vandecappelle <arnout@mind.be> a
écrit :

>
>
> On 17/11/2019 15:19, Jérémy ROSEN wrote:
> > hmmm, I'm kinda torn on that one..
> >
> > The only difference I see betweed no machine-id and an empty machine-id
> is
> > systemd-firstboot and ConditionFirstBoot
>
>  That's exactly the point I think.
>
>  I assumed, however, that ConditionFirstBoot was evaluated by
> systemd-firstboot,
> but that's not the case then?
>
>
>
no, all Condition* are handled by PID1.
systemd-firstboot.service itself relies on ConditionFirstBoot to decide if
the binary needs to be started


> > if the file is empty or missing and systemd can write to it, systemd
> will set it
> > up (not sure what happens for Arnout's symlink, that's worth testing)
> > however, if the file exists but is empty we will skip systemd-firstboot
> >
> > for context, systemd-firstboot does the following:
> > * set (unconditionnaly) a machine-id
>
>  Are you saying that if systemd-firstboot it not enabled, no machine-id
> will be
> generated?
>
> no, it's a bit more complex, and i'd need to check the details.

my understanding is that systemd-firstboot will always override the
machine-id whereas on a normal boot,
the machine-id will be used if available and generated/written if not



> > * sets the locale (LANG= LC_MESSGES)
> > * sets the hostname
> > * sets the timezone
> > * sets the root password
> >
> > systemd-firstboot can use values from its command line, but when
> launched on the
> > target, the point is usually to prompt the user for answers.
> > (as a side note, we can call it with --root=$DESTDIR once we have
> host-systemd)
> >
> > So I would tend to consider systemd-firstboot as "an uncommon case"
> because
> > prompting user on first boot is not something we see often in the
> embedded space
> >
> > The good news is: we already know when systemd-firstboot is used because
> it's a
> > config option of the systemd package.
> >
> > ConditionFirstBoot is a bit more useful for us.. it allows to easily run
> a
> > service only at first boot (or skip it on first boot)
>
>  Well, here are a few things you'd typically want to generate on first
> boot:
>
> - machine id;
> - ssh keys;
> - various certificates;
> - maybe a hashed root password based on a serial number of MAC address;
> - maybe a random or hashed hostname;
> - register on the phone-home server;
> - resize the data partition based on actual size of the eMMC.
>
>  None of them interactively, of course.
>
>  So apparently systemd-firstboot can only do the first one. That's indeed
> not
> very useful.
>
>
> > we also have a big clue if the system is meant to be read-only via the
> "remount
> > RW" option
>
>  Yeah, but it's just a clue. You can have a writeable machine ID without a
> remount RW option in a number of situations:
>
> - custom skeleton
> - fstab is overwritten by fs-overlay
> - /etc is (bind)-mounted to a writeable partition
> - /etc/machine-id is symlinked into a writeable partition
> - root is squashfs (so it's not writeable even if RW is on)
>
>
> > So, putting it all together
> > * if we use systemd-firstboot, we don't want a machine-id
>
>  ... at least, not an empty one :-)
>
> > * if we have a read-only rootfs, we want an empty machine-id
> > * in other cases, we can deal with both, but it's better to have no
> machine-id
> > * I don't think that the concept of first boot makes any sense with a
> read-only
> > rootfs...
>
>  With readonly *rootfs* it certainly does, but not without any persistent
> storage at all.
>
> >
> > Could something like that be enough ? can we trust "remount RW" ?
> > maybe "remount RW" should be renamed "create a RW filesystem" and enable
> various
> > tweaks related to RO vs RW
>
>  As written above: no.
>
>
>  The problem is: we're not a distro. We leave too much freedom for the
> user to
> integrate things in various ways to be able to make assumptions about what
> is
> the right way to do things. So, the only thing we can do is to give a
> decent
> out-of-the-box experience, and let the user figure out how to tweak things
> -
> possibly adding a config option for a common situation that is easily
> handled in
> a generic way. The other thing we can do is to provide documentation about
> the
> proper way to integrate things in different scenarios.
>
>  I'm starting to agree that this option is maybe not that great.
>
>  Actually, come to think of it, maybe it's better to never create the
> machine-id
> at all. What is the effect (with a readonly rootfs) if machine-id doesn't
> exist?
> Does it block booting, does the boot take a little longer, or do we boot
> with a
> different machine-id all the time? And in the latter case, is there a real
> problem with that?
>

on a RO filesystem
* if /etc/machine-id exists and contains a machine-id, systemd will use it
* if it's an empty file, systemd will bind-mount a file with a machine-id
over the file
* if the file does not exist, systemd can't bind mount over the file...
systemd itself should be fine, but other services might not, it's hard to
tell

So the recommended way for a read-only rootfs is to have that empty file,
which is why this is what currently have.
it works both for RW and RO filesystems..


A longer-term solution would be to add a patch upstream to allow to specify
in a systemd configuration file what is
the file to check for first boot instead of /etc/machine-id.

I can look into that, but for the short term, i'd rather break
ConditionFirstBoot than read-only rootfs

so... not sure what to advise at that point. that choice is something a bit
too specialized for a config option
especially if we plan to drop it in the future

so not sure where to go...


>
>  Regards,
>  Arnout
>
> >
> > my two cents...
> >
> >
> > Le dim. 17 nov. 2019 à 14:39, Arnout Vandecappelle <arnout@mind.be
> > <mailto:arnout@mind.be>> a écrit :
> >
> >      Hi Bartosz,
> >
> >     On 17/11/2019 12:33, Bartosz Bilas wrote:
> >     > In case of e.g writable systems there is no neccesity to have
> >                      writeable                    necessity
> >
> >     > pre-installed empty machine-id file on target because systemd
> >     > generates that automatically during system boot.
> >     > Also in case of having empty machine-id file we are not able
> >     > to use a service with ConditionFirstBoot because systemd recognizes
> >     > machine-id file in system therefore we can't detect new system
> instance
> >     boots up.
> >     > Set this option as enable by default to keep compatibility with
> old builds.
> >
> >      Nice that you take a look at this! I'm immediately going to follow
> it up with
> >     more feature requests though :-)
> >
> >     - If there is a writeable persistent partition, but /etc is not
> writeable, can
> >     we create a symlink and will systemd still create it correctly at
> first boot?
> >
> >     - Maybe if BR2_TARGET_GENERIC_REMOUNT_ROOTFS_RW is set, we should
> default it or
> >     even force it to no.
> >
> >
> >      Regards,
> >      Arnout
> >
> >     >
> >     > Signed-off-by: Bartosz Bilas <b.bilas@grinn-global.com
> >     <mailto:b.bilas@grinn-global.com>>
> >     > ---
> >     >  package/systemd/Config.in  | 13 +++++++++++++
> >     >  package/systemd/systemd.mk <http://systemd.mk> |  2 ++
> >     >  2 files changed, 15 insertions(+)
> >     >
> >     > diff --git a/package/systemd/Config.in b/package/systemd/Config.in
> >     > index aef39abe27..fadc1a32c8 100644
> >     > --- a/package/systemd/Config.in
> >     > +++ b/package/systemd/Config.in
> >     > @@ -112,6 +112,19 @@ config BR2_PACKAGE_SYSTEMD_BOOT_EFI_ARCH
> >     >       default "x64"   if BR2_x86_64
> >     >       depends on BR2_PACKAGE_SYSTEMD_BOOT
> >     >
> >     > +config BR2_PACKAGE_SYSTEMD_MACHINEID_FILE
> >     > +     bool "Install empty machine id file"
> >     > +     default y
> >     > +     help
> >     > +       The /etc/machine-id file contains the unique machine ID
> >     > +       of the local system that is set during installation or
> >     > +       boot. The machine ID is a single newline-terminated,
> >     > +       hexadecimal, 32-character, lowercase ID. When decoded from
> >     > +       hexadecimal, this corresponds to a 16-byte/128-bit value.
> >     > +       This ID may not be all zeros.
> >     > +
> >     > +
> https://www.freedesktop.org/software/systemd/man/machine-id.html
> >     > +
> >     >  config BR2_PACKAGE_SYSTEMD_JOURNAL_GATEWAY
> >     >       bool "HTTP server for journal events"
> >     >       select BR2_PACKAGE_LIBMICROHTTPD
> >     > diff --git a/package/systemd/systemd.mk <http://systemd.mk>
> >     b/package/systemd/systemd.mk <http://systemd.mk>
> >     > index 92490eb86b..fc348fe120 100644
> >     > --- a/package/systemd/systemd.mk <http://systemd.mk>
> >     > +++ b/package/systemd/systemd.mk <http://systemd.mk>
> >     > @@ -461,9 +461,11 @@ define SYSTEMD_INSTALL_INIT_HOOK
> >     >
> >
>   $(TARGET_DIR)/etc/systemd/system/multi-user.target.wants/remote-fs.target
> >     >  endef
> >     >
> >     > +ifeq ($(BR2_PACKAGE_SYSTEMD_MACHINEID_FILE),y)
> >     >  define SYSTEMD_INSTALL_MACHINEID_HOOK
> >     >       touch $(TARGET_DIR)/etc/machine-id
> >     >  endef
> >     > +endif
> >     >
> >     >  SYSTEMD_POST_INSTALL_TARGET_HOOKS += \
> >     >       SYSTEMD_INSTALL_TARGET_CRYPTSETUP \
> >     >
> >     _______________________________________________
> >     buildroot mailing list
> >     buildroot@busybox.net <mailto:buildroot@busybox.net>
> >     http://lists.busybox.net/mailman/listinfo/buildroot
> >
> >
> >
> > --
> > SMILE  <http://www.smile.eu/>
> >
> > 20 rue des Jardins
> > 92600 Asnières-sur-Seine
> >
> >
> > *Jérémy ROSEN*
> > Architecte technique
> >
> > email jeremy.rosen@smile.fr <mailto:jeremy.rosen@smile.fr>
> > phone  +33 6 88 25 87 42
> > url http://www.smile.eu <http://www.smile.eu/>
> >
> > Twitter <https://twitter.com/GroupeSmile> Facebook
> > <https://www.facebook.com/smileopensource> LinkedIn
> > <https://www.linkedin.com/company/smile> Github <
> https://github.com/Smile-SA>
> >
> >
> > Découvrez l’univers Smile, rendez-vous sur smile.eu
> > <
> https://www.smile.eu/fr/publications/livres-blancs/yocto?utm_source=signature&utm_medium=email&utm_campaign=signature
> >
>
Arnout Vandecappelle Nov. 17, 2019, 5:34 p.m. UTC | #5
Hi Jérémy,

 Thanks for your input!

On 17/11/2019 17:14, Jérémy ROSEN wrote:
> Argh, each time I simplify a bit, it comes back to bite me :)
> 
> Le dim. 17 nov. 2019 à 17:00, Arnout Vandecappelle <arnout@mind.be
> <mailto:arnout@mind.be>> a écrit :
> 
> 
> 
>     On 17/11/2019 15:19, Jérémy ROSEN wrote:
>     > hmmm, I'm kinda torn on that one..
>     >
>     > The only difference I see betweed no machine-id and an empty machine-id is
>     > systemd-firstboot and ConditionFirstBoot
> 
>      That's exactly the point I think.
> 
>      I assumed, however, that ConditionFirstBoot was evaluated by systemd-firstboot,
>     but that's not the case then?
> 
> 
> 
> no, all Condition* are handled by PID1. 
> systemd-firstboot.service itself relies on ConditionFirstBoot to decide if the
> binary needs to be started

 Of course, that makes a lot of sense if you think about it. I should do more of
that :-)


>     > if the file is empty or missing and systemd can write to it, systemd will
>     set it
>     > up (not sure what happens for Arnout's symlink, that's worth testing)
>     > however, if the file exists but is empty we will skip systemd-firstboot
>     >
>     > for context, systemd-firstboot does the following:
>     > * set (unconditionnaly) a machine-id
> 
>      Are you saying that if systemd-firstboot it not enabled, no machine-id will be
>     generated?
> 
> no, it's a bit more complex, and i'd need to check the details.
> 
> my understanding is that systemd-firstboot will always override the machine-id
> whereas on a normal boot, 
> the machine-id will be used if available and generated/written if not

 That also makes sense.

[snip]
>      Actually, come to think of it, maybe it's better to never create the machine-id
>     at all. What is the effect (with a readonly rootfs) if machine-id doesn't exist?
>     Does it block booting, does the boot take a little longer, or do we boot with a
>     different machine-id all the time? And in the latter case, is there a real
>     problem with that?
> 
> 
> on a RO filesystem
> * if /etc/machine-id exists and contains a machine-id, systemd will use it
> * if it's an empty file, systemd will bind-mount a file with a machine-id over
> the file

 Where I suppose the machine-id is regenerated on every boot.

> * if the file does not exist, systemd can't bind mount over the file... systemd
> itself should be fine, but other services might not, it's hard to tell

 That's not good...

> 
> So the recommended way for a read-only rootfs is to have that empty file, which
> is why this is what currently have.
> it works both for RW and RO filesystems..

 Another option would be to make it a symlink to a file in tmpfs, but that
doesn't help because the net effect is the same: machine-id is regenerated on
every boot :-(


> A longer-term solution would be to add a patch upstream to allow to specify in a
> systemd configuration file what is 
> the file to check for first boot instead of /etc/machine-id.

 No, that makes very little sense. If a system has any kind of persistency, then
machine-id should definitely be one of the things that is persisted, so it's a
perfect flag for firstboot.

 Upstream's vision is that /etc should be persistent and empty on factory reset
[1]. But we're far away from having an empty /etc in Buildroot...



> I can look into that, but for the short term, i'd rather break
> ConditionFirstBoot than read-only rootfs
> 
> so... not sure what to advise at that point. that choice is something a bit too
> specialized for a config option
> especially if we plan to drop it in the future
> 
> so not sure where to go...

 As is often the case: it looks to me that the status quo is the best we can do :-)

 Regards,
 Arnout

[1] http://0pointer.net/blog/projects/stateless.html

[snip]
Jérémy ROSEN Nov. 17, 2019, 5:47 p.m. UTC | #6
>
>
> [snip]
> >      Actually, come to think of it, maybe it's better to never create
> the machine-id
> >     at all. What is the effect (with a readonly rootfs) if machine-id
> doesn't exist?
> >     Does it block booting, does the boot take a little longer, or do we
> boot with a
> >     different machine-id all the time? And in the latter case, is there
> a real
> >     problem with that?
> >
> >
> > on a RO filesystem
> > * if /etc/machine-id exists and contains a machine-id, systemd will use
> it
> > * if it's an empty file, systemd will bind-mount a file with a
> machine-id over
> > the file
>
>  Where I suppose the machine-id is regenerated on every boot.
>
> yes, which is fine, that's the intended behaviour


> > * if the file does not exist, systemd can't bind mount over the file...
> systemd
> > itself should be fine, but other services might not, it's hard to tell
>
>  That's not good...
>
I just tested, and systemd logs an error message that it can't boot in that
case.

more precisely, when systemd starts
* either /etc is readwrite
* or it should have a /etc/machine-id
so we need the file, since our rootfs is read-only when systemd starts
* it's just an error message, everything turns out fine once we remount rw
and we save the machine-id only
* sounds like a bug from upstream... read-only rootfs and read-only rootfs
that become RW are supposed to be supported


>
> >
> > So the recommended way for a read-only rootfs is to have that empty
> file, which
> > is why this is what currently have.
> > it works both for RW and RO filesystems..
>
>  Another option would be to make it a symlink to a file in tmpfs, but that
> doesn't help because the net effect is the same: machine-id is regenerated
> on
> every boot :-(
>
>
regenerated every boot is fine, i'm just not sure in what way a symlink to
a tmpfs would be different/better than what systemd currently does

symlink to a r/w partition makes sense, but that would not work for
ConditionFirstBoot


>
> > A longer-term solution would be to add a patch upstream to allow to
> specify in a
> > systemd configuration file what is
> > the file to check for first boot instead of /etc/machine-id.
>
>  No, that makes very little sense. If a system has any kind of
> persistency, then
> machine-id should definitely be one of the things that is persisted, so
> it's a
> perfect flag for firstboot.
>
> no because it makes sense to have it be a symlink to a RW file... in which
case it exists even in first boot..



>  Upstream's vision is that /etc should be persistent and empty on factory
> reset
> [1]. But we're far away from having an empty /etc in Buildroot...
>
> yeah, but I think the assumption empty /etc == first boot is wrong...
in the real world, the assumption is no /etc/machine-id == first boot makes
more sense, but it doesn't work for us.
No... upstream patch is really the way to go (and i'm already working on it
:P )

>
>
> > I can look into that, but for the short term, i'd rather break
> > ConditionFirstBoot than read-only rootfs
> >
> > so... not sure what to advise at that point. that choice is something a
> bit too
> > specialized for a config option
> > especially if we plan to drop it in the future
> >
> > so not sure where to go...
>
>  As is often the case: it looks to me that the status quo is the best we
> can do :-)
>

it could be slightly improved by removing /etc/machine-id when
systemd-firstboot is selected, but I agree...
Until we can change and configure how to detect first-boot, we are kinda
stuck...


>
>  Regards,
>  Arnout
>
> [1] http://0pointer.net/blog/projects/stateless.html
>
> [snip]
>
Thomas Petazzoni Nov. 19, 2019, 8:40 a.m. UTC | #7
Hello,

On Sun, 17 Nov 2019 17:00:05 +0100
Arnout Vandecappelle <arnout@mind.be> wrote:

> > Could something like that be enough ? can we trust "remount RW" ?
> > maybe "remount RW" should be renamed "create a RW filesystem" and enable various
> > tweaks related to RO vs RW  
> 
>  As written above: no.
> 
>  The problem is: we're not a distro.

Agreed.

> We leave too much freedom for the user to
> integrate things in various ways to be able to make assumptions about what is
> the right way to do things. So, the only thing we can do is to give a decent
> out-of-the-box experience, and let the user figure out how to tweak things -
> possibly adding a config option for a common situation that is easily handled in
> a generic way. The other thing we can do is to provide documentation about the
> proper way to integrate things in different scenarios.
> 
>  I'm starting to agree that this option is maybe not that great.

But I would in fact not come to the same conclusion. Having this empty
machine-id file is useless and causes problems when the filesystem is
R/W. So for the sake of supporting the R/O case (for which we create
this empty machine-id file), we make the R/W experience less good.

So I'd say that the right approach is to not do too much integration by
precisely having the option proposed by Bartosz, with many the tweak
that it should default y if rootfs is really, and default disable
otherwise, but while still being an option that the user can tweak,
because as you rightfully explained, the RW/RO remount option is just a
clue, not a definitive answer on whether /etc is writable or not.

To me, having this option matches the Buildroot way: we are not a
distro, we don't enforce how the system should work, so we provide the
appropriate options, while making sure the option has the most sensible
default values.

Generally speaking, Buildroot kind of supports "out of the box" two use
cases:

 (1) The root filesystem is completely read-write.

 (2) The root filesystem is completely read-only, and all files that need
     to be written are stored in tmpfs, and therefore are volatile.

I.e, we do not have any explicit support for what is I guess a much
more common use case than (2):

 (3) The root filesystem is completely read-only, but there is another
     read-write partition somewhere that stores the information that can
     change but needs to be persistent (user configuration, etc.)

Since we don't have explicit support for (3), there is no way we can
properly support machine-id and ConditionFirstBoot in the case of (2),
because there's nowhere we can store /etc/machine-id.

So the best we can do is in the case of (2), default to creating an
empty /etc/machine-id, while giving the possibility for the user
implementing (3) in its own way, to disable the creation of the empty
/etc/machine-id.

Best regards,

Thomas
Jérémy ROSEN Nov. 19, 2019, 10:15 a.m. UTC | #8
As a side-note, I am working with upstream to have a better support of (3)
: https://github.com/systemd/systemd/pull/14059

I am a bit cautious about the new config option because it seems too
"advanced" for a config option (for me, it's something that should be set
in a post-image or overlay) but that's open to discussion.

Please wait a little before applying this patch, if that's the way
buildroot wants to go, so my pull-request above is solved and we might
backport it to ease our transition.

Cheers
Jérémy

Le mar. 19 nov. 2019 à 09:40, Thomas Petazzoni <thomas.petazzoni@bootlin.com>
a écrit :

> Hello,
>
> On Sun, 17 Nov 2019 17:00:05 +0100
> Arnout Vandecappelle <arnout@mind.be> wrote:
>
> > > Could something like that be enough ? can we trust "remount RW" ?
> > > maybe "remount RW" should be renamed "create a RW filesystem" and
> enable various
> > > tweaks related to RO vs RW
> >
> >  As written above: no.
> >
> >  The problem is: we're not a distro.
>
> Agreed.
>
> > We leave too much freedom for the user to
> > integrate things in various ways to be able to make assumptions about
> what is
> > the right way to do things. So, the only thing we can do is to give a
> decent
> > out-of-the-box experience, and let the user figure out how to tweak
> things -
> > possibly adding a config option for a common situation that is easily
> handled in
> > a generic way. The other thing we can do is to provide documentation
> about the
> > proper way to integrate things in different scenarios.
> >
> >  I'm starting to agree that this option is maybe not that great.
>
> But I would in fact not come to the same conclusion. Having this empty
> machine-id file is useless and causes problems when the filesystem is
> R/W. So for the sake of supporting the R/O case (for which we create
> this empty machine-id file), we make the R/W experience less good.
>
> So I'd say that the right approach is to not do too much integration by
> precisely having the option proposed by Bartosz, with many the tweak
> that it should default y if rootfs is really, and default disable
> otherwise, but while still being an option that the user can tweak,
> because as you rightfully explained, the RW/RO remount option is just a
> clue, not a definitive answer on whether /etc is writable or not.
>
> To me, having this option matches the Buildroot way: we are not a
> distro, we don't enforce how the system should work, so we provide the
> appropriate options, while making sure the option has the most sensible
> default values.
>
> Generally speaking, Buildroot kind of supports "out of the box" two use
> cases:
>
>  (1) The root filesystem is completely read-write.
>
>  (2) The root filesystem is completely read-only, and all files that need
>      to be written are stored in tmpfs, and therefore are volatile.
>
> I.e, we do not have any explicit support for what is I guess a much
> more common use case than (2):
>
>  (3) The root filesystem is completely read-only, but there is another
>      read-write partition somewhere that stores the information that can
>      change but needs to be persistent (user configuration, etc.)
>
> Since we don't have explicit support for (3), there is no way we can
> properly support machine-id and ConditionFirstBoot in the case of (2),
> because there's nowhere we can store /etc/machine-id.
>
> So the best we can do is in the case of (2), default to creating an
> empty /etc/machine-id, while giving the possibility for the user
> implementing (3) in its own way, to disable the creation of the empty
> /etc/machine-id.
>
> Best regards,
>
> Thomas
> --
> Thomas Petazzoni, CTO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>

Patch
diff mbox series

diff --git a/package/systemd/Config.in b/package/systemd/Config.in
index aef39abe27..fadc1a32c8 100644
--- a/package/systemd/Config.in
+++ b/package/systemd/Config.in
@@ -112,6 +112,19 @@  config BR2_PACKAGE_SYSTEMD_BOOT_EFI_ARCH
 	default "x64"   if BR2_x86_64
 	depends on BR2_PACKAGE_SYSTEMD_BOOT
 
+config BR2_PACKAGE_SYSTEMD_MACHINEID_FILE
+	bool "Install empty machine id file"
+	default y
+	help
+	  The /etc/machine-id file contains the unique machine ID
+	  of the local system that is set during installation or
+	  boot. The machine ID is a single newline-terminated,
+	  hexadecimal, 32-character, lowercase ID. When decoded from
+	  hexadecimal, this corresponds to a 16-byte/128-bit value.
+	  This ID may not be all zeros.
+
+	  https://www.freedesktop.org/software/systemd/man/machine-id.html
+
 config BR2_PACKAGE_SYSTEMD_JOURNAL_GATEWAY
 	bool "HTTP server for journal events"
 	select BR2_PACKAGE_LIBMICROHTTPD
diff --git a/package/systemd/systemd.mk b/package/systemd/systemd.mk
index 92490eb86b..fc348fe120 100644
--- a/package/systemd/systemd.mk
+++ b/package/systemd/systemd.mk
@@ -461,9 +461,11 @@  define SYSTEMD_INSTALL_INIT_HOOK
 		$(TARGET_DIR)/etc/systemd/system/multi-user.target.wants/remote-fs.target
 endef
 
+ifeq ($(BR2_PACKAGE_SYSTEMD_MACHINEID_FILE),y)
 define SYSTEMD_INSTALL_MACHINEID_HOOK
 	touch $(TARGET_DIR)/etc/machine-id
 endef
+endif
 
 SYSTEMD_POST_INSTALL_TARGET_HOOKS += \
 	SYSTEMD_INSTALL_TARGET_CRYPTSETUP \