Patchwork [RFC] new target: live filesystem

login
register
mail settings
Submitter Jeremy Rosen
Date Dec. 6, 2012, 1:36 p.m.
Message ID <1354800968-16745-1-git-send-email-jeremy.rosen@openwide.fr>
Download mbox | patch
Permalink /patch/204226/
State Rejected
Headers show

Comments

Jeremy Rosen - Dec. 6, 2012, 1:36 p.m.
add a new target to deploy a live filesystem to be used with NFS or as a chroot

Signed-off-by: Jérémy Rosen <jeremy.rosen@openwide.fr>
---
v2 : implement Arnoult's suggestion, update manual entry
v3 : improve documentation for the chroot case, more suggestions by Arnoult
---
 docs/manual/beyond-buildroot.txt     |   20 ++++++++------------
 fs/Config.in                         |    1 +
 fs/live/Config.in                    |   14 ++++++++++++++
 fs/live/live.mk                      |   19 +++++++++++++++++++
 support/dependencies/dependencies.sh |    8 ++++++++
 5 files changed, 50 insertions(+), 12 deletions(-)
 create mode 100644 fs/live/Config.in
 create mode 100644 fs/live/live.mk
Arnout Vandecappelle - Dec. 10, 2012, 7:11 a.m.
Peter, can you make the call if this has a chance of being accepted?
Otherwise Jérémy is wasting his time...


On 06/12/12 14:36, Jérémy Rosen wrote:
> add a new target to deploy a live filesystem to be used with NFS or as a chroot

  The commit message should be word-wrapped at +- 75 characters IIRC.

>
> Signed-off-by: Jérémy Rosen<jeremy.rosen@openwide.fr>
> ---
> v2 : implement Arnoult's suggestion, update manual entry

  Arnout

> v3 : improve documentation for the chroot case, more suggestions by Arnoult
> ---
[snip]
> diff --git a/fs/Config.in b/fs/Config.in
> index da4c5ff..664d2f6 100644
> --- a/fs/Config.in
> +++ b/fs/Config.in
> @@ -11,5 +11,6 @@ source "fs/romfs/Config.in"
>   source "fs/squashfs/Config.in"
>   source "fs/tar/Config.in"
>   source "fs/ubifs/Config.in"
> +source "fs/live/Config.in"

  Was there a problem with sorting this alphabetically?

>
>   endmenu
> diff --git a/fs/live/Config.in b/fs/live/Config.in
> new file mode 100644
> index 0000000..a79f1dc
> --- /dev/null
> +++ b/fs/live/Config.in
> @@ -0,0 +1,14 @@
> +config BR2_TARGET_ROOTFS_LIVE
> +	bool "live root filesystem"
> +	help
> +	  Create a live image of the root filesystem that is directly
> +	  usable as over NFS or chroot.

  usable over NFS or chroot.

> +
> +config BR2_TARGET_ROOTFS_LIVE_DEST
> +	string "live image location"
> +	depends on BR2_TARGET_ROOTFS_LIVE
> +	default "$(BINARIES_DIR)/live"
> +	help
> +	  The directory where the image should be stored.
> +	  This directory will be emptied and recreated

  How about:

The directory where the live root filesystem will be
installed. Any changes to this tree are lost.

> +

  Redundant empty line.

> diff --git a/fs/live/live.mk b/fs/live/live.mk
> new file mode 100644
> index 0000000..52f7444
> --- /dev/null
> +++ b/fs/live/live.mk
> @@ -0,0 +1,19 @@
> +#############################################################
> +#
> +# Build the live root filesystem directory
> +#
> +#############################################################
> +
> +
> +define ROOTFS_LIVE_CMD
> +	sudo rsync -a --no-o --no-g --delete-during $(TARGET_DIR)/ \
> +		$(BR2_TARGET_ROOTFS_LIVE_DEST)/
> +endef
> +
> +define ROOTFS_LIVE_INIT
> +  if [ -z $(shell which sudo) ] ; then echo "sudo seems to not be installed on the host system" ; false ; fi
> +endef
> +
> +ROOTFS_LIVE_PRE_GEN_HOOKS += ROOTFS_LIVE_INIT

  Since it's already in dependencies.sh, this piece is redundant.

> +
> +$(eval $(call ROOTFS_TARGET,live))
> diff --git a/support/dependencies/dependencies.sh b/support/dependencies/dependencies.sh
> index 7a02512..ebaabbb 100755
> --- a/support/dependencies/dependencies.sh
> +++ b/support/dependencies/dependencies.sh
> @@ -158,6 +158,7 @@ if grep ^BR2_TOOLCHAIN_BUILDROOT=y $CONFIG_FILE>  /dev/null&&  \
>          exit 1 ;
>      fi
>   fi
> +
>   if grep -q ^BR2_PACKAGE_CLASSPATH=y $CONFIG_FILE ; then
>       for prog in javac jar; do
>   	if ! which $prog>  /dev/null ; then
> @@ -166,3 +167,10 @@ if grep -q ^BR2_PACKAGE_CLASSPATH=y $CONFIG_FILE ; then
>   	fi
>       done
>   fi
> +
> +if grep ^BR2_TARGET_ROOTFS_LIVE=y $CONFIG_FILE>  /dev/null ; then

  We use 'grep -q' rather than redirecting to /dev/null.

> +   if ! which sudo>  /dev/null ; then
> +       /bin/echo -e "\nYou need sudo installed on your build machine to build a live filesystem\n"
> +       exit 1 ;
> +   fi
> +fi

  Maybe, in addition to the mere existence of sudo, you should also
check if it the user is actually in sudoers.  'sudo -l rsync' should
do the trick.

  Regards,
  Arnout
Jeremy Rosen - Dec. 10, 2012, 10:51 a.m.
As usual, sniped what I agree with and v4 is on the way

> > diff --git a/fs/Config.in b/fs/Config.in
> > index da4c5ff..664d2f6 100644
> > --- a/fs/Config.in
> > +++ b/fs/Config.in
> > @@ -11,5 +11,6 @@ source "fs/romfs/Config.in"
> >   source "fs/squashfs/Config.in"
> >   source "fs/tar/Config.in"
> >   source "fs/ubifs/Config.in"
> > +source "fs/live/Config.in"
> 
>   Was there a problem with sorting this alphabetically?

just a mixup between my "sort alphabetically" and my "live" patches... fixing that

> > +
> > +if grep ^BR2_TARGET_ROOTFS_LIVE=y $CONFIG_FILE>  /dev/null ; then
> 
>   We use 'grep -q' rather than redirecting to /dev/null.
> 

I copied how it was done for locale that redirect to /dev/null, thus my mistake

I'll fix that in v4, but not the locale ones (to avoid mixup in my patch)



Thx for the comments, i'll wait a little to see if more comments come and then i'll send a V4

I also need to see what to do  with "make distclean" when the live filesystem is in output/

currently there are all sorts of permission-denied errors, and i'm not sure what to do here...

* I could use the config options to run "sudo rm" but i'm not sure we want to read the config option in distclean
* I could just leave it as is and just say the user is smart enough to run "sudo make distclean" in that case
* I could blindly run "sudo rm" in output but that sounds a bit dangerous


I tend to think the second option is the best, but i'd gladly get feedback on that one
Jeremy Rosen - Dec. 17, 2012, 8:58 a.m.
Hello everybody

there is a little bit more work to do on this patch (mainly updating a few more places in the documentation) but i'd like to have a go/no-go before I do that...

Peter, could I have a final call on this ?

    Regards

    Jérémy Rosen

fight key loggers : write some perl using vim

----- Mail original -----
> De: "Jérémy Rosen" <jeremy.rosen@openwide.fr>
> À: buildroot@busybox.net
> Cc: "Jérémy Rosen" <jeremy.rosen@openwide.fr>
> Envoyé: Jeudi 6 Décembre 2012 14:36:07
> Objet: [PATCH] [RFC] new target: live filesystem
> 
> add a new target to deploy a live filesystem to be used with NFS or
> as a chroot
> 
> Signed-off-by: Jérémy Rosen <jeremy.rosen@openwide.fr>
> ---
> v2 : implement Arnoult's suggestion, update manual entry
> v3 : improve documentation for the chroot case, more suggestions by
> Arnoult
> ---
>  docs/manual/beyond-buildroot.txt     |   20 ++++++++------------
>  fs/Config.in                         |    1 +
>  fs/live/Config.in                    |   14 ++++++++++++++
>  fs/live/live.mk                      |   19 +++++++++++++++++++
>  support/dependencies/dependencies.sh |    8 ++++++++
>  5 files changed, 50 insertions(+), 12 deletions(-)
>  create mode 100644 fs/live/Config.in
>  create mode 100644 fs/live/live.mk
> 
> diff --git a/docs/manual/beyond-buildroot.txt
> b/docs/manual/beyond-buildroot.txt
> index a87b584..17ccc1a 100644
> --- a/docs/manual/beyond-buildroot.txt
> +++ b/docs/manual/beyond-buildroot.txt
> @@ -9,19 +9,15 @@ Boot the generated images
>  NFS boot
>  ~~~~~~~~
>  
> -To achieve NFS-boot, enable _tar root filesystem_ in the _Filesystem
> -images_ menu.
> +To achieve NFS-boot, enable _live root filesystem_ in the
> _Filesystem
> +images_ menu and select a _live image location_ to choose where the
> live
> +filesystem will be deployed. you can use _$(BINARIES_DIR)_ to easily
> +build in +/path/to/output_dir/images+
>  
> -After a complete build, just run the following commands to setup the
> -NFS-root directory:
> +You will be asked for a password during the build. This is needed to
> create
> +device entries in the target filesystem
>  
> --------------------
> -sudo tar -xavf /path/to/output_dir/rootfs.tar -C
> /path/to/nfs_root_dir
> --------------------
> -
> -Remember to add this path to +/etc/exports+.
> -
> -Then, you can execute a NFS-boot from your target.
> +You will need to add the _live image location_  to +/etc/exports+.
>  
>  Chroot
>  ------
> @@ -29,7 +25,7 @@ Chroot
>  If you want to chroot in a generated image, then there are few thing
>  you should be aware of:
>  
> -* you should setup the new root from the _tar root filesystem_
> image;
> +* you should use the _live root filesystem_ image;
>  
>  * either the selected target architecture is compatible with your
>  host
>    machine, or you should use some +qemu-*+ binary and correctly set
>    it
> diff --git a/fs/Config.in b/fs/Config.in
> index da4c5ff..664d2f6 100644
> --- a/fs/Config.in
> +++ b/fs/Config.in
> @@ -11,5 +11,6 @@ source "fs/romfs/Config.in"
>  source "fs/squashfs/Config.in"
>  source "fs/tar/Config.in"
>  source "fs/ubifs/Config.in"
> +source "fs/live/Config.in"
>  
>  endmenu
> diff --git a/fs/live/Config.in b/fs/live/Config.in
> new file mode 100644
> index 0000000..a79f1dc
> --- /dev/null
> +++ b/fs/live/Config.in
> @@ -0,0 +1,14 @@
> +config BR2_TARGET_ROOTFS_LIVE
> +	bool "live root filesystem"
> +	help
> +	  Create a live image of the root filesystem that is directly
> +	  usable as over NFS or chroot.
> +
> +config BR2_TARGET_ROOTFS_LIVE_DEST
> +	string "live image location"
> +	depends on BR2_TARGET_ROOTFS_LIVE
> +	default "$(BINARIES_DIR)/live"
> +	help
> +	  The directory where the image should be stored.
> +	  This directory will be emptied and recreated
> +
> diff --git a/fs/live/live.mk b/fs/live/live.mk
> new file mode 100644
> index 0000000..52f7444
> --- /dev/null
> +++ b/fs/live/live.mk
> @@ -0,0 +1,19 @@
> +#############################################################
> +#
> +# Build the live root filesystem directory
> +#
> +#############################################################
> +
> +
> +define ROOTFS_LIVE_CMD
> +	sudo rsync -a --no-o --no-g --delete-during $(TARGET_DIR)/ \
> +		$(BR2_TARGET_ROOTFS_LIVE_DEST)/
> +endef
> +
> +define ROOTFS_LIVE_INIT
> +  if [ -z $(shell which sudo) ] ; then echo "sudo seems to not be
> installed on the host system" ; false ; fi
> +endef
> +
> +ROOTFS_LIVE_PRE_GEN_HOOKS += ROOTFS_LIVE_INIT
> +
> +$(eval $(call ROOTFS_TARGET,live))
> diff --git a/support/dependencies/dependencies.sh
> b/support/dependencies/dependencies.sh
> index 7a02512..ebaabbb 100755
> --- a/support/dependencies/dependencies.sh
> +++ b/support/dependencies/dependencies.sh
> @@ -158,6 +158,7 @@ if grep ^BR2_TOOLCHAIN_BUILDROOT=y $CONFIG_FILE >
> /dev/null && \
>         exit 1 ;
>     fi
>  fi
> +
>  if grep -q ^BR2_PACKAGE_CLASSPATH=y $CONFIG_FILE ; then
>      for prog in javac jar; do
>  	if ! which $prog > /dev/null ; then
> @@ -166,3 +167,10 @@ if grep -q ^BR2_PACKAGE_CLASSPATH=y $CONFIG_FILE
> ; then
>  	fi
>      done
>  fi
> +
> +if grep ^BR2_TARGET_ROOTFS_LIVE=y $CONFIG_FILE > /dev/null ; then
> +   if ! which sudo > /dev/null ; then
> +       /bin/echo -e "\nYou need sudo installed on your build machine
> to build a live filesystem\n"
> +       exit 1 ;
> +   fi
> +fi
> --
> 1.7.10.4
> 
>
Richard Genoud - Dec. 17, 2012, 12:41 p.m.
2012/12/17 Jeremy Rosen <jeremy.rosen@openwide.fr>:
> Hello everybody
>
> there is a little bit more work to do on this patch (mainly updating a few more places in the documentation) but i'd like to have a go/no-go before I do that...

I was looking for that kind of feature, I hope it will be accepted.
For now, I use the output/target tree and those parameters in the
/etc/exports file:
*(rw,all_squash,no_subtree_check,anonuid=my_uid,anongid=my_gid)

But that means that all the FS is owned by my user.

Regards,
Richard.
Samuel Martin - Dec. 17, 2012, 4:08 p.m.
Hi Richard,

2012/12/17 Richard Genoud <richard.genoud@gmail.com>:
> 2012/12/17 Jeremy Rosen <jeremy.rosen@openwide.fr>:
>> Hello everybody
>>
>> there is a little bit more work to do on this patch (mainly updating a few more places in the documentation) but i'd like to have a go/no-go before I do that...
>
> I was looking for that kind of feature, I hope it will be accepted.
> For now, I use the output/target tree and those parameters in the
> /etc/exports file:
> *(rw,all_squash,no_subtree_check,anonuid=my_uid,anongid=my_gid)
>
> But that means that all the FS is owned by my user.

There are plenty of good reasons to not use output/target as root for
a chroot or a NFS root, as explained here:
http://buildroot.org/downloads/manual/manual.html#faq-why-not-use-target-as-chroot

And here, you'll find the right way to do this (so far):
http://buildroot.org/downloads/manual/manual.html#_nfs_boot


Regards,
Richard Genoud - Dec. 17, 2012, 4:35 p.m.
2012/12/17 Samuel Martin <s.martin49@gmail.com>:
> Hi Richard,
> There are plenty of good reasons to not use output/target as root for
> a chroot or a NFS root, as explained here:
> http://buildroot.org/downloads/manual/manual.html#faq-why-not-use-target-as-chroot
>
> And here, you'll find the right way to do this (so far):
> http://buildroot.org/downloads/manual/manual.html#_nfs_boot
>
Yes, I know...
It's just more convenient (fast) for me like that:
When I recompile a small software (with make mysoft-reconfigure), I'd
like it to be fast, and de-tar-ing the whole RFS takes some time.
But, it's true that it's not the right way to do it.
(I also always forgot to sudo when I want to copy something to the
nfsroot, so, like that, the tree is mine and there's no cp problem)

Thanks for the links though !

Regards,
Richard.
Arnout Vandecappelle - Jan. 2, 2013, 5:38 p.m.
More comments on top of what I sent before...

On 12/06/12 14:36, Jérémy Rosen wrote:
> add a new target to deploy a live filesystem to be used with NFS or as a chroot
>
> Signed-off-by: Jérémy Rosen<jeremy.rosen@openwide.fr>

Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

  I used this patch (after some more adaptation) to rsync the rootfs to a 
running target over ssh. This is ideal for development. It's 
unfortunately a bit hackish because ssh tries to use /root/.ssh/... under 
fakeroot.

[snip]
> ---
> v2 : implement Arnoult's suggestion, update manual entry
> v3 : improve documentation for the chroot case, more suggestions by Arnoult
> ---
>   docs/manual/beyond-buildroot.txt     |   20 ++++++++------------
>   fs/Config.in                         |    1 +
>   fs/live/Config.in                    |   14 ++++++++++++++
>   fs/live/live.mk                      |   19 +++++++++++++++++++
>   support/dependencies/dependencies.sh |    8 ++++++++
>   5 files changed, 50 insertions(+), 12 deletions(-)
>   create mode 100644 fs/live/Config.in
>   create mode 100644 fs/live/live.mk
>
> diff --git a/docs/manual/beyond-buildroot.txt b/docs/manual/beyond-buildroot.txt
> index a87b584..17ccc1a 100644
> --- a/docs/manual/beyond-buildroot.txt
> +++ b/docs/manual/beyond-buildroot.txt
> @@ -9,19 +9,15 @@ Boot the generated images
>   NFS boot
>   ~~~~~~~~
>
> -To achieve NFS-boot, enable _tar root filesystem_ in the _Filesystem
> -images_ menu.
> +To achieve NFS-boot, enable _live root filesystem_ in the _Filesystem
> +images_ menu and select a _live image location_ to choose where the live
> +filesystem will be deployed. you can use _$(BINARIES_DIR)_ to easily

  Trailing whitespace.


> +build in +/path/to/output_dir/images+
>
> -After a complete build, just run the following commands to setup the
> -NFS-root directory:
> +You will be asked for a password during the build. This is needed to create
> +device entries in the target filesystem

  Note exactly correct.  Something like:

Since the NFS root must be installed as the real root user, you need to 
have sudo access. You will be asked for a password during the build.

>
> --------------------
> -sudo tar -xavf /path/to/output_dir/rootfs.tar -C /path/to/nfs_root_dir
> --------------------
> -
> -Remember to add this path to +/etc/exports+.
> -
> -Then, you can execute a NFS-boot from your target.
> +You will need to add the _live image location_  to +/etc/exports+.
>

[snip]

> +define ROOTFS_LIVE_INIT
> +  if [ -z $(shell which sudo) ] ; then echo "sudo seems to not be installed on the host system" ; false ; fi

  Trailing whitespace.

> +endef
> +
> +ROOTFS_LIVE_PRE_GEN_HOOKS += ROOTFS_LIVE_INIT
> +
> +$(eval $(call ROOTFS_TARGET,live))
[snip]
Jeremy Rosen - Jan. 3, 2013, 8:54 a.m.
unfortunately I had the final word and this patch won't be included in master...

I'd still be interested in your changes if they make sense to add in my patch,

I might post that patch one last time if you are interested though...

    Regards

    Jérémy Rosen

fight key loggers : write some perl using vim

Patch

diff --git a/docs/manual/beyond-buildroot.txt b/docs/manual/beyond-buildroot.txt
index a87b584..17ccc1a 100644
--- a/docs/manual/beyond-buildroot.txt
+++ b/docs/manual/beyond-buildroot.txt
@@ -9,19 +9,15 @@  Boot the generated images
 NFS boot
 ~~~~~~~~
 
-To achieve NFS-boot, enable _tar root filesystem_ in the _Filesystem
-images_ menu.
+To achieve NFS-boot, enable _live root filesystem_ in the _Filesystem
+images_ menu and select a _live image location_ to choose where the live
+filesystem will be deployed. you can use _$(BINARIES_DIR)_ to easily 
+build in +/path/to/output_dir/images+
 
-After a complete build, just run the following commands to setup the
-NFS-root directory:
+You will be asked for a password during the build. This is needed to create
+device entries in the target filesystem
 
--------------------
-sudo tar -xavf /path/to/output_dir/rootfs.tar -C /path/to/nfs_root_dir
--------------------
-
-Remember to add this path to +/etc/exports+.
-
-Then, you can execute a NFS-boot from your target.
+You will need to add the _live image location_  to +/etc/exports+.
 
 Chroot
 ------
@@ -29,7 +25,7 @@  Chroot
 If you want to chroot in a generated image, then there are few thing
 you should be aware of:
 
-* you should setup the new root from the _tar root filesystem_ image;
+* you should use the _live root filesystem_ image;
 
 * either the selected target architecture is compatible with your host
   machine, or you should use some +qemu-*+ binary and correctly set it
diff --git a/fs/Config.in b/fs/Config.in
index da4c5ff..664d2f6 100644
--- a/fs/Config.in
+++ b/fs/Config.in
@@ -11,5 +11,6 @@  source "fs/romfs/Config.in"
 source "fs/squashfs/Config.in"
 source "fs/tar/Config.in"
 source "fs/ubifs/Config.in"
+source "fs/live/Config.in"
 
 endmenu
diff --git a/fs/live/Config.in b/fs/live/Config.in
new file mode 100644
index 0000000..a79f1dc
--- /dev/null
+++ b/fs/live/Config.in
@@ -0,0 +1,14 @@ 
+config BR2_TARGET_ROOTFS_LIVE
+	bool "live root filesystem"
+	help
+	  Create a live image of the root filesystem that is directly
+	  usable as over NFS or chroot.
+
+config BR2_TARGET_ROOTFS_LIVE_DEST
+	string "live image location"
+	depends on BR2_TARGET_ROOTFS_LIVE
+	default "$(BINARIES_DIR)/live"
+	help
+	  The directory where the image should be stored.
+	  This directory will be emptied and recreated
+
diff --git a/fs/live/live.mk b/fs/live/live.mk
new file mode 100644
index 0000000..52f7444
--- /dev/null
+++ b/fs/live/live.mk
@@ -0,0 +1,19 @@ 
+#############################################################
+#
+# Build the live root filesystem directory
+#
+#############################################################
+
+
+define ROOTFS_LIVE_CMD
+	sudo rsync -a --no-o --no-g --delete-during $(TARGET_DIR)/ \
+		$(BR2_TARGET_ROOTFS_LIVE_DEST)/
+endef
+
+define ROOTFS_LIVE_INIT
+  if [ -z $(shell which sudo) ] ; then echo "sudo seems to not be installed on the host system" ; false ; fi 
+endef
+
+ROOTFS_LIVE_PRE_GEN_HOOKS += ROOTFS_LIVE_INIT
+
+$(eval $(call ROOTFS_TARGET,live))
diff --git a/support/dependencies/dependencies.sh b/support/dependencies/dependencies.sh
index 7a02512..ebaabbb 100755
--- a/support/dependencies/dependencies.sh
+++ b/support/dependencies/dependencies.sh
@@ -158,6 +158,7 @@  if grep ^BR2_TOOLCHAIN_BUILDROOT=y $CONFIG_FILE > /dev/null && \
        exit 1 ;
    fi
 fi
+
 if grep -q ^BR2_PACKAGE_CLASSPATH=y $CONFIG_FILE ; then
     for prog in javac jar; do
 	if ! which $prog > /dev/null ; then
@@ -166,3 +167,10 @@  if grep -q ^BR2_PACKAGE_CLASSPATH=y $CONFIG_FILE ; then
 	fi
     done
 fi
+
+if grep ^BR2_TARGET_ROOTFS_LIVE=y $CONFIG_FILE > /dev/null ; then
+   if ! which sudo > /dev/null ; then
+       /bin/echo -e "\nYou need sudo installed on your build machine to build a live filesystem\n"
+       exit 1 ;
+   fi
+fi