| Message ID | 1379215295-1805-1-git-send-email-danomimanchego123@gmail.com |
|---|---|
| State | Superseded |
| Headers | show |
Hi Danomi, On Sat, Sep 14, 2013 at 11:21:35PM -0400, Danomi Manchego wrote: > By default, the UBI FS target creates a ubinize configuration > file on-the-fly, for a single volume. Add an option to specify > a custom config file. > > E.g., one might want to deploy a system with a volume for the > built ubifs image, a volume for a future upgrade image, and a > volume for user data. > > Signed-off-by: Danomi Manchego <danomimanchego123@gmail.com> > --- > fs/ubifs/Config.in | 12 ++++++++++++ > fs/ubifs/ubi.mk | 12 +++++++++--- > 2 files changed, 21 insertions(+), 3 deletions(-) > > diff --git a/fs/ubifs/Config.in b/fs/ubifs/Config.in > index 70ea81e..32ca730 100644 > --- a/fs/ubifs/Config.in > +++ b/fs/ubifs/Config.in > @@ -119,6 +119,18 @@ config BR2_TARGET_ROOTFS_UBI_SUBSIZE > The value provided here is passed to the -s/--sub-page-size > option of ubinize. > > +config BR2_TARGET_ROOTFS_UBI_USE_CUSTOM_CONFIG > + bool "Use custom config file" > + help > + Select to use a custom volume configuration file. Otherwise, > + a configuration file will be generated automatically. > + > +config BR2_TARGET_ROOTFS_UBI_CUSTOM_CONFIG_FILE > + string "Configuration file path" > + depends on BR2_TARGET_ROOTFS_UBI_USE_CUSTOM_CONFIG > + help > + Path to the volume configuration file > + > config BR2_TARGET_ROOTFS_UBI_OPTS > string "Additional ubinize options" > help > diff --git a/fs/ubifs/ubi.mk b/fs/ubifs/ubi.mk > index 08c952c..1304bd5 100644 > --- a/fs/ubifs/ubi.mk > +++ b/fs/ubifs/ubi.mk > @@ -14,10 +14,16 @@ UBI_UBINIZE_OPTS += $(call qstrip,$(BR2_TARGET_ROOTFS_UBI_OPTS)) > > ROOTFS_UBI_DEPENDENCIES = rootfs-ubifs > > +ifeq ($(BR2_TARGET_ROOTFS_UBI_USE_CUSTOM_CONFIG),y) > +UBI_UBINIZE_PREP_CFG = \ > + cp $(BR2_TARGET_ROOTFS_UBI_CUSTOM_CONFIG_FILE) ./ubinize.cfg You should check whether BR2_TARGET_ROOTFS_UBI_CUSTOM_CONFIG_FILE is an empty string. > +else > +UBI_UBINIZE_PREP_CFG = \ > + cp fs/ubifs/ubinize.cfg . ; echo "image=$@fs" >> ./ubinize.cfg > +endif > + > define ROOTFS_UBI_CMD > - cp fs/ubifs/ubinize.cfg . ;\ > - echo "image=$@fs" \ > - >> ./ubinize.cfg ;\ > + $(UBI_UBINIZE_PREP_CFG) ; \ > $(HOST_DIR)/usr/sbin/ubinize -o $@ $(UBI_UBINIZE_OPTS) ubinize.cfg ;\ > rm ubinize.cfg > endef > -- > 1.7.9.5
Hi Baruch, On Sun, Sep 15, 2013 at 12:38 AM, Baruch Siach <baruch@tkos.co.il> wrote: > > +ifeq ($(BR2_TARGET_ROOTFS_UBI_USE_CUSTOM_CONFIG),y) > > +UBI_UBINIZE_PREP_CFG = \ > > + cp $(BR2_TARGET_ROOTFS_UBI_CUSTOM_CONFIG_FILE) ./ubinize.cfg > > You should check whether BR2_TARGET_ROOTFS_UBI_CUSTOM_CONFIG_FILE is an empty > string. I was trying to follow the pattern in linux.mk and uboot.mk: ifeq ($(BR2_LINUX_KERNEL_IMAGE_TARGET_CUSTOM),y) LINUX_IMAGE_NAME=$(call qstrip,$(BR2_LINUX_KERNEL_IMAGE_TARGET_NAME)) and else ifeq ($(BR2_TARGET_UBOOT_FORMAT_CUSTOM),y) UBOOT_BIN = $(call qstrip,$(BR2_TARGET_UBOOT_FORMAT_CUSTOM_NAME)) But maybe they are not perfect analogies, since those custom entries are one of N choices in big if/ese-if tests. It would be easy to change my single if(...,y) test to check for a non-empty file string. I'll wait and see if there are any other comments, then send a v2. Thanks, Danomi -
Dear Danomi Manchego, On Sat, 14 Sep 2013 23:21:35 -0400, Danomi Manchego wrote: > By default, the UBI FS target creates a ubinize configuration > file on-the-fly, for a single volume. Add an option to specify > a custom config file. > > E.g., one might want to deploy a system with a volume for the > built ubifs image, a volume for a future upgrade image, and a > volume for user data. > > Signed-off-by: Danomi Manchego <danomimanchego123@gmail.com> I believe I'm ok with the general idea. Some comments below, though. > +config BR2_TARGET_ROOTFS_UBI_USE_CUSTOM_CONFIG > + bool "Use custom config file" > + help > + Select to use a custom volume configuration file. Otherwise, > + a configuration file will be generated automatically. I'd like to have a bit more documentation here, and especially explicitly mention that we're talking about the configuration file passed to the 'ubinize' tool. Also, I'm not sure the name 'volume configuration file' is correct: the ubinize configuration file does not describe the configuration of one volume, but rather the configuration of all volumes that will be contained in an UBI "space". Some maybe something: Select this option to use a custom ubinize configuration file, rather than the default configuration used by Buildroot (which consists in generating an UBI image that contains a single volume, itself containing the root filesystem). Passing a custom ubinize configuration file will allow you to create several volumes. However, I'd like to add more detail on using the rootfs UBIFS image in a volume (i.e $@fs in the existing code), but I'm not sure what suggestion to give to the user: i.e, what image= line should he put in his custom configuration file? > +config BR2_TARGET_ROOTFS_UBI_CUSTOM_CONFIG_FILE > + string "Configuration file path" > + depends on BR2_TARGET_ROOTFS_UBI_USE_CUSTOM_CONFIG > + help > + Path to the volume configuration file Path to the ubinize configuration file. > + > config BR2_TARGET_ROOTFS_UBI_OPTS > string "Additional ubinize options" > help > diff --git a/fs/ubifs/ubi.mk b/fs/ubifs/ubi.mk > index 08c952c..1304bd5 100644 > --- a/fs/ubifs/ubi.mk > +++ b/fs/ubifs/ubi.mk > @@ -14,10 +14,16 @@ UBI_UBINIZE_OPTS += $(call qstrip,$(BR2_TARGET_ROOTFS_UBI_OPTS)) > > ROOTFS_UBI_DEPENDENCIES = rootfs-ubifs > > +ifeq ($(BR2_TARGET_ROOTFS_UBI_USE_CUSTOM_CONFIG),y) > +UBI_UBINIZE_PREP_CFG = \ > + cp $(BR2_TARGET_ROOTFS_UBI_CUSTOM_CONFIG_FILE) ./ubinize.cfg > +else > +UBI_UBINIZE_PREP_CFG = \ > + cp fs/ubifs/ubinize.cfg . ; echo "image=$@fs" >> ./ubinize.cfg > +endif > + > define ROOTFS_UBI_CMD > - cp fs/ubifs/ubinize.cfg . ;\ Am I looking wrong, or are we copying ubinize.cfg to the root of the Buildroot source tree? If that's the case, then it's really really wrong, as the source tree should be read-only. Instead, we should copy it to some location in $(O). > - echo "image=$@fs" \ > - >> ./ubinize.cfg ;\ > + $(UBI_UBINIZE_PREP_CFG) ; \ > $(HOST_DIR)/usr/sbin/ubinize -o $@ $(UBI_UBINIZE_OPTS) ubinize.cfg ;\ > rm ubinize.cfg > endef Thomas
Thomas, On Sun, Sep 15, 2013 at 2:00 PM, Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote: >> +config BR2_TARGET_ROOTFS_UBI_USE_CUSTOM_CONFIG >> + bool "Use custom config file" >> + help >> + Select to use a custom volume configuration file. Otherwise, >> + a configuration file will be generated automatically. > > I'd like to have a bit more documentation here, and especially > explicitly mention that we're talking about the configuration file > passed to the 'ubinize' tool. Also, I'm not sure the name 'volume > configuration file' is correct: the ubinize configuration file does not > describe the configuration of one volume, but rather the configuration > of all volumes that will be contained in an UBI "space". > > Some maybe something: > > Select this option to use a custom ubinize configuration file, > rather than the default configuration used by Buildroot (which > consists in generating an UBI image that contains a single > volume, itself containing the root filesystem). Passing a > custom ubinize configuration file will allow you to create > several volumes. > > However, I'd like to add more detail on using the rootfs UBIFS image in > a volume (i.e $@fs in the existing code), but I'm not sure what > suggestion to give to the user: i.e, what image= line should he put in > his custom configuration file? What I found was that the image path is relative to the $(TOPDIR). So if you are providing your own file, you need to know your $(O) and either specify a path relative to $(TOPDIR), or an absolute path. i played with the idea of having a sed line or something to substitute in a file path, but decided against. I though that if you are providing your own config file, then it could refer to multiple images in different output directories (for whatever reason), in which case you have to know where your files are anyway. Maybe the description needs something like: >> +config BR2_TARGET_ROOTFS_UBI_CUSTOM_CONFIG_FILE >> + string "Configuration file path" >> + depends on BR2_TARGET_ROOTFS_UBI_USE_CUSTOM_CONFIG >> + help >> + Path to the volume configuration file > > Path to the ubinize configuration file. Okay. >> +ifeq ($(BR2_TARGET_ROOTFS_UBI_USE_CUSTOM_CONFIG),y) >> +UBI_UBINIZE_PREP_CFG = \ >> + cp $(BR2_TARGET_ROOTFS_UBI_CUSTOM_CONFIG_FILE) ./ubinize.cfg >> +else >> +UBI_UBINIZE_PREP_CFG = \ >> + cp fs/ubifs/ubinize.cfg . ; echo "image=$@fs" >> ./ubinize.cfg >> +endif >> + >> define ROOTFS_UBI_CMD >> - cp fs/ubifs/ubinize.cfg . ;\ > > Am I looking wrong, or are we copying ubinize.cfg to the root of the > Buildroot source tree? If that's the case, then it's really really > wrong, as the source tree should be read-only. Instead, we should copy > it to some location in $(O). Correct - that's the way it was, so I didn't change it. But it would be easy to change "./ubinize.cfg" to "$(O)/ubinize.cfg". I'll do that.
Dear Danomi Manchego, On Sun, 15 Sep 2013 14:37:21 -0400, Danomi Manchego wrote: > > However, I'd like to add more detail on using the rootfs UBIFS > > image in a volume (i.e $@fs in the existing code), but I'm not sure > > what suggestion to give to the user: i.e, what image= line should > > he put in his custom configuration file? > > What I found was that the image path is relative to the $(TOPDIR). So > if you are providing your own file, you need to know your $(O) and > either specify a path relative to $(TOPDIR), or an absolute path. i > played with the idea of having a sed line or something to substitute > in a file path, but decided against. I though that if you are > providing your own config file, then it could refer to multiple images > in different output directories (for whatever reason), in which case > you have to know where your files are anyway. Using sed sounds like a good idea. Just provide a keyword that the user can use in his custom configuration file, and that Buildroot will automatically replace by the path to the UBIFS root filesystem image. Of course, for the other volumes, the user can specify whichever image he wants. So, the user could write something like: [rootfs] mode=ubi vol_id=0 vol_type=dynamic vol_name=rootfs vol_alignment=1 vol_flags=autoresize image=BR2_ROOTFS_UBIFS_PATH [datafs] mode=ubi vol_id=1 vol_type=dynamic vol_name=datafs vol_alignment=1 vol_flags=autoresize image=/some/other/location.img and Buildroot replaces BR2_ROOTFS_UBIFS_PATH by the proper $(O)/images/ubifs.img. Note that this could also be used in the fs/ubifs/ubinize.cfg file, therefore making the default and custom cases more similar: ifeq ($(BR2_TARGET_ROOTFS_UBI_USE_CUSTOM_CONFIG),y) UBINIZE_CONFIG_FILE_PATH = $(call qstrip,$(BR2_TARGET_ROOTFS_UBI_CUSTOM_CONFIG_FILE)) else UBINIZE_CONFIG_FILE_PATH = fs/ubifs/ubinize.cfg endef define ROOTFS_UBI_CMD cp $(UBINIZE_CONFIG_FILE_PATH) $(BUILD_DIR)/ubinize.cfg $(SED) 's/BR2_ROOTFS_UBIFS_PATH/$@fs/' $(BUILD_DIR)/ubinize.cfg $(HOST_DIR)/usr/sbin/ubinize -o $@ $(UBI_UBINIZE_OPTS) $(BUILD_DIR)/ubinize.cfg rm $(BUILD_DIR)/ubinize.cfg endef Of course, the BR2_ROOTFS_UBIFS_PATH trick would have to be properly documented in the configuration option help text. Best regards, Thomas
On Sun, Sep 15, 2013 at 2:53 PM, Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote: > and Buildroot replaces BR2_ROOTFS_UBIFS_PATH by the proper > $(O)/images/ubifs.img. > > Note that this could also be used in the fs/ubifs/ubinize.cfg file, > therefore making the default and custom cases more similar: > > ifeq ($(BR2_TARGET_ROOTFS_UBI_USE_CUSTOM_CONFIG),y) > UBINIZE_CONFIG_FILE_PATH = $(call qstrip,$(BR2_TARGET_ROOTFS_UBI_CUSTOM_CONFIG_FILE)) > else > UBINIZE_CONFIG_FILE_PATH = fs/ubifs/ubinize.cfg > endef > > define ROOTFS_UBI_CMD > cp $(UBINIZE_CONFIG_FILE_PATH) $(BUILD_DIR)/ubinize.cfg > $(SED) 's/BR2_ROOTFS_UBIFS_PATH/$@fs/' $(BUILD_DIR)/ubinize.cfg > $(HOST_DIR)/usr/sbin/ubinize -o $@ $(UBI_UBINIZE_OPTS) $(BUILD_DIR)/ubinize.cfg > rm $(BUILD_DIR)/ubinize.cfg > endef That does appeal to me; I think that we can combine the cp and the sed lines as well ... $(SED) 's/BR2_ROOTFS_UBIFS_PATH/$@fs/' $(UBINIZE_CONFIG_FILE_PATH) > $(BUILD_DIR)/ubinize.cfg Cleaner all the way around. I'll look into it tonight. Thanks, Danomi -
Dear Danomi Manchego, On Sun, 15 Sep 2013 15:08:28 -0400, Danomi Manchego wrote: > That does appeal to me; I think that we can combine the cp and the sed > lines as well ... > > $(SED) 's/BR2_ROOTFS_UBIFS_PATH/$@fs/' > $(UBINIZE_CONFIG_FILE_PATH) > $(BUILD_DIR)/ubinize.cfg That won't work. $(SED) expands to sed -e -i, i.e it does the sed in-place. Thomas
Hi Thomas and Danomi, On 15 September 2013 16:16, Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote: [...] > On Sun, 15 Sep 2013 15:08:28 -0400, Danomi Manchego wrote: > >> That does appeal to me; I think that we can combine the cp and the sed >> lines as well ... >> >> $(SED) 's/BR2_ROOTFS_UBIFS_PATH/$@fs/' >> $(UBINIZE_CONFIG_FILE_PATH) > $(BUILD_DIR)/ubinize.cfg > > That won't work. $(SED) expands to sed -e -i, i.e it does the sed > in-place. > What's the status of this? It would be interesting to pick it up. I'm currently using a post-image script to create UBI images (using ubinize). On the other side, I'm not using ubifs but rather squashfs on top of ubiblock [1]. I was thinking about keeping the post-image script approach, which means I need host mtd/ubi tools. Thomas: Do you think adding mtd/ubi host tools is a sane approach? Something like what Yann recently did for squashfs [2]. Speaking of Yann's, I've also considered an extension of his fs/custom work. Yann: How do you think that would fit? Ideas? Just for reference, here's how my post-image script looks like: # Create UBI rootfs stuff # mkdir -p board/ppst/linux/user fallocate -l 10M board/ppst/linux/user/vfat.img mkfs.vfat board/ppst/linux/user/vfat.img mkfs.ubifs -r board/ppst/linux/user/ -m 2048 -o user.ubifs -e 126976 -c 100 mkfs.ubifs -r board/ppst/linux/config/ -m 2048 -o config.ubifs -e 126976 -c 100 ubinize -o output/images/rootfs.ubi.img -p 128KiB -m 2048 -O 2048 board/ppst/linux/ubinize-rootfs.cfg rm user.ubifs config.ubifs # Create UBI kernel # ubinize -o output/images/linux.ubi.img -p 128KiB -m 2048 -O 2048 board/ppst/linux/ubinize-linux.cfg [1] http://git.free-electrons.com/users/ezequiel-garcia/linux/log/?h=ubiblock-v4-cache-optional [2] http://patchwork.ozlabs.org/patch/306629/
Ezequiel, On Mon, Jan 27, 2014 at 6:17 PM, Ezequiel García <ezequiel@vanguardiasur.com.ar> wrote: > Hi Thomas and Danomi, > > On 15 September 2013 16:16, Thomas Petazzoni > <thomas.petazzoni@free-electrons.com> wrote: > [...] >> On Sun, 15 Sep 2013 15:08:28 -0400, Danomi Manchego wrote: >> >>> That does appeal to me; I think that we can combine the cp and the sed >>> lines as well ... >>> >>> $(SED) 's/BR2_ROOTFS_UBIFS_PATH/$@fs/' >>> $(UBINIZE_CONFIG_FILE_PATH) > $(BUILD_DIR)/ubinize.cfg >> >> That won't work. $(SED) expands to sed -e -i, i.e it does the sed >> in-place. >> > > What's the status of this? It would be interesting to pick it up. A slightly renamed v2 of this patch was submitted in September, found at: http://patchwork.ozlabs.org/patch/275088/ I applied the patch to the buildroot that we use at work, and have been using it quite regularly to produce rootfs.ubi images. No issues encountered. Like a child at Christmas, I am have been waiting for the 10-oldest-patches cycle to reach Sept. 2013 to see if this patch finally will be accepted. Danomi -
Dear Ezequiel García,
On Mon, 27 Jan 2014 21:17:05 -0200, Ezequiel García wrote:
> What's the status of this? It would be interesting to pick it up.
As Danomi said, a new version was posted at
http://patchwork.ozlabs.org/patch/275088/. It is waiting for you to
give your Tested-by about it :-)
Thanks!
Thomas
On 28 January 2014 13:35, Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote: > On Mon, 27 Jan 2014 21:17:05 -0200, Ezequiel García wrote: > >> What's the status of this? It would be interesting to pick it up. > > As Danomi said, a new version was posted at > http://patchwork.ozlabs.org/patch/275088/. It is waiting for you to > give your Tested-by about it :-) > Sure, I will.
On 2014-01-27 21:17 -0200, Ezequiel García spake thusly: > Hi Thomas and Danomi, > > On 15 September 2013 16:16, Thomas Petazzoni > <thomas.petazzoni@free-electrons.com> wrote: > [...] > > On Sun, 15 Sep 2013 15:08:28 -0400, Danomi Manchego wrote: > > > >> That does appeal to me; I think that we can combine the cp and the sed > >> lines as well ... > >> > >> $(SED) 's/BR2_ROOTFS_UBIFS_PATH/$@fs/' > >> $(UBINIZE_CONFIG_FILE_PATH) > $(BUILD_DIR)/ubinize.cfg > > > > That won't work. $(SED) expands to sed -e -i, i.e it does the sed > > in-place. > > > > What's the status of this? It would be interesting to pick it up. IIRC, I have it in my tree somewhere... I'll update and re-submit... [--SNIP--] > Thomas: Do you think adding mtd/ubi host tools is a sane approach? > Something like what Yann recently did for squashfs [2]. I think it would be a good idea, since it is perfectly valid that a post-build script wants to call the UBI tools. > Speaking of Yann's, I've also considered an extension of his fs/custom work. > Yann: How do you think that would fit? Ideas? I don't know much about UBI, but I don't see why we could not add it to the genimages infra. > Just for reference, here's how my post-image script looks like: > > # Create UBI rootfs stuff > # > mkdir -p board/ppst/linux/user > fallocate -l 10M board/ppst/linux/user/vfat.img > mkfs.vfat board/ppst/linux/user/vfat.img > mkfs.ubifs -r board/ppst/linux/user/ -m 2048 -o user.ubifs -e 126976 -c 100 > mkfs.ubifs -r board/ppst/linux/config/ -m 2048 -o config.ubifs -e 126976 -c 100 > ubinize -o output/images/rootfs.ubi.img -p 128KiB -m 2048 -O 2048 > board/ppst/linux/ubinize-rootfs.cfg > rm user.ubifs config.ubifs > > # Create UBI kernel > # > ubinize -o output/images/linux.ubi.img -p 128KiB -m 2048 -O 2048 > board/ppst/linux/ubinize-linux.cfg All you do above could probably be added easily to genimages, it seems. Regards, Yann E. MORIN.
diff --git a/fs/ubifs/Config.in b/fs/ubifs/Config.in index 70ea81e..32ca730 100644 --- a/fs/ubifs/Config.in +++ b/fs/ubifs/Config.in @@ -119,6 +119,18 @@ config BR2_TARGET_ROOTFS_UBI_SUBSIZE The value provided here is passed to the -s/--sub-page-size option of ubinize. +config BR2_TARGET_ROOTFS_UBI_USE_CUSTOM_CONFIG + bool "Use custom config file" + help + Select to use a custom volume configuration file. Otherwise, + a configuration file will be generated automatically. + +config BR2_TARGET_ROOTFS_UBI_CUSTOM_CONFIG_FILE + string "Configuration file path" + depends on BR2_TARGET_ROOTFS_UBI_USE_CUSTOM_CONFIG + help + Path to the volume configuration file + config BR2_TARGET_ROOTFS_UBI_OPTS string "Additional ubinize options" help diff --git a/fs/ubifs/ubi.mk b/fs/ubifs/ubi.mk index 08c952c..1304bd5 100644 --- a/fs/ubifs/ubi.mk +++ b/fs/ubifs/ubi.mk @@ -14,10 +14,16 @@ UBI_UBINIZE_OPTS += $(call qstrip,$(BR2_TARGET_ROOTFS_UBI_OPTS)) ROOTFS_UBI_DEPENDENCIES = rootfs-ubifs +ifeq ($(BR2_TARGET_ROOTFS_UBI_USE_CUSTOM_CONFIG),y) +UBI_UBINIZE_PREP_CFG = \ + cp $(BR2_TARGET_ROOTFS_UBI_CUSTOM_CONFIG_FILE) ./ubinize.cfg +else +UBI_UBINIZE_PREP_CFG = \ + cp fs/ubifs/ubinize.cfg . ; echo "image=$@fs" >> ./ubinize.cfg +endif + define ROOTFS_UBI_CMD - cp fs/ubifs/ubinize.cfg . ;\ - echo "image=$@fs" \ - >> ./ubinize.cfg ;\ + $(UBI_UBINIZE_PREP_CFG) ; \ $(HOST_DIR)/usr/sbin/ubinize -o $@ $(UBI_UBINIZE_OPTS) ubinize.cfg ;\ rm ubinize.cfg endef
By default, the UBI FS target creates a ubinize configuration file on-the-fly, for a single volume. Add an option to specify a custom config file. E.g., one might want to deploy a system with a volume for the built ubifs image, a volume for a future upgrade image, and a volume for user data. Signed-off-by: Danomi Manchego <danomimanchego123@gmail.com> --- fs/ubifs/Config.in | 12 ++++++++++++ fs/ubifs/ubi.mk | 12 +++++++++--- 2 files changed, 21 insertions(+), 3 deletions(-)