[RESEND,1/2] fs/ubi: decouple ubi & ubifs

Message ID 1521829279-5156-2-git-send-email-ben.whitten@gmail.com
State New
Headers show
Series
  • Verified boot tooling
Related show

Commit Message

Ben Whitten March 23, 2018, 6:21 p.m.
As ubi is a container it shouldn't enforce a ubifs rootfs to be built,
it is reasonable to have a squashfs ro filesystem within a ubi.
If ubifs is selected ubi containing ubifs is automatically selected as
to not break existing configs

Signed-off-by: Ben Whitten <ben.whitten@gmail.com>
---
 fs/Config.in       |  3 ++-
 fs/ubi/Config.in   | 34 +++++++++++++++++++++++++++++++---
 fs/ubi/ubi.mk      | 14 +++++++++++++-
 fs/ubi/ubinize.cfg |  4 ++--
 fs/ubifs/Config.in |  2 ++
 5 files changed, 50 insertions(+), 7 deletions(-)

Comments

Matt Weber March 24, 2018, 3:17 a.m. | #1
Ben,

On Fri, Mar 23, 2018 at 1:21 PM, Ben Whitten <ben.whitten@gmail.com> wrote:
> As ubi is a container it shouldn't enforce a ubifs rootfs to be built,
> it is reasonable to have a squashfs ro filesystem within a ubi.
> If ubifs is selected ubi containing ubifs is automatically selected as
> to not break existing configs

What are you thoughts on the usecase of needing the Kernel/DTB FIT (or
other firmware) images also in UBI?

i.e. when online creating the UBI partitions, I can see creating a
single UBI and then just treating it like a raw device to store a
series of blobs (firmware, rootfs & kernel).  Generation of something
like that offline feels like a post image activity.   However, I guess
that case could just be a new image containerization option which acts
like a general hook to take a series of files (for each create a UBI
container).  On target for that option, there would be more MTD
devices vs a single (I'd have to revisit the limitations of a single
UBI vs one for each image as I'm unsure of which is better).

I wonder how often a design would end up using a combination of a post
script and buildroot doing UBI generation where they have to maintain
both?  Depending on that,  I think I'd argue for defaulting to a post
script

Matt
Ben Whitten March 26, 2018, 9:05 p.m. | #2
On 24 March 2018 at 03:17, Matthew Weber
<matthew.weber@rockwellcollins.com> wrote:
> Ben,
>
> On Fri, Mar 23, 2018 at 1:21 PM, Ben Whitten <ben.whitten@gmail.com> wrote:
>> As ubi is a container it shouldn't enforce a ubifs rootfs to be built,
>> it is reasonable to have a squashfs ro filesystem within a ubi.
>> If ubifs is selected ubi containing ubifs is automatically selected as
>> to not break existing configs
>
> What are you thoughts on the usecase of needing the Kernel/DTB FIT (or
> other firmware) images also in UBI?
>
> i.e. when online creating the UBI partitions, I can see creating a
> single UBI and then just treating it like a raw device to store a
> series of blobs (firmware, rootfs & kernel).  Generation of something
> like that offline feels like a post image activity.

So in my context the final 'image' is something I would deliver to our
factory for
flashing in production. In this case our UBI needs it all, kernel a/b
rootfs a/b etc.
And its the individual artifacts that sw update gets generated from.

> However, I guess
> that case could just be a new image containerization option which acts
> like a general hook to take a series of files (for each create a UBI
> container).  On target for that option, there would be more MTD
> devices vs a single (I'd have to revisit the limitations of a single
> UBI vs one for each image as I'm unsure of which is better).

It makes sense to have the UBI as large as possible to take advantage of wear
leveling across the full flash, instead of artificially limiting their
range. Thats what
the volumes would be for within the UBI.

> I wonder how often a design would end up using a combination of a post
> script and buildroot doing UBI generation where they have to maintain
> both?  Depending on that,  I think I'd argue for defaulting to a post
> script

True. I think justing having BR generate artifacts then have genimage assemble
to what ever format required seems to be the defacto solution.
Perhaps that could be integrated as an option instead of having to call it in
numerous boards as a post script.

Ben
Arnout Vandecappelle March 31, 2018, 9:35 a.m. | #3
Hi Ben,

On 26-03-18 23:05, Ben Whitten wrote:
> On 24 March 2018 at 03:17, Matthew Weber
> <matthew.weber@rockwellcollins.com> wrote:
>> Ben,
>>
>> On Fri, Mar 23, 2018 at 1:21 PM, Ben Whitten <ben.whitten@gmail.com> wrote:
>>> As ubi is a container it shouldn't enforce a ubifs rootfs to be built,
>>> it is reasonable to have a squashfs ro filesystem within a ubi.
>>> If ubifs is selected ubi containing ubifs is automatically selected as
>>> to not break existing configs
>>
>> What are you thoughts on the usecase of needing the Kernel/DTB FIT (or
>> other firmware) images also in UBI?
>>
>> i.e. when online creating the UBI partitions, I can see creating a
>> single UBI and then just treating it like a raw device to store a
>> series of blobs (firmware, rootfs & kernel).  Generation of something
>> like that offline feels like a post image activity.
> 
> So in my context the final 'image' is something I would deliver to our
> factory for
> flashing in production. In this case our UBI needs it all, kernel a/b
> rootfs a/b etc.

 I generally find it easier in practice to format the UBI partition "online" in
a production script, and then add the UBI(FS) volumes to it one by one. That
makes it easier to make sure the UBI extends to the full partition. Otherwise,
it depends a bit on your flasher what is done with bad blocks in your image.

> And its the individual artifacts that sw update gets generated from.
> 
>> However, I guess
>> that case could just be a new image containerization option which acts
>> like a general hook to take a series of files (for each create a UBI
>> container).  On target for that option, there would be more MTD
>> devices vs a single (I'd have to revisit the limitations of a single
>> UBI vs one for each image as I'm unsure of which is better).
> 
> It makes sense to have the UBI as large as possible to take advantage of wear
> leveling across the full flash, instead of artificially limiting their
> range. Thats what
> the volumes would be for within the UBI.
> 
>> I wonder how often a design would end up using a combination of a post
>> script and buildroot doing UBI generation where they have to maintain
>> both?  Depending on that,  I think I'd argue for defaulting to a post
>> script
> 
> True. I think justing having BR generate artifacts then have genimage assemble
> to what ever format required seems to be the defacto solution.
> Perhaps that could be integrated as an option instead of having to call it in
> numerous boards as a post script.

 Well, you can usually use support/scripts/genimage.sh directly as the
post-image script and just set the appropriate options in
BR2_ROOTFS_POST_SCRIPT_ARGS. Like is done in numerous defconfigs.

 Regards,
 Arnout

Patch

diff --git a/fs/Config.in b/fs/Config.in
index c25b01c..771220b 100644
--- a/fs/Config.in
+++ b/fs/Config.in
@@ -11,8 +11,9 @@  source "fs/jffs2/Config.in"
 source "fs/romfs/Config.in"
 source "fs/squashfs/Config.in"
 source "fs/tar/Config.in"
-source "fs/ubi/Config.in"
 source "fs/ubifs/Config.in"
 source "fs/yaffs2/Config.in"
 
+comment "Container"
+source "fs/ubi/Config.in"
 endmenu
diff --git a/fs/ubi/Config.in b/fs/ubi/Config.in
index 5fa2f1f..ae91aa2 100644
--- a/fs/ubi/Config.in
+++ b/fs/ubi/Config.in
@@ -1,11 +1,30 @@ 
 config BR2_TARGET_ROOTFS_UBI
-	bool "ubi image containing an ubifs root filesystem"
-	select BR2_TARGET_ROOTFS_UBIFS
+	bool "ubi image containing a root filesystem"
 	help
-	  Build an ubi image from the ubifs one (with ubinize).
+	  Build a ubi image from the selected filesystems (with ubinize).
 
 if BR2_TARGET_ROOTFS_UBI
 
+choice
+	prompt "UBI Container"
+	default BR2_TARGET_ROOTFS_UBIFS
+
+config BR2_TARGET_ROOTFS_UBI_CONTAINS_UBIFS
+	bool "containing a ubifs filesystem"
+	depends on BR2_TARGET_ROOTFS_UBIFS
+	help
+	  Select this option to have the ubi contain the ubifs filesystem
+	  this is the default behaviour.
+
+config BR2_TARGET_ROOTFS_UBI_CONTAINS_SQUASHFS
+	bool "containing a squashfs filesystem"
+	depends on BR2_TARGET_ROOTFS_SQUASHFS
+	help
+	  Select this option to have the ubi contain the squashfs
+	  filesystem. Useful for readonly systems.
+
+endchoice
+
 config BR2_TARGET_ROOTFS_UBI_PEBSIZE
 	hex "physical eraseblock size"
 	default 0x20000
@@ -14,6 +33,15 @@  config BR2_TARGET_ROOTFS_UBI_PEBSIZE
 	  flash chip the ubi image is created for. The value provided
 	  here is passed to the -p/--peb-size option of ubinize.
 
+config BR2_TARGET_ROOTFS_UBI_MINIOSIZE
+        hex "minimum I/O unit size"
+	depends on !BR2_TARGET_ROOTFS_UBIFS
+        default 0x800
+        help
+          Minimum I/O unit size. The value provided here is passed
+          to the -m/--min-io-size option of ubinize.
+
+
 config BR2_TARGET_ROOTFS_UBI_SUBSIZE
 	int "sub-page size"
 	default 512
diff --git a/fs/ubi/ubi.mk b/fs/ubi/ubi.mk
index c78feda..2532a02 100644
--- a/fs/ubi/ubi.mk
+++ b/fs/ubi/ubi.mk
@@ -4,7 +4,12 @@ 
 #
 ################################################################################
 
+ifneq ($(BR2_TARGET_ROOTFS_UBIFS),y)
+UBI_UBINIZE_OPTS := -m $(BR2_TARGET_ROOTFS_UBI_MINIOSIZE)
+else
 UBI_UBINIZE_OPTS := -m $(BR2_TARGET_ROOTFS_UBIFS_MINIOSIZE)
+endif
+
 UBI_UBINIZE_OPTS += -p $(BR2_TARGET_ROOTFS_UBI_PEBSIZE)
 ifneq ($(BR2_TARGET_ROOTFS_UBI_SUBSIZE),0)
 UBI_UBINIZE_OPTS += -s $(BR2_TARGET_ROOTFS_UBI_SUBSIZE)
@@ -12,7 +17,14 @@  endif
 
 UBI_UBINIZE_OPTS += $(call qstrip,$(BR2_TARGET_ROOTFS_UBI_OPTS))
 
+ifeq ($(BR2_TARGET_ROOTFS_UBI_CONTAINS_UBIFS),y)
 ROOTFS_UBI_DEPENDENCIES = rootfs-ubifs
+ROOTFS_UBI_ROOTFS_EXT = .ubifs
+else ifeq ($(BR2_TARGET_ROOTFS_UBI_CONTAINS_SQUASHFS),y)
+ROOTFS_UBI_DEPENDENCIES = rootfs-squashfs
+ROOTFS_UBI_ROOTFS_EXT = .squashfs
+endif
+ROOTFS_UBI_DEPENDENCIES += host-mtd
 
 ifeq ($(BR2_TARGET_ROOTFS_UBI_USE_CUSTOM_CONFIG),y)
 UBINIZE_CONFIG_FILE_PATH = $(call qstrip,$(BR2_TARGET_ROOTFS_UBI_CUSTOM_CONFIG_FILE))
@@ -23,7 +35,7 @@  endif
 # don't use sed -i as it misbehaves on systems with SELinux enabled when this is
 # executed through fakeroot (see #9386)
 define ROOTFS_UBI_CMD
-	sed 's;BR2_ROOTFS_UBIFS_PATH;$@fs;' \
+	sed 's;BR2_ROOTFS_PATH;$(BINARIES_DIR)/rootfs$(ROOTFS_UBI_ROOTFS_EXT);' \
 		$(UBINIZE_CONFIG_FILE_PATH) > $(BUILD_DIR)/ubinize.cfg
 	$(HOST_DIR)/sbin/ubinize -o $@ $(UBI_UBINIZE_OPTS) $(BUILD_DIR)/ubinize.cfg
 	rm $(BUILD_DIR)/ubinize.cfg
diff --git a/fs/ubi/ubinize.cfg b/fs/ubi/ubinize.cfg
index b0aace7..66e4a0f 100644
--- a/fs/ubi/ubinize.cfg
+++ b/fs/ubi/ubinize.cfg
@@ -1,8 +1,8 @@ 
-[ubifs]
+[rootfs]
 mode=ubi
 vol_id=0
 vol_type=dynamic
 vol_name=rootfs
 vol_alignment=1
 vol_flags=autoresize
-image=BR2_ROOTFS_UBIFS_PATH
+image=BR2_ROOTFS_PATH
diff --git a/fs/ubifs/Config.in b/fs/ubifs/Config.in
index e79ab9a..30929ee 100644
--- a/fs/ubifs/Config.in
+++ b/fs/ubifs/Config.in
@@ -1,5 +1,7 @@ 
 config BR2_TARGET_ROOTFS_UBIFS
 	bool "ubifs root filesystem"
+	depends on BR2_TARGET_ROOTFS_UBI
+	select BR2_TARGET_ROOTFS_UBI_CONTAINS_UBIFS
 	help
 	  Build a ubifs root filesystem