diff mbox series

[v3] package/zerofree: new package

Message ID 20220717091411.8598-1-vincent.stehle@laposte.net
State Accepted
Headers show
Series [v3] package/zerofree: new package | expand

Commit Message

Vincent Stehlé July 17, 2022, 9:14 a.m. UTC 
zerofree is a utility which scans the free blocks in an ext2 filesystem and
fills any non-zero blocks with zeroes.

https://frippery.org/uml/

Tested-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Vincent Stehlé <vincent.stehle@laposte.net>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---


Hi,

This is the third version of a patch adding a zerofree package. This version
adds a simple test, which works for me both locally and on gitlab.

As the package itself has not changed since v2[1] I have added Julien's
"Tested-by" following his e-mail[2] (thanks!).

Best regards,
Vincent.

Changes in v3:
- Add a simple test to verify that zerofree is working.

Changes in v2:
- Use a workaround to allow building with musl.

[1]: https://lists.buildroot.org/pipermail/buildroot/2022-June/645338.html
[2]: https://lists.buildroot.org/pipermail/buildroot/2022-June/645338.html


 DEVELOPERS                                    |  2 +
 package/Config.in                             |  1 +
 package/zerofree/Config.in                    | 13 ++++++
 package/zerofree/zerofree.hash                |  3 ++
 package/zerofree/zerofree.mk                  | 27 ++++++++++++
 .../testing/tests/package/test_zerofree.py    | 41 +++++++++++++++++++
 6 files changed, 87 insertions(+)
 create mode 100644 package/zerofree/Config.in
 create mode 100644 package/zerofree/zerofree.hash
 create mode 100644 package/zerofree/zerofree.mk
 create mode 100644 support/testing/tests/package/test_zerofree.py

Comments

Yann E. MORIN July 17, 2022, 12:14 p.m. UTC | #1 
Vincent, All,

On 2022-07-17 11:14 +0200, Vincent Stehlé via buildroot spake thusly:
> zerofree is a utility which scans the free blocks in an ext2 filesystem and
> fills any non-zero blocks with zeroes.
> 
> https://frippery.org/uml/
> 
> Tested-by: Julien Olivain <ju.o@free.fr>
> Signed-off-by: Vincent Stehlé <vincent.stehle@laposte.net>
> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>

Applied to master, thanks, with a few fixes (of which the review by
Bernd):

  - fix MMU dependency for comment; reword comment
  - fix multi-line assignment of ZEROFREE_CFLAGS
  - do not add comment trailing after assignment
  - extend commit log to explain why we need the workaround
  - use TARGET_CONFIGURE_OPTS, drop explicit CC=
  - install to explicit destination file

Please note that we may need to patch the Makefile, to be able to pass
the LDFLAGS.

Indeed, since the compile and link steps are done with a single call to
gcc (that's OK!), we'd be missing TARGET_LDFLAGS when tey are not empty.

However, assing those as part of the CFLAGS is not going to make it:
LDFLAGS should be after the linked libraries (e.g. in case we want to
shoe-horn a secific library for linking after -lext2).

Care to have a look at that?

Regards,
Yann E. MORIN.

> ---
> 
> 
> Hi,
> 
> This is the third version of a patch adding a zerofree package. This version
> adds a simple test, which works for me both locally and on gitlab.
> 
> As the package itself has not changed since v2[1] I have added Julien's
> "Tested-by" following his e-mail[2] (thanks!).
> 
> Best regards,
> Vincent.
> 
> Changes in v3:
> - Add a simple test to verify that zerofree is working.
> 
> Changes in v2:
> - Use a workaround to allow building with musl.
> 
> [1]: https://lists.buildroot.org/pipermail/buildroot/2022-June/645338.html
> [2]: https://lists.buildroot.org/pipermail/buildroot/2022-June/645338.html
> 
> 
>  DEVELOPERS                                    |  2 +
>  package/Config.in                             |  1 +
>  package/zerofree/Config.in                    | 13 ++++++
>  package/zerofree/zerofree.hash                |  3 ++
>  package/zerofree/zerofree.mk                  | 27 ++++++++++++
>  .../testing/tests/package/test_zerofree.py    | 41 +++++++++++++++++++
>  6 files changed, 87 insertions(+)
>  create mode 100644 package/zerofree/Config.in
>  create mode 100644 package/zerofree/zerofree.hash
>  create mode 100644 package/zerofree/zerofree.mk
>  create mode 100644 support/testing/tests/package/test_zerofree.py
> 
> diff --git a/DEVELOPERS b/DEVELOPERS
> index 71dd7d3d6c..0fc73f311f 100644
> --- a/DEVELOPERS
> +++ b/DEVELOPERS
> @@ -2917,8 +2917,10 @@ F:	configs/uevm5432_defconfig
>  F:	package/i7z/
>  F:	package/msr-tools/
>  F:	package/pixz/
> +F:	package/zerofree/
>  F:	support/testing/tests/package/test_msr_tools*
>  F:	support/testing/tests/package/test_pixz.py
> +F:	support/testing/tests/package/test_zerofree.py
>  
>  N:	Vinicius Tinti <viniciustinti@gmail.com>
>  F:	package/python-thrift/
> diff --git a/package/Config.in b/package/Config.in
> index 230aa7a2c2..3feb47c534 100644
> --- a/package/Config.in
> +++ b/package/Config.in
> @@ -239,6 +239,7 @@ menu "Filesystem and flash utilities"
>  	source "package/udftools/Config.in"
>  	source "package/unionfs/Config.in"
>  	source "package/xfsprogs/Config.in"
> +	source "package/zerofree/Config.in"
>  	source "package/zfs/Config.in"
>  endmenu
>  
> diff --git a/package/zerofree/Config.in b/package/zerofree/Config.in
> new file mode 100644
> index 0000000000..2ce2901a27
> --- /dev/null
> +++ b/package/zerofree/Config.in
> @@ -0,0 +1,13 @@
> +config BR2_PACKAGE_ZEROFREE
> +	bool "zerofree"
> +	depends on BR2_USE_MMU # e2fsprogs
> +	depends on !BR2_STATIC_LIBS
> +	select BR2_PACKAGE_E2FSPROGS
> +	help
> +	  zerofree is a utility which scans the free blocks in an ext2
> +	  filesystem and fills any non-zero blocks with zeroes.
> +
> +	  https://frippery.org/uml/
> +
> +comment "zerofree needs MMU and dynamic library"
> +	depends on !BR2_USE_MMU || BR2_STATIC_LIBS
> diff --git a/package/zerofree/zerofree.hash b/package/zerofree/zerofree.hash
> new file mode 100644
> index 0000000000..3399611787
> --- /dev/null
> +++ b/package/zerofree/zerofree.hash
> @@ -0,0 +1,3 @@
> +# Locally calculated
> +sha256  956bc861b55ba0a2b7593c58d32339dab1a0e7da6ea2b813d27c80f08b723867  zerofree-1.1.1.tgz
> +sha256  90daae00475a992a367da5b0658469a5d1c4449dbbe964c5b7246e1aec92f491  COPYING
> diff --git a/package/zerofree/zerofree.mk b/package/zerofree/zerofree.mk
> new file mode 100644
> index 0000000000..ca5a019de7
> --- /dev/null
> +++ b/package/zerofree/zerofree.mk
> @@ -0,0 +1,27 @@
> +################################################################################
> +#
> +# zerofree
> +#
> +################################################################################
> +
> +ZEROFREE_VERSION = 1.1.1
> +ZEROFREE_SOURCE = zerofree-$(ZEROFREE_VERSION).tgz
> +ZEROFREE_SITE = https://frippery.org/uml
> +ZEROFREE_LICENSE = GPL-2.0
> +ZEROFREE_LICENSE_FILE = COPYING
> +ZEROFREE_DEPENDENCIES = e2fsprogs
> +
> +# We use the same workaround as in https://bugs.gentoo.org/716136
> +# to build with musl.
> +ZEROFREE_CFLAGS = $(TARGET_CFLAGS) \
> +	$(if $(BR2_TOOLCHAIN_USES_MUSL),-DHAVE_SYS_TYPES_H) # dev_t
> +
> +define ZEROFREE_BUILD_CMDS
> +	$(MAKE) CC="$(TARGET_CC)" CFLAGS="$(ZEROFREE_CFLAGS)" -C $(@D) all
> +endef
> +
> +define ZEROFREE_INSTALL_TARGET_CMDS
> +	$(INSTALL) -D -m 0755 $(@D)/zerofree $(TARGET_DIR)/usr/bin
> +endef
> +
> +$(eval $(generic-package))
> diff --git a/support/testing/tests/package/test_zerofree.py b/support/testing/tests/package/test_zerofree.py
> new file mode 100644
> index 0000000000..75eea346d3
> --- /dev/null
> +++ b/support/testing/tests/package/test_zerofree.py
> @@ -0,0 +1,41 @@
> +import os
> +import infra.basetest
> +import subprocess
> +
> +
> +class TestZerofree(infra.basetest.BRTest):
> +    config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
> +        """
> +        BR2_PACKAGE_ZEROFREE=y
> +        BR2_TARGET_ROOTFS_CPIO=y
> +        # BR2_TARGET_ROOTFS_TAR is not set
> +        """
> +
> +    def test_run(self):
> +        # Prepare disk image.
> +        # We keep it small (8 MB) for the sake of test time.
> +        disk_file = os.path.join(self.outputdir, "disk.img")
> +        subprocess.check_call(
> +            ["dd", "if=/dev/zero", f"of={disk_file}", "bs=1M", "count=8"],
> +            stdout=self.emulator.logfile,
> +            stderr=self.emulator.logfile)
> +
> +        # Run the emulator with a drive.
> +        cpio_file = os.path.join(self.builddir, "images", "rootfs.cpio")
> +        self.emulator.boot(arch="armv5",
> +                           kernel="builtin",
> +                           options=[
> +                                "-initrd", cpio_file,
> +                                "-drive", f"file={disk_file},format=raw"])
> +        self.emulator.login()
> +
> +        # Prepare filesystem.
> +        output, exit_code = self.emulator.run("mkfs.ext4 /dev/sda")
> +        self.assertEqual(exit_code, 0)
> +        self.assertIn('Creating filesystem', output[2])
> +
> +        # Run zerofree on newly created filesystem.
> +        cmd = "zerofree -v /dev/sda"
> +        output, exit_code = self.emulator.run(cmd, timeout=60)
> +        self.assertEqual(exit_code, 0)
> +        self.assertIn('/8192', output[-1])  # total number of blocks
> -- 
> 2.35.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot
diff mbox series

Patch

diff --git a/DEVELOPERS b/DEVELOPERS
index 71dd7d3d6c..0fc73f311f 100644
--- a/DEVELOPERS
+++ b/DEVELOPERS
@@ -2917,8 +2917,10 @@  F:	configs/uevm5432_defconfig
 F:	package/i7z/
 F:	package/msr-tools/
 F:	package/pixz/
+F:	package/zerofree/
 F:	support/testing/tests/package/test_msr_tools*
 F:	support/testing/tests/package/test_pixz.py
+F:	support/testing/tests/package/test_zerofree.py
 
 N:	Vinicius Tinti <viniciustinti@gmail.com>
 F:	package/python-thrift/
diff --git a/package/Config.in b/package/Config.in
index 230aa7a2c2..3feb47c534 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -239,6 +239,7 @@  menu "Filesystem and flash utilities"
 	source "package/udftools/Config.in"
 	source "package/unionfs/Config.in"
 	source "package/xfsprogs/Config.in"
+	source "package/zerofree/Config.in"
 	source "package/zfs/Config.in"
 endmenu
 
diff --git a/package/zerofree/Config.in b/package/zerofree/Config.in
new file mode 100644
index 0000000000..2ce2901a27
--- /dev/null
+++ b/package/zerofree/Config.in
@@ -0,0 +1,13 @@ 
+config BR2_PACKAGE_ZEROFREE
+	bool "zerofree"
+	depends on BR2_USE_MMU # e2fsprogs
+	depends on !BR2_STATIC_LIBS
+	select BR2_PACKAGE_E2FSPROGS
+	help
+	  zerofree is a utility which scans the free blocks in an ext2
+	  filesystem and fills any non-zero blocks with zeroes.
+
+	  https://frippery.org/uml/
+
+comment "zerofree needs MMU and dynamic library"
+	depends on !BR2_USE_MMU || BR2_STATIC_LIBS
diff --git a/package/zerofree/zerofree.hash b/package/zerofree/zerofree.hash
new file mode 100644
index 0000000000..3399611787
--- /dev/null
+++ b/package/zerofree/zerofree.hash
@@ -0,0 +1,3 @@ 
+# Locally calculated
+sha256  956bc861b55ba0a2b7593c58d32339dab1a0e7da6ea2b813d27c80f08b723867  zerofree-1.1.1.tgz
+sha256  90daae00475a992a367da5b0658469a5d1c4449dbbe964c5b7246e1aec92f491  COPYING
diff --git a/package/zerofree/zerofree.mk b/package/zerofree/zerofree.mk
new file mode 100644
index 0000000000..ca5a019de7
--- /dev/null
+++ b/package/zerofree/zerofree.mk
@@ -0,0 +1,27 @@ 
+################################################################################
+#
+# zerofree
+#
+################################################################################
+
+ZEROFREE_VERSION = 1.1.1
+ZEROFREE_SOURCE = zerofree-$(ZEROFREE_VERSION).tgz
+ZEROFREE_SITE = https://frippery.org/uml
+ZEROFREE_LICENSE = GPL-2.0
+ZEROFREE_LICENSE_FILE = COPYING
+ZEROFREE_DEPENDENCIES = e2fsprogs
+
+# We use the same workaround as in https://bugs.gentoo.org/716136
+# to build with musl.
+ZEROFREE_CFLAGS = $(TARGET_CFLAGS) \
+	$(if $(BR2_TOOLCHAIN_USES_MUSL),-DHAVE_SYS_TYPES_H) # dev_t
+
+define ZEROFREE_BUILD_CMDS
+	$(MAKE) CC="$(TARGET_CC)" CFLAGS="$(ZEROFREE_CFLAGS)" -C $(@D) all
+endef
+
+define ZEROFREE_INSTALL_TARGET_CMDS
+	$(INSTALL) -D -m 0755 $(@D)/zerofree $(TARGET_DIR)/usr/bin
+endef
+
+$(eval $(generic-package))
diff --git a/support/testing/tests/package/test_zerofree.py b/support/testing/tests/package/test_zerofree.py
new file mode 100644
index 0000000000..75eea346d3
--- /dev/null
+++ b/support/testing/tests/package/test_zerofree.py
@@ -0,0 +1,41 @@ 
+import os
+import infra.basetest
+import subprocess
+
+
+class TestZerofree(infra.basetest.BRTest):
+    config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
+        """
+        BR2_PACKAGE_ZEROFREE=y
+        BR2_TARGET_ROOTFS_CPIO=y
+        # BR2_TARGET_ROOTFS_TAR is not set
+        """
+
+    def test_run(self):
+        # Prepare disk image.
+        # We keep it small (8 MB) for the sake of test time.
+        disk_file = os.path.join(self.outputdir, "disk.img")
+        subprocess.check_call(
+            ["dd", "if=/dev/zero", f"of={disk_file}", "bs=1M", "count=8"],
+            stdout=self.emulator.logfile,
+            stderr=self.emulator.logfile)
+
+        # Run the emulator with a drive.
+        cpio_file = os.path.join(self.builddir, "images", "rootfs.cpio")
+        self.emulator.boot(arch="armv5",
+                           kernel="builtin",
+                           options=[
+                                "-initrd", cpio_file,
+                                "-drive", f"file={disk_file},format=raw"])
+        self.emulator.login()
+
+        # Prepare filesystem.
+        output, exit_code = self.emulator.run("mkfs.ext4 /dev/sda")
+        self.assertEqual(exit_code, 0)
+        self.assertIn('Creating filesystem', output[2])
+
+        # Run zerofree on newly created filesystem.
+        cmd = "zerofree -v /dev/sda"
+        output, exit_code = self.emulator.run(cmd, timeout=60)
+        self.assertEqual(exit_code, 0)
+        self.assertIn('/8192', output[-1])  # total number of blocks