| Message ID | 20190311162028.13172-1-thomas.petazzoni@bootlin.com |
|---|---|
| Headers | show |
| Series | Proposal for dm-verity support | expand |
Hi Thomas, On 3/11/19 5:20 PM, Thomas Petazzoni wrote: > Hello, > > This patch series is a proposal to add support for dm-verity to > OpenWRT. While I am familiar with build systems in general (I am one > of the core developers of Buildroot), this is my first ever > contribution to OpenWRT, so I am definitely not sure that the approach > is correct and I'm interested in getting feedback. > > dm-verity is a Linux kernel Device Mapper target that verifies that > the data in a block device has not been tampered with, by checking it > at runtime against a hash tree, itself verified by a root hash, which > is passed from a trusted source. dm-verity only supports read > operations, so we only support the read-only squashfs root filesystem > in this series. > > This "hash tree" is a bunch of metadata that needs to be stored on > non-volatile storage. It can be appended to the filesystem data, or > stored on a separate block device/partition. We have chosen to support > only the case where it is appended to the filesystem data. This sounds interesting, from a community perspective I do not like secure boot, because it makes it harder to hack the devices, but I know that many vendors are interested in this. > > In the proposed series: > > - The first four patches introduce new host packages. The first three > are simply dependencies needed for cryptsetup, which is the tool > used to generate the hash tree at build time. > > - The fifth and sixth patches introduce a way to include a U-Boot > script inside a FIT image. Indeed, to set up a dm-verity device at > boot time, you need to pass a lot of details to the kernel that > describe the dm-verity device, including the root hash. Those > details need to be trusted: having them as part of the FIT image > allows to leverage the signing capabilities of FIT images. > > - The seventh patch adds the code itself that generates the dm-verity > capable squashfs image, and a script that produces the U-Boot > script with the various parameters needed to setup the DM device at > boot time. How do you handle the overlay filesystem? An attacker could place there some new binaries which would just replace the original ones. > - The eighth patch adds two kernel patches that allow setting up a DM > device at boot time. In the upstream kernel, setting up a DM device > requires userspace tools and therefore an initramfs, which is > unpractical. Those two patches have been submitted numerous times > by folks from Google and Redhat, but have remained out of tree so > far. We know this problem in that area. ;-) > - The remaining patches are just related to enabling this mechanism > on Armada XP GP, which is the platform I used to work on this > topic. > > This work was tested on Armada XP GP, with both MMC and NAND storage. > > One aspect that is not solved by this patch series is the logic in the > fstools programs to set up the overlay at boot time. Indeed, when > there is a squashfs filesystem, fstools assumes that it can use the > space after the squashfs filesystem for its overlay (in the MMC > storage case). It is not the case with dm-verity, because we have the > hash tree after the squashfs filesystem. This is something I intend to > work on. There are some kernel patches (?) which detect how big the squashfs filesystem is and then create an extra partition there. You should probably make this detection aware of dm-verify. > Thanks in advance for your feedback, > > Thomas Petazzoni > > Thomas Petazzoni (11): > tools/libaio: new package > tools/lvm2: new package > tools/popt: new package > tools/cryptsetup: new package > scripts/mkits.sh: extend with -s option to include a U-Boot script > include/image-commands.mk: extend Build/fit for U-Boot script > integration > include/image.mk: add support for building a dm-verity enabled > squashfs image > target/linux/generic: add patches to support dm-verity volume at boot > target/linux/mvebu: enable UBI factory image on Armada XP GP > target/linux/mvebu/config-4.14: enable options needed for dm-verity > target/linux/mvebu: generate a FIT image on Armada XP GP with > dm-verity > > config/Config-images.in | 4 + > include/image-commands.mk | 1 + > include/image.mk | 12 + > scripts/mkits.sh | 21 +- > scripts/prepare-dm-verity-uboot-script.sh | 41 ++ > ...-to-directly-boot-to-a-mapped-device.patch | 633 ++++++++++++++++++ > ...l-add-a-device-mapper-ioctl-function.patch | 98 +++ > target/linux/mvebu/config-4.14 | 11 + > target/linux/mvebu/image/cortex-a9.mk | 15 + > tools/Makefile | 3 + > tools/cryptsetup/Makefile | 28 + > .../patches/0001-dont-use-c89.patch | 13 + > tools/libaio/Makefile | 33 + > tools/lvm2/Makefile | 47 ++ > tools/popt/Makefile | 22 + > 15 files changed, 980 insertions(+), 2 deletions(-) > create mode 100755 scripts/prepare-dm-verity-uboot-script.sh > create mode 100644 target/linux/generic/pending-4.14/960-init-add-support-to-directly-boot-to-a-mapped-device.patch > create mode 100644 target/linux/generic/pending-4.14/961-dm-ioctl-add-a-device-mapper-ioctl-function.patch > create mode 100644 tools/cryptsetup/Makefile > create mode 100644 tools/cryptsetup/patches/0001-dont-use-c89.patch > create mode 100644 tools/libaio/Makefile > create mode 100644 tools/lvm2/Makefile > create mode 100644 tools/popt/Makefile >
Hello Hauke, On Mon, 25 Mar 2019 18:31:19 +0100 Hauke Mehrtens <hauke@hauke-m.de> wrote: > > This "hash tree" is a bunch of metadata that needs to be stored on > > non-volatile storage. It can be appended to the filesystem data, or > > stored on a separate block device/partition. We have chosen to support > > only the case where it is appended to the filesystem data. > > This sounds interesting, from a community perspective I do not like > secure boot, because it makes it harder to hack the devices, but I know > that many vendors are interested in this. Indeed. That being said, dm-verity per-se is not sufficient to achieve security: the root hash needs to come from a trusted source, i.e typically a signed kernel image. But I agree overall dm-verity is part of a plan to lock down devices. > > - The seventh patch adds the code itself that generates the dm-verity > > capable squashfs image, and a script that produces the U-Boot > > script with the various parameters needed to setup the DM device at > > boot time. > > How do you handle the overlay filesystem? An attacker could place there > some new binaries which would just replace the original ones. At the moment, I have a hacky patch on fstools that simply disables mounting an overlay filesystem entirely, i.e the system is completely read-only. I am not sure yet how to turn this into a clean solution that can be accepted upstream: fstools is currently doing all its overlay logic in a very automated way, with not much configuration to adjust its behavior. I was thinking of adding a kernel argument like openwrt.overlayfs={none,ram,default} to be able to force a certain behavior with the overlay, but I'm open to suggestions. > > - The eighth patch adds two kernel patches that allow setting up a DM > > device at boot time. In the upstream kernel, setting up a DM device > > requires userspace tools and therefore an initramfs, which is > > unpractical. Those two patches have been submitted numerous times > > by folks from Google and Redhat, but have remained out of tree so > > far. > > We know this problem in that area. ;-) As I replied to your review on patch 08/11, the 5.1 kernel will have support for setting up DM devices on the kernel command line, it has been merged upstream. > There are some kernel patches (?) which detect how big the squashfs > filesystem is and then create an extra partition there. You should > probably make this detection aware of dm-verify. It's actually user-space code in fstools that does this, at least for the MMC case. It looks at the squashfs filesystem size, and then creates a loop device that starts right after the squashfs filesystem, and uses it as a f2fs filesystem to store the overlay information. As explained above, I've worked-around this stuff for now with a hacky patch in fstools to completely disable setting up an overlayfs. Best regards, Thomas
Hi Thomas, On 3/25/19 7:07 PM, Thomas Petazzoni wrote: > Hello Hauke, > > On Mon, 25 Mar 2019 18:31:19 +0100 > Hauke Mehrtens <hauke@hauke-m.de> wrote: > >>> This "hash tree" is a bunch of metadata that needs to be stored on >>> non-volatile storage. It can be appended to the filesystem data, or >>> stored on a separate block device/partition. We have chosen to support >>> only the case where it is appended to the filesystem data. >> >> This sounds interesting, from a community perspective I do not like >> secure boot, because it makes it harder to hack the devices, but I know >> that many vendors are interested in this. > > Indeed. That being said, dm-verity per-se is not sufficient to achieve > security: the root hash needs to come from a trusted source, i.e > typically a signed kernel image. But I agree overall dm-verity is part > of a plan to lock down devices. > >>> - The seventh patch adds the code itself that generates the dm-verity >>> capable squashfs image, and a script that produces the U-Boot >>> script with the various parameters needed to setup the DM device at >>> boot time. >> >> How do you handle the overlay filesystem? An attacker could place there >> some new binaries which would just replace the original ones. > > At the moment, I have a hacky patch on fstools that simply disables > mounting an overlay filesystem entirely, i.e the system is completely > read-only. I am not sure yet how to turn this into a clean solution > that can be accepted upstream: fstools is currently doing all its > overlay logic in a very automated way, with not much configuration to > adjust its behavior. I was thinking of adding a kernel argument like > openwrt.overlayfs={none,ram,default} to be able to force a certain > behavior with the overlay, but I'm open to suggestions. Using some boot arguments sounds like a good solution, but I am not an expert on the file system handling. The default has to be the current behavior, because we do not have control over all boot loaders, I assume that people who need this special behavior have control over their boot loader. Do you know if it is possible to support dm-verify also for the overlay file system? >>> - The eighth patch adds two kernel patches that allow setting up a DM >>> device at boot time. In the upstream kernel, setting up a DM device >>> requires userspace tools and therefore an initramfs, which is >>> unpractical. Those two patches have been submitted numerous times >>> by folks from Google and Redhat, but have remained out of tree so >>> far. >> >> We know this problem in that area. ;-) > > As I replied to your review on patch 08/11, the 5.1 kernel will have > support for setting up DM devices on the kernel command line, it has > been merged upstream. It would be nice if you could backport the upstream version to kernel 4.14 and 4.19, you do not have to care about the old kernels, when we move to the next LTS kernel we can just remove the patches. >> There are some kernel patches (?) which detect how big the squashfs >> filesystem is and then create an extra partition there. You should >> probably make this detection aware of dm-verify. > > It's actually user-space code in fstools that does this, at least for > the MMC case. It looks at the squashfs filesystem size, and then > creates a loop device that starts right after the squashfs filesystem, > and uses it as a f2fs filesystem to store the overlay information. > > As explained above, I've worked-around this stuff for now with a hacky > patch in fstools to completely disable setting up an overlayfs. > > Best regards, > > Thomas > Hauke
Hello Hauke, On Mon, 25 Mar 2019 23:13:17 +0100 Hauke Mehrtens <hauke@hauke-m.de> wrote: > Using some boot arguments sounds like a good solution, but I am not an > expert on the file system handling. OK, thanks. Do you know who would be the appropriate person to discuss this ? > The default has to be the current > behavior, because we do not have control over all boot loaders, I assume > that people who need this special behavior have control over their boot > loader. Yes of course the default would be to preserve the current behavior. > Do you know if it is possible to support dm-verify also for the overlay > file system? dm-verity by essence only supports read-only accesses. dm-verity generates a tree of hashes at "build" time, i.e with "veritysetup format" and at runtime, dm-verity checks that the hash of the blocks being read matches the hash stored in the hash tree. So the data blocks cannot be changed: any change in a data block will cause a hash mismatch, which results in an I/O error: it's exactly what dm-verity wants to detect, that the data has been tampered with. > > As I replied to your review on patch 08/11, the 5.1 kernel will have > > support for setting up DM devices on the kernel command line, it has > > been merged upstream. > > It would be nice if you could backport the upstream version to kernel > 4.14 and 4.19, you do not have to care about the old kernels, when we > move to the next LTS kernel we can just remove the patches. OK, I'll see if the upstream version is reasonable enough to be backported. Best regards, Thomas
Hello, This patch series is a proposal to add support for dm-verity to OpenWRT. While I am familiar with build systems in general (I am one of the core developers of Buildroot), this is my first ever contribution to OpenWRT, so I am definitely not sure that the approach is correct and I'm interested in getting feedback. dm-verity is a Linux kernel Device Mapper target that verifies that the data in a block device has not been tampered with, by checking it at runtime against a hash tree, itself verified by a root hash, which is passed from a trusted source. dm-verity only supports read operations, so we only support the read-only squashfs root filesystem in this series. This "hash tree" is a bunch of metadata that needs to be stored on non-volatile storage. It can be appended to the filesystem data, or stored on a separate block device/partition. We have chosen to support only the case where it is appended to the filesystem data. In the proposed series: - The first four patches introduce new host packages. The first three are simply dependencies needed for cryptsetup, which is the tool used to generate the hash tree at build time. - The fifth and sixth patches introduce a way to include a U-Boot script inside a FIT image. Indeed, to set up a dm-verity device at boot time, you need to pass a lot of details to the kernel that describe the dm-verity device, including the root hash. Those details need to be trusted: having them as part of the FIT image allows to leverage the signing capabilities of FIT images. - The seventh patch adds the code itself that generates the dm-verity capable squashfs image, and a script that produces the U-Boot script with the various parameters needed to setup the DM device at boot time. - The eighth patch adds two kernel patches that allow setting up a DM device at boot time. In the upstream kernel, setting up a DM device requires userspace tools and therefore an initramfs, which is unpractical. Those two patches have been submitted numerous times by folks from Google and Redhat, but have remained out of tree so far. - The remaining patches are just related to enabling this mechanism on Armada XP GP, which is the platform I used to work on this topic. This work was tested on Armada XP GP, with both MMC and NAND storage. One aspect that is not solved by this patch series is the logic in the fstools programs to set up the overlay at boot time. Indeed, when there is a squashfs filesystem, fstools assumes that it can use the space after the squashfs filesystem for its overlay (in the MMC storage case). It is not the case with dm-verity, because we have the hash tree after the squashfs filesystem. This is something I intend to work on. Thanks in advance for your feedback, Thomas Petazzoni Thomas Petazzoni (11): tools/libaio: new package tools/lvm2: new package tools/popt: new package tools/cryptsetup: new package scripts/mkits.sh: extend with -s option to include a U-Boot script include/image-commands.mk: extend Build/fit for U-Boot script integration include/image.mk: add support for building a dm-verity enabled squashfs image target/linux/generic: add patches to support dm-verity volume at boot target/linux/mvebu: enable UBI factory image on Armada XP GP target/linux/mvebu/config-4.14: enable options needed for dm-verity target/linux/mvebu: generate a FIT image on Armada XP GP with dm-verity config/Config-images.in | 4 + include/image-commands.mk | 1 + include/image.mk | 12 + scripts/mkits.sh | 21 +- scripts/prepare-dm-verity-uboot-script.sh | 41 ++ ...-to-directly-boot-to-a-mapped-device.patch | 633 ++++++++++++++++++ ...l-add-a-device-mapper-ioctl-function.patch | 98 +++ target/linux/mvebu/config-4.14 | 11 + target/linux/mvebu/image/cortex-a9.mk | 15 + tools/Makefile | 3 + tools/cryptsetup/Makefile | 28 + .../patches/0001-dont-use-c89.patch | 13 + tools/libaio/Makefile | 33 + tools/lvm2/Makefile | 47 ++ tools/popt/Makefile | 22 + 15 files changed, 980 insertions(+), 2 deletions(-) create mode 100755 scripts/prepare-dm-verity-uboot-script.sh create mode 100644 target/linux/generic/pending-4.14/960-init-add-support-to-directly-boot-to-a-mapped-device.patch create mode 100644 target/linux/generic/pending-4.14/961-dm-ioctl-add-a-device-mapper-ioctl-function.patch create mode 100644 tools/cryptsetup/Makefile create mode 100644 tools/cryptsetup/patches/0001-dont-use-c89.patch create mode 100644 tools/libaio/Makefile create mode 100644 tools/lvm2/Makefile create mode 100644 tools/popt/Makefile