| Message ID | CAJ+vNU1+oxD9HxbtVsoG+JCxPZZps7d-5ugvyzbMZycWxC3i0Q@mail.gmail.com |
|---|---|
| State | Not Applicable |
| Headers | show |
| Series | IMX8M OP-TEE | expand |
Hi Tim On Mon, 2021-10-11 at 15:15 -0700, Tim Harvey wrote: > Greetings, > > Is anyone successfully booting U-Boot with OP-TEE support on the IMX8M? I am not aware that anybody at Toradex played with OP-TEE so far but let me ask around again. However, maybe Igor or Oleksandr (added to CC, now both with Foundries) might have some experience. I know that Igor used to work on this kind of stuff. > My understanding is that you need to add tee.bin to the images in the > FIT image and include it in loadables following the ATF. > > While this was done with arch/arm/mach-imx/mkimage_fit_atf.sh before > the switch to binman by simply having tee.bin in the U-Boot directory > and passing in TEE_LOAD_ADDR (or accepting the default of 0xfe000000) > once you switch to binman it needs to be added for your board. > > Additionally in order to use OP-TEE from U-Boot (ie for dek_blob > command) you need to add a node with compatible=linaro,optee-tz as > well. > > I've done the following to add OP-TEE for imx8mm_venice: > diff --git a/configs/imx8mm_venice_defconfig b/configs/imx8mm_venice_defconfig > index d85827b588..85d2b20810 100644 > -u-boot.dtsi > index e0fa9ff4bf..7a71b974e1 100644 > --- a/arch/arm/dts/imx8mm-venice-u-boot.dtsi > +++ b/arch/arm/dts/imx8mm-venice-u-boot.dtsi > @@ -10,6 +10,13 @@ > multiple-images; > }; > > + firmware { > + optee { > + compatible = "linaro,optee-tz"; > + method = "smc"; > + }; > + }; > + > wdt-reboot { > compatible = "wdt-reboot"; > wdt = <&wdog1>; > @@ -152,6 +159,16 @@ > }; > }; > > + tee { > + description = "TEE firmware"; > + type = "firmware"; > + arch = "arm64"; > + compression = "none"; > + data = "tee.bin"; > + load = <0xbe000000>; > + entry = <0xbe000000>; > + }; > + > @fdt-SEQ { > description = "NAME"; > type = "flat_dt"; > @@ -165,7 +182,7 @@ > @config-SEQ { > description = "NAME"; > firmware = "uboot"; > - loadables = "atf"; > + loadables = "atf", "tee"; > fdt = "fdt-SEQ"; > }; > }; > > > However, when I attempt to boot I hang when the ATF is run. > > Where does the TEE_LOAD_ADDR come from specifically? I would think > this needs to be defined when building tee and needs to match the load > address used in the FIT image. It appears that perhaps this is supped > to be DDR_BASE + DDR_SIZE - 32MIB but I'm not entirely sure. > > I'm currently using NXP's ATF (imx_5.4.3_2.0.0) and NXP's TEE > (imx_5.4.70_2.3.0) and would also like to understand if NXP's branches > are strictly required here and if so what the pros and cons of using > them are. > > Anyone using IMX8MM OP-TEE that could point me in the right direction? > > Best regards, > > Tim BTW: Have you tried my latest stuff [1] on venice? Did/does it work? And yeah, my cover letter subject is slightly misleading as I forgot to update that one (;-p). [1] https://marc.info/?l=u-boot&m=163372696806292 Cheers Marcel
On Mon, Oct 11, 2021 at 3:29 PM Marcel Ziswiler <marcel.ziswiler@toradex.com> wrote: > > Hi Tim > > On Mon, 2021-10-11 at 15:15 -0700, Tim Harvey wrote: > > Greetings, > > > > Is anyone successfully booting U-Boot with OP-TEE support on the IMX8M? > > I am not aware that anybody at Toradex played with OP-TEE so far but let me ask around again. > > However, maybe Igor or Oleksandr (added to CC, now both with Foundries) might have some experience. I know that > Igor used to work on this kind of stuff. > > > My understanding is that you need to add tee.bin to the images in the > > FIT image and include it in loadables following the ATF. > > > > While this was done with arch/arm/mach-imx/mkimage_fit_atf.sh before > > the switch to binman by simply having tee.bin in the U-Boot directory > > and passing in TEE_LOAD_ADDR (or accepting the default of 0xfe000000) > > once you switch to binman it needs to be added for your board. > > > > Additionally in order to use OP-TEE from U-Boot (ie for dek_blob > > command) you need to add a node with compatible=linaro,optee-tz as > > well. > > > > I've done the following to add OP-TEE for imx8mm_venice: > > diff --git a/configs/imx8mm_venice_defconfig b/configs/imx8mm_venice_defconfig > > index d85827b588..85d2b20810 100644 > > -u-boot.dtsi > > index e0fa9ff4bf..7a71b974e1 100644 > > --- a/arch/arm/dts/imx8mm-venice-u-boot.dtsi > > +++ b/arch/arm/dts/imx8mm-venice-u-boot.dtsi > > @@ -10,6 +10,13 @@ > > multiple-images; > > }; > > > > + firmware { > > + optee { > > + compatible = "linaro,optee-tz"; > > + method = "smc"; > > + }; > > + }; > > + > > wdt-reboot { > > compatible = "wdt-reboot"; > > wdt = <&wdog1>; > > @@ -152,6 +159,16 @@ > > }; > > }; > > > > + tee { > > + description = "TEE firmware"; > > + type = "firmware"; > > + arch = "arm64"; > > + compression = "none"; > > + data = "tee.bin"; > > + load = <0xbe000000>; > > + entry = <0xbe000000>; > > + }; > > + > > @fdt-SEQ { > > description = "NAME"; > > type = "flat_dt"; > > @@ -165,7 +182,7 @@ > > @config-SEQ { > > description = "NAME"; > > firmware = "uboot"; > > - loadables = "atf"; > > + loadables = "atf", "tee"; > > fdt = "fdt-SEQ"; > > }; > > }; > > > > > > However, when I attempt to boot I hang when the ATF is run. > > > > Where does the TEE_LOAD_ADDR come from specifically? I would think > > this needs to be defined when building tee and needs to match the load > > address used in the FIT image. It appears that perhaps this is supped > > to be DDR_BASE + DDR_SIZE - 32MIB but I'm not entirely sure. > > > > I'm currently using NXP's ATF (imx_5.4.3_2.0.0) and NXP's TEE > > (imx_5.4.70_2.3.0) and would also like to understand if NXP's branches > > are strictly required here and if so what the pros and cons of using > > them are. > > > > Anyone using IMX8MM OP-TEE that could point me in the right direction? > > > > Best regards, > > > > Tim > > BTW: Have you tried my latest stuff [1] on venice? Did/does it work? And yeah, my cover letter subject is > slightly misleading as I forgot to update that one (;-p). > Marcel, Thanks for checking into OP-TEE for me. I haven't tested your series yet as I tried to apply it and it failed. Do you have a git repo I can cherry-pick from or what did you base it on top of? Thanks, Tim
Hi Tim On Mon, 2021-10-11 at 16:32 -0700, Tim Harvey wrote: > > ... > Marcel, > > Thanks for checking into OP-TEE for me. Sure thing. I meanwhile got reminded that Sergio Prado actually held a talk about this topic back at the Embedded World [1]. Not sure whether he played with mainline U-Boot and OP-TEE though. > I haven't tested your series yet as I tried to apply it and it failed. It should actually be based on latest imx/master but depends on a few other series as indicated in the cover letter. > Do you have a git repo I can cherry-pick from or what did you base it > on top of? For your convenience I just pushed my branch to my personal github now as well [2]. > Thanks, Thanks you! > Tim [1] https://www.youtube.com/watch?v=sBVzSQ5uvUw [2] https://github.com/ziswiler/u-boot/tree/20211008-imx-master_common_binman_v2 Cheers Marcel
On Mon, Oct 11, 2021 at 3:15 PM Tim Harvey <tharvey@gateworks.com> wrote: > > Greetings, > > Is anyone successfully booting U-Boot with OP-TEE support on the IMX8M? > > My understanding is that you need to add tee.bin to the images in the > FIT image and include it in loadables following the ATF. > > While this was done with arch/arm/mach-imx/mkimage_fit_atf.sh before > the switch to binman by simply having tee.bin in the U-Boot directory > and passing in TEE_LOAD_ADDR (or accepting the default of 0xfe000000) > once you switch to binman it needs to be added for your board. > > Additionally in order to use OP-TEE from U-Boot (ie for dek_blob > command) you need to add a node with compatible=linaro,optee-tz as > well. > > I've done the following to add OP-TEE for imx8mm_venice: > diff --git a/configs/imx8mm_venice_defconfig b/configs/imx8mm_venice_defconfig > index d85827b588..85d2b20810 100644 > -u-boot.dtsi > index e0fa9ff4bf..7a71b974e1 100644 > --- a/arch/arm/dts/imx8mm-venice-u-boot.dtsi > +++ b/arch/arm/dts/imx8mm-venice-u-boot.dtsi > @@ -10,6 +10,13 @@ > multiple-images; > }; > > + firmware { > + optee { > + compatible = "linaro,optee-tz"; > + method = "smc"; > + }; > + }; > + > wdt-reboot { > compatible = "wdt-reboot"; > wdt = <&wdog1>; > @@ -152,6 +159,16 @@ > }; > }; > > + tee { > + description = "TEE firmware"; > + type = "firmware"; > + arch = "arm64"; > + compression = "none"; > + data = "tee.bin"; > + load = <0xbe000000>; > + entry = <0xbe000000>; > + }; > + > @fdt-SEQ { > description = "NAME"; > type = "flat_dt"; > @@ -165,7 +182,7 @@ > @config-SEQ { > description = "NAME"; > firmware = "uboot"; > - loadables = "atf"; > + loadables = "atf", "tee"; > fdt = "fdt-SEQ"; > }; > }; > > > However, when I attempt to boot I hang when the ATF is run. > > Where does the TEE_LOAD_ADDR come from specifically? I would think > this needs to be defined when building tee and needs to match the load > address used in the FIT image. It appears that perhaps this is supped > to be DDR_BASE + DDR_SIZE - 32MIB but I'm not entirely sure. > > I'm currently using NXP's ATF (imx_5.4.3_2.0.0) and NXP's TEE > (imx_5.4.70_2.3.0) and would also like to understand if NXP's branches > are strictly required here and if so what the pros and cons of using > them are. > > Anyone using IMX8MM OP-TEE that could point me in the right direction? > Here's what I've learned so far via various responses, research and testing. There are multiple boot flows that can be used with OP-TEE: 1. SPL -> ATF -> OP-TEE -> U-boot -> LInux 2. SPL -> OP-TEE -> U-boot -> Linux I'm not really clear what the pros and cons of each are - can anyone shed some light on this? Method #1 is what I've found the most info on and have mostly gotten working so I will go over that here. As IMX8MM runs on Cortex-A Arm TrustZone is available and thus TF-A can be used. By the way, I'm a bit confused as to ATF vs TF-A. From what I can find: - ATF (Arm Trusted Firmware) is a Trusted Firmware reference implementation of secure world software for Armv7-A, Armv8-A, and Armv8-M architectures. - TF-A (Trusted Firmware-A) also appears to provide the same thing Are these the same? Which one is it that NXP uses for imx-atf [1] When OP-TEE is used with TF-A the TF-A gets invoked with the TEE as bl32 and main u-boot as bl33. Once it has done its startup TF-A jumps into the bl32(OP-TEE) for the TEE startup, returns to TF-A and then jumps to bl33(U-Boot). For this boot flow on IMX8MM you need to do the following: - build TF-A (bl31.bin) with OP-TEE as a Secure Payload Dispatcher (SPD) (add 'SPD=opteed' to env) - build OP-TEE (tee.bin) - add tee.bin to U-Boot FIT image along with TF-A binary in images node * for boards still using arch/arm/mach-imx/mkimage_fit_atf.sh FIT generation this is done automatically when tee.bin is U-Boot's build directory but you also need to set TEE_LOAD_ADDR env to specify the load addr of the TEE which depends on the board DRAM size * for boards using binman the generation of the FIT image has moved to using the 'binman' tool thus adding tee.bin requires you to modify your board's u-boot.dtsi itb node to add tee.bin to itb/fit/images/ and add a reference to it in the loadables property of your boot configuration following atf. This is shown in my patch above except there I neglected to add the required 'tee_blob: blob-ext { filename = "tee.bin"; }. Here you must set the TEE_LOAD_ADDR as the load address manually. - To enable OP-TEE support in U-Boot a /firmware/optee node needs to be added to your board u-boot.dtsi with 'compatible = "linaro,optee-tz"' and 'method = "smc"' nodes * This node is used by the U-Boot dek_blob command * If present then U-Boot will also copy it to your Linux dtb via image_setup_libfdt->optee_copy_fdt_nodes - To enable dek_blob in U-Boot enable CONFIG_CMD_DEKBLOB=y which will enable CONFIG_IMX_OPTEE_DEK_ENCAP=y - To enable optee driver in Linux enable CONFIG_TEE and CONFIG_OPTEE and optionally CONFIG_HW_RANDOM_OPTEE - Also, I believe you need to add memory reservations in the dt via a /reserved-memory/optee_core and /reserved-memory/optee_shm which will also be copied to the linux dtb as well if present and also depend on the OP-TEE memory location/configuration If you are using NXP's imx-optee-os [2] for IMX8MM then you can use the 'mx8mmevk' platform which simply configures the following which can be overridden via env: CFG_UART_BASE ?= UART2_BASE CFG_DDR_SIZE ?= 0x80000000 In core/arch/arm/plat-imx/conf.mk for imx8mm CFG_DDR_SIZE is used to compute CFG_TZDRAM_START where optee gets loaded (TEE_LOAD_ADDR): # put optee end of ddr for AARCH64 CFG_TZDRAM_START ?= ($(CFG_DRAM_BASE) - 0x02000000 + $(CFG_DDR_SIZE)) CFG_TZDRAM_SIZE ?= 0x01c00000 CFG_SHMEM_START ?= ($(CFG_TZDRAM_START) + $(CFG_TZDRAM_SIZE)) CFG_SHMEM_SIZE ?= 0x00400000 thus: CFG_DDR_SIZE = 0x40000000 (1GiB) results in CFG_TZDRAM_START=0x7e000000 (TEE_LOAD_ADDR) and CFG_SHMEM_START=0x7fc00000 CFG_DDR_SIZE = 0x80000000 (2GiB) results in CFG_TZDRAM_START=0xbe000000 (TEE_LOAD_ADDR) and CFG_SHMEM_START=0xbfc00000 CFG_DDR_SIZE = 0x100000000 (4GiB) results in CFG_TZDRAM_START=0x13e000000 (TEE_LOAD_ADDR) and CFG_SHMEM_START=0x13fc00000 So for a 2GiB board your dt changes for a board already using binman should look like this: @@ -6,6 +6,29 @@ #include "imx8mm-u-boot.dtsi" / { + firmware { + optee { + compatible = "linaro,optee-tz"; + method = "smc"; + }; + }; + + reserved-memory { + #address-cells = <2>; + #size-cells = <2>; + ranges; + + /* 32MiB */ + optee_core@be000000 { + reg = <0x00 0xbe000000 0x00 0x2000000>; + }; + + /* 4K */ + optee_shm@bfc00000 { + reg = <0x00 0xbfc00000 0x00 0x400000>; + }; + }; + binman: binman { multiple-images; }; @@ -152,6 +175,19 @@ }; }; + tee { + description = "TEE firmware"; + type = "firmware"; + arch = "arm64"; + compression = "none"; + load = <0xbe000000>; + entry = <0xbe000000>; + + tee_blob: blob-ext { + filename = "tee.bin"; + }; + }; + @fdt-SEQ { description = "NAME"; type = "flat_dt"; @@ -165,7 +201,7 @@ @config-SEQ { description = "NAME"; firmware = "uboot"; - loadables = "atf"; + loadables = "atf", "tee"; fdt = "fdt-SEQ"; }; }; I have done all of the above using imx-atf [1] and imx-optee-os [2] on a 2GiB IMX8MM board and successfully boot to U-Boot where I verify dek_blob works, and then successfully booting to Linux where I verify that the Linux optee driver probes. But, what confuses me relates to optee memory usage and location (TEE_LOAD_ADDR) based on testing on 1GiB and 4GiB IMX8MM boards: - CFG_DDR_SIZE = 0x40000000 (1GiB); TEE_LOAD_ADDR=0x7e000000 - panic's at core/arch/arm/mm/core_mmu.c <check_pa_matches_va> on a 1GiB DRAM board (why? makes no sense) - panic's at core/arch/arm/mm/core_mmu.c <check_pa_matches_va> on a 2GiB DRAM board (this makes sense as 0x7e000000 should be out of bounds!) - hangs in opteed_init on a 4GiB DRAM board (why?) - CFG_DDR_SIZE = 0x80000000 (2GiB); TEE_LOAD_ADDR=0xbe000000 - boots on a 1GiB DRAM board... shouldn't that crash when accessing 0xbe000000? - boots on a 2GiB DRAM board - boots on a 4GiB DRAM board CFG_DDR_SIZE = 0x100000000 (4GiB); TEE_LOAD_ADDR=0x13e000000 - fails building: 'Error: u-boot.its:63.13-24 Value out of range for 32-bit array element' - also this may be an issue as currently IMX8MM boards have DRAM limited to 3GiB due to various IP drivers not being 64bit ready Also, I find that the memory reservations produce an error when I TFTP a file due to (tftp_init_load_addr -> lmb_init_and_reserve) u-boot=> tftpboot $fdt_addr_r venice/imx8mm-venice-gw73xx-0x.dtb Using ethernet@30be0000 device TFTP from server 192.168.1.146; our IP address is 192.168.1.1 Filename 'venice/imx8mm-venice-gw73xx-0x.dtb'. ERROR: reserving fdt memory region failed (addr=bfc00000 size=400000 flags=0) Can anyone explain the memory configurations needed for IMX8MM OP-TEE? Best regards, Tim [1] https://source.codeaurora.org/external/imx/imx-atf/ [2] https://source.codeaurora.org/external/imx/imx-optee-os/
Hi Tim, On Thu, Oct 14, 2021 at 10:03 PM Tim Harvey <tharvey@gateworks.com> wrote: > > On Mon, Oct 11, 2021 at 3:15 PM Tim Harvey <tharvey@gateworks.com> wrote: > > > > Greetings, > > > > Is anyone successfully booting U-Boot with OP-TEE support on the IMX8M? > > > > My understanding is that you need to add tee.bin to the images in the > > FIT image and include it in loadables following the ATF. > > > > While this was done with arch/arm/mach-imx/mkimage_fit_atf.sh before > > the switch to binman by simply having tee.bin in the U-Boot directory > > and passing in TEE_LOAD_ADDR (or accepting the default of 0xfe000000) > > once you switch to binman it needs to be added for your board. > > > > Additionally in order to use OP-TEE from U-Boot (ie for dek_blob > > command) you need to add a node with compatible=linaro,optee-tz as > > well. > > > > I've done the following to add OP-TEE for imx8mm_venice: > > diff --git a/configs/imx8mm_venice_defconfig b/configs/imx8mm_venice_defconfig > > index d85827b588..85d2b20810 100644 > > -u-boot.dtsi > > index e0fa9ff4bf..7a71b974e1 100644 > > --- a/arch/arm/dts/imx8mm-venice-u-boot.dtsi > > +++ b/arch/arm/dts/imx8mm-venice-u-boot.dtsi > > @@ -10,6 +10,13 @@ > > multiple-images; > > }; > > > > + firmware { > > + optee { > > + compatible = "linaro,optee-tz"; > > + method = "smc"; > > + }; > > + }; > > + > > wdt-reboot { > > compatible = "wdt-reboot"; > > wdt = <&wdog1>; > > @@ -152,6 +159,16 @@ > > }; > > }; > > > > + tee { > > + description = "TEE firmware"; > > + type = "firmware"; > > + arch = "arm64"; > > + compression = "none"; > > + data = "tee.bin"; > > + load = <0xbe000000>; > > + entry = <0xbe000000>; > > + }; > > + > > @fdt-SEQ { > > description = "NAME"; > > type = "flat_dt"; > > @@ -165,7 +182,7 @@ > > @config-SEQ { > > description = "NAME"; > > firmware = "uboot"; > > - loadables = "atf"; > > + loadables = "atf", "tee"; > > fdt = "fdt-SEQ"; > > }; > > }; > > > > > > However, when I attempt to boot I hang when the ATF is run. > > > > Where does the TEE_LOAD_ADDR come from specifically? I would think > > this needs to be defined when building tee and needs to match the load > > address used in the FIT image. It appears that perhaps this is supped > > to be DDR_BASE + DDR_SIZE - 32MIB but I'm not entirely sure. > > > > I'm currently using NXP's ATF (imx_5.4.3_2.0.0) and NXP's TEE > > (imx_5.4.70_2.3.0) and would also like to understand if NXP's branches > > are strictly required here and if so what the pros and cons of using > > them are. > > > > Anyone using IMX8MM OP-TEE that could point me in the right direction? > > > > Here's what I've learned so far via various responses, research and testing. > > There are multiple boot flows that can be used with OP-TEE: > 1. SPL -> ATF -> OP-TEE -> U-boot -> LInux This is the case for ARMv8: TF-A provides runtime service here as Secure Monitor (running on EL3), arbitrating context switches between Secure/Non-secure worlds. OP-TEE OS core is running on Secure EL1. > 2. SPL -> OP-TEE -> U-boot -> Linux This is the case for ARMv7, where there is only one Secure PL1, so OP-TEE also fulfills the function of SM (arbitrating context switches between NW/SW). > > I'm not really clear what the pros and cons of each are - can anyone > shed some light on this? > > Method #1 is what I've found the most info on and have mostly gotten > working so I will go over that here. As IMX8MM runs on Cortex-A Arm > TrustZone is available and thus TF-A can be used. > > By the way, I'm a bit confused as to ATF vs TF-A. From what I can find: > - ATF (Arm Trusted Firmware) is a Trusted Firmware reference > implementation of secure world software for Armv7-A, Armv8-A, and > Armv8-M architectures. > - TF-A (Trusted Firmware-A) also appears to provide the same thing > Are these the same? Which one is it that NXP uses for imx-atf [1] It's the same product, which I guess was renamed to avoid confusion, as ARM introduced Trusted Firmware M for M-cores. > > When OP-TEE is used with TF-A the TF-A gets invoked with the TEE as > bl32 and main u-boot as bl33. Once it has done its startup TF-A jumps > into the bl32(OP-TEE) for the TEE startup, returns to TF-A and then > jumps to bl33(U-Boot). > > For this boot flow on IMX8MM you need to do the following: > - build TF-A (bl31.bin) with OP-TEE as a Secure Payload Dispatcher > (SPD) (add 'SPD=opteed' to env) > - build OP-TEE (tee.bin) > - add tee.bin to U-Boot FIT image along with TF-A binary in images node > * for boards still using arch/arm/mach-imx/mkimage_fit_atf.sh FIT > generation this is done automatically when tee.bin is U-Boot's build > directory but you also need to set TEE_LOAD_ADDR env to specify the > load addr of the TEE which depends on the board DRAM size > * for boards using binman the generation of the FIT image has moved > to using the 'binman' tool thus adding tee.bin requires you to modify > your board's u-boot.dtsi itb node to add tee.bin to itb/fit/images/ > and add a reference to it in the loadables property of your boot > configuration following atf. This is shown in my patch above except > there I neglected to add the required 'tee_blob: blob-ext { filename = > "tee.bin"; }. Here you must set the TEE_LOAD_ADDR as the load address > manually. > - To enable OP-TEE support in U-Boot a /firmware/optee node needs to > be added to your board u-boot.dtsi with 'compatible = > "linaro,optee-tz"' and 'method = "smc"' nodes > * This node is used by the U-Boot dek_blob command Well, to be more precise it's just for probing optee (drivers/tee/optee) driver. It can be also used in AVB 2.0 implementation (in this case OP-TEE is used for RPMB eMMC access for storing rollback indexes or other persistent values to RPMB) > * If present then U-Boot will also copy it to your Linux dtb via > image_setup_libfdt->optee_copy_fdt_nodes > - To enable dek_blob in U-Boot enable CONFIG_CMD_DEKBLOB=y which will > enable CONFIG_IMX_OPTEE_DEK_ENCAP=y > - To enable optee driver in Linux enable CONFIG_TEE and CONFIG_OPTEE > and optionally CONFIG_HW_RANDOM_OPTEE > - Also, I believe you need to add memory reservations in the dt via a > /reserved-memory/optee_core and /reserved-memory/optee_shm which will > also be copied to the linux dtb as well if present and also depend on > the OP-TEE memory location/configuration > > If you are using NXP's imx-optee-os [2] for IMX8MM then you can use > the 'mx8mmevk' platform which simply configures the following which > can be overridden via env: > CFG_UART_BASE ?= UART2_BASE > CFG_DDR_SIZE ?= 0x80000000 > > In core/arch/arm/plat-imx/conf.mk for imx8mm CFG_DDR_SIZE is used to > compute CFG_TZDRAM_START where optee gets loaded (TEE_LOAD_ADDR): > # put optee end of ddr for AARCH64 > CFG_TZDRAM_START ?= ($(CFG_DRAM_BASE) - 0x02000000 + $(CFG_DDR_SIZE)) > CFG_TZDRAM_SIZE ?= 0x01c00000 > CFG_SHMEM_START ?= ($(CFG_TZDRAM_START) + $(CFG_TZDRAM_SIZE)) > CFG_SHMEM_SIZE ?= 0x00400000 > > thus: > CFG_DDR_SIZE = 0x40000000 (1GiB) results in > CFG_TZDRAM_START=0x7e000000 (TEE_LOAD_ADDR) and > CFG_SHMEM_START=0x7fc00000 > CFG_DDR_SIZE = 0x80000000 (2GiB) results in > CFG_TZDRAM_START=0xbe000000 (TEE_LOAD_ADDR) and > CFG_SHMEM_START=0xbfc00000 > CFG_DDR_SIZE = 0x100000000 (4GiB) results in > CFG_TZDRAM_START=0x13e000000 (TEE_LOAD_ADDR) and > CFG_SHMEM_START=0x13fc00000 > > So for a 2GiB board your dt changes for a board already using binman > should look like this: > @@ -6,6 +6,29 @@ > #include "imx8mm-u-boot.dtsi" > > / { > + firmware { > + optee { > + compatible = "linaro,optee-tz"; > + method = "smc"; > + }; > + }; > + > + reserved-memory { > + #address-cells = <2>; > + #size-cells = <2>; > + ranges; > + > + /* 32MiB */ > + optee_core@be000000 { > + reg = <0x00 0xbe000000 0x00 0x2000000>; > + }; > + > + /* 4K */ > + optee_shm@bfc00000 { > + reg = <0x00 0xbfc00000 0x00 0x400000>; > + }; > + }; > + > binman: binman { > multiple-images; > }; > @@ -152,6 +175,19 @@ > }; > }; > > + tee { > + description = "TEE firmware"; > + type = "firmware"; > + arch = "arm64"; > + compression = "none"; > + load = <0xbe000000>; > + entry = <0xbe000000>; > + > + tee_blob: blob-ext { > + filename = "tee.bin"; > + }; > + }; > + > @fdt-SEQ { > description = "NAME"; > type = "flat_dt"; > @@ -165,7 +201,7 @@ > @config-SEQ { > description = "NAME"; > firmware = "uboot"; > - loadables = "atf"; > + loadables = "atf", "tee"; > fdt = "fdt-SEQ"; > }; > }; > > I have done all of the above using imx-atf [1] and imx-optee-os [2] on > a 2GiB IMX8MM board and successfully boot to U-Boot where I verify > dek_blob works, and then successfully booting to Linux where I verify > that the Linux optee driver probes. > > But, what confuses me relates to optee memory usage and location > (TEE_LOAD_ADDR) based on testing on 1GiB and 4GiB IMX8MM boards: > - CFG_DDR_SIZE = 0x40000000 (1GiB); TEE_LOAD_ADDR=0x7e000000 > - panic's at core/arch/arm/mm/core_mmu.c <check_pa_matches_va> on a > 1GiB DRAM board (why? makes no sense) > - panic's at core/arch/arm/mm/core_mmu.c <check_pa_matches_va> on a > 2GiB DRAM board (this makes sense as 0x7e000000 should be out of > bounds!) > - hangs in opteed_init on a 4GiB DRAM board (why?) > - CFG_DDR_SIZE = 0x80000000 (2GiB); TEE_LOAD_ADDR=0xbe000000 > - boots on a 1GiB DRAM board... shouldn't that crash when accessing > 0xbe000000? > - boots on a 2GiB DRAM board > - boots on a 4GiB DRAM board > CFG_DDR_SIZE = 0x100000000 (4GiB); TEE_LOAD_ADDR=0x13e000000 > - fails building: 'Error: u-boot.its:63.13-24 Value out of range for > 32-bit array element' > - also this may be an issue as currently IMX8MM boards have DRAM > limited to 3GiB due to various IP drivers not being 64bit ready > > Also, I find that the memory reservations produce an error when I TFTP > a file due to (tftp_init_load_addr -> lmb_init_and_reserve) > u-boot=> tftpboot $fdt_addr_r venice/imx8mm-venice-gw73xx-0x.dtb > Using ethernet@30be0000 device > TFTP from server 192.168.1.146; our IP address is 192.168.1.1 > Filename 'venice/imx8mm-venice-gw73xx-0x.dtb'. > ERROR: reserving fdt memory region failed (addr=bfc00000 size=400000 flags=0) > > Can anyone explain the memory configurations needed for IMX8MM OP-TEE? Well, I'd suggest just to take existing buildroot based setup with vanilla OP-TEE for Verdin iMX8MM ([1], [2], [3]), which I created ~2 years ago and just take a look what build flags are used there. I'm not sure if it's still works, as I don't have Verdin SoM at hand, however this definitely will save you a lot of time trying to build/bring up everything manually. Also there is a makefile for iMX8MQ EVK [4], which is currently maintained by Jens Wiklander from Linaro (never tried to use it, but looks like it's regularly updated, so it should just work out of the box). [1] https://github.com/OP-TEE/manifest/blob/master/verdin.xml [2] https://github.com/OP-TEE/build/blob/master/verdin.mk [3] https://optee.readthedocs.io/en/latest/building/index.html [4] https://github.com/OP-TEE/build/blob/master/imx.mk > > Best regards, > > Tim > [1] https://source.codeaurora.org/external/imx/imx-atf/ > [2] https://source.codeaurora.org/external/imx/imx-optee-os/ Hope that helps! Regards, Igor -- Best regards - Freundliche Grüsse - Meilleures salutations Igor Opaniuk Embedded Software Engineer T: +380 938364067 E: igor.opaniuk@foundries.io W: www.foundries.io
On Wed, Oct 20, 2021 at 9:47 AM Igor Opaniuk <igor.opaniuk@foundries.io> wrote: > > Hi Tim, > > On Thu, Oct 14, 2021 at 10:03 PM Tim Harvey <tharvey@gateworks.com> wrote: > > > > On Mon, Oct 11, 2021 at 3:15 PM Tim Harvey <tharvey@gateworks.com> wrote: > > > > > > Greetings, > > > > > > Is anyone successfully booting U-Boot with OP-TEE support on the IMX8M? > > > > > > My understanding is that you need to add tee.bin to the images in the > > > FIT image and include it in loadables following the ATF. > > > > > > While this was done with arch/arm/mach-imx/mkimage_fit_atf.sh before > > > the switch to binman by simply having tee.bin in the U-Boot directory > > > and passing in TEE_LOAD_ADDR (or accepting the default of 0xfe000000) > > > once you switch to binman it needs to be added for your board. > > > > > > Additionally in order to use OP-TEE from U-Boot (ie for dek_blob > > > command) you need to add a node with compatible=linaro,optee-tz as > > > well. > > > > > > I've done the following to add OP-TEE for imx8mm_venice: > > > diff --git a/configs/imx8mm_venice_defconfig b/configs/imx8mm_venice_defconfig > > > index d85827b588..85d2b20810 100644 > > > -u-boot.dtsi > > > index e0fa9ff4bf..7a71b974e1 100644 > > > --- a/arch/arm/dts/imx8mm-venice-u-boot.dtsi > > > +++ b/arch/arm/dts/imx8mm-venice-u-boot.dtsi > > > @@ -10,6 +10,13 @@ > > > multiple-images; > > > }; > > > > > > + firmware { > > > + optee { > > > + compatible = "linaro,optee-tz"; > > > + method = "smc"; > > > + }; > > > + }; > > > + > > > wdt-reboot { > > > compatible = "wdt-reboot"; > > > wdt = <&wdog1>; > > > @@ -152,6 +159,16 @@ > > > }; > > > }; > > > > > > + tee { > > > + description = "TEE firmware"; > > > + type = "firmware"; > > > + arch = "arm64"; > > > + compression = "none"; > > > + data = "tee.bin"; > > > + load = <0xbe000000>; > > > + entry = <0xbe000000>; > > > + }; > > > + > > > @fdt-SEQ { > > > description = "NAME"; > > > type = "flat_dt"; > > > @@ -165,7 +182,7 @@ > > > @config-SEQ { > > > description = "NAME"; > > > firmware = "uboot"; > > > - loadables = "atf"; > > > + loadables = "atf", "tee"; > > > fdt = "fdt-SEQ"; > > > }; > > > }; > > > > > > > > > However, when I attempt to boot I hang when the ATF is run. > > > > > > Where does the TEE_LOAD_ADDR come from specifically? I would think > > > this needs to be defined when building tee and needs to match the load > > > address used in the FIT image. It appears that perhaps this is supped > > > to be DDR_BASE + DDR_SIZE - 32MIB but I'm not entirely sure. > > > > > > I'm currently using NXP's ATF (imx_5.4.3_2.0.0) and NXP's TEE > > > (imx_5.4.70_2.3.0) and would also like to understand if NXP's branches > > > are strictly required here and if so what the pros and cons of using > > > them are. > > > > > > Anyone using IMX8MM OP-TEE that could point me in the right direction? > > > > > > > Here's what I've learned so far via various responses, research and testing. > > > > There are multiple boot flows that can be used with OP-TEE: > > 1. SPL -> ATF -> OP-TEE -> U-boot -> LInux > This is the case for ARMv8: TF-A provides runtime service here as > Secure Monitor (running on EL3), arbitrating context switches between > Secure/Non-secure worlds. OP-TEE OS core is running on Secure EL1. > > > 2. SPL -> OP-TEE -> U-boot -> Linux > This is the case for ARMv7, where there is only one Secure PL1, > so OP-TEE also fulfills the function of SM (arbitrating context > switches between NW/SW). > Igor, Thanks for the explanation - this makes much more sense now! > > > > I'm not really clear what the pros and cons of each are - can anyone > > shed some light on this? > > > > Method #1 is what I've found the most info on and have mostly gotten > > working so I will go over that here. As IMX8MM runs on Cortex-A Arm > > TrustZone is available and thus TF-A can be used. > > > > By the way, I'm a bit confused as to ATF vs TF-A. From what I can find: > > - ATF (Arm Trusted Firmware) is a Trusted Firmware reference > > implementation of secure world software for Armv7-A, Armv8-A, and > > Armv8-M architectures. > > - TF-A (Trusted Firmware-A) also appears to provide the same thing > > Are these the same? Which one is it that NXP uses for imx-atf [1] > It's the same product, which I guess was renamed to avoid confusion, > as ARM introduced Trusted Firmware M for M-cores. > > > > > > When OP-TEE is used with TF-A the TF-A gets invoked with the TEE as > > bl32 and main u-boot as bl33. Once it has done its startup TF-A jumps > > into the bl32(OP-TEE) for the TEE startup, returns to TF-A and then > > jumps to bl33(U-Boot). > > > > For this boot flow on IMX8MM you need to do the following: > > - build TF-A (bl31.bin) with OP-TEE as a Secure Payload Dispatcher > > (SPD) (add 'SPD=opteed' to env) > > - build OP-TEE (tee.bin) > > - add tee.bin to U-Boot FIT image along with TF-A binary in images node > > * for boards still using arch/arm/mach-imx/mkimage_fit_atf.sh FIT > > generation this is done automatically when tee.bin is U-Boot's build > > directory but you also need to set TEE_LOAD_ADDR env to specify the > > load addr of the TEE which depends on the board DRAM size > > * for boards using binman the generation of the FIT image has moved > > to using the 'binman' tool thus adding tee.bin requires you to modify > > your board's u-boot.dtsi itb node to add tee.bin to itb/fit/images/ > > and add a reference to it in the loadables property of your boot > > configuration following atf. This is shown in my patch above except > > there I neglected to add the required 'tee_blob: blob-ext { filename = > > "tee.bin"; }. Here you must set the TEE_LOAD_ADDR as the load address > > manually. > > - To enable OP-TEE support in U-Boot a /firmware/optee node needs to > > be added to your board u-boot.dtsi with 'compatible = > > "linaro,optee-tz"' and 'method = "smc"' nodes > > * This node is used by the U-Boot dek_blob command > Well, to be more precise it's just for probing optee (drivers/tee/optee) > driver. It can be also used in AVB 2.0 implementation (in this case OP-TEE > is used for RPMB eMMC access for storing > rollback indexes or other persistent values to RPMB) > > > * If present then U-Boot will also copy it to your Linux dtb via > > image_setup_libfdt->optee_copy_fdt_nodes > > - To enable dek_blob in U-Boot enable CONFIG_CMD_DEKBLOB=y which will > > enable CONFIG_IMX_OPTEE_DEK_ENCAP=y > > - To enable optee driver in Linux enable CONFIG_TEE and CONFIG_OPTEE > > and optionally CONFIG_HW_RANDOM_OPTEE > > - Also, I believe you need to add memory reservations in the dt via a > > /reserved-memory/optee_core and /reserved-memory/optee_shm which will > > also be copied to the linux dtb as well if present and also depend on > > the OP-TEE memory location/configuration > > > > If you are using NXP's imx-optee-os [2] for IMX8MM then you can use > > the 'mx8mmevk' platform which simply configures the following which > > can be overridden via env: > > CFG_UART_BASE ?= UART2_BASE > > CFG_DDR_SIZE ?= 0x80000000 > > > > In core/arch/arm/plat-imx/conf.mk for imx8mm CFG_DDR_SIZE is used to > > compute CFG_TZDRAM_START where optee gets loaded (TEE_LOAD_ADDR): > > # put optee end of ddr for AARCH64 > > CFG_TZDRAM_START ?= ($(CFG_DRAM_BASE) - 0x02000000 + $(CFG_DDR_SIZE)) > > CFG_TZDRAM_SIZE ?= 0x01c00000 > > CFG_SHMEM_START ?= ($(CFG_TZDRAM_START) + $(CFG_TZDRAM_SIZE)) > > CFG_SHMEM_SIZE ?= 0x00400000 > > > > thus: > > CFG_DDR_SIZE = 0x40000000 (1GiB) results in > > CFG_TZDRAM_START=0x7e000000 (TEE_LOAD_ADDR) and > > CFG_SHMEM_START=0x7fc00000 > > CFG_DDR_SIZE = 0x80000000 (2GiB) results in > > CFG_TZDRAM_START=0xbe000000 (TEE_LOAD_ADDR) and > > CFG_SHMEM_START=0xbfc00000 > > CFG_DDR_SIZE = 0x100000000 (4GiB) results in > > CFG_TZDRAM_START=0x13e000000 (TEE_LOAD_ADDR) and > > CFG_SHMEM_START=0x13fc00000 > > > > So for a 2GiB board your dt changes for a board already using binman > > should look like this: > > @@ -6,6 +6,29 @@ > > #include "imx8mm-u-boot.dtsi" > > > > / { > > + firmware { > > + optee { > > + compatible = "linaro,optee-tz"; > > + method = "smc"; > > + }; > > + }; > > + > > + reserved-memory { > > + #address-cells = <2>; > > + #size-cells = <2>; > > + ranges; > > + > > + /* 32MiB */ > > + optee_core@be000000 { > > + reg = <0x00 0xbe000000 0x00 0x2000000>; > > + }; > > + > > + /* 4K */ > > + optee_shm@bfc00000 { > > + reg = <0x00 0xbfc00000 0x00 0x400000>; > > + }; > > + }; > > + > > binman: binman { > > multiple-images; > > }; > > @@ -152,6 +175,19 @@ > > }; > > }; > > > > + tee { > > + description = "TEE firmware"; > > + type = "firmware"; > > + arch = "arm64"; > > + compression = "none"; > > + load = <0xbe000000>; > > + entry = <0xbe000000>; > > + > > + tee_blob: blob-ext { > > + filename = "tee.bin"; > > + }; > > + }; > > + > > @fdt-SEQ { > > description = "NAME"; > > type = "flat_dt"; > > @@ -165,7 +201,7 @@ > > @config-SEQ { > > description = "NAME"; > > firmware = "uboot"; > > - loadables = "atf"; > > + loadables = "atf", "tee"; > > fdt = "fdt-SEQ"; > > }; > > }; > > > > I have done all of the above using imx-atf [1] and imx-optee-os [2] on > > a 2GiB IMX8MM board and successfully boot to U-Boot where I verify > > dek_blob works, and then successfully booting to Linux where I verify > > that the Linux optee driver probes. > > > > But, what confuses me relates to optee memory usage and location > > (TEE_LOAD_ADDR) based on testing on 1GiB and 4GiB IMX8MM boards: > > - CFG_DDR_SIZE = 0x40000000 (1GiB); TEE_LOAD_ADDR=0x7e000000 > > - panic's at core/arch/arm/mm/core_mmu.c <check_pa_matches_va> on a > > 1GiB DRAM board (why? makes no sense) > > - panic's at core/arch/arm/mm/core_mmu.c <check_pa_matches_va> on a > > 2GiB DRAM board (this makes sense as 0x7e000000 should be out of > > bounds!) > > - hangs in opteed_init on a 4GiB DRAM board (why?) > > - CFG_DDR_SIZE = 0x80000000 (2GiB); TEE_LOAD_ADDR=0xbe000000 > > - boots on a 1GiB DRAM board... shouldn't that crash when accessing > > 0xbe000000? > > - boots on a 2GiB DRAM board > > - boots on a 4GiB DRAM board > > CFG_DDR_SIZE = 0x100000000 (4GiB); TEE_LOAD_ADDR=0x13e000000 > > - fails building: 'Error: u-boot.its:63.13-24 Value out of range for > > 32-bit array element' > > - also this may be an issue as currently IMX8MM boards have DRAM > > limited to 3GiB due to various IP drivers not being 64bit ready > > > > Also, I find that the memory reservations produce an error when I TFTP > > a file due to (tftp_init_load_addr -> lmb_init_and_reserve) > > u-boot=> tftpboot $fdt_addr_r venice/imx8mm-venice-gw73xx-0x.dtb > > Using ethernet@30be0000 device > > TFTP from server 192.168.1.146; our IP address is 192.168.1.1 > > Filename 'venice/imx8mm-venice-gw73xx-0x.dtb'. > > ERROR: reserving fdt memory region failed (addr=bfc00000 size=400000 flags=0) > > > > Can anyone explain the memory configurations needed for IMX8MM OP-TEE? > > Well, I'd suggest just to take existing buildroot based setup with > vanilla OP-TEE for > Verdin iMX8MM ([1], [2], [3]), which I created ~2 years ago and just take a look > what build flags are used there. I'm not sure if it's still works, as > I don't have > Verdin SoM at hand, however this definitely will save you a lot of > time trying to build/bring > up everything manually. > > Also there is a makefile for iMX8MQ EVK [4], which is currently maintained by > Jens Wiklander from Linaro (never tried to use it, but looks like it's > regularly updated, so > it should just work out of the box). > > [1] https://github.com/OP-TEE/manifest/blob/master/verdin.xml > [2] https://github.com/OP-TEE/build/blob/master/verdin.mk > [3] https://optee.readthedocs.io/en/latest/building/index.html > [4] https://github.com/OP-TEE/build/blob/master/imx.mk I did manage to get OP-TEE booting properly on 1GiB IMX8MM boards. An additional step above is to set BL32_BASE to what I calculate above as TEE_LOAD_ADDR when building the ATF as by default it is set in plat/imx/imx8m/imx8mm/platform.mk to 0xbe000000 so my issue was that the ATF jump address was wrong. Also above my /reserved-memory/optee_core node is defined wrong as a size of 32MiB (0x2000000) overflows the optee_shm region thus the optee_core size should be set to 0x1c00000 (this did not produce a failure to boot, just a kernel warning/error). I still do not have OP-TEE booting properly on a 4GiB IMX8MM board however. Placing OP-TEE at the end of DRAM as imx8mm configuration does puts it in a 64bit address space which if defined properly in the its file (ie TEE_LOAD_ADDRESS="0x1 0x3fc00000" for pre-binman boards) ends up failing to boot via eMMC due to the emmc driver (drivers/mmc/fsl_esdhc_imx.c) not yet supporting 64bit addressing. I have tried moving OP-TEE to lower memory such as the IMX8MN does by specifying BL32_BASE=0x56000000 when building the ATF and TEE_LOAD_ADDRESS=0x56000000 when building a pre-binman U-Boot) but this ended up hanging in the ATF as well for still unknown reasons. Best regards, Tim
diff --git a/configs/imx8mm_venice_defconfig b/configs/imx8mm_venice_defconfig index d85827b588..85d2b20810 100644 -u-boot.dtsi index e0fa9ff4bf..7a71b974e1 100644 --- a/arch/arm/dts/imx8mm-venice-u-boot.dtsi +++ b/arch/arm/dts/imx8mm-venice-u-boot.dtsi @@ -10,6 +10,13 @@ multiple-images; }; + firmware { + optee { + compatible = "linaro,optee-tz"; + method = "smc"; + }; + }; + wdt-reboot { compatible = "wdt-reboot"; wdt = <&wdog1>; @@ -152,6 +159,16 @@ }; }; + tee { + description = "TEE firmware"; + type = "firmware"; + arch = "arm64"; + compression = "none"; + data = "tee.bin"; + load = <0xbe000000>; + entry = <0xbe000000>; + }; + @fdt-SEQ { description = "NAME"; type = "flat_dt"; @@ -165,7 +182,7 @@ @config-SEQ { description = "NAME"; firmware = "uboot"; - loadables = "atf"; + loadables = "atf", "tee"; fdt = "fdt-SEQ"; }; };