mbox series

[0/5] qemu-arm64: Allow booting via Trusted Firmware

Message ID 20200924001715.30975-1-andre.przywara@arm.com
Headers show
Series qemu-arm64: Allow booting via Trusted Firmware | expand

Message

Andre Przywara Sept. 24, 2020, 12:17 a.m. UTC
U-Boot on QEMU-arm64 can be used in two configurations: Loaded directly
via QEMU's -bios option, or as a non-secure payload (BL33) via
ARM Trusted Firmware-A (TF-A).
In the latter case we need to define CONFIG_TFABOOT, to accommodate
the first flash bank being secure only, and manually set SYS_TEXT_BASE
to the address configured in TF-A (currently 0x60000000).

To avoid this poorly documented adventure, we enable a position
independent build, and also let the flash regions be always detected
through the DTB. This results in a single build to work under both
scenarios, and also allows to move the BL33 load address in TF-A to
something lower in the future.

For this to work, we have to first make PIE work when booted from ROM.
While writing to ROM should not hurt, it might trigger CFI flash
sequences, and indeed crashes for me in the middle of the fixup routine.
This is covered by patch 1/5, which skips the whole fixup routine if the
offset is actually 0 (as it is in our case).
Also we have to decouple the relative initial stack pointer from the
PIE option, as we always need to use the fixed version, pointing to
RAM (patch 2/5).
Patch 3/5 drops the hard-coded flash address, instead U-Boot can already
read all required information from QEMU's DTB.
Patch 4/5 is a cleanup, while the last patch enables the PIE build.

With this series the very same u-boot.bin file works when directly loaded
from the QEMU command line (-bios), but also when embedded into TF-A's
fip.bin, removing the need for case-specific build options.

Please have a look!

Cheers,
Andre

Andre Przywara (5):
  arm64: PIE: Skip fixups if distance is zero
  arm64: PIE: Allow fixed stack pointer
  qemu-arm: Remove need to specify flash banks
  qemu: Drop ARCH_SUPPORT_TFABOOT
  qemu/arm64: Enable POSITION_INDEPENDENT

 arch/arm/Kconfig             | 4 ++--
 arch/arm/cpu/armv8/start.S   | 3 ++-
 configs/qemu_arm64_defconfig | 1 +
 include/configs/qemu-arm.h   | 8 +-------
 4 files changed, 6 insertions(+), 10 deletions(-)

Comments

Amit Singh Tomar Sept. 24, 2020, 7:57 a.m. UTC | #1
Hi,

Andre Przywara (5):

>   arm64: PIE: Skip fixups if distance is zero
>   arm64: PIE: Allow fixed stack pointer
>   qemu-arm: Remove need to specify flash banks
>   qemu: Drop ARCH_SUPPORT_TFABOOT
>   qemu/arm64: Enable POSITION_INDEPENDENT
>
>  arch/arm/Kconfig             | 4 ++--
>  arch/arm/cpu/armv8/start.S   | 3 ++-
>  configs/qemu_arm64_defconfig | 1 +
>  include/configs/qemu-arm.h   | 8 +-------
>  4 files changed, 6 insertions(+), 10 deletions(-)
>
> --
> 2.17.5
>
>
I tried testing this series but don't see any output while loading U-Boot
from ROM:

# ./qemu-system-aarch64 -M virt -cpu cortex-a57 -nographic -smp 1 -bios
u-boot.bin

strangely enough SP is having a value of 0 after execution:

(qemu) info registers
 PC=0000000000001a00 X00=540000a0f100303f X01=000000000007c000
X02=0000000000000000 X03=00000000401fe000 X04=0000000000000000
X05=0000000000000000 X06=0000000000000030 X07=00000000401fe008
X08=0000000000000000 X09=14000008d51e115f X10=0000000000000000
X11=0000000000000000 X12=0000000000000000 X13=0000000000000000
X14=0000000000000000 X15=0000000000000000 X16=0000000000000000
X17=0000000000000000 X18=0000000000000000 X19=0000000000000000
X20=0000000000000000 X21=0000000000000000 X22=0000000000000000
X23=0000000000000000 X24=0000000000000000 X25=0000000000000000
X26=0000000000000000 X27=0000000000000000 X28=0000000000000000
X29=00000000000000c8 X30=14000008d51e39cb  SP=0000000000000000
PSTATE=000003c5 ---- EL1h     FPCR=00000000 FPSR=00000000

Wondering , if I have missed something ?

Thanks
-Amit
Andre Przywara Sept. 24, 2020, 8:39 a.m. UTC | #2
On 24/09/2020 08:57, Amit Tomar wrote:
> Hi,
> 
> Andre Przywara (5):
> 
>       arm64: PIE: Skip fixups if distance is zero
>       arm64: PIE: Allow fixed stack pointer
>       qemu-arm: Remove need to specify flash banks
>       qemu: Drop ARCH_SUPPORT_TFABOOT
>       qemu/arm64: Enable POSITION_INDEPENDENT
> 
>      arch/arm/Kconfig             | 4 ++--
>      arch/arm/cpu/armv8/start.S   | 3 ++-
>      configs/qemu_arm64_defconfig | 1 +
>      include/configs/qemu-arm.h   | 8 +-------
>      4 files changed, 6 insertions(+), 10 deletions(-)
> 
>     -- 
>     2.17.5
> 
> 
> I tried testing this series but don't see any output while loading
> U-Boot from ROM:
> 
> # ./qemu-system-aarch64 -M virt -cpu cortex-a57 -nographic -smp 1 -bios
> u-boot.bin

I can't reproduce this (read: works for me).
I tried with cross-gcc 9.2 & QEMU 5.0.0, and with Ubuntu-arm64 (GCC 7.5
& QEMU 2.11.1).
Did you apply against master, used qemu_arm64_defconfig, cleaned the
build directory? Is your (self-built?) QEMU working? Are you using the
actual generated binary?

I pushed the branch to
https://github.com/Andre-ARM/u-boot/commits/qemu-tfa, please try this.

Cheers,
Andre

> 
> strangely enough SP is having a value of 0 after execution:
> 
> (qemu) info registers
>  PC=0000000000001a00 X00=540000a0f100303f X01=000000000007c000
> X02=0000000000000000 X03=00000000401fe000 X04=0000000000000000
> X05=0000000000000000 X06=0000000000000030 X07=00000000401fe008
> X08=0000000000000000 X09=14000008d51e115f X10=0000000000000000
> X11=0000000000000000 X12=0000000000000000 X13=0000000000000000
> X14=0000000000000000 X15=0000000000000000 X16=0000000000000000
> X17=0000000000000000 X18=0000000000000000 X19=0000000000000000
> X20=0000000000000000 X21=0000000000000000 X22=0000000000000000
> X23=0000000000000000 X24=0000000000000000 X25=0000000000000000
> X26=0000000000000000 X27=0000000000000000 X28=0000000000000000
> X29=00000000000000c8 X30=14000008d51e39cb  SP=0000000000000000
> PSTATE=000003c5 ---- EL1h     FPCR=00000000 FPSR=00000000
> 
> Wondering , if I have missed something ?
> 
> Thanks
> -Amit
Heinrich Schuchardt Sept. 24, 2020, 8:44 a.m. UTC | #3
On 24.09.20 09:57, Amit Tomar wrote:
> Hi,
>
> Andre Przywara (5):
>
>       arm64: PIE: Skip fixups if distance is zero
>       arm64: PIE: Allow fixed stack pointer
>       qemu-arm: Remove need to specify flash banks
>       qemu: Drop ARCH_SUPPORT_TFABOOT
>       qemu/arm64: Enable POSITION_INDEPENDENT
>
>      arch/arm/Kconfig             | 4 ++--
>      arch/arm/cpu/armv8/start.S   | 3 ++-
>      configs/qemu_arm64_defconfig | 1 +
>      include/configs/qemu-arm.h   | 8 +-------
>      4 files changed, 6 insertions(+), 10 deletions(-)
>
>     --
>     2.17.5
>
>
> I tried testing this series but don't see any output while loading
> U-Boot from ROM:
>
> # ./qemu-system-aarch64 -M virt -cpu cortex-a57 -nographic -smp 1 -bios
> u-boot.bin

TF-A runs at EL3 so you should test with
-machine virt,secure=true,virtualization=true

The bios parameter has to point to the TF-A bl1.bin binary. see TF-A's
docs/plat/qemu.rst: BL1 is used as the BootROM, supplied with the -bios
argument.

U-Boot is BL33.

Best regards

Heinrich

>
> strangely enough SP is having a value of 0 after execution:
>
> (qemu) info registers
>  PC=0000000000001a00 X00=540000a0f100303f X01=000000000007c000
> X02=0000000000000000 X03=00000000401fe000 X04=0000000000000000
> X05=0000000000000000 X06=0000000000000030 X07=00000000401fe008
> X08=0000000000000000 X09=14000008d51e115f X10=0000000000000000
> X11=0000000000000000 X12=0000000000000000 X13=0000000000000000
> X14=0000000000000000 X15=0000000000000000 X16=0000000000000000
> X17=0000000000000000 X18=0000000000000000 X19=0000000000000000
> X20=0000000000000000 X21=0000000000000000 X22=0000000000000000
> X23=0000000000000000 X24=0000000000000000 X25=0000000000000000
> X26=0000000000000000 X27=0000000000000000 X28=0000000000000000
> X29=00000000000000c8 X30=14000008d51e39cb  SP=0000000000000000
> PSTATE=000003c5 ---- EL1h     FPCR=00000000 FPSR=00000000
>
> Wondering , if I have missed something ?
>
> Thanks
> -Amit
Ard Biesheuvel Sept. 24, 2020, 8:52 a.m. UTC | #4
On Thu, 24 Sep 2020 at 09:58, Amit Tomar <atomar25opensource@gmail.com> wrote:
>
> Hi,
>
> Andre Przywara (5):
>>
>>   arm64: PIE: Skip fixups if distance is zero
>>   arm64: PIE: Allow fixed stack pointer
>>   qemu-arm: Remove need to specify flash banks
>>   qemu: Drop ARCH_SUPPORT_TFABOOT
>>   qemu/arm64: Enable POSITION_INDEPENDENT
>>
>>  arch/arm/Kconfig             | 4 ++--
>>  arch/arm/cpu/armv8/start.S   | 3 ++-
>>  configs/qemu_arm64_defconfig | 1 +
>>  include/configs/qemu-arm.h   | 8 +-------
>>  4 files changed, 6 insertions(+), 10 deletions(-)
>>
>> --
>> 2.17.5
>>
>
> I tried testing this series but don't see any output while loading U-Boot from ROM:
>
> # ./qemu-system-aarch64 -M virt -cpu cortex-a57 -nographic -smp 1 -bios u-boot.bin
>
> strangely enough SP is having a value of 0 after execution:
>
> (qemu) info registers
>  PC=0000000000001a00 X00=540000a0f100303f X01=000000000007c000
> X02=0000000000000000 X03=00000000401fe000 X04=0000000000000000
> X05=0000000000000000 X06=0000000000000030 X07=00000000401fe008
> X08=0000000000000000 X09=14000008d51e115f X10=0000000000000000
> X11=0000000000000000 X12=0000000000000000 X13=0000000000000000
> X14=0000000000000000 X15=0000000000000000 X16=0000000000000000
> X17=0000000000000000 X18=0000000000000000 X19=0000000000000000
> X20=0000000000000000 X21=0000000000000000 X22=0000000000000000
> X23=0000000000000000 X24=0000000000000000 X25=0000000000000000
> X26=0000000000000000 X27=0000000000000000 X28=0000000000000000
> X29=00000000000000c8 X30=14000008d51e39cb  SP=0000000000000000
> PSTATE=000003c5 ---- EL1h     FPCR=00000000 FPSR=00000000
>
> Wondering , if I have missed something ?
>

Did you regenerate the .config? Otherwise, CONFIG_INIT_SP_RELATIVE may
still be enabled.
Amit Singh Tomar Sept. 24, 2020, 9:10 a.m. UTC | #5
>
> Did you regenerate the .config? Otherwise, CONFIG_INIT_SP_RELATIVE may
> still be enabled.
>

I cloned a fresh U-boot source (with top commit 55004fa43364e) , and the
top of it applied these patches.
After generating the .config using "qemu_arm64_defconfig" where
CONFIG_INIT_SP_RELATIVE is disabled.

#
# ARM architecture
#
CONFIG_ARM64=y
CONFIG_POSITION_INDEPENDENT=y
# CONFIG_INIT_SP_RELATIVE is not set
# CONFIG_GIC_V3_ITS is not set

Using ./qemu-system-aarch64 --version
QEMU emulator version 5.0.50 (v5.0.0-2210-g45db94cc90c2-dirty)

U-boot is crossed compile using
"gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu"

Thanks
-Amit
Amit Singh Tomar Sept. 24, 2020, 9:13 a.m. UTC | #6
>
> >
> > I tried testing this series but don't see any output while loading
> > U-Boot from ROM:
> >
> > # ./qemu-system-aarch64 -M virt -cpu cortex-a57 -nographic -smp 1 -bios
> > u-boot.bin
>
> TF-A runs at EL3 so you should test with
> -machine virt,secure=true,virtualization=true
>
> The bios parameter has to point to the TF-A bl1.bin binary. see TF-A's
> docs/plat/qemu.rst: BL1 is used as the BootROM, supplied with the -bios
> argument.
>
> But, wanted to test it without loading U-boot from TF-A , for instance if
> I just disable the
>
   CONFIG_POSITION_INDEPENDENT, it works:

./qemu-system-aarch64 -M virt -cpu cortex-a57 -nographic -smp 1 -bios
u-boot.bin

U-Boot 2020.10-rc5-00020-gca11959a34f4 (Sep 24 2020 - 14:34:25 +0530)

DRAM:  128 MiB
Flash: 128 MiB
*** Warning - bad CRC, using default environment

PCI: Failed autoconfig bar 14
In:    pl011@9000000
Out:   pl011@9000000
Err:   pl011@9000000
Net:   No ethernet found.

Thanks
-Amit
Andre Przywara Sept. 24, 2020, 9:26 a.m. UTC | #7
On 24/09/2020 09:44, Heinrich Schuchardt wrote:
> On 24.09.20 09:57, Amit Tomar wrote:
>> Hi,
>>
>> Andre Przywara (5):
>>
>>       arm64: PIE: Skip fixups if distance is zero
>>       arm64: PIE: Allow fixed stack pointer
>>       qemu-arm: Remove need to specify flash banks
>>       qemu: Drop ARCH_SUPPORT_TFABOOT
>>       qemu/arm64: Enable POSITION_INDEPENDENT
>>
>>      arch/arm/Kconfig             | 4 ++--
>>      arch/arm/cpu/armv8/start.S   | 3 ++-
>>      configs/qemu_arm64_defconfig | 1 +
>>      include/configs/qemu-arm.h   | 8 +-------
>>      4 files changed, 6 insertions(+), 10 deletions(-)
>>
>>     --
>>     2.17.5
>>
>>
>> I tried testing this series but don't see any output while loading
>> U-Boot from ROM:
>>
>> # ./qemu-system-aarch64 -M virt -cpu cortex-a57 -nographic -smp 1 -bios
>> u-boot.bin
> 
> TF-A runs at EL3 so you should test with
> -machine virt,secure=true,virtualization=true

This is true, but the point of this series is to work both within TF-A
and without it. So it should work (and does for me!) with Amit's line above.

> The bios parameter has to point to the TF-A bl1.bin binary. see TF-A's
> docs/plat/qemu.rst: BL1 is used as the BootROM, supplied with the -bios
> argument.

Yeah, and it seems to be even more complicated, since you have to glue
fip.bin within a certain offset to bl1.bin, into one file to give to
-bios. I use:
$ cp build/qemu/debug/bl1.bin flash.bin
$ dd if=build/qemu/debug/fip.bin of=flash.bin bs=4k seek=64

... and make sure to give QEMU more than 512MB of RAM, since the default
load address for U-Boot is there. Actually fixing this annoyance was the
main motivation for this series.

Once people agreed that this series is making some sense, I plan to move
the load address in TF-A and improve the documentation in both TF-A and
QEMU on this.

Thanks!
Andre

> 
> U-Boot is BL33.
> 
> Best regards
> 
> Heinrich
> 
>>
>> strangely enough SP is having a value of 0 after execution:
>>
>> (qemu) info registers
>>  PC=0000000000001a00 X00=540000a0f100303f X01=000000000007c000
>> X02=0000000000000000 X03=00000000401fe000 X04=0000000000000000
>> X05=0000000000000000 X06=0000000000000030 X07=00000000401fe008
>> X08=0000000000000000 X09=14000008d51e115f X10=0000000000000000
>> X11=0000000000000000 X12=0000000000000000 X13=0000000000000000
>> X14=0000000000000000 X15=0000000000000000 X16=0000000000000000
>> X17=0000000000000000 X18=0000000000000000 X19=0000000000000000
>> X20=0000000000000000 X21=0000000000000000 X22=0000000000000000
>> X23=0000000000000000 X24=0000000000000000 X25=0000000000000000
>> X26=0000000000000000 X27=0000000000000000 X28=0000000000000000
>> X29=00000000000000c8 X30=14000008d51e39cb  SP=0000000000000000
>> PSTATE=000003c5 ---- EL1h     FPCR=00000000 FPSR=00000000
>>
>> Wondering , if I have missed something ?
>>
>> Thanks
>> -Amit
>
Tom Rini Sept. 29, 2020, 1:01 p.m. UTC | #8
On Thu, Sep 24, 2020 at 01:17:10AM +0100, Andre Przywara wrote:

> U-Boot on QEMU-arm64 can be used in two configurations: Loaded directly
> via QEMU's -bios option, or as a non-secure payload (BL33) via
> ARM Trusted Firmware-A (TF-A).
> In the latter case we need to define CONFIG_TFABOOT, to accommodate
> the first flash bank being secure only, and manually set SYS_TEXT_BASE
> to the address configured in TF-A (currently 0x60000000).
> 
> To avoid this poorly documented adventure, we enable a position
> independent build, and also let the flash regions be always detected
> through the DTB. This results in a single build to work under both
> scenarios, and also allows to move the BL33 load address in TF-A to
> something lower in the future.
> 
> For this to work, we have to first make PIE work when booted from ROM.
> While writing to ROM should not hurt, it might trigger CFI flash
> sequences, and indeed crashes for me in the middle of the fixup routine.
> This is covered by patch 1/5, which skips the whole fixup routine if the
> offset is actually 0 (as it is in our case).
> Also we have to decouple the relative initial stack pointer from the
> PIE option, as we always need to use the fixed version, pointing to
> RAM (patch 2/5).
> Patch 3/5 drops the hard-coded flash address, instead U-Boot can already
> read all required information from QEMU's DTB.
> Patch 4/5 is a cleanup, while the last patch enables the PIE build.
> 
> With this series the very same u-boot.bin file works when directly loaded
> from the QEMU command line (-bios), but also when embedded into TF-A's
> fip.bin, removing the need for case-specific build options.
> 
> Please have a look!

This sounds very useful.  Would it be possible to extend CI to create
the files for the TF-A case and also run that through test.py?