[U-Boot,v3,2/3] efi_loader: ARM: run EFI payloads non-secure

Message ID 20180613224108.13372-3-kettenis@openbsd.org
State Superseded
Delegated to: Alexander Graf
Headers show
Series
  • efi_loader: ARM: add support for ARMV7_NONSEC=y
Related show

Commit Message

Mark Kettenis June 13, 2018, 10:41 p.m.
If desired (and possible) switch into HYP mode or non-secure SVC mode
before calling the entry point of an EFI application.  This allows
U-Boot to provide a usable PSCI implementation and makes it possible
to boot kernels into hypervisor mode using an EFI bootloader.

Based on diffs from Heinrich Schuchardt and Alexander Graf.

Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
---
 cmd/bootefi.c | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

Comments

Marc Zyngier June 14, 2018, 11:54 a.m. | #1
On 13/06/18 23:41, Mark Kettenis wrote:
> If desired (and possible) switch into HYP mode or non-secure SVC mode
> before calling the entry point of an EFI application.  This allows
> U-Boot to provide a usable PSCI implementation and makes it possible
> to boot kernels into hypervisor mode using an EFI bootloader.
> 
> Based on diffs from Heinrich Schuchardt and Alexander Graf.
> 
> Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> ---
>  cmd/bootefi.c | 32 ++++++++++++++++++++++++++++++++
>  1 file changed, 32 insertions(+)
> 
> diff --git a/cmd/bootefi.c b/cmd/bootefi.c
> index 707d159bac..12a6b84ce6 100644
> --- a/cmd/bootefi.c
> +++ b/cmd/bootefi.c
> @@ -20,6 +20,11 @@
>  #include <asm-generic/unaligned.h>
>  #include <linux/linkage.h>
>  
> +#ifdef CONFIG_ARMV7_NONSEC
> +#include <asm/armv7.h>
> +#include <asm/secure.h>
> +#endif
> +
>  DECLARE_GLOBAL_DATA_PTR;
>  
>  #define OBJ_LIST_NOT_INITIALIZED 1
> @@ -189,6 +194,18 @@ static efi_status_t efi_run_in_el2(EFIAPI efi_status_t (*entry)(
>  }
>  #endif
>  
> +#ifdef CONFIG_ARMV7_NONSEC
> +static efi_status_t efi_run_in_hyp(EFIAPI efi_status_t (*entry)(
> +			efi_handle_t image_handle, struct efi_system_table *st),
> +			efi_handle_t image_handle, struct efi_system_table *st)
> +{
> +	/* Enable caches again */
> +	dcache_enable();
> +
> +	return efi_do_enter(image_handle, st, entry);
> +}
> +#endif
> +
>  /* Carve out DT reserved memory ranges */
>  static efi_status_t efi_carve_out_dt_rsv(void *fdt)
>  {
> @@ -338,6 +355,21 @@ static efi_status_t do_bootefi_exec(void *efi,
>  	}
>  #endif
>  
> +#ifdef CONFIG_ARMV7_NONSEC
> +	if (armv7_boot_nonsec()) {
> +		dcache_disable();	/* flush cache before switch to HYP */
> +

What is the rational for disabling/enabling caches across the transition
to HYP? I'm sure there is a good reason, but I'd rather see it explained
here.

> +		armv7_init_nonsec();
> +		secure_ram_addr(_do_nonsec_entry)(efi_run_in_hyp,
> +				(uintptr_t)entry,
> +				(uintptr_t)loaded_image_info_obj.handle,
> +				(uintptr_t)&systab);
> +
> +		/* Should never reach here, efi exits with longjmp */
> +		while (1) { }
> +	}
> +#endif
> +
>  	ret = efi_do_enter(loaded_image_info_obj.handle, &systab, entry);
>  
>  exit:
> 

Thanks,

	M.
Mark Kettenis June 14, 2018, 8:55 p.m. | #2
> From: Marc Zyngier <marc.zyngier@arm.com>
> Date: Thu, 14 Jun 2018 12:54:53 +0100
> 
> On 13/06/18 23:41, Mark Kettenis wrote:
> > If desired (and possible) switch into HYP mode or non-secure SVC mode
> > before calling the entry point of an EFI application.  This allows
> > U-Boot to provide a usable PSCI implementation and makes it possible
> > to boot kernels into hypervisor mode using an EFI bootloader.
> > 
> > Based on diffs from Heinrich Schuchardt and Alexander Graf.
> > 
> > Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> > ---
> >  cmd/bootefi.c | 32 ++++++++++++++++++++++++++++++++
> >  1 file changed, 32 insertions(+)
> > 
> > diff --git a/cmd/bootefi.c b/cmd/bootefi.c
> > index 707d159bac..12a6b84ce6 100644
> > --- a/cmd/bootefi.c
> > +++ b/cmd/bootefi.c
> > @@ -20,6 +20,11 @@
> >  #include <asm-generic/unaligned.h>
> >  #include <linux/linkage.h>
> >  
> > +#ifdef CONFIG_ARMV7_NONSEC
> > +#include <asm/armv7.h>
> > +#include <asm/secure.h>
> > +#endif
> > +
> >  DECLARE_GLOBAL_DATA_PTR;
> >  
> >  #define OBJ_LIST_NOT_INITIALIZED 1
> > @@ -189,6 +194,18 @@ static efi_status_t efi_run_in_el2(EFIAPI efi_status_t (*entry)(
> >  }
> >  #endif
> >  
> > +#ifdef CONFIG_ARMV7_NONSEC
> > +static efi_status_t efi_run_in_hyp(EFIAPI efi_status_t (*entry)(
> > +			efi_handle_t image_handle, struct efi_system_table *st),
> > +			efi_handle_t image_handle, struct efi_system_table *st)
> > +{
> > +	/* Enable caches again */
> > +	dcache_enable();
> > +
> > +	return efi_do_enter(image_handle, st, entry);
> > +}
> > +#endif
> > +
> >  /* Carve out DT reserved memory ranges */
> >  static efi_status_t efi_carve_out_dt_rsv(void *fdt)
> >  {
> > @@ -338,6 +355,21 @@ static efi_status_t do_bootefi_exec(void *efi,
> >  	}
> >  #endif
> >  
> > +#ifdef CONFIG_ARMV7_NONSEC
> > +	if (armv7_boot_nonsec()) {
> > +		dcache_disable();	/* flush cache before switch to HYP */
> > +
> 
> What is the rational for disabling/enabling caches across the transition
> to HYP? I'm sure there is a good reason, but I'd rather see it explained
> here.

Can't say I fully understan why.  But the AArch64 code does this as
well and if I don't flush the cache here the contents of efi_gd (which
gets initialized before the switch) sometimes gets lost.

> > +		armv7_init_nonsec();
> > +		secure_ram_addr(_do_nonsec_entry)(efi_run_in_hyp,
> > +				(uintptr_t)entry,
> > +				(uintptr_t)loaded_image_info_obj.handle,
> > +				(uintptr_t)&systab);
> > +
> > +		/* Should never reach here, efi exits with longjmp */
> > +		while (1) { }
> > +	}
> > +#endif
> > +
> >  	ret = efi_do_enter(loaded_image_info_obj.handle, &systab, entry);
> >  
> >  exit:
> >
Marc Zyngier June 15, 2018, 7:59 a.m. | #3
On 14/06/18 21:55, Mark Kettenis wrote:
>> From: Marc Zyngier <marc.zyngier@arm.com>
>> Date: Thu, 14 Jun 2018 12:54:53 +0100
>>
>> On 13/06/18 23:41, Mark Kettenis wrote:
>>> If desired (and possible) switch into HYP mode or non-secure SVC mode
>>> before calling the entry point of an EFI application.  This allows
>>> U-Boot to provide a usable PSCI implementation and makes it possible
>>> to boot kernels into hypervisor mode using an EFI bootloader.
>>>
>>> Based on diffs from Heinrich Schuchardt and Alexander Graf.
>>>
>>> Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
>>> ---
>>>  cmd/bootefi.c | 32 ++++++++++++++++++++++++++++++++
>>>  1 file changed, 32 insertions(+)
>>>
>>> diff --git a/cmd/bootefi.c b/cmd/bootefi.c
>>> index 707d159bac..12a6b84ce6 100644
>>> --- a/cmd/bootefi.c
>>> +++ b/cmd/bootefi.c
>>> @@ -20,6 +20,11 @@
>>>  #include <asm-generic/unaligned.h>
>>>  #include <linux/linkage.h>
>>>  
>>> +#ifdef CONFIG_ARMV7_NONSEC
>>> +#include <asm/armv7.h>
>>> +#include <asm/secure.h>
>>> +#endif
>>> +
>>>  DECLARE_GLOBAL_DATA_PTR;
>>>  
>>>  #define OBJ_LIST_NOT_INITIALIZED 1
>>> @@ -189,6 +194,18 @@ static efi_status_t efi_run_in_el2(EFIAPI efi_status_t (*entry)(
>>>  }
>>>  #endif
>>>  
>>> +#ifdef CONFIG_ARMV7_NONSEC
>>> +static efi_status_t efi_run_in_hyp(EFIAPI efi_status_t (*entry)(
>>> +			efi_handle_t image_handle, struct efi_system_table *st),
>>> +			efi_handle_t image_handle, struct efi_system_table *st)
>>> +{
>>> +	/* Enable caches again */
>>> +	dcache_enable();
>>> +
>>> +	return efi_do_enter(image_handle, st, entry);
>>> +}
>>> +#endif
>>> +
>>>  /* Carve out DT reserved memory ranges */
>>>  static efi_status_t efi_carve_out_dt_rsv(void *fdt)
>>>  {
>>> @@ -338,6 +355,21 @@ static efi_status_t do_bootefi_exec(void *efi,
>>>  	}
>>>  #endif
>>>  
>>> +#ifdef CONFIG_ARMV7_NONSEC
>>> +	if (armv7_boot_nonsec()) {
>>> +		dcache_disable();	/* flush cache before switch to HYP */
>>> +
>>
>> What is the rational for disabling/enabling caches across the transition
>> to HYP? I'm sure there is a good reason, but I'd rather see it explained
>> here.
> 
> Can't say I fully understan why.  But the AArch64 code does this as
> well and if I don't flush the cache here the contents of efi_gd (which
> gets initialized before the switch) sometimes gets lost.

I guess the following can happen:

- EL1 code (or SVC for AArch32) has its MMU enabled, and caches are on
- Writes from EL1 are nicely sitting in the dcache
- Enter EL2 (HYP) where the MMU is off, and thus the caches are too
- The uncached accesses do not hit in the cache, and sh*t happens

dcache_disable also cleans to the PoC, making sure that everything is
visible even when the MMU and caches are off. I have the strong feeling
that dcache_enable is utterly useless as I don't think you install any
page table at HYP (that code was never designed to run anything other
than jumping into the kernel).

It would make a lot more sense if you installed id-mapped page tables at
HYP too in order to enable the caches, and geta  bit of performance back
(otherwise anything you run at HYP will negatively compare to the speed
of an anaemic snail stuck on sand).

Thanks,

	M.
Mark Kettenis June 15, 2018, 12:51 p.m. | #4
> From: Marc Zyngier <marc.zyngier@arm.com>
> Date: Fri, 15 Jun 2018 08:59:59 +0100
> 
> On 14/06/18 21:55, Mark Kettenis wrote:
> >> From: Marc Zyngier <marc.zyngier@arm.com>
> >> Date: Thu, 14 Jun 2018 12:54:53 +0100
> >>
> >> On 13/06/18 23:41, Mark Kettenis wrote:
> >>> If desired (and possible) switch into HYP mode or non-secure SVC mode
> >>> before calling the entry point of an EFI application.  This allows
> >>> U-Boot to provide a usable PSCI implementation and makes it possible
> >>> to boot kernels into hypervisor mode using an EFI bootloader.
> >>>
> >>> Based on diffs from Heinrich Schuchardt and Alexander Graf.
> >>>
> >>> Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> >>> ---
> >>>  cmd/bootefi.c | 32 ++++++++++++++++++++++++++++++++
> >>>  1 file changed, 32 insertions(+)
> >>>
> >>> diff --git a/cmd/bootefi.c b/cmd/bootefi.c
> >>> index 707d159bac..12a6b84ce6 100644
> >>> --- a/cmd/bootefi.c
> >>> +++ b/cmd/bootefi.c
> >>> @@ -20,6 +20,11 @@
> >>>  #include <asm-generic/unaligned.h>
> >>>  #include <linux/linkage.h>
> >>>  
> >>> +#ifdef CONFIG_ARMV7_NONSEC
> >>> +#include <asm/armv7.h>
> >>> +#include <asm/secure.h>
> >>> +#endif
> >>> +
> >>>  DECLARE_GLOBAL_DATA_PTR;
> >>>  
> >>>  #define OBJ_LIST_NOT_INITIALIZED 1
> >>> @@ -189,6 +194,18 @@ static efi_status_t efi_run_in_el2(EFIAPI efi_status_t (*entry)(
> >>>  }
> >>>  #endif
> >>>  
> >>> +#ifdef CONFIG_ARMV7_NONSEC
> >>> +static efi_status_t efi_run_in_hyp(EFIAPI efi_status_t (*entry)(
> >>> +			efi_handle_t image_handle, struct efi_system_table *st),
> >>> +			efi_handle_t image_handle, struct efi_system_table *st)
> >>> +{
> >>> +	/* Enable caches again */
> >>> +	dcache_enable();
> >>> +
> >>> +	return efi_do_enter(image_handle, st, entry);
> >>> +}
> >>> +#endif
> >>> +
> >>>  /* Carve out DT reserved memory ranges */
> >>>  static efi_status_t efi_carve_out_dt_rsv(void *fdt)
> >>>  {
> >>> @@ -338,6 +355,21 @@ static efi_status_t do_bootefi_exec(void *efi,
> >>>  	}
> >>>  #endif
> >>>  
> >>> +#ifdef CONFIG_ARMV7_NONSEC
> >>> +	if (armv7_boot_nonsec()) {
> >>> +		dcache_disable();	/* flush cache before switch to HYP */
> >>> +
> >>
> >> What is the rational for disabling/enabling caches across the transition
> >> to HYP? I'm sure there is a good reason, but I'd rather see it explained
> >> here.
> > 
> > Can't say I fully understan why.  But the AArch64 code does this as
> > well and if I don't flush the cache here the contents of efi_gd (which
> > gets initialized before the switch) sometimes gets lost.
> 
> I guess the following can happen:
> 
> - EL1 code (or SVC for AArch32) has its MMU enabled, and caches are on
> - Writes from EL1 are nicely sitting in the dcache
> - Enter EL2 (HYP) where the MMU is off, and thus the caches are too
> - The uncached accesses do not hit in the cache, and sh*t happens
> 
> dcache_disable also cleans to the PoC, making sure that everything is
> visible even when the MMU and caches are off. I have the strong feeling
> that dcache_enable is utterly useless as I don't think you install any
> page table at HYP (that code was never designed to run anything other
> than jumping into the kernel).

There actually is code in arch/arm/lib/cache-cp15.c to set up the page
tables for HYP and enable the MMU.  And it would run as part of the
dcache_enable() call if CONFIG_ARMV7_LPAE was defined.  But that isn't
set on the boards I'm looking at.  I'll have a go at enabling that option.

> It would make a lot more sense if you installed id-mapped page tables at
> HYP too in order to enable the caches, and geta  bit of performance back
> (otherwise anything you run at HYP will negatively compare to the speed
> of an anaemic snail stuck on sand).

On the Allwinner A20 it certainly does crawl; console output is really
slow.  Didn't notice it on the NXP i.MX7D board though.  Anyway,
thanks for the hint!
Marc Zyngier June 15, 2018, 1:20 p.m. | #5
On 15/06/18 13:51, Mark Kettenis wrote:
>> From: Marc Zyngier <marc.zyngier@arm.com>
>> Date: Fri, 15 Jun 2018 08:59:59 +0100
>>
>> On 14/06/18 21:55, Mark Kettenis wrote:
>>>> From: Marc Zyngier <marc.zyngier@arm.com>
>>>> Date: Thu, 14 Jun 2018 12:54:53 +0100
>>>>
>>>> On 13/06/18 23:41, Mark Kettenis wrote:
>>>>> If desired (and possible) switch into HYP mode or non-secure SVC mode
>>>>> before calling the entry point of an EFI application.  This allows
>>>>> U-Boot to provide a usable PSCI implementation and makes it possible
>>>>> to boot kernels into hypervisor mode using an EFI bootloader.
>>>>>
>>>>> Based on diffs from Heinrich Schuchardt and Alexander Graf.
>>>>>
>>>>> Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
>>>>> ---
>>>>>  cmd/bootefi.c | 32 ++++++++++++++++++++++++++++++++
>>>>>  1 file changed, 32 insertions(+)
>>>>>
>>>>> diff --git a/cmd/bootefi.c b/cmd/bootefi.c
>>>>> index 707d159bac..12a6b84ce6 100644
>>>>> --- a/cmd/bootefi.c
>>>>> +++ b/cmd/bootefi.c
>>>>> @@ -20,6 +20,11 @@
>>>>>  #include <asm-generic/unaligned.h>
>>>>>  #include <linux/linkage.h>
>>>>>  
>>>>> +#ifdef CONFIG_ARMV7_NONSEC
>>>>> +#include <asm/armv7.h>
>>>>> +#include <asm/secure.h>
>>>>> +#endif
>>>>> +
>>>>>  DECLARE_GLOBAL_DATA_PTR;
>>>>>  
>>>>>  #define OBJ_LIST_NOT_INITIALIZED 1
>>>>> @@ -189,6 +194,18 @@ static efi_status_t efi_run_in_el2(EFIAPI efi_status_t (*entry)(
>>>>>  }
>>>>>  #endif
>>>>>  
>>>>> +#ifdef CONFIG_ARMV7_NONSEC
>>>>> +static efi_status_t efi_run_in_hyp(EFIAPI efi_status_t (*entry)(
>>>>> +			efi_handle_t image_handle, struct efi_system_table *st),
>>>>> +			efi_handle_t image_handle, struct efi_system_table *st)
>>>>> +{
>>>>> +	/* Enable caches again */
>>>>> +	dcache_enable();
>>>>> +
>>>>> +	return efi_do_enter(image_handle, st, entry);
>>>>> +}
>>>>> +#endif
>>>>> +
>>>>>  /* Carve out DT reserved memory ranges */
>>>>>  static efi_status_t efi_carve_out_dt_rsv(void *fdt)
>>>>>  {
>>>>> @@ -338,6 +355,21 @@ static efi_status_t do_bootefi_exec(void *efi,
>>>>>  	}
>>>>>  #endif
>>>>>  
>>>>> +#ifdef CONFIG_ARMV7_NONSEC
>>>>> +	if (armv7_boot_nonsec()) {
>>>>> +		dcache_disable();	/* flush cache before switch to HYP */
>>>>> +
>>>>
>>>> What is the rational for disabling/enabling caches across the transition
>>>> to HYP? I'm sure there is a good reason, but I'd rather see it explained
>>>> here.
>>>
>>> Can't say I fully understan why.  But the AArch64 code does this as
>>> well and if I don't flush the cache here the contents of efi_gd (which
>>> gets initialized before the switch) sometimes gets lost.
>>
>> I guess the following can happen:
>>
>> - EL1 code (or SVC for AArch32) has its MMU enabled, and caches are on
>> - Writes from EL1 are nicely sitting in the dcache
>> - Enter EL2 (HYP) where the MMU is off, and thus the caches are too
>> - The uncached accesses do not hit in the cache, and sh*t happens
>>
>> dcache_disable also cleans to the PoC, making sure that everything is
>> visible even when the MMU and caches are off. I have the strong feeling
>> that dcache_enable is utterly useless as I don't think you install any
>> page table at HYP (that code was never designed to run anything other
>> than jumping into the kernel).
> 
> There actually is code in arch/arm/lib/cache-cp15.c to set up the page
> tables for HYP and enable the MMU.  And it would run as part of the
> dcache_enable() call if CONFIG_ARMV7_LPAE was defined.  But that isn't
> set on the boards I'm looking at.  I'll have a go at enabling that option.

Yeah, you most definitely want to have that one, and LPAE is the only
thing that makes sense if you have a virtualization-capable CPU.

> 
>> It would make a lot more sense if you installed id-mapped page tables at
>> HYP too in order to enable the caches, and geta  bit of performance back
>> (otherwise anything you run at HYP will negatively compare to the speed
>> of an anaemic snail stuck on sand).
> 
> On the Allwinner A20 it certainly does crawl; console output is really
> slow.  Didn't notice it on the NXP i.MX7D board though.  Anyway,
> thanks for the hint!

No worries.

	M.
Heinrich Schuchardt Aug. 31, 2018, 5:37 p.m. | #6
On 06/14/2018 12:41 AM, Mark Kettenis wrote:
> If desired (and possible) switch into HYP mode or non-secure SVC mode
> before calling the entry point of an EFI application.  This allows
> U-Boot to provide a usable PSCI implementation and makes it possible
> to boot kernels into hypervisor mode using an EFI bootloader.
> 
> Based on diffs from Heinrich Schuchardt and Alexander Graf.
> 
> Signed-off-by: Mark Kettenis <kettenis@openbsd.org>

bootefi hello fails on vexpress_ca15_tc2_defconfig when run on qemu with

        QEMU_AUDIO_DRV=none qemu-system-arm \
        -M vexpress-a15 -cpu cortex-a15 -kernel u-boot \
        -net user -net nic,model=lan9118 \
        -m 1024M --nographic \
        -drive if=sd,file=img.vexpress,media=disk,format=raw

Bisection points to
efi_loader: ARM: run EFI payloads non-secure
commit dc500c369486fbe04000fd325c46bb309e4a1827

Best regards

Heinrich Schuchardt
Mark Kettenis Aug. 31, 2018, 6:45 p.m. | #7
> From: Heinrich Schuchardt <xypron.glpk@gmx.de>
> Date: Fri, 31 Aug 2018 19:37:25 +0200
> 
> On 06/14/2018 12:41 AM, Mark Kettenis wrote:
> > If desired (and possible) switch into HYP mode or non-secure SVC mode
> > before calling the entry point of an EFI application.  This allows
> > U-Boot to provide a usable PSCI implementation and makes it possible
> > to boot kernels into hypervisor mode using an EFI bootloader.
> > 
> > Based on diffs from Heinrich Schuchardt and Alexander Graf.
> > 
> > Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> 
> bootefi hello fails on vexpress_ca15_tc2_defconfig when run on qemu with
> 
>         QEMU_AUDIO_DRV=none qemu-system-arm \
>         -M vexpress-a15 -cpu cortex-a15 -kernel u-boot \
>         -net user -net nic,model=lan9118 \
>         -m 1024M --nographic \
>         -drive if=sd,file=img.vexpress,media=disk,format=raw

Works for me with:

$ qemu-system-arm --version
QEMU emulator version 3.0.0
Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers

=> bootefi hello
Scanning disks on mmc...
MMC Device 1 not found
MMC Device 2 not found
MMC Device 3 not found
Found 3 disks
WARNING: booting without device tree
## Starting EFI application at a0008000 ...
WARNING: using memory device/image path, this may confuse some payloads!
Hello, world!
Running on UEFI 2.7
Have SMBIOS table
Load options: <none>
## Application terminated, r = 0

That is with CONFIG_CMD_BOOTEFI_HELLO=y added to the
vexpress_ca15_tc2_defconfig of course.

> Bisection points to
> efi_loader: ARM: run EFI payloads non-secure
> commit dc500c369486fbe04000fd325c46bb309e4a1827

That suggests an issue with emulation if the mode switching
instructions or HYP support in qemu.  Or a toolchain issue of course.

Cheers,

Mark
Alexander Graf Sept. 1, 2018, 10:21 a.m. | #8
On 31.08.18 20:45, Mark Kettenis wrote:
>> From: Heinrich Schuchardt <xypron.glpk@gmx.de>
>> Date: Fri, 31 Aug 2018 19:37:25 +0200
>>
>> On 06/14/2018 12:41 AM, Mark Kettenis wrote:
>>> If desired (and possible) switch into HYP mode or non-secure SVC mode
>>> before calling the entry point of an EFI application.  This allows
>>> U-Boot to provide a usable PSCI implementation and makes it possible
>>> to boot kernels into hypervisor mode using an EFI bootloader.
>>>
>>> Based on diffs from Heinrich Schuchardt and Alexander Graf.
>>>
>>> Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
>>
>> bootefi hello fails on vexpress_ca15_tc2_defconfig when run on qemu with
>>
>>         QEMU_AUDIO_DRV=none qemu-system-arm \
>>         -M vexpress-a15 -cpu cortex-a15 -kernel u-boot \
>>         -net user -net nic,model=lan9118 \
>>         -m 1024M --nographic \
>>         -drive if=sd,file=img.vexpress,media=disk,format=raw
> 
> Works for me with:
> 
> $ qemu-system-arm --version
> QEMU emulator version 3.0.0
> Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers
> 
> => bootefi hello
> Scanning disks on mmc...
> MMC Device 1 not found
> MMC Device 2 not found
> MMC Device 3 not found
> Found 3 disks
> WARNING: booting without device tree
> ## Starting EFI application at a0008000 ...
> WARNING: using memory device/image path, this may confuse some payloads!
> Hello, world!
> Running on UEFI 2.7
> Have SMBIOS table
> Load options: <none>
> ## Application terminated, r = 0
> 
> That is with CONFIG_CMD_BOOTEFI_HELLO=y added to the
> vexpress_ca15_tc2_defconfig of course.
> 
>> Bisection points to
>> efi_loader: ARM: run EFI payloads non-secure
>> commit dc500c369486fbe04000fd325c46bb309e4a1827
> 
> That suggests an issue with emulation if the mode switching
> instructions or HYP support in qemu.  Or a toolchain issue of course.

Or maybe Heinrich's QEMU version starts up in a different EL mode?


Alex
Heinrich Schuchardt Sept. 22, 2018, 11:30 p.m. | #9
On 09/01/2018 12:21 PM, Alexander Graf wrote:
> 
> 
> On 31.08.18 20:45, Mark Kettenis wrote:
>>> From: Heinrich Schuchardt <xypron.glpk@gmx.de>
>>> Date: Fri, 31 Aug 2018 19:37:25 +0200
>>>
>>> On 06/14/2018 12:41 AM, Mark Kettenis wrote:
>>>> If desired (and possible) switch into HYP mode or non-secure SVC mode
>>>> before calling the entry point of an EFI application.  This allows
>>>> U-Boot to provide a usable PSCI implementation and makes it possible
>>>> to boot kernels into hypervisor mode using an EFI bootloader.
>>>>
>>>> Based on diffs from Heinrich Schuchardt and Alexander Graf.
>>>>
>>>> Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
>>>
>>> bootefi hello fails on vexpress_ca15_tc2_defconfig when run on qemu with
>>>
>>>         QEMU_AUDIO_DRV=none qemu-system-arm \
>>>         -M vexpress-a15 -cpu cortex-a15 -kernel u-boot \
>>>         -net user -net nic,model=lan9118 \
>>>         -m 1024M --nographic \
>>>         -drive if=sd,file=img.vexpress,media=disk,format=raw
>>
>> Works for me with:
>>
>> $ qemu-system-arm --version
>> QEMU emulator version 3.0.0
>> Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers
>>
>> => bootefi hello
>> Scanning disks on mmc...
>> MMC Device 1 not found
>> MMC Device 2 not found
>> MMC Device 3 not found
>> Found 3 disks
>> WARNING: booting without device tree
>> ## Starting EFI application at a0008000 ...
>> WARNING: using memory device/image path, this may confuse some payloads!
>> Hello, world!
>> Running on UEFI 2.7
>> Have SMBIOS table
>> Load options: <none>
>> ## Application terminated, r = 0
>>
>> That is with CONFIG_CMD_BOOTEFI_HELLO=y added to the
>> vexpress_ca15_tc2_defconfig of course.
>>
>>> Bisection points to
>>> efi_loader: ARM: run EFI payloads non-secure
>>> commit dc500c369486fbe04000fd325c46bb309e4a1827
>>
>> That suggests an issue with emulation if the mode switching
>> instructions or HYP support in qemu.  Or a toolchain issue of course.
> 
> Or maybe Heinrich's QEMU version starts up in a different EL mode?
> 
> 
> Alex
> _______________________________________________
> U-Boot mailing list
> U-Boot@lists.denx.de
> https://lists.denx.de/listinfo/u-boot
> 
The problem is reproducible with qemu 3.0.0.

git reset --hard dc500c369486fbe04000fd325c46bb309e4a1827

--- a/lib/efi_loader/Kconfig
+++ b/lib/efi_loader/Kconfig
@@ -2,7 +2,6 @@ config EFI_LOADER
        bool "Support running EFI Applications in U-Boot"
        depends on (ARM || X86 || RISCV) && OF_LIBFDT
        # We do not support bootefi booting ARMv7 in non-secure mode
-       depends on !ARMV7_NONSEC
        # We need EFI_STUB_64BIT to be set on x86_64 with EFI_STUB
        depends on !EFI_STUB || !X86_64 || EFI_STUB_64BIT
        # We need EFI_STUB_32BIT to be set on x86_32 with EFI_STUB

make vexpress_ca15_tc2_defconfig

Set CONFIG_CMD_BOOTEFI_HELLO=y

export CROSS_COMPILE=arm-linux-gnueabihf-
make

QEMU_AUDIO_DRV=none qemu-system-arm \
-M vexpress-a15 -cpu cortex-a15 -kernel u-boot \
-netdev \
user,id=net0,tftp=tftp,net=192.168.76.0/24,dhcpstart=192.168.76.9 \
-net nic,model=lan9118,netdev=net0 \
-m 1024M --nographic \
-drive if=sd,file=img.vexpress,media=disk,format=raw

=> bootefi hello
Scanning disks on mmc...
MMC Device 1 not found
MMC Device 2 not found
MMC Device 3 not found
Found 2 disks
WARNING: booting without device tree
## Starting EFI application at a0008000 ...
WARNING: using memory device/image path, this may confuse some payloads!

No further output after this line :(

Best regards

Heinrich
Jonathan Gray Sept. 23, 2018, 2:39 a.m. | #10
On Sun, Sep 23, 2018 at 01:30:11AM +0200, Heinrich Schuchardt wrote:
> On 09/01/2018 12:21 PM, Alexander Graf wrote:
> > 
> > 
> > On 31.08.18 20:45, Mark Kettenis wrote:
> >>> From: Heinrich Schuchardt <xypron.glpk@gmx.de>
> >>> Date: Fri, 31 Aug 2018 19:37:25 +0200
> >>>
> >>> On 06/14/2018 12:41 AM, Mark Kettenis wrote:
> >>>> If desired (and possible) switch into HYP mode or non-secure SVC mode
> >>>> before calling the entry point of an EFI application.  This allows
> >>>> U-Boot to provide a usable PSCI implementation and makes it possible
> >>>> to boot kernels into hypervisor mode using an EFI bootloader.
> >>>>
> >>>> Based on diffs from Heinrich Schuchardt and Alexander Graf.
> >>>>
> >>>> Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> >>>
> >>> bootefi hello fails on vexpress_ca15_tc2_defconfig when run on qemu with
> >>>
> >>>         QEMU_AUDIO_DRV=none qemu-system-arm \
> >>>         -M vexpress-a15 -cpu cortex-a15 -kernel u-boot \
> >>>         -net user -net nic,model=lan9118 \
> >>>         -m 1024M --nographic \
> >>>         -drive if=sd,file=img.vexpress,media=disk,format=raw
> >>
> >> Works for me with:
> >>
> >> $ qemu-system-arm --version
> >> QEMU emulator version 3.0.0
> >> Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers
> >>
> >> => bootefi hello
> >> Scanning disks on mmc...
> >> MMC Device 1 not found
> >> MMC Device 2 not found
> >> MMC Device 3 not found
> >> Found 3 disks
> >> WARNING: booting without device tree
> >> ## Starting EFI application at a0008000 ...
> >> WARNING: using memory device/image path, this may confuse some payloads!
> >> Hello, world!
> >> Running on UEFI 2.7
> >> Have SMBIOS table
> >> Load options: <none>
> >> ## Application terminated, r = 0
> >>
> >> That is with CONFIG_CMD_BOOTEFI_HELLO=y added to the
> >> vexpress_ca15_tc2_defconfig of course.
> >>
> >>> Bisection points to
> >>> efi_loader: ARM: run EFI payloads non-secure
> >>> commit dc500c369486fbe04000fd325c46bb309e4a1827
> >>
> >> That suggests an issue with emulation if the mode switching
> >> instructions or HYP support in qemu.  Or a toolchain issue of course.
> > 
> > Or maybe Heinrich's QEMU version starts up in a different EL mode?
> > 
> > 
> > Alex
> > _______________________________________________
> > U-Boot mailing list
> > U-Boot@lists.denx.de
> > https://lists.denx.de/listinfo/u-boot
> > 
> The problem is reproducible with qemu 3.0.0.
> 
> git reset --hard dc500c369486fbe04000fd325c46bb309e4a1827
> 
> --- a/lib/efi_loader/Kconfig
> +++ b/lib/efi_loader/Kconfig
> @@ -2,7 +2,6 @@ config EFI_LOADER
>         bool "Support running EFI Applications in U-Boot"
>         depends on (ARM || X86 || RISCV) && OF_LIBFDT
>         # We do not support bootefi booting ARMv7 in non-secure mode
> -       depends on !ARMV7_NONSEC
>         # We need EFI_STUB_64BIT to be set on x86_64 with EFI_STUB
>         depends on !EFI_STUB || !X86_64 || EFI_STUB_64BIT
>         # We need EFI_STUB_32BIT to be set on x86_32 with EFI_STUB
> 
> make vexpress_ca15_tc2_defconfig
> 
> Set CONFIG_CMD_BOOTEFI_HELLO=y
> 
> export CROSS_COMPILE=arm-linux-gnueabihf-
> make
> 
> QEMU_AUDIO_DRV=none qemu-system-arm \
> -M vexpress-a15 -cpu cortex-a15 -kernel u-boot \
> -netdev \
> user,id=net0,tftp=tftp,net=192.168.76.0/24,dhcpstart=192.168.76.9 \
> -net nic,model=lan9118,netdev=net0 \
> -m 1024M --nographic \
> -drive if=sd,file=img.vexpress,media=disk,format=raw
> 
> => bootefi hello
> Scanning disks on mmc...
> MMC Device 1 not found
> MMC Device 2 not found
> MMC Device 3 not found
> Found 2 disks
> WARNING: booting without device tree
> ## Starting EFI application at a0008000 ...
> WARNING: using memory device/image path, this may confuse some payloads!
> 
> No further output after this line :(

bootefi hello works fine with 2018.09 and qemu 3.0 here

gmake PYTHON=python2.7 CROSS_COMPILE=arm-none-eabi- vexpress_ca15_tc2_defconfig O=build/vexpress
gmake PYTHON=python2.7 CROSS_COMPILE=arm-none-eabi- O=build/vexpress menuconfig
gmake PYTHON=python2.7 CROSS_COMPILE=arm-none-eabi- O=build/vexpress all

$ arm-none-eabi-gcc -v
Using built-in specs.
COLLECT_GCC=arm-none-eabi-gcc
COLLECT_LTO_WRAPPER=/usr/local/libexec/gcc/arm-none-eabi/6.3.1/lto-wrapper
Target: arm-none-eabi
Configured with: /usr/obj/ports/arm-none-eabi-gcc-linaro-6.3.2017.02-arm/gcc-linaro-6.3-2017.02/configure --enable-languages=c,c++ --enable-multilib --enable-interwork --with-gmp=/usr/local --with-newlib --disable-lto --enable-cpp --target=arm-none-eabi --disable-shared --disable-nls --disable-werror --prefix=/usr/local --sysconfdir=/etc --mandir=/usr/local/man --infodir=/usr/local/info --localstatedir=/var --disable-silent-rules --disable-gtk-doc
Thread model: single
gcc version 6.3.1 20170109 (Linaro GCC 6.3-2017.02)

$ qemu-system-arm --version
QEMU emulator version 3.0.0
Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers

qemu-system-arm -m 1024 -M vexpress-a15 -kernel build/vexpress/u-boot -serial stdio

U-Boot 2018.09-00001-g2a9fbd55c3 (Sep 23 2018 - 12:24:33 +1000)

DRAM:  1 GiB
WARNING: Caches not enabled
Flash: 128 MiB
MMC:   MMC: 0
*** Warning - bad CRC, using default environment

In:    serial
Out:   serial
Err:   serial
Net:   smc911x-0
Hit any key to stop autoboot:  0
=> bootefi hello
Scanning disks on mmc...
Card did not respond to voltage select!
MMC Device 1 not found
MMC Device 2 not found
MMC Device 3 not found
Found 0 disks
WARNING: booting without device tree
## Starting EFI application at a0008000 ...
WARNING: using memory device/image path, this may confuse some payloads!
Hello, world!
Running on UEFI 2.7
Have SMBIOS table
Load options: <none>
## Application terminated, r = 0
=>
Heinrich Schuchardt Sept. 23, 2018, 3:38 a.m. | #11
On 09/23/2018 04:39 AM, Jonathan Gray wrote:
> On Sun, Sep 23, 2018 at 01:30:11AM +0200, Heinrich Schuchardt wrote:
>> On 09/01/2018 12:21 PM, Alexander Graf wrote:
>>>
>>>
>>> On 31.08.18 20:45, Mark Kettenis wrote:
>>>>> From: Heinrich Schuchardt <xypron.glpk@gmx.de>
>>>>> Date: Fri, 31 Aug 2018 19:37:25 +0200
>>>>>
>>>>> On 06/14/2018 12:41 AM, Mark Kettenis wrote:
>>>>>> If desired (and possible) switch into HYP mode or non-secure SVC mode
>>>>>> before calling the entry point of an EFI application.  This allows
>>>>>> U-Boot to provide a usable PSCI implementation and makes it possible
>>>>>> to boot kernels into hypervisor mode using an EFI bootloader.
>>>>>>
>>>>>> Based on diffs from Heinrich Schuchardt and Alexander Graf.
>>>>>>
>>>>>> Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
>>>>>
>>>>> bootefi hello fails on vexpress_ca15_tc2_defconfig when run on qemu with
>>>>>
>>>>>         QEMU_AUDIO_DRV=none qemu-system-arm \
>>>>>         -M vexpress-a15 -cpu cortex-a15 -kernel u-boot \
>>>>>         -net user -net nic,model=lan9118 \
>>>>>         -m 1024M --nographic \
>>>>>         -drive if=sd,file=img.vexpress,media=disk,format=raw
>>>>
>>>> Works for me with:
>>>>
>>>> $ qemu-system-arm --version
>>>> QEMU emulator version 3.0.0
>>>> Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers
>>>>
>>>> => bootefi hello
>>>> Scanning disks on mmc...
>>>> MMC Device 1 not found
>>>> MMC Device 2 not found
>>>> MMC Device 3 not found
>>>> Found 3 disks
>>>> WARNING: booting without device tree
>>>> ## Starting EFI application at a0008000 ...
>>>> WARNING: using memory device/image path, this may confuse some payloads!
>>>> Hello, world!
>>>> Running on UEFI 2.7
>>>> Have SMBIOS table
>>>> Load options: <none>
>>>> ## Application terminated, r = 0
>>>>
>>>> That is with CONFIG_CMD_BOOTEFI_HELLO=y added to the
>>>> vexpress_ca15_tc2_defconfig of course.
>>>>
>>>>> Bisection points to
>>>>> efi_loader: ARM: run EFI payloads non-secure
>>>>> commit dc500c369486fbe04000fd325c46bb309e4a1827
>>>>
>>>> That suggests an issue with emulation if the mode switching
>>>> instructions or HYP support in qemu.  Or a toolchain issue of course.
>>>
>>> Or maybe Heinrich's QEMU version starts up in a different EL mode?
>>>
>>>
>>> Alex
>>> _______________________________________________
>>> U-Boot mailing list
>>> U-Boot@lists.denx.de
>>> https://lists.denx.de/listinfo/u-boot
>>>
>> The problem is reproducible with qemu 3.0.0.
>>
>> git reset --hard dc500c369486fbe04000fd325c46bb309e4a1827
>>
>> --- a/lib/efi_loader/Kconfig
>> +++ b/lib/efi_loader/Kconfig
>> @@ -2,7 +2,6 @@ config EFI_LOADER
>>         bool "Support running EFI Applications in U-Boot"
>>         depends on (ARM || X86 || RISCV) && OF_LIBFDT
>>         # We do not support bootefi booting ARMv7 in non-secure mode
>> -       depends on !ARMV7_NONSEC
>>         # We need EFI_STUB_64BIT to be set on x86_64 with EFI_STUB
>>         depends on !EFI_STUB || !X86_64 || EFI_STUB_64BIT
>>         # We need EFI_STUB_32BIT to be set on x86_32 with EFI_STUB
>>
>> make vexpress_ca15_tc2_defconfig
>>
>> Set CONFIG_CMD_BOOTEFI_HELLO=y
>>
>> export CROSS_COMPILE=arm-linux-gnueabihf-
>> make
>>
>> QEMU_AUDIO_DRV=none qemu-system-arm \
>> -M vexpress-a15 -cpu cortex-a15 -kernel u-boot \
>> -netdev \
>> user,id=net0,tftp=tftp,net=192.168.76.0/24,dhcpstart=192.168.76.9 \
>> -net nic,model=lan9118,netdev=net0 \
>> -m 1024M --nographic \
>> -drive if=sd,file=img.vexpress,media=disk,format=raw
>>
>> => bootefi hello
>> Scanning disks on mmc...
>> MMC Device 1 not found
>> MMC Device 2 not found
>> MMC Device 3 not found
>> Found 2 disks
>> WARNING: booting without device tree
>> ## Starting EFI application at a0008000 ...
>> WARNING: using memory device/image path, this may confuse some payloads!
>>
>> No further output after this line :(
> 
> bootefi hello works fine with 2018.09 and qemu 3.0 here
> 
> gmake PYTHON=python2.7 CROSS_COMPILE=arm-none-eabi- vexpress_ca15_tc2_defconfig O=build/vexpress
> gmake PYTHON=python2.7 CROSS_COMPILE=arm-none-eabi- O=build/vexpress menuconfig
> gmake PYTHON=python2.7 CROSS_COMPILE=arm-none-eabi- O=build/vexpress all
> 
> $ arm-none-eabi-gcc -v
> Using built-in specs.
> COLLECT_GCC=arm-none-eabi-gcc
> COLLECT_LTO_WRAPPER=/usr/local/libexec/gcc/arm-none-eabi/6.3.1/lto-wrapper
> Target: arm-none-eabi
> Configured with: /usr/obj/ports/arm-none-eabi-gcc-linaro-6.3.2017.02-arm/gcc-linaro-6.3-2017.02/configure --enable-languages=c,c++ --enable-multilib --enable-interwork --with-gmp=/usr/local --with-newlib --disable-lto --enable-cpp --target=arm-none-eabi --disable-shared --disable-nls --disable-werror --prefix=/usr/local --sysconfdir=/etc --mandir=/usr/local/man --infodir=/usr/local/info --localstatedir=/var --disable-silent-rules --disable-gtk-doc
> Thread model: single
> gcc version 6.3.1 20170109 (Linaro GCC 6.3-2017.02)
> 
> $ qemu-system-arm --version
> QEMU emulator version 3.0.0
> Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers
> 
> qemu-system-arm -m 1024 -M vexpress-a15 -kernel build/vexpress/u-boot -serial stdio
> 
> U-Boot 2018.09-00001-g2a9fbd55c3 (Sep 23 2018 - 12:24:33 +1000)
> 
> DRAM:  1 GiB
> WARNING: Caches not enabled
> Flash: 128 MiB
> MMC:   MMC: 0
> *** Warning - bad CRC, using default environment
> 
> In:    serial
> Out:   serial
> Err:   serial
> Net:   smc911x-0
> Hit any key to stop autoboot:  0
> => bootefi hellopi
> Scanning disks on mmc...
> Card did not respond to voltage select!
> MMC Device 1 not found
> MMC Device 2 not found
> MMC Device 3 not found
> Found 0 disks
> WARNING: booting without device tree
> ## Starting EFI application at a0008000 ...
> WARNING: using memory device/image path, this may confuse some payloads!
> Hello, world!
> Running on UEFI 2.7
> Have SMBIOS table
> Load options: <none>
> ## Application terminated, r = 0
> => 
> 
The difference seems to be in the compiler:
With the recipe above arm-none-eabi-gcc 6.3.1 is working but
arm-linux-gnueabihf-gcc 8.2.0 (as supplied with Debian Buster) is still
failing.

Errors like out of bound accesses may become apparent or disappearing
depending of generated addresses. So it may be by chance that we do not
see the error with gcc 6.3.1. This does not imply that the code is correct.

Best regards

Heinrich
Mark Kettenis Sept. 23, 2018, 6:22 p.m. | #12
> From: Heinrich Schuchardt <xypron.glpk@gmx.de>
> Date: Sun, 23 Sep 2018 05:38:29 +0200
> 
> On 09/23/2018 04:39 AM, Jonathan Gray wrote:
> > On Sun, Sep 23, 2018 at 01:30:11AM +0200, Heinrich Schuchardt wrote:
> >> On 09/01/2018 12:21 PM, Alexander Graf wrote:
> >>>
> >>>
> >>> On 31.08.18 20:45, Mark Kettenis wrote:
> >>>>> From: Heinrich Schuchardt <xypron.glpk@gmx.de>
> >>>>> Date: Fri, 31 Aug 2018 19:37:25 +0200
> >>>>>
> >>>>> On 06/14/2018 12:41 AM, Mark Kettenis wrote:
> >>>>>> If desired (and possible) switch into HYP mode or non-secure SVC mode
> >>>>>> before calling the entry point of an EFI application.  This allows
> >>>>>> U-Boot to provide a usable PSCI implementation and makes it possible
> >>>>>> to boot kernels into hypervisor mode using an EFI bootloader.
> >>>>>>
> >>>>>> Based on diffs from Heinrich Schuchardt and Alexander Graf.
> >>>>>>
> >>>>>> Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> >>>>>
> >>>>> bootefi hello fails on vexpress_ca15_tc2_defconfig when run on qemu with
> >>>>>
> >>>>>         QEMU_AUDIO_DRV=none qemu-system-arm \
> >>>>>         -M vexpress-a15 -cpu cortex-a15 -kernel u-boot \
> >>>>>         -net user -net nic,model=lan9118 \
> >>>>>         -m 1024M --nographic \
> >>>>>         -drive if=sd,file=img.vexpress,media=disk,format=raw
> >>>>
> >>>> Works for me with:
> >>>>
> >>>> $ qemu-system-arm --version
> >>>> QEMU emulator version 3.0.0
> >>>> Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers
> >>>>
> >>>> => bootefi hello
> >>>> Scanning disks on mmc...
> >>>> MMC Device 1 not found
> >>>> MMC Device 2 not found
> >>>> MMC Device 3 not found
> >>>> Found 3 disks
> >>>> WARNING: booting without device tree
> >>>> ## Starting EFI application at a0008000 ...
> >>>> WARNING: using memory device/image path, this may confuse some payloads!
> >>>> Hello, world!
> >>>> Running on UEFI 2.7
> >>>> Have SMBIOS table
> >>>> Load options: <none>
> >>>> ## Application terminated, r = 0
> >>>>
> >>>> That is with CONFIG_CMD_BOOTEFI_HELLO=y added to the
> >>>> vexpress_ca15_tc2_defconfig of course.
> >>>>
> >>>>> Bisection points to
> >>>>> efi_loader: ARM: run EFI payloads non-secure
> >>>>> commit dc500c369486fbe04000fd325c46bb309e4a1827
> >>>>
> >>>> That suggests an issue with emulation if the mode switching
> >>>> instructions or HYP support in qemu.  Or a toolchain issue of course.
> >>>
> >>> Or maybe Heinrich's QEMU version starts up in a different EL mode?
> >>>
> >>>
> >>> Alex
> >>> _______________________________________________
> >>> U-Boot mailing list
> >>> U-Boot@lists.denx.de
> >>> https://lists.denx.de/listinfo/u-boot
> >>>
> >> The problem is reproducible with qemu 3.0.0.
> >>
> >> git reset --hard dc500c369486fbe04000fd325c46bb309e4a1827
> >>
> >> --- a/lib/efi_loader/Kconfig
> >> +++ b/lib/efi_loader/Kconfig
> >> @@ -2,7 +2,6 @@ config EFI_LOADER
> >>         bool "Support running EFI Applications in U-Boot"
> >>         depends on (ARM || X86 || RISCV) && OF_LIBFDT
> >>         # We do not support bootefi booting ARMv7 in non-secure mode
> >> -       depends on !ARMV7_NONSEC
> >>         # We need EFI_STUB_64BIT to be set on x86_64 with EFI_STUB
> >>         depends on !EFI_STUB || !X86_64 || EFI_STUB_64BIT
> >>         # We need EFI_STUB_32BIT to be set on x86_32 with EFI_STUB
> >>
> >> make vexpress_ca15_tc2_defconfig
> >>
> >> Set CONFIG_CMD_BOOTEFI_HELLO=y
> >>
> >> export CROSS_COMPILE=arm-linux-gnueabihf-
> >> make
> >>
> >> QEMU_AUDIO_DRV=none qemu-system-arm \
> >> -M vexpress-a15 -cpu cortex-a15 -kernel u-boot \
> >> -netdev \
> >> user,id=net0,tftp=tftp,net=192.168.76.0/24,dhcpstart=192.168.76.9 \
> >> -net nic,model=lan9118,netdev=net0 \
> >> -m 1024M --nographic \
> >> -drive if=sd,file=img.vexpress,media=disk,format=raw
> >>
> >> => bootefi hello
> >> Scanning disks on mmc...
> >> MMC Device 1 not found
> >> MMC Device 2 not found
> >> MMC Device 3 not found
> >> Found 2 disks
> >> WARNING: booting without device tree
> >> ## Starting EFI application at a0008000 ...
> >> WARNING: using memory device/image path, this may confuse some payloads!
> >>
> >> No further output after this line :(
> > 
> > bootefi hello works fine with 2018.09 and qemu 3.0 here
> > 
> > gmake PYTHON=python2.7 CROSS_COMPILE=arm-none-eabi- vexpress_ca15_tc2_defconfig O=build/vexpress
> > gmake PYTHON=python2.7 CROSS_COMPILE=arm-none-eabi- O=build/vexpress menuconfig
> > gmake PYTHON=python2.7 CROSS_COMPILE=arm-none-eabi- O=build/vexpress all
> > 
> > $ arm-none-eabi-gcc -v
> > Using built-in specs.
> > COLLECT_GCC=arm-none-eabi-gcc
> > COLLECT_LTO_WRAPPER=/usr/local/libexec/gcc/arm-none-eabi/6.3.1/lto-wrapper
> > Target: arm-none-eabi
> > Configured with: /usr/obj/ports/arm-none-eabi-gcc-linaro-6.3.2017.02-arm/gcc-linaro-6.3-2017.02/configure --enable-languages=c,c++ --enable-multilib --enable-interwork --with-gmp=/usr/local --with-newlib --disable-lto --enable-cpp --target=arm-none-eabi --disable-shared --disable-nls --disable-werror --prefix=/usr/local --sysconfdir=/etc --mandir=/usr/local/man --infodir=/usr/local/info --localstatedir=/var --disable-silent-rules --disable-gtk-doc
> > Thread model: single
> > gcc version 6.3.1 20170109 (Linaro GCC 6.3-2017.02)
> > 
> > $ qemu-system-arm --version
> > QEMU emulator version 3.0.0
> > Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers
> > 
> > qemu-system-arm -m 1024 -M vexpress-a15 -kernel build/vexpress/u-boot -serial stdio
> > 
> > U-Boot 2018.09-00001-g2a9fbd55c3 (Sep 23 2018 - 12:24:33 +1000)
> > 
> > DRAM:  1 GiB
> > WARNING: Caches not enabled
> > Flash: 128 MiB
> > MMC:   MMC: 0
> > *** Warning - bad CRC, using default environment
> > 
> > In:    serial
> > Out:   serial
> > Err:   serial
> > Net:   smc911x-0
> > Hit any key to stop autoboot:  0
> > => bootefi hellopi
> > Scanning disks on mmc...
> > Card did not respond to voltage select!
> > MMC Device 1 not found
> > MMC Device 2 not found
> > MMC Device 3 not found
> > Found 0 disks
> > WARNING: booting without device tree
> > ## Starting EFI application at a0008000 ...
> > WARNING: using memory device/image path, this may confuse some payloads!
> > Hello, world!
> > Running on UEFI 2.7
> > Have SMBIOS table
> > Load options: <none>
> > ## Application terminated, r = 0
> > => 
> > 
> The difference seems to be in the compiler:
> With the recipe above arm-none-eabi-gcc 6.3.1 is working but
> arm-linux-gnueabihf-gcc 8.2.0 (as supplied with Debian Buster) is still
> failing.

Are you sure it's the compiler that causes the breakage and not the
linker?  Apparently there was a recent change in binutils that affects
sectipn alignment.  See commit
792b204798453d96b00e0817e8472c19455e92a2 for a workaround.  It
wouldn't surprise me more tweaks like that are necessary.

> Errors like out of bound accesses may become apparent or disappearing
> depending of generated addresses. So it may be by chance that we do not
> see the error with gcc 6.3.1. This does not imply that the code is correct.

Everything is possible.  But there is very little C code involved
here.  It's mostly scary assembly code...

Cheers,

Mark

Patch

diff --git a/cmd/bootefi.c b/cmd/bootefi.c
index 707d159bac..12a6b84ce6 100644
--- a/cmd/bootefi.c
+++ b/cmd/bootefi.c
@@ -20,6 +20,11 @@ 
 #include <asm-generic/unaligned.h>
 #include <linux/linkage.h>
 
+#ifdef CONFIG_ARMV7_NONSEC
+#include <asm/armv7.h>
+#include <asm/secure.h>
+#endif
+
 DECLARE_GLOBAL_DATA_PTR;
 
 #define OBJ_LIST_NOT_INITIALIZED 1
@@ -189,6 +194,18 @@  static efi_status_t efi_run_in_el2(EFIAPI efi_status_t (*entry)(
 }
 #endif
 
+#ifdef CONFIG_ARMV7_NONSEC
+static efi_status_t efi_run_in_hyp(EFIAPI efi_status_t (*entry)(
+			efi_handle_t image_handle, struct efi_system_table *st),
+			efi_handle_t image_handle, struct efi_system_table *st)
+{
+	/* Enable caches again */
+	dcache_enable();
+
+	return efi_do_enter(image_handle, st, entry);
+}
+#endif
+
 /* Carve out DT reserved memory ranges */
 static efi_status_t efi_carve_out_dt_rsv(void *fdt)
 {
@@ -338,6 +355,21 @@  static efi_status_t do_bootefi_exec(void *efi,
 	}
 #endif
 
+#ifdef CONFIG_ARMV7_NONSEC
+	if (armv7_boot_nonsec()) {
+		dcache_disable();	/* flush cache before switch to HYP */
+
+		armv7_init_nonsec();
+		secure_ram_addr(_do_nonsec_entry)(efi_run_in_hyp,
+				(uintptr_t)entry,
+				(uintptr_t)loaded_image_info_obj.handle,
+				(uintptr_t)&systab);
+
+		/* Should never reach here, efi exits with longjmp */
+		while (1) { }
+	}
+#endif
+
 	ret = efi_do_enter(loaded_image_info_obj.handle, &systab, entry);
 
 exit: