mbox series

[00/14] tests: acpi: add UEFI (ARM) testing support

Message ID 1547566866-129386-1-git-send-email-imammedo@redhat.com
Headers show
Series tests: acpi: add UEFI (ARM) testing support | expand

Message

Igor Mammedov Jan. 15, 2019, 3:40 p.m. UTC
Series adds support for ACPI tables located above 4G. It only adds 64-bit
handling necessary for testing arm/virt board (i.e. might be not complete
wrt spec) and a test build of UEFI (AVMF) firmware that provides an entry
point[1] for fetching ACPI tables (RSDP pointer).

Series depends on:
   [PATCH v2 0/8] tests: apci: consolidate and cleanup  ACPI test code
   it's queued on PCI tree git://git.kernel.org/pub/scm/virt/kvm/mst/qemu.git pci
and requires following EDK2 patches to enable testing feature:
   1) https://github.com/lersek/edk2/commits/acpi_test_support

Git tree for testing:
   https://github.com/imammedo/qemu.git acpi_arm_tests_v1

Igor Mammedov (14):
  tests: acpi: add uefi_find_rsdp_addr() helper
  tests: acpi: make RSDT test routine handle XSDT
  tests: acpi: rename acpi_parse_rsdp_table() into
    acpi_fetch_rsdp_table()
  tests: acpi: make pointer to RSDP 64bit
  tests: acpi: fetch X_DSDT if pointer to DSDT is 0
  tests: acpi: add reference blobs arm/virt board testcase
  tests: acpi: skip FACS table if board uses hw reduced ACPI profile
  tests: acpi: introduce an abilty start tests with UEFI firmware
  tests: acpi: move boot_sector_init() into x86 tests branch
  tests: acpi: ignore SMBIOS tests when UEFI firmware is used
  tests: acpi: add AVMF firmware blobs
  tests: acpi: prepare AVMF firmware blobs to be used by
    bios-tables-test
  tests: acpi: add simple arm/virt testcase
  tests: acpi: refactor rebuild-expected-aml.sh to dump ACPI tables for
    a specified list of targets

 tests/acpi-utils.h                      |   4 +-
 Makefile                                |   3 +-
 pc-bios/avmf.img                        | Bin 0 -> 2097152 bytes
 pc-bios/avmf_vars.img                   | Bin 0 -> 786432 bytes
 tests/Makefile.include                  |  19 +++++-
 tests/acpi-utils.c                      |  57 ++++++++++++----
 tests/bios-tables-test.c                | 113 +++++++++++++++++++++++---------
 tests/data/acpi/rebuild-expected-aml.sh |  23 ++++---
 tests/data/acpi/virt/APIC               | Bin 0 -> 168 bytes
 tests/data/acpi/virt/DSDT               | Bin 0 -> 18476 bytes
 tests/data/acpi/virt/FACP               | Bin 0 -> 268 bytes
 tests/data/acpi/virt/GTDT               | Bin 0 -> 96 bytes
 tests/data/acpi/virt/MCFG               | Bin 0 -> 60 bytes
 tests/data/acpi/virt/SPCR               | Bin 0 -> 80 bytes
 tests/vmgenid-test.c                    |   2 +-
 15 files changed, 160 insertions(+), 61 deletions(-)
 create mode 100644 pc-bios/avmf.img
 create mode 100644 pc-bios/avmf_vars.img
 create mode 100644 tests/data/acpi/virt/APIC
 create mode 100644 tests/data/acpi/virt/DSDT
 create mode 100644 tests/data/acpi/virt/FACP
 create mode 100644 tests/data/acpi/virt/GTDT
 create mode 100644 tests/data/acpi/virt/MCFG
 create mode 100644 tests/data/acpi/virt/SPCR

Comments

Igor Mammedov Jan. 15, 2019, 3:40 p.m. UTC | #1
adds test blobs for default arm/virt machine testcase

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
 tests/data/acpi/virt/APIC | Bin 0 -> 168 bytes
 tests/data/acpi/virt/DSDT | Bin 0 -> 18476 bytes
 tests/data/acpi/virt/FACP | Bin 0 -> 268 bytes
 tests/data/acpi/virt/GTDT | Bin 0 -> 96 bytes
 tests/data/acpi/virt/MCFG | Bin 0 -> 60 bytes
 tests/data/acpi/virt/SPCR | Bin 0 -> 80 bytes
 6 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 tests/data/acpi/virt/APIC
 create mode 100644 tests/data/acpi/virt/DSDT
 create mode 100644 tests/data/acpi/virt/FACP
 create mode 100644 tests/data/acpi/virt/GTDT
 create mode 100644 tests/data/acpi/virt/MCFG
 create mode 100644 tests/data/acpi/virt/SPCR

diff --git a/tests/data/acpi/virt/APIC b/tests/data/acpi/virt/APIC
new file mode 100644
index 0000000000000000000000000000000000000000..797dfde2841c51b7e72065602e99ce1714347f0d
GIT binary patch
literal 168
zcmZ<^@N{0mz`($~*~#D8BUr&HBEZ=ZD8>jB1F=Cg4Dd+6SPUF6788)c?E~X6Fu>G{
hBZPn~MyPrgD9sGlkD?67;f3451Xcqw&w(L;0RYV=2>}2A

literal 0
HcmV?d00001

diff --git a/tests/data/acpi/virt/DSDT b/tests/data/acpi/virt/DSDT
new file mode 100644
index 0000000000000000000000000000000000000000..20e85c7f89f645c69935c615c07084e221419960
GIT binary patch
literal 18476
zcmc)ScU)EVAII^7km3at6myGa+R|dUS|(gjDG>rqi;BuJN5jfYEla7?tSq%Xt!(eT
zr!AYd_uhN&!S8+U(D&=t`StIw9$oH>d%mB0pK#D~&LeJRL*=*uql2K;?26j>=!V`E
z6YJuY`dmg31mXSgWB#J~S-UqiR5Ud<cZ(Mn7iTw(uCB~0kJnWzh6dS9<Etx!#^$Q5
zcx_Gk!TOrf#l<BhsRy&0;`I#$-C~^=whh9GZG-$EIH7frk<mvrM_ZLw*5`%~G&Yxv
z9Mh1RGG=Ujt)>jdl!92h)D&$WWX;hthf7M5uZl}Dl25#TNmhEvu#pquBa=&ZuBsU?
zNU5HsVO)7EM{DBc|GlzR+b&ufK3RFzF7@fJLGsy(?FFt|xgHw}TBWeXJ_0W|JtBPC
ze~f4qtRGR58c`9xic&YHN5oo1&B(GDr9Pu9az<v<jMg@z%x4UWoRJkZBim*S@)?68
zXKWBPBW5!O`-~xxGg?Q@$h8?me8$ko8Ev9ww6hsQea5iJ8QD=Y@@>X2pRsl1jJ8oT
zI@pY@ZAJ-xjMnWRv8Wk^He(x~5xJZ4ha|nLZ)h{N^%>FoX>Qc(=wdUr^BK|mX<pQf
zZZ>0kpAo&Ewu_q4!)6Tk8PWS``=}YcY{m{gBYHp0kDAfjW{mI|(fesZ)QpX6M!C<3
z-cLJ3&DhvxRQQbO{j_7$jQ%zwZZk@w_tV0t8JpUSN}mzEpLU9xvANBt@)^<l>4s4=
z2H1>|J|lWR?Hn~@OPevuXGHI(U7}_b+YJ3Rp7<Lo{JWtvdOz(NHKWvKRQrtR{j^)u
zj6pVIN1qYBpLUO$F~nx<<TIl8(;iVXhS`iUJ|lWR?HM&=8=FyMGs>d((_T?CwzC;y
zeMa<tS`;;7xXq|d%~<euW?j5G^+M{#))ki*57U85TnA*yDhm%|sz&LyqGGIWbzr4i
z9iiog>%s@e)fW`SdejB+pgPzu=p7X6ze?Sk6-*5#>0_Xck_RDm_2W7&y)ZK;$m)=j
zmDAD^jB3z`<oyiYF9|y2hM$kMQk146Q&ARl$ji!YX~_t}HQFv!;VNy|F8Nquoi<Hp
zxKi^I+v=DpxoxV#mFZ1&KomZsHchNlhAY$0l9^bUeg&C9xH7$IW^$!AO{`RgEA@&J
z&!VHl<hH3w855i804k+Sr#m-*bA!ZlrkxukSEgUQ(w(j0)FH86L3&diaJf_I&Ngti
zfwK*rN_S?%nVndf{*veb&7DejwuMs%^U1GX;!Dz&PNh3zaK_-&0i8RQ?#zKR2hJQi
zmG0CTg?&GB;nV@3JC*LtgEJ4#JUW%`YzJpMICa42PNh5B!`U9r_H-)Usgo&l=EJE2
zOm`~XSpa7NoCS0$-Pr-o4shy#)SXIqc7(GdoE_;@y0Z|@LO68*>rSORJHgor&Q5eH
z-MJy08^WmrUUw?p*%{8xaCW9s=}w($+V`^yoH{^ur_!BW;p_@$S2~sM>;`8yICVhn
zPNh4$!`U6q?sO{M*#piVaOwcuol1B1gtI4{J?T`svlpDb;M4)TJC*J%g0l$DB081s
z><wpcICX&UPNh5hz}W}RK6EPGxe=Tj!Ko7hcPic47tX$L_N7zl&W+*R7*3r)xKruQ
zesK1Kvmc#GclL*~Kb$(@aHrCpo4~mVoSV?8bmyjUZVIPPP~53>=Vowj2IppUD&4s`
zoSVa`6B>6a-MIyvTfn&mol18OfO7zxIstO0(wzh090=z?I+gC+63#8*)CrS2mG0aM
z&aL3wicY0FbzRQ>ye)=PCs^)Ox>J|qv@(6<SpsJXol1A=BAiyHk337^)Crk8mG0D~
zH?2$`-<H8yMyJx9y5MHcL2&8>&Yenk>N1--2g5m-PNh3_am}1V;M579JC*L#B{g#n
zg>xvKN_Xl)nmLETsS`wZD&48eXXe}*&aLTGx>Fa;%()GmI-zu@(w(|gX3lNl+?GzI
zJ9UA~oZG>v6Hs?5-KooB=G-36?depyQy0U`IUG)%u)0&}PF(^s=MHf0K&R53Bj6kX
z=LkBL?ktD19L{n&mG0C<E^}7ESwW}Low~GT&N!TLI+gC!1ud;ip8;3GSxKkTow|&r
zmFYi+s^F}mQ|V4!yfWuVI7iZ{bf+#^nR67Jqv%w+Qx~etIU3H<bSmAc%Twm8hO?SZ
zr8{*|%A7mGxg(uQcj{7<Id_6{Cpwkx)CDMWj)8Lwol1A=vXeP$;H;rj=}uj2GUr%0
z$I_{Er!Fy>vlh-;I+gCMgR>6KIy#l^+!@ZD;oO-{r9124tcSCnPNh2=;B0`iflj46
z$H6%c&T(`q-8ml4@o<i(Q|ZoK;M@hyUFcN0a{`<b;G96G(w!6GoCxPcI+gC+70zAZ
z+?7tHJ9mR~H#m2rQ|Zn|I2++?q*Lk6-QnCF&fV!$x^oXW_keQ`I+gC+6V5&1+>=hF
zJDcEag0qQEr91b6b1yjeqEqS4z2V#&&b{eWx^o{m_knXCI+gC61m`3;C()^N=e}_6
z3+KLcD&488-uCD1esJzbr_!C1;hYTTWIC1Z+#k;U;oP52r8}p<IR(xsbSm9B70#(}
zPNh@n&S`K?gL4|4N_S3&b2^;U=~TM&05}hT^8h-P?mQ6A1K~W7PNh2!g7Y9a5291)
z&V%7R7|w&~RJwBpoHO8@L8sE4hroFVoQKe<bmyUP9t!87bSm9>7@UW}c^I8acg}=!
zCY&?rRJ!wUI1h*Oa5|OloCW7BIA_tRbmtLp9s%bObSm9B8_wBq&Zbl8&N*<-fpZR>
zN_WnMb1s~7=~TM&NH~v#^GG_C?mP<4qu@M>PNh5N!8s4kd2}k>c{H3y!+A8FN_QRu
z=P_^|L#NW6^WmHi=X^Sq?py%p0yr1YsdQ&EoXv1H)2Vdlv2Y#>=dpAu-FY0G$H93V
zol18e59jf49#5yzoeSYy2<JjNmF`>w=OQ>4(W!LjVmKGWxtLC+J5PY~1UOHiQ|Znn
za4vy!37txJo(SiOaGpq~(w!&4c@mr_(W!Lj$#9+w=gD*`-MJLbrEo5#Q|ZoA;5-G+
zQ|MH>^Hexbh4WN8mF`>y=Q22#(W!LjayXa6xtva=J6FKD0?rk5D&2V+oTtHg8l6ga
zo(|{faGp-5(w%3(c?O(k(5ZCinQ)#7=b3aW-FX(AXTf<Eol19}4d>Z#o=vCHo#()L
z4xH!EsdVSLaGne2xpXSsc^;hS!Fe8?N_U<Q=lO7+Pp8tI7r=P|oEOlkbmxU|UI^!f
zbSmAs63&%yuB21x&Wqr@2+oV>RJ!wGI4_3tVmg)Xyadim;Jk!Rr8`%_xeCrzbSm9>
zDV&$Wc`2PrcU}hPWpG|br_!C5!+ANJm(!_q=M`{X0p}HTD&2V{oL9nmC7nulUIph>
za9%~H(w$esc{Q9@)2VdlHE>=7=QVUH-FYpX*TQ)%ol19J2j_KgUPq_Wo!7&8J)GCm
zsdVQJaNYpt4Rk8qc_W-R!g(W|N_XA_=S^_lM5of7H^X@|oHx^{bmuK_-U8<>bSm9>
zE1b8&c`Kbtcisl)ZE)U3r_!Cb!+ATLx6`R~=N)j~0p}fbD&2V}oOi-`C!I=n-Ua7f
zaNb3y(w%q1c{iMQ)2VdlJ#gLw=RI^P-FYvZ_riHEol1A!2j_io-bbg>o%h3eKb-f|
zsdVQ9a6SO%19U3g`5>GR!ucSbN_Rd4=R<HlM5of7tKnP?=W05Y?tB=|hv9sfPNh2^
zf%6eKAE8s}&PU;V6wXKKRJ!vqI3I)aF*=p*d>qcl;e4D<r90QaxdzTPbSmBX1e{O6
z`2?LxcRmT{lW;ysr_!BI!TA)NPtmD#=hJXL4d>HzD&6@EoX^1d44q1MJ`3lwa6U_?
z(w%GJTnpz~I+gBx4$kM`e2z|~JD-R1c{rb^Q|ZnZ;Cunj7wA;F^F=scg!4r@mF|2A
z&X?ePiB6?EUxxE#IA5ky>CRW+d<D)|=v2D%RXAUT^Hn;P?tBf-*Wi4OPNh3vhx2tf
zU#C;)&NtwE1I{<-RJ!v`INyZxO*)nCd<)LE;Czctr90n-^KCfarc>$8ci?;n&Uffk
zy7OH)--YvCI+gBx56<`Ce2-40JKu-%eK_BzQ|ZnR;QRp259n07^Fug4g!4l>mG1lq
z&X3^yh)$(DKZf&TI6tOS>CR8!`~=QV=v2D%Q#e0`^HVyN?)(hS&*1!wPNh3Phx2nd
zKc`da&M)Bn0?se!RJ!v^IKPDROFEVA{0h#m;QWeCr8~cd^J_T2rc>$8Z{Yj}&Tr^c
zy7OB&zlHN#I+gDH4$kl3{EkkgJHLnXdpN(RQ|Znh;QRs3ALvxN^G7&;g!4x_mG1lr
z&Y$4?iB6?Ee}?mCIDe*7>CRu^`~}Wm=v2D%S2%x#^H(~R?)(kT-{Aa>PNh44hx2zh
zf2ULF&OhM%1I|C_RJ!v|IRAw6Pdb(E{0q*%;QWhDr91zI^KUr+rc>$87C2krY@t(`
zbT&3uXX$^8vEMh17mrN-KB;c&^rjx|VmXO7^5`2R-^e3;qYr+ruys>IeM3fSRO<I%
z!(UeYU!yjT7?u1SN2PvU``<?Oix`#s97m;oYy00u^^X{p{9I~OVSZD*qC8mDP8;Tr
z&`n`&`|y2Fg6#T=@goaHw~5VMoENmp)gwWmZ$=PgEb1Htxf$VI{gdC)^7ruM-Igu&
zNJegEvb1$#^A<gt5iHrl)+9EVuiKXJpY-ObkKyO%1grjU&z#*bzOF9Fj*qJ!6BLeZ
z+f>^S&ss7)h*wT1Svj`NiYWyhWBH9WZ<PH~)MLaM6K0k_u>C8OmrRdkX@gRI%+|-U
z8DWHT!aHT*s2N9wx3Qi<_+e#-<twIU%$TPKOJdo{(Os6WShsERf&9b+Gr|SoBdexg
zl%HCXm3U;;^umEnl?^pnEBhp0)!LzJK57^|w`N)A&uhA_j@PVgyJDKYkeL}7;f>w|
zCa1oxxGDL|)s4+HS@l)vx2#F-LE(GJgg*#nvEqMxyAr#GzF9>hQs1W3hy3tk_y#kh
z+l;EKP5s1`C*DVANccWF>wb|tH9P&T8$ss!chlJ<F+$r`RTuP)^**V)_Lt<pdO-VH
q*Pxf~WCr0A=(!5>nyQM+f`xSx>MLUN8=H&5JIVJQNjl<q-v0n?ptJx0

literal 0
HcmV?d00001

diff --git a/tests/data/acpi/virt/FACP b/tests/data/acpi/virt/FACP
new file mode 100644
index 0000000000000000000000000000000000000000..27de99f51bfe846b1f8796ace49d83f5b33a1aed
GIT binary patch
literal 268
ycmZ>BbPnKQWME+3?d0$55v<@85#a0w6axw|fY>0Kx<CNcIA#XwTY+i=(L4a*H3tCz

literal 0
HcmV?d00001

diff --git a/tests/data/acpi/virt/GTDT b/tests/data/acpi/virt/GTDT
new file mode 100644
index 0000000000000000000000000000000000000000..10107a65e958ff6495bb8c17d63d0539690f59f6
GIT binary patch
literal 96
zcmZ<{aS2IaU|?Xn>E!S15v<@85#a0&6k`O6f!H7#8OTC8azL5|h^3)?DJYFj0RVOU
B2mt^9

literal 0
HcmV?d00001

diff --git a/tests/data/acpi/virt/MCFG b/tests/data/acpi/virt/MCFG
new file mode 100644
index 0000000000000000000000000000000000000000..e8987e1af0ec3829770bf4fe11fab02b06160dd2
GIT binary patch
literal 60
scmeZuc5}C3U|?YMck*}k2v%^42ypfViZKGkKx`0=1Oyx)oc|yS05YNo0RR91

literal 0
HcmV?d00001

diff --git a/tests/data/acpi/virt/SPCR b/tests/data/acpi/virt/SPCR
new file mode 100644
index 0000000000000000000000000000000000000000..377271a0e7817cc21a28c02123a89facad63604f
GIT binary patch
literal 80
zcmWFza1IJ!U|?VpcJg=j2v%^42yhMtiZKGkKx`1r48#l^3?L>agsBLmm>C$E7#RKo
I0Z0r60QF4^0RR91

literal 0
HcmV?d00001
Laszlo Ersek Jan. 15, 2019, 8:47 p.m. UTC | #2
On 01/15/19 16:41, Igor Mammedov wrote:
> Add firmware blobs built with PcdAcpiTestSupport=TRUE,
> that puts RSDP address in RAM after 1Mb aligned GUID
>   AB87A6B1-2034-BDA0-71BD-375007757785
> so that tests could scan and find it in RAM once firmware's
> initialized ACPI tables.
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> ---
>  Makefile              |   3 ++-
>  pc-bios/avmf.img      | Bin 0 -> 2097152 bytes
>  pc-bios/avmf_vars.img | Bin 0 -> 786432 bytes
>  3 files changed, 2 insertions(+), 1 deletion(-)
>  create mode 100644 pc-bios/avmf.img
>  create mode 100644 pc-bios/avmf_vars.img

"AVMF" is not a great name. "AAVMF" is a downstream name alright, but
many dislike it in upstream use. "edk2-aarch64" or "edk2-ArmVirtQemu"
would be more precise, but those are verbose. Sigh, why are names so
hard. What does everyone think?

Also, do you have to care about the license of firmware images built
from edk2 when bundling such images? Since edk2 commit 9a67ba261fe9
("ArmVirtPkg: Replace obsoleted network drivers from platform DSC/FDF.",
2018-12-14), you cannot build the ArmVirtQemu (aka AAVMF) firmware
without OpenSSL. Thus, the resultant license is 2-BSDL + OpenSSL -- for
now anyway.

Because, in the near future, that might change again. edk2 is in the
process of moving to Apache-2.0, and OpenSSL will also move (AFAICT) to
Apache-2.0 starting with release 3.0.0. (See commit 151333164ece,
"Change license to the Apache License v2.0", 2018-12-06.)

Thanks
Laszlo
Igor Mammedov Jan. 16, 2019, 12:29 p.m. UTC | #3
On Tue, 15 Jan 2019 21:47:49 +0100
Laszlo Ersek <lersek@redhat.com> wrote:

> On 01/15/19 16:41, Igor Mammedov wrote:
> > Add firmware blobs built with PcdAcpiTestSupport=TRUE,
> > that puts RSDP address in RAM after 1Mb aligned GUID
> >   AB87A6B1-2034-BDA0-71BD-375007757785
> > so that tests could scan and find it in RAM once firmware's
> > initialized ACPI tables.
> > 
> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > ---
> >  Makefile              |   3 ++-
> >  pc-bios/avmf.img      | Bin 0 -> 2097152 bytes
> >  pc-bios/avmf_vars.img | Bin 0 -> 786432 bytes
> >  3 files changed, 2 insertions(+), 1 deletion(-)
> >  create mode 100644 pc-bios/avmf.img
> >  create mode 100644 pc-bios/avmf_vars.img  
> 
> "AVMF" is not a great name. "AAVMF" is a downstream name alright, but
> many dislike it in upstream use. "edk2-aarch64" or "edk2-ArmVirtQemu"
> would be more precise, but those are verbose. Sigh, why are names so
> hard. What does everyone think?
I'm fine with either version.
 
> Also, do you have to care about the license of firmware images built
> from edk2 when bundling such images? Since edk2 commit 9a67ba261fe9
> ("ArmVirtPkg: Replace obsoleted network drivers from platform DSC/FDF.",
> 2018-12-14), you cannot build the ArmVirtQemu (aka AAVMF) firmware
> without OpenSSL. Thus, the resultant license is 2-BSDL + OpenSSL -- for
> now anyway.
> 
> Because, in the near future, that might change again. edk2 is in the
> process of moving to Apache-2.0, and OpenSSL will also move (AFAICT) to
> Apache-2.0 starting with release 3.0.0. (See commit 151333164ece,
> "Change license to the Apache License v2.0", 2018-12-06.)
That's another reason to look into EFI app approach (a simple one without
dependencies) ans let distro to provide firmware image.

> Thanks
> Laszlo
Michael S. Tsirkin Jan. 16, 2019, 4:01 p.m. UTC | #4
On Wed, Jan 16, 2019 at 01:29:53PM +0100, Igor Mammedov wrote:
> On Tue, 15 Jan 2019 21:47:49 +0100
> Laszlo Ersek <lersek@redhat.com> wrote:
> 
> > On 01/15/19 16:41, Igor Mammedov wrote:
> > > Add firmware blobs built with PcdAcpiTestSupport=TRUE,
> > > that puts RSDP address in RAM after 1Mb aligned GUID
> > >   AB87A6B1-2034-BDA0-71BD-375007757785
> > > so that tests could scan and find it in RAM once firmware's
> > > initialized ACPI tables.
> > > 
> > > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > > ---
> > >  Makefile              |   3 ++-
> > >  pc-bios/avmf.img      | Bin 0 -> 2097152 bytes
> > >  pc-bios/avmf_vars.img | Bin 0 -> 786432 bytes
> > >  3 files changed, 2 insertions(+), 1 deletion(-)
> > >  create mode 100644 pc-bios/avmf.img
> > >  create mode 100644 pc-bios/avmf_vars.img  
> > 
> > "AVMF" is not a great name. "AAVMF" is a downstream name alright, but
> > many dislike it in upstream use. "edk2-aarch64" or "edk2-ArmVirtQemu"
> > would be more precise, but those are verbose. Sigh, why are names so
> > hard. What does everyone think?
> I'm fine with either version.
>  
> > Also, do you have to care about the license of firmware images built
> > from edk2 when bundling such images? Since edk2 commit 9a67ba261fe9
> > ("ArmVirtPkg: Replace obsoleted network drivers from platform DSC/FDF.",
> > 2018-12-14), you cannot build the ArmVirtQemu (aka AAVMF) firmware
> > without OpenSSL. Thus, the resultant license is 2-BSDL + OpenSSL -- for
> > now anyway.
> > 
> > Because, in the near future, that might change again. edk2 is in the
> > process of moving to Apache-2.0, and OpenSSL will also move (AFAICT) to
> > Apache-2.0 starting with release 3.0.0. (See commit 151333164ece,
> > "Change license to the Apache License v2.0", 2018-12-06.)
> That's another reason to look into EFI app approach (a simple one without
> dependencies) ans let distro to provide firmware image.

That will mean supporting very old firmware with the app.
I'm all for the EFI app approach for modularity
but I think we should include the batteries with this toy.


> > Thanks
> > Laszlo
Laszlo Ersek Jan. 17, 2019, 8:53 a.m. UTC | #5
Hello Michael,

On 01/16/19 17:01, Michael S. Tsirkin wrote:
> On Wed, Jan 16, 2019 at 01:29:53PM +0100, Igor Mammedov wrote:
>> On Tue, 15 Jan 2019 21:47:49 +0100
>> Laszlo Ersek <lersek@redhat.com> wrote:
>>
>>> On 01/15/19 16:41, Igor Mammedov wrote:
>>>> Add firmware blobs built with PcdAcpiTestSupport=TRUE,
>>>> that puts RSDP address in RAM after 1Mb aligned GUID
>>>>   AB87A6B1-2034-BDA0-71BD-375007757785
>>>> so that tests could scan and find it in RAM once firmware's
>>>> initialized ACPI tables.
>>>>
>>>> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
>>>> ---
>>>>  Makefile              |   3 ++-
>>>>  pc-bios/avmf.img      | Bin 0 -> 2097152 bytes
>>>>  pc-bios/avmf_vars.img | Bin 0 -> 786432 bytes
>>>>  3 files changed, 2 insertions(+), 1 deletion(-)
>>>>  create mode 100644 pc-bios/avmf.img
>>>>  create mode 100644 pc-bios/avmf_vars.img  
>>>
>>> "AVMF" is not a great name. "AAVMF" is a downstream name alright, but
>>> many dislike it in upstream use. "edk2-aarch64" or "edk2-ArmVirtQemu"
>>> would be more precise, but those are verbose. Sigh, why are names so
>>> hard. What does everyone think?
>> I'm fine with either version.
>>  
>>> Also, do you have to care about the license of firmware images built
>>> from edk2 when bundling such images? Since edk2 commit 9a67ba261fe9
>>> ("ArmVirtPkg: Replace obsoleted network drivers from platform DSC/FDF.",
>>> 2018-12-14), you cannot build the ArmVirtQemu (aka AAVMF) firmware
>>> without OpenSSL. Thus, the resultant license is 2-BSDL + OpenSSL -- for
>>> now anyway.
>>>
>>> Because, in the near future, that might change again. edk2 is in the
>>> process of moving to Apache-2.0, and OpenSSL will also move (AFAICT) to
>>> Apache-2.0 starting with release 3.0.0. (See commit 151333164ece,
>>> "Change license to the Apache License v2.0", 2018-12-06.)
>> That's another reason to look into EFI app approach (a simple one without
>> dependencies) ans let distro to provide firmware image.
> 
> That will mean supporting very old firmware with the app.
> I'm all for the EFI app approach for modularity
> but I think we should include the batteries with this toy.

There are two approaches here. (Both require that edk2 be present in the
QEMU source tree as a submodule, and both require QEMU to receive a
script for building edk2 in *some* way.)


(1) Carry the bios-tables-test helper UEFI app as an "out of tree"
module, from the perspective of edk2; carry it natively in the QEMU tree.

* Edk2 supports this use case as a first class citizen.

* In this case, the UEFI application is permitted to link only such
libraries from edk2 that the resultant binary inherit no platform
dependencies. The app can only be made dependent on a minimum UEFI spec
release, if necessary. The resultant binary can be run on any conformant
UEFI implementation, including physical hardware.

An example for "platform dependency prohibition" is that the X64 build
of the app could not print debug messages to the QEMU debug port (like
the rest of OVMF does), only to the UEFI console. On OVMF, that console
would direct the debug messages to the serial port and the graphics card.

Regarding a minimum UEFI spec release, the oldest UEFI spec I have on my
disk is "2.3.1, Errata C", dated "June 27, 2012". I'm not aware of
anything in the helper app that requires more recent spec features.

* Should we have to extend the UEFI app with a feature, we could easily
do that in sync with the consumer QEMU test code.

* The build output to commit to the QEMU repo would be an ISO image
containing the UEFI binary as a "boot loader". It would not contain
OpenSSL bits. The derived license would be a combination of core edk2's
license and our out-of-edk2-module's license.


(2) Carry the bios-tables-test helper *functionality* (not necessarily a
standalone UEFI application) in edk2; add a custom build flag to the
OVMF and ArmVirtQemu firmware platforms to enable the helper functionality.

* In this case, the test helper logic is permitted to rely on platform
internals. For example, on X64, it could print debug messages to the
QEMU debug port, like the rest of OVMF does.

* Whenever a new feature became necessary, edk2 would get new patches,
then QEMU would bump the submodule commit reference accordingly.

* The build output to commit to the QEMU repo would be a custom firmware
image (built with the special build flag mentioned above), and no other
bootable media would be formatted / saved. The custom firmware image
would contain OpenSSL bits. The image's license would be derived from
edk2 and from the OpenSSL submodule used by edk2.

* Also, the custom firmware image would not be suitable for "production"
use, so it couldn't be at once bundled under pc-bios as well -- that
would remain a separate RFE.


Since last night, I have some rough patches for (1), including the
QEMU-side Makefile + build script.

Regarding (2), I also have the edk2-native code ready (I had posted it a
few weeks back -- that's what Igor used now). For the QEMU side of
approach (2), I reckon I could reuse most of the build script I already
have for (1).

Could we please decide for (1) vs (2), before I put more work into (1)?

Thanks
Laszlo
Laszlo Ersek Jan. 17, 2019, 8:58 a.m. UTC | #6
On 01/17/19 09:53, Laszlo Ersek wrote:
> Hello Michael,
> 
> On 01/16/19 17:01, Michael S. Tsirkin wrote:
>> On Wed, Jan 16, 2019 at 01:29:53PM +0100, Igor Mammedov wrote:
>>> On Tue, 15 Jan 2019 21:47:49 +0100
>>> Laszlo Ersek <lersek@redhat.com> wrote:
>>>
>>>> On 01/15/19 16:41, Igor Mammedov wrote:
>>>>> Add firmware blobs built with PcdAcpiTestSupport=TRUE,
>>>>> that puts RSDP address in RAM after 1Mb aligned GUID
>>>>>   AB87A6B1-2034-BDA0-71BD-375007757785
>>>>> so that tests could scan and find it in RAM once firmware's
>>>>> initialized ACPI tables.
>>>>>
>>>>> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
>>>>> ---
>>>>>  Makefile              |   3 ++-
>>>>>  pc-bios/avmf.img      | Bin 0 -> 2097152 bytes
>>>>>  pc-bios/avmf_vars.img | Bin 0 -> 786432 bytes
>>>>>  3 files changed, 2 insertions(+), 1 deletion(-)
>>>>>  create mode 100644 pc-bios/avmf.img
>>>>>  create mode 100644 pc-bios/avmf_vars.img  
>>>>
>>>> "AVMF" is not a great name. "AAVMF" is a downstream name alright, but
>>>> many dislike it in upstream use. "edk2-aarch64" or "edk2-ArmVirtQemu"
>>>> would be more precise, but those are verbose. Sigh, why are names so
>>>> hard. What does everyone think?
>>> I'm fine with either version.
>>>  
>>>> Also, do you have to care about the license of firmware images built
>>>> from edk2 when bundling such images? Since edk2 commit 9a67ba261fe9
>>>> ("ArmVirtPkg: Replace obsoleted network drivers from platform DSC/FDF.",
>>>> 2018-12-14), you cannot build the ArmVirtQemu (aka AAVMF) firmware
>>>> without OpenSSL. Thus, the resultant license is 2-BSDL + OpenSSL -- for
>>>> now anyway.
>>>>
>>>> Because, in the near future, that might change again. edk2 is in the
>>>> process of moving to Apache-2.0, and OpenSSL will also move (AFAICT) to
>>>> Apache-2.0 starting with release 3.0.0. (See commit 151333164ece,
>>>> "Change license to the Apache License v2.0", 2018-12-06.)
>>> That's another reason to look into EFI app approach (a simple one without
>>> dependencies) ans let distro to provide firmware image.
>>
>> That will mean supporting very old firmware with the app.
>> I'm all for the EFI app approach for modularity
>> but I think we should include the batteries with this toy.
> 
> There are two approaches here. (Both require that edk2 be present in the
> QEMU source tree as a submodule, and both require QEMU to receive a
> script for building edk2 in *some* way.)
> 
> 
> (1) Carry the bios-tables-test helper UEFI app as an "out of tree"
> module, from the perspective of edk2; carry it natively in the QEMU tree.
> 
> * Edk2 supports this use case as a first class citizen.
> 
> * In this case, the UEFI application is permitted to link only such
> libraries from edk2 that the resultant binary inherit no platform
> dependencies. The app can only be made dependent on a minimum UEFI spec
> release, if necessary. The resultant binary can be run on any conformant
> UEFI implementation, including physical hardware.
> 
> An example for "platform dependency prohibition" is that the X64 build
> of the app could not print debug messages to the QEMU debug port (like
> the rest of OVMF does), only to the UEFI console. On OVMF, that console
> would direct the debug messages to the serial port and the graphics card.
> 
> Regarding a minimum UEFI spec release, the oldest UEFI spec I have on my
> disk is "2.3.1, Errata C", dated "June 27, 2012". I'm not aware of
> anything in the helper app that requires more recent spec features.
> 
> * Should we have to extend the UEFI app with a feature, we could easily
> do that in sync with the consumer QEMU test code.
> 
> * The build output to commit to the QEMU repo would be an ISO image
> containing the UEFI binary as a "boot loader". It would not contain
> OpenSSL bits. The derived license would be a combination of core edk2's
> license and our out-of-edk2-module's license.
> 
> 
> (2) Carry the bios-tables-test helper *functionality* (not necessarily a
> standalone UEFI application) in edk2; add a custom build flag to the
> OVMF and ArmVirtQemu firmware platforms to enable the helper functionality.
> 
> * In this case, the test helper logic is permitted to rely on platform
> internals. For example, on X64, it could print debug messages to the
> QEMU debug port, like the rest of OVMF does.
> 
> * Whenever a new feature became necessary, edk2 would get new patches,
> then QEMU would bump the submodule commit reference accordingly.
> 
> * The build output to commit to the QEMU repo would be a custom firmware
> image (built with the special build flag mentioned above), and no other
> bootable media would be formatted / saved. The custom firmware image
> would contain OpenSSL bits. The image's license would be derived from
> edk2 and from the OpenSSL submodule used by edk2.
> 
> * Also, the custom firmware image would not be suitable for "production"
> use, so it couldn't be at once bundled under pc-bios as well -- that
> would remain a separate RFE.
> 
> 
> Since last night, I have some rough patches for (1), including the
> QEMU-side Makefile + build script.
> 
> Regarding (2), I also have the edk2-native code ready (I had posted it a
> few weeks back -- that's what Igor used now). For the QEMU side of
> approach (2), I reckon I could reuse most of the build script I already
> have for (1).
> 
> Could we please decide for (1) vs (2), before I put more work into (1)?

Sorry, borked English, I meant "decide between (1) and (2)". I didn't
intend to express a preference (I don't have one).

Thanks,
Laszlo
Gerd Hoffmann Jan. 17, 2019, 10:22 a.m. UTC | #7
Hi,

> >>>>  create mode 100644 pc-bios/avmf.img
> >>>>  create mode 100644 pc-bios/avmf_vars.img  
> >>>
> >>> "AVMF" is not a great name. "AAVMF" is a downstream name alright, but
> >>> many dislike it in upstream use. "edk2-aarch64" or "edk2-ArmVirtQemu"
> >>> would be more precise, but those are verbose. Sigh, why are names so
> >>> hard. What does everyone think?
> >> I'm fine with either version.

How about placing them in pc-bios/efi-$arch subdirs and not renaming the
files, i.e. that would be ...

	pc-bios/efi-aarch64/QEMU_EFI.fd
	pc-bios/efi-aarch64/QEMU_VARS.fd

... for arm, and ...

	pc-bios/efi-x86_64/OVMF_CODE.fd
	pc-bios/efi-x86_64/OVMF_VARS.fd

... for x86.

> Could we please decide for (1) vs (2), before I put more work into (1)?

I'd tend to prefer (1).

cheers,
  Gerd
Laszlo Ersek Jan. 17, 2019, 12:54 p.m. UTC | #8
On 01/17/19 11:22, Gerd Hoffmann wrote:
>   Hi,
> 
>>>>>>  create mode 100644 pc-bios/avmf.img
>>>>>>  create mode 100644 pc-bios/avmf_vars.img  
>>>>>
>>>>> "AVMF" is not a great name. "AAVMF" is a downstream name alright, but
>>>>> many dislike it in upstream use. "edk2-aarch64" or "edk2-ArmVirtQemu"
>>>>> would be more precise, but those are verbose. Sigh, why are names so
>>>>> hard. What does everyone think?
>>>> I'm fine with either version.
> 
> How about placing them in pc-bios/efi-$arch subdirs and not renaming the
> files, i.e. that would be ...
> 
> 	pc-bios/efi-aarch64/QEMU_EFI.fd
> 	pc-bios/efi-aarch64/QEMU_VARS.fd
> 
> ... for arm, and ...
> 
> 	pc-bios/efi-x86_64/OVMF_CODE.fd
> 	pc-bios/efi-x86_64/OVMF_VARS.fd
> 
> ... for x86.

That sounds good to me. One thing to note is that the arm/aarch64 images
have to be padded to 64MB, so I generally append ".padded" to those file
names. Would that be OK? Any better ideas?

>> Could we please decide for (1) vs (2), before I put more work into (1)?
> 
> I'd tend to prefer (1).

Thanks. I have a patch set that's almost suitable for posting as an RFC.
I should split the last patch and write some sensible commit messages.

BTW, the bundling under pc-bios is a bit larger task than it immediately
appears:

- there are many build options to consider (as you know perfectly well
  :) ),

- plus now we have the "docs/interop/firmware.json" schema too, hence
  whatever images we build for end-user consumption, should likely be
  accompanied by metafiles that conform to this schema.

I think once we introduce the "roms/edk2" submodule for the current
purpose, we could address the pc-bios binaries (+metafiles) in a
separate series, on top.

Thanks,
Laszlo
Gerd Hoffmann Jan. 17, 2019, 2:09 p.m. UTC | #9
On Thu, Jan 17, 2019 at 01:54:51PM +0100, Laszlo Ersek wrote:
> On 01/17/19 11:22, Gerd Hoffmann wrote:
> >   Hi,
> > 
> >>>>>>  create mode 100644 pc-bios/avmf.img
> >>>>>>  create mode 100644 pc-bios/avmf_vars.img  
> >>>>>
> >>>>> "AVMF" is not a great name. "AAVMF" is a downstream name alright, but
> >>>>> many dislike it in upstream use. "edk2-aarch64" or "edk2-ArmVirtQemu"
> >>>>> would be more precise, but those are verbose. Sigh, why are names so
> >>>>> hard. What does everyone think?
> >>>> I'm fine with either version.
> > 
> > How about placing them in pc-bios/efi-$arch subdirs and not renaming the
> > files, i.e. that would be ...
> > 
> > 	pc-bios/efi-aarch64/QEMU_EFI.fd
> > 	pc-bios/efi-aarch64/QEMU_VARS.fd
> > 
> > ... for arm, and ...
> > 
> > 	pc-bios/efi-x86_64/OVMF_CODE.fd
> > 	pc-bios/efi-x86_64/OVMF_VARS.fd
> > 
> > ... for x86.
> 
> That sounds good to me. One thing to note is that the arm/aarch64 images
> have to be padded to 64MB, so I generally append ".padded" to those file
> names. Would that be OK? Any better ideas?

Ah, right, the arm versions can't be used as-is with pflash.  In my rpm
builds I've named the padded version "QEMU_EFI-pflash.raw".  Using
.padded looks fine to me too.

Other idea:  Does it make sense to use qcow2 for the pflash images?
i.e. "qemu-img create -f qcow2 -b QEMU_EFI.fd -F raw QEMU_EFI.qcow2 64M"?

cheers,
  Gerd
Igor Mammedov Jan. 17, 2019, 3:42 p.m. UTC | #10
On Thu, 17 Jan 2019 13:54:51 +0100
Laszlo Ersek <lersek@redhat.com> wrote:

> On 01/17/19 11:22, Gerd Hoffmann wrote:
> >   Hi,
> >   
> >>>>>>  create mode 100644 pc-bios/avmf.img
> >>>>>>  create mode 100644 pc-bios/avmf_vars.img    
> >>>>>
> >>>>> "AVMF" is not a great name. "AAVMF" is a downstream name alright, but
> >>>>> many dislike it in upstream use. "edk2-aarch64" or "edk2-ArmVirtQemu"
> >>>>> would be more precise, but those are verbose. Sigh, why are names so
> >>>>> hard. What does everyone think?  
> >>>> I'm fine with either version.  
> > 
> > How about placing them in pc-bios/efi-$arch subdirs and not renaming the
> > files, i.e. that would be ...
> > 
> > 	pc-bios/efi-aarch64/QEMU_EFI.fd
> > 	pc-bios/efi-aarch64/QEMU_VARS.fd
> > 
> > ... for arm, and ...
> > 
> > 	pc-bios/efi-x86_64/OVMF_CODE.fd
> > 	pc-bios/efi-x86_64/OVMF_VARS.fd
> > 
> > ... for x86.  
if it's non production images (i.e. openssl-less) than maybe use
tests/data/acpi instead of pc-bios for now, once we have pc-bios ones
ready we drop test specific and use production ones.

> That sounds good to me. One thing to note is that the arm/aarch64 images
> have to be padded to 64MB, so I generally append ".padded" to those file
> names. Would that be OK? Any better ideas?
Images could be pre-padded and ready for commit, wrt large size we can commit
them compressed and decompress for using in tests instead of padding padding
I'm doing now before I use them.

> >> Could we please decide for (1) vs (2), before I put more work into (1)?  
> > 
> > I'd tend to prefer (1).  

+1

> 
> Thanks. I have a patch set that's almost suitable for posting as an RFC.
> I should split the last patch and write some sensible commit messages.
> 
> BTW, the bundling under pc-bios is a bit larger task than it immediately
> appears:
> 
> - there are many build options to consider (as you know perfectly well
>   :) ),
> 
> - plus now we have the "docs/interop/firmware.json" schema too, hence
>   whatever images we build for end-user consumption, should likely be
>   accompanied by metafiles that conform to this schema.
> 
> I think once we introduce the "roms/edk2" submodule for the current
> purpose, we could address the pc-bios binaries (+metafiles) in a
> separate series, on top.
> 
> Thanks,
> Laszlo
>
Laszlo Ersek Jan. 18, 2019, 11:23 p.m. UTC | #11
On 01/17/19 15:09, Gerd Hoffmann wrote:
> On Thu, Jan 17, 2019 at 01:54:51PM +0100, Laszlo Ersek wrote:
>> On 01/17/19 11:22, Gerd Hoffmann wrote:
>>>   Hi,
>>>
>>>>>>>>  create mode 100644 pc-bios/avmf.img
>>>>>>>>  create mode 100644 pc-bios/avmf_vars.img  
>>>>>>>
>>>>>>> "AVMF" is not a great name. "AAVMF" is a downstream name alright, but
>>>>>>> many dislike it in upstream use. "edk2-aarch64" or "edk2-ArmVirtQemu"
>>>>>>> would be more precise, but those are verbose. Sigh, why are names so
>>>>>>> hard. What does everyone think?
>>>>>> I'm fine with either version.
>>>
>>> How about placing them in pc-bios/efi-$arch subdirs and not renaming the
>>> files, i.e. that would be ...
>>>
>>> 	pc-bios/efi-aarch64/QEMU_EFI.fd
>>> 	pc-bios/efi-aarch64/QEMU_VARS.fd
>>>
>>> ... for arm, and ...
>>>
>>> 	pc-bios/efi-x86_64/OVMF_CODE.fd
>>> 	pc-bios/efi-x86_64/OVMF_VARS.fd
>>>
>>> ... for x86.
>>
>> That sounds good to me. One thing to note is that the arm/aarch64 images
>> have to be padded to 64MB, so I generally append ".padded" to those file
>> names. Would that be OK? Any better ideas?
> 
> Ah, right, the arm versions can't be used as-is with pflash.  In my rpm
> builds I've named the padded version "QEMU_EFI-pflash.raw".  Using
> .padded looks fine to me too.
> 
> Other idea:  Does it make sense to use qcow2 for the pflash images?
> i.e. "qemu-img create -f qcow2 -b QEMU_EFI.fd -F raw QEMU_EFI.qcow2 64M"?

That's a long story.

In a perfect world, the answer would be, "qcow2 (and all its features)
make perfect sense for pflash images". In the world we have however,
"savevm" exists, which might dump live guest RAM into whichever qcow2
disk it finds first. See
<https://bugzilla.redhat.com/show_bug.cgi?id=1214187>.

(I apologize to any non-RedHatters reading this; that's a private RHBZ
for some obscure reason, and I dare not open it up myself.)

See also libvirt commit 9e2465834f4b ("qemu: snapshot: Forbid internal
snapshots with pflash firmware", 2017-03-24).

So, for now, it remains the case that we're better off with raw, at
least for the writeable (=varstore) pflash chip.

In addition, when libvirt composes the cmdline, it specifies format=raw
for unit=0 (i.e., firmware image pflash) as well. (That's actually a
good thing, because we shouldn't auto-detect the format, and qcow2 is
out (for now), so we should specify format=raw. In the future, the
domain XML schema may have to be extended with "format" too.)

... For now, it's probably best to check the unpadded FDs into git, and
pad them only in "make install".

Thanks,
Laszlo