How to reserve guest physical region for ACPI

On Mon, Dec 28, 2015 at 10:39:04AM +0800, Xiao Guangrong wrote:
> 
> Hi Michael, Paolo,
> 
> Now it is the time to return to the challenge that how to reserve guest
> physical region internally used by ACPI.
> 
> Igor suggested that:
> | An alternative place to allocate reserve from could be high memory.
> | For pc we have "reserved-memory-end" which currently makes sure
> | that hotpluggable memory range isn't used by firmware
> (https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg00926.html)

I don't want to tie things to reserved-memory-end because this
does not scale: next time we need to reserve memory,
we'll need to find yet another way to figure out what is where.

I would like ./hw/acpi/bios-linker-loader.c interface to be extended to
support 64 bit RAM instead (and maybe a way to allocate and
zero-initialize buffer without loading it through fwcfg), this way bios
does the allocation, and addresses can be patched into acpi.

See patch at the bottom that might be handy.

> he also innovated a way to use 64-bit address in DSDT/SSDT.rev = 1:
> | when writing ASL one shall make sure that only XP supported
> | features are in global scope, which is evaluated when tables
> | are loaded and features of rev2 and higher are inside methods.
> | That way XP doesn't crash as far as it doesn't evaluate unsupported
> | opcode and one can guard those opcodes checking _REV object if neccesary.
> (https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg01010.html)

Yes, this technique works.

An alternative is to add an XSDT, XP ignores that.
XSDT at the moment breaks OVMF (because it loads both
the RSDT and the XSDT, which is wrong), but I think
Laszlo was working on a fix for that.

> Michael, Paolo, what do you think about these ideas?
> 
> Thanks!



So using a patch below, we can add Name(PQRS, 0x0) at the top of the
SSDT (or bottom, or add a separate SSDT just for that).  It returns the
current offset so we can add that to the linker.

Won't work if you append the Name to the Aml structure (these can be
nested to arbitrary depth using aml_append), so using plain GArray for
this API makes sense to me.

--->

acpi: add build_append_named_dword, returning an offset in buffer

This is a very limited form of support for runtime patching -
similar in functionality to what we can do with ACPI_EXTRACT
macros in python, but implemented in C.

This is to allow ACPI code direct access to data tables -
which is exactly what DataTableRegion is there for, except
no known windows release so far implements DataTableRegion.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

---

On Mon, 28 Dec 2015 14:50:15 +0200
"Michael S. Tsirkin" <mst@redhat.com> wrote:

> On Mon, Dec 28, 2015 at 10:39:04AM +0800, Xiao Guangrong wrote:
> > 
> > Hi Michael, Paolo,
> > 
> > Now it is the time to return to the challenge that how to reserve guest
> > physical region internally used by ACPI.
> > 
> > Igor suggested that:
> > | An alternative place to allocate reserve from could be high memory.
> > | For pc we have "reserved-memory-end" which currently makes sure
> > | that hotpluggable memory range isn't used by firmware
> > (https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg00926.html)
> 
> I don't want to tie things to reserved-memory-end because this
> does not scale: next time we need to reserve memory,
> we'll need to find yet another way to figure out what is where.
Could you elaborate a bit more on a problem you're seeing?

To me it looks like it scales rather well.
For example lets imagine that we adding a device
that has some on device memory that should be mapped into GPA
code to do so would look like:

  pc_machine_device_plug_cb(dev)
  {
   ...
   if (dev == OUR_NEW_DEVICE_TYPE) {
       memory_region_add_subregion(as, current_reserved_end, &dev->mr);
       set_new_reserved_end(current_reserved_end + memory_region_size(&dev->mr));
   }
  }

we can practically add any number of new devices that way.

 
> I would like ./hw/acpi/bios-linker-loader.c interface to be extended to
> support 64 bit RAM instead (and maybe a way to allocate and
> zero-initialize buffer without loading it through fwcfg), this way bios
> does the allocation, and addresses can be patched into acpi.
and then guest side needs to parse/execute some AML that would
initialize QEMU side so it would know where to write data.

bios-linker-loader is a great interface for initializing some
guest owned data and linking it together but I think it adds
unnecessary complexity and is misused if it's used to handle
device owned data/on device memory in this and VMGID cases.

There was RFC on list to make BIOS boot from NVDIMM already
doing some ACPI table lookup/parsing. Now if they were forced
to also parse and execute AML to initialize QEMU with guest
allocated address that would complicate them quite a bit.
While with NVDIMM control memory region mapped directly by QEMU,
respective patches don't need in any way to initialize QEMU,
all they would need just read necessary data from control region.

Also using bios-linker-loader takes away some usable RAM
from guest and in the end that doesn't scale,
the more devices I add the less usable RAM is left for guest OS
while all the device needs is a piece of GPA address space
that would belong to it.

> 
> See patch at the bottom that might be handy.
> 
> > he also innovated a way to use 64-bit address in DSDT/SSDT.rev = 1:
> > | when writing ASL one shall make sure that only XP supported
> > | features are in global scope, which is evaluated when tables
> > | are loaded and features of rev2 and higher are inside methods.
> > | That way XP doesn't crash as far as it doesn't evaluate unsupported
> > | opcode and one can guard those opcodes checking _REV object if neccesary.
> > (https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg01010.html)
> 
> Yes, this technique works.
> 
> An alternative is to add an XSDT, XP ignores that.
> XSDT at the moment breaks OVMF (because it loads both
> the RSDT and the XSDT, which is wrong), but I think
> Laszlo was working on a fix for that.
Using XSDT would increase ACPI tables occupied RAM
as it would duplicate DSDT + non XP supported AML
at global namespace.

So far we've managed keep DSDT compatible with XP while
introducing features from v2 and higher ACPI revisions as
AML that is only evaluated on demand.
We can continue doing so unless we have to unconditionally
add incompatible AML at global scope.


> 
> > Michael, Paolo, what do you think about these ideas?
> > 
> > Thanks!
> 
> 
> 
> So using a patch below, we can add Name(PQRS, 0x0) at the top of the
> SSDT (or bottom, or add a separate SSDT just for that).  It returns the
> current offset so we can add that to the linker.
> 
> Won't work if you append the Name to the Aml structure (these can be
> nested to arbitrary depth using aml_append), so using plain GArray for
> this API makes sense to me.
> 
> --->
> 
> acpi: add build_append_named_dword, returning an offset in buffer
> 
> This is a very limited form of support for runtime patching -
> similar in functionality to what we can do with ACPI_EXTRACT
> macros in python, but implemented in C.
> 
> This is to allow ACPI code direct access to data tables -
> which is exactly what DataTableRegion is there for, except
> no known windows release so far implements DataTableRegion.
unsupported means Windows will BSOD, so it's practically
unusable unless MS will patch currently existing Windows
versions.

Another thing about DataTableRegion is that ACPI tables are
supposed to have static content which matches checksum in
table the header while you are trying to use it for dynamic
data. It would be cleaner/more compatible to teach
bios-linker-loader to just allocate memory and patch AML
with the allocated address.

Also if OperationRegion() is used, then one has to patch
DefOpRegion directly as RegionOffset must be Integer,
using variable names is not permitted there.

> 
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> 
> ---
> 
> diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
> index 1b632dc..f8998ea 100644
> --- a/include/hw/acpi/aml-build.h
> +++ b/include/hw/acpi/aml-build.h
> @@ -286,4 +286,7 @@ void acpi_build_tables_cleanup(AcpiBuildTables *tables, bool mfre);
>  void
>  build_rsdt(GArray *table_data, GArray *linker, GArray *table_offsets);
>  
> +int
> +build_append_named_dword(GArray *array, const char *name_format, ...);
> +
>  #endif
> diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
> index 0d4b324..7f9fa65 100644
> --- a/hw/acpi/aml-build.c
> +++ b/hw/acpi/aml-build.c
> @@ -262,6 +262,32 @@ static void build_append_int(GArray *table, uint64_t value)
>      }
>  }
>  
> +/* Build NAME(XXXX, 0x0) where 0x0 is encoded as a qword,
> + * and return the offset to 0x0 for runtime patching.
> + *
> + * Warning: runtime patching is best avoided. Only use this as
> + * a replacement for DataTableRegion (for guests that don't
> + * support it).
> + */
> +int
> +build_append_named_qword(GArray *array, const char *name_format, ...)
> +{
> +    int offset;
> +    va_list ap;
> +
> +    va_start(ap, name_format);
> +    build_append_namestringv(array, name_format, ap);
> +    va_end(ap);
> +
> +    build_append_byte(array, 0x0E); /* QWordPrefix */
> +
> +    offset = array->len;
> +    build_append_int_noprefix(array, 0x0, 8);
> +    assert(array->len == offset + 8);
> +
> +    return offset;
> +}
> +
>  static GPtrArray *alloc_list;
>  
>  static Aml *aml_alloc(void)
> 
>

On Wed, Dec 30, 2015 at 04:55:54PM +0100, Igor Mammedov wrote:
> On Mon, 28 Dec 2015 14:50:15 +0200
> "Michael S. Tsirkin" <mst@redhat.com> wrote:
> 
> > On Mon, Dec 28, 2015 at 10:39:04AM +0800, Xiao Guangrong wrote:
> > > 
> > > Hi Michael, Paolo,
> > > 
> > > Now it is the time to return to the challenge that how to reserve guest
> > > physical region internally used by ACPI.
> > > 
> > > Igor suggested that:
> > > | An alternative place to allocate reserve from could be high memory.
> > > | For pc we have "reserved-memory-end" which currently makes sure
> > > | that hotpluggable memory range isn't used by firmware
> > > (https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg00926.html)
> > 
> > I don't want to tie things to reserved-memory-end because this
> > does not scale: next time we need to reserve memory,
> > we'll need to find yet another way to figure out what is where.
> Could you elaborate a bit more on a problem you're seeing?
> 
> To me it looks like it scales rather well.
> For example lets imagine that we adding a device
> that has some on device memory that should be mapped into GPA
> code to do so would look like:
> 
>   pc_machine_device_plug_cb(dev)
>   {
>    ...
>    if (dev == OUR_NEW_DEVICE_TYPE) {
>        memory_region_add_subregion(as, current_reserved_end, &dev->mr);
>        set_new_reserved_end(current_reserved_end + memory_region_size(&dev->mr));
>    }
>   }
> 
> we can practically add any number of new devices that way.

Yes but we'll have to build a host side allocator for these, and that's
nasty. We'll also have to maintain these addresses indefinitely (at
least per machine version) as they are guest visible.
Not only that, there's no way for guest to know if we move things
around, so basically we'll never be able to change addresses.


>  
> > I would like ./hw/acpi/bios-linker-loader.c interface to be extended to
> > support 64 bit RAM instead (and maybe a way to allocate and
> > zero-initialize buffer without loading it through fwcfg), this way bios
> > does the allocation, and addresses can be patched into acpi.
> and then guest side needs to parse/execute some AML that would
> initialize QEMU side so it would know where to write data.

Well not really - we can put it in a data table, by itself
so it's easy to find.

AML is only needed if access from ACPI is desired.


> bios-linker-loader is a great interface for initializing some
> guest owned data and linking it together but I think it adds
> unnecessary complexity and is misused if it's used to handle
> device owned data/on device memory in this and VMGID cases.

I want a generic interface for guest to enumerate these things.  linker
seems quite reasonable but if you see a reason why it won't do, or want
to propose a better interface, fine.

PCI would do, too - though windows guys had concerns about
returning PCI BARs from ACPI.


> There was RFC on list to make BIOS boot from NVDIMM already
> doing some ACPI table lookup/parsing. Now if they were forced
> to also parse and execute AML to initialize QEMU with guest
> allocated address that would complicate them quite a bit.

If they just need to find a table by name, it won't be
too bad, would it?

> While with NVDIMM control memory region mapped directly by QEMU,
> respective patches don't need in any way to initialize QEMU,
> all they would need just read necessary data from control region.
> 
> Also using bios-linker-loader takes away some usable RAM
> from guest and in the end that doesn't scale,
> the more devices I add the less usable RAM is left for guest OS
> while all the device needs is a piece of GPA address space
> that would belong to it.

I don't get this comment. I don't think it's MMIO that is wanted.
If it's backed by qemu virtual memory then it's RAM.

> > 
> > See patch at the bottom that might be handy.
> > 
> > > he also innovated a way to use 64-bit address in DSDT/SSDT.rev = 1:
> > > | when writing ASL one shall make sure that only XP supported
> > > | features are in global scope, which is evaluated when tables
> > > | are loaded and features of rev2 and higher are inside methods.
> > > | That way XP doesn't crash as far as it doesn't evaluate unsupported
> > > | opcode and one can guard those opcodes checking _REV object if neccesary.
> > > (https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg01010.html)
> > 
> > Yes, this technique works.
> > 
> > An alternative is to add an XSDT, XP ignores that.
> > XSDT at the moment breaks OVMF (because it loads both
> > the RSDT and the XSDT, which is wrong), but I think
> > Laszlo was working on a fix for that.
> Using XSDT would increase ACPI tables occupied RAM
> as it would duplicate DSDT + non XP supported AML
> at global namespace.

Not at all - I posted patches linking to same
tables from both RSDT and XSDT at some point.
Only the list of pointers would be different.

> So far we've managed keep DSDT compatible with XP while
> introducing features from v2 and higher ACPI revisions as
> AML that is only evaluated on demand.
> We can continue doing so unless we have to unconditionally
> add incompatible AML at global scope.
> 

Yes.

> > 
> > > Michael, Paolo, what do you think about these ideas?
> > > 
> > > Thanks!
> > 
> > 
> > 
> > So using a patch below, we can add Name(PQRS, 0x0) at the top of the
> > SSDT (or bottom, or add a separate SSDT just for that).  It returns the
> > current offset so we can add that to the linker.
> > 
> > Won't work if you append the Name to the Aml structure (these can be
> > nested to arbitrary depth using aml_append), so using plain GArray for
> > this API makes sense to me.
> > 
> > --->
> > 
> > acpi: add build_append_named_dword, returning an offset in buffer
> > 
> > This is a very limited form of support for runtime patching -
> > similar in functionality to what we can do with ACPI_EXTRACT
> > macros in python, but implemented in C.
> > 
> > This is to allow ACPI code direct access to data tables -
> > which is exactly what DataTableRegion is there for, except
> > no known windows release so far implements DataTableRegion.
> unsupported means Windows will BSOD, so it's practically
> unusable unless MS will patch currently existing Windows
> versions.

Yes. That's why my patch allows patching SSDT without using
DataTableRegion.

> Another thing about DataTableRegion is that ACPI tables are
> supposed to have static content which matches checksum in
> table the header while you are trying to use it for dynamic
> data. It would be cleaner/more compatible to teach
> bios-linker-loader to just allocate memory and patch AML
> with the allocated address.

Yes - if address is static, you need to put it outside
the table. Can come right before or right after this.

> Also if OperationRegion() is used, then one has to patch
> DefOpRegion directly as RegionOffset must be Integer,
> using variable names is not permitted there.

I am not sure the comment was understood correctly.
The comment says really "we can't use DataTableRegion
so here is an alternative".

> 
> > 
> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > 
> > ---
> > 
> > diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
> > index 1b632dc..f8998ea 100644
> > --- a/include/hw/acpi/aml-build.h
> > +++ b/include/hw/acpi/aml-build.h
> > @@ -286,4 +286,7 @@ void acpi_build_tables_cleanup(AcpiBuildTables *tables, bool mfre);
> >  void
> >  build_rsdt(GArray *table_data, GArray *linker, GArray *table_offsets);
> >  
> > +int
> > +build_append_named_dword(GArray *array, const char *name_format, ...);
> > +
> >  #endif
> > diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
> > index 0d4b324..7f9fa65 100644
> > --- a/hw/acpi/aml-build.c
> > +++ b/hw/acpi/aml-build.c
> > @@ -262,6 +262,32 @@ static void build_append_int(GArray *table, uint64_t value)
> >      }
> >  }
> >  
> > +/* Build NAME(XXXX, 0x0) where 0x0 is encoded as a qword,
> > + * and return the offset to 0x0 for runtime patching.
> > + *
> > + * Warning: runtime patching is best avoided. Only use this as
> > + * a replacement for DataTableRegion (for guests that don't
> > + * support it).
> > + */
> > +int
> > +build_append_named_qword(GArray *array, const char *name_format, ...)
> > +{
> > +    int offset;
> > +    va_list ap;
> > +
> > +    va_start(ap, name_format);
> > +    build_append_namestringv(array, name_format, ap);
> > +    va_end(ap);
> > +
> > +    build_append_byte(array, 0x0E); /* QWordPrefix */
> > +
> > +    offset = array->len;
> > +    build_append_int_noprefix(array, 0x0, 8);
> > +    assert(array->len == offset + 8);
> > +
> > +    return offset;
> > +}
> > +
> >  static GPtrArray *alloc_list;
> >  
> >  static Aml *aml_alloc(void)
> > 
> >

Michael CC'd me on the grandparent of the email below. I'll try to add
my thoughts in a single go, with regard to OVMF.

On 12/30/15 20:52, Michael S. Tsirkin wrote:
> On Wed, Dec 30, 2015 at 04:55:54PM +0100, Igor Mammedov wrote:
>> On Mon, 28 Dec 2015 14:50:15 +0200
>> "Michael S. Tsirkin" <mst@redhat.com> wrote:
>>
>>> On Mon, Dec 28, 2015 at 10:39:04AM +0800, Xiao Guangrong wrote:
>>>>
>>>> Hi Michael, Paolo,
>>>>
>>>> Now it is the time to return to the challenge that how to reserve guest
>>>> physical region internally used by ACPI.
>>>>
>>>> Igor suggested that:
>>>> | An alternative place to allocate reserve from could be high memory.
>>>> | For pc we have "reserved-memory-end" which currently makes sure
>>>> | that hotpluggable memory range isn't used by firmware
>>>> (https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg00926.html)

OVMF has no support for the "reserved-memory-end" fw_cfg file. The
reason is that nobody wrote that patch, nor asked for the patch to be
written. (Not implying that just requesting the patch would be
sufficient for the patch to be written.)

>>> I don't want to tie things to reserved-memory-end because this
>>> does not scale: next time we need to reserve memory,
>>> we'll need to find yet another way to figure out what is where.
>> Could you elaborate a bit more on a problem you're seeing?
>>
>> To me it looks like it scales rather well.
>> For example lets imagine that we adding a device
>> that has some on device memory that should be mapped into GPA
>> code to do so would look like:
>>
>>   pc_machine_device_plug_cb(dev)
>>   {
>>    ...
>>    if (dev == OUR_NEW_DEVICE_TYPE) {
>>        memory_region_add_subregion(as, current_reserved_end, &dev->mr);
>>        set_new_reserved_end(current_reserved_end + memory_region_size(&dev->mr));
>>    }
>>   }
>>
>> we can practically add any number of new devices that way.
> 
> Yes but we'll have to build a host side allocator for these, and that's
> nasty. We'll also have to maintain these addresses indefinitely (at
> least per machine version) as they are guest visible.
> Not only that, there's no way for guest to know if we move things
> around, so basically we'll never be able to change addresses.
> 
> 
>>  
>>> I would like ./hw/acpi/bios-linker-loader.c interface to be extended to
>>> support 64 bit RAM instead

This looks quite doable in OVMF, as long as the blob to allocate from
high memory contains *zero* ACPI tables.

(
Namely, each ACPI table is installed from the containing fw_cfg blob
with EFI_ACPI_TABLE_PROTOCOL.InstallAcpiTable(), and the latter has its
own allocation policy for the *copies* of ACPI tables it installs.

This allocation policy is left unspecified in the section of the UEFI
spec that governs EFI_ACPI_TABLE_PROTOCOL.

The current policy in edk2 (= the reference implementation) seems to be
"allocate from under 4GB". It is currently being changed to "try to
allocate from under 4GB, and if that fails, retry from high memory". (It
is motivated by Aarch64 machines that may have no DRAM at all under 4GB.)
)

>>> (and maybe a way to allocate and
>>> zero-initialize buffer without loading it through fwcfg),

Sounds reasonable.

>>> this way bios
>>> does the allocation, and addresses can be patched into acpi.
>> and then guest side needs to parse/execute some AML that would
>> initialize QEMU side so it would know where to write data.
> 
> Well not really - we can put it in a data table, by itself
> so it's easy to find.

Do you mean acpi_tb_find_table(), acpi_get_table_by_index() /
acpi_get_table_with_size()?

> 
> AML is only needed if access from ACPI is desired.
> 
> 
>> bios-linker-loader is a great interface for initializing some
>> guest owned data and linking it together but I think it adds
>> unnecessary complexity and is misused if it's used to handle
>> device owned data/on device memory in this and VMGID cases.
> 
> I want a generic interface for guest to enumerate these things.  linker
> seems quite reasonable but if you see a reason why it won't do, or want
> to propose a better interface, fine.

* The guest could do the following:
- while processing the ALLOCATE commands, it would make a note where in
GPA space each fw_cfg blob gets allocated
- at the end the guest would prepare a temporary array with a predefined
record format, that associates each fw_cfg blob's name with the concrete
allocation address
- it would create an FWCfgDmaAccess stucture pointing at this array,
with a new "control" bit set (or something similar)
- the guest could write the address of the FWCfgDmaAccess struct to the
appropriate register, as always.

* Another idea would be a GET_ALLOCATION_ADDRESS linker/loader command,
specifying:
- the fw_cfg blob's name, for which to retrieve the guest-allocated
  address (this command could only follow the matching ALLOCATE
  command, never precede it)
- a flag whether the address should be written to IO or MMIO space
  (would be likely IO on x86, MMIO on ARM)
- a unique uint64_t key (could be the 16-bit fw_cfg selector value that
  identifies the blob, actually!)
- a uint64_t (IO or MMIO) address to write the unique key and then the
  allocation address to.

Either way, QEMU could learn about all the relevant guest-side
allocation addresses in a low number of traps. In addition, AML code
wouldn't have to reflect any allocation addresses to QEMU, ever.

> 
> PCI would do, too - though windows guys had concerns about
> returning PCI BARs from ACPI.
> 
> 
>> There was RFC on list to make BIOS boot from NVDIMM already
>> doing some ACPI table lookup/parsing. Now if they were forced
>> to also parse and execute AML to initialize QEMU with guest
>> allocated address that would complicate them quite a bit.
> 
> If they just need to find a table by name, it won't be
> too bad, would it?
> 
>> While with NVDIMM control memory region mapped directly by QEMU,
>> respective patches don't need in any way to initialize QEMU,
>> all they would need just read necessary data from control region.
>>
>> Also using bios-linker-loader takes away some usable RAM
>> from guest and in the end that doesn't scale,
>> the more devices I add the less usable RAM is left for guest OS
>> while all the device needs is a piece of GPA address space
>> that would belong to it.
> 
> I don't get this comment. I don't think it's MMIO that is wanted.
> If it's backed by qemu virtual memory then it's RAM.
> 
>>>
>>> See patch at the bottom that might be handy.

I've given up on Microsoft implementing DataTableRegion. (It's sad, really.)

From last year I have a WIP version of "docs/vmgenid.txt" that is based
on Michael's build_append_named_dword() function. If
GET_ALLOCATION_ADDRESS above looks good, then I could simplify the ACPI
stuff in that text file (and hopefully post it soon after for comments?)

>>>
>>>> he also innovated a way to use 64-bit address in DSDT/SSDT.rev = 1:
>>>> | when writing ASL one shall make sure that only XP supported
>>>> | features are in global scope, which is evaluated when tables
>>>> | are loaded and features of rev2 and higher are inside methods.
>>>> | That way XP doesn't crash as far as it doesn't evaluate unsupported
>>>> | opcode and one can guard those opcodes checking _REV object if neccesary.
>>>> (https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg01010.html)
>>>
>>> Yes, this technique works.

Agreed.

>>>
>>> An alternative is to add an XSDT, XP ignores that.
>>> XSDT at the moment breaks OVMF (because it loads both
>>> the RSDT and the XSDT, which is wrong), but I think
>>> Laszlo was working on a fix for that.

We have to distinguish two use cases here.

* The first is the case when QEMU prepares both an XSDT and an RSDT, and
links at least one common ACPI table from both. This would cause OVMF to
pass the same source (= to-be-copied) table to
EFI_ACPI_TABLE_PROTOCOL.InstallAcpiTable() twice, with one of the
following outcomes:

- there would be two instances of the same table (think e.g. SSDT)
- the second attempt would be rejected (e.g. FADT) and that error would
  terminate the linker-loader procedure.

This issue would not be too hard to overcome, with a simple "memoization
technique". After the initial loading & linking of the tables, OVMF
could remember the addresses of the "source" ACPI tables, and could
avoid passing already installed source tables to
EFI_ACPI_TABLE_PROTOCOL.InstallAcpiTable() for a second time.

* The second use case is when an ACPI table is linked *only* from QEMU's
XSDT. This is much harder to fix, because
EFI_ACPI_TABLE_PROTOCOL.InstallAcpiTable() in edk2 links the copy of the
passed-in table into *both* RSDT and XSDT, automatically. And, again,
the UEFI spec doesn't provide a way to control this from the caller
(i.e. from within OVMF).

I have tried earlier to effect a change in the specification of
EFI_ACPI_TABLE_PROTOCOL.InstallAcpiTable(), on the ASWG and USWG mailing
lists. (At that time I was trying to expose UEFI memory *type* to the
caller, from which the copy of the ACPI table being installed should be
allocated from.) Alas, I received no answers at all.

All in all I strongly recommend the "place rev2+ objects in method
scope" trick, over the "link it from the XSDT only" trick.

>> Using XSDT would increase ACPI tables occupied RAM
>> as it would duplicate DSDT + non XP supported AML
>> at global namespace.
> 
> Not at all - I posted patches linking to same
> tables from both RSDT and XSDT at some point.

Yes, at <http://thread.gmane.org/gmane.comp.emulators.qemu/342559>. This
could be made work in OVMF with the above mentioned memoization stuff.

> Only the list of pointers would be different.

I don't recommend that, see the second case above.

Thanks
Laszlo

>> So far we've managed keep DSDT compatible with XP while
>> introducing features from v2 and higher ACPI revisions as
>> AML that is only evaluated on demand.
>> We can continue doing so unless we have to unconditionally
>> add incompatible AML at global scope.
>>
> 
> Yes.
> 
>>>
>>>> Michael, Paolo, what do you think about these ideas?
>>>>
>>>> Thanks!
>>>
>>>
>>>
>>> So using a patch below, we can add Name(PQRS, 0x0) at the top of the
>>> SSDT (or bottom, or add a separate SSDT just for that).  It returns the
>>> current offset so we can add that to the linker.
>>>
>>> Won't work if you append the Name to the Aml structure (these can be
>>> nested to arbitrary depth using aml_append), so using plain GArray for
>>> this API makes sense to me.
>>>
>>> --->
>>>
>>> acpi: add build_append_named_dword, returning an offset in buffer
>>>
>>> This is a very limited form of support for runtime patching -
>>> similar in functionality to what we can do with ACPI_EXTRACT
>>> macros in python, but implemented in C.
>>>
>>> This is to allow ACPI code direct access to data tables -
>>> which is exactly what DataTableRegion is there for, except
>>> no known windows release so far implements DataTableRegion.
>> unsupported means Windows will BSOD, so it's practically
>> unusable unless MS will patch currently existing Windows
>> versions.
> 
> Yes. That's why my patch allows patching SSDT without using
> DataTableRegion.
> 
>> Another thing about DataTableRegion is that ACPI tables are
>> supposed to have static content which matches checksum in
>> table the header while you are trying to use it for dynamic
>> data. It would be cleaner/more compatible to teach
>> bios-linker-loader to just allocate memory and patch AML
>> with the allocated address.
> 
> Yes - if address is static, you need to put it outside
> the table. Can come right before or right after this.
> 
>> Also if OperationRegion() is used, then one has to patch
>> DefOpRegion directly as RegionOffset must be Integer,
>> using variable names is not permitted there.
> 
> I am not sure the comment was understood correctly.
> The comment says really "we can't use DataTableRegion
> so here is an alternative".
> 
>>
>>>
>>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>>>
>>> ---
>>>
>>> diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
>>> index 1b632dc..f8998ea 100644
>>> --- a/include/hw/acpi/aml-build.h
>>> +++ b/include/hw/acpi/aml-build.h
>>> @@ -286,4 +286,7 @@ void acpi_build_tables_cleanup(AcpiBuildTables *tables, bool mfre);
>>>  void
>>>  build_rsdt(GArray *table_data, GArray *linker, GArray *table_offsets);
>>>  
>>> +int
>>> +build_append_named_dword(GArray *array, const char *name_format, ...);
>>> +
>>>  #endif
>>> diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
>>> index 0d4b324..7f9fa65 100644
>>> --- a/hw/acpi/aml-build.c
>>> +++ b/hw/acpi/aml-build.c
>>> @@ -262,6 +262,32 @@ static void build_append_int(GArray *table, uint64_t value)
>>>      }
>>>  }
>>>  
>>> +/* Build NAME(XXXX, 0x0) where 0x0 is encoded as a qword,
>>> + * and return the offset to 0x0 for runtime patching.
>>> + *
>>> + * Warning: runtime patching is best avoided. Only use this as
>>> + * a replacement for DataTableRegion (for guests that don't
>>> + * support it).
>>> + */
>>> +int
>>> +build_append_named_qword(GArray *array, const char *name_format, ...)
>>> +{
>>> +    int offset;
>>> +    va_list ap;
>>> +
>>> +    va_start(ap, name_format);
>>> +    build_append_namestringv(array, name_format, ap);
>>> +    va_end(ap);
>>> +
>>> +    build_append_byte(array, 0x0E); /* QWordPrefix */
>>> +
>>> +    offset = array->len;
>>> +    build_append_int_noprefix(array, 0x0, 8);
>>> +    assert(array->len == offset + 8);
>>> +
>>> +    return offset;
>>> +}
>>> +
>>>  static GPtrArray *alloc_list;
>>>  
>>>  static Aml *aml_alloc(void)
>>>
>>>

On Wed, 30 Dec 2015 21:52:32 +0200
"Michael S. Tsirkin" <mst@redhat.com> wrote:

> On Wed, Dec 30, 2015 at 04:55:54PM +0100, Igor Mammedov wrote:
> > On Mon, 28 Dec 2015 14:50:15 +0200
> > "Michael S. Tsirkin" <mst@redhat.com> wrote:
> >   
> > > On Mon, Dec 28, 2015 at 10:39:04AM +0800, Xiao Guangrong wrote:  
> > > > 
> > > > Hi Michael, Paolo,
> > > > 
> > > > Now it is the time to return to the challenge that how to reserve guest
> > > > physical region internally used by ACPI.
> > > > 
> > > > Igor suggested that:
> > > > | An alternative place to allocate reserve from could be high memory.
> > > > | For pc we have "reserved-memory-end" which currently makes sure
> > > > | that hotpluggable memory range isn't used by firmware
> > > > (https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg00926.html)  
> > > 
> > > I don't want to tie things to reserved-memory-end because this
> > > does not scale: next time we need to reserve memory,
> > > we'll need to find yet another way to figure out what is where.  
> > Could you elaborate a bit more on a problem you're seeing?
> > 
> > To me it looks like it scales rather well.
> > For example lets imagine that we adding a device
> > that has some on device memory that should be mapped into GPA
> > code to do so would look like:
> > 
> >   pc_machine_device_plug_cb(dev)
> >   {
> >    ...
> >    if (dev == OUR_NEW_DEVICE_TYPE) {
> >        memory_region_add_subregion(as, current_reserved_end, &dev->mr);
> >        set_new_reserved_end(current_reserved_end + memory_region_size(&dev->mr));
> >    }
> >   }
> > 
> > we can practically add any number of new devices that way.  
> 
> Yes but we'll have to build a host side allocator for these, and that's
> nasty. We'll also have to maintain these addresses indefinitely (at
> least per machine version) as they are guest visible.
> Not only that, there's no way for guest to know if we move things
> around, so basically we'll never be able to change addresses.
simplistic GPA allocator in snippet above does the job,

if one unconditionally adds a device in new version then yes
code has to have compat code based on machine version.
But that applies to any device that gas a state to migrate
or to any address space layout change.

However device that directly maps addresses doesn't have to
have fixed address though, it could behave the same way as
PCI device with BARs, with only difference that its
MemoryRegions are mapped before guest is running vs
BARs mapped by BIOS.
It could be worth to create a generic base device class
that would do above. Then it could be inherited from and
extended by concrete device implementations.

> >    
> > > I would like ./hw/acpi/bios-linker-loader.c interface to be extended to
> > > support 64 bit RAM instead (and maybe a way to allocate and
> > > zero-initialize buffer without loading it through fwcfg), this way bios
> > > does the allocation, and addresses can be patched into acpi.  
> > and then guest side needs to parse/execute some AML that would
> > initialize QEMU side so it would know where to write data.  
> 
> Well not really - we can put it in a data table, by itself
> so it's easy to find.
> 
> AML is only needed if access from ACPI is desired.
in both cases (VMGEN, NVDIMM) access from ACPI is required
as minimum to write address back to QEMU and for NVDIM
to pass _DSM method data between guest and QEMU.

> 
> 
> > bios-linker-loader is a great interface for initializing some
> > guest owned data and linking it together but I think it adds
> > unnecessary complexity and is misused if it's used to handle
> > device owned data/on device memory in this and VMGID cases.  
> 
> I want a generic interface for guest to enumerate these things.  linker
> seems quite reasonable but if you see a reason why it won't do, or want
> to propose a better interface, fine.
> 
> PCI would do, too - though windows guys had concerns about
> returning PCI BARs from ACPI.
There were potential issues with pSeries bootloader that treated
PCI_CLASS_MEMORY_RAM as conventional RAM but it was fixed.
Could you point out to discussion about windows issues?

What VMGEN patches that used PCI for mapping purposes were
stuck at, was that it was suggested to use PCI_CLASS_MEMORY_RAM
class id but we couldn't agree on it.

VMGEN v13 with full discussion is here
https://patchwork.ozlabs.org/patch/443554/
So to continue with this route we would need to pick some other
driver less class id so windows won't prompt for driver or
maybe supply our own driver stub to guarantee that no one
would touch it. Any suggestions?

> 
> 
> > There was RFC on list to make BIOS boot from NVDIMM already
> > doing some ACPI table lookup/parsing. Now if they were forced
> > to also parse and execute AML to initialize QEMU with guest
> > allocated address that would complicate them quite a bit.  
> 
> If they just need to find a table by name, it won't be
> too bad, would it?
that's what they were doing scanning memory for static NVDIMM table.
However if it were DataTable, BIOS side would have to execute
AML so that the table address could be told to QEMU.

In case of direct mapping or PCI BAR there is no need to initialize
QEMU side from AML.
That also saves us IO port where this address should be written
if bios-linker-loader approach is used.

> 
> > While with NVDIMM control memory region mapped directly by QEMU,
> > respective patches don't need in any way to initialize QEMU,
> > all they would need just read necessary data from control region.
> > 
> > Also using bios-linker-loader takes away some usable RAM
> > from guest and in the end that doesn't scale,
> > the more devices I add the less usable RAM is left for guest OS
> > while all the device needs is a piece of GPA address space
> > that would belong to it.  
> 
> I don't get this comment. I don't think it's MMIO that is wanted.
> If it's backed by qemu virtual memory then it's RAM.
Then why don't allocate video card VRAM the same way and try to explain
user that a guest started with '-m 128 -device cirrus-vga,vgamem_mb=64Mb'
only has 64Mb of available RAM because of we think that on device VRAM
is also RAM.

Maybe I've used MMIO term wrongly here but it roughly reflects the idea
that on device memory (whether it's VRAM, NVDIMM control block or VMGEN
area) is not allocated from guest's usable RAM (as described in E820)
but rather directly mapped in guest's GPA and doesn't consume available
RAM as guest sees it. That's also the way it's done on real hardware.

What we need in case of VMGEN ID and NVDIMM is on device memory
that could be directly accessed by guest.
Both direct mapping or PCI BAR do that job and we could use simple
static AML without any patching.

> > > 
> > > See patch at the bottom that might be handy.
> > >   
> > > > he also innovated a way to use 64-bit address in DSDT/SSDT.rev = 1:
> > > > | when writing ASL one shall make sure that only XP supported
> > > > | features are in global scope, which is evaluated when tables
> > > > | are loaded and features of rev2 and higher are inside methods.
> > > > | That way XP doesn't crash as far as it doesn't evaluate unsupported
> > > > | opcode and one can guard those opcodes checking _REV object if neccesary.
> > > > (https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg01010.html)  
> > > 
> > > Yes, this technique works.
> > > 
> > > An alternative is to add an XSDT, XP ignores that.
> > > XSDT at the moment breaks OVMF (because it loads both
> > > the RSDT and the XSDT, which is wrong), but I think
> > > Laszlo was working on a fix for that.  
> > Using XSDT would increase ACPI tables occupied RAM
> > as it would duplicate DSDT + non XP supported AML
> > at global namespace.  
> 
> Not at all - I posted patches linking to same
> tables from both RSDT and XSDT at some point.
> Only the list of pointers would be different.
if you put XP incompatible AML in separate SSDT and link it
only from XSDT than that would work but if incompatibility
is in DSDT, one would have to provide compat DSDT for RSDT
an incompat DSDT for XSDT.

So far policy was don't try to run guest OS on QEMU
configuration that isn't supported by it.
For example we use VAR_PACKAGE when running with more
than 255 VCPUs (commit b4f4d5481) which BSODs XP.

So we can continue with that policy with out resorting to
using both RSDT and XSDT,
It would be even easier as all AML would be dynamically
generated and DSDT would only contain AML elements for
a concrete QEMU configuration.

> > So far we've managed keep DSDT compatible with XP while
> > introducing features from v2 and higher ACPI revisions as
> > AML that is only evaluated on demand.
> > We can continue doing so unless we have to unconditionally
> > add incompatible AML at global scope.
> >   
> 
> Yes.
> 
> > >   
> > > > Michael, Paolo, what do you think about these ideas?
> > > > 
> > > > Thanks!  
> > > 
> > > 
> > > 
> > > So using a patch below, we can add Name(PQRS, 0x0) at the top of the
> > > SSDT (or bottom, or add a separate SSDT just for that).  It returns the
> > > current offset so we can add that to the linker.
> > > 
> > > Won't work if you append the Name to the Aml structure (these can be
> > > nested to arbitrary depth using aml_append), so using plain GArray for
> > > this API makes sense to me.
> > >   
> > > --->  
> > > 
> > > acpi: add build_append_named_dword, returning an offset in buffer
> > > 
> > > This is a very limited form of support for runtime patching -
> > > similar in functionality to what we can do with ACPI_EXTRACT
> > > macros in python, but implemented in C.
> > > 
> > > This is to allow ACPI code direct access to data tables -
> > > which is exactly what DataTableRegion is there for, except
> > > no known windows release so far implements DataTableRegion.  
> > unsupported means Windows will BSOD, so it's practically
> > unusable unless MS will patch currently existing Windows
> > versions.  
> 
> Yes. That's why my patch allows patching SSDT without using
> DataTableRegion.
> 
> > Another thing about DataTableRegion is that ACPI tables are
> > supposed to have static content which matches checksum in
> > table the header while you are trying to use it for dynamic
> > data. It would be cleaner/more compatible to teach
> > bios-linker-loader to just allocate memory and patch AML
> > with the allocated address.  
> 
> Yes - if address is static, you need to put it outside
> the table. Can come right before or right after this.
> 
> > Also if OperationRegion() is used, then one has to patch
> > DefOpRegion directly as RegionOffset must be Integer,
> > using variable names is not permitted there.  
> 
> I am not sure the comment was understood correctly.
> The comment says really "we can't use DataTableRegion
> so here is an alternative".
so how are you going to access data at which patched
NameString point to?
for that you'd need a normal patched OperationRegion
as well since DataTableRegion isn't usable.

> 
> >   
> > > 
> > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > > 
> > > ---
> > > 
> > > diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
> > > index 1b632dc..f8998ea 100644
> > > --- a/include/hw/acpi/aml-build.h
> > > +++ b/include/hw/acpi/aml-build.h
> > > @@ -286,4 +286,7 @@ void acpi_build_tables_cleanup(AcpiBuildTables *tables, bool mfre);
> > >  void
> > >  build_rsdt(GArray *table_data, GArray *linker, GArray *table_offsets);
> > >  
> > > +int
> > > +build_append_named_dword(GArray *array, const char *name_format, ...);
> > > +
> > >  #endif
> > > diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
> > > index 0d4b324..7f9fa65 100644
> > > --- a/hw/acpi/aml-build.c
> > > +++ b/hw/acpi/aml-build.c
> > > @@ -262,6 +262,32 @@ static void build_append_int(GArray *table, uint64_t value)
> > >      }
> > >  }
> > >  
> > > +/* Build NAME(XXXX, 0x0) where 0x0 is encoded as a qword,
> > > + * and return the offset to 0x0 for runtime patching.
> > > + *
> > > + * Warning: runtime patching is best avoided. Only use this as
> > > + * a replacement for DataTableRegion (for guests that don't
> > > + * support it).
> > > + */
> > > +int
> > > +build_append_named_qword(GArray *array, const char *name_format, ...)
> > > +{
> > > +    int offset;
> > > +    va_list ap;
> > > +
> > > +    va_start(ap, name_format);
> > > +    build_append_namestringv(array, name_format, ap);
> > > +    va_end(ap);
> > > +
> > > +    build_append_byte(array, 0x0E); /* QWordPrefix */
> > > +
> > > +    offset = array->len;
> > > +    build_append_int_noprefix(array, 0x0, 8);
> > > +    assert(array->len == offset + 8);
> > > +
> > > +    return offset;
> > > +}
> > > +
> > >  static GPtrArray *alloc_list;
> > >  
> > >  static Aml *aml_alloc(void)
> > > 
> > >   
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Tue, Jan 05, 2016 at 05:30:25PM +0100, Igor Mammedov wrote:
> > > bios-linker-loader is a great interface for initializing some
> > > guest owned data and linking it together but I think it adds
> > > unnecessary complexity and is misused if it's used to handle
> > > device owned data/on device memory in this and VMGID cases.  
> > 
> > I want a generic interface for guest to enumerate these things.  linker
> > seems quite reasonable but if you see a reason why it won't do, or want
> > to propose a better interface, fine.
> > 
> > PCI would do, too - though windows guys had concerns about
> > returning PCI BARs from ACPI.
> There were potential issues with pSeries bootloader that treated
> PCI_CLASS_MEMORY_RAM as conventional RAM but it was fixed.
> Could you point out to discussion about windows issues?
> 
> What VMGEN patches that used PCI for mapping purposes were
> stuck at, was that it was suggested to use PCI_CLASS_MEMORY_RAM
> class id but we couldn't agree on it.
> 
> VMGEN v13 with full discussion is here
> https://patchwork.ozlabs.org/patch/443554/
> So to continue with this route we would need to pick some other
> driver less class id so windows won't prompt for driver or
> maybe supply our own driver stub to guarantee that no one
> would touch it. Any suggestions?

Pick any device/vendor id pair for which windows specifies no driver.
There's a small risk that this will conflict with some
guest but I think it's minimal.


> > 
> > 
> > > There was RFC on list to make BIOS boot from NVDIMM already
> > > doing some ACPI table lookup/parsing. Now if they were forced
> > > to also parse and execute AML to initialize QEMU with guest
> > > allocated address that would complicate them quite a bit.  
> > 
> > If they just need to find a table by name, it won't be
> > too bad, would it?
> that's what they were doing scanning memory for static NVDIMM table.
> However if it were DataTable, BIOS side would have to execute
> AML so that the table address could be told to QEMU.

Not at all. You can find any table by its signature without
parsing AML.


> In case of direct mapping or PCI BAR there is no need to initialize
> QEMU side from AML.
> That also saves us IO port where this address should be written
> if bios-linker-loader approach is used.
> 
> > 
> > > While with NVDIMM control memory region mapped directly by QEMU,
> > > respective patches don't need in any way to initialize QEMU,
> > > all they would need just read necessary data from control region.
> > > 
> > > Also using bios-linker-loader takes away some usable RAM
> > > from guest and in the end that doesn't scale,
> > > the more devices I add the less usable RAM is left for guest OS
> > > while all the device needs is a piece of GPA address space
> > > that would belong to it.  
> > 
> > I don't get this comment. I don't think it's MMIO that is wanted.
> > If it's backed by qemu virtual memory then it's RAM.
> Then why don't allocate video card VRAM the same way and try to explain
> user that a guest started with '-m 128 -device cirrus-vga,vgamem_mb=64Mb'
> only has 64Mb of available RAM because of we think that on device VRAM
> is also RAM.
> 
> Maybe I've used MMIO term wrongly here but it roughly reflects the idea
> that on device memory (whether it's VRAM, NVDIMM control block or VMGEN
> area) is not allocated from guest's usable RAM (as described in E820)
> but rather directly mapped in guest's GPA and doesn't consume available
> RAM as guest sees it. That's also the way it's done on real hardware.
> 
> What we need in case of VMGEN ID and NVDIMM is on device memory
> that could be directly accessed by guest.
> Both direct mapping or PCI BAR do that job and we could use simple
> static AML without any patching.

At least with VMGEN the issue is that there's an AML method
that returns the physical address.
Then if guest OS moves the BAR (which is legal), it will break
since caller has no way to know it's related to the BAR.


> > > > 
> > > > See patch at the bottom that might be handy.
> > > >   
> > > > > he also innovated a way to use 64-bit address in DSDT/SSDT.rev = 1:
> > > > > | when writing ASL one shall make sure that only XP supported
> > > > > | features are in global scope, which is evaluated when tables
> > > > > | are loaded and features of rev2 and higher are inside methods.
> > > > > | That way XP doesn't crash as far as it doesn't evaluate unsupported
> > > > > | opcode and one can guard those opcodes checking _REV object if neccesary.
> > > > > (https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg01010.html)  
> > > > 
> > > > Yes, this technique works.
> > > > 
> > > > An alternative is to add an XSDT, XP ignores that.
> > > > XSDT at the moment breaks OVMF (because it loads both
> > > > the RSDT and the XSDT, which is wrong), but I think
> > > > Laszlo was working on a fix for that.  
> > > Using XSDT would increase ACPI tables occupied RAM
> > > as it would duplicate DSDT + non XP supported AML
> > > at global namespace.  
> > 
> > Not at all - I posted patches linking to same
> > tables from both RSDT and XSDT at some point.
> > Only the list of pointers would be different.
> if you put XP incompatible AML in separate SSDT and link it
> only from XSDT than that would work but if incompatibility
> is in DSDT, one would have to provide compat DSDT for RSDT
> an incompat DSDT for XSDT.

So don't do this.

> So far policy was don't try to run guest OS on QEMU
> configuration that isn't supported by it.

It's better if guests don't see some features but
don't crash. It's not always possible of course but
we should try to avoid this.

> For example we use VAR_PACKAGE when running with more
> than 255 VCPUs (commit b4f4d5481) which BSODs XP.

Yes. And it's because we violate the spec, DSDT
should not have this stuff.

> So we can continue with that policy with out resorting to
> using both RSDT and XSDT,
> It would be even easier as all AML would be dynamically
> generated and DSDT would only contain AML elements for
> a concrete QEMU configuration.

I'd prefer XSDT but I won't nack it if you do it in DSDT.
I think it's not spec compliant but guests do not
seem to care.

> > > So far we've managed keep DSDT compatible with XP while
> > > introducing features from v2 and higher ACPI revisions as
> > > AML that is only evaluated on demand.
> > > We can continue doing so unless we have to unconditionally
> > > add incompatible AML at global scope.
> > >   
> > 
> > Yes.
> > 
> > > >   
> > > > > Michael, Paolo, what do you think about these ideas?
> > > > > 
> > > > > Thanks!  
> > > > 
> > > > 
> > > > 
> > > > So using a patch below, we can add Name(PQRS, 0x0) at the top of the
> > > > SSDT (or bottom, or add a separate SSDT just for that).  It returns the
> > > > current offset so we can add that to the linker.
> > > > 
> > > > Won't work if you append the Name to the Aml structure (these can be
> > > > nested to arbitrary depth using aml_append), so using plain GArray for
> > > > this API makes sense to me.
> > > >   
> > > > --->  
> > > > 
> > > > acpi: add build_append_named_dword, returning an offset in buffer
> > > > 
> > > > This is a very limited form of support for runtime patching -
> > > > similar in functionality to what we can do with ACPI_EXTRACT
> > > > macros in python, but implemented in C.
> > > > 
> > > > This is to allow ACPI code direct access to data tables -
> > > > which is exactly what DataTableRegion is there for, except
> > > > no known windows release so far implements DataTableRegion.  
> > > unsupported means Windows will BSOD, so it's practically
> > > unusable unless MS will patch currently existing Windows
> > > versions.  
> > 
> > Yes. That's why my patch allows patching SSDT without using
> > DataTableRegion.
> > 
> > > Another thing about DataTableRegion is that ACPI tables are
> > > supposed to have static content which matches checksum in
> > > table the header while you are trying to use it for dynamic
> > > data. It would be cleaner/more compatible to teach
> > > bios-linker-loader to just allocate memory and patch AML
> > > with the allocated address.  
> > 
> > Yes - if address is static, you need to put it outside
> > the table. Can come right before or right after this.
> > 
> > > Also if OperationRegion() is used, then one has to patch
> > > DefOpRegion directly as RegionOffset must be Integer,
> > > using variable names is not permitted there.  
> > 
> > I am not sure the comment was understood correctly.
> > The comment says really "we can't use DataTableRegion
> > so here is an alternative".
> so how are you going to access data at which patched
> NameString point to?
> for that you'd need a normal patched OperationRegion
> as well since DataTableRegion isn't usable.

For VMGENID you would patch the method that
returns the address - you do not need an op region
as you never access it.

I don't know about NVDIMM. Maybe OperationRegion can
use the patched NameString? Will need some thought.

> > 
> > >   
> > > > 
> > > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > > > 
> > > > ---
> > > > 
> > > > diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
> > > > index 1b632dc..f8998ea 100644
> > > > --- a/include/hw/acpi/aml-build.h
> > > > +++ b/include/hw/acpi/aml-build.h
> > > > @@ -286,4 +286,7 @@ void acpi_build_tables_cleanup(AcpiBuildTables *tables, bool mfre);
> > > >  void
> > > >  build_rsdt(GArray *table_data, GArray *linker, GArray *table_offsets);
> > > >  
> > > > +int
> > > > +build_append_named_dword(GArray *array, const char *name_format, ...);
> > > > +
> > > >  #endif
> > > > diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
> > > > index 0d4b324..7f9fa65 100644
> > > > --- a/hw/acpi/aml-build.c
> > > > +++ b/hw/acpi/aml-build.c
> > > > @@ -262,6 +262,32 @@ static void build_append_int(GArray *table, uint64_t value)
> > > >      }
> > > >  }
> > > >  
> > > > +/* Build NAME(XXXX, 0x0) where 0x0 is encoded as a qword,
> > > > + * and return the offset to 0x0 for runtime patching.
> > > > + *
> > > > + * Warning: runtime patching is best avoided. Only use this as
> > > > + * a replacement for DataTableRegion (for guests that don't
> > > > + * support it).
> > > > + */
> > > > +int
> > > > +build_append_named_qword(GArray *array, const char *name_format, ...)
> > > > +{
> > > > +    int offset;
> > > > +    va_list ap;
> > > > +
> > > > +    va_start(ap, name_format);
> > > > +    build_append_namestringv(array, name_format, ap);
> > > > +    va_end(ap);
> > > > +
> > > > +    build_append_byte(array, 0x0E); /* QWordPrefix */
> > > > +
> > > > +    offset = array->len;
> > > > +    build_append_int_noprefix(array, 0x0, 8);
> > > > +    assert(array->len == offset + 8);
> > > > +
> > > > +    return offset;
> > > > +}
> > > > +
> > > >  static GPtrArray *alloc_list;
> > > >  
> > > >  static Aml *aml_alloc(void)
> > > > 
> > > >   
> > --
> > To unsubscribe from this list: send the line "unsubscribe kvm" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html

On 01/05/16 17:43, Michael S. Tsirkin wrote:
> On Tue, Jan 05, 2016 at 05:30:25PM +0100, Igor Mammedov wrote:
>>>> bios-linker-loader is a great interface for initializing some
>>>> guest owned data and linking it together but I think it adds
>>>> unnecessary complexity and is misused if it's used to handle
>>>> device owned data/on device memory in this and VMGID cases.  
>>>
>>> I want a generic interface for guest to enumerate these things.  linker
>>> seems quite reasonable but if you see a reason why it won't do, or want
>>> to propose a better interface, fine.
>>>
>>> PCI would do, too - though windows guys had concerns about
>>> returning PCI BARs from ACPI.
>> There were potential issues with pSeries bootloader that treated
>> PCI_CLASS_MEMORY_RAM as conventional RAM but it was fixed.
>> Could you point out to discussion about windows issues?
>>
>> What VMGEN patches that used PCI for mapping purposes were
>> stuck at, was that it was suggested to use PCI_CLASS_MEMORY_RAM
>> class id but we couldn't agree on it.
>>
>> VMGEN v13 with full discussion is here
>> https://patchwork.ozlabs.org/patch/443554/
>> So to continue with this route we would need to pick some other
>> driver less class id so windows won't prompt for driver or
>> maybe supply our own driver stub to guarantee that no one
>> would touch it. Any suggestions?
> 
> Pick any device/vendor id pair for which windows specifies no driver.
> There's a small risk that this will conflict with some
> guest but I think it's minimal.
> 
> 
>>>
>>>
>>>> There was RFC on list to make BIOS boot from NVDIMM already
>>>> doing some ACPI table lookup/parsing. Now if they were forced
>>>> to also parse and execute AML to initialize QEMU with guest
>>>> allocated address that would complicate them quite a bit.  
>>>
>>> If they just need to find a table by name, it won't be
>>> too bad, would it?
>> that's what they were doing scanning memory for static NVDIMM table.
>> However if it were DataTable, BIOS side would have to execute
>> AML so that the table address could be told to QEMU.
> 
> Not at all. You can find any table by its signature without
> parsing AML.
> 
> 
>> In case of direct mapping or PCI BAR there is no need to initialize
>> QEMU side from AML.
>> That also saves us IO port where this address should be written
>> if bios-linker-loader approach is used.
>>
>>>
>>>> While with NVDIMM control memory region mapped directly by QEMU,
>>>> respective patches don't need in any way to initialize QEMU,
>>>> all they would need just read necessary data from control region.
>>>>
>>>> Also using bios-linker-loader takes away some usable RAM
>>>> from guest and in the end that doesn't scale,
>>>> the more devices I add the less usable RAM is left for guest OS
>>>> while all the device needs is a piece of GPA address space
>>>> that would belong to it.  
>>>
>>> I don't get this comment. I don't think it's MMIO that is wanted.
>>> If it's backed by qemu virtual memory then it's RAM.
>> Then why don't allocate video card VRAM the same way and try to explain
>> user that a guest started with '-m 128 -device cirrus-vga,vgamem_mb=64Mb'
>> only has 64Mb of available RAM because of we think that on device VRAM
>> is also RAM.
>>
>> Maybe I've used MMIO term wrongly here but it roughly reflects the idea
>> that on device memory (whether it's VRAM, NVDIMM control block or VMGEN
>> area) is not allocated from guest's usable RAM (as described in E820)
>> but rather directly mapped in guest's GPA and doesn't consume available
>> RAM as guest sees it. That's also the way it's done on real hardware.
>>
>> What we need in case of VMGEN ID and NVDIMM is on device memory
>> that could be directly accessed by guest.
>> Both direct mapping or PCI BAR do that job and we could use simple
>> static AML without any patching.
> 
> At least with VMGEN the issue is that there's an AML method
> that returns the physical address.
> Then if guest OS moves the BAR (which is legal), it will break
> since caller has no way to know it's related to the BAR.
> 
> 
>>>>>
>>>>> See patch at the bottom that might be handy.
>>>>>   
>>>>>> he also innovated a way to use 64-bit address in DSDT/SSDT.rev = 1:
>>>>>> | when writing ASL one shall make sure that only XP supported
>>>>>> | features are in global scope, which is evaluated when tables
>>>>>> | are loaded and features of rev2 and higher are inside methods.
>>>>>> | That way XP doesn't crash as far as it doesn't evaluate unsupported
>>>>>> | opcode and one can guard those opcodes checking _REV object if neccesary.
>>>>>> (https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg01010.html)  
>>>>>
>>>>> Yes, this technique works.
>>>>>
>>>>> An alternative is to add an XSDT, XP ignores that.
>>>>> XSDT at the moment breaks OVMF (because it loads both
>>>>> the RSDT and the XSDT, which is wrong), but I think
>>>>> Laszlo was working on a fix for that.  
>>>> Using XSDT would increase ACPI tables occupied RAM
>>>> as it would duplicate DSDT + non XP supported AML
>>>> at global namespace.  
>>>
>>> Not at all - I posted patches linking to same
>>> tables from both RSDT and XSDT at some point.
>>> Only the list of pointers would be different.
>> if you put XP incompatible AML in separate SSDT and link it
>> only from XSDT than that would work but if incompatibility
>> is in DSDT, one would have to provide compat DSDT for RSDT
>> an incompat DSDT for XSDT.
> 
> So don't do this.
> 
>> So far policy was don't try to run guest OS on QEMU
>> configuration that isn't supported by it.
> 
> It's better if guests don't see some features but
> don't crash. It's not always possible of course but
> we should try to avoid this.
> 
>> For example we use VAR_PACKAGE when running with more
>> than 255 VCPUs (commit b4f4d5481) which BSODs XP.
> 
> Yes. And it's because we violate the spec, DSDT
> should not have this stuff.
> 
>> So we can continue with that policy with out resorting to
>> using both RSDT and XSDT,
>> It would be even easier as all AML would be dynamically
>> generated and DSDT would only contain AML elements for
>> a concrete QEMU configuration.
> 
> I'd prefer XSDT but I won't nack it if you do it in DSDT.
> I think it's not spec compliant but guests do not
> seem to care.
> 
>>>> So far we've managed keep DSDT compatible with XP while
>>>> introducing features from v2 and higher ACPI revisions as
>>>> AML that is only evaluated on demand.
>>>> We can continue doing so unless we have to unconditionally
>>>> add incompatible AML at global scope.
>>>>   
>>>
>>> Yes.
>>>
>>>>>   
>>>>>> Michael, Paolo, what do you think about these ideas?
>>>>>>
>>>>>> Thanks!  
>>>>>
>>>>>
>>>>>
>>>>> So using a patch below, we can add Name(PQRS, 0x0) at the top of the
>>>>> SSDT (or bottom, or add a separate SSDT just for that).  It returns the
>>>>> current offset so we can add that to the linker.
>>>>>
>>>>> Won't work if you append the Name to the Aml structure (these can be
>>>>> nested to arbitrary depth using aml_append), so using plain GArray for
>>>>> this API makes sense to me.
>>>>>   
>>>>> --->  
>>>>>
>>>>> acpi: add build_append_named_dword, returning an offset in buffer
>>>>>
>>>>> This is a very limited form of support for runtime patching -
>>>>> similar in functionality to what we can do with ACPI_EXTRACT
>>>>> macros in python, but implemented in C.
>>>>>
>>>>> This is to allow ACPI code direct access to data tables -
>>>>> which is exactly what DataTableRegion is there for, except
>>>>> no known windows release so far implements DataTableRegion.  
>>>> unsupported means Windows will BSOD, so it's practically
>>>> unusable unless MS will patch currently existing Windows
>>>> versions.  
>>>
>>> Yes. That's why my patch allows patching SSDT without using
>>> DataTableRegion.
>>>
>>>> Another thing about DataTableRegion is that ACPI tables are
>>>> supposed to have static content which matches checksum in
>>>> table the header while you are trying to use it for dynamic
>>>> data. It would be cleaner/more compatible to teach
>>>> bios-linker-loader to just allocate memory and patch AML
>>>> with the allocated address.  
>>>
>>> Yes - if address is static, you need to put it outside
>>> the table. Can come right before or right after this.
>>>
>>>> Also if OperationRegion() is used, then one has to patch
>>>> DefOpRegion directly as RegionOffset must be Integer,
>>>> using variable names is not permitted there.  
>>>
>>> I am not sure the comment was understood correctly.
>>> The comment says really "we can't use DataTableRegion
>>> so here is an alternative".
>> so how are you going to access data at which patched
>> NameString point to?
>> for that you'd need a normal patched OperationRegion
>> as well since DataTableRegion isn't usable.
> 
> For VMGENID you would patch the method that
> returns the address - you do not need an op region
> as you never access it.
> 
> I don't know about NVDIMM. Maybe OperationRegion can
> use the patched NameString? Will need some thought.

Xiao Guangrong has patches on the list that already solve this.

  [Qemu-devel] [PATCH 0/6] NVDIMM ACPI: introduce the framework of QEMU
                           emulated DSM

  http://thread.gmane.org/gmane.comp.emulators.kvm.devel/145138

I very briefly skimmed that series.

(Side note: I sort of dislike that with the approach seen in that
series, nvdimm and vmgenid would *both* have to have their own ioports
for telling QEMU about the guest-allocated address. See the rough
GET_ALLOCATION_ADDRESS idea in my earlier post in this thread for one
way to generalize this.)

In any case, in order to stay on topic, AFAICS in patch 3/6, Xiao
Guangrong creates a method called "MEMA". That method consists of a
single return statement that returns a 64-bit integer constant. This
returned constant is patched by the linker/loader.

Then in patch 5/6, there seems to be another method (named "NCAL"?) that
calls MEMA, then uses MEMA's return value to dynamically create the NRAM
operation region, apparently scoped to the NCAL method.

This is possible because the <RegionOffset> symbol (from the expansion
of <DefOpRegion>) is "TermArg => Integer". Patch 4/6 modifies
aml_operation_region() so that it exposes this capability.

... Again, this is just from a superficial skimming; it would have
helped quite a bit if Xiao Guangrong had appended a decompiled ACPI dump
to the 0/6 blurb (or even a documentation file).

Thanks
Laszlo

> 
>>>
>>>>   
>>>>>
>>>>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>>>>>
>>>>> ---
>>>>>
>>>>> diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
>>>>> index 1b632dc..f8998ea 100644
>>>>> --- a/include/hw/acpi/aml-build.h
>>>>> +++ b/include/hw/acpi/aml-build.h
>>>>> @@ -286,4 +286,7 @@ void acpi_build_tables_cleanup(AcpiBuildTables *tables, bool mfre);
>>>>>  void
>>>>>  build_rsdt(GArray *table_data, GArray *linker, GArray *table_offsets);
>>>>>  
>>>>> +int
>>>>> +build_append_named_dword(GArray *array, const char *name_format, ...);
>>>>> +
>>>>>  #endif
>>>>> diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
>>>>> index 0d4b324..7f9fa65 100644
>>>>> --- a/hw/acpi/aml-build.c
>>>>> +++ b/hw/acpi/aml-build.c
>>>>> @@ -262,6 +262,32 @@ static void build_append_int(GArray *table, uint64_t value)
>>>>>      }
>>>>>  }
>>>>>  
>>>>> +/* Build NAME(XXXX, 0x0) where 0x0 is encoded as a qword,
>>>>> + * and return the offset to 0x0 for runtime patching.
>>>>> + *
>>>>> + * Warning: runtime patching is best avoided. Only use this as
>>>>> + * a replacement for DataTableRegion (for guests that don't
>>>>> + * support it).
>>>>> + */
>>>>> +int
>>>>> +build_append_named_qword(GArray *array, const char *name_format, ...)
>>>>> +{
>>>>> +    int offset;
>>>>> +    va_list ap;
>>>>> +
>>>>> +    va_start(ap, name_format);
>>>>> +    build_append_namestringv(array, name_format, ap);
>>>>> +    va_end(ap);
>>>>> +
>>>>> +    build_append_byte(array, 0x0E); /* QWordPrefix */
>>>>> +
>>>>> +    offset = array->len;
>>>>> +    build_append_int_noprefix(array, 0x0, 8);
>>>>> +    assert(array->len == offset + 8);
>>>>> +
>>>>> +    return offset;
>>>>> +}
>>>>> +
>>>>>  static GPtrArray *alloc_list;
>>>>>  
>>>>>  static Aml *aml_alloc(void)
>>>>>
>>>>>   
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

On 01/06/2016 12:43 AM, Michael S. Tsirkin wrote:

>>> Yes - if address is static, you need to put it outside
>>> the table. Can come right before or right after this.
>>>
>>>> Also if OperationRegion() is used, then one has to patch
>>>> DefOpRegion directly as RegionOffset must be Integer,
>>>> using variable names is not permitted there.
>>>
>>> I am not sure the comment was understood correctly.
>>> The comment says really "we can't use DataTableRegion
>>> so here is an alternative".
>> so how are you going to access data at which patched
>> NameString point to?
>> for that you'd need a normal patched OperationRegion
>> as well since DataTableRegion isn't usable.
>
> For VMGENID you would patch the method that
> returns the address - you do not need an op region
> as you never access it.
>
> I don't know about NVDIMM. Maybe OperationRegion can
> use the patched NameString? Will need some thought.

The ACPI spec says that the offsetTerm in OperationRegion
is evaluated as Int, so the named object is allowed to be
used in OperationRegion, that is exact what my patchset
is doing (http://marc.info/?l=kvm&m=145193395624537&w=2):

+    dsm_mem = aml_arg(3);
+    aml_append(method, aml_store(aml_call0(NVDIMM_GET_DSM_MEM), dsm_mem));

+    aml_append(method, aml_operation_region("NRAM", AML_SYSTEM_MEMORY,
+                                            dsm_mem, TARGET_PAGE_SIZE));

We hide the int64 object which is patched by BIOS in the method,
NVDIMM_GET_DSM_MEM, to make windows XP happy.

However, the disadvantages i see are:
a) as Igor pointed out, we need a way to tell QEMU what is the patched
    address, in NVDIMM ACPI, we used a 64 bit IO ports to pass the address
    to QEMU.

b) BIOS allocated memory is RAM based so it stops us to use MMIO in ACPI,
    MMIO is the more scalable resource than IO port as it has larger region
    and supports 64 bits operation.

On Mon, 4 Jan 2016 21:17:31 +0100
Laszlo Ersek <lersek@redhat.com> wrote:

> Michael CC'd me on the grandparent of the email below. I'll try to add
> my thoughts in a single go, with regard to OVMF.
> 
> On 12/30/15 20:52, Michael S. Tsirkin wrote:
> > On Wed, Dec 30, 2015 at 04:55:54PM +0100, Igor Mammedov wrote:  
> >> On Mon, 28 Dec 2015 14:50:15 +0200
> >> "Michael S. Tsirkin" <mst@redhat.com> wrote:
> >>  
> >>> On Mon, Dec 28, 2015 at 10:39:04AM +0800, Xiao Guangrong wrote:  
> >>>>
> >>>> Hi Michael, Paolo,
> >>>>
> >>>> Now it is the time to return to the challenge that how to reserve guest
> >>>> physical region internally used by ACPI.
> >>>>
> >>>> Igor suggested that:
> >>>> | An alternative place to allocate reserve from could be high memory.
> >>>> | For pc we have "reserved-memory-end" which currently makes sure
> >>>> | that hotpluggable memory range isn't used by firmware
> >>>> (https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg00926.html)  
> 
> OVMF has no support for the "reserved-memory-end" fw_cfg file. The
> reason is that nobody wrote that patch, nor asked for the patch to be
> written. (Not implying that just requesting the patch would be
> sufficient for the patch to be written.)
> 
> >>> I don't want to tie things to reserved-memory-end because this
> >>> does not scale: next time we need to reserve memory,
> >>> we'll need to find yet another way to figure out what is where.  
> >> Could you elaborate a bit more on a problem you're seeing?
> >>
> >> To me it looks like it scales rather well.
> >> For example lets imagine that we adding a device
> >> that has some on device memory that should be mapped into GPA
> >> code to do so would look like:
> >>
> >>   pc_machine_device_plug_cb(dev)
> >>   {
> >>    ...
> >>    if (dev == OUR_NEW_DEVICE_TYPE) {
> >>        memory_region_add_subregion(as, current_reserved_end, &dev->mr);
> >>        set_new_reserved_end(current_reserved_end + memory_region_size(&dev->mr));
> >>    }
> >>   }
> >>
> >> we can practically add any number of new devices that way.  
> > 
> > Yes but we'll have to build a host side allocator for these, and that's
> > nasty. We'll also have to maintain these addresses indefinitely (at
> > least per machine version) as they are guest visible.
> > Not only that, there's no way for guest to know if we move things
> > around, so basically we'll never be able to change addresses.
> > 
> >   
> >>    
> >>> I would like ./hw/acpi/bios-linker-loader.c interface to be extended to
> >>> support 64 bit RAM instead  
> 
> This looks quite doable in OVMF, as long as the blob to allocate from
> high memory contains *zero* ACPI tables.
> 
> (
> Namely, each ACPI table is installed from the containing fw_cfg blob
> with EFI_ACPI_TABLE_PROTOCOL.InstallAcpiTable(), and the latter has its
> own allocation policy for the *copies* of ACPI tables it installs.
> 
> This allocation policy is left unspecified in the section of the UEFI
> spec that governs EFI_ACPI_TABLE_PROTOCOL.
> 
> The current policy in edk2 (= the reference implementation) seems to be
> "allocate from under 4GB". It is currently being changed to "try to
> allocate from under 4GB, and if that fails, retry from high memory". (It
> is motivated by Aarch64 machines that may have no DRAM at all under 4GB.)
> )
> 
> >>> (and maybe a way to allocate and
> >>> zero-initialize buffer without loading it through fwcfg),  
> 
> Sounds reasonable.
> 
> >>> this way bios
> >>> does the allocation, and addresses can be patched into acpi.  
> >> and then guest side needs to parse/execute some AML that would
> >> initialize QEMU side so it would know where to write data.  
> > 
> > Well not really - we can put it in a data table, by itself
> > so it's easy to find.  
> 
> Do you mean acpi_tb_find_table(), acpi_get_table_by_index() /
> acpi_get_table_with_size()?
> 
> > 
> > AML is only needed if access from ACPI is desired.
> > 
> >   
> >> bios-linker-loader is a great interface for initializing some
> >> guest owned data and linking it together but I think it adds
> >> unnecessary complexity and is misused if it's used to handle
> >> device owned data/on device memory in this and VMGID cases.  
> > 
> > I want a generic interface for guest to enumerate these things.  linker
> > seems quite reasonable but if you see a reason why it won't do, or want
> > to propose a better interface, fine.  
> 
> * The guest could do the following:
> - while processing the ALLOCATE commands, it would make a note where in
> GPA space each fw_cfg blob gets allocated
> - at the end the guest would prepare a temporary array with a predefined
> record format, that associates each fw_cfg blob's name with the concrete
> allocation address
> - it would create an FWCfgDmaAccess stucture pointing at this array,
> with a new "control" bit set (or something similar)
> - the guest could write the address of the FWCfgDmaAccess struct to the
> appropriate register, as always.
> 
> * Another idea would be a GET_ALLOCATION_ADDRESS linker/loader command,
> specifying:
> - the fw_cfg blob's name, for which to retrieve the guest-allocated
>   address (this command could only follow the matching ALLOCATE
>   command, never precede it)
> - a flag whether the address should be written to IO or MMIO space
>   (would be likely IO on x86, MMIO on ARM)
> - a unique uint64_t key (could be the 16-bit fw_cfg selector value that
>   identifies the blob, actually!)
> - a uint64_t (IO or MMIO) address to write the unique key and then the
>   allocation address to.
> 
> Either way, QEMU could learn about all the relevant guest-side
> allocation addresses in a low number of traps. In addition, AML code
> wouldn't have to reflect any allocation addresses to QEMU, ever.
That would be nice trick. I see 2 issues here:
 1. ACPI tables blob is build atomically when one guest tries to read it
    from fw_cfg so patched addresses have to be communicated
    to QEMU before that.
 2. Mo important I think that we are miss-using linker-loader
    interface here, trying to from allocate buffer in guest RAM
    an so consuming it while all we need a window into device
    memory mapped somewhere outside of RAM occupied  address space.

> 
> > 
> > PCI would do, too - though windows guys had concerns about
> > returning PCI BARs from ACPI.
> > 
> >   
> >> There was RFC on list to make BIOS boot from NVDIMM already
> >> doing some ACPI table lookup/parsing. Now if they were forced
> >> to also parse and execute AML to initialize QEMU with guest
> >> allocated address that would complicate them quite a bit.  
> > 
> > If they just need to find a table by name, it won't be
> > too bad, would it?
> >   
> >> While with NVDIMM control memory region mapped directly by QEMU,
> >> respective patches don't need in any way to initialize QEMU,
> >> all they would need just read necessary data from control region.
> >>
> >> Also using bios-linker-loader takes away some usable RAM
> >> from guest and in the end that doesn't scale,
> >> the more devices I add the less usable RAM is left for guest OS
> >> while all the device needs is a piece of GPA address space
> >> that would belong to it.  
> > 
> > I don't get this comment. I don't think it's MMIO that is wanted.
> > If it's backed by qemu virtual memory then it's RAM.
> >   
> >>>
> >>> See patch at the bottom that might be handy.  
> 
> I've given up on Microsoft implementing DataTableRegion. (It's sad, really.)
> 
> From last year I have a WIP version of "docs/vmgenid.txt" that is based
> on Michael's build_append_named_dword() function. If
> GET_ALLOCATION_ADDRESS above looks good, then I could simplify the ACPI
> stuff in that text file (and hopefully post it soon after for comments?)
> 
> >>>  
> >>>> he also innovated a way to use 64-bit address in DSDT/SSDT.rev = 1:
> >>>> | when writing ASL one shall make sure that only XP supported
> >>>> | features are in global scope, which is evaluated when tables
> >>>> | are loaded and features of rev2 and higher are inside methods.
> >>>> | That way XP doesn't crash as far as it doesn't evaluate unsupported
> >>>> | opcode and one can guard those opcodes checking _REV object if neccesary.
> >>>> (https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg01010.html)  
> >>>
> >>> Yes, this technique works.  
> 
> Agreed.
> 
> >>>
> >>> An alternative is to add an XSDT, XP ignores that.
> >>> XSDT at the moment breaks OVMF (because it loads both
> >>> the RSDT and the XSDT, which is wrong), but I think
> >>> Laszlo was working on a fix for that.  
> 
> We have to distinguish two use cases here.
> 
> * The first is the case when QEMU prepares both an XSDT and an RSDT, and
> links at least one common ACPI table from both. This would cause OVMF to
> pass the same source (= to-be-copied) table to
> EFI_ACPI_TABLE_PROTOCOL.InstallAcpiTable() twice, with one of the
> following outcomes:
> 
> - there would be two instances of the same table (think e.g. SSDT)
> - the second attempt would be rejected (e.g. FADT) and that error would
>   terminate the linker-loader procedure.
> 
> This issue would not be too hard to overcome, with a simple "memoization
> technique". After the initial loading & linking of the tables, OVMF
> could remember the addresses of the "source" ACPI tables, and could
> avoid passing already installed source tables to
> EFI_ACPI_TABLE_PROTOCOL.InstallAcpiTable() for a second time.
> 
> * The second use case is when an ACPI table is linked *only* from QEMU's
> XSDT. This is much harder to fix, because
> EFI_ACPI_TABLE_PROTOCOL.InstallAcpiTable() in edk2 links the copy of the
> passed-in table into *both* RSDT and XSDT, automatically. And, again,
> the UEFI spec doesn't provide a way to control this from the caller
> (i.e. from within OVMF).
> 
> I have tried earlier to effect a change in the specification of
> EFI_ACPI_TABLE_PROTOCOL.InstallAcpiTable(), on the ASWG and USWG mailing
> lists. (At that time I was trying to expose UEFI memory *type* to the
> caller, from which the copy of the ACPI table being installed should be
> allocated from.) Alas, I received no answers at all.
> 
> All in all I strongly recommend the "place rev2+ objects in method
> scope" trick, over the "link it from the XSDT only" trick.
> 
> >> Using XSDT would increase ACPI tables occupied RAM
> >> as it would duplicate DSDT + non XP supported AML
> >> at global namespace.  
> > 
> > Not at all - I posted patches linking to same
> > tables from both RSDT and XSDT at some point.  
> 
> Yes, at <http://thread.gmane.org/gmane.comp.emulators.qemu/342559>. This
> could be made work in OVMF with the above mentioned memoization stuff.
> 
> > Only the list of pointers would be different.  
> 
> I don't recommend that, see the second case above.
> 
> Thanks
> Laszlo
> 
> >> So far we've managed keep DSDT compatible with XP while
> >> introducing features from v2 and higher ACPI revisions as
> >> AML that is only evaluated on demand.
> >> We can continue doing so unless we have to unconditionally
> >> add incompatible AML at global scope.
> >>  
> > 
> > Yes.
> >   
> >>>  
> >>>> Michael, Paolo, what do you think about these ideas?
> >>>>
> >>>> Thanks!  
> >>>
> >>>
> >>>
> >>> So using a patch below, we can add Name(PQRS, 0x0) at the top of the
> >>> SSDT (or bottom, or add a separate SSDT just for that).  It returns the
> >>> current offset so we can add that to the linker.
> >>>
> >>> Won't work if you append the Name to the Aml structure (these can be
> >>> nested to arbitrary depth using aml_append), so using plain GArray for
> >>> this API makes sense to me.
> >>>  
> >>> --->  
> >>>
> >>> acpi: add build_append_named_dword, returning an offset in buffer
> >>>
> >>> This is a very limited form of support for runtime patching -
> >>> similar in functionality to what we can do with ACPI_EXTRACT
> >>> macros in python, but implemented in C.
> >>>
> >>> This is to allow ACPI code direct access to data tables -
> >>> which is exactly what DataTableRegion is there for, except
> >>> no known windows release so far implements DataTableRegion.  
> >> unsupported means Windows will BSOD, so it's practically
> >> unusable unless MS will patch currently existing Windows
> >> versions.  
> > 
> > Yes. That's why my patch allows patching SSDT without using
> > DataTableRegion.
> >   
> >> Another thing about DataTableRegion is that ACPI tables are
> >> supposed to have static content which matches checksum in
> >> table the header while you are trying to use it for dynamic
> >> data. It would be cleaner/more compatible to teach
> >> bios-linker-loader to just allocate memory and patch AML
> >> with the allocated address.  
> > 
> > Yes - if address is static, you need to put it outside
> > the table. Can come right before or right after this.
> >   
> >> Also if OperationRegion() is used, then one has to patch
> >> DefOpRegion directly as RegionOffset must be Integer,
> >> using variable names is not permitted there.  
> > 
> > I am not sure the comment was understood correctly.
> > The comment says really "we can't use DataTableRegion
> > so here is an alternative".
> >   
> >>  
> >>>
> >>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> >>>
> >>> ---
> >>>
> >>> diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
> >>> index 1b632dc..f8998ea 100644
> >>> --- a/include/hw/acpi/aml-build.h
> >>> +++ b/include/hw/acpi/aml-build.h
> >>> @@ -286,4 +286,7 @@ void acpi_build_tables_cleanup(AcpiBuildTables *tables, bool mfre);
> >>>  void
> >>>  build_rsdt(GArray *table_data, GArray *linker, GArray *table_offsets);
> >>>  
> >>> +int
> >>> +build_append_named_dword(GArray *array, const char *name_format, ...);
> >>> +
> >>>  #endif
> >>> diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
> >>> index 0d4b324..7f9fa65 100644
> >>> --- a/hw/acpi/aml-build.c
> >>> +++ b/hw/acpi/aml-build.c
> >>> @@ -262,6 +262,32 @@ static void build_append_int(GArray *table, uint64_t value)
> >>>      }
> >>>  }
> >>>  
> >>> +/* Build NAME(XXXX, 0x0) where 0x0 is encoded as a qword,
> >>> + * and return the offset to 0x0 for runtime patching.
> >>> + *
> >>> + * Warning: runtime patching is best avoided. Only use this as
> >>> + * a replacement for DataTableRegion (for guests that don't
> >>> + * support it).
> >>> + */
> >>> +int
> >>> +build_append_named_qword(GArray *array, const char *name_format, ...)
> >>> +{
> >>> +    int offset;
> >>> +    va_list ap;
> >>> +
> >>> +    va_start(ap, name_format);
> >>> +    build_append_namestringv(array, name_format, ap);
> >>> +    va_end(ap);
> >>> +
> >>> +    build_append_byte(array, 0x0E); /* QWordPrefix */
> >>> +
> >>> +    offset = array->len;
> >>> +    build_append_int_noprefix(array, 0x0, 8);
> >>> +    assert(array->len == offset + 8);
> >>> +
> >>> +    return offset;
> >>> +}
> >>> +
> >>>  static GPtrArray *alloc_list;
> >>>  
> >>>  static Aml *aml_alloc(void)
> >>>
> >>>  
>

On 01/05/16 18:08, Igor Mammedov wrote:
> On Mon, 4 Jan 2016 21:17:31 +0100
> Laszlo Ersek <lersek@redhat.com> wrote:
> 
>> Michael CC'd me on the grandparent of the email below. I'll try to add
>> my thoughts in a single go, with regard to OVMF.
>>
>> On 12/30/15 20:52, Michael S. Tsirkin wrote:
>>> On Wed, Dec 30, 2015 at 04:55:54PM +0100, Igor Mammedov wrote:  
>>>> On Mon, 28 Dec 2015 14:50:15 +0200
>>>> "Michael S. Tsirkin" <mst@redhat.com> wrote:
>>>>  
>>>>> On Mon, Dec 28, 2015 at 10:39:04AM +0800, Xiao Guangrong wrote:  
>>>>>>
>>>>>> Hi Michael, Paolo,
>>>>>>
>>>>>> Now it is the time to return to the challenge that how to reserve guest
>>>>>> physical region internally used by ACPI.
>>>>>>
>>>>>> Igor suggested that:
>>>>>> | An alternative place to allocate reserve from could be high memory.
>>>>>> | For pc we have "reserved-memory-end" which currently makes sure
>>>>>> | that hotpluggable memory range isn't used by firmware
>>>>>> (https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg00926.html)  
>>
>> OVMF has no support for the "reserved-memory-end" fw_cfg file. The
>> reason is that nobody wrote that patch, nor asked for the patch to be
>> written. (Not implying that just requesting the patch would be
>> sufficient for the patch to be written.)
>>
>>>>> I don't want to tie things to reserved-memory-end because this
>>>>> does not scale: next time we need to reserve memory,
>>>>> we'll need to find yet another way to figure out what is where.  
>>>> Could you elaborate a bit more on a problem you're seeing?
>>>>
>>>> To me it looks like it scales rather well.
>>>> For example lets imagine that we adding a device
>>>> that has some on device memory that should be mapped into GPA
>>>> code to do so would look like:
>>>>
>>>>   pc_machine_device_plug_cb(dev)
>>>>   {
>>>>    ...
>>>>    if (dev == OUR_NEW_DEVICE_TYPE) {
>>>>        memory_region_add_subregion(as, current_reserved_end, &dev->mr);
>>>>        set_new_reserved_end(current_reserved_end + memory_region_size(&dev->mr));
>>>>    }
>>>>   }
>>>>
>>>> we can practically add any number of new devices that way.  
>>>
>>> Yes but we'll have to build a host side allocator for these, and that's
>>> nasty. We'll also have to maintain these addresses indefinitely (at
>>> least per machine version) as they are guest visible.
>>> Not only that, there's no way for guest to know if we move things
>>> around, so basically we'll never be able to change addresses.
>>>
>>>   
>>>>    
>>>>> I would like ./hw/acpi/bios-linker-loader.c interface to be extended to
>>>>> support 64 bit RAM instead  
>>
>> This looks quite doable in OVMF, as long as the blob to allocate from
>> high memory contains *zero* ACPI tables.
>>
>> (
>> Namely, each ACPI table is installed from the containing fw_cfg blob
>> with EFI_ACPI_TABLE_PROTOCOL.InstallAcpiTable(), and the latter has its
>> own allocation policy for the *copies* of ACPI tables it installs.
>>
>> This allocation policy is left unspecified in the section of the UEFI
>> spec that governs EFI_ACPI_TABLE_PROTOCOL.
>>
>> The current policy in edk2 (= the reference implementation) seems to be
>> "allocate from under 4GB". It is currently being changed to "try to
>> allocate from under 4GB, and if that fails, retry from high memory". (It
>> is motivated by Aarch64 machines that may have no DRAM at all under 4GB.)
>> )
>>
>>>>> (and maybe a way to allocate and
>>>>> zero-initialize buffer without loading it through fwcfg),  
>>
>> Sounds reasonable.
>>
>>>>> this way bios
>>>>> does the allocation, and addresses can be patched into acpi.  
>>>> and then guest side needs to parse/execute some AML that would
>>>> initialize QEMU side so it would know where to write data.  
>>>
>>> Well not really - we can put it in a data table, by itself
>>> so it's easy to find.  
>>
>> Do you mean acpi_tb_find_table(), acpi_get_table_by_index() /
>> acpi_get_table_with_size()?
>>
>>>
>>> AML is only needed if access from ACPI is desired.
>>>
>>>   
>>>> bios-linker-loader is a great interface for initializing some
>>>> guest owned data and linking it together but I think it adds
>>>> unnecessary complexity and is misused if it's used to handle
>>>> device owned data/on device memory in this and VMGID cases.  
>>>
>>> I want a generic interface for guest to enumerate these things.  linker
>>> seems quite reasonable but if you see a reason why it won't do, or want
>>> to propose a better interface, fine.  
>>
>> * The guest could do the following:
>> - while processing the ALLOCATE commands, it would make a note where in
>> GPA space each fw_cfg blob gets allocated
>> - at the end the guest would prepare a temporary array with a predefined
>> record format, that associates each fw_cfg blob's name with the concrete
>> allocation address
>> - it would create an FWCfgDmaAccess stucture pointing at this array,
>> with a new "control" bit set (or something similar)
>> - the guest could write the address of the FWCfgDmaAccess struct to the
>> appropriate register, as always.
>>
>> * Another idea would be a GET_ALLOCATION_ADDRESS linker/loader command,
>> specifying:
>> - the fw_cfg blob's name, for which to retrieve the guest-allocated
>>   address (this command could only follow the matching ALLOCATE
>>   command, never precede it)
>> - a flag whether the address should be written to IO or MMIO space
>>   (would be likely IO on x86, MMIO on ARM)
>> - a unique uint64_t key (could be the 16-bit fw_cfg selector value that
>>   identifies the blob, actually!)
>> - a uint64_t (IO or MMIO) address to write the unique key and then the
>>   allocation address to.
>>
>> Either way, QEMU could learn about all the relevant guest-side
>> allocation addresses in a low number of traps. In addition, AML code
>> wouldn't have to reflect any allocation addresses to QEMU, ever.

> That would be nice trick. I see 2 issues here:
>  1. ACPI tables blob is build atomically when one guest tries to read it
>     from fw_cfg so patched addresses have to be communicated
>     to QEMU before that.

I don't understand issue #1. I think it is okay if the allocation
happens strictly after QEMU refreshes / regenerates the ACPI payload.
Namely, the guest-allocated addresses have two uses:
- references from within the ACPI payload
- references from the QEMU side, for device operation.

The first purpose is covered by the linker/loader itself (that is,
GET_ALLOCATION_ADDRESS would be used *in addition* to ADD_POINTER). The
second purpose would be covered by GET_ALLOCATION_ADDRESS.

>  2. Mo important I think that we are miss-using linker-loader
>     interface here, trying to from allocate buffer in guest RAM
>     an so consuming it while all we need a window into device
>     memory mapped somewhere outside of RAM occupied  address space.

But, more importantly, I definitely see your point with issue #2. I'm
neutral on the question whether this should be solved with the ACPI
linker/loader or with something else. I'm perfectly fine with "something
else", as long as it is generic enough. The above GET_ALLOCATION_ADDRESS
idea is relevant *only if* the ACPI linker/loader is deemed the best
solution here.

(Heck, if the linker/loader avenue is rejected here, that's the least
work for me! :))

Thanks
Laszlo

> 
>>
>>>
>>> PCI would do, too - though windows guys had concerns about
>>> returning PCI BARs from ACPI.
>>>
>>>   
>>>> There was RFC on list to make BIOS boot from NVDIMM already
>>>> doing some ACPI table lookup/parsing. Now if they were forced
>>>> to also parse and execute AML to initialize QEMU with guest
>>>> allocated address that would complicate them quite a bit.  
>>>
>>> If they just need to find a table by name, it won't be
>>> too bad, would it?
>>>   
>>>> While with NVDIMM control memory region mapped directly by QEMU,
>>>> respective patches don't need in any way to initialize QEMU,
>>>> all they would need just read necessary data from control region.
>>>>
>>>> Also using bios-linker-loader takes away some usable RAM
>>>> from guest and in the end that doesn't scale,
>>>> the more devices I add the less usable RAM is left for guest OS
>>>> while all the device needs is a piece of GPA address space
>>>> that would belong to it.  
>>>
>>> I don't get this comment. I don't think it's MMIO that is wanted.
>>> If it's backed by qemu virtual memory then it's RAM.
>>>   
>>>>>
>>>>> See patch at the bottom that might be handy.  
>>
>> I've given up on Microsoft implementing DataTableRegion. (It's sad, really.)
>>
>> From last year I have a WIP version of "docs/vmgenid.txt" that is based
>> on Michael's build_append_named_dword() function. If
>> GET_ALLOCATION_ADDRESS above looks good, then I could simplify the ACPI
>> stuff in that text file (and hopefully post it soon after for comments?)
>>
>>>>>  
>>>>>> he also innovated a way to use 64-bit address in DSDT/SSDT.rev = 1:
>>>>>> | when writing ASL one shall make sure that only XP supported
>>>>>> | features are in global scope, which is evaluated when tables
>>>>>> | are loaded and features of rev2 and higher are inside methods.
>>>>>> | That way XP doesn't crash as far as it doesn't evaluate unsupported
>>>>>> | opcode and one can guard those opcodes checking _REV object if neccesary.
>>>>>> (https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg01010.html)  
>>>>>
>>>>> Yes, this technique works.  
>>
>> Agreed.
>>
>>>>>
>>>>> An alternative is to add an XSDT, XP ignores that.
>>>>> XSDT at the moment breaks OVMF (because it loads both
>>>>> the RSDT and the XSDT, which is wrong), but I think
>>>>> Laszlo was working on a fix for that.  
>>
>> We have to distinguish two use cases here.
>>
>> * The first is the case when QEMU prepares both an XSDT and an RSDT, and
>> links at least one common ACPI table from both. This would cause OVMF to
>> pass the same source (= to-be-copied) table to
>> EFI_ACPI_TABLE_PROTOCOL.InstallAcpiTable() twice, with one of the
>> following outcomes:
>>
>> - there would be two instances of the same table (think e.g. SSDT)
>> - the second attempt would be rejected (e.g. FADT) and that error would
>>   terminate the linker-loader procedure.
>>
>> This issue would not be too hard to overcome, with a simple "memoization
>> technique". After the initial loading & linking of the tables, OVMF
>> could remember the addresses of the "source" ACPI tables, and could
>> avoid passing already installed source tables to
>> EFI_ACPI_TABLE_PROTOCOL.InstallAcpiTable() for a second time.
>>
>> * The second use case is when an ACPI table is linked *only* from QEMU's
>> XSDT. This is much harder to fix, because
>> EFI_ACPI_TABLE_PROTOCOL.InstallAcpiTable() in edk2 links the copy of the
>> passed-in table into *both* RSDT and XSDT, automatically. And, again,
>> the UEFI spec doesn't provide a way to control this from the caller
>> (i.e. from within OVMF).
>>
>> I have tried earlier to effect a change in the specification of
>> EFI_ACPI_TABLE_PROTOCOL.InstallAcpiTable(), on the ASWG and USWG mailing
>> lists. (At that time I was trying to expose UEFI memory *type* to the
>> caller, from which the copy of the ACPI table being installed should be
>> allocated from.) Alas, I received no answers at all.
>>
>> All in all I strongly recommend the "place rev2+ objects in method
>> scope" trick, over the "link it from the XSDT only" trick.
>>
>>>> Using XSDT would increase ACPI tables occupied RAM
>>>> as it would duplicate DSDT + non XP supported AML
>>>> at global namespace.  
>>>
>>> Not at all - I posted patches linking to same
>>> tables from both RSDT and XSDT at some point.  
>>
>> Yes, at <http://thread.gmane.org/gmane.comp.emulators.qemu/342559>. This
>> could be made work in OVMF with the above mentioned memoization stuff.
>>
>>> Only the list of pointers would be different.  
>>
>> I don't recommend that, see the second case above.
>>
>> Thanks
>> Laszlo
>>
>>>> So far we've managed keep DSDT compatible with XP while
>>>> introducing features from v2 and higher ACPI revisions as
>>>> AML that is only evaluated on demand.
>>>> We can continue doing so unless we have to unconditionally
>>>> add incompatible AML at global scope.
>>>>  
>>>
>>> Yes.
>>>   
>>>>>  
>>>>>> Michael, Paolo, what do you think about these ideas?
>>>>>>
>>>>>> Thanks!  
>>>>>
>>>>>
>>>>>
>>>>> So using a patch below, we can add Name(PQRS, 0x0) at the top of the
>>>>> SSDT (or bottom, or add a separate SSDT just for that).  It returns the
>>>>> current offset so we can add that to the linker.
>>>>>
>>>>> Won't work if you append the Name to the Aml structure (these can be
>>>>> nested to arbitrary depth using aml_append), so using plain GArray for
>>>>> this API makes sense to me.
>>>>>  
>>>>> --->  
>>>>>
>>>>> acpi: add build_append_named_dword, returning an offset in buffer
>>>>>
>>>>> This is a very limited form of support for runtime patching -
>>>>> similar in functionality to what we can do with ACPI_EXTRACT
>>>>> macros in python, but implemented in C.
>>>>>
>>>>> This is to allow ACPI code direct access to data tables -
>>>>> which is exactly what DataTableRegion is there for, except
>>>>> no known windows release so far implements DataTableRegion.  
>>>> unsupported means Windows will BSOD, so it's practically
>>>> unusable unless MS will patch currently existing Windows
>>>> versions.  
>>>
>>> Yes. That's why my patch allows patching SSDT without using
>>> DataTableRegion.
>>>   
>>>> Another thing about DataTableRegion is that ACPI tables are
>>>> supposed to have static content which matches checksum in
>>>> table the header while you are trying to use it for dynamic
>>>> data. It would be cleaner/more compatible to teach
>>>> bios-linker-loader to just allocate memory and patch AML
>>>> with the allocated address.  
>>>
>>> Yes - if address is static, you need to put it outside
>>> the table. Can come right before or right after this.
>>>   
>>>> Also if OperationRegion() is used, then one has to patch
>>>> DefOpRegion directly as RegionOffset must be Integer,
>>>> using variable names is not permitted there.  
>>>
>>> I am not sure the comment was understood correctly.
>>> The comment says really "we can't use DataTableRegion
>>> so here is an alternative".
>>>   
>>>>  
>>>>>
>>>>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>>>>>
>>>>> ---
>>>>>
>>>>> diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
>>>>> index 1b632dc..f8998ea 100644
>>>>> --- a/include/hw/acpi/aml-build.h
>>>>> +++ b/include/hw/acpi/aml-build.h
>>>>> @@ -286,4 +286,7 @@ void acpi_build_tables_cleanup(AcpiBuildTables *tables, bool mfre);
>>>>>  void
>>>>>  build_rsdt(GArray *table_data, GArray *linker, GArray *table_offsets);
>>>>>  
>>>>> +int
>>>>> +build_append_named_dword(GArray *array, const char *name_format, ...);
>>>>> +
>>>>>  #endif
>>>>> diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
>>>>> index 0d4b324..7f9fa65 100644
>>>>> --- a/hw/acpi/aml-build.c
>>>>> +++ b/hw/acpi/aml-build.c
>>>>> @@ -262,6 +262,32 @@ static void build_append_int(GArray *table, uint64_t value)
>>>>>      }
>>>>>  }
>>>>>  
>>>>> +/* Build NAME(XXXX, 0x0) where 0x0 is encoded as a qword,
>>>>> + * and return the offset to 0x0 for runtime patching.
>>>>> + *
>>>>> + * Warning: runtime patching is best avoided. Only use this as
>>>>> + * a replacement for DataTableRegion (for guests that don't
>>>>> + * support it).
>>>>> + */
>>>>> +int
>>>>> +build_append_named_qword(GArray *array, const char *name_format, ...)
>>>>> +{
>>>>> +    int offset;
>>>>> +    va_list ap;
>>>>> +
>>>>> +    va_start(ap, name_format);
>>>>> +    build_append_namestringv(array, name_format, ap);
>>>>> +    va_end(ap);
>>>>> +
>>>>> +    build_append_byte(array, 0x0E); /* QWordPrefix */
>>>>> +
>>>>> +    offset = array->len;
>>>>> +    build_append_int_noprefix(array, 0x0, 8);
>>>>> +    assert(array->len == offset + 8);
>>>>> +
>>>>> +    return offset;
>>>>> +}
>>>>> +
>>>>>  static GPtrArray *alloc_list;
>>>>>  
>>>>>  static Aml *aml_alloc(void)
>>>>>
>>>>>  
>>
>

On Tue, 5 Jan 2016 18:22:33 +0100
Laszlo Ersek <lersek@redhat.com> wrote:

> On 01/05/16 18:08, Igor Mammedov wrote:
> > On Mon, 4 Jan 2016 21:17:31 +0100
> > Laszlo Ersek <lersek@redhat.com> wrote:
> >   
> >> Michael CC'd me on the grandparent of the email below. I'll try to add
> >> my thoughts in a single go, with regard to OVMF.
> >>
> >> On 12/30/15 20:52, Michael S. Tsirkin wrote:  
> >>> On Wed, Dec 30, 2015 at 04:55:54PM +0100, Igor Mammedov wrote:    
> >>>> On Mon, 28 Dec 2015 14:50:15 +0200
> >>>> "Michael S. Tsirkin" <mst@redhat.com> wrote:
> >>>>    
> >>>>> On Mon, Dec 28, 2015 at 10:39:04AM +0800, Xiao Guangrong wrote:    
> >>>>>>
> >>>>>> Hi Michael, Paolo,
> >>>>>>
> >>>>>> Now it is the time to return to the challenge that how to reserve guest
> >>>>>> physical region internally used by ACPI.
> >>>>>>
> >>>>>> Igor suggested that:
> >>>>>> | An alternative place to allocate reserve from could be high memory.
> >>>>>> | For pc we have "reserved-memory-end" which currently makes sure
> >>>>>> | that hotpluggable memory range isn't used by firmware
> >>>>>> (https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg00926.html)    
> >>
> >> OVMF has no support for the "reserved-memory-end" fw_cfg file. The
> >> reason is that nobody wrote that patch, nor asked for the patch to be
> >> written. (Not implying that just requesting the patch would be
> >> sufficient for the patch to be written.)
> >>  
> >>>>> I don't want to tie things to reserved-memory-end because this
> >>>>> does not scale: next time we need to reserve memory,
> >>>>> we'll need to find yet another way to figure out what is where.    
> >>>> Could you elaborate a bit more on a problem you're seeing?
> >>>>
> >>>> To me it looks like it scales rather well.
> >>>> For example lets imagine that we adding a device
> >>>> that has some on device memory that should be mapped into GPA
> >>>> code to do so would look like:
> >>>>
> >>>>   pc_machine_device_plug_cb(dev)
> >>>>   {
> >>>>    ...
> >>>>    if (dev == OUR_NEW_DEVICE_TYPE) {
> >>>>        memory_region_add_subregion(as, current_reserved_end, &dev->mr);
> >>>>        set_new_reserved_end(current_reserved_end + memory_region_size(&dev->mr));
> >>>>    }
> >>>>   }
> >>>>
> >>>> we can practically add any number of new devices that way.    
> >>>
> >>> Yes but we'll have to build a host side allocator for these, and that's
> >>> nasty. We'll also have to maintain these addresses indefinitely (at
> >>> least per machine version) as they are guest visible.
> >>> Not only that, there's no way for guest to know if we move things
> >>> around, so basically we'll never be able to change addresses.
> >>>
> >>>     
> >>>>      
> >>>>> I would like ./hw/acpi/bios-linker-loader.c interface to be extended to
> >>>>> support 64 bit RAM instead    
> >>
> >> This looks quite doable in OVMF, as long as the blob to allocate from
> >> high memory contains *zero* ACPI tables.
> >>
> >> (
> >> Namely, each ACPI table is installed from the containing fw_cfg blob
> >> with EFI_ACPI_TABLE_PROTOCOL.InstallAcpiTable(), and the latter has its
> >> own allocation policy for the *copies* of ACPI tables it installs.
> >>
> >> This allocation policy is left unspecified in the section of the UEFI
> >> spec that governs EFI_ACPI_TABLE_PROTOCOL.
> >>
> >> The current policy in edk2 (= the reference implementation) seems to be
> >> "allocate from under 4GB". It is currently being changed to "try to
> >> allocate from under 4GB, and if that fails, retry from high memory". (It
> >> is motivated by Aarch64 machines that may have no DRAM at all under 4GB.)
> >> )
> >>  
> >>>>> (and maybe a way to allocate and
> >>>>> zero-initialize buffer without loading it through fwcfg),    
> >>
> >> Sounds reasonable.
> >>  
> >>>>> this way bios
> >>>>> does the allocation, and addresses can be patched into acpi.    
> >>>> and then guest side needs to parse/execute some AML that would
> >>>> initialize QEMU side so it would know where to write data.    
> >>>
> >>> Well not really - we can put it in a data table, by itself
> >>> so it's easy to find.    
> >>
> >> Do you mean acpi_tb_find_table(), acpi_get_table_by_index() /
> >> acpi_get_table_with_size()?
> >>  
> >>>
> >>> AML is only needed if access from ACPI is desired.
> >>>
> >>>     
> >>>> bios-linker-loader is a great interface for initializing some
> >>>> guest owned data and linking it together but I think it adds
> >>>> unnecessary complexity and is misused if it's used to handle
> >>>> device owned data/on device memory in this and VMGID cases.    
> >>>
> >>> I want a generic interface for guest to enumerate these things.  linker
> >>> seems quite reasonable but if you see a reason why it won't do, or want
> >>> to propose a better interface, fine.    
> >>
> >> * The guest could do the following:
> >> - while processing the ALLOCATE commands, it would make a note where in
> >> GPA space each fw_cfg blob gets allocated
> >> - at the end the guest would prepare a temporary array with a predefined
> >> record format, that associates each fw_cfg blob's name with the concrete
> >> allocation address
> >> - it would create an FWCfgDmaAccess stucture pointing at this array,
> >> with a new "control" bit set (or something similar)
> >> - the guest could write the address of the FWCfgDmaAccess struct to the
> >> appropriate register, as always.
> >>
> >> * Another idea would be a GET_ALLOCATION_ADDRESS linker/loader command,
> >> specifying:
> >> - the fw_cfg blob's name, for which to retrieve the guest-allocated
> >>   address (this command could only follow the matching ALLOCATE
> >>   command, never precede it)
> >> - a flag whether the address should be written to IO or MMIO space
> >>   (would be likely IO on x86, MMIO on ARM)
> >> - a unique uint64_t key (could be the 16-bit fw_cfg selector value that
> >>   identifies the blob, actually!)
> >> - a uint64_t (IO or MMIO) address to write the unique key and then the
> >>   allocation address to.
> >>
> >> Either way, QEMU could learn about all the relevant guest-side
> >> allocation addresses in a low number of traps. In addition, AML code
> >> wouldn't have to reflect any allocation addresses to QEMU, ever.  
> 
> > That would be nice trick. I see 2 issues here:
> >  1. ACPI tables blob is build atomically when one guest tries to read it
> >     from fw_cfg so patched addresses have to be communicated
> >     to QEMU before that.  
> 
> I don't understand issue #1. I think it is okay if the allocation
> happens strictly after QEMU refreshes / regenerates the ACPI payload.
> Namely, the guest-allocated addresses have two uses:
> - references from within the ACPI payload
If references are from AML, then AML should be patched by linker,
which is tricky and forces us to invent duplicate AML API that
would be able to tell linker where AML object should be patched
(Michael's patch in this thread as example)

It would be better if linker would communicate addresses to QEMU
before AML is built, so that AML would use already present
in QEMU addresses and doesn't have to be patched at all.

> - references from the QEMU side, for device operation.
> 
> The first purpose is covered by the linker/loader itself (that is,
> GET_ALLOCATION_ADDRESS would be used *in addition* to ADD_POINTER). The
> second purpose would be covered by GET_ALLOCATION_ADDRESS.
> 
> >  2. Mo important I think that we are miss-using linker-loader
> >     interface here, trying to from allocate buffer in guest RAM
> >     an so consuming it while all we need a window into device
> >     memory mapped somewhere outside of RAM occupied  address space.  
> 
> But, more importantly, I definitely see your point with issue #2. I'm
> neutral on the question whether this should be solved with the ACPI
> linker/loader or with something else. I'm perfectly fine with "something
> else", as long as it is generic enough. The above GET_ALLOCATION_ADDRESS
> idea is relevant *only if* the ACPI linker/loader is deemed the best
> solution here.
> 
> (Heck, if the linker/loader avenue is rejected here, that's the least
> work for me! :))
> 
> Thanks
> Laszlo
> 
> >   
> >>  
> >>>
> >>> PCI would do, too - though windows guys had concerns about
> >>> returning PCI BARs from ACPI.
> >>>
> >>>     
> >>>> There was RFC on list to make BIOS boot from NVDIMM already
> >>>> doing some ACPI table lookup/parsing. Now if they were forced
> >>>> to also parse and execute AML to initialize QEMU with guest
> >>>> allocated address that would complicate them quite a bit.    
> >>>
> >>> If they just need to find a table by name, it won't be
> >>> too bad, would it?
> >>>     
> >>>> While with NVDIMM control memory region mapped directly by QEMU,
> >>>> respective patches don't need in any way to initialize QEMU,
> >>>> all they would need just read necessary data from control region.
> >>>>
> >>>> Also using bios-linker-loader takes away some usable RAM
> >>>> from guest and in the end that doesn't scale,
> >>>> the more devices I add the less usable RAM is left for guest OS
> >>>> while all the device needs is a piece of GPA address space
> >>>> that would belong to it.    
> >>>
> >>> I don't get this comment. I don't think it's MMIO that is wanted.
> >>> If it's backed by qemu virtual memory then it's RAM.
> >>>     
> >>>>>
> >>>>> See patch at the bottom that might be handy.    
> >>
> >> I've given up on Microsoft implementing DataTableRegion. (It's sad, really.)
> >>
> >> From last year I have a WIP version of "docs/vmgenid.txt" that is based
> >> on Michael's build_append_named_dword() function. If
> >> GET_ALLOCATION_ADDRESS above looks good, then I could simplify the ACPI
> >> stuff in that text file (and hopefully post it soon after for comments?)
> >>  
> >>>>>    
> >>>>>> he also innovated a way to use 64-bit address in DSDT/SSDT.rev = 1:
> >>>>>> | when writing ASL one shall make sure that only XP supported
> >>>>>> | features are in global scope, which is evaluated when tables
> >>>>>> | are loaded and features of rev2 and higher are inside methods.
> >>>>>> | That way XP doesn't crash as far as it doesn't evaluate unsupported
> >>>>>> | opcode and one can guard those opcodes checking _REV object if neccesary.
> >>>>>> (https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg01010.html)    
> >>>>>
> >>>>> Yes, this technique works.    
> >>
> >> Agreed.
> >>  
> >>>>>
> >>>>> An alternative is to add an XSDT, XP ignores that.
> >>>>> XSDT at the moment breaks OVMF (because it loads both
> >>>>> the RSDT and the XSDT, which is wrong), but I think
> >>>>> Laszlo was working on a fix for that.    
> >>
> >> We have to distinguish two use cases here.
> >>
> >> * The first is the case when QEMU prepares both an XSDT and an RSDT, and
> >> links at least one common ACPI table from both. This would cause OVMF to
> >> pass the same source (= to-be-copied) table to
> >> EFI_ACPI_TABLE_PROTOCOL.InstallAcpiTable() twice, with one of the
> >> following outcomes:
> >>
> >> - there would be two instances of the same table (think e.g. SSDT)
> >> - the second attempt would be rejected (e.g. FADT) and that error would
> >>   terminate the linker-loader procedure.
> >>
> >> This issue would not be too hard to overcome, with a simple "memoization
> >> technique". After the initial loading & linking of the tables, OVMF
> >> could remember the addresses of the "source" ACPI tables, and could
> >> avoid passing already installed source tables to
> >> EFI_ACPI_TABLE_PROTOCOL.InstallAcpiTable() for a second time.
> >>
> >> * The second use case is when an ACPI table is linked *only* from QEMU's
> >> XSDT. This is much harder to fix, because
> >> EFI_ACPI_TABLE_PROTOCOL.InstallAcpiTable() in edk2 links the copy of the
> >> passed-in table into *both* RSDT and XSDT, automatically. And, again,
> >> the UEFI spec doesn't provide a way to control this from the caller
> >> (i.e. from within OVMF).
> >>
> >> I have tried earlier to effect a change in the specification of
> >> EFI_ACPI_TABLE_PROTOCOL.InstallAcpiTable(), on the ASWG and USWG mailing
> >> lists. (At that time I was trying to expose UEFI memory *type* to the
> >> caller, from which the copy of the ACPI table being installed should be
> >> allocated from.) Alas, I received no answers at all.
> >>
> >> All in all I strongly recommend the "place rev2+ objects in method
> >> scope" trick, over the "link it from the XSDT only" trick.
> >>  
> >>>> Using XSDT would increase ACPI tables occupied RAM
> >>>> as it would duplicate DSDT + non XP supported AML
> >>>> at global namespace.    
> >>>
> >>> Not at all - I posted patches linking to same
> >>> tables from both RSDT and XSDT at some point.    
> >>
> >> Yes, at <http://thread.gmane.org/gmane.comp.emulators.qemu/342559>. This
> >> could be made work in OVMF with the above mentioned memoization stuff.
> >>  
> >>> Only the list of pointers would be different.    
> >>
> >> I don't recommend that, see the second case above.
> >>
> >> Thanks
> >> Laszlo
> >>  
> >>>> So far we've managed keep DSDT compatible with XP while
> >>>> introducing features from v2 and higher ACPI revisions as
> >>>> AML that is only evaluated on demand.
> >>>> We can continue doing so unless we have to unconditionally
> >>>> add incompatible AML at global scope.
> >>>>    
> >>>
> >>> Yes.
> >>>     
> >>>>>    
> >>>>>> Michael, Paolo, what do you think about these ideas?
> >>>>>>
> >>>>>> Thanks!    
> >>>>>
> >>>>>
> >>>>>
> >>>>> So using a patch below, we can add Name(PQRS, 0x0) at the top of the
> >>>>> SSDT (or bottom, or add a separate SSDT just for that).  It returns the
> >>>>> current offset so we can add that to the linker.
> >>>>>
> >>>>> Won't work if you append the Name to the Aml structure (these can be
> >>>>> nested to arbitrary depth using aml_append), so using plain GArray for
> >>>>> this API makes sense to me.
> >>>>>    
> >>>>> --->    
> >>>>>
> >>>>> acpi: add build_append_named_dword, returning an offset in buffer
> >>>>>
> >>>>> This is a very limited form of support for runtime patching -
> >>>>> similar in functionality to what we can do with ACPI_EXTRACT
> >>>>> macros in python, but implemented in C.
> >>>>>
> >>>>> This is to allow ACPI code direct access to data tables -
> >>>>> which is exactly what DataTableRegion is there for, except
> >>>>> no known windows release so far implements DataTableRegion.    
> >>>> unsupported means Windows will BSOD, so it's practically
> >>>> unusable unless MS will patch currently existing Windows
> >>>> versions.    
> >>>
> >>> Yes. That's why my patch allows patching SSDT without using
> >>> DataTableRegion.
> >>>     
> >>>> Another thing about DataTableRegion is that ACPI tables are
> >>>> supposed to have static content which matches checksum in
> >>>> table the header while you are trying to use it for dynamic
> >>>> data. It would be cleaner/more compatible to teach
> >>>> bios-linker-loader to just allocate memory and patch AML
> >>>> with the allocated address.    
> >>>
> >>> Yes - if address is static, you need to put it outside
> >>> the table. Can come right before or right after this.
> >>>     
> >>>> Also if OperationRegion() is used, then one has to patch
> >>>> DefOpRegion directly as RegionOffset must be Integer,
> >>>> using variable names is not permitted there.    
> >>>
> >>> I am not sure the comment was understood correctly.
> >>> The comment says really "we can't use DataTableRegion
> >>> so here is an alternative".
> >>>     
> >>>>    
> >>>>>
> >>>>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> >>>>>
> >>>>> ---
> >>>>>
> >>>>> diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
> >>>>> index 1b632dc..f8998ea 100644
> >>>>> --- a/include/hw/acpi/aml-build.h
> >>>>> +++ b/include/hw/acpi/aml-build.h
> >>>>> @@ -286,4 +286,7 @@ void acpi_build_tables_cleanup(AcpiBuildTables *tables, bool mfre);
> >>>>>  void
> >>>>>  build_rsdt(GArray *table_data, GArray *linker, GArray *table_offsets);
> >>>>>  
> >>>>> +int
> >>>>> +build_append_named_dword(GArray *array, const char *name_format, ...);
> >>>>> +
> >>>>>  #endif
> >>>>> diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
> >>>>> index 0d4b324..7f9fa65 100644
> >>>>> --- a/hw/acpi/aml-build.c
> >>>>> +++ b/hw/acpi/aml-build.c
> >>>>> @@ -262,6 +262,32 @@ static void build_append_int(GArray *table, uint64_t value)
> >>>>>      }
> >>>>>  }
> >>>>>  
> >>>>> +/* Build NAME(XXXX, 0x0) where 0x0 is encoded as a qword,
> >>>>> + * and return the offset to 0x0 for runtime patching.
> >>>>> + *
> >>>>> + * Warning: runtime patching is best avoided. Only use this as
> >>>>> + * a replacement for DataTableRegion (for guests that don't
> >>>>> + * support it).
> >>>>> + */
> >>>>> +int
> >>>>> +build_append_named_qword(GArray *array, const char *name_format, ...)
> >>>>> +{
> >>>>> +    int offset;
> >>>>> +    va_list ap;
> >>>>> +
> >>>>> +    va_start(ap, name_format);
> >>>>> +    build_append_namestringv(array, name_format, ap);
> >>>>> +    va_end(ap);
> >>>>> +
> >>>>> +    build_append_byte(array, 0x0E); /* QWordPrefix */
> >>>>> +
> >>>>> +    offset = array->len;
> >>>>> +    build_append_int_noprefix(array, 0x0, 8);
> >>>>> +    assert(array->len == offset + 8);
> >>>>> +
> >>>>> +    return offset;
> >>>>> +}
> >>>>> +
> >>>>>  static GPtrArray *alloc_list;
> >>>>>  
> >>>>>  static Aml *aml_alloc(void)
> >>>>>
> >>>>>    
> >>  
> >   
>

On 01/06/16 14:39, Igor Mammedov wrote:
> On Tue, 5 Jan 2016 18:22:33 +0100
> Laszlo Ersek <lersek@redhat.com> wrote:
> 
>> On 01/05/16 18:08, Igor Mammedov wrote:
>>> On Mon, 4 Jan 2016 21:17:31 +0100
>>> Laszlo Ersek <lersek@redhat.com> wrote:
>>>   
>>>> Michael CC'd me on the grandparent of the email below. I'll try to add
>>>> my thoughts in a single go, with regard to OVMF.
>>>>
>>>> On 12/30/15 20:52, Michael S. Tsirkin wrote:  
>>>>> On Wed, Dec 30, 2015 at 04:55:54PM +0100, Igor Mammedov wrote:    
>>>>>> On Mon, 28 Dec 2015 14:50:15 +0200
>>>>>> "Michael S. Tsirkin" <mst@redhat.com> wrote:
>>>>>>    
>>>>>>> On Mon, Dec 28, 2015 at 10:39:04AM +0800, Xiao Guangrong wrote:    
>>>>>>>>
>>>>>>>> Hi Michael, Paolo,
>>>>>>>>
>>>>>>>> Now it is the time to return to the challenge that how to reserve guest
>>>>>>>> physical region internally used by ACPI.
>>>>>>>>
>>>>>>>> Igor suggested that:
>>>>>>>> | An alternative place to allocate reserve from could be high memory.
>>>>>>>> | For pc we have "reserved-memory-end" which currently makes sure
>>>>>>>> | that hotpluggable memory range isn't used by firmware
>>>>>>>> (https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg00926.html)    
>>>>
>>>> OVMF has no support for the "reserved-memory-end" fw_cfg file. The
>>>> reason is that nobody wrote that patch, nor asked for the patch to be
>>>> written. (Not implying that just requesting the patch would be
>>>> sufficient for the patch to be written.)
>>>>  
>>>>>>> I don't want to tie things to reserved-memory-end because this
>>>>>>> does not scale: next time we need to reserve memory,
>>>>>>> we'll need to find yet another way to figure out what is where.    
>>>>>> Could you elaborate a bit more on a problem you're seeing?
>>>>>>
>>>>>> To me it looks like it scales rather well.
>>>>>> For example lets imagine that we adding a device
>>>>>> that has some on device memory that should be mapped into GPA
>>>>>> code to do so would look like:
>>>>>>
>>>>>>   pc_machine_device_plug_cb(dev)
>>>>>>   {
>>>>>>    ...
>>>>>>    if (dev == OUR_NEW_DEVICE_TYPE) {
>>>>>>        memory_region_add_subregion(as, current_reserved_end, &dev->mr);
>>>>>>        set_new_reserved_end(current_reserved_end + memory_region_size(&dev->mr));
>>>>>>    }
>>>>>>   }
>>>>>>
>>>>>> we can practically add any number of new devices that way.    
>>>>>
>>>>> Yes but we'll have to build a host side allocator for these, and that's
>>>>> nasty. We'll also have to maintain these addresses indefinitely (at
>>>>> least per machine version) as they are guest visible.
>>>>> Not only that, there's no way for guest to know if we move things
>>>>> around, so basically we'll never be able to change addresses.
>>>>>
>>>>>     
>>>>>>      
>>>>>>> I would like ./hw/acpi/bios-linker-loader.c interface to be extended to
>>>>>>> support 64 bit RAM instead    
>>>>
>>>> This looks quite doable in OVMF, as long as the blob to allocate from
>>>> high memory contains *zero* ACPI tables.
>>>>
>>>> (
>>>> Namely, each ACPI table is installed from the containing fw_cfg blob
>>>> with EFI_ACPI_TABLE_PROTOCOL.InstallAcpiTable(), and the latter has its
>>>> own allocation policy for the *copies* of ACPI tables it installs.
>>>>
>>>> This allocation policy is left unspecified in the section of the UEFI
>>>> spec that governs EFI_ACPI_TABLE_PROTOCOL.
>>>>
>>>> The current policy in edk2 (= the reference implementation) seems to be
>>>> "allocate from under 4GB". It is currently being changed to "try to
>>>> allocate from under 4GB, and if that fails, retry from high memory". (It
>>>> is motivated by Aarch64 machines that may have no DRAM at all under 4GB.)
>>>> )
>>>>  
>>>>>>> (and maybe a way to allocate and
>>>>>>> zero-initialize buffer without loading it through fwcfg),    
>>>>
>>>> Sounds reasonable.
>>>>  
>>>>>>> this way bios
>>>>>>> does the allocation, and addresses can be patched into acpi.    
>>>>>> and then guest side needs to parse/execute some AML that would
>>>>>> initialize QEMU side so it would know where to write data.    
>>>>>
>>>>> Well not really - we can put it in a data table, by itself
>>>>> so it's easy to find.    
>>>>
>>>> Do you mean acpi_tb_find_table(), acpi_get_table_by_index() /
>>>> acpi_get_table_with_size()?
>>>>  
>>>>>
>>>>> AML is only needed if access from ACPI is desired.
>>>>>
>>>>>     
>>>>>> bios-linker-loader is a great interface for initializing some
>>>>>> guest owned data and linking it together but I think it adds
>>>>>> unnecessary complexity and is misused if it's used to handle
>>>>>> device owned data/on device memory in this and VMGID cases.    
>>>>>
>>>>> I want a generic interface for guest to enumerate these things.  linker
>>>>> seems quite reasonable but if you see a reason why it won't do, or want
>>>>> to propose a better interface, fine.    
>>>>
>>>> * The guest could do the following:
>>>> - while processing the ALLOCATE commands, it would make a note where in
>>>> GPA space each fw_cfg blob gets allocated
>>>> - at the end the guest would prepare a temporary array with a predefined
>>>> record format, that associates each fw_cfg blob's name with the concrete
>>>> allocation address
>>>> - it would create an FWCfgDmaAccess stucture pointing at this array,
>>>> with a new "control" bit set (or something similar)
>>>> - the guest could write the address of the FWCfgDmaAccess struct to the
>>>> appropriate register, as always.
>>>>
>>>> * Another idea would be a GET_ALLOCATION_ADDRESS linker/loader command,
>>>> specifying:
>>>> - the fw_cfg blob's name, for which to retrieve the guest-allocated
>>>>   address (this command could only follow the matching ALLOCATE
>>>>   command, never precede it)
>>>> - a flag whether the address should be written to IO or MMIO space
>>>>   (would be likely IO on x86, MMIO on ARM)
>>>> - a unique uint64_t key (could be the 16-bit fw_cfg selector value that
>>>>   identifies the blob, actually!)
>>>> - a uint64_t (IO or MMIO) address to write the unique key and then the
>>>>   allocation address to.
>>>>
>>>> Either way, QEMU could learn about all the relevant guest-side
>>>> allocation addresses in a low number of traps. In addition, AML code
>>>> wouldn't have to reflect any allocation addresses to QEMU, ever.  
>>
>>> That would be nice trick. I see 2 issues here:
>>>  1. ACPI tables blob is build atomically when one guest tries to read it
>>>     from fw_cfg so patched addresses have to be communicated
>>>     to QEMU before that.  
>>
>> I don't understand issue #1. I think it is okay if the allocation
>> happens strictly after QEMU refreshes / regenerates the ACPI payload.
>> Namely, the guest-allocated addresses have two uses:
>> - references from within the ACPI payload
> If references are from AML, then AML should be patched by linker,
> which is tricky and forces us to invent duplicate AML API that
> would be able to tell linker where AML object should be patched
> (Michael's patch in this thread as example)

Yes, such minimal AML patching is necessary.

> It would be better if linker would communicate addresses to QEMU
> before AML is built, so that AML would use already present
> in QEMU addresses and doesn't have to be patched at all.

I dislike this.

First, this would duplicate part of the linker's functionality in the host.

Second, it would lead to an ugly ping-pong between host and guest. First
QEMU has to create the full ACPI payload, with placeholder constants in
the AML. Then the guest could retrieve the *size* of the ACPI payload
(the fw_cf gblobs), and perform the allocations. Then QEMU would fix up
the AML. Then the guest would download the fw_cfg blobs. Then the guest
linker would fix up the data tables. Ugly ugly ugly.

I think Michael's and Xiao Guangrong's solutions to the minimal AML
patching (= patch named dword / qword object, or the constant return
value in a minimal method) is quite feasible.

How about this:

+------------------+            +-----------------------+
|Single DWORD      |            | 4KB system memory     |
|object or         |            | operation region      | ---------+
|DWORD-returning   |            | hosting a single      |          |
|method in AML,    | ---------> | "QEMU parameter"      | -----+   |
|to be patched with|            | structure, with       |      |   |
|Michael's or      |            | pointers, small       |      |   |
|Xiao Guangrong's  |            | scalars, and padding. |      |   |
|trick             |            | Call this QPRM ("QEMU |      |   |
+------------------+            | parameters").         |      |   |
                                +-----------------------+      |   |
                                                               |   |
                                +-----------------------+ <----+   |
                                | "NRAM" operation      |          |
                                | region for NVDIMM     |          |
                                +-----------------------+          |
                                                                   |
                                +--------------------------+       |
                                | Another operation region | <-----+
                                | for another device       |
                                +--------------------------+

                                ...


Here's the idea formulated in a sequence of steps:

(1) In QEMU, create a single DWORD object, or DWORD-returning simple
method, that we *do* patch, with Michael's or Xiao Guangrong's trick,
using the ACPI linker. This would be the *only* such trick.

(2) This object or method would provide the GPA of a 4KB fw_cfg blob.
This fw_cfg blob would start with 36 zero bytes (for reasons specific to
OVMF; let me skip those for now). The rest of the blob would carry a
structure that we would actually define in the QEMU source code, as a type.

Fields of this structure would be:
- pointers (4-byte or 8-byte)
- small scalars (like a 128-bit GUID)
- padding

This structure would be named QPRM ("QEMU parameters").

(3) We add an *unconditional*, do-nothing device to the DSDT whose
initialization function evaluates the DWORD object (or DWORD-returning
method), and writes the result (= the guest-allocated address of QPRM)
to a hard-coded IO (or MMIO) port range.

(4) This port range would be backed by a *single* MemoryRegion in QEMU,
and the address written by the guest would be stored in a global
variable (or a singleton object anyway).

(5) In the guest AML, an unconditional QPRM operation region would
overlay the blob, with fields matching the QEMU structure type.

(6) Whenever a new device is introduced in QEMU that needs a dedicated
system memory operation region in the guest (nvdimm, vmgenid), we add a
new field to QPRM.

If the required region is very small (just a few scalars, like with
vmgenid), then the field is placed directly in QPRM (with the necessary
padding).

Otherwise (see the NRAM operation region for nvdimm) we add a new fw_cfg
blob, and an ADD_POINTER command for relocating the referencing field in
QPRM.

(7) The device models in QEMU can follow the pointers in guest memory,
from the initially stashed address of QPRM, through the necessary
pointer fields in QPRM, to the final operation regions.

(8) The device model-specific AML in the guest can do the same
traversal. It can fetch the right pointer field from QPRM, and define a
new operation region (like NRAM) based on that value.


All in all this is just another layer of indirection, same as the
DataTableRegion idea, except that the parameter table would be located
by a central patched DWORD object or method, not by ACPI SDT signature /
OEM ID / OEM table ID.

If we can agree on this, I could work on the device model-independent
steps (1-5), and perhaps do (6) and (8) for vmgenid on top.

Thanks
Laszlo

On Wed, 6 Jan 2016 01:07:45 +0800
Xiao Guangrong <guangrong.xiao@linux.intel.com> wrote:

> On 01/06/2016 12:43 AM, Michael S. Tsirkin wrote:
> 
> >>> Yes - if address is static, you need to put it outside
> >>> the table. Can come right before or right after this.
> >>>  
> >>>> Also if OperationRegion() is used, then one has to patch
> >>>> DefOpRegion directly as RegionOffset must be Integer,
> >>>> using variable names is not permitted there.  
> >>>
> >>> I am not sure the comment was understood correctly.
> >>> The comment says really "we can't use DataTableRegion
> >>> so here is an alternative".  
> >> so how are you going to access data at which patched
> >> NameString point to?
> >> for that you'd need a normal patched OperationRegion
> >> as well since DataTableRegion isn't usable.  
> >
> > For VMGENID you would patch the method that
> > returns the address - you do not need an op region
> > as you never access it.
> >
> > I don't know about NVDIMM. Maybe OperationRegion can
> > use the patched NameString? Will need some thought.  
> 
> The ACPI spec says that the offsetTerm in OperationRegion
> is evaluated as Int, so the named object is allowed to be
> used in OperationRegion, that is exact what my patchset
> is doing (http://marc.info/?l=kvm&m=145193395624537&w=2):
that's not my reading of spec:
"
DefOpRegion := OpRegionOp NameString RegionSpace RegionOffset RegionLen
RegionOffset := TermArg => Integer
TermArg := Type2Opcode | DataObject | ArgObj | LocalObj
"

Named object is not allowed per spec, but you've used ArgObj which is
allowed, even Windows ok with such dynamic OperationRegion.

> 
> +    dsm_mem = aml_arg(3);
> +    aml_append(method, aml_store(aml_call0(NVDIMM_GET_DSM_MEM), dsm_mem));
> 
> +    aml_append(method, aml_operation_region("NRAM", AML_SYSTEM_MEMORY,
> +                                            dsm_mem, TARGET_PAGE_SIZE));
> 
> We hide the int64 object which is patched by BIOS in the method,
> NVDIMM_GET_DSM_MEM, to make windows XP happy.
considering that NRAM is allocated in low mem it's even fine to move
OperationRegion into object scope to get rid of IASL warnings
about declariong Named object inside method, but the you'd need to
patch it directly as the only choice for RegionOffset would be DataObject

> 
> However, the disadvantages i see are:
> a) as Igor pointed out, we need a way to tell QEMU what is the patched
>     address, in NVDIMM ACPI, we used a 64 bit IO ports to pass the address
>     to QEMU.
> 
> b) BIOS allocated memory is RAM based so it stops us to use MMIO in ACPI,
>     MMIO is the more scalable resource than IO port as it has larger region
>     and supports 64 bits operation.

On Tue, 5 Jan 2016 18:43:02 +0200
"Michael S. Tsirkin" <mst@redhat.com> wrote:

> On Tue, Jan 05, 2016 at 05:30:25PM +0100, Igor Mammedov wrote:
> > > > bios-linker-loader is a great interface for initializing some
> > > > guest owned data and linking it together but I think it adds
> > > > unnecessary complexity and is misused if it's used to handle
> > > > device owned data/on device memory in this and VMGID cases.    
> > > 
> > > I want a generic interface for guest to enumerate these things.  linker
> > > seems quite reasonable but if you see a reason why it won't do, or want
> > > to propose a better interface, fine.
> > > 
> > > PCI would do, too - though windows guys had concerns about
> > > returning PCI BARs from ACPI.  
> > There were potential issues with pSeries bootloader that treated
> > PCI_CLASS_MEMORY_RAM as conventional RAM but it was fixed.
> > Could you point out to discussion about windows issues?
> > 
> > What VMGEN patches that used PCI for mapping purposes were
> > stuck at, was that it was suggested to use PCI_CLASS_MEMORY_RAM
> > class id but we couldn't agree on it.
> > 
> > VMGEN v13 with full discussion is here
> > https://patchwork.ozlabs.org/patch/443554/
> > So to continue with this route we would need to pick some other
> > driver less class id so windows won't prompt for driver or
> > maybe supply our own driver stub to guarantee that no one
> > would touch it. Any suggestions?  
> 
> Pick any device/vendor id pair for which windows specifies no driver.
> There's a small risk that this will conflict with some
> guest but I think it's minimal.
device/vendor id pair was QEMU specific so doesn't conflicts with anything
issue we were trying to solve was to prevent windows asking for driver
even though it does so only once if told not to ask again.

That's why PCI_CLASS_MEMORY_RAM was selected as it's generic driver-less
device descriptor in INF file which matches as the last resort if
there isn't any other diver that's matched device by device/vendor id pair.

> 
> 
> > > 
> > >   
> > > > There was RFC on list to make BIOS boot from NVDIMM already
> > > > doing some ACPI table lookup/parsing. Now if they were forced
> > > > to also parse and execute AML to initialize QEMU with guest
> > > > allocated address that would complicate them quite a bit.    
> > > 
> > > If they just need to find a table by name, it won't be
> > > too bad, would it?  
> > that's what they were doing scanning memory for static NVDIMM table.
> > However if it were DataTable, BIOS side would have to execute
> > AML so that the table address could be told to QEMU.  
> 
> Not at all. You can find any table by its signature without
> parsing AML.
yep, and then BIOS would need to tell its address to QEMU
writing to IO port which is allocated statically in QEMU
for this purpose and is described in AML only on guest side.

> 
> 
> > In case of direct mapping or PCI BAR there is no need to initialize
> > QEMU side from AML.
> > That also saves us IO port where this address should be written
> > if bios-linker-loader approach is used.
> >   
> > >   
> > > > While with NVDIMM control memory region mapped directly by QEMU,
> > > > respective patches don't need in any way to initialize QEMU,
> > > > all they would need just read necessary data from control region.
> > > > 
> > > > Also using bios-linker-loader takes away some usable RAM
> > > > from guest and in the end that doesn't scale,
> > > > the more devices I add the less usable RAM is left for guest OS
> > > > while all the device needs is a piece of GPA address space
> > > > that would belong to it.    
> > > 
> > > I don't get this comment. I don't think it's MMIO that is wanted.
> > > If it's backed by qemu virtual memory then it's RAM.  
> > Then why don't allocate video card VRAM the same way and try to explain
> > user that a guest started with '-m 128 -device cirrus-vga,vgamem_mb=64Mb'
> > only has 64Mb of available RAM because of we think that on device VRAM
> > is also RAM.
> > 
> > Maybe I've used MMIO term wrongly here but it roughly reflects the idea
> > that on device memory (whether it's VRAM, NVDIMM control block or VMGEN
> > area) is not allocated from guest's usable RAM (as described in E820)
> > but rather directly mapped in guest's GPA and doesn't consume available
> > RAM as guest sees it. That's also the way it's done on real hardware.
> > 
> > What we need in case of VMGEN ID and NVDIMM is on device memory
> > that could be directly accessed by guest.
> > Both direct mapping or PCI BAR do that job and we could use simple
> > static AML without any patching.  
> 
> At least with VMGEN the issue is that there's an AML method
> that returns the physical address.
> Then if guest OS moves the BAR (which is legal), it will break
> since caller has no way to know it's related to the BAR.
I've found a following MS doc "Firmware Allocation of PCI Device Resources in Windows". It looks like when MS implemented resource rebalancing in
Vista they pushed a compat change to PCI specs.
That ECN is called "Ignore PCI Boot Configuration_DSM Function"
and can be found here:
https://pcisig.com/sites/default/files/specification_documents/ECR-Ignorebootconfig-final.pdf

It looks like it's possible to forbid rebalancing per
device/bridge if it has _DMS method that returns "do not
ignore the boot configuration of PCI resources".

 
> > > > > 
> > > > > See patch at the bottom that might be handy.
> > > > >     
> > > > > > he also innovated a way to use 64-bit address in DSDT/SSDT.rev = 1:
> > > > > > | when writing ASL one shall make sure that only XP supported
> > > > > > | features are in global scope, which is evaluated when tables
> > > > > > | are loaded and features of rev2 and higher are inside methods.
> > > > > > | That way XP doesn't crash as far as it doesn't evaluate unsupported
> > > > > > | opcode and one can guard those opcodes checking _REV object if neccesary.
> > > > > > (https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg01010.html)    
> > > > > 
> > > > > Yes, this technique works.
> > > > > 
> > > > > An alternative is to add an XSDT, XP ignores that.
> > > > > XSDT at the moment breaks OVMF (because it loads both
> > > > > the RSDT and the XSDT, which is wrong), but I think
> > > > > Laszlo was working on a fix for that.    
> > > > Using XSDT would increase ACPI tables occupied RAM
> > > > as it would duplicate DSDT + non XP supported AML
> > > > at global namespace.    
> > > 
> > > Not at all - I posted patches linking to same
> > > tables from both RSDT and XSDT at some point.
> > > Only the list of pointers would be different.  
> > if you put XP incompatible AML in separate SSDT and link it
> > only from XSDT than that would work but if incompatibility
> > is in DSDT, one would have to provide compat DSDT for RSDT
> > an incompat DSDT for XSDT.  
> 
> So don't do this.
well spec says "An ACPI-compatible OS must use the XSDT if present",
which I read as tables pointed by RSDT MUST be pointed by XSDT
as well and RSDT MUST NOT not be used.

so if we put incompatible changes in a separate SSDT and put
it only in XSDT that might work. Showstopper here is OVMF which
has issues with it as Laszlo pointed out.

Also since Windows implements only subset of spec XSDT trick
would cover only XP based versions while the rest will see and
use XSDT pointed tables which still could have incompatible
AML with some of the later windows versions.


> 
> > So far policy was don't try to run guest OS on QEMU
> > configuration that isn't supported by it.  
> 
> It's better if guests don't see some features but
> don't crash. It's not always possible of course but
> we should try to avoid this.
> 
> > For example we use VAR_PACKAGE when running with more
> > than 255 VCPUs (commit b4f4d5481) which BSODs XP.  
> 
> Yes. And it's because we violate the spec, DSDT
> should not have this stuff.
> 
> > So we can continue with that policy with out resorting to
> > using both RSDT and XSDT,
> > It would be even easier as all AML would be dynamically
> > generated and DSDT would only contain AML elements for
> > a concrete QEMU configuration.  
> 
> I'd prefer XSDT but I won't nack it if you do it in DSDT.
> I think it's not spec compliant but guests do not
> seem to care.
> 
> > > > So far we've managed keep DSDT compatible with XP while
> > > > introducing features from v2 and higher ACPI revisions as
> > > > AML that is only evaluated on demand.
> > > > We can continue doing so unless we have to unconditionally
> > > > add incompatible AML at global scope.
> > > >     
> > > 
> > > Yes.
> > >   
> > > > >     
> > > > > > Michael, Paolo, what do you think about these ideas?
> > > > > > 
> > > > > > Thanks!    
> > > > > 
> > > > > 
> > > > > 
> > > > > So using a patch below, we can add Name(PQRS, 0x0) at the top of the
> > > > > SSDT (or bottom, or add a separate SSDT just for that).  It returns the
> > > > > current offset so we can add that to the linker.
> > > > > 
> > > > > Won't work if you append the Name to the Aml structure (these can be
> > > > > nested to arbitrary depth using aml_append), so using plain GArray for
> > > > > this API makes sense to me.
> > > > >     
> > > > > --->    
> > > > > 
> > > > > acpi: add build_append_named_dword, returning an offset in buffer
> > > > > 
> > > > > This is a very limited form of support for runtime patching -
> > > > > similar in functionality to what we can do with ACPI_EXTRACT
> > > > > macros in python, but implemented in C.
> > > > > 
> > > > > This is to allow ACPI code direct access to data tables -
> > > > > which is exactly what DataTableRegion is there for, except
> > > > > no known windows release so far implements DataTableRegion.    
> > > > unsupported means Windows will BSOD, so it's practically
> > > > unusable unless MS will patch currently existing Windows
> > > > versions.    
> > > 
> > > Yes. That's why my patch allows patching SSDT without using
> > > DataTableRegion.
> > >   
> > > > Another thing about DataTableRegion is that ACPI tables are
> > > > supposed to have static content which matches checksum in
> > > > table the header while you are trying to use it for dynamic
> > > > data. It would be cleaner/more compatible to teach
> > > > bios-linker-loader to just allocate memory and patch AML
> > > > with the allocated address.    
> > > 
> > > Yes - if address is static, you need to put it outside
> > > the table. Can come right before or right after this.
> > >   
> > > > Also if OperationRegion() is used, then one has to patch
> > > > DefOpRegion directly as RegionOffset must be Integer,
> > > > using variable names is not permitted there.    
> > > 
> > > I am not sure the comment was understood correctly.
> > > The comment says really "we can't use DataTableRegion
> > > so here is an alternative".  
> > so how are you going to access data at which patched
> > NameString point to?
> > for that you'd need a normal patched OperationRegion
> > as well since DataTableRegion isn't usable.  
> 
> For VMGENID you would patch the method that
> returns the address - you do not need an op region
> as you never access it.
> 
> I don't know about NVDIMM. Maybe OperationRegion can
> use the patched NameString? Will need some thought.
> 
> > >   
> > > >     
> > > > > 
> > > > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > > > > 
> > > > > ---
> > > > > 
> > > > > diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
> > > > > index 1b632dc..f8998ea 100644
> > > > > --- a/include/hw/acpi/aml-build.h
> > > > > +++ b/include/hw/acpi/aml-build.h
> > > > > @@ -286,4 +286,7 @@ void acpi_build_tables_cleanup(AcpiBuildTables *tables, bool mfre);
> > > > >  void
> > > > >  build_rsdt(GArray *table_data, GArray *linker, GArray *table_offsets);
> > > > >  
> > > > > +int
> > > > > +build_append_named_dword(GArray *array, const char *name_format, ...);
> > > > > +
> > > > >  #endif
> > > > > diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
> > > > > index 0d4b324..7f9fa65 100644
> > > > > --- a/hw/acpi/aml-build.c
> > > > > +++ b/hw/acpi/aml-build.c
> > > > > @@ -262,6 +262,32 @@ static void build_append_int(GArray *table, uint64_t value)
> > > > >      }
> > > > >  }
> > > > >  
> > > > > +/* Build NAME(XXXX, 0x0) where 0x0 is encoded as a qword,
> > > > > + * and return the offset to 0x0 for runtime patching.
> > > > > + *
> > > > > + * Warning: runtime patching is best avoided. Only use this as
> > > > > + * a replacement for DataTableRegion (for guests that don't
> > > > > + * support it).
> > > > > + */
> > > > > +int
> > > > > +build_append_named_qword(GArray *array, const char *name_format, ...)
> > > > > +{
> > > > > +    int offset;
> > > > > +    va_list ap;
> > > > > +
> > > > > +    va_start(ap, name_format);
> > > > > +    build_append_namestringv(array, name_format, ap);
> > > > > +    va_end(ap);
> > > > > +
> > > > > +    build_append_byte(array, 0x0E); /* QWordPrefix */
> > > > > +
> > > > > +    offset = array->len;
> > > > > +    build_append_int_noprefix(array, 0x0, 8);
> > > > > +    assert(array->len == offset + 8);
> > > > > +
> > > > > +    return offset;
> > > > > +}
> > > > > +
> > > > >  static GPtrArray *alloc_list;
> > > > >  
> > > > >  static Aml *aml_alloc(void)
> > > > > 
> > > > >     
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe kvm" in
> > > the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at  http://vger.kernel.org/majordomo-info.html  
>

On Thu, Jan 07, 2016 at 11:30:25AM +0100, Igor Mammedov wrote:
> On Tue, 5 Jan 2016 18:43:02 +0200
> "Michael S. Tsirkin" <mst@redhat.com> wrote:
> 
> > On Tue, Jan 05, 2016 at 05:30:25PM +0100, Igor Mammedov wrote:
> > > > > bios-linker-loader is a great interface for initializing some
> > > > > guest owned data and linking it together but I think it adds
> > > > > unnecessary complexity and is misused if it's used to handle
> > > > > device owned data/on device memory in this and VMGID cases.    
> > > > 
> > > > I want a generic interface for guest to enumerate these things.  linker
> > > > seems quite reasonable but if you see a reason why it won't do, or want
> > > > to propose a better interface, fine.
> > > > 
> > > > PCI would do, too - though windows guys had concerns about
> > > > returning PCI BARs from ACPI.  
> > > There were potential issues with pSeries bootloader that treated
> > > PCI_CLASS_MEMORY_RAM as conventional RAM but it was fixed.
> > > Could you point out to discussion about windows issues?
> > > 
> > > What VMGEN patches that used PCI for mapping purposes were
> > > stuck at, was that it was suggested to use PCI_CLASS_MEMORY_RAM
> > > class id but we couldn't agree on it.
> > > 
> > > VMGEN v13 with full discussion is here
> > > https://patchwork.ozlabs.org/patch/443554/
> > > So to continue with this route we would need to pick some other
> > > driver less class id so windows won't prompt for driver or
> > > maybe supply our own driver stub to guarantee that no one
> > > would touch it. Any suggestions?  
> > 
> > Pick any device/vendor id pair for which windows specifies no driver.
> > There's a small risk that this will conflict with some
> > guest but I think it's minimal.
> device/vendor id pair was QEMU specific so doesn't conflicts with anything
> issue we were trying to solve was to prevent windows asking for driver
> even though it does so only once if told not to ask again.
> 
> That's why PCI_CLASS_MEMORY_RAM was selected as it's generic driver-less
> device descriptor in INF file which matches as the last resort if
> there isn't any other diver that's matched device by device/vendor id pair.

I think this is the only class in this inf.
If you can't use it, you must use an existing device/vendor id pair,
there's some risk involved but probably not much.

> > 
> > 
> > > > 
> > > >   
> > > > > There was RFC on list to make BIOS boot from NVDIMM already
> > > > > doing some ACPI table lookup/parsing. Now if they were forced
> > > > > to also parse and execute AML to initialize QEMU with guest
> > > > > allocated address that would complicate them quite a bit.    
> > > > 
> > > > If they just need to find a table by name, it won't be
> > > > too bad, would it?  
> > > that's what they were doing scanning memory for static NVDIMM table.
> > > However if it were DataTable, BIOS side would have to execute
> > > AML so that the table address could be told to QEMU.  
> > 
> > Not at all. You can find any table by its signature without
> > parsing AML.
> yep, and then BIOS would need to tell its address to QEMU
> writing to IO port which is allocated statically in QEMU
> for this purpose and is described in AML only on guest side.

io ports are an ABI too but they are way easier to
maintain.

> > 
> > 
> > > In case of direct mapping or PCI BAR there is no need to initialize
> > > QEMU side from AML.
> > > That also saves us IO port where this address should be written
> > > if bios-linker-loader approach is used.
> > >   
> > > >   
> > > > > While with NVDIMM control memory region mapped directly by QEMU,
> > > > > respective patches don't need in any way to initialize QEMU,
> > > > > all they would need just read necessary data from control region.
> > > > > 
> > > > > Also using bios-linker-loader takes away some usable RAM
> > > > > from guest and in the end that doesn't scale,
> > > > > the more devices I add the less usable RAM is left for guest OS
> > > > > while all the device needs is a piece of GPA address space
> > > > > that would belong to it.    
> > > > 
> > > > I don't get this comment. I don't think it's MMIO that is wanted.
> > > > If it's backed by qemu virtual memory then it's RAM.  
> > > Then why don't allocate video card VRAM the same way and try to explain
> > > user that a guest started with '-m 128 -device cirrus-vga,vgamem_mb=64Mb'
> > > only has 64Mb of available RAM because of we think that on device VRAM
> > > is also RAM.
> > > 
> > > Maybe I've used MMIO term wrongly here but it roughly reflects the idea
> > > that on device memory (whether it's VRAM, NVDIMM control block or VMGEN
> > > area) is not allocated from guest's usable RAM (as described in E820)
> > > but rather directly mapped in guest's GPA and doesn't consume available
> > > RAM as guest sees it. That's also the way it's done on real hardware.
> > > 
> > > What we need in case of VMGEN ID and NVDIMM is on device memory
> > > that could be directly accessed by guest.
> > > Both direct mapping or PCI BAR do that job and we could use simple
> > > static AML without any patching.  
> > 
> > At least with VMGEN the issue is that there's an AML method
> > that returns the physical address.
> > Then if guest OS moves the BAR (which is legal), it will break
> > since caller has no way to know it's related to the BAR.
> I've found a following MS doc "Firmware Allocation of PCI Device Resources in Windows". It looks like when MS implemented resource rebalancing in
> Vista they pushed a compat change to PCI specs.
> That ECN is called "Ignore PCI Boot Configuration_DSM Function"
> and can be found here:
> https://pcisig.com/sites/default/files/specification_documents/ECR-Ignorebootconfig-final.pdf
> 
> It looks like it's possible to forbid rebalancing per
> device/bridge if it has _DMS method that returns "do not
> ignore the boot configuration of PCI resources".

I'll have to study this but we don't want that
globally, do we?
This restricts hotplug functionality significantly.

>  
> > > > > > 
> > > > > > See patch at the bottom that might be handy.
> > > > > >     
> > > > > > > he also innovated a way to use 64-bit address in DSDT/SSDT.rev = 1:
> > > > > > > | when writing ASL one shall make sure that only XP supported
> > > > > > > | features are in global scope, which is evaluated when tables
> > > > > > > | are loaded and features of rev2 and higher are inside methods.
> > > > > > > | That way XP doesn't crash as far as it doesn't evaluate unsupported
> > > > > > > | opcode and one can guard those opcodes checking _REV object if neccesary.
> > > > > > > (https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg01010.html)    
> > > > > > 
> > > > > > Yes, this technique works.
> > > > > > 
> > > > > > An alternative is to add an XSDT, XP ignores that.
> > > > > > XSDT at the moment breaks OVMF (because it loads both
> > > > > > the RSDT and the XSDT, which is wrong), but I think
> > > > > > Laszlo was working on a fix for that.    
> > > > > Using XSDT would increase ACPI tables occupied RAM
> > > > > as it would duplicate DSDT + non XP supported AML
> > > > > at global namespace.    
> > > > 
> > > > Not at all - I posted patches linking to same
> > > > tables from both RSDT and XSDT at some point.
> > > > Only the list of pointers would be different.  
> > > if you put XP incompatible AML in separate SSDT and link it
> > > only from XSDT than that would work but if incompatibility
> > > is in DSDT, one would have to provide compat DSDT for RSDT
> > > an incompat DSDT for XSDT.  
> > 
> > So don't do this.
> well spec says "An ACPI-compatible OS must use the XSDT if present",
> which I read as tables pointed by RSDT MUST be pointed by XSDT
> as well and RSDT MUST NOT not be used.
>
> so if we put incompatible changes in a separate SSDT and put
> it only in XSDT that might work. Showstopper here is OVMF which
> has issues with it as Laszlo pointed out.

But that's just a bug.

> Also since Windows implements only subset of spec XSDT trick
> would cover only XP based versions while the rest will see and
> use XSDT pointed tables which still could have incompatible
> AML with some of the later windows versions.

We'll have to see what these are exactly.
If it's methods in SSDT we can check the version supported
by the ASPM.

> 
> > 
> > > So far policy was don't try to run guest OS on QEMU
> > > configuration that isn't supported by it.  
> > 
> > It's better if guests don't see some features but
> > don't crash. It's not always possible of course but
> > we should try to avoid this.
> > 
> > > For example we use VAR_PACKAGE when running with more
> > > than 255 VCPUs (commit b4f4d5481) which BSODs XP.  
> > 
> > Yes. And it's because we violate the spec, DSDT
> > should not have this stuff.
> > 
> > > So we can continue with that policy with out resorting to
> > > using both RSDT and XSDT,
> > > It would be even easier as all AML would be dynamically
> > > generated and DSDT would only contain AML elements for
> > > a concrete QEMU configuration.  
> > 
> > I'd prefer XSDT but I won't nack it if you do it in DSDT.
> > I think it's not spec compliant but guests do not
> > seem to care.
> > 
> > > > > So far we've managed keep DSDT compatible with XP while
> > > > > introducing features from v2 and higher ACPI revisions as
> > > > > AML that is only evaluated on demand.
> > > > > We can continue doing so unless we have to unconditionally
> > > > > add incompatible AML at global scope.
> > > > >     
> > > > 
> > > > Yes.
> > > >   
> > > > > >     
> > > > > > > Michael, Paolo, what do you think about these ideas?
> > > > > > > 
> > > > > > > Thanks!    
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > So using a patch below, we can add Name(PQRS, 0x0) at the top of the
> > > > > > SSDT (or bottom, or add a separate SSDT just for that).  It returns the
> > > > > > current offset so we can add that to the linker.
> > > > > > 
> > > > > > Won't work if you append the Name to the Aml structure (these can be
> > > > > > nested to arbitrary depth using aml_append), so using plain GArray for
> > > > > > this API makes sense to me.
> > > > > >     
> > > > > > --->    
> > > > > > 
> > > > > > acpi: add build_append_named_dword, returning an offset in buffer
> > > > > > 
> > > > > > This is a very limited form of support for runtime patching -
> > > > > > similar in functionality to what we can do with ACPI_EXTRACT
> > > > > > macros in python, but implemented in C.
> > > > > > 
> > > > > > This is to allow ACPI code direct access to data tables -
> > > > > > which is exactly what DataTableRegion is there for, except
> > > > > > no known windows release so far implements DataTableRegion.    
> > > > > unsupported means Windows will BSOD, so it's practically
> > > > > unusable unless MS will patch currently existing Windows
> > > > > versions.    
> > > > 
> > > > Yes. That's why my patch allows patching SSDT without using
> > > > DataTableRegion.
> > > >   
> > > > > Another thing about DataTableRegion is that ACPI tables are
> > > > > supposed to have static content which matches checksum in
> > > > > table the header while you are trying to use it for dynamic
> > > > > data. It would be cleaner/more compatible to teach
> > > > > bios-linker-loader to just allocate memory and patch AML
> > > > > with the allocated address.    
> > > > 
> > > > Yes - if address is static, you need to put it outside
> > > > the table. Can come right before or right after this.
> > > >   
> > > > > Also if OperationRegion() is used, then one has to patch
> > > > > DefOpRegion directly as RegionOffset must be Integer,
> > > > > using variable names is not permitted there.    
> > > > 
> > > > I am not sure the comment was understood correctly.
> > > > The comment says really "we can't use DataTableRegion
> > > > so here is an alternative".  
> > > so how are you going to access data at which patched
> > > NameString point to?
> > > for that you'd need a normal patched OperationRegion
> > > as well since DataTableRegion isn't usable.  
> > 
> > For VMGENID you would patch the method that
> > returns the address - you do not need an op region
> > as you never access it.
> > 
> > I don't know about NVDIMM. Maybe OperationRegion can
> > use the patched NameString? Will need some thought.
> > 
> > > >   
> > > > >     
> > > > > > 
> > > > > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > > > > > 
> > > > > > ---
> > > > > > 
> > > > > > diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
> > > > > > index 1b632dc..f8998ea 100644
> > > > > > --- a/include/hw/acpi/aml-build.h
> > > > > > +++ b/include/hw/acpi/aml-build.h
> > > > > > @@ -286,4 +286,7 @@ void acpi_build_tables_cleanup(AcpiBuildTables *tables, bool mfre);
> > > > > >  void
> > > > > >  build_rsdt(GArray *table_data, GArray *linker, GArray *table_offsets);
> > > > > >  
> > > > > > +int
> > > > > > +build_append_named_dword(GArray *array, const char *name_format, ...);
> > > > > > +
> > > > > >  #endif
> > > > > > diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
> > > > > > index 0d4b324..7f9fa65 100644
> > > > > > --- a/hw/acpi/aml-build.c
> > > > > > +++ b/hw/acpi/aml-build.c
> > > > > > @@ -262,6 +262,32 @@ static void build_append_int(GArray *table, uint64_t value)
> > > > > >      }
> > > > > >  }
> > > > > >  
> > > > > > +/* Build NAME(XXXX, 0x0) where 0x0 is encoded as a qword,
> > > > > > + * and return the offset to 0x0 for runtime patching.
> > > > > > + *
> > > > > > + * Warning: runtime patching is best avoided. Only use this as
> > > > > > + * a replacement for DataTableRegion (for guests that don't
> > > > > > + * support it).
> > > > > > + */
> > > > > > +int
> > > > > > +build_append_named_qword(GArray *array, const char *name_format, ...)
> > > > > > +{
> > > > > > +    int offset;
> > > > > > +    va_list ap;
> > > > > > +
> > > > > > +    va_start(ap, name_format);
> > > > > > +    build_append_namestringv(array, name_format, ap);
> > > > > > +    va_end(ap);
> > > > > > +
> > > > > > +    build_append_byte(array, 0x0E); /* QWordPrefix */
> > > > > > +
> > > > > > +    offset = array->len;
> > > > > > +    build_append_int_noprefix(array, 0x0, 8);
> > > > > > +    assert(array->len == offset + 8);
> > > > > > +
> > > > > > +    return offset;
> > > > > > +}
> > > > > > +
> > > > > >  static GPtrArray *alloc_list;
> > > > > >  
> > > > > >  static Aml *aml_alloc(void)
> > > > > > 
> > > > > >     
> > > > --
> > > > To unsubscribe from this list: send the line "unsubscribe kvm" in
> > > > the body of a message to majordomo@vger.kernel.org
> > > > More majordomo info at  http://vger.kernel.org/majordomo-info.html  
> >

On Thu, 7 Jan 2016 12:54:30 +0200
"Michael S. Tsirkin" <mst@redhat.com> wrote:

> On Thu, Jan 07, 2016 at 11:30:25AM +0100, Igor Mammedov wrote:
> > On Tue, 5 Jan 2016 18:43:02 +0200
> > "Michael S. Tsirkin" <mst@redhat.com> wrote:
> >   
> > > On Tue, Jan 05, 2016 at 05:30:25PM +0100, Igor Mammedov wrote:  
> > > > > > bios-linker-loader is a great interface for initializing some
> > > > > > guest owned data and linking it together but I think it adds
> > > > > > unnecessary complexity and is misused if it's used to handle
> > > > > > device owned data/on device memory in this and VMGID cases.      
> > > > > 
> > > > > I want a generic interface for guest to enumerate these things.  linker
> > > > > seems quite reasonable but if you see a reason why it won't do, or want
> > > > > to propose a better interface, fine.
> > > > > 
> > > > > PCI would do, too - though windows guys had concerns about
> > > > > returning PCI BARs from ACPI.    
> > > > There were potential issues with pSeries bootloader that treated
> > > > PCI_CLASS_MEMORY_RAM as conventional RAM but it was fixed.
> > > > Could you point out to discussion about windows issues?
> > > > 
> > > > What VMGEN patches that used PCI for mapping purposes were
> > > > stuck at, was that it was suggested to use PCI_CLASS_MEMORY_RAM
> > > > class id but we couldn't agree on it.
> > > > 
> > > > VMGEN v13 with full discussion is here
> > > > https://patchwork.ozlabs.org/patch/443554/
> > > > So to continue with this route we would need to pick some other
> > > > driver less class id so windows won't prompt for driver or
> > > > maybe supply our own driver stub to guarantee that no one
> > > > would touch it. Any suggestions?    
> > > 
> > > Pick any device/vendor id pair for which windows specifies no driver.
> > > There's a small risk that this will conflict with some
> > > guest but I think it's minimal.  
> > device/vendor id pair was QEMU specific so doesn't conflicts with anything
> > issue we were trying to solve was to prevent windows asking for driver
> > even though it does so only once if told not to ask again.
> > 
> > That's why PCI_CLASS_MEMORY_RAM was selected as it's generic driver-less
> > device descriptor in INF file which matches as the last resort if
> > there isn't any other diver that's matched device by device/vendor id pair.  
> 
> I think this is the only class in this inf.
> If you can't use it, you must use an existing device/vendor id pair,
> there's some risk involved but probably not much.
I can't wrap my head around this answer, could you rephrase it?

As far as I see we can use PCI_CLASS_MEMORY_RAM with qemu's device/vendor ids.
In that case Windows associates it with dummy "Generic RAM controller".

The same happens with some NVIDIA cards if NVIDIA drivers are not installed,
if we install drivers then Windows binds NVIDIA's PCI_CLASS_MEMORY_RAM with
concrete driver that manages VRAM the way NVIDIA wants it.

So I think we can use it with low risk.

If we use existing device/vendor id pair with some driver then driver
will fail to initialize and as minimum we would get device marked as
not working in Device-Manager. Any way if you have in mind a concrete
existing device/vendor id pair feel free to suggest it.

> 
> > > 
> > >   
> > > > > 
> > > > >     
> > > > > > There was RFC on list to make BIOS boot from NVDIMM already
> > > > > > doing some ACPI table lookup/parsing. Now if they were forced
> > > > > > to also parse and execute AML to initialize QEMU with guest
> > > > > > allocated address that would complicate them quite a bit.      
> > > > > 
> > > > > If they just need to find a table by name, it won't be
> > > > > too bad, would it?    
> > > > that's what they were doing scanning memory for static NVDIMM table.
> > > > However if it were DataTable, BIOS side would have to execute
> > > > AML so that the table address could be told to QEMU.    
> > > 
> > > Not at all. You can find any table by its signature without
> > > parsing AML.  
> > yep, and then BIOS would need to tell its address to QEMU
> > writing to IO port which is allocated statically in QEMU
> > for this purpose and is described in AML only on guest side.  
> 
> io ports are an ABI too but they are way easier to
> maintain.
It's pretty much the same as GPA addresses only it's much more limited resource.
Otherwise one has to do the same tricks to maintain ABI.

> 
> > > 
> > >   
> > > > In case of direct mapping or PCI BAR there is no need to initialize
> > > > QEMU side from AML.
> > > > That also saves us IO port where this address should be written
> > > > if bios-linker-loader approach is used.
> > > >     
> > > > >     
> > > > > > While with NVDIMM control memory region mapped directly by QEMU,
> > > > > > respective patches don't need in any way to initialize QEMU,
> > > > > > all they would need just read necessary data from control region.
> > > > > > 
> > > > > > Also using bios-linker-loader takes away some usable RAM
> > > > > > from guest and in the end that doesn't scale,
> > > > > > the more devices I add the less usable RAM is left for guest OS
> > > > > > while all the device needs is a piece of GPA address space
> > > > > > that would belong to it.      
> > > > > 
> > > > > I don't get this comment. I don't think it's MMIO that is wanted.
> > > > > If it's backed by qemu virtual memory then it's RAM.    
> > > > Then why don't allocate video card VRAM the same way and try to explain
> > > > user that a guest started with '-m 128 -device cirrus-vga,vgamem_mb=64Mb'
> > > > only has 64Mb of available RAM because of we think that on device VRAM
> > > > is also RAM.
> > > > 
> > > > Maybe I've used MMIO term wrongly here but it roughly reflects the idea
> > > > that on device memory (whether it's VRAM, NVDIMM control block or VMGEN
> > > > area) is not allocated from guest's usable RAM (as described in E820)
> > > > but rather directly mapped in guest's GPA and doesn't consume available
> > > > RAM as guest sees it. That's also the way it's done on real hardware.
> > > > 
> > > > What we need in case of VMGEN ID and NVDIMM is on device memory
> > > > that could be directly accessed by guest.
> > > > Both direct mapping or PCI BAR do that job and we could use simple
> > > > static AML without any patching.    
> > > 
> > > At least with VMGEN the issue is that there's an AML method
> > > that returns the physical address.
> > > Then if guest OS moves the BAR (which is legal), it will break
> > > since caller has no way to know it's related to the BAR.  
> > I've found a following MS doc "Firmware Allocation of PCI Device Resources in Windows". It looks like when MS implemented resource rebalancing in
> > Vista they pushed a compat change to PCI specs.
> > That ECN is called "Ignore PCI Boot Configuration_DSM Function"
> > and can be found here:
> > https://pcisig.com/sites/default/files/specification_documents/ECR-Ignorebootconfig-final.pdf
> > 
> > It looks like it's possible to forbid rebalancing per
> > device/bridge if it has _DMS method that returns "do not
> > ignore the boot configuration of PCI resources".  
> 
> I'll have to study this but we don't want that
> globally, do we?
no need to do it globally, adding _DSM to a device, we don't wish
to be rebalanced, should be sufficient to lock down specific resources.

actually existence of spec implies that if there is a boot configured
device with resources described in ACPI table and there isn't _DSM
method enabling rebalancing for it, then rebalancing is not permitted.
It should be easy to make an experiment to verify what Windows would do.

So if this approach would work and we agree on going with it, I could work
on redoing VMGENv13 series using _DSM as described.
That would simplify implementing this kind of devices vs bios-linker approach i.e.:
 - free RAM occupied by linker blob
 - free IO port
 - avoid 2 or 3 layers of indirection - which makes understanding of code much easier
 - avoid runtime AML patching and simplify AML and its composing parts
 - there won't be need for BIOS to get IO port from fw_cfg and write
   there GPA as well no need for table lookup.
 - much easier to write unit tests, i.e. use the same qtest device testing
   technique without necessity of running actual guest code.
   i.e. no binary code blobs like we have for running bios-tables test in TCG mode.


> This restricts hotplug functionality significantly.
> 
> >    
> > > > > > > 
> > > > > > > See patch at the bottom that might be handy.
> > > > > > >       
> > > > > > > > he also innovated a way to use 64-bit address in DSDT/SSDT.rev = 1:
> > > > > > > > | when writing ASL one shall make sure that only XP supported
> > > > > > > > | features are in global scope, which is evaluated when tables
> > > > > > > > | are loaded and features of rev2 and higher are inside methods.
> > > > > > > > | That way XP doesn't crash as far as it doesn't evaluate unsupported
> > > > > > > > | opcode and one can guard those opcodes checking _REV object if neccesary.
> > > > > > > > (https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg01010.html)      
> > > > > > > 
> > > > > > > Yes, this technique works.
> > > > > > > 
> > > > > > > An alternative is to add an XSDT, XP ignores that.
> > > > > > > XSDT at the moment breaks OVMF (because it loads both
> > > > > > > the RSDT and the XSDT, which is wrong), but I think
> > > > > > > Laszlo was working on a fix for that.      
> > > > > > Using XSDT would increase ACPI tables occupied RAM
> > > > > > as it would duplicate DSDT + non XP supported AML
> > > > > > at global namespace.      
> > > > > 
> > > > > Not at all - I posted patches linking to same
> > > > > tables from both RSDT and XSDT at some point.
> > > > > Only the list of pointers would be different.    
> > > > if you put XP incompatible AML in separate SSDT and link it
> > > > only from XSDT than that would work but if incompatibility
> > > > is in DSDT, one would have to provide compat DSDT for RSDT
> > > > an incompat DSDT for XSDT.    
> > > 
> > > So don't do this.  
> > well spec says "An ACPI-compatible OS must use the XSDT if present",
> > which I read as tables pointed by RSDT MUST be pointed by XSDT
> > as well and RSDT MUST NOT not be used.
> >
> > so if we put incompatible changes in a separate SSDT and put
> > it only in XSDT that might work. Showstopper here is OVMF which
> > has issues with it as Laszlo pointed out.  
> 
> But that's just a bug.
> 
> > Also since Windows implements only subset of spec XSDT trick
> > would cover only XP based versions while the rest will see and
> > use XSDT pointed tables which still could have incompatible
> > AML with some of the later windows versions.  
> 
> We'll have to see what these are exactly.
> If it's methods in SSDT we can check the version supported
> by the ASPM.
I see only VAR_PACKAGE as such object so far.

64-bit PCI0._CRS probably won't crash 32-bit Vista and later as
it should be able to parse 64-bit Integers as defined by ACPI 2.0.

> 
> >   
> > >   
> > > > So far policy was don't try to run guest OS on QEMU
> > > > configuration that isn't supported by it.    
> > > 
> > > It's better if guests don't see some features but
> > > don't crash. It's not always possible of course but
> > > we should try to avoid this.
> > >   
> > > > For example we use VAR_PACKAGE when running with more
> > > > than 255 VCPUs (commit b4f4d5481) which BSODs XP.    
> > > 
> > > Yes. And it's because we violate the spec, DSDT
> > > should not have this stuff.
> > >   
> > > > So we can continue with that policy with out resorting to
> > > > using both RSDT and XSDT,
> > > > It would be even easier as all AML would be dynamically
> > > > generated and DSDT would only contain AML elements for
> > > > a concrete QEMU configuration.    
> > > 
> > > I'd prefer XSDT but I won't nack it if you do it in DSDT.
> > > I think it's not spec compliant but guests do not
> > > seem to care.
> > >   
> > > > > > So far we've managed keep DSDT compatible with XP while
> > > > > > introducing features from v2 and higher ACPI revisions as
> > > > > > AML that is only evaluated on demand.
> > > > > > We can continue doing so unless we have to unconditionally
> > > > > > add incompatible AML at global scope.
> > > > > >       
> > > > > 
> > > > > Yes.
> > > > >     
> > > > > > >       
> > > > > > > > Michael, Paolo, what do you think about these ideas?
> > > > > > > > 
> > > > > > > > Thanks!      
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > So using a patch below, we can add Name(PQRS, 0x0) at the top of the
> > > > > > > SSDT (or bottom, or add a separate SSDT just for that).  It returns the
> > > > > > > current offset so we can add that to the linker.
> > > > > > > 
> > > > > > > Won't work if you append the Name to the Aml structure (these can be
> > > > > > > nested to arbitrary depth using aml_append), so using plain GArray for
> > > > > > > this API makes sense to me.
> > > > > > >       
> > > > > > > --->      
> > > > > > > 
> > > > > > > acpi: add build_append_named_dword, returning an offset in buffer
> > > > > > > 
> > > > > > > This is a very limited form of support for runtime patching -
> > > > > > > similar in functionality to what we can do with ACPI_EXTRACT
> > > > > > > macros in python, but implemented in C.
> > > > > > > 
> > > > > > > This is to allow ACPI code direct access to data tables -
> > > > > > > which is exactly what DataTableRegion is there for, except
> > > > > > > no known windows release so far implements DataTableRegion.      
> > > > > > unsupported means Windows will BSOD, so it's practically
> > > > > > unusable unless MS will patch currently existing Windows
> > > > > > versions.      
> > > > > 
> > > > > Yes. That's why my patch allows patching SSDT without using
> > > > > DataTableRegion.
> > > > >     
> > > > > > Another thing about DataTableRegion is that ACPI tables are
> > > > > > supposed to have static content which matches checksum in
> > > > > > table the header while you are trying to use it for dynamic
> > > > > > data. It would be cleaner/more compatible to teach
> > > > > > bios-linker-loader to just allocate memory and patch AML
> > > > > > with the allocated address.      
> > > > > 
> > > > > Yes - if address is static, you need to put it outside
> > > > > the table. Can come right before or right after this.
> > > > >     
> > > > > > Also if OperationRegion() is used, then one has to patch
> > > > > > DefOpRegion directly as RegionOffset must be Integer,
> > > > > > using variable names is not permitted there.      
> > > > > 
> > > > > I am not sure the comment was understood correctly.
> > > > > The comment says really "we can't use DataTableRegion
> > > > > so here is an alternative".    
> > > > so how are you going to access data at which patched
> > > > NameString point to?
> > > > for that you'd need a normal patched OperationRegion
> > > > as well since DataTableRegion isn't usable.    
> > > 
> > > For VMGENID you would patch the method that
> > > returns the address - you do not need an op region
> > > as you never access it.
> > > 
> > > I don't know about NVDIMM. Maybe OperationRegion can
> > > use the patched NameString? Will need some thought.
> > >   
> > > > >     
> > > > > >       
> > > > > > > 
> > > > > > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > > > > > > 
> > > > > > > ---
> > > > > > > 
> > > > > > > diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
> > > > > > > index 1b632dc..f8998ea 100644
> > > > > > > --- a/include/hw/acpi/aml-build.h
> > > > > > > +++ b/include/hw/acpi/aml-build.h
> > > > > > > @@ -286,4 +286,7 @@ void acpi_build_tables_cleanup(AcpiBuildTables *tables, bool mfre);
> > > > > > >  void
> > > > > > >  build_rsdt(GArray *table_data, GArray *linker, GArray *table_offsets);
> > > > > > >  
> > > > > > > +int
> > > > > > > +build_append_named_dword(GArray *array, const char *name_format, ...);
> > > > > > > +
> > > > > > >  #endif
> > > > > > > diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
> > > > > > > index 0d4b324..7f9fa65 100644
> > > > > > > --- a/hw/acpi/aml-build.c
> > > > > > > +++ b/hw/acpi/aml-build.c
> > > > > > > @@ -262,6 +262,32 @@ static void build_append_int(GArray *table, uint64_t value)
> > > > > > >      }
> > > > > > >  }
> > > > > > >  
> > > > > > > +/* Build NAME(XXXX, 0x0) where 0x0 is encoded as a qword,
> > > > > > > + * and return the offset to 0x0 for runtime patching.
> > > > > > > + *
> > > > > > > + * Warning: runtime patching is best avoided. Only use this as
> > > > > > > + * a replacement for DataTableRegion (for guests that don't
> > > > > > > + * support it).
> > > > > > > + */
> > > > > > > +int
> > > > > > > +build_append_named_qword(GArray *array, const char *name_format, ...)
> > > > > > > +{
> > > > > > > +    int offset;
> > > > > > > +    va_list ap;
> > > > > > > +
> > > > > > > +    va_start(ap, name_format);
> > > > > > > +    build_append_namestringv(array, name_format, ap);
> > > > > > > +    va_end(ap);
> > > > > > > +
> > > > > > > +    build_append_byte(array, 0x0E); /* QWordPrefix */
> > > > > > > +
> > > > > > > +    offset = array->len;
> > > > > > > +    build_append_int_noprefix(array, 0x0, 8);
> > > > > > > +    assert(array->len == offset + 8);
> > > > > > > +
> > > > > > > +    return offset;
> > > > > > > +}
> > > > > > > +
> > > > > > >  static GPtrArray *alloc_list;
> > > > > > >  
> > > > > > >  static Aml *aml_alloc(void)
> > > > > > > 
> > > > > > >       
> > > > > --
> > > > > To unsubscribe from this list: send the line "unsubscribe kvm" in
> > > > > the body of a message to majordomo@vger.kernel.org
> > > > > More majordomo info at  http://vger.kernel.org/majordomo-info.html    
> > >

On Mon, 4 Jan 2016 21:17:31 +0100
Laszlo Ersek <lersek@redhat.com> wrote:

> Michael CC'd me on the grandparent of the email below. I'll try to add
> my thoughts in a single go, with regard to OVMF.
> 
> On 12/30/15 20:52, Michael S. Tsirkin wrote:
> > On Wed, Dec 30, 2015 at 04:55:54PM +0100, Igor Mammedov wrote:  
> >> On Mon, 28 Dec 2015 14:50:15 +0200
> >> "Michael S. Tsirkin" <mst@redhat.com> wrote:
> >>  
> >>> On Mon, Dec 28, 2015 at 10:39:04AM +0800, Xiao Guangrong wrote:  
> >>>>
> >>>> Hi Michael, Paolo,
> >>>>
> >>>> Now it is the time to return to the challenge that how to reserve guest
> >>>> physical region internally used by ACPI.
> >>>>
> >>>> Igor suggested that:
> >>>> | An alternative place to allocate reserve from could be high memory.
> >>>> | For pc we have "reserved-memory-end" which currently makes sure
> >>>> | that hotpluggable memory range isn't used by firmware
> >>>> (https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg00926.html)  
> 
> OVMF has no support for the "reserved-memory-end" fw_cfg file. The
> reason is that nobody wrote that patch, nor asked for the patch to be
> written. (Not implying that just requesting the patch would be
> sufficient for the patch to be written.)
Hijacking this part of thread to check if OVMF would work with memory-hotplug
and if it needs "reserved-memory-end" support at all.

How OVMF determines which GPA ranges to use for initializing PCI BARs
at boot time, more specifically 64-bit BARs.

On 01/07/16 11:54, Michael S. Tsirkin wrote:
> On Thu, Jan 07, 2016 at 11:30:25AM +0100, Igor Mammedov wrote:
>> On Tue, 5 Jan 2016 18:43:02 +0200
>> "Michael S. Tsirkin" <mst@redhat.com> wrote:
>>
>>> On Tue, Jan 05, 2016 at 05:30:25PM +0100, Igor Mammedov wrote:

...

>>>>>>> An alternative is to add an XSDT, XP ignores that.
>>>>>>> XSDT at the moment breaks OVMF (because it loads both
>>>>>>> the RSDT and the XSDT, which is wrong), but I think
>>>>>>> Laszlo was working on a fix for that.    
>>>>>> Using XSDT would increase ACPI tables occupied RAM
>>>>>> as it would duplicate DSDT + non XP supported AML
>>>>>> at global namespace.    
>>>>>
>>>>> Not at all - I posted patches linking to same
>>>>> tables from both RSDT and XSDT at some point.
>>>>> Only the list of pointers would be different.  
>>>> if you put XP incompatible AML in separate SSDT and link it
>>>> only from XSDT than that would work but if incompatibility
>>>> is in DSDT, one would have to provide compat DSDT for RSDT
>>>> an incompat DSDT for XSDT.  
>>>
>>> So don't do this.
>> well spec says "An ACPI-compatible OS must use the XSDT if present",
>> which I read as tables pointed by RSDT MUST be pointed by XSDT
>> as well and RSDT MUST NOT not be used.
>>
>> so if we put incompatible changes in a separate SSDT and put
>> it only in XSDT that might work. Showstopper here is OVMF which
>> has issues with it as Laszlo pointed out.
> 
> But that's just a bug.

Yes, but the bug (actually: lack of feature) is in the UEFI
specification. The current EFI_ACPI_TABLE_PROTOCOL implementation in
edk2 conforms to the specification. In order to expose the functionality
that the above trick needs, the UEFI spec has to be changed. In my
(limited, admittedly) experience, that's an uphill battle.

[...]

On 01/07/16 14:42, Igor Mammedov wrote:
> On Thu, 7 Jan 2016 12:54:30 +0200
> "Michael S. Tsirkin" <mst@redhat.com> wrote:
> 
>> On Thu, Jan 07, 2016 at 11:30:25AM +0100, Igor Mammedov wrote:
>>> On Tue, 5 Jan 2016 18:43:02 +0200
>>> "Michael S. Tsirkin" <mst@redhat.com> wrote:
>>>   
>>>> On Tue, Jan 05, 2016 at 05:30:25PM +0100, Igor Mammedov wrote:  
>>>>>>> bios-linker-loader is a great interface for initializing some
>>>>>>> guest owned data and linking it together but I think it adds
>>>>>>> unnecessary complexity and is misused if it's used to handle
>>>>>>> device owned data/on device memory in this and VMGID cases.      
>>>>>>
>>>>>> I want a generic interface for guest to enumerate these things.  linker
>>>>>> seems quite reasonable but if you see a reason why it won't do, or want
>>>>>> to propose a better interface, fine.
>>>>>>
>>>>>> PCI would do, too - though windows guys had concerns about
>>>>>> returning PCI BARs from ACPI.    
>>>>> There were potential issues with pSeries bootloader that treated
>>>>> PCI_CLASS_MEMORY_RAM as conventional RAM but it was fixed.
>>>>> Could you point out to discussion about windows issues?
>>>>>
>>>>> What VMGEN patches that used PCI for mapping purposes were
>>>>> stuck at, was that it was suggested to use PCI_CLASS_MEMORY_RAM
>>>>> class id but we couldn't agree on it.
>>>>>
>>>>> VMGEN v13 with full discussion is here
>>>>> https://patchwork.ozlabs.org/patch/443554/
>>>>> So to continue with this route we would need to pick some other
>>>>> driver less class id so windows won't prompt for driver or
>>>>> maybe supply our own driver stub to guarantee that no one
>>>>> would touch it. Any suggestions?    
>>>>
>>>> Pick any device/vendor id pair for which windows specifies no driver.
>>>> There's a small risk that this will conflict with some
>>>> guest but I think it's minimal.  
>>> device/vendor id pair was QEMU specific so doesn't conflicts with anything
>>> issue we were trying to solve was to prevent windows asking for driver
>>> even though it does so only once if told not to ask again.
>>>
>>> That's why PCI_CLASS_MEMORY_RAM was selected as it's generic driver-less
>>> device descriptor in INF file which matches as the last resort if
>>> there isn't any other diver that's matched device by device/vendor id pair.  
>>
>> I think this is the only class in this inf.
>> If you can't use it, you must use an existing device/vendor id pair,
>> there's some risk involved but probably not much.
> I can't wrap my head around this answer, could you rephrase it?
> 
> As far as I see we can use PCI_CLASS_MEMORY_RAM with qemu's device/vendor ids.
> In that case Windows associates it with dummy "Generic RAM controller".
> 
> The same happens with some NVIDIA cards if NVIDIA drivers are not installed,
> if we install drivers then Windows binds NVIDIA's PCI_CLASS_MEMORY_RAM with
> concrete driver that manages VRAM the way NVIDIA wants it.
> 
> So I think we can use it with low risk.
> 
> If we use existing device/vendor id pair with some driver then driver
> will fail to initialize and as minimum we would get device marked as
> not working in Device-Manager. Any way if you have in mind a concrete
> existing device/vendor id pair feel free to suggest it.
> 
>>
>>>>
>>>>   
>>>>>>
>>>>>>     
>>>>>>> There was RFC on list to make BIOS boot from NVDIMM already
>>>>>>> doing some ACPI table lookup/parsing. Now if they were forced
>>>>>>> to also parse and execute AML to initialize QEMU with guest
>>>>>>> allocated address that would complicate them quite a bit.      
>>>>>>
>>>>>> If they just need to find a table by name, it won't be
>>>>>> too bad, would it?    
>>>>> that's what they were doing scanning memory for static NVDIMM table.
>>>>> However if it were DataTable, BIOS side would have to execute
>>>>> AML so that the table address could be told to QEMU.    
>>>>
>>>> Not at all. You can find any table by its signature without
>>>> parsing AML.  
>>> yep, and then BIOS would need to tell its address to QEMU
>>> writing to IO port which is allocated statically in QEMU
>>> for this purpose and is described in AML only on guest side.  
>>
>> io ports are an ABI too but they are way easier to
>> maintain.
> It's pretty much the same as GPA addresses only it's much more limited resource.
> Otherwise one has to do the same tricks to maintain ABI.
> 
>>
>>>>
>>>>   
>>>>> In case of direct mapping or PCI BAR there is no need to initialize
>>>>> QEMU side from AML.
>>>>> That also saves us IO port where this address should be written
>>>>> if bios-linker-loader approach is used.
>>>>>     
>>>>>>     
>>>>>>> While with NVDIMM control memory region mapped directly by QEMU,
>>>>>>> respective patches don't need in any way to initialize QEMU,
>>>>>>> all they would need just read necessary data from control region.
>>>>>>>
>>>>>>> Also using bios-linker-loader takes away some usable RAM
>>>>>>> from guest and in the end that doesn't scale,
>>>>>>> the more devices I add the less usable RAM is left for guest OS
>>>>>>> while all the device needs is a piece of GPA address space
>>>>>>> that would belong to it.      
>>>>>>
>>>>>> I don't get this comment. I don't think it's MMIO that is wanted.
>>>>>> If it's backed by qemu virtual memory then it's RAM.    
>>>>> Then why don't allocate video card VRAM the same way and try to explain
>>>>> user that a guest started with '-m 128 -device cirrus-vga,vgamem_mb=64Mb'
>>>>> only has 64Mb of available RAM because of we think that on device VRAM
>>>>> is also RAM.
>>>>>
>>>>> Maybe I've used MMIO term wrongly here but it roughly reflects the idea
>>>>> that on device memory (whether it's VRAM, NVDIMM control block or VMGEN
>>>>> area) is not allocated from guest's usable RAM (as described in E820)
>>>>> but rather directly mapped in guest's GPA and doesn't consume available
>>>>> RAM as guest sees it. That's also the way it's done on real hardware.
>>>>>
>>>>> What we need in case of VMGEN ID and NVDIMM is on device memory
>>>>> that could be directly accessed by guest.
>>>>> Both direct mapping or PCI BAR do that job and we could use simple
>>>>> static AML without any patching.    
>>>>
>>>> At least with VMGEN the issue is that there's an AML method
>>>> that returns the physical address.
>>>> Then if guest OS moves the BAR (which is legal), it will break
>>>> since caller has no way to know it's related to the BAR.  
>>> I've found a following MS doc "Firmware Allocation of PCI Device Resources in Windows". It looks like when MS implemented resource rebalancing in
>>> Vista they pushed a compat change to PCI specs.
>>> That ECN is called "Ignore PCI Boot Configuration_DSM Function"
>>> and can be found here:
>>> https://pcisig.com/sites/default/files/specification_documents/ECR-Ignorebootconfig-final.pdf
>>>
>>> It looks like it's possible to forbid rebalancing per
>>> device/bridge if it has _DMS method that returns "do not
>>> ignore the boot configuration of PCI resources".  
>>
>> I'll have to study this but we don't want that
>> globally, do we?
> no need to do it globally, adding _DSM to a device, we don't wish
> to be rebalanced, should be sufficient to lock down specific resources.
> 
> actually existence of spec implies that if there is a boot configured
> device with resources described in ACPI table and there isn't _DSM
> method enabling rebalancing for it, then rebalancing is not permitted.
> It should be easy to make an experiment to verify what Windows would do.
> 
> So if this approach would work and we agree on going with it, I could work
> on redoing VMGENv13 series using _DSM as described.
> That would simplify implementing this kind of devices vs bios-linker approach i.e.:
>  - free RAM occupied by linker blob
>  - free IO port
>  - avoid 2 or 3 layers of indirection - which makes understanding of code much easier
>  - avoid runtime AML patching and simplify AML and its composing parts
>  - there won't be need for BIOS to get IO port from fw_cfg and write
>    there GPA as well no need for table lookup.
>  - much easier to write unit tests, i.e. use the same qtest device testing
>    technique without necessity of running actual guest code.
>    i.e. no binary code blobs like we have for running bios-tables test in TCG mode.

No objections on my part. If it works, it works for me!

Laszlo


> 
> 
>> This restricts hotplug functionality significantly.
>>
>>>    
>>>>>>>>
>>>>>>>> See patch at the bottom that might be handy.
>>>>>>>>       
>>>>>>>>> he also innovated a way to use 64-bit address in DSDT/SSDT.rev = 1:
>>>>>>>>> | when writing ASL one shall make sure that only XP supported
>>>>>>>>> | features are in global scope, which is evaluated when tables
>>>>>>>>> | are loaded and features of rev2 and higher are inside methods.
>>>>>>>>> | That way XP doesn't crash as far as it doesn't evaluate unsupported
>>>>>>>>> | opcode and one can guard those opcodes checking _REV object if neccesary.
>>>>>>>>> (https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg01010.html)      
>>>>>>>>
>>>>>>>> Yes, this technique works.
>>>>>>>>
>>>>>>>> An alternative is to add an XSDT, XP ignores that.
>>>>>>>> XSDT at the moment breaks OVMF (because it loads both
>>>>>>>> the RSDT and the XSDT, which is wrong), but I think
>>>>>>>> Laszlo was working on a fix for that.      
>>>>>>> Using XSDT would increase ACPI tables occupied RAM
>>>>>>> as it would duplicate DSDT + non XP supported AML
>>>>>>> at global namespace.      
>>>>>>
>>>>>> Not at all - I posted patches linking to same
>>>>>> tables from both RSDT and XSDT at some point.
>>>>>> Only the list of pointers would be different.    
>>>>> if you put XP incompatible AML in separate SSDT and link it
>>>>> only from XSDT than that would work but if incompatibility
>>>>> is in DSDT, one would have to provide compat DSDT for RSDT
>>>>> an incompat DSDT for XSDT.    
>>>>
>>>> So don't do this.  
>>> well spec says "An ACPI-compatible OS must use the XSDT if present",
>>> which I read as tables pointed by RSDT MUST be pointed by XSDT
>>> as well and RSDT MUST NOT not be used.
>>>
>>> so if we put incompatible changes in a separate SSDT and put
>>> it only in XSDT that might work. Showstopper here is OVMF which
>>> has issues with it as Laszlo pointed out.  
>>
>> But that's just a bug.
>>
>>> Also since Windows implements only subset of spec XSDT trick
>>> would cover only XP based versions while the rest will see and
>>> use XSDT pointed tables which still could have incompatible
>>> AML with some of the later windows versions.  
>>
>> We'll have to see what these are exactly.
>> If it's methods in SSDT we can check the version supported
>> by the ASPM.
> I see only VAR_PACKAGE as such object so far.
> 
> 64-bit PCI0._CRS probably won't crash 32-bit Vista and later as
> it should be able to parse 64-bit Integers as defined by ACPI 2.0.
> 
>>
>>>   
>>>>   
>>>>> So far policy was don't try to run guest OS on QEMU
>>>>> configuration that isn't supported by it.    
>>>>
>>>> It's better if guests don't see some features but
>>>> don't crash. It's not always possible of course but
>>>> we should try to avoid this.
>>>>   
>>>>> For example we use VAR_PACKAGE when running with more
>>>>> than 255 VCPUs (commit b4f4d5481) which BSODs XP.    
>>>>
>>>> Yes. And it's because we violate the spec, DSDT
>>>> should not have this stuff.
>>>>   
>>>>> So we can continue with that policy with out resorting to
>>>>> using both RSDT and XSDT,
>>>>> It would be even easier as all AML would be dynamically
>>>>> generated and DSDT would only contain AML elements for
>>>>> a concrete QEMU configuration.    
>>>>
>>>> I'd prefer XSDT but I won't nack it if you do it in DSDT.
>>>> I think it's not spec compliant but guests do not
>>>> seem to care.
>>>>   
>>>>>>> So far we've managed keep DSDT compatible with XP while
>>>>>>> introducing features from v2 and higher ACPI revisions as
>>>>>>> AML that is only evaluated on demand.
>>>>>>> We can continue doing so unless we have to unconditionally
>>>>>>> add incompatible AML at global scope.
>>>>>>>       
>>>>>>
>>>>>> Yes.
>>>>>>     
>>>>>>>>       
>>>>>>>>> Michael, Paolo, what do you think about these ideas?
>>>>>>>>>
>>>>>>>>> Thanks!      
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> So using a patch below, we can add Name(PQRS, 0x0) at the top of the
>>>>>>>> SSDT (or bottom, or add a separate SSDT just for that).  It returns the
>>>>>>>> current offset so we can add that to the linker.
>>>>>>>>
>>>>>>>> Won't work if you append the Name to the Aml structure (these can be
>>>>>>>> nested to arbitrary depth using aml_append), so using plain GArray for
>>>>>>>> this API makes sense to me.
>>>>>>>>       
>>>>>>>> --->      
>>>>>>>>
>>>>>>>> acpi: add build_append_named_dword, returning an offset in buffer
>>>>>>>>
>>>>>>>> This is a very limited form of support for runtime patching -
>>>>>>>> similar in functionality to what we can do with ACPI_EXTRACT
>>>>>>>> macros in python, but implemented in C.
>>>>>>>>
>>>>>>>> This is to allow ACPI code direct access to data tables -
>>>>>>>> which is exactly what DataTableRegion is there for, except
>>>>>>>> no known windows release so far implements DataTableRegion.      
>>>>>>> unsupported means Windows will BSOD, so it's practically
>>>>>>> unusable unless MS will patch currently existing Windows
>>>>>>> versions.      
>>>>>>
>>>>>> Yes. That's why my patch allows patching SSDT without using
>>>>>> DataTableRegion.
>>>>>>     
>>>>>>> Another thing about DataTableRegion is that ACPI tables are
>>>>>>> supposed to have static content which matches checksum in
>>>>>>> table the header while you are trying to use it for dynamic
>>>>>>> data. It would be cleaner/more compatible to teach
>>>>>>> bios-linker-loader to just allocate memory and patch AML
>>>>>>> with the allocated address.      
>>>>>>
>>>>>> Yes - if address is static, you need to put it outside
>>>>>> the table. Can come right before or right after this.
>>>>>>     
>>>>>>> Also if OperationRegion() is used, then one has to patch
>>>>>>> DefOpRegion directly as RegionOffset must be Integer,
>>>>>>> using variable names is not permitted there.      
>>>>>>
>>>>>> I am not sure the comment was understood correctly.
>>>>>> The comment says really "we can't use DataTableRegion
>>>>>> so here is an alternative".    
>>>>> so how are you going to access data at which patched
>>>>> NameString point to?
>>>>> for that you'd need a normal patched OperationRegion
>>>>> as well since DataTableRegion isn't usable.    
>>>>
>>>> For VMGENID you would patch the method that
>>>> returns the address - you do not need an op region
>>>> as you never access it.
>>>>
>>>> I don't know about NVDIMM. Maybe OperationRegion can
>>>> use the patched NameString? Will need some thought.
>>>>   
>>>>>>     
>>>>>>>       
>>>>>>>>
>>>>>>>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>>>>>>>>
>>>>>>>> ---
>>>>>>>>
>>>>>>>> diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
>>>>>>>> index 1b632dc..f8998ea 100644
>>>>>>>> --- a/include/hw/acpi/aml-build.h
>>>>>>>> +++ b/include/hw/acpi/aml-build.h
>>>>>>>> @@ -286,4 +286,7 @@ void acpi_build_tables_cleanup(AcpiBuildTables *tables, bool mfre);
>>>>>>>>  void
>>>>>>>>  build_rsdt(GArray *table_data, GArray *linker, GArray *table_offsets);
>>>>>>>>  
>>>>>>>> +int
>>>>>>>> +build_append_named_dword(GArray *array, const char *name_format, ...);
>>>>>>>> +
>>>>>>>>  #endif
>>>>>>>> diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
>>>>>>>> index 0d4b324..7f9fa65 100644
>>>>>>>> --- a/hw/acpi/aml-build.c
>>>>>>>> +++ b/hw/acpi/aml-build.c
>>>>>>>> @@ -262,6 +262,32 @@ static void build_append_int(GArray *table, uint64_t value)
>>>>>>>>      }
>>>>>>>>  }
>>>>>>>>  
>>>>>>>> +/* Build NAME(XXXX, 0x0) where 0x0 is encoded as a qword,
>>>>>>>> + * and return the offset to 0x0 for runtime patching.
>>>>>>>> + *
>>>>>>>> + * Warning: runtime patching is best avoided. Only use this as
>>>>>>>> + * a replacement for DataTableRegion (for guests that don't
>>>>>>>> + * support it).
>>>>>>>> + */
>>>>>>>> +int
>>>>>>>> +build_append_named_qword(GArray *array, const char *name_format, ...)
>>>>>>>> +{
>>>>>>>> +    int offset;
>>>>>>>> +    va_list ap;
>>>>>>>> +
>>>>>>>> +    va_start(ap, name_format);
>>>>>>>> +    build_append_namestringv(array, name_format, ap);
>>>>>>>> +    va_end(ap);
>>>>>>>> +
>>>>>>>> +    build_append_byte(array, 0x0E); /* QWordPrefix */
>>>>>>>> +
>>>>>>>> +    offset = array->len;
>>>>>>>> +    build_append_int_noprefix(array, 0x0, 8);
>>>>>>>> +    assert(array->len == offset + 8);
>>>>>>>> +
>>>>>>>> +    return offset;
>>>>>>>> +}
>>>>>>>> +
>>>>>>>>  static GPtrArray *alloc_list;
>>>>>>>>  
>>>>>>>>  static Aml *aml_alloc(void)
>>>>>>>>
>>>>>>>>       
>>>>>> --
>>>>>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html    
>>>>   
>

On 01/07/16 14:51, Igor Mammedov wrote:
> On Mon, 4 Jan 2016 21:17:31 +0100
> Laszlo Ersek <lersek@redhat.com> wrote:
> 
>> Michael CC'd me on the grandparent of the email below. I'll try to add
>> my thoughts in a single go, with regard to OVMF.
>>
>> On 12/30/15 20:52, Michael S. Tsirkin wrote:
>>> On Wed, Dec 30, 2015 at 04:55:54PM +0100, Igor Mammedov wrote:  
>>>> On Mon, 28 Dec 2015 14:50:15 +0200
>>>> "Michael S. Tsirkin" <mst@redhat.com> wrote:
>>>>  
>>>>> On Mon, Dec 28, 2015 at 10:39:04AM +0800, Xiao Guangrong wrote:  
>>>>>>
>>>>>> Hi Michael, Paolo,
>>>>>>
>>>>>> Now it is the time to return to the challenge that how to reserve guest
>>>>>> physical region internally used by ACPI.
>>>>>>
>>>>>> Igor suggested that:
>>>>>> | An alternative place to allocate reserve from could be high memory.
>>>>>> | For pc we have "reserved-memory-end" which currently makes sure
>>>>>> | that hotpluggable memory range isn't used by firmware
>>>>>> (https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg00926.html)  
>>
>> OVMF has no support for the "reserved-memory-end" fw_cfg file. The
>> reason is that nobody wrote that patch, nor asked for the patch to be
>> written. (Not implying that just requesting the patch would be
>> sufficient for the patch to be written.)
> Hijacking this part of thread to check if OVMF would work with memory-hotplug
> and if it needs "reserved-memory-end" support at all.
> 
> How OVMF determines which GPA ranges to use for initializing PCI BARs
> at boot time,

I'm glad you asked this question. This is an utterly sordid area that goes back quite a bit. We've discussed it several times in the past; for example: if you recall the "etc/pci-info" discussion...

Fact is, OVMF has no way to dynamically determine the PCI MMIO aperture to allocate BARs from. (Obviously parsing AML is out of question, especially at the early stage of the firmware where this information would be necessary. Plus that would be a chicken-egg problem anyway: QEMU composes the CRS in the AML *based on* the enumeration that was completed by the guest.)

Search "OvmfPkg/PlatformPei/Platform.c" for the string "PciBase"; it all originates there. I can also quote it:

    UINT32  TopOfLowRam;
    UINT32  PciBase;

    TopOfLowRam = GetSystemMemorySizeBelow4gb ();
    if (mHostBridgeDevId == INTEL_Q35_MCH_DEVICE_ID) {
      //
      // A 3GB base will always fall into Q35's 32-bit PCI host aperture,
      // regardless of the Q35 MMCONFIG BAR. Correspondingly, QEMU never lets
      // the RAM below 4 GB exceed it.
      //
      PciBase = BASE_2GB + BASE_1GB;
      ASSERT (TopOfLowRam <= PciBase);
    } else {
      PciBase = (TopOfLowRam < BASE_2GB) ? BASE_2GB : TopOfLowRam;
    }

    ...

    AddIoMemoryRangeHob (PciBase, 0xFC000000);

That's it.

In the past, it has repeatedly occurred that OVMF's calculation wouldn't match QEMU's calculation. Then PCI MMIO BARs were allocated outside of QEMU's actual MMIO aperture. This caused two things:
- video display not working (due to framebuffer being accessed in bogus place),
- Windows and Linux guests noticing that the BARs were outside of the range exposed in the _CRS, and disabling devices etc.

We kept duct-taping this, with patches in both OVMF and QEMU (see e.g. Gerd's QEMU commit ddaaefb4dd42).

It has been working fine for quite a long time now, but it is still not dynamic -- the calculations are duplicated between QEMU and OVMF.

To this day, I maintain that the "etc/pci-info" fw_cfg file would have been ideal for OVMF's purposes; and I still don't understand why it was ultimately removed.

> more specifically 64-bit BARs.

Ha. Haha. Hahahaha.

OVMF doesn't support 64-bit BARs *at all*. In order to implement that, I would have to (1) understand PCI about ten billion percent better than I do now, (2) extend the mostly *impenetrable* PCI host bridge / root bridge driver in "OvmfPkg/PciHostBridgeDxe" to support this functionality.

Unfortunately, the parts of the UEFI & Platform Init specs that seem to talk about this functionality are super complex and obscure.

We have plans with Marcel and others to understand this better and perhaps do something about it.

Anyway, the basic premise bears repeating: even for the 32-bit case, OVMF has no way to dynamically retrieve the PCI hole's boundaries from QEMU.

Honestly, I'm confused. If "reserved-memory-end" is exposed over fw_cfg, and it -- apparently! -- partakes in communicating the 64-bit PCI hole to the guest, then why again was "etc/pci-info" removed in the first place?

Thanks
Laszlo

On 01/07/2016 05:21 PM, Igor Mammedov wrote:
> On Wed, 6 Jan 2016 01:07:45 +0800
> Xiao Guangrong <guangrong.xiao@linux.intel.com> wrote:
>
>> On 01/06/2016 12:43 AM, Michael S. Tsirkin wrote:
>>
>>>>> Yes - if address is static, you need to put it outside
>>>>> the table. Can come right before or right after this.
>>>>>
>>>>>> Also if OperationRegion() is used, then one has to patch
>>>>>> DefOpRegion directly as RegionOffset must be Integer,
>>>>>> using variable names is not permitted there.
>>>>>
>>>>> I am not sure the comment was understood correctly.
>>>>> The comment says really "we can't use DataTableRegion
>>>>> so here is an alternative".
>>>> so how are you going to access data at which patched
>>>> NameString point to?
>>>> for that you'd need a normal patched OperationRegion
>>>> as well since DataTableRegion isn't usable.
>>>
>>> For VMGENID you would patch the method that
>>> returns the address - you do not need an op region
>>> as you never access it.
>>>
>>> I don't know about NVDIMM. Maybe OperationRegion can
>>> use the patched NameString? Will need some thought.
>>
>> The ACPI spec says that the offsetTerm in OperationRegion
>> is evaluated as Int, so the named object is allowed to be
>> used in OperationRegion, that is exact what my patchset
>> is doing (http://marc.info/?l=kvm&m=145193395624537&w=2):
> that's not my reading of spec:
> "
> DefOpRegion := OpRegionOp NameString RegionSpace RegionOffset RegionLen
> RegionOffset := TermArg => Integer
> TermArg := Type2Opcode | DataObject | ArgObj | LocalObj
> "
>
> Named object is not allowed per spec, but you've used ArgObj which is
> allowed, even Windows ok with such dynamic OperationRegion.

Sorry, Named object i was talking about is something like this:
Name("SOTH", int(0x10000))

I am checking acpi spec, and this is a formal NamedObj definition in
that spec, my fault.

>
>>
>> +    dsm_mem = aml_arg(3);
>> +    aml_append(method, aml_store(aml_call0(NVDIMM_GET_DSM_MEM), dsm_mem));
>>
>> +    aml_append(method, aml_operation_region("NRAM", AML_SYSTEM_MEMORY,
>> +                                            dsm_mem, TARGET_PAGE_SIZE));
>>
>> We hide the int64 object which is patched by BIOS in the method,
>> NVDIMM_GET_DSM_MEM, to make windows XP happy.
> considering that NRAM is allocated in low mem it's even fine to move
> OperationRegion into object scope to get rid of IASL warnings
> about declariong Named object inside method, but the you'd need to
> patch it directly as the only choice for RegionOffset would be DataObject
>

Yes, it is. So it is depends on the question in my reply of another thread:
http://marc.info/?l=kvm&m=145222487605390&w=2
Can we assume that BIOS allocated address is always 32 bits?

If yes, we also need not make ssdt as v2.

On 01/08/16 05:21, Xiao Guangrong wrote:
> 
> 
> On 01/07/2016 05:21 PM, Igor Mammedov wrote:
>> On Wed, 6 Jan 2016 01:07:45 +0800
>> Xiao Guangrong <guangrong.xiao@linux.intel.com> wrote:
>>
>>> On 01/06/2016 12:43 AM, Michael S. Tsirkin wrote:
>>>
>>>>>> Yes - if address is static, you need to put it outside
>>>>>> the table. Can come right before or right after this.
>>>>>>
>>>>>>> Also if OperationRegion() is used, then one has to patch
>>>>>>> DefOpRegion directly as RegionOffset must be Integer,
>>>>>>> using variable names is not permitted there.
>>>>>>
>>>>>> I am not sure the comment was understood correctly.
>>>>>> The comment says really "we can't use DataTableRegion
>>>>>> so here is an alternative".
>>>>> so how are you going to access data at which patched
>>>>> NameString point to?
>>>>> for that you'd need a normal patched OperationRegion
>>>>> as well since DataTableRegion isn't usable.
>>>>
>>>> For VMGENID you would patch the method that
>>>> returns the address - you do not need an op region
>>>> as you never access it.
>>>>
>>>> I don't know about NVDIMM. Maybe OperationRegion can
>>>> use the patched NameString? Will need some thought.
>>>
>>> The ACPI spec says that the offsetTerm in OperationRegion
>>> is evaluated as Int, so the named object is allowed to be
>>> used in OperationRegion, that is exact what my patchset
>>> is doing (http://marc.info/?l=kvm&m=145193395624537&w=2):
>> that's not my reading of spec:
>> "
>> DefOpRegion := OpRegionOp NameString RegionSpace RegionOffset RegionLen
>> RegionOffset := TermArg => Integer
>> TermArg := Type2Opcode | DataObject | ArgObj | LocalObj
>> "
>>
>> Named object is not allowed per spec, but you've used ArgObj which is
>> allowed, even Windows ok with such dynamic OperationRegion.
> 
> Sorry, Named object i was talking about is something like this:
> Name("SOTH", int(0x10000))
> 
> I am checking acpi spec, and this is a formal NamedObj definition in
> that spec, my fault.
> 
>>
>>>
>>> +    dsm_mem = aml_arg(3);
>>> +    aml_append(method, aml_store(aml_call0(NVDIMM_GET_DSM_MEM),
>>> dsm_mem));
>>>
>>> +    aml_append(method, aml_operation_region("NRAM", AML_SYSTEM_MEMORY,
>>> +                                            dsm_mem,
>>> TARGET_PAGE_SIZE));
>>>
>>> We hide the int64 object which is patched by BIOS in the method,
>>> NVDIMM_GET_DSM_MEM, to make windows XP happy.
>> considering that NRAM is allocated in low mem it's even fine to move
>> OperationRegion into object scope to get rid of IASL warnings
>> about declariong Named object inside method, but the you'd need to
>> patch it directly as the only choice for RegionOffset would be DataObject
>>
> 
> Yes, it is. So it is depends on the question in my reply of another thread:
> http://marc.info/?l=kvm&m=145222487605390&w=2
> Can we assume that BIOS allocated address is always 32 bits?

As far as OVMF is concerned: you can assume this at the moment, yes.

Thanks
Laszlo

> If yes, we also need not make ssdt as v2.

On Fri, 8 Jan 2016 12:21:09 +0800
Xiao Guangrong <guangrong.xiao@linux.intel.com> wrote:

> On 01/07/2016 05:21 PM, Igor Mammedov wrote:
> > On Wed, 6 Jan 2016 01:07:45 +0800
> > Xiao Guangrong <guangrong.xiao@linux.intel.com> wrote:
> >  
> >> On 01/06/2016 12:43 AM, Michael S. Tsirkin wrote:
> >>  
> >>>>> Yes - if address is static, you need to put it outside
> >>>>> the table. Can come right before or right after this.
> >>>>>  
> >>>>>> Also if OperationRegion() is used, then one has to patch
> >>>>>> DefOpRegion directly as RegionOffset must be Integer,
> >>>>>> using variable names is not permitted there.  
> >>>>>
> >>>>> I am not sure the comment was understood correctly.
> >>>>> The comment says really "we can't use DataTableRegion
> >>>>> so here is an alternative".  
> >>>> so how are you going to access data at which patched
> >>>> NameString point to?
> >>>> for that you'd need a normal patched OperationRegion
> >>>> as well since DataTableRegion isn't usable.  
> >>>
> >>> For VMGENID you would patch the method that
> >>> returns the address - you do not need an op region
> >>> as you never access it.
> >>>
> >>> I don't know about NVDIMM. Maybe OperationRegion can
> >>> use the patched NameString? Will need some thought.  
> >>
> >> The ACPI spec says that the offsetTerm in OperationRegion
> >> is evaluated as Int, so the named object is allowed to be
> >> used in OperationRegion, that is exact what my patchset
> >> is doing (http://marc.info/?l=kvm&m=145193395624537&w=2):  
> > that's not my reading of spec:
> > "
> > DefOpRegion := OpRegionOp NameString RegionSpace RegionOffset RegionLen
> > RegionOffset := TermArg => Integer
> > TermArg := Type2Opcode | DataObject | ArgObj | LocalObj
> > "
> >
> > Named object is not allowed per spec, but you've used ArgObj which is
> > allowed, even Windows ok with such dynamic OperationRegion.  
> 
> Sorry, Named object i was talking about is something like this:
> Name("SOTH", int(0x10000))
> 
> I am checking acpi spec, and this is a formal NamedObj definition in
> that spec, my fault.
> 
> >  
> >>
> >> +    dsm_mem = aml_arg(3);
> >> +    aml_append(method, aml_store(aml_call0(NVDIMM_GET_DSM_MEM), dsm_mem));
> >>
> >> +    aml_append(method, aml_operation_region("NRAM", AML_SYSTEM_MEMORY,
> >> +                                            dsm_mem, TARGET_PAGE_SIZE));
> >>
> >> We hide the int64 object which is patched by BIOS in the method,
> >> NVDIMM_GET_DSM_MEM, to make windows XP happy.  
> > considering that NRAM is allocated in low mem it's even fine to move
> > OperationRegion into object scope to get rid of IASL warnings
> > about declariong Named object inside method, but the you'd need to
> > patch it directly as the only choice for RegionOffset would be DataObject
> >  
> 
> Yes, it is. So it is depends on the question in my reply of another thread:
> http://marc.info/?l=kvm&m=145222487605390&w=2
> Can we assume that BIOS allocated address is always 32 bits?
> 
> If yes, we also need not make ssdt as v2.
For SeaBIOS it's so for now.