[qemu-web] Add a blog post documenting Spectre/Meltdown options for QEMU 2.11.1

This blog entry is intended as a follow-up to the original entry in
January regarding Spectre/Meltdown and the proposed changes to address
them in the upcoming 2.11.1 release.

This entry is meant to accompany the 2.11.1 release (planned for
2018-02-14) and document how to make use of the new options for
various architectures.

Cc: Eduardo Habkost <ehabkost@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Peter Maydell <peter.maydell@linaro.org>
Cc: Suraj Jitindar Singh <sjitindarsingh@gmail.com>
Cc: David Gibson <david@gibson.dropbear.id.au>
Cc: Christian Borntraeger <borntraeger@de.ibm.com>
Cc: Cornelia Huck <cohuck@redhat.com>
Cc: Thomas Huth <thuth@redhat.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---

The pseries/s390 bits have gotten some initial review (thanks Suraj/Christian),
but it can definitely use some additional review on the x86 side of things.

Also, Peter if think anything extra should to be mentioned on the ARM side just
let me know what to add.

 .../2018-02-14-qemu-2-11-1-and-spectre-update.md   | 180 +++++++++++++++++++++
 1 file changed, 180 insertions(+)
 create mode 100644 _posts/2018-02-14-qemu-2-11-1-and-spectre-update.md

On 2/13/2018 at 5:11 PM, Michael Roth <mdroth@linux.vnet.ibm.com> wrote:
> This blog entry is intended as a follow‑up to the original entry in
> January regarding Spectre/Meltdown and the proposed changes to address
> them in the upcoming 2.11.1 release.
> 
> This entry is meant to accompany the 2.11.1 release (planned for
> 2018‑02‑14) and document how to make use of the new options for
> various architectures.
> 
> Cc: Eduardo Habkost <ehabkost@redhat.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Peter Maydell <peter.maydell@linaro.org>
> Cc: Suraj Jitindar Singh <sjitindarsingh@gmail.com>
> Cc: David Gibson <david@gibson.dropbear.id.au>
> Cc: Christian Borntraeger <borntraeger@de.ibm.com>
> Cc: Cornelia Huck <cohuck@redhat.com>
> Cc: Thomas Huth <thuth@redhat.com>
> Signed‑off‑by: Michael Roth <mdroth@linux.vnet.ibm.com>
> ‑‑‑
> 
> The pseries/s390 bits have gotten some initial review (thanks 
> Suraj/Christian),
> but it can definitely use some additional review on the x86 side of things.
> 
> Also, Peter if think anything extra should to be mentioned on the ARM side 
> just
> let me know what to add.
> 
>  .../2018‑02‑14‑qemu‑2‑11‑1‑and‑spectre‑update.md   | 180 +++++++++++++++++++++
>  1 file changed, 180 insertions(+)
>  create mode 100644 _posts/2018‑02‑14‑qemu‑2‑11‑1‑and‑spectre‑update.md
> 
> diff ‑‑git a/_posts/2018‑02‑14‑qemu‑2‑11‑1‑and‑spectre‑update.md 
> b/_posts/2018‑02‑14‑qemu‑2‑11‑1‑and‑spectre‑update.md
> new file mode 100644
> index 0000000..7cdea59
> ‑‑‑ /dev/null
> +++ b/_posts/2018‑02‑14‑qemu‑2‑11‑1‑and‑spectre‑update.md
> @@ ‑0,0 +1,180 @@
> +‑‑‑
> +layout: post
> +title:  "QEMU 2.11.1 and making use of Spectre/Meltdown mitigation for KVM 
> guests"
> +date: 2018‑02‑14 10:35:44 ‑0600
> +author: Michael Roth
> +categories: [meltdown, spectre, security, x86, ppc, s390, releases, 'qemu 
> 2.11']
> +‑‑‑
> +
> +In a [previous post](https://www.qemu.org/2018/01/04/spectre/) it was
> +detailed how QEMU/KVM might be affected by Spectre/Meltdown attacks, and 
> what
> +the plan was to mitigate them in QEMU 2.11.1 (and eventually QEMU 2.12).
> +
> +QEMU 2.11.1 is now available, and contains the aforementioned mitigations 
> for
> +x86 guests, along with additional mitigation functionality for pseries and
> +s390 guests (ARM guests do not currently require additional QEMU patches).
> +However, enabling this functionality requires additional configuration 
> beyond
> +just updating QEMU, which we hope to address with this post.
> +
> +Please note that, as mentioned in the previous blog post, QEMU/KVM 
> generally
> +has the same requirements as other unpriviledged processes running on the
> +host WRT Spectre/Meltdown mitigation. What is being addressed here is
> +enabling a guest operating system to enable the same (or similar) 
> mitigations
> +to protect itself from unpriviledged guest processes. Thus, the
> +patches/requirements listed here are specific to that goal and should not 
> be
> +regarded as the full set of requirements to enable mitigations on the host
> +side (though in some cases there is some overlap between the two WRT 
> required
> +patches/etc).
> +
> +Also please note that this is a best‑effort from the QEMU/KVM community, and
> +these mitigations rely on a mix of additional kernel/firmware/microcode
> +updates that are in some cases not available publically, or may not yet be
> +implemented in some distros, so users are highly encouraged to consult with
> +their respective vendors/distros to confirm whether all the required
> +components are in place. We do our best to highlight the requirements here,
> +but this may not be an exhaustive list.
> +
> +
> +## enabling mitigations for x86 KVM guests
> +
> +For x86 guests there are 2 additional CPU flags associated with
> +Spectre/Meltdown mitigation
: **spec‑ctrl**, and **ibpb**. These flags
> +expose additional functionality made available through new microcode
> +updates for certain Intel/AMD processors that can be used to mitigate
> +various attack vectors related to Spectre. (Meltdown mitigation via KPTI
> +does not require additional CPU functionality or microcode, and does not
> +require an updated QEMU, only the related guest/host kernel patches).
> +
> +These CPU flags:
> +
> +* spec‑ctrl: exposes Indirect Branch Restricted Speculation (IBRS)
> +* ibpb: exposes Indirect Branch Prediction Barriers
> +
> +are both features requiring guest/host kernel updates, as well as
> +microcode updates for Intel and recent AMD processors. The status of
> +these kernel patches upstream is still in flux, but most supported
> +distros have some form of the patches that is sufficient to make use
> +of the features. The current status/availability of microcode updates
> +depends on your CPU architecture/model. Please check with your
> +vendor/distro to confirm these prerequisites are available/installed.
> +
> +Generally, for Intel CPUs with updated microcode, **spec‑ctrl** will
> +enable both IBRS and IBPB functionality. For AMD EPYC processors,
> +**ibpb** can be used to enable IBPB specifically, and is thought to
> +be sufficient by itself that particular architecture.

be sufficient by itself for that particular architecture

> +
> +These flags can be set in a similar manner as other CPU flags, i.e.:
> +
> +    qemu‑system‑x86_64 ‑cpu qemu64,+spec‑ctrl,... ...
> +    qemu‑system‑x86_64 ‑cpu IvyBridge,+spec‑ctrl,... ...
> +    qemu‑system‑x86_64 ‑cpu EPYC,+ibpb
> +    etc...

kind of a nit, but might as well be consistent here, as in:
qemu-system-x86_64 -cpu EPYC,+ibpb,... ...


Thanks for putting this together!

Bruce

On Tue, Feb 13, 2018 at 06:11:05PM -0600, Michael Roth wrote:
> This blog entry is intended as a follow-up to the original entry in
> January regarding Spectre/Meltdown and the proposed changes to address
> them in the upcoming 2.11.1 release.
> 
> This entry is meant to accompany the 2.11.1 release (planned for
> 2018-02-14) and document how to make use of the new options for
> various architectures.
> 
> Cc: Eduardo Habkost <ehabkost@redhat.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Peter Maydell <peter.maydell@linaro.org>
> Cc: Suraj Jitindar Singh <sjitindarsingh@gmail.com>
> Cc: David Gibson <david@gibson.dropbear.id.au>
> Cc: Christian Borntraeger <borntraeger@de.ibm.com>
> Cc: Cornelia Huck <cohuck@redhat.com>
> Cc: Thomas Huth <thuth@redhat.com>
> Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> ---
> 
> The pseries/s390 bits have gotten some initial review (thanks Suraj/Christian),
> but it can definitely use some additional review on the x86 side of things.
> 
> Also, Peter if think anything extra should to be mentioned on the ARM side just
> let me know what to add.
> 
>  .../2018-02-14-qemu-2-11-1-and-spectre-update.md   | 180 +++++++++++++++++++++
>  1 file changed, 180 insertions(+)
>  create mode 100644 _posts/2018-02-14-qemu-2-11-1-and-spectre-update.md
> 
> diff --git a/_posts/2018-02-14-qemu-2-11-1-and-spectre-update.md b/_posts/2018-02-14-qemu-2-11-1-and-spectre-update.md
> new file mode 100644
> index 0000000..7cdea59
> --- /dev/null
> +++ b/_posts/2018-02-14-qemu-2-11-1-and-spectre-update.md
> @@ -0,0 +1,180 @@

> +Please note that, as mentioned in the previous blog post, QEMU/KVM generally
> +has the same requirements as other unpriviledged processes running on the
> +host WRT Spectre/Meltdown mitigation.

Is this actually still considered accurate wrt the host QEMU ? I was under
the believe that life is more complicated for QEMU/KVM wrt Spectre and that
it will require more protection than other unpriv processes on the host in
some cases.

Regards,
Daniel

On 14.02.2018 01:11, Michael Roth wrote:
> This blog entry is intended as a follow-up to the original entry in
> January regarding Spectre/Meltdown and the proposed changes to address
> them in the upcoming 2.11.1 release.
> 
> This entry is meant to accompany the 2.11.1 release (planned for
> 2018-02-14) and document how to make use of the new options for
> various architectures.

Thanks for writing this post, I think it will be quite helpful for the
users! Some comments below...

> Cc: Eduardo Habkost <ehabkost@redhat.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Peter Maydell <peter.maydell@linaro.org>
> Cc: Suraj Jitindar Singh <sjitindarsingh@gmail.com>
> Cc: David Gibson <david@gibson.dropbear.id.au>
> Cc: Christian Borntraeger <borntraeger@de.ibm.com>
> Cc: Cornelia Huck <cohuck@redhat.com>
> Cc: Thomas Huth <thuth@redhat.com>
> Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> ---
> 
> The pseries/s390 bits have gotten some initial review (thanks Suraj/Christian),
> but it can definitely use some additional review on the x86 side of things.
> 
> Also, Peter if think anything extra should to be mentioned on the ARM side just
> let me know what to add.
> 
>  .../2018-02-14-qemu-2-11-1-and-spectre-update.md   | 180 +++++++++++++++++++++
>  1 file changed, 180 insertions(+)
>  create mode 100644 _posts/2018-02-14-qemu-2-11-1-and-spectre-update.md
> 
> diff --git a/_posts/2018-02-14-qemu-2-11-1-and-spectre-update.md b/_posts/2018-02-14-qemu-2-11-1-and-spectre-update.md
> new file mode 100644
> index 0000000..7cdea59
> --- /dev/null
> +++ b/_posts/2018-02-14-qemu-2-11-1-and-spectre-update.md
> @@ -0,0 +1,180 @@
> +---
> +layout: post
> +title:  "QEMU 2.11.1 and making use of Spectre/Meltdown mitigation for KVM guests"
> +date: 2018-02-14 10:35:44 -0600
> +author: Michael Roth
> +categories: [meltdown, spectre, security, x86, ppc, s390, releases, 'qemu 2.11']

Maybe use "s390x" instead of "s390" ? ... we're using s390x as target
name in the sources, so this would be more consistent.

> +---
> +
> +In a [previous post](https://www.qemu.org/2018/01/04/spectre/) it was

I'm not a native speaker, so please simply ignore me if I write stupid
things, but I think I'd rather prefer active voice here:

A previous post described how QEMU/KVM might be ...

> +detailed how QEMU/KVM might be affected by Spectre/Meltdown attacks, and what
> +the plan was to mitigate them in QEMU 2.11.1 (and eventually QEMU 2.12).
> +
> +QEMU 2.11.1 is now available, and contains the aforementioned mitigations for
> +x86 guests, along with additional mitigation functionality for pseries and
> +s390 guests (ARM guests do not currently require additional QEMU patches).
> +However, enabling this functionality requires additional configuration beyond
> +just updating QEMU, which we hope to address with this post.

"which we hope" sounds unconfident to me. Maybe rather write "which we
want" instead?

> +Please note that, as mentioned in the previous blog post, QEMU/KVM generally
> +has the same requirements as other unpriviledged processes running on the

s/unpriviledged/unprivileged/g

> +host WRT Spectre/Meltdown mitigation. What is being addressed here is

s/WRT/with regards to/g ... not every non-native speaker knows that
abbreviation.

> +enabling a guest operating system to enable the same (or similar) mitigations
> +to protect itself from unpriviledged guest processes. Thus, the

I'd maybe add a "running under the guest operating system" after the
"guest processes" to make it clear that we're not talking about
something like cross-VM exploits here.

> +patches/requirements listed here are specific to that goal and should not be
> +regarded as the full set of requirements to enable mitigations on the host
> +side (though in some cases there is some overlap between the two WRT required
> +patches/etc).
> +
> +Also please note that this is a best-effort from the QEMU/KVM community, and
> +these mitigations rely on a mix of additional kernel/firmware/microcode
> +updates that are in some cases not available publically, or may not yet be

Maybe: s/publically/publicly (yet)/ ?

> +implemented in some distros, so users are highly encouraged to consult with
> +their respective vendors/distros to confirm whether all the required
> +components are in place. We do our best to highlight the requirements here,
> +but this may not be an exhaustive list.
> +
> +
> +## enabling mitigations for x86 KVM guests

Maybe: s/enabling/Enabling/ ?

Should we maybe also rather talk about "mitigation features" or
"mitigation flasgs" instead of "mitigations", since the mitigation
itself has rather to be enabled by the guest OS instead?

> +For x86 guests there are 2 additional CPU flags associated with
> +Spectre/Meltdown mitigation: **spec-ctrl**, and **ibpb**.

I think I'd rather move the bullet list below to this place here instead...

> These flags
> +expose additional functionality made available through new microcode
> +updates for certain Intel/AMD processors that can be used to mitigate
> +various attack vectors related to Spectre. (Meltdown mitigation via KPTI
> +does not require additional CPU functionality or microcode, and does not
> +require an updated QEMU, only the related guest/host kernel patches).
> +
> +These CPU flags:
> +
> +* spec-ctrl: exposes Indirect Branch Restricted Speculation (IBRS)
> +* ibpb: exposes Indirect Branch Prediction Barriers

... and then simply start this sentence with "For being able to use both
CPU flags a guest and host kernel update is required ..." or so.

> +are both features requiring guest/host kernel updates, as well as
> +microcode updates for Intel and recent AMD processors. The status of
> +these kernel patches upstream is still in flux, but most supported
> +distros have some form of the patches that is sufficient to make use
> +of the features. The current status/availability of microcode updates
> +depends on your CPU architecture/model. Please check with your
> +vendor/distro to confirm these prerequisites are available/installed.
> +
> +Generally, for Intel CPUs with updated microcode, **spec-ctrl** will
> +enable both IBRS and IBPB functionality. For AMD EPYC processors,
> +**ibpb** can be used to enable IBPB specifically, and is thought to
> +be sufficient by itself that particular architecture.

s/itself that/itself on that/ ?

> +These flags can be set in a similar manner as other CPU flags, i.e.:
> +
> +    qemu-system-x86_64 -cpu qemu64,+spec-ctrl,... ...
> +    qemu-system-x86_64 -cpu IvyBridge,+spec-ctrl,... ...
> +    qemu-system-x86_64 -cpu EPYC,+ibpb
> +    etc...
> +
> +Additionally, for management stacks that lack support for setting
> +specific CPU flags, a set of new CPU types have been added which
> +enable the appropriate CPU flags automatically:
> +
> +    qemu-system-x86_64 -cpu Nehalem-IBRS ...
> +    qemu-system-x86_64 -cpu Westmere-IBRS ...
> +    qemu-system-x86_64 -cpu SandyBridge-IBRS ...
> +    qemu-system-x86_64 -cpu IvyBridge-IBRS ...
> +    qemu-system-x86_64 -cpu Haswell-IBRS ...
> +    qemu-system-x86_64 -cpu Haswell-noTSX-IBRS ...
> +    qemu-system-x86_64 -cpu Broadwell-IBRS ...
> +    qemu-system-x86_64 -cpu Broadwell-noTSX-IBRS ...
> +    qemu-system-x86_64 -cpu Skylake-Client-IBRS ...
> +    qemu-system-x86_64 -cpu Skylake-Server-IBRS ...
> +    qemu-system-x86_64 -cpu EPYC-IBPB ...
> +
> +With these settings enabled, guests may still require additional
> +configuration to enable IBRS/IBPB, which may vary somewhat from one
> +distro to another. For RHEL guests, the following resource may be
> +useful:
> +
> +* https://access.redhat.com/articles/3311301
> +
> +WRT migration compatibility, **spec-ctrl**/**ibrs** (or the corresponding
> +CPU type) should be set the same on both source/target to maintain
> +compatibility. Thus, guests will need to be rebooted to make use of
> +the new features.
> +
> +
> +## enabling mitigations for pseries KVM guests
> +
> +For pseries guests there are 3 tri-state -machine options/capabilities
> +relating to Spectre/Meltdown mitigation: **cap-cfpc**, **cap-sbbc**,
> +**cap-ibs**, which each correspond to a set of host machine capabilities
> +advertised by the KVM kernel module in new/patched host kernels that can
> +be used to mitigate various aspects of Spectre/Meltdown:
> +
> +* cap-cfpc: Cache Flush on Privilege Change
> +* cap-sbbc: Speculation Barrier Bounds Checking
> +* cap-ibs: Indirect Branch Serialisation
> +
> +Each option can be set to one of "broken", "workaround", or "fixed", which
> +correspond, respectively, to instructing the guest whether the host is
> +vulnerable, has OS-level workarounds available, or has hardware/firmware
> +that does not require OS-level workarounds. Based on these options, QEMU
> +will perform checks to validate whether the specified settings are available
> +on the current host and pass these settings on to the guest kernel. At a
> +minimum, any setting other than "broken" will require a host kernel that has
> +some form of the following patches:
> +
> +    commit 3214d01f139b7544e870fc0b7fcce8da13c1cb51
> +    KVM: PPC: Book3S: Provide information about hardware/firmware CVE workarounds
> +  
> +    commit 191eccb1580939fb0d47deb405b82a85b0379070
> +    powerpc/pseries: Add H_GET_CPU_CHARACTERISTICS flags & wrapper
> +
> +and whether a host will support "workaround" and "fixed" settings for each
> +option will depend on the hardware/firmware level of the host system.
> +
> +In turn, to make use of "workaround" or "fixed" settings for each option,
> +the guest kernel will require at least the following set of patches:
> +
> +* https://lists.ozlabs.org/pipermail/linuxppc-dev/2018-January/167455.html
> +
> +These are available upstream and have been backported to a number of stable
> +kernels. Please check with your vendor/distro to confirm the required
> +hardware/firmware and guest kernel patches are available/installed.
> +
> +By default, all three options, **cap-cfpc**, **cap-sbbc**, and **cap-ibs**
> +default to "broken" to maintain compatibility with previous versions of QEMU

"By default ... default to ..." - I'd suggest to drop the initial "By
default".

> +and unpatched host kernels. To enable them you must start QEMU with the
> +desired mitigation strategy specified explicitly. For example:
> +
> +    qemu-system-ppc64 ... \
> +      -machine pseries-2.11,cap-cfpc=workaround,cap-sbbc=workaround,cap-ibs=fixed
> +
> +WRT migration compatibility, setting any of these features to a value other
> +than "broken" will require an identical setting for that option on the
> +source/destination guest. To enable these settings your guests will need to
> +be rebooted at some point.
> +
> +
> +## enabling mitigations for s390 KVM guests
> +
> +For s390 guests there are 2 CPU options relating to Spectre/Meltdown:
> +
> +* bpb: Branch prediction blocking
> +* ppa15: PPA15 is installed
> +
> +**bpb** requires a host kernel patched with:
> +
> +    commit 35b3fde6203b932b2b1a5b53b3d8808abc9c4f60
> +    KVM: s390: wire up bpb feature
> +
> +and both **bpb** and **ppa15** require a firmware with the appropriate support
> +level as well as guest kernel patches to enable the functionality within
> +guests. Please check with your distro/vendor to confirm.
> +
> +Both **bpb** and **ppa15** are enabled by default with newer/patched host
> +kernels, and can also be set manually. For example:
> +
> +    qemu-system-s390x -M s390-ccw-virtio-2.11 ... \
> +      -cpu zEC12,bpb=on,ppa15=on 

IIRC we only enable them by default with "-cpu host" ? Cornelia, David,
Christian, can you confirm?
So maybe better rephrase the above to:

Both **bpb** and **ppa15** are enabled by default when using "-cpu host"
and when the host kernels supports these facilities. For other CPU
models, the flags have to be set manually. For example:

    qemu-system-s390x -M s390-ccw-virtio-2.11 ... \
      -cpu zEC12,bpb=on,ppa15=on

> +WRT to migration, enabling **bpb** requires the source/target also have **bpb**
> +enabled. Since this is enabled by default, you must ensure that **bpb**=off if

s/**bpb**=off/**bpb**=off is used/ ?

> +you wish to maintain migration compatibility with existing guests, or take
> +steps to reboot guests with **bpb** enabled prior to migrating them.

 Thomas

On Tue, 13 Feb 2018 18:11:05 -0600
Michael Roth <mdroth@linux.vnet.ibm.com> wrote:

> This blog entry is intended as a follow-up to the original entry in
> January regarding Spectre/Meltdown and the proposed changes to address
> them in the upcoming 2.11.1 release.
> 
> This entry is meant to accompany the 2.11.1 release (planned for
> 2018-02-14) and document how to make use of the new options for
> various architectures.
> 
> Cc: Eduardo Habkost <ehabkost@redhat.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Peter Maydell <peter.maydell@linaro.org>
> Cc: Suraj Jitindar Singh <sjitindarsingh@gmail.com>
> Cc: David Gibson <david@gibson.dropbear.id.au>
> Cc: Christian Borntraeger <borntraeger@de.ibm.com>
> Cc: Cornelia Huck <cohuck@redhat.com>
> Cc: Thomas Huth <thuth@redhat.com>
> Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> ---
> 
> The pseries/s390 bits have gotten some initial review (thanks Suraj/Christian),
> but it can definitely use some additional review on the x86 side of things.
> 
> Also, Peter if think anything extra should to be mentioned on the ARM side just
> let me know what to add.
> 
>  .../2018-02-14-qemu-2-11-1-and-spectre-update.md   | 180 +++++++++++++++++++++
>  1 file changed, 180 insertions(+)
>  create mode 100644 _posts/2018-02-14-qemu-2-11-1-and-spectre-update.md

[some comments/questions regarding s390 cpu models, adding DavidH on cc:]

> +## enabling mitigations for s390 KVM guests
> +
> +For s390 guests there are 2 CPU options relating to Spectre/Meltdown:

s/options/feature bits/ ?

> +
> +* bpb: Branch prediction blocking
> +* ppa15: PPA15 is installed
> +
> +**bpb** requires a host kernel patched with:
> +
> +    commit 35b3fde6203b932b2b1a5b53b3d8808abc9c4f60
> +    KVM: s390: wire up bpb feature
> +
> +and both **bpb** and **ppa15** require a firmware with the appropriate support
> +level as well as guest kernel patches to enable the functionality within
> +guests. Please check with your distro/vendor to confirm.
> +
> +Both **bpb** and **ppa15** are enabled by default with newer/patched host
> +kernels, and can also be set manually. For example:
> +
> +    qemu-system-s390x -M s390-ccw-virtio-2.11 ... \
> +      -cpu zEC12,bpb=on,ppa15=on 

Do we also want to add that bpb/ppa15 are on if you use the _full_
model (as opposed to the _base_ model)? Or is this going into too much
detail about the cpu model?

> +
> +WRT to migration, enabling **bpb** requires the source/target also have **bpb**

Either "WRT migration", or "With regard to migration" :)

> +enabled. Since this is enabled by default, you must ensure that **bpb**=off if
> +you wish to maintain migration compatibility with existing guests, or take
> +steps to reboot guests with **bpb** enabled prior to migrating them.

This paragraph confuses me a bit. Both bpb and ppa15 are guest visible,
aren't they?

On 02/14/2018 10:11 AM, Cornelia Huck wrote:
> On Tue, 13 Feb 2018 18:11:05 -0600
> Michael Roth <mdroth@linux.vnet.ibm.com> wrote:
> 
>> This blog entry is intended as a follow-up to the original entry in
>> January regarding Spectre/Meltdown and the proposed changes to address
>> them in the upcoming 2.11.1 release.
>>
>> This entry is meant to accompany the 2.11.1 release (planned for
>> 2018-02-14) and document how to make use of the new options for
>> various architectures.
>>
>> Cc: Eduardo Habkost <ehabkost@redhat.com>
>> Cc: Paolo Bonzini <pbonzini@redhat.com>
>> Cc: Peter Maydell <peter.maydell@linaro.org>
>> Cc: Suraj Jitindar Singh <sjitindarsingh@gmail.com>
>> Cc: David Gibson <david@gibson.dropbear.id.au>
>> Cc: Christian Borntraeger <borntraeger@de.ibm.com>
>> Cc: Cornelia Huck <cohuck@redhat.com>
>> Cc: Thomas Huth <thuth@redhat.com>
>> Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
>> ---
>>
>> The pseries/s390 bits have gotten some initial review (thanks Suraj/Christian),
>> but it can definitely use some additional review on the x86 side of things.
>>
>> Also, Peter if think anything extra should to be mentioned on the ARM side just
>> let me know what to add.
>>
>>  .../2018-02-14-qemu-2-11-1-and-spectre-update.md   | 180 +++++++++++++++++++++
>>  1 file changed, 180 insertions(+)
>>  create mode 100644 _posts/2018-02-14-qemu-2-11-1-and-spectre-update.md
> 
> [some comments/questions regarding s390 cpu models, adding DavidH on cc:]
> 
>> +## enabling mitigations for s390 KVM guests
>> +
>> +For s390 guests there are 2 CPU options relating to Spectre/Meltdown:
> 
> s/options/feature bits/ ?
> 
>> +
>> +* bpb: Branch prediction blocking
>> +* ppa15: PPA15 is installed
>> +
>> +**bpb** requires a host kernel patched with:
>> +
>> +    commit 35b3fde6203b932b2b1a5b53b3d8808abc9c4f60
>> +    KVM: s390: wire up bpb feature
>> +
>> +and both **bpb** and **ppa15** require a firmware with the appropriate support
>> +level as well as guest kernel patches to enable the functionality within
>> +guests. Please check with your distro/vendor to confirm.
>> +
>> +Both **bpb** and **ppa15** are enabled by default with newer/patched host
>> +kernels, and can also be set manually. For example:
>> +
>> +    qemu-system-s390x -M s390-ccw-virtio-2.11 ... \
>> +      -cpu zEC12,bpb=on,ppa15=on 
> 
> Do we also want to add that bpb/ppa15 are on if you use the _full_
> model (as opposed to the _base_ model)? Or is this going into too much
> detail about the cpu model?

full model is just an internal implementation.
Either use
- host passthrough
- host model
- add bpb and ppa15 manually to the choosen model (e.g. z13,bpb=on,ppa15=on)

> 
>> +
>> +WRT to migration, enabling **bpb** requires the source/target also have **bpb**
> 
> Either "WRT migration", or "With regard to migration" :)
> 
>> +enabled. Since this is enabled by default, you must ensure that **bpb**=off if
>> +you wish to maintain migration compatibility with existing guests, or take
>> +steps to reboot guests with **bpb** enabled prior to migrating them.
> 
> This paragraph confuses me a bit. Both bpb and ppa15 are guest visible,
> aren't they?
>

On Wed, 14 Feb 2018 10:05:24 +0100
Thomas Huth <thuth@redhat.com> wrote:

> On 14.02.2018 01:11, Michael Roth wrote:

> > +## enabling mitigations for s390 KVM guests
> > +
> > +For s390 guests there are 2 CPU options relating to Spectre/Meltdown:
> > +
> > +* bpb: Branch prediction blocking
> > +* ppa15: PPA15 is installed
> > +
> > +**bpb** requires a host kernel patched with:
> > +
> > +    commit 35b3fde6203b932b2b1a5b53b3d8808abc9c4f60
> > +    KVM: s390: wire up bpb feature
> > +
> > +and both **bpb** and **ppa15** require a firmware with the appropriate support
> > +level as well as guest kernel patches to enable the functionality within
> > +guests. Please check with your distro/vendor to confirm.
> > +
> > +Both **bpb** and **ppa15** are enabled by default with newer/patched host
> > +kernels, and can also be set manually. For example:
> > +
> > +    qemu-system-s390x -M s390-ccw-virtio-2.11 ... \
> > +      -cpu zEC12,bpb=on,ppa15=on   
> 
> IIRC we only enable them by default with "-cpu host" ? Cornelia, David,
> Christian, can you confirm?

-cpu host enables them if present, as does specifying the full model
(which will fail if not present on the host).

> So maybe better rephrase the above to:
> 
> Both **bpb** and **ppa15** are enabled by default when using "-cpu host"
> and when the host kernels supports these facilities. For other CPU

"and when both the host hardware and the host kernel supports..." ?

(Although that's still a bit misleading, as we only require the bpb KVM
interface; otherwise, the controls are pretty much independent from
what the host is doing IIUC.]

> models, the flags have to be set manually. For example:
> 
>     qemu-system-s390x -M s390-ccw-virtio-2.11 ... \
>       -cpu zEC12,bpb=on,ppa15=on
> 
> > +WRT to migration, enabling **bpb** requires the source/target also have **bpb**
> > +enabled. Since this is enabled by default, you must ensure that **bpb**=off if  
> 
> s/**bpb**=off/**bpb**=off is used/ ?
> 
> > +you wish to maintain migration compatibility with existing guests, or take
> > +steps to reboot guests with **bpb** enabled prior to migrating them.  
> 
>  Thomas

On 14.02.2018 10:18, Christian Borntraeger wrote:
> 
> 
> On 02/14/2018 10:11 AM, Cornelia Huck wrote:
>> On Tue, 13 Feb 2018 18:11:05 -0600
>> Michael Roth <mdroth@linux.vnet.ibm.com> wrote:
>>
>>> This blog entry is intended as a follow-up to the original entry in
>>> January regarding Spectre/Meltdown and the proposed changes to address
>>> them in the upcoming 2.11.1 release.
>>>
>>> This entry is meant to accompany the 2.11.1 release (planned for
>>> 2018-02-14) and document how to make use of the new options for
>>> various architectures.
>>>
>>> Cc: Eduardo Habkost <ehabkost@redhat.com>
>>> Cc: Paolo Bonzini <pbonzini@redhat.com>
>>> Cc: Peter Maydell <peter.maydell@linaro.org>
>>> Cc: Suraj Jitindar Singh <sjitindarsingh@gmail.com>
>>> Cc: David Gibson <david@gibson.dropbear.id.au>
>>> Cc: Christian Borntraeger <borntraeger@de.ibm.com>
>>> Cc: Cornelia Huck <cohuck@redhat.com>
>>> Cc: Thomas Huth <thuth@redhat.com>
>>> Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
>>> ---
>>>
>>> The pseries/s390 bits have gotten some initial review (thanks Suraj/Christian),
>>> but it can definitely use some additional review on the x86 side of things.
>>>
>>> Also, Peter if think anything extra should to be mentioned on the ARM side just
>>> let me know what to add.
>>>
>>>  .../2018-02-14-qemu-2-11-1-and-spectre-update.md   | 180 +++++++++++++++++++++
>>>  1 file changed, 180 insertions(+)
>>>  create mode 100644 _posts/2018-02-14-qemu-2-11-1-and-spectre-update.md
>>
>> [some comments/questions regarding s390 cpu models, adding DavidH on cc:]
>>
>>> +## enabling mitigations for s390 KVM guests
>>> +
>>> +For s390 guests there are 2 CPU options relating to Spectre/Meltdown:
>>
>> s/options/feature bits/ ?
>>
>>> +
>>> +* bpb: Branch prediction blocking
>>> +* ppa15: PPA15 is installed
>>> +
>>> +**bpb** requires a host kernel patched with:
>>> +
>>> +    commit 35b3fde6203b932b2b1a5b53b3d8808abc9c4f60
>>> +    KVM: s390: wire up bpb feature
>>> +
>>> +and both **bpb** and **ppa15** require a firmware with the appropriate support
>>> +level as well as guest kernel patches to enable the functionality within
>>> +guests. Please check with your distro/vendor to confirm.
>>> +
>>> +Both **bpb** and **ppa15** are enabled by default with newer/patched host
>>> +kernels, and can also be set manually. For example:
>>> +
>>> +    qemu-system-s390x -M s390-ccw-virtio-2.11 ... \
>>> +      -cpu zEC12,bpb=on,ppa15=on 
>>
>> Do we also want to add that bpb/ppa15 are on if you use the _full_
>> model (as opposed to the _base_ model)? Or is this going into too much
>> detail about the cpu model?
> 
> full model is just an internal implementation.
> Either use
> - host passthrough
> - host model
> - add bpb and ppa15 manually to the choosen model (e.g. z13,bpb=on,ppa15=on)

Ack. They won't be enabled by existing base (e.g. z13-base) or default
(e.g. z13) CPU models. The full models are internal only.

So "-cpu host" or "-cpu [MODEL],bpb=on,ppa15=on" are the only two
options on the QEMU command line level.

On 14/02/2018 09:51, Daniel P. Berrangé wrote:
>> +Please note that, as mentioned in the previous blog post, QEMU/KVM generally
>> +has the same requirements as other unpriviledged processes running on the
>> +host WRT Spectre/Meltdown mitigation.
>
> Is this actually still considered accurate wrt the host QEMU ? I was under
> the believe that life is more complicated for QEMU/KVM wrt Spectre and that
> it will require more protection than other unpriv processes on the host in
> some cases.

The plan is for KVM to ensure that QEMU can be treated as yet another
unprivileged process.  Anything else would require applying the same
care to all of QEMU's dependencies.

Paolo

Quoting Paolo Bonzini (2018-02-14 04:33:29)
> On 14/02/2018 09:51, Daniel P. Berrangé wrote:
> >> +Please note that, as mentioned in the previous blog post, QEMU/KVM generally
> >> +has the same requirements as other unpriviledged processes running on the
> >> +host WRT Spectre/Meltdown mitigation.
> >
> > Is this actually still considered accurate wrt the host QEMU ? I was under
> > the believe that life is more complicated for QEMU/KVM wrt Spectre and that
> > it will require more protection than other unpriv processes on the host in
> > some cases.
> 
> The plan is for KVM to ensure that QEMU can be treated as yet another
> unprivileged process.  Anything else would require applying the same
> care to all of QEMU's dependencies.

Would the following re-wording be reasonable? The main goal of the
statement is to stress that additional patches pertaining to general
host-side security are still needed to secure a QEMU/KVM host, not
so much to suggest that there isn't anything needed beyond that.

-Please note that, as mentioned in the previous blog post, QEMU/KVM generally
-has the same requirements as other unpriviledged processes running on the
-host WRT Spectre/Meltdown mitigation. What is being addressed here is
-enabling a guest operating system to enable the same (or similar) mitigations
-to protect itself from unpriviledged guest processes. Thus, the
-patches/requirements listed here are specific to that goal and should not be
-regarded as the full set of requirements to enable mitigations on the host
-side (though in some cases there is some overlap between the two WRT required
-patches/etc).

+Please note that QEMU/KVM has at least the same requirements as other
+unpriviledged processes running on the host WRT Spectre/Meltdown
+mitigation. What is being addressed here is enabling a guest operating system
+to enable the same (or similar) mitigations to protect itself from
+unpriviledged guest processes. Thus, the patches/requirements listed here are
+specific to that goal and should not be regarded as the full set of
+requirements to enable mitigations on the host side (though in some cases
+there is some overlap between the two WRT required patches/etc).


> 
> Paolo
>

* Michael Roth (mdroth@linux.vnet.ibm.com) wrote:
> This blog entry is intended as a follow-up to the original entry in
> January regarding Spectre/Meltdown and the proposed changes to address
> them in the upcoming 2.11.1 release.
> 
> This entry is meant to accompany the 2.11.1 release (planned for
> 2018-02-14) and document how to make use of the new options for
> various architectures.
> 
> Cc: Eduardo Habkost <ehabkost@redhat.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Peter Maydell <peter.maydell@linaro.org>
> Cc: Suraj Jitindar Singh <sjitindarsingh@gmail.com>
> Cc: David Gibson <david@gibson.dropbear.id.au>
> Cc: Christian Borntraeger <borntraeger@de.ibm.com>
> Cc: Cornelia Huck <cohuck@redhat.com>
> Cc: Thomas Huth <thuth@redhat.com>
> Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> ---
> 
> The pseries/s390 bits have gotten some initial review (thanks Suraj/Christian),
> but it can definitely use some additional review on the x86 side of things.
> 
> Also, Peter if think anything extra should to be mentioned on the ARM side just
> let me know what to add.
> 
>  .../2018-02-14-qemu-2-11-1-and-spectre-update.md   | 180 +++++++++++++++++++++
>  1 file changed, 180 insertions(+)
>  create mode 100644 _posts/2018-02-14-qemu-2-11-1-and-spectre-update.md
> 
> diff --git a/_posts/2018-02-14-qemu-2-11-1-and-spectre-update.md b/_posts/2018-02-14-qemu-2-11-1-and-spectre-update.md
> new file mode 100644
> index 0000000..7cdea59
> --- /dev/null
> +++ b/_posts/2018-02-14-qemu-2-11-1-and-spectre-update.md
> @@ -0,0 +1,180 @@
> +---
> +layout: post
> +title:  "QEMU 2.11.1 and making use of Spectre/Meltdown mitigation for KVM guests"
> +date: 2018-02-14 10:35:44 -0600
> +author: Michael Roth
> +categories: [meltdown, spectre, security, x86, ppc, s390, releases, 'qemu 2.11']
> +---
> +
> +In a [previous post](https://www.qemu.org/2018/01/04/spectre/) it was
> +detailed how QEMU/KVM might be affected by Spectre/Meltdown attacks, and what
> +the plan was to mitigate them in QEMU 2.11.1 (and eventually QEMU 2.12).
> +
> +QEMU 2.11.1 is now available, and contains the aforementioned mitigations for
> +x86 guests, along with additional mitigation functionality for pseries and
> +s390 guests (ARM guests do not currently require additional QEMU patches).
> +However, enabling this functionality requires additional configuration beyond
> +just updating QEMU, which we hope to address with this post.
> +
> +Please note that, as mentioned in the previous blog post, QEMU/KVM generally
> +has the same requirements as other unpriviledged processes running on the
> +host WRT Spectre/Meltdown mitigation. What is being addressed here is
> +enabling a guest operating system to enable the same (or similar) mitigations
> +to protect itself from unpriviledged guest processes. Thus, the
> +patches/requirements listed here are specific to that goal and should not be
> +regarded as the full set of requirements to enable mitigations on the host
> +side (though in some cases there is some overlap between the two WRT required
> +patches/etc).
> +
> +Also please note that this is a best-effort from the QEMU/KVM community, and
> +these mitigations rely on a mix of additional kernel/firmware/microcode
> +updates that are in some cases not available publically, or may not yet be
> +implemented in some distros, so users are highly encouraged to consult with
> +their respective vendors/distros to confirm whether all the required
> +components are in place. We do our best to highlight the requirements here,
> +but this may not be an exhaustive list.
> +
> +
> +## enabling mitigations for x86 KVM guests
> +
> +For x86 guests there are 2 additional CPU flags associated with
> +Spectre/Meltdown mitigation: **spec-ctrl**, and **ibpb**. These flags
> +expose additional functionality made available through new microcode
> +updates for certain Intel/AMD processors that can be used to mitigate
> +various attack vectors related to Spectre. (Meltdown mitigation via KPTI
> +does not require additional CPU functionality or microcode, and does not
> +require an updated QEMU, only the related guest/host kernel patches).
> +
> +These CPU flags:
> +
> +* spec-ctrl: exposes Indirect Branch Restricted Speculation (IBRS)
> +* ibpb: exposes Indirect Branch Prediction Barriers
> +
> +are both features requiring guest/host kernel updates, as well as
> +microcode updates for Intel and recent AMD processors. The status of
> +these kernel patches upstream is still in flux, but most supported
> +distros have some form of the patches that is sufficient to make use
> +of the features. The current status/availability of microcode updates
> +depends on your CPU architecture/model. Please check with your
> +vendor/distro to confirm these prerequisites are available/installed.
> +
> +Generally, for Intel CPUs with updated microcode, **spec-ctrl** will
> +enable both IBRS and IBPB functionality. For AMD EPYC processors,
> +**ibpb** can be used to enable IBPB specifically, and is thought to
> +be sufficient by itself that particular architecture.
> +
> +These flags can be set in a similar manner as other CPU flags, i.e.:
> +
> +    qemu-system-x86_64 -cpu qemu64,+spec-ctrl,... ...
> +    qemu-system-x86_64 -cpu IvyBridge,+spec-ctrl,... ...
> +    qemu-system-x86_64 -cpu EPYC,+ibpb
> +    etc...
> +
> +Additionally, for management stacks that lack support for setting
> +specific CPU flags, a set of new CPU types have been added which
> +enable the appropriate CPU flags automatically:
> +
> +    qemu-system-x86_64 -cpu Nehalem-IBRS ...
> +    qemu-system-x86_64 -cpu Westmere-IBRS ...
> +    qemu-system-x86_64 -cpu SandyBridge-IBRS ...
> +    qemu-system-x86_64 -cpu IvyBridge-IBRS ...
> +    qemu-system-x86_64 -cpu Haswell-IBRS ...
> +    qemu-system-x86_64 -cpu Haswell-noTSX-IBRS ...
> +    qemu-system-x86_64 -cpu Broadwell-IBRS ...
> +    qemu-system-x86_64 -cpu Broadwell-noTSX-IBRS ...
> +    qemu-system-x86_64 -cpu Skylake-Client-IBRS ...
> +    qemu-system-x86_64 -cpu Skylake-Server-IBRS ...
> +    qemu-system-x86_64 -cpu EPYC-IBPB ...
> +
> +With these settings enabled, guests may still require additional
> +configuration to enable IBRS/IBPB, which may vary somewhat from one
> +distro to another. For RHEL guests, the following resource may be
> +useful:
> +
> +* https://access.redhat.com/articles/3311301
> +
> +WRT migration compatibility, **spec-ctrl**/**ibrs** (or the corresponding
> +CPU type) should be set the same on both source/target to maintain
> +compatibility. Thus, guests will need to be rebooted to make use of
> +the new features.

There is an unfortunate wrinkle with respect to CPU type compatibility
on x86;  the retpoline code is looking like it has to do special
cases for Skylake and other recent CPUs (fluching the return buffer) and
it does this on the basis of CPUid/model rather than flags;  so from
QEMUs point of view, the normal trick we do about giving guests a false
earlier CPU name (e.g. -cpu IvyBridge) and expecting them to run across
a wide range of more modern hardware may not be secure.

Dave

> +## enabling mitigations for pseries KVM guests
> +
> +For pseries guests there are 3 tri-state -machine options/capabilities
> +relating to Spectre/Meltdown mitigation: **cap-cfpc**, **cap-sbbc**,
> +**cap-ibs**, which each correspond to a set of host machine capabilities
> +advertised by the KVM kernel module in new/patched host kernels that can
> +be used to mitigate various aspects of Spectre/Meltdown:
> +
> +* cap-cfpc: Cache Flush on Privilege Change
> +* cap-sbbc: Speculation Barrier Bounds Checking
> +* cap-ibs: Indirect Branch Serialisation
> +
> +Each option can be set to one of "broken", "workaround", or "fixed", which
> +correspond, respectively, to instructing the guest whether the host is
> +vulnerable, has OS-level workarounds available, or has hardware/firmware
> +that does not require OS-level workarounds. Based on these options, QEMU
> +will perform checks to validate whether the specified settings are available
> +on the current host and pass these settings on to the guest kernel. At a
> +minimum, any setting other than "broken" will require a host kernel that has
> +some form of the following patches:
> +
> +    commit 3214d01f139b7544e870fc0b7fcce8da13c1cb51
> +    KVM: PPC: Book3S: Provide information about hardware/firmware CVE workarounds
> +  
> +    commit 191eccb1580939fb0d47deb405b82a85b0379070
> +    powerpc/pseries: Add H_GET_CPU_CHARACTERISTICS flags & wrapper
> +
> +and whether a host will support "workaround" and "fixed" settings for each
> +option will depend on the hardware/firmware level of the host system.
> +
> +In turn, to make use of "workaround" or "fixed" settings for each option,
> +the guest kernel will require at least the following set of patches:
> +
> +* https://lists.ozlabs.org/pipermail/linuxppc-dev/2018-January/167455.html
> +
> +These are available upstream and have been backported to a number of stable
> +kernels. Please check with your vendor/distro to confirm the required
> +hardware/firmware and guest kernel patches are available/installed.
> +
> +By default, all three options, **cap-cfpc**, **cap-sbbc**, and **cap-ibs**
> +default to "broken" to maintain compatibility with previous versions of QEMU
> +and unpatched host kernels. To enable them you must start QEMU with the
> +desired mitigation strategy specified explicitly. For example:
> +
> +    qemu-system-ppc64 ... \
> +      -machine pseries-2.11,cap-cfpc=workaround,cap-sbbc=workaround,cap-ibs=fixed
> +
> +WRT migration compatibility, setting any of these features to a value other
> +than "broken" will require an identical setting for that option on the
> +source/destination guest. To enable these settings your guests will need to
> +be rebooted at some point.
> +
> +
> +## enabling mitigations for s390 KVM guests
> +
> +For s390 guests there are 2 CPU options relating to Spectre/Meltdown:
> +
> +* bpb: Branch prediction blocking
> +* ppa15: PPA15 is installed
> +
> +**bpb** requires a host kernel patched with:
> +
> +    commit 35b3fde6203b932b2b1a5b53b3d8808abc9c4f60
> +    KVM: s390: wire up bpb feature
> +
> +and both **bpb** and **ppa15** require a firmware with the appropriate support
> +level as well as guest kernel patches to enable the functionality within
> +guests. Please check with your distro/vendor to confirm.
> +
> +Both **bpb** and **ppa15** are enabled by default with newer/patched host
> +kernels, and can also be set manually. For example:
> +
> +    qemu-system-s390x -M s390-ccw-virtio-2.11 ... \
> +      -cpu zEC12,bpb=on,ppa15=on 
> +
> +WRT to migration, enabling **bpb** requires the source/target also have **bpb**
> +enabled. Since this is enabled by default, you must ensure that **bpb**=off if
> +you wish to maintain migration compatibility with existing guests, or take
> +steps to reboot guests with **bpb** enabled prior to migrating them.
> -- 
> 2.11.0
> 
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Quoting Dr. David Alan Gilbert (2018-02-14 10:50:40)
> * Michael Roth (mdroth@linux.vnet.ibm.com) wrote:
> > This blog entry is intended as a follow-up to the original entry in
> > January regarding Spectre/Meltdown and the proposed changes to address
> > them in the upcoming 2.11.1 release.
> > 
> > This entry is meant to accompany the 2.11.1 release (planned for
> > 2018-02-14) and document how to make use of the new options for
> > various architectures.
> > 

<snip>

> > +
> > +
> > +## enabling mitigations for x86 KVM guests
> > +
> > +For x86 guests there are 2 additional CPU flags associated with
> > +Spectre/Meltdown mitigation: **spec-ctrl**, and **ibpb**. These flags
> > +expose additional functionality made available through new microcode
> > +updates for certain Intel/AMD processors that can be used to mitigate
> > +various attack vectors related to Spectre. (Meltdown mitigation via KPTI
> > +does not require additional CPU functionality or microcode, and does not
> > +require an updated QEMU, only the related guest/host kernel patches).
> > +
> > +These CPU flags:
> > +
> > +* spec-ctrl: exposes Indirect Branch Restricted Speculation (IBRS)
> > +* ibpb: exposes Indirect Branch Prediction Barriers
> > +
> > +are both features requiring guest/host kernel updates, as well as
> > +microcode updates for Intel and recent AMD processors. The status of
> > +these kernel patches upstream is still in flux, but most supported
> > +distros have some form of the patches that is sufficient to make use
> > +of the features. The current status/availability of microcode updates
> > +depends on your CPU architecture/model. Please check with your
> > +vendor/distro to confirm these prerequisites are available/installed.
> > +
> > +Generally, for Intel CPUs with updated microcode, **spec-ctrl** will
> > +enable both IBRS and IBPB functionality. For AMD EPYC processors,
> > +**ibpb** can be used to enable IBPB specifically, and is thought to
> > +be sufficient by itself that particular architecture.
> > +
> > +These flags can be set in a similar manner as other CPU flags, i.e.:
> > +
> > +    qemu-system-x86_64 -cpu qemu64,+spec-ctrl,... ...
> > +    qemu-system-x86_64 -cpu IvyBridge,+spec-ctrl,... ...
> > +    qemu-system-x86_64 -cpu EPYC,+ibpb
> > +    etc...
> > +
> > +Additionally, for management stacks that lack support for setting
> > +specific CPU flags, a set of new CPU types have been added which
> > +enable the appropriate CPU flags automatically:
> > +
> > +    qemu-system-x86_64 -cpu Nehalem-IBRS ...
> > +    qemu-system-x86_64 -cpu Westmere-IBRS ...
> > +    qemu-system-x86_64 -cpu SandyBridge-IBRS ...
> > +    qemu-system-x86_64 -cpu IvyBridge-IBRS ...
> > +    qemu-system-x86_64 -cpu Haswell-IBRS ...
> > +    qemu-system-x86_64 -cpu Haswell-noTSX-IBRS ...
> > +    qemu-system-x86_64 -cpu Broadwell-IBRS ...
> > +    qemu-system-x86_64 -cpu Broadwell-noTSX-IBRS ...
> > +    qemu-system-x86_64 -cpu Skylake-Client-IBRS ...
> > +    qemu-system-x86_64 -cpu Skylake-Server-IBRS ...
> > +    qemu-system-x86_64 -cpu EPYC-IBPB ...
> > +
> > +With these settings enabled, guests may still require additional
> > +configuration to enable IBRS/IBPB, which may vary somewhat from one
> > +distro to another. For RHEL guests, the following resource may be
> > +useful:
> > +
> > +* https://access.redhat.com/articles/3311301
> > +
> > +WRT migration compatibility, **spec-ctrl**/**ibrs** (or the corresponding
> > +CPU type) should be set the same on both source/target to maintain
> > +compatibility. Thus, guests will need to be rebooted to make use of
> > +the new features.
> 
> There is an unfortunate wrinkle with respect to CPU type compatibility
> on x86;  the retpoline code is looking like it has to do special
> cases for Skylake and other recent CPUs (fluching the return buffer) and
> it does this on the basis of CPUid/model rather than flags;  so from
> QEMUs point of view, the normal trick we do about giving guests a false
> earlier CPU name (e.g. -cpu IvyBridge) and expecting them to run across
> a wide range of more modern hardware may not be secure.

Hmm, that's referring to this thread and Eduardo's concerns?
  https://lkml.org/lkml/2018/1/20/162

In the context of enabling IBRS in guests, wouldn't
Skylake host <-> non-Skylake host migration still be safe, assuming the
IBRS bits are properly enabled? Even with the patches proposed in that
thread?

For the purposes of this document I was sort of staying away from
retpoline since AFAIK IBRS is still considered more complete and seems
to be more widely deployed. But perhaps we can add note to clarify and
maybe help users plan better for future developments of the sort. Maybe
something like the following?


Note that this document centers around Spectre mitigation via IBRS/IBPB
features. Alternative approaches to addressing Spectre exist in the form
of 'retpoline'-enabled kernels and are outside the scope of this
document. However, if you plan on utilizing such a kernel in place of IBRS,
either now or in the future, note that some proposed forms of the code
still rely on IBRS as a fallback for certain architectures like Skylake.
In such cases it may be necessary to ensure that guests running on Skylake
hosts utilize a matching -cpu Skylake* model with appropriate IBRS options
enabled, otherwise such fallbacks may be unavailable and leave the guest
vulnerable. For more background on this particular consideration you can
view the discussion here:

https://lkml.org/lkml/2018/1/29/634

> 
> Dave
> 

<snip>

> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
>

* Michael Roth (mdroth@linux.vnet.ibm.com) wrote:
> Quoting Dr. David Alan Gilbert (2018-02-14 10:50:40)
> > * Michael Roth (mdroth@linux.vnet.ibm.com) wrote:
> > > This blog entry is intended as a follow-up to the original entry in
> > > January regarding Spectre/Meltdown and the proposed changes to address
> > > them in the upcoming 2.11.1 release.
> > > 
> > > This entry is meant to accompany the 2.11.1 release (planned for
> > > 2018-02-14) and document how to make use of the new options for
> > > various architectures.
> > > 
> 
> <snip>
> 
> > > +
> > > +
> > > +## enabling mitigations for x86 KVM guests
> > > +
> > > +For x86 guests there are 2 additional CPU flags associated with
> > > +Spectre/Meltdown mitigation: **spec-ctrl**, and **ibpb**. These flags
> > > +expose additional functionality made available through new microcode
> > > +updates for certain Intel/AMD processors that can be used to mitigate
> > > +various attack vectors related to Spectre. (Meltdown mitigation via KPTI
> > > +does not require additional CPU functionality or microcode, and does not
> > > +require an updated QEMU, only the related guest/host kernel patches).
> > > +
> > > +These CPU flags:
> > > +
> > > +* spec-ctrl: exposes Indirect Branch Restricted Speculation (IBRS)
> > > +* ibpb: exposes Indirect Branch Prediction Barriers
> > > +
> > > +are both features requiring guest/host kernel updates, as well as
> > > +microcode updates for Intel and recent AMD processors. The status of
> > > +these kernel patches upstream is still in flux, but most supported
> > > +distros have some form of the patches that is sufficient to make use
> > > +of the features. The current status/availability of microcode updates
> > > +depends on your CPU architecture/model. Please check with your
> > > +vendor/distro to confirm these prerequisites are available/installed.
> > > +
> > > +Generally, for Intel CPUs with updated microcode, **spec-ctrl** will
> > > +enable both IBRS and IBPB functionality. For AMD EPYC processors,
> > > +**ibpb** can be used to enable IBPB specifically, and is thought to
> > > +be sufficient by itself that particular architecture.
> > > +
> > > +These flags can be set in a similar manner as other CPU flags, i.e.:
> > > +
> > > +    qemu-system-x86_64 -cpu qemu64,+spec-ctrl,... ...
> > > +    qemu-system-x86_64 -cpu IvyBridge,+spec-ctrl,... ...
> > > +    qemu-system-x86_64 -cpu EPYC,+ibpb
> > > +    etc...
> > > +
> > > +Additionally, for management stacks that lack support for setting
> > > +specific CPU flags, a set of new CPU types have been added which
> > > +enable the appropriate CPU flags automatically:
> > > +
> > > +    qemu-system-x86_64 -cpu Nehalem-IBRS ...
> > > +    qemu-system-x86_64 -cpu Westmere-IBRS ...
> > > +    qemu-system-x86_64 -cpu SandyBridge-IBRS ...
> > > +    qemu-system-x86_64 -cpu IvyBridge-IBRS ...
> > > +    qemu-system-x86_64 -cpu Haswell-IBRS ...
> > > +    qemu-system-x86_64 -cpu Haswell-noTSX-IBRS ...
> > > +    qemu-system-x86_64 -cpu Broadwell-IBRS ...
> > > +    qemu-system-x86_64 -cpu Broadwell-noTSX-IBRS ...
> > > +    qemu-system-x86_64 -cpu Skylake-Client-IBRS ...
> > > +    qemu-system-x86_64 -cpu Skylake-Server-IBRS ...
> > > +    qemu-system-x86_64 -cpu EPYC-IBPB ...
> > > +
> > > +With these settings enabled, guests may still require additional
> > > +configuration to enable IBRS/IBPB, which may vary somewhat from one
> > > +distro to another. For RHEL guests, the following resource may be
> > > +useful:
> > > +
> > > +* https://access.redhat.com/articles/3311301
> > > +
> > > +WRT migration compatibility, **spec-ctrl**/**ibrs** (or the corresponding
> > > +CPU type) should be set the same on both source/target to maintain
> > > +compatibility. Thus, guests will need to be rebooted to make use of
> > > +the new features.
> > 
> > There is an unfortunate wrinkle with respect to CPU type compatibility
> > on x86;  the retpoline code is looking like it has to do special
> > cases for Skylake and other recent CPUs (fluching the return buffer) and
> > it does this on the basis of CPUid/model rather than flags;  so from
> > QEMUs point of view, the normal trick we do about giving guests a false
> > earlier CPU name (e.g. -cpu IvyBridge) and expecting them to run across
> > a wide range of more modern hardware may not be secure.
> 
> Hmm, that's referring to this thread and Eduardo's concerns?
>   https://lkml.org/lkml/2018/1/20/162

Yes.

> In the context of enabling IBRS in guests, wouldn't
> Skylake host <-> non-Skylake host migration still be safe, assuming the
> IBRS bits are properly enabled? Even with the patches proposed in that
> thread?

That's my understanding.

> For the purposes of this document I was sort of staying away from
> retpoline since AFAIK IBRS is still considered more complete and seems
> to be more widely deployed.

I think the retpoline stuff is catching up.

> But perhaps we can add note to clarify and
> maybe help users plan better for future developments of the sort. Maybe
> something like the following?
> 
> 
> Note that this document centers around Spectre mitigation via IBRS/IBPB
> features. Alternative approaches to addressing Spectre exist in the form
> of 'retpoline'-enabled kernels and are outside the scope of this
> document. However, if you plan on utilizing such a kernel in place of IBRS,
> either now or in the future, note that some proposed forms of the code
> still rely on IBRS as a fallback for certain architectures like Skylake.
> In such cases it may be necessary to ensure that guests running on Skylake
> hosts utilize a matching -cpu Skylake* model with appropriate IBRS options
> enabled, otherwise such fallbacks may be unavailable and leave the guest
> vulnerable. For more background on this particular consideration you can
> view the discussion here:
> 
> https://lkml.org/lkml/2018/1/29/634

It's a bit weird to end up pointing to them to a lkml post;
Paolo: Any ideas on anything better to say?

Dave

> > 
> > Dave
> > 
> 
> <snip>
> 
> > --
> > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> > 
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

On 16/02/2018 12:57, Dr. David Alan Gilbert wrote:
> It's a bit weird to end up pointing to them to a lkml post;
> Paolo: Any ideas on anything better to say?

IBRS is really that much slower than retpolines on pre-Skylake machines,
so it's a hard call to tell people to use it.  It looks like Intel is
going to add Yet Another Bit to clean up the Skylake mess; in the
meanwhile I would just use retpolines even on Skylake and newer
machines, because the Spectre attacks are much more theoretical than
Meltdown, but it's good to point out the issue for the paranoid folks.

Paolo