Patchwork VM can not boot after commit 235e898

login
register
mail settings
Submitter Alexander Graf
Date July 24, 2013, 4:26 p.m.
Message ID <51F00041.3010005@suse.de>
Download mbox | patch
Permalink /patch/261457/
State New
Headers show

Comments

Alexander Graf - July 24, 2013, 4:26 p.m.
On 07/24/2013 06:17 PM, Gleb Natapov wrote:
> On Wed, Jul 24, 2013 at 05:31:14PM +0200, Alexander Graf wrote:
>> On 07/24/2013 05:21 PM, Gleb Natapov wrote:
>>> On Wed, Jul 24, 2013 at 05:16:09PM +0200, Paolo Bonzini wrote:
>>>> Il 24/07/2013 11:58, Alexander Graf ha scritto:
>>>>>>> No QEMU or kvm crashes, no error message printed, I mean it just hangs, even no BIOS information are printed.
>>>>>>> And "top" shows QEMU consumes 100% cpu.
>>>>>>>
>>>>>>> When I define DEBUG_KVM in kvm-all.c, and run QEMU(this time I boot a normal OS disk),
>>>>>>> # x86_64-softmmu/qemu-system-x86_64 -enable-kvm -hda /mnt/nfs/Images/debian-append.img
>>>>>>> kvm_init_vcpu
>>>>>>> kvm_cpu_exec()
>>>>>>> handle_io
>>>>>>> handle_io
>>>>>>> handle_io
>>>>>>> handle_io
>>>>>>>
>>>>>>> Only 4 debug messages(handle_io) are printed, then nothing is shown, and "top" shows QEMU process uses 100% CPU.
>>>>> After this we're running in an endless loop of:
>>>>>
>>>>>   qemu-system-x86-9298  [003] ...1 162090.918845: kvm_emulate_insn: f0000:c489:66 ea 91 c4 0f 00 08 00 (prot16)
>>>>>   qemu-system-x86-9298  [003] d..2 162090.918846: kvm_entry: vcpu 0
>>>>>
>>>>>    (qemu) x /i $pc
>>>>>    0x00000000000fc489:  ljmpl  $0x8,$0xfc491
>>>>>
>>>>> With current master, qemu-system-x86_64 -enable-kvm is broken on at least 3.7 kernels (openSUSE 12.3).
>>>>>
>>>>> Gleb, I don't remember all the glorious details of ljmpl, but would it have to raise an MMIO request for a read-only memory slot which it fails to do?
>>>> The point of KVM_CAP_READONLY_MEM should be that it doesn't.
>>>>
>>> Yes, it should not. Can you provide complete trace of kvm and kvmmmu
>>> event up until failure?
>> Sure! These are all trace events up to the loop that I was able to
>> fetch from the "kvm" and "kvmmmu" event bucket in
>> /sys/kernel/debug/tracing.
>>
> You should start using trace-cmd :) It even disassembles for you.
>
>>   qemu-system-x86-13150 [000] d..2 185370.441825: kvm_entry: vcpu 0
>>   qemu-system-x86-13150 [000] d..2 185370.441826: kvm_exit: reason EXCEPTION_NMI rip 0xc486 info 0 80000b0d
>>   qemu-system-x86-13150 [000] ...1 185370.441826: kvm_emulate_insn: f0000:c486:0f 22 c0 (real)
> This mov CR0 that sets PE bit.
>
>>   qemu-system-x86-13150 [000] d..2 185370.441829: kvm_entry: vcpu 0
>>   qemu-system-x86-13150 [000] ...1 185370.441830: kvm_emulate_insn: f0000:c489:66 ea 91 c4 0f 00 08 00 (prot16)
> Here jmp is emulated because vcpu state is invalid, but for some reason
> emulation does not fail and does not succeed. Never saw such thing

It works just fine with older QEMU:

  qemu-system-x86-9448  [001] d..2 162748.223935: kvm_exit: reason 
IO_INSTRUCTION rip 0xc471 info 920040 0
  qemu-system-x86-9448  [001] ...1 162748.223936: kvm_pio: pio_write at 
0x92 size 1 count 1
  qemu-system-x86-9448  [001] ...1 162748.223936: kvm_userspace_exit: 
reason KVM_EXIT_IO (2)
  qemu-system-x86-9448  [001] d..2 162748.223939: kvm_entry: vcpu 0
  qemu-system-x86-9448  [001] d..2 162748.223940: kvm_exit: reason 
EXCEPTION_NMI rip 0xc473 info 0 80000b0d
  qemu-system-x86-9448  [001] ...1 162748.223942: kvm_emulate_insn: 
f0000:c473:2e 0f 01 1e e0 d3 (real)
  qemu-system-x86-9448  [001] d..2 162748.223945: kvm_entry: vcpu 0
  qemu-system-x86-9448  [001] d..2 162748.223946: kvm_exit: reason 
EXCEPTION_NMI rip 0xc479 info 0 80000b0d
  qemu-system-x86-9448  [001] ...1 162748.223947: kvm_emulate_insn: 
f0000:c479:2e 0f 01 16 a0 d3 (real)
  qemu-system-x86-9448  [001] d..2 162748.223948: kvm_entry: vcpu 0
  qemu-system-x86-9448  [001] d..2 162748.223948: kvm_exit: reason 
EXCEPTION_NMI rip 0xc47f info 0 80000b0d
  qemu-system-x86-9448  [001] ...1 162748.223950: kvm_emulate_insn: 
f0000:c47f:0f 20 c0 (real)
  qemu-system-x86-9448  [001] d..2 162748.223951: kvm_entry: vcpu 0
  qemu-system-x86-9448  [001] d..2 162748.223951: kvm_exit: reason 
EXCEPTION_NMI rip 0xc486 info 0 80000b0d
  qemu-system-x86-9448  [001] ...1 162748.223952: kvm_emulate_insn: 
f0000:c486:0f 22 c0 (real)
  qemu-system-x86-9448  [001] d..2 162748.223955: kvm_entry: vcpu 0
  qemu-system-x86-9448  [001] ...1 162748.223956: kvm_emulate_insn: 
f0000:c489:66 ea 91 c4 0f 00 08 00 (prot16)
  qemu-system-x86-9448  [001] d..2 162748.223959: kvm_entry: vcpu 0
  qemu-system-x86-9448  [001] ...1 162748.223960: kvm_emulate_insn: 
0:fc491:b8 10 00 00 00 (prot32)
  qemu-system-x86-9448  [001] d..2 162748.223961: kvm_entry: vcpu 0
  qemu-system-x86-9448  [001] ...1 162748.223961: kvm_emulate_insn: 
0:fc496:8e d8 (prot32)
  qemu-system-x86-9448  [001] d..2 162748.223963: kvm_entry: vcpu 0
  qemu-system-x86-9448  [001] ...1 162748.223964: kvm_emulate_insn: 
0:fc498:8e c0 (prot32)
  qemu-system-x86-9448  [001] d..2 162748.223965: kvm_entry: vcpu 0
[...]

> before. Are you saying configuring BIOS memslot differently solves the
> problem?

Git bisect pointed to the commit mentioned in this email. The following 
patch also gets me a working guest again:



Alex
Gleb Natapov - July 24, 2013, 4:53 p.m.
On Wed, Jul 24, 2013 at 06:26:41PM +0200, Alexander Graf wrote:
> >before. Are you saying configuring BIOS memslot differently solves the
> >problem?
> 
> Git bisect pointed to the commit mentioned in this email. The
> following patch also gets me a working guest again:
> 
> diff --git a/kvm-all.c b/kvm-all.c
> index 4fb4ccb..deca9e5 100644
> --- a/kvm-all.c
> +++ b/kvm-all.c
> @@ -1455,7 +1455,7 @@ int kvm_init(void)
>          s->irq_set_ioctl = KVM_IRQ_LINE_STATUS;
>      }
> 
> -#ifdef KVM_CAP_READONLY_MEM
> +#if 0 //def KVM_CAP_READONLY_MEM
>      kvm_readonly_mem_allowed =
>          (kvm_check_extension(s, KVM_CAP_READONLY_MEM) > 0);
>  #endif
> 
Can you disable emulate_invalid_state on 3.7? What happens on upstream kernel
(works for me obviously :)).

--
			Gleb.
Alexander Graf - July 24, 2013, 8:25 p.m.
On 07/24/2013 06:53 PM, Gleb Natapov wrote:
> On Wed, Jul 24, 2013 at 06:26:41PM +0200, Alexander Graf wrote:
>>> before. Are you saying configuring BIOS memslot differently solves the
>>> problem?
>> Git bisect pointed to the commit mentioned in this email. The
>> following patch also gets me a working guest again:
>>
>> diff --git a/kvm-all.c b/kvm-all.c
>> index 4fb4ccb..deca9e5 100644
>> --- a/kvm-all.c
>> +++ b/kvm-all.c
>> @@ -1455,7 +1455,7 @@ int kvm_init(void)
>>           s->irq_set_ioctl = KVM_IRQ_LINE_STATUS;
>>       }
>>
>> -#ifdef KVM_CAP_READONLY_MEM
>> +#if 0 //def KVM_CAP_READONLY_MEM
>>       kvm_readonly_mem_allowed =
>>           (kvm_check_extension(s, KVM_CAP_READONLY_MEM)>  0);
>>   #endif
>>
> Can you disable emulate_invalid_state on 3.7?

I could only find emulate_invalid_guest_state. I suppose you mean that 
one? :)

$ rmmod kvm-intel
$ modprobe kvm-intel emulate_invalid_guest_state=n
$ ./x86_64-softmmu/qemu-system-x86_64 -nographic -kernel /boot/vmlinuz 
-append console=ttyS0 -bios pc-bios/bios.bin -enable-kvm
QEMU 1.5.50 monitor - type 'help' for more information
(qemu)
KVM: entry failed, hardware error 0x80000021

If you're running a guest on an Intel machine without unrestricted mode
support, the failure can be most likely due to the guest entering an invalid
state for Intel VT. For example, the guest maybe running in big real mode
which is not supported on less recent Intel processors.

EAX=00000011 EBX=18ae1000 ECX=00006a12 EDX=000fffa9
ESI=07feb50d EDI=00000000 EBP=000069d2 ESP=000069d2
EIP=0000c489 EFL=00010006 [-----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =fd39 000fd390 ffffffff 00809300 DPL=0 DS16 [-WA]
CS =f000 000f0000 0000ffff 00009b00 DPL=0 CS16 [-RA]
SS =0000 00000000 0000ffff 00009300 DPL=0 DS16 [-WA]
DS =0030 00000000 ffffffff 00809300 DPL=0 DS16 [-WA]
FS =0030 00000000 ffffffff 00809300 DPL=0 DS16 [-WA]
GS =c900 000c9000 ffffffff 00809300 DPL=0 DS16 [-WA]
LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT
TR =0000 00000000 0000ffff 00008b00 DPL=0 TSS32-busy
GDT=     000fd3a8 00000037
IDT=     000fd3e6 00000000
CR0=00000011 CR2=00000000 CR3=00000000 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 
DR3=0000000000000000
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000000
Code=01 1e e0 d3 2e 0f 01 16 a0 d3 0f 20 c0 66 83 c8 01 0f 22 c0 <66> ea 
91 c4 0f 00 08 00 b8 10 00 00 00 8e d8 8e c0 8e d0 8e e0 8e e8 89 c8 ff 
e2 89 c1 b8
QEMU: Terminated


> What happens on upstream kernel
> (works for me obviously :)).

kvm-kmod from 3.9 works.


Alex
Andreas Färber - July 24, 2013, 8:34 p.m.
Am 24.07.2013 18:53, schrieb Gleb Natapov:
> What happens on upstream kernel
> (works for me obviously :)).

3.10.x has been working fine for me on openSUSE 12.3.

Andreas
Gleb Natapov - July 25, 2013, 11:30 a.m.
On Wed, Jul 24, 2013 at 10:25:49PM +0200, Alexander Graf wrote:
> On 07/24/2013 06:53 PM, Gleb Natapov wrote:
> >On Wed, Jul 24, 2013 at 06:26:41PM +0200, Alexander Graf wrote:
> >>>before. Are you saying configuring BIOS memslot differently solves the
> >>>problem?
> >>Git bisect pointed to the commit mentioned in this email. The
> >>following patch also gets me a working guest again:
> >>
> >>diff --git a/kvm-all.c b/kvm-all.c
> >>index 4fb4ccb..deca9e5 100644
> >>--- a/kvm-all.c
> >>+++ b/kvm-all.c
> >>@@ -1455,7 +1455,7 @@ int kvm_init(void)
> >>          s->irq_set_ioctl = KVM_IRQ_LINE_STATUS;
> >>      }
> >>
> >>-#ifdef KVM_CAP_READONLY_MEM
> >>+#if 0 //def KVM_CAP_READONLY_MEM
> >>      kvm_readonly_mem_allowed =
> >>          (kvm_check_extension(s, KVM_CAP_READONLY_MEM)>  0);
> >>  #endif
> >>
> >Can you disable emulate_invalid_state on 3.7?
> 
> I could only find emulate_invalid_guest_state. I suppose you mean
> that one? :)
> 
That one will do :)

> $ rmmod kvm-intel
> $ modprobe kvm-intel emulate_invalid_guest_state=n
> $ ./x86_64-softmmu/qemu-system-x86_64 -nographic -kernel
> /boot/vmlinuz -append console=ttyS0 -bios pc-bios/bios.bin
> -enable-kvm
> QEMU 1.5.50 monitor - type 'help' for more information
> (qemu)
> KVM: entry failed, hardware error 0x80000021
> 
Yeah, emulate_invalid_guest_state=0 was broken for a while. Can you try
applying a4d3326c2de46fd7bcc47d1e8786efccfc152f81 on top of 3.7
and try again with emulate_invalid_guest_state=0?

> >What happens on upstream kernel
> >(works for me obviously :)).
> 
> kvm-kmod from 3.9 works.
> 
Doing backwards bisect to see where it was fixed would be interesting.

--
			Gleb.

Patch

diff --git a/kvm-all.c b/kvm-all.c
index 4fb4ccb..deca9e5 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -1455,7 +1455,7 @@  int kvm_init(void)
          s->irq_set_ioctl = KVM_IRQ_LINE_STATUS;
      }

-#ifdef KVM_CAP_READONLY_MEM
+#if 0 //def KVM_CAP_READONLY_MEM
      kvm_readonly_mem_allowed =
          (kvm_check_extension(s, KVM_CAP_READONLY_MEM) > 0);
  #endif