Patchwork [Oneiric,1/1] x86, kdump, ioapic: Reset remote-IRR in clear_IO_APIC

login
register
mail settings
Submitter Brad Figg
Date Dec. 8, 2011, 7:24 p.m.
Message ID <1323372257-16226-2-git-send-email-brad.figg@canonical.com>
Download mbox | patch
Permalink /patch/130216/
State New
Headers show

Comments

Brad Figg - Dec. 8, 2011, 7:24 p.m.
From: Suresh Siddha <suresh.b.siddha@intel.com>

BugLink: http://bugs.launchpad.net/bugs/901830

In the kdump scenario mentioned below, we can have a case where
the device using level triggered interrupt will not generate any
interrupts in the kdump kernel.

1. IO-APIC sends a level triggered interrupt to the CPU's local APIC.

2. Kernel crashed before the CPU services this interrupt, leaving
   the remote-IRR in the IO-APIC set.

3. kdump kernel boot sequence does clear_IO_APIC() as part of IO-APIC
   initialization. But this fails to reset remote-IRR bit of the
   IO-APIC RTE as the remote-IRR bit is read-only.

4. Device using that level triggered entry can't generate any
   more interrupts because of the remote-IRR bit.

In clear_IO_APIC_pin(), check if the remote-IRR bit is set and if
so do an explicit attempt to clear it (by doing EOI write on
modern io-apic's and changing trigger mode to edge/level on
older io-apic's). Also before doing the explicit EOI to the
io-apic, ensure that the trigger mode is indeed set to level.
This will enable the explicit EOI to the io-apic to reset the
remote-IRR bit.

Tested-by: Leonardo Chiquitto <lchiquitto@novell.com>
Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com>
Fixes: https://bugzilla.novell.com/show_bug.cgi?id=701686
Cc: Rafael Wysocki <rjw@novell.com>
Cc: Maciej W. Rozycki <macro@linux-mips.org>
Cc: Thomas Renninger <trenn@suse.de>
Cc: jbeulich@novell.com
Cc: yinghai@kernel.org
Link: http://lkml.kernel.org/r/20110825190657.157502602@sbsiddha-desk.sc.intel.com
Signed-off-by: Ingo Molnar <mingo@elte.hu>
(cherry picked from commit 1e75b31d638d5242ca8e9771dfdcbd28a5f041df)
Signed-off-by: Brad Figg <brad.figg@canonical.com>
---
 arch/x86/kernel/apic/io_apic.c |   48 +++++++++++++++++++++++++++++++++++++++-
 1 files changed, 47 insertions(+), 1 deletions(-)
Herton Ronaldo Krzesinski - Dec. 9, 2011, 2:29 a.m.
On Thu, Dec 08, 2011 at 11:24:17AM -0800, Brad Figg wrote:
> From: Suresh Siddha <suresh.b.siddha@intel.com>
> 
> BugLink: http://bugs.launchpad.net/bugs/901830
> 
> In the kdump scenario mentioned below, we can have a case where
> the device using level triggered interrupt will not generate any
> interrupts in the kdump kernel.
> 
> 1. IO-APIC sends a level triggered interrupt to the CPU's local APIC.
> 
> 2. Kernel crashed before the CPU services this interrupt, leaving
>    the remote-IRR in the IO-APIC set.
> 
> 3. kdump kernel boot sequence does clear_IO_APIC() as part of IO-APIC
>    initialization. But this fails to reset remote-IRR bit of the
>    IO-APIC RTE as the remote-IRR bit is read-only.
> 
> 4. Device using that level triggered entry can't generate any
>    more interrupts because of the remote-IRR bit.
> 
> In clear_IO_APIC_pin(), check if the remote-IRR bit is set and if
> so do an explicit attempt to clear it (by doing EOI write on
> modern io-apic's and changing trigger mode to edge/level on
> older io-apic's). Also before doing the explicit EOI to the
> io-apic, ensure that the trigger mode is indeed set to level.
> This will enable the explicit EOI to the io-apic to reset the
> remote-IRR bit.
> 
> Tested-by: Leonardo Chiquitto <lchiquitto@novell.com>
> Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com>
> Fixes: https://bugzilla.novell.com/show_bug.cgi?id=701686
> Cc: Rafael Wysocki <rjw@novell.com>
> Cc: Maciej W. Rozycki <macro@linux-mips.org>
> Cc: Thomas Renninger <trenn@suse.de>
> Cc: jbeulich@novell.com
> Cc: yinghai@kernel.org
> Link: http://lkml.kernel.org/r/20110825190657.157502602@sbsiddha-desk.sc.intel.com
> Signed-off-by: Ingo Molnar <mingo@elte.hu>
> (cherry picked from commit 1e75b31d638d5242ca8e9771dfdcbd28a5f041df)
> Signed-off-by: Brad Figg <brad.figg@canonical.com>
> ---
>  arch/x86/kernel/apic/io_apic.c |   48 +++++++++++++++++++++++++++++++++++++++-
>  1 files changed, 47 insertions(+), 1 deletions(-)
> 
> diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
> index e529339..ab40b22 100644
> --- a/arch/x86/kernel/apic/io_apic.c
> +++ b/arch/x86/kernel/apic/io_apic.c
> @@ -593,10 +593,56 @@ static void clear_IO_APIC_pin(unsigned int apic, unsigned int pin)
>  	entry = ioapic_read_entry(apic, pin);
>  	if (entry.delivery_mode == dest_SMI)
>  		return;
> +
> +	/*
> +	 * Make sure the entry is masked and re-read the contents to check
> +	 * if it is a level triggered pin and if the remote-IRR is set.
> +	 */
> +	if (!entry.mask) {
> +		entry.mask = 1;
> +		ioapic_write_entry(apic, pin, entry);
> +		entry = ioapic_read_entry(apic, pin);
> +	}
> +
> +	if (entry.irr) {
> +		/*
> +		 * Make sure the trigger mode is set to level. Explicit EOI
> +		 * doesn't clear the remote-IRR if the trigger mode is not
> +		 * set to level.
> +		 */
> +		if (!entry.trigger) {
> +			entry.trigger = IOAPIC_LEVEL;
> +			ioapic_write_entry(apic, pin, entry);
> +		}
> +
> +		if (mpc_ioapic_ver(apic) >= 0x20) {
> +			unsigned long flags;
> +
> +			raw_spin_lock_irqsave(&ioapic_lock, flags);
> +			io_apic_eoi(apic, entry.vector);
> +			raw_spin_unlock_irqrestore(&ioapic_lock, flags);
> +		} else {
> +			/*
> +			 * Mechanism by which we clear remote-IRR in this
> +			 * case is by changing the trigger mode to edge and
> +			 * back to level.
> +			 */
> +			entry.trigger = IOAPIC_EDGE;
> +			ioapic_write_entry(apic, pin, entry);
> +			entry.trigger = IOAPIC_LEVEL;
> +			ioapic_write_entry(apic, pin, entry);
> +		}
> +	}
> +
>  	/*
> -	 * Disable it in the IO-APIC irq-routing table:
> +	 * Clear the rest of the bits in the IO-APIC RTE except for the mask
> +	 * bit.
>  	 */
>  	ioapic_mask_entry(apic, pin);
> +	entry = ioapic_read_entry(apic, pin);
> +	if (entry.irr)
> +		printk(KERN_ERR "Unable to reset IRR for apic: %d, pin :%d\n",
> +		       mpc_ioapic_id(apic), pin);
>  }
>  
>  static void clear_IO_APIC (void)
> -- 
> 1.7.5.4
> 
> 
> -- 
> kernel-team mailing list
> kernel-team@lists.ubuntu.com
> https://lists.ubuntu.com/mailman/listinfo/kernel-team
>

Patch

diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index e529339..ab40b22 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -593,10 +593,56 @@  static void clear_IO_APIC_pin(unsigned int apic, unsigned int pin)
 	entry = ioapic_read_entry(apic, pin);
 	if (entry.delivery_mode == dest_SMI)
 		return;
+
+	/*
+	 * Make sure the entry is masked and re-read the contents to check
+	 * if it is a level triggered pin and if the remote-IRR is set.
+	 */
+	if (!entry.mask) {
+		entry.mask = 1;
+		ioapic_write_entry(apic, pin, entry);
+		entry = ioapic_read_entry(apic, pin);
+	}
+
+	if (entry.irr) {
+		/*
+		 * Make sure the trigger mode is set to level. Explicit EOI
+		 * doesn't clear the remote-IRR if the trigger mode is not
+		 * set to level.
+		 */
+		if (!entry.trigger) {
+			entry.trigger = IOAPIC_LEVEL;
+			ioapic_write_entry(apic, pin, entry);
+		}
+
+		if (mpc_ioapic_ver(apic) >= 0x20) {
+			unsigned long flags;
+
+			raw_spin_lock_irqsave(&ioapic_lock, flags);
+			io_apic_eoi(apic, entry.vector);
+			raw_spin_unlock_irqrestore(&ioapic_lock, flags);
+		} else {
+			/*
+			 * Mechanism by which we clear remote-IRR in this
+			 * case is by changing the trigger mode to edge and
+			 * back to level.
+			 */
+			entry.trigger = IOAPIC_EDGE;
+			ioapic_write_entry(apic, pin, entry);
+			entry.trigger = IOAPIC_LEVEL;
+			ioapic_write_entry(apic, pin, entry);
+		}
+	}
+
 	/*
-	 * Disable it in the IO-APIC irq-routing table:
+	 * Clear the rest of the bits in the IO-APIC RTE except for the mask
+	 * bit.
 	 */
 	ioapic_mask_entry(apic, pin);
+	entry = ioapic_read_entry(apic, pin);
+	if (entry.irr)
+		printk(KERN_ERR "Unable to reset IRR for apic: %d, pin :%d\n",
+		       mpc_ioapic_id(apic), pin);
 }
 
 static void clear_IO_APIC (void)