| Message ID | 20091016140918.3981cfa2@marrow.netinsight.se |
|---|---|
| State | New, archived |
| Headers | show |
On Fri, 2009-10-16 at 14:09 +0200, Simon Kagstrom wrote: > The core functionality is implemented as per Linus suggestion from > > http://lists.infradead.org/pipermail/linux-mtd/2009-October/027620.html > > (with the kmsg_dump implementation by Linus). A struct kmsg_dumper has > been added which contains a callback to dump the kernel log buffers on > crashes. The kmsg_dump function gets called from oops_exit() and panic() > and invokes this callbacks with the crash reason. > > Signed-off-by: Simon Kagstrom <simon.kagstrom@netinsight.net> > Reviewed-by: Anders Grafstrom <anders.grafstrom@netinsight.net> > Reviewed-by: Ingo Molnar <mingo@elte.hu> > --- > The bikeshed paint is drying: > > ChangeLog: > * (Ingo Molnar): Flatten lock use > * (Artem, Aaro): Fix style issues > > include/linux/kmsg_dump.h | 44 ++++++++++++++++++ > kernel/panic.c | 3 + > kernel/printk.c | 112 +++++++++++++++++++++++++++++++++++++++++++++ > 3 files changed, 159 insertions(+), 0 deletions(-) > create mode 100644 include/linux/kmsg_dump.h I wonder, via which tree this should go in. We are going to have mtdoops depend on this. Should we go one of these ways: * this patch goes to one of Ingo's tip trees, so we can pull it and work on top. * we have Ingo's / Linus' acks, and this goes via the MTD tree. ?
* Artem Bityutskiy <dedekind1@gmail.com> wrote: > On Fri, 2009-10-16 at 14:09 +0200, Simon Kagstrom wrote: > > The core functionality is implemented as per Linus suggestion from > > > > http://lists.infradead.org/pipermail/linux-mtd/2009-October/027620.html > > > > (with the kmsg_dump implementation by Linus). A struct kmsg_dumper has > > been added which contains a callback to dump the kernel log buffers on > > crashes. The kmsg_dump function gets called from oops_exit() and panic() > > and invokes this callbacks with the crash reason. > > > > Signed-off-by: Simon Kagstrom <simon.kagstrom@netinsight.net> > > Reviewed-by: Anders Grafstrom <anders.grafstrom@netinsight.net> > > Reviewed-by: Ingo Molnar <mingo@elte.hu> > > --- > > The bikeshed paint is drying: > > > > ChangeLog: > > * (Ingo Molnar): Flatten lock use > > * (Artem, Aaro): Fix style issues > > > > include/linux/kmsg_dump.h | 44 ++++++++++++++++++ > > kernel/panic.c | 3 + > > kernel/printk.c | 112 +++++++++++++++++++++++++++++++++++++++++++++ > > 3 files changed, 159 insertions(+), 0 deletions(-) > > create mode 100644 include/linux/kmsg_dump.h > > I wonder, via which tree this should go in. We are going to have mtdoops > depend on this. Should we go one of these ways: > > * this patch goes to one of Ingo's tip trees, so we can pull it and > work on top. > * we have Ingo's / Linus' acks, and this goes via the MTD tree. > ? Either way is good to me. Linus, do you have any preferences? Ingo
On Mon, 19 Oct 2009, Ingo Molnar wrote: > > > > I wonder, via which tree this should go in. We are going to have mtdoops > > depend on this. Should we go one of these ways: > > > > * this patch goes to one of Ingo's tip trees, so we can pull it and > > work on top. > > * we have Ingo's / Linus' acks, and this goes via the MTD tree. > > ? > > Either way is good to me. Linus, do you have any preferences? Either works for me. Linus
On Thu, 22 Oct 2009 08:33:11 +0900 (JST) Linus Torvalds <torvalds@linux-foundation.org> wrote: > > > I wonder, via which tree this should go in. We are going to have mtdoops > > > depend on this. Should we go one of these ways: > > > > > > * this patch goes to one of Ingo's tip trees, so we can pull it and > > > work on top. > > > * we have Ingo's / Linus' acks, and this goes via the MTD tree. > > > ? > > > > Either way is good to me. Linus, do you have any preferences? > > Either works for me. I would put it all through Artems MTD tree then so that we keep the new API and the user of it close together then. Thanks to everyone for the reviewing and suggestions! // Simon
On Thu, 2009-10-22 at 08:25 +0200, Simon Kagstrom wrote: > On Thu, 22 Oct 2009 08:33:11 +0900 (JST) > Linus Torvalds <torvalds@linux-foundation.org> wrote: > > > > > I wonder, via which tree this should go in. We are going to have mtdoops > > > > depend on this. Should we go one of these ways: > > > > > > > > * this patch goes to one of Ingo's tip trees, so we can pull it and > > > > work on top. > > > > * we have Ingo's / Linus' acks, and this goes via the MTD tree. > > > > ? > > > > > > Either way is good to me. Linus, do you have any preferences? > > > > Either works for me. > > I would put it all through Artems MTD tree then so that we keep the new > API and the user of it close together then. > > Thanks to everyone for the reviewing and suggestions! > It's dwmw2's tree actually. I just act as his secretary :-) David, could you push this patch to your tree? Ingo's and Linus' acks are there, AFAIU. Simon, I'll look at your other mtdoops patches as well a bit later.
(Removing some people from Cc:) On Thu, 22 Oct 2009 09:36:08 +0300 Artem Bityutskiy <dedekind1@gmail.com> wrote: > > I would put it all through Artems MTD tree then so that we keep the new > > API and the user of it close together then. > > It's dwmw2's tree actually. I just act as his secretary :-) > > David, could you push this patch to your tree? Ingo's and Linus' acks > are there, AFAIU. I know it's Davids tree :-) I just thought it would be a good idea to keep everything together through your tree first before asking David to merge it. But it's better to take in only the core patch first as you suggest. // Simon
On Thu, 2009-10-22 at 08:42 +0200, Simon Kagstrom wrote: > (Removing some people from Cc:) > > On Thu, 22 Oct 2009 09:36:08 +0300 > Artem Bityutskiy <dedekind1@gmail.com> wrote: > > > I would put it all through Artems MTD tree then so that we keep the new > > > API and the user of it close together then. > > > > It's dwmw2's tree actually. I just act as his secretary :-) > > > > David, could you push this patch to your tree? Ingo's and Linus' acks > > are there, AFAIU. > > I know it's Davids tree :-) > > I just thought it would be a good idea to keep everything together > through your tree first before asking David to merge it. > > > But it's better to take in only the core patch first as you suggest. Well, let's wait for what dwmw2 does. But for now I just put this patch to my l2 tree. FYI, I never meant to have this tree for developers, so I always rebased it. Can you leave with this? Or you want me to create a branch which I won't re-base?
On Thu, 22 Oct 2009 11:52:56 +0300 Artem Bityutskiy <dedekind1@gmail.com> wrote: > Well, let's wait for what dwmw2 does. But for now I just put this patch > to my l2 tree. > > FYI, I never meant to have this tree for developers, so I always rebased > it. Can you leave with this? Or you want me to create a branch which I > won't re-base? I'm fine with having it rebased. Thanks, // Simon
* Artem Bityutskiy <dedekind1@gmail.com> wrote: > On Thu, 2009-10-22 at 08:25 +0200, Simon Kagstrom wrote: > > On Thu, 22 Oct 2009 08:33:11 +0900 (JST) > > Linus Torvalds <torvalds@linux-foundation.org> wrote: > > > > > > > I wonder, via which tree this should go in. We are going to have mtdoops > > > > > depend on this. Should we go one of these ways: > > > > > > > > > > * this patch goes to one of Ingo's tip trees, so we can pull it and > > > > > work on top. > > > > > * we have Ingo's / Linus' acks, and this goes via the MTD tree. > > > > > ? > > > > > > > > Either way is good to me. Linus, do you have any preferences? > > > > > > Either works for me. > > > > I would put it all through Artems MTD tree then so that we keep the new > > API and the user of it close together then. > > > > Thanks to everyone for the reviewing and suggestions! > > > > It's dwmw2's tree actually. I just act as his secretary :-) > > David, could you push this patch to your tree? Ingo's and Linus' acks > are there, AFAIU. Yeah, the latest bits also have my: Acked-by: Ingo Molnar <mingo@elte.hu> Ingo
Hi all, On Thu, 2009-10-22 at 08:25 +0200, ext Simon Kagstrom wrote: > Thanks to everyone for the reviewing and suggestions! > I have a couple of questions: 1. If somebody writes a module that uses dumper for uploading the oopses/panics logs via some pay-per-byte medium, since he has no way to know in a module if the panic_on_oops flag is set, he'll have to upload both oops and the following panic, because he does not know for sure that the panic was caused by the oops. Hence he pays twice for the same information, right? I can think of a couple of way to figure it out in the module itself, but I could not think of any clean way to do it. 2. We tried to use panic notifiers mechanism to print additional information that we want to see in mtdoops logs and it worked well, but having the kmsg_dump(KMSG_DUMP_PANIC) before the atomic_notifier_call_chain() breaks this functionality. Can we the call kmsg_dump() after the notifiers had been invoked? Thanks, Atal. > // Simon > -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/
On Fri, 23 Oct 2009 18:53:22 +0300 "Shargorodsky Atal (EXT-Teleca/Helsinki)" <ext-atal.shargorodsky@nokia.com> wrote: > 1. If somebody writes a module that uses dumper for uploading the > oopses/panics logs via some pay-per-byte medium, since he has no way > to know in a module if the panic_on_oops flag is set, he'll have > to upload both oops and the following panic, because he does not > know for sure that the panic was caused by the oops. Hence he > pays twice for the same information, right? > > I can think of a couple of way to figure it out in the module > itself, but I could not think of any clean way to do it. This is correct, and the mtdoops driver has some provisions to handle this. First, there is a parameter to the module to specify whether oopses should be dumped at all - I added this for the particular case that someone has panic_on_oops set. Second, it does not dump oopses directly anyway, but puts it in a work queue. That way, if panic_on_oops is set, it will store the panic but the oops (called from the workqueue) will not get written anyway. > 2. We tried to use panic notifiers mechanism to print additional > information that we want to see in mtdoops logs and it worked well, > but having the kmsg_dump(KMSG_DUMP_PANIC) before the > atomic_notifier_call_chain() breaks this functionality. > Can we the call kmsg_dump() after the notifiers had been invoked? Well, it depends I think. The code currently looks like this: kmsg_dump(KMSG_DUMP_PANIC); /* * If we have crashed and we have a crash kernel loaded let it handle * everything else. * Do we want to call this before we try to display a message? */ crash_kexec(NULL); [... Comments removed] atomic_notifier_call_chain(&panic_notifier_list, 0, buf); And moving kdump_msg() after crash_kexec() will make us miss the message if we have a kexec crash kernel as well. I realise that these two approaches might be complementary and are not likely to be used at the same time, but it's still something to think about. Then again, maybe it's possible to move the panic notifiers above crash_kexec() as well, which would solve things nicely. // Simon
On Mon, 2009-10-26 at 08:41 +0100, ext Simon Kagstrom wrote: > On Fri, 23 Oct 2009 18:53:22 +0300 > "Shargorodsky Atal (EXT-Teleca/Helsinki)" <ext-atal.shargorodsky@nokia.com> wrote: > > > 1. If somebody writes a module that uses dumper for uploading the > > oopses/panics logs via some pay-per-byte medium, since he has no way > > to know in a module if the panic_on_oops flag is set, he'll have > > to upload both oops and the following panic, because he does not > > know for sure that the panic was caused by the oops. Hence he > > pays twice for the same information, right? > > > > I can think of a couple of way to figure it out in the module > > itself, but I could not think of any clean way to do it. > > This is correct, and the mtdoops driver has some provisions to handle > this. First, there is a parameter to the module to specify whether > oopses should be dumped at all - I added this for the particular case > that someone has panic_on_oops set. > It takes care of most of the situations, but panic_on_oops can be changed any time, even after the module is loaded. While I think that exporting oops_on_panic is a wrong thing to do, I believe that dumpers differ a bit from the rest of the modules in that aspect and should be at least hinted about this flag setting. Does it not make sense? > Second, it does not dump oopses directly anyway, but puts it in a work > queue. That way, if panic_on_oops is set, it will store the panic but > the oops (called from the workqueue) will not get written anyway. > AFAIK, mtdoops does not put oopses in a work queue. And if by any chance it does, then I think it's wrong and might lead to missed oopses, as the oops might be because of the work queues themselves, or it might look to the kernel like some non-fatal fault, but actually it's a sign of a much more catastrophic failure - IOMMU device garbaging memory, for instance. But anyway, I was not talking about mtdoops. In fact, I was not talking about any particular module, I just described some situation which looks a bit problematic to me. > > 2. We tried to use panic notifiers mechanism to print additional > > information that we want to see in mtdoops logs and it worked well, > > but having the kmsg_dump(KMSG_DUMP_PANIC) before the > > atomic_notifier_call_chain() breaks this functionality. > > Can we the call kmsg_dump() after the notifiers had been invoked? > > Well, it depends I think. The code currently looks like this: > > kmsg_dump(KMSG_DUMP_PANIC); > /* > * If we have crashed and we have a crash kernel loaded let it handle > * everything else. > * Do we want to call this before we try to display a message? > */ > crash_kexec(NULL); > [... Comments removed] > atomic_notifier_call_chain(&panic_notifier_list, 0, buf); > > And moving kdump_msg() after crash_kexec() will make us miss the > message if we have a kexec crash kernel as well. I realise that these > two approaches might be complementary and are not likely to be used at > the same time, but it's still something to think about. > > Then again, maybe it's possible to move the panic notifiers above > crash_kexec() as well, which would solve things nicely. > Which leaves me no choice but just ask the question, as it bothering me for some time: does anybody know why we try to crash_kexec() at so early stage? > // Simon
On Mon, 26 Oct 2009 12:36:33 +0200 "Shargorodsky Atal (EXT-Teleca/Helsinki)" <ext-atal.shargorodsky@nokia.com> wrote: > > > I can think of a couple of way to figure it out in the module > > > itself, but I could not think of any clean way to do it. > > > > This is correct, and the mtdoops driver has some provisions to handle > > this. First, there is a parameter to the module to specify whether > > oopses should be dumped at all - I added this for the particular case > > that someone has panic_on_oops set. > > It takes care of most of the situations, but panic_on_oops > can be changed any time, even after the module is loaded. Yes, but this parameter is settable at runtime as well by writing to /sys/module/mtdoops/parameters/dump_oops. > > Second, it does not dump oopses directly anyway, but puts it in a work > > queue. That way, if panic_on_oops is set, it will store the panic but > > the oops (called from the workqueue) will not get written anyway. > > > > AFAIK, mtdoops does not put oopses in a work queue. And if by any chance > it does, then I think it's wrong and might lead to missed oopses, as > the oops might be because of the work queues themselves, or it might > look to the kernel like some non-fatal fault, but actually it's a > sign of a much more catastrophic failure - IOMMU device garbaging > memory, for instance. I was referring to my patches to it, sorry. It's in the patch "[PATCH v7 5/5]: mtdoops: refactor as a kmsg_dumper" (as well as the parameter to dump oopses at all). There are other situations which will make dumping problematic as well, e.g., crashes in the mtd code, so there are certainly some cases which will be difficult to catch. But in the panic_on_oops case or oops-in-interrupt, the oops won't be missed and won't be outputted twice for mtdoops. Anyway, I understand your problem and agree that it would be good to fix. Moving up crash_kexec() and the notifiers will at least fix your second issue. For the double-output-of-oopses, I don't see a good way to fix it unless relying on the module to correct it like above. // Simon
On Mon, 2009-10-26 at 12:53 +0100, ext Simon Kagstrom wrote: > On Mon, 26 Oct 2009 12:36:33 +0200 > "Shargorodsky Atal (EXT-Teleca/Helsinki)" <ext-atal.shargorodsky@nokia.com> wrote: > > > > > I can think of a couple of way to figure it out in the module > > > > itself, but I could not think of any clean way to do it. > > > > > > This is correct, and the mtdoops driver has some provisions to handle > > > this. First, there is a parameter to the module to specify whether > > > oopses should be dumped at all - I added this for the particular case > > > that someone has panic_on_oops set. > > > > It takes care of most of the situations, but panic_on_oops > > can be changed any time, even after the module is loaded. > > Yes, but this parameter is settable at runtime as well by writing > to /sys/module/mtdoops/parameters/dump_oops. > > > > Second, it does not dump oopses directly anyway, but puts it in a work > > > queue. That way, if panic_on_oops is set, it will store the panic but > > > the oops (called from the workqueue) will not get written anyway. > > > > > > > AFAIK, mtdoops does not put oopses in a work queue. And if by any chance > > it does, then I think it's wrong and might lead to missed oopses, as > > the oops might be because of the work queues themselves, or it might > > look to the kernel like some non-fatal fault, but actually it's a > > sign of a much more catastrophic failure - IOMMU device garbaging > > memory, for instance. > > I was referring to my patches to it, sorry. It's in the patch "[PATCH v7 5/5]: > mtdoops: refactor as a kmsg_dumper" (as well as the parameter to dump > oopses at all). > > There are other situations which will make dumping problematic as well, > e.g., crashes in the mtd code, so there are certainly some cases which > will be difficult to catch. But in the panic_on_oops case or > oops-in-interrupt, the oops won't be missed and won't be outputted > twice for mtdoops. > > > Anyway, I understand your problem and agree that it would be good to > fix. Moving up crash_kexec() and the notifiers will at least fix your > second issue. For the double-output-of-oopses, I don't see a good way > to fix it unless relying on the module to correct it like above. > How about adding KMSG_DUMP_LAST_OOPS_BEFORE_PANIC or something to the kmsg_dump_reason enum, and making the kmsg_dump look like kmsg_dump(panic_on_oops ? KMSG_DUMP_LAST_OOPS_BEFORE_PANIC : KMSG_DUMP_OOPS); in oops_exit. Then let the dumpers decide what they want to do about it. Just a thought. And since you have no objections about moving notifiers up, it looks like the second issue will be resolved, I believe Artem will take care of it. :) Thanks a lot, Atal. > // Simon
On Mon, 26 Oct 2009 17:13:27 +0200 "Shargorodsky Atal (EXT-Teleca/Helsinki)" <ext-atal.shargorodsky@nokia.com> wrote: > How about adding KMSG_DUMP_LAST_OOPS_BEFORE_PANIC or something to the > kmsg_dump_reason enum, and making the kmsg_dump look like > kmsg_dump(panic_on_oops ? KMSG_DUMP_LAST_OOPS_BEFORE_PANIC : KMSG_DUMP_OOPS); > in oops_exit. Then let the dumpers decide what they want to do about it. > Just a thought. It would also have to take in_interrupt() in account since that will also lead to a panic. > And since you have no objections about moving notifiers up, it looks > like the second issue will be resolved, I believe Artem > will take care of it. :) Well, I have no objections to it, but I also know very little of the reasoning to put it there in the first place, so don't count my vote for very much here :-) // Simon
diff --git a/include/linux/kmsg_dump.h b/include/linux/kmsg_dump.h new file mode 100644 index 0000000..7f089ec --- /dev/null +++ b/include/linux/kmsg_dump.h @@ -0,0 +1,44 @@ +/* + * linux/include/kmsg_dump.h + * + * Copyright (C) 2009 Net Insight AB + * + * Author: Simon Kagstrom <simon.kagstrom@netinsight.net> + * + * This file is subject to the terms and conditions of the GNU General Public + * License. See the file COPYING in the main directory of this archive + * for more details. + */ +#ifndef _LINUX_KMSG_DUMP_H +#define _LINUX_KMSG_DUMP_H + +#include <linux/list.h> + +enum kmsg_dump_reason { + KMSG_DUMP_OOPS, + KMSG_DUMP_PANIC, +}; + +/** + * struct kmsg_dumper - kernel crash message dumper structure + * @dump: The callback which gets called on crashes. The buffer is passed + * as two sections, where s1 (length l1) contains the older + * messages and s2 (length l2) contains the newer. + * @list: Entry in the dumper list (private) + * @registered: Flag that specifies if this is already registered + */ +struct kmsg_dumper { + void (*dump)(struct kmsg_dumper *dumper, enum kmsg_dump_reason reason, + const char *s1, unsigned long l1, + const char *s2, unsigned long l2); + struct list_head list; + int registered; +}; + +void kmsg_dump(enum kmsg_dump_reason reason); + +int kmsg_dump_register(struct kmsg_dumper *dumper); + +int kmsg_dump_unregister(struct kmsg_dumper *dumper); + +#endif /* _LINUX_KMSG_DUMP_H */ diff --git a/kernel/panic.c b/kernel/panic.c index c0b33b8..763296d 100644 --- a/kernel/panic.c +++ b/kernel/panic.c @@ -10,6 +10,7 @@ */ #include <linux/debug_locks.h> #include <linux/interrupt.h> +#include <linux/kmsg_dump.h> #include <linux/kallsyms.h> #include <linux/notifier.h> #include <linux/module.h> @@ -76,6 +77,7 @@ NORET_TYPE void panic(const char * fmt, ...) dump_stack(); #endif + kmsg_dump(KMSG_DUMP_PANIC); /* * If we have crashed and we have a crash kernel loaded let it handle * everything else. @@ -341,6 +343,7 @@ void oops_exit(void) { do_oops_enter_exit(); print_oops_end_marker(); + kmsg_dump(KMSG_DUMP_OOPS); } #ifdef WANT_WARN_ON_SLOWPATH diff --git a/kernel/printk.c b/kernel/printk.c index f38b07f..f711b99 100644 --- a/kernel/printk.c +++ b/kernel/printk.c @@ -33,6 +33,7 @@ #include <linux/bootmem.h> #include <linux/syscalls.h> #include <linux/kexec.h> +#include <linux/kmsg_dump.h> #include <asm/uaccess.h> @@ -1405,3 +1406,114 @@ bool printk_timed_ratelimit(unsigned long *caller_jiffies, } EXPORT_SYMBOL(printk_timed_ratelimit); #endif + +static DEFINE_SPINLOCK(dump_list_lock); +static LIST_HEAD(dump_list); + +/** + * kmsg_dump_register - register a kernel log dumper. + * @dump: pointer to the kmsg_dumper structure + * + * Adds a kernel log dumper to the system. The dump callback in the + * structure will be called when the kernel oopses or panics and must be + * set. Returns zero on success and %-EINVAL or %-EBUSY otherwise. + */ +int kmsg_dump_register(struct kmsg_dumper *dumper) +{ + unsigned long flags; + int err = -EBUSY; + + /* The dump callback needs to be set */ + if (!dumper->dump) + return -EINVAL; + + spin_lock_irqsave(&dump_list_lock, flags); + /* Don't allow registering multiple times */ + if (!dumper->registered) { + dumper->registered = 1; + list_add_tail(&dumper->list, &dump_list); + err = 0; + } + spin_unlock_irqrestore(&dump_list_lock, flags); + + return err; +} +EXPORT_SYMBOL_GPL(kmsg_dump_register); + +/** + * kmsg_dump_unregister - unregister a kmsg dumper. + * @dump: pointer to the kmsg_dumper structure + * + * Removes a dump device from the system. Returns zero on success and + * %-EINVAL otherwise. + */ +int kmsg_dump_unregister(struct kmsg_dumper *dumper) +{ + unsigned long flags; + int err = -EINVAL; + + spin_lock_irqsave(&dump_list_lock, flags); + if (dumper->registered) { + dumper->registered = 0; + list_del(&dumper->list); + err = 0; + } + spin_unlock_irqrestore(&dump_list_lock, flags); + + return err; +} +EXPORT_SYMBOL_GPL(kmsg_dump_unregister); + +static const char const *kmsg_reasons[] = { + [KMSG_DUMP_OOPS] = "oops", + [KMSG_DUMP_PANIC] = "panic", +}; + +static const char *kmsg_to_str(enum kmsg_dump_reason reason) +{ + if (reason >= ARRAY_SIZE(kmsg_reasons) || reason < 0) + return "unknown"; + + return kmsg_reasons[reason]; +} + +/** + * kmsg_dump - dump kernel log to kernel message dumpers. + * @reason: the reason (oops, panic etc) for dumping + * + * Iterate through each of the dump devices and call the oops/panic + * callbacks with the log buffer. + */ +void kmsg_dump(enum kmsg_dump_reason reason) +{ + unsigned long len = ACCESS_ONCE(log_end); + struct kmsg_dumper *dumper; + const char *s1, *s2; + unsigned long l1, l2; + unsigned long flags; + + s1 = ""; + l1 = 0; + s2 = log_buf; + l2 = len; + + /* Have we rotated around the circular buffer? */ + if (len > log_buf_len) { + unsigned long pos = len & LOG_BUF_MASK; + + s1 = log_buf + pos; + l1 = log_buf_len - pos; + + s2 = log_buf; + l2 = pos; + } + + if (!spin_trylock_irqsave(&dump_list_lock, flags)) { + printk(KERN_ERR "dump_kmsg: dump list lock is held during %s, skipping dump\n", + kmsg_to_str(reason)); + return; + } + list_for_each_entry(dumper, &dump_list, list) + dumper->dump(dumper, reason, s1, l1, s2, l2); + spin_unlock_irqrestore(&dump_list_lock, flags); +}