| Message ID | 1380189591-1368-1-git-send-email-vfalico@redhat.com |
|---|---|
| State | Changes Requested |
| Headers | show |
On Thu, Sep 26, 2013 at 11:59:51AM +0200, Veaceslav Falico wrote: > Currently, we first do kobject_put(&entry->kobj) and the kfree(entry), > however kobject_put() doesn't guarantee us that it was the last reference > and that the kobj isn't used currently by someone else, so after we > kfree(entry) with the struct kobject - other users will begin using the > freed memory, instead of the actual kobject. > > Fix this by using the kobject->release callback, which is called last when > the kobject is indeed not used and is cleaned up - it's msi_kobj_release(), > which can do the kfree(entry) safely (kobject_put/cleanup doesn't use the > kobj itself after ->release() was called, so we're safe). > > In case we've failed to create the sysfs directories - just kfree() > it - cause we don't have the kobjects attached. > > Also, remove the same functionality from populate_msi_sysfs(), cause on > failure we anyway call free_msi_irqs(), which will take care of all the > kobjects properly. > > And add the forgotten pci_dev_put(pdev) in case of failure to register the > kobject in populate_msi_sysfs(). > > CC: Bjorn Helgaas <bhelgaas@google.com> > CC: Neil Horman <nhorman@tuxdriver.com> > CC: Greg Kroah-Hartman <gregkh@linuxfoundation.org> > CC: linux-pci@vger.kernel.org > CC: linux-kernel@vger.kernel.org > Signed-off-by: Veaceslav Falico <vfalico@redhat.com> > --- > Acked-by: Neil Horman <nhorman@tuxdriver.com> -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Sep 26, 2013 at 11:59:51AM +0200, Veaceslav Falico wrote: >Currently, we first do kobject_put(&entry->kobj) and the kfree(entry), >however kobject_put() doesn't guarantee us that it was the last reference >and that the kobj isn't used currently by someone else, so after we >kfree(entry) with the struct kobject - other users will begin using the >freed memory, instead of the actual kobject. Hi Bjorn, I've seen that you've dropped this bugfix (and the 3 cleanup patches) with "Changes Requested", however I don't recall any request to change this. I'm really sorry for bugging - but I need this fix to get included for my testing to work :(. This fix fixes the critical bug when we free the entry with the kobject .../msi_irqs/IRQ_NR still being used, however the issue with msi_irqs kset itself still exists, and really is a different issue - and I'm trying to find a fix for it. Thank you, and sorry for the mess and the noise. > >Fix this by using the kobject->release callback, which is called last when >the kobject is indeed not used and is cleaned up - it's msi_kobj_release(), >which can do the kfree(entry) safely (kobject_put/cleanup doesn't use the >kobj itself after ->release() was called, so we're safe). > >In case we've failed to create the sysfs directories - just kfree() >it - cause we don't have the kobjects attached. > >Also, remove the same functionality from populate_msi_sysfs(), cause on >failure we anyway call free_msi_irqs(), which will take care of all the >kobjects properly. > >And add the forgotten pci_dev_put(pdev) in case of failure to register the >kobject in populate_msi_sysfs(). > >CC: Bjorn Helgaas <bhelgaas@google.com> >CC: Neil Horman <nhorman@tuxdriver.com> >CC: Greg Kroah-Hartman <gregkh@linuxfoundation.org> >CC: linux-pci@vger.kernel.org >CC: linux-kernel@vger.kernel.org >Signed-off-by: Veaceslav Falico <vfalico@redhat.com> >--- > >Notes: > v1 -> v2: > Make it as a standalone patch, which is a bugfix, and add the forgotten > pci_dev_put() so that it won't break bisecting. The pci_dev_put() will > go away anyway in the following patchset, which cleans removes > kobject_del and useless pci_dev_get/put(). Rebased on linux-pci/next. > > drivers/pci/msi.c | 31 ++++++++++++------------------- > 1 file changed, 12 insertions(+), 19 deletions(-) > >diff --git a/drivers/pci/msi.c b/drivers/pci/msi.c >index d5f90d6..5d70f49 100644 >--- a/drivers/pci/msi.c >+++ b/drivers/pci/msi.c >@@ -374,19 +374,22 @@ static void free_msi_irqs(struct pci_dev *dev) > iounmap(entry->mask_base); > } > >+ list_del(&entry->list); >+ > /* > * Its possible that we get into this path > * When populate_msi_sysfs fails, which means the entries > * were not registered with sysfs. In that case don't >- * unregister them. >+ * unregister them, and just free. Otherwise the >+ * kobject->release will take care of freeing the entry via >+ * msi_kobj_release(). > */ > if (entry->kobj.parent) { > kobject_del(&entry->kobj); > kobject_put(&entry->kobj); >+ } else { >+ kfree(entry); > } >- >- list_del(&entry->list); >- kfree(entry); > } > } > >@@ -509,6 +512,7 @@ static void msi_kobj_release(struct kobject *kobj) > struct msi_desc *entry = to_msi_desc(kobj); > > pci_dev_put(entry->dev); >+ kfree(entry); > } > > static struct kobj_type msi_irq_ktype = { >@@ -522,7 +526,6 @@ static int populate_msi_sysfs(struct pci_dev *pdev) > struct msi_desc *entry; > struct kobject *kobj; > int ret; >- int count = 0; > > pdev->msi_kset = kset_create_and_add("msi_irqs", NULL, &pdev->dev.kobj); > if (!pdev->msi_kset) >@@ -534,23 +537,13 @@ static int populate_msi_sysfs(struct pci_dev *pdev) > pci_dev_get(pdev); > ret = kobject_init_and_add(kobj, &msi_irq_ktype, NULL, > "%u", entry->irq); >- if (ret) >- goto out_unroll; >- >- count++; >+ if (ret) { >+ pci_dev_put(pdev); >+ return ret; >+ } > } > > return 0; >- >-out_unroll: >- list_for_each_entry(entry, &pdev->msi_list, list) { >- if (!count) >- break; >- kobject_del(&entry->kobj); >- kobject_put(&entry->kobj); >- count--; >- } >- return ret; > } > > /** >-- >1.8.4 > -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Sat, Sep 28, 2013 at 3:37 PM, Veaceslav Falico <vfalico@redhat.com> wrote: > On Thu, Sep 26, 2013 at 11:59:51AM +0200, Veaceslav Falico wrote: >> >> Currently, we first do kobject_put(&entry->kobj) and the kfree(entry), >> however kobject_put() doesn't guarantee us that it was the last reference >> and that the kobj isn't used currently by someone else, so after we >> kfree(entry) with the struct kobject - other users will begin using the >> freed memory, instead of the actual kobject. > > > Hi Bjorn, > > I've seen that you've dropped this bugfix (and the 3 cleanup patches) with > "Changes Requested", however I don't recall any request to change this. I talked to Greg KH about this recently, and he said he might take a look at doing a more extensive cleanup of populate_msi_sysfs() using attribute groups, so I don't know if you want to wait and see whether he does anything, or go ahead on the path you were on. If you continue, my advice is: - Put all these patches in a single series with a version number (I think the next posting would be v3) to help me keep track of them. - In populate_msi_sysfs(), drop the pci_dev_get() (or explain why it's needed). My reasoning is that the "msi_irqs" kset should already hold a reference on the pdev (acquired in kset_create_and_add() -> kset_register() -> kobject_add_internal()), and each irq entry should hold a reference on the kset (see kobject_add_internal() again), so it is redundant to acquire a reference on the pdev directly. This means dropping the pci_dev_put() in msi_kobj_release(), of course. - Move the kfree(entry) from free_msi_irqs() to msi_kobj_release() (I think one of your patches already did this). - In populate_msi_sysfs(), drop the kobject_del() in the out_unroll loop. I think we would only need that if there were a way to create a new irq entry in "msi_irqs" before the old irq entry was released. But I don't think that's possible. We only create irq entries in populate_msi_sysfs(), which always starts with a fresh, empty "msi_irqs" kset. - In free_msi_irqs(), similarly remove the kobject_del(). - Add a kobject_del() before each kset_unregister(dev->msi_kset) call. This will remove "msi_irqs" from sysfs, so future creates will succeed even if somebody still has the old "msi_irqs" open. - Keep the msi_kset cleanup in populate_msi_sysfs() instead of relying on free_msi_irqs(). I think it's less error prone to keep the creation and error path cleanup in the same function. Bjorn -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, Oct 04, 2013 at 10:46:31AM -0600, Bjorn Helgaas wrote: >On Sat, Sep 28, 2013 at 3:37 PM, Veaceslav Falico <vfalico@redhat.com> wrote: >> On Thu, Sep 26, 2013 at 11:59:51AM +0200, Veaceslav Falico wrote: >>> >>> Currently, we first do kobject_put(&entry->kobj) and the kfree(entry), >>> however kobject_put() doesn't guarantee us that it was the last reference >>> and that the kobj isn't used currently by someone else, so after we >>> kfree(entry) with the struct kobject - other users will begin using the >>> freed memory, instead of the actual kobject. >> >> >> Hi Bjorn, >> >> I've seen that you've dropped this bugfix (and the 3 cleanup patches) with >> "Changes Requested", however I don't recall any request to change this. > >I talked to Greg KH about this recently, and he said he might take a >look at doing a more extensive cleanup of populate_msi_sysfs() using >attribute groups, so I don't know if you want to wait and see whether >he does anything, or go ahead on the path you were on. Sorry for the delay, was sick. I'll continue going ahead, however if Greg/you don't really need it or are working on it - please say now, so I'll stop waisting your time. > >If you continue, my advice is: > > - Put all these patches in a single series with a version number (I >think the next posting would be v3) to help me keep track of them. Will do, if/when there'll be next version. Now they're divided into 1 bugfix and 1 cleanup patchset. > > - In populate_msi_sysfs(), drop the pci_dev_get() (or explain why >it's needed). My reasoning is that the "msi_irqs" kset should already >hold a reference on the pdev (acquired in kset_create_and_add() -> >kset_register() -> kobject_add_internal()), and each irq entry should >hold a reference on the kset (see kobject_add_internal() again), so >it is redundant to acquire a reference on the pdev directly. This >means dropping the pci_dev_put() in msi_kobj_release(), of course. It's done in my patch pci: remove redundant pci_dev_get/put() on kobject (un)register http://patchwork.ozlabs.org/patch/278201/ > >- Move the kfree(entry) from free_msi_irqs() to msi_kobj_release() (I >think one of your patches already did this). It's done in my patch msi: free msi_desc entry only after we've released the kobject http://patchwork.ozlabs.org/patch/278150/ > > - In populate_msi_sysfs(), drop the kobject_del() in the out_unroll >loop. I think we would only need that if there were a way to create a >new irq entry in "msi_irqs" before the old irq entry was released. >But I don't think that's possible. We only create irq entries in >populate_msi_sysfs(), which always starts with a fresh, empty >"msi_irqs" kset. It's done in my patch msi: free msi_desc entry only after we've released the kobject http://patchwork.ozlabs.org/patch/278150/ > > - In free_msi_irqs(), similarly remove the kobject_del(). It's done in my patch msi: remove useless kobject_del() in free_msi_irqs() http://patchwork.ozlabs.org/patch/278202/ > > - Add a kobject_del() before each kset_unregister(dev->msi_kset) >call. This will remove "msi_irqs" from sysfs, so future creates will >succeed even if somebody still has the old "msi_irqs" open. I think it's done in your patch kobject: remove kset from sysfs immediately in kset_unregister() http://patchwork.ozlabs.org/patch/281618/ So it'll collide and use kobject_del() twice. Or did you actually drop your patch? > > - Keep the msi_kset cleanup in populate_msi_sysfs() instead of >relying on free_msi_irqs(). I think it's less error prone to keep the >creation and error path cleanup in the same function. It is less error prone, however the current design is that "once we fail something while creating irqs, always call free_msi_irqs()", so, if we add the msi_kset cleanup to populate_msi_sysfs() (it wasn't there before, so we can't 'keep' it) - we'll have to verify if it was don in free_msi_irqs(), cause free_msi_irqs() is being called not only on rollback in msi_capability_init(), but also in pci_disable_msi() and friends. > >Bjorn -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/drivers/pci/msi.c b/drivers/pci/msi.c index d5f90d6..5d70f49 100644 --- a/drivers/pci/msi.c +++ b/drivers/pci/msi.c @@ -374,19 +374,22 @@ static void free_msi_irqs(struct pci_dev *dev) iounmap(entry->mask_base); } + list_del(&entry->list); + /* * Its possible that we get into this path * When populate_msi_sysfs fails, which means the entries * were not registered with sysfs. In that case don't - * unregister them. + * unregister them, and just free. Otherwise the + * kobject->release will take care of freeing the entry via + * msi_kobj_release(). */ if (entry->kobj.parent) { kobject_del(&entry->kobj); kobject_put(&entry->kobj); + } else { + kfree(entry); } - - list_del(&entry->list); - kfree(entry); } } @@ -509,6 +512,7 @@ static void msi_kobj_release(struct kobject *kobj) struct msi_desc *entry = to_msi_desc(kobj); pci_dev_put(entry->dev); + kfree(entry); } static struct kobj_type msi_irq_ktype = { @@ -522,7 +526,6 @@ static int populate_msi_sysfs(struct pci_dev *pdev) struct msi_desc *entry; struct kobject *kobj; int ret; - int count = 0; pdev->msi_kset = kset_create_and_add("msi_irqs", NULL, &pdev->dev.kobj); if (!pdev->msi_kset) @@ -534,23 +537,13 @@ static int populate_msi_sysfs(struct pci_dev *pdev) pci_dev_get(pdev); ret = kobject_init_and_add(kobj, &msi_irq_ktype, NULL, "%u", entry->irq); - if (ret) - goto out_unroll; - - count++; + if (ret) { + pci_dev_put(pdev); + return ret; + } } return 0; - -out_unroll: - list_for_each_entry(entry, &pdev->msi_list, list) { - if (!count) - break; - kobject_del(&entry->kobj); - kobject_put(&entry->kobj); - count--; - } - return ret; } /**
Currently, we first do kobject_put(&entry->kobj) and the kfree(entry), however kobject_put() doesn't guarantee us that it was the last reference and that the kobj isn't used currently by someone else, so after we kfree(entry) with the struct kobject - other users will begin using the freed memory, instead of the actual kobject. Fix this by using the kobject->release callback, which is called last when the kobject is indeed not used and is cleaned up - it's msi_kobj_release(), which can do the kfree(entry) safely (kobject_put/cleanup doesn't use the kobj itself after ->release() was called, so we're safe). In case we've failed to create the sysfs directories - just kfree() it - cause we don't have the kobjects attached. Also, remove the same functionality from populate_msi_sysfs(), cause on failure we anyway call free_msi_irqs(), which will take care of all the kobjects properly. And add the forgotten pci_dev_put(pdev) in case of failure to register the kobject in populate_msi_sysfs(). CC: Bjorn Helgaas <bhelgaas@google.com> CC: Neil Horman <nhorman@tuxdriver.com> CC: Greg Kroah-Hartman <gregkh@linuxfoundation.org> CC: linux-pci@vger.kernel.org CC: linux-kernel@vger.kernel.org Signed-off-by: Veaceslav Falico <vfalico@redhat.com> --- Notes: v1 -> v2: Make it as a standalone patch, which is a bugfix, and add the forgotten pci_dev_put() so that it won't break bisecting. The pci_dev_put() will go away anyway in the following patchset, which cleans removes kobject_del and useless pci_dev_get/put(). Rebased on linux-pci/next. drivers/pci/msi.c | 31 ++++++++++++------------------- 1 file changed, 12 insertions(+), 19 deletions(-)