| Message ID | 20210204165831.2703772-2-daniel.vetter@ffwll.ch |
|---|---|
| State | New |
| Headers | show |
| Series | pci sysfs file iomem revoke support | expand |
[+cc Oliver, Pali, Krzysztof] s/also/Also/ in subject On Thu, Feb 04, 2021 at 05:58:30PM +0100, Daniel Vetter wrote: > We are already doing this for all the regular sysfs files on PCI > devices, but not yet on the legacy io files on the PCI buses. Thus far > now problem, but in the next patch I want to wire up iomem revoke > support. That needs the vfs up an running already to make so that > iomem_get_mapping() works. s/now problem/no problem/ s/an running/and running/ s/so that/sure that/ ? iomem_get_mapping() doesn't exist; I don't know what that should be. > Wire it up exactly like the existing code. Note that > pci_remove_legacy_files() doesn't need a check since the one for > pci_bus->legacy_io is sufficient. I'm not sure exactly what you mean by "the existing code." I could probably figure it out, but it would save time to mention the existing function here. This looks like another instance where we should really apply Oliver's idea of converting these to attribute_groups [1]. The cover letter mentions options discussed with Greg in [2], but I don't think the "sysfs_initialized" hack vs attribute_groups was part of that discussion. It's not absolutely a show-stopper, but it *is* a shame to extend the sysfs_initialized hack if attribute_groups could do this more cleanly and help solve more than one issue. Bjorn [1] https://lore.kernel.org/r/CAOSf1CHss03DBSDO4PmTtMp0tCEu5kScn704ZEwLKGXQzBfqaA@mail.gmail.com [2] https://lore.kernel.org/dri-devel/CAKMK7uGrdDrbtj0OyzqQc0CGrQwc2F3tFJU9vLfm2jjufAZ5YQ@mail.gmail.com/ > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com> > Cc: Stephen Rothwell <sfr@canb.auug.org.au> > Cc: Jason Gunthorpe <jgg@ziepe.ca> > Cc: Kees Cook <keescook@chromium.org> > Cc: Dan Williams <dan.j.williams@intel.com> > Cc: Andrew Morton <akpm@linux-foundation.org> > Cc: John Hubbard <jhubbard@nvidia.com> > Cc: Jérôme Glisse <jglisse@redhat.com> > Cc: Jan Kara <jack@suse.cz> > Cc: Dan Williams <dan.j.williams@intel.com> > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> > Cc: linux-mm@kvack.org > Cc: linux-arm-kernel@lists.infradead.org > Cc: linux-samsung-soc@vger.kernel.org > Cc: linux-media@vger.kernel.org > Cc: Bjorn Helgaas <bhelgaas@google.com> > Cc: linux-pci@vger.kernel.org > --- > drivers/pci/pci-sysfs.c | 7 +++++++ > 1 file changed, 7 insertions(+) > > diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c > index fb072f4b3176..0c45b4f7b214 100644 > --- a/drivers/pci/pci-sysfs.c > +++ b/drivers/pci/pci-sysfs.c > @@ -927,6 +927,9 @@ void pci_create_legacy_files(struct pci_bus *b) > { > int error; > > + if (!sysfs_initialized) > + return; > + > b->legacy_io = kcalloc(2, sizeof(struct bin_attribute), > GFP_ATOMIC); > if (!b->legacy_io) > @@ -1448,6 +1451,7 @@ void pci_remove_sysfs_dev_files(struct pci_dev *pdev) > static int __init pci_sysfs_init(void) > { > struct pci_dev *pdev = NULL; > + struct pci_bus *pbus = NULL; > int retval; > > sysfs_initialized = 1; > @@ -1459,6 +1463,9 @@ static int __init pci_sysfs_init(void) > } > } > > + while ((pbus = pci_find_next_bus(pbus))) > + pci_create_legacy_files(pbus); > + > return 0; > } > late_initcall(pci_sysfs_init); > -- > 2.30.0 > > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Thursday 04 February 2021 15:50:19 Bjorn Helgaas wrote: > [+cc Oliver, Pali, Krzysztof] Just to note that extending or using sysfs_initialized introduces another race condition into kernel code which results in PCI fatal errors. Details are in email discussion which Bjorn already sent. > s/also/Also/ in subject > > On Thu, Feb 04, 2021 at 05:58:30PM +0100, Daniel Vetter wrote: > > We are already doing this for all the regular sysfs files on PCI > > devices, but not yet on the legacy io files on the PCI buses. Thus far > > now problem, but in the next patch I want to wire up iomem revoke > > support. That needs the vfs up an running already to make so that > > iomem_get_mapping() works. > > s/now problem/no problem/ > s/an running/and running/ > s/so that/sure that/ ? > > iomem_get_mapping() doesn't exist; I don't know what that should be. > > > Wire it up exactly like the existing code. Note that > > pci_remove_legacy_files() doesn't need a check since the one for > > pci_bus->legacy_io is sufficient. > > I'm not sure exactly what you mean by "the existing code." I could > probably figure it out, but it would save time to mention the existing > function here. > > This looks like another instance where we should really apply Oliver's > idea of converting these to attribute_groups [1]. > > The cover letter mentions options discussed with Greg in [2], but I > don't think the "sysfs_initialized" hack vs attribute_groups was part > of that discussion. > > It's not absolutely a show-stopper, but it *is* a shame to extend the > sysfs_initialized hack if attribute_groups could do this more cleanly > and help solve more than one issue. > > Bjorn > > [1] https://lore.kernel.org/r/CAOSf1CHss03DBSDO4PmTtMp0tCEu5kScn704ZEwLKGXQzBfqaA@mail.gmail.com > [2] https://lore.kernel.org/dri-devel/CAKMK7uGrdDrbtj0OyzqQc0CGrQwc2F3tFJU9vLfm2jjufAZ5YQ@mail.gmail.com/ > > > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com> > > Cc: Stephen Rothwell <sfr@canb.auug.org.au> > > Cc: Jason Gunthorpe <jgg@ziepe.ca> > > Cc: Kees Cook <keescook@chromium.org> > > Cc: Dan Williams <dan.j.williams@intel.com> > > Cc: Andrew Morton <akpm@linux-foundation.org> > > Cc: John Hubbard <jhubbard@nvidia.com> > > Cc: Jérôme Glisse <jglisse@redhat.com> > > Cc: Jan Kara <jack@suse.cz> > > Cc: Dan Williams <dan.j.williams@intel.com> > > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> > > Cc: linux-mm@kvack.org > > Cc: linux-arm-kernel@lists.infradead.org > > Cc: linux-samsung-soc@vger.kernel.org > > Cc: linux-media@vger.kernel.org > > Cc: Bjorn Helgaas <bhelgaas@google.com> > > Cc: linux-pci@vger.kernel.org > > --- > > drivers/pci/pci-sysfs.c | 7 +++++++ > > 1 file changed, 7 insertions(+) > > > > diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c > > index fb072f4b3176..0c45b4f7b214 100644 > > --- a/drivers/pci/pci-sysfs.c > > +++ b/drivers/pci/pci-sysfs.c > > @@ -927,6 +927,9 @@ void pci_create_legacy_files(struct pci_bus *b) > > { > > int error; > > > > + if (!sysfs_initialized) > > + return; > > + > > b->legacy_io = kcalloc(2, sizeof(struct bin_attribute), > > GFP_ATOMIC); > > if (!b->legacy_io) > > @@ -1448,6 +1451,7 @@ void pci_remove_sysfs_dev_files(struct pci_dev *pdev) > > static int __init pci_sysfs_init(void) > > { > > struct pci_dev *pdev = NULL; > > + struct pci_bus *pbus = NULL; > > int retval; > > > > sysfs_initialized = 1; > > @@ -1459,6 +1463,9 @@ static int __init pci_sysfs_init(void) > > } > > } > > > > + while ((pbus = pci_find_next_bus(pbus))) > > + pci_create_legacy_files(pbus); > > + > > return 0; > > } > > late_initcall(pci_sysfs_init); > > -- > > 2.30.0 > > > > > > _______________________________________________ > > linux-arm-kernel mailing list > > linux-arm-kernel@lists.infradead.org > > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Thu, Feb 4, 2021 at 10:50 PM Bjorn Helgaas <helgaas@kernel.org> wrote: > > [+cc Oliver, Pali, Krzysztof] > > s/also/Also/ in subject > > On Thu, Feb 04, 2021 at 05:58:30PM +0100, Daniel Vetter wrote: > > We are already doing this for all the regular sysfs files on PCI > > devices, but not yet on the legacy io files on the PCI buses. Thus far > > now problem, but in the next patch I want to wire up iomem revoke > > support. That needs the vfs up an running already to make so that > > iomem_get_mapping() works. > > s/now problem/no problem/ > s/an running/and running/ > s/so that/sure that/ ? > > iomem_get_mapping() doesn't exist; I don't know what that should be. Series is based on top of linux-next, where iomem_get_mapping exists. This patch fixes the 2nd patch in this series, which I had to take out of my branch because it failed. > > Wire it up exactly like the existing code. Note that > > pci_remove_legacy_files() doesn't need a check since the one for > > pci_bus->legacy_io is sufficient. > > I'm not sure exactly what you mean by "the existing code." I could > probably figure it out, but it would save time to mention the existing > function here. Sorry, I meant the existing code in pci_create_sysfs_dev_files(). > This looks like another instance where we should really apply Oliver's > idea of converting these to attribute_groups [1]. > > The cover letter mentions options discussed with Greg in [2], but I > don't think the "sysfs_initialized" hack vs attribute_groups was part > of that discussion. Hm not sure the attribute_groups works. The problem is that I cant set up the attributes before the vfs layer is initialized, because before that point the iomem_get_mapping function doesn't return anything useful (well it crashes), because it needs to have an inode available. So if you want to set up the attributes earlier, we'd need some kind of callback, which Greg didn't like. > It's not absolutely a show-stopper, but it *is* a shame to extend the > sysfs_initialized hack if attribute_groups could do this more cleanly > and help solve more than one issue. So I think I have yet another init ordering problem here, but not sure. -Daniel > > Bjorn > > [1] https://lore.kernel.org/r/CAOSf1CHss03DBSDO4PmTtMp0tCEu5kScn704ZEwLKGXQzBfqaA@mail.gmail.com > [2] https://lore.kernel.org/dri-devel/CAKMK7uGrdDrbtj0OyzqQc0CGrQwc2F3tFJU9vLfm2jjufAZ5YQ@mail.gmail.com/ > > > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com> > > Cc: Stephen Rothwell <sfr@canb.auug.org.au> > > Cc: Jason Gunthorpe <jgg@ziepe.ca> > > Cc: Kees Cook <keescook@chromium.org> > > Cc: Dan Williams <dan.j.williams@intel.com> > > Cc: Andrew Morton <akpm@linux-foundation.org> > > Cc: John Hubbard <jhubbard@nvidia.com> > > Cc: Jérôme Glisse <jglisse@redhat.com> > > Cc: Jan Kara <jack@suse.cz> > > Cc: Dan Williams <dan.j.williams@intel.com> > > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> > > Cc: linux-mm@kvack.org > > Cc: linux-arm-kernel@lists.infradead.org > > Cc: linux-samsung-soc@vger.kernel.org > > Cc: linux-media@vger.kernel.org > > Cc: Bjorn Helgaas <bhelgaas@google.com> > > Cc: linux-pci@vger.kernel.org > > --- > > drivers/pci/pci-sysfs.c | 7 +++++++ > > 1 file changed, 7 insertions(+) > > > > diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c > > index fb072f4b3176..0c45b4f7b214 100644 > > --- a/drivers/pci/pci-sysfs.c > > +++ b/drivers/pci/pci-sysfs.c > > @@ -927,6 +927,9 @@ void pci_create_legacy_files(struct pci_bus *b) > > { > > int error; > > > > + if (!sysfs_initialized) > > + return; > > + > > b->legacy_io = kcalloc(2, sizeof(struct bin_attribute), > > GFP_ATOMIC); > > if (!b->legacy_io) > > @@ -1448,6 +1451,7 @@ void pci_remove_sysfs_dev_files(struct pci_dev *pdev) > > static int __init pci_sysfs_init(void) > > { > > struct pci_dev *pdev = NULL; > > + struct pci_bus *pbus = NULL; > > int retval; > > > > sysfs_initialized = 1; > > @@ -1459,6 +1463,9 @@ static int __init pci_sysfs_init(void) > > } > > } > > > > + while ((pbus = pci_find_next_bus(pbus))) > > + pci_create_legacy_files(pbus); > > + > > return 0; > > } > > late_initcall(pci_sysfs_init); > > -- > > 2.30.0 > > > > > > _______________________________________________ > > linux-arm-kernel mailing list > > linux-arm-kernel@lists.infradead.org > > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Thu, Feb 4, 2021 at 11:24 PM Pali Rohár <pali@kernel.org> wrote: > > On Thursday 04 February 2021 15:50:19 Bjorn Helgaas wrote: > > [+cc Oliver, Pali, Krzysztof] > > Just to note that extending or using sysfs_initialized introduces > another race condition into kernel code which results in PCI fatal > errors. Details are in email discussion which Bjorn already sent. Yeah I wondered why this doesn't race, but since the history goes back to pre-git times I figured it would have been addressed somehow already if it indeed does race. -Daniel > > s/also/Also/ in subject > > > > On Thu, Feb 04, 2021 at 05:58:30PM +0100, Daniel Vetter wrote: > > > We are already doing this for all the regular sysfs files on PCI > > > devices, but not yet on the legacy io files on the PCI buses. Thus far > > > now problem, but in the next patch I want to wire up iomem revoke > > > support. That needs the vfs up an running already to make so that > > > iomem_get_mapping() works. > > > > s/now problem/no problem/ > > s/an running/and running/ > > s/so that/sure that/ ? > > > > iomem_get_mapping() doesn't exist; I don't know what that should be. > > > > > Wire it up exactly like the existing code. Note that > > > pci_remove_legacy_files() doesn't need a check since the one for > > > pci_bus->legacy_io is sufficient. > > > > I'm not sure exactly what you mean by "the existing code." I could > > probably figure it out, but it would save time to mention the existing > > function here. > > > > This looks like another instance where we should really apply Oliver's > > idea of converting these to attribute_groups [1]. > > > > The cover letter mentions options discussed with Greg in [2], but I > > don't think the "sysfs_initialized" hack vs attribute_groups was part > > of that discussion. > > > > It's not absolutely a show-stopper, but it *is* a shame to extend the > > sysfs_initialized hack if attribute_groups could do this more cleanly > > and help solve more than one issue. > > > > Bjorn > > > > [1] https://lore.kernel.org/r/CAOSf1CHss03DBSDO4PmTtMp0tCEu5kScn704ZEwLKGXQzBfqaA@mail.gmail.com > > [2] https://lore.kernel.org/dri-devel/CAKMK7uGrdDrbtj0OyzqQc0CGrQwc2F3tFJU9vLfm2jjufAZ5YQ@mail.gmail.com/ > > > > > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com> > > > Cc: Stephen Rothwell <sfr@canb.auug.org.au> > > > Cc: Jason Gunthorpe <jgg@ziepe.ca> > > > Cc: Kees Cook <keescook@chromium.org> > > > Cc: Dan Williams <dan.j.williams@intel.com> > > > Cc: Andrew Morton <akpm@linux-foundation.org> > > > Cc: John Hubbard <jhubbard@nvidia.com> > > > Cc: Jérôme Glisse <jglisse@redhat.com> > > > Cc: Jan Kara <jack@suse.cz> > > > Cc: Dan Williams <dan.j.williams@intel.com> > > > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> > > > Cc: linux-mm@kvack.org > > > Cc: linux-arm-kernel@lists.infradead.org > > > Cc: linux-samsung-soc@vger.kernel.org > > > Cc: linux-media@vger.kernel.org > > > Cc: Bjorn Helgaas <bhelgaas@google.com> > > > Cc: linux-pci@vger.kernel.org > > > --- > > > drivers/pci/pci-sysfs.c | 7 +++++++ > > > 1 file changed, 7 insertions(+) > > > > > > diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c > > > index fb072f4b3176..0c45b4f7b214 100644 > > > --- a/drivers/pci/pci-sysfs.c > > > +++ b/drivers/pci/pci-sysfs.c > > > @@ -927,6 +927,9 @@ void pci_create_legacy_files(struct pci_bus *b) > > > { > > > int error; > > > > > > + if (!sysfs_initialized) > > > + return; > > > + > > > b->legacy_io = kcalloc(2, sizeof(struct bin_attribute), > > > GFP_ATOMIC); > > > if (!b->legacy_io) > > > @@ -1448,6 +1451,7 @@ void pci_remove_sysfs_dev_files(struct pci_dev *pdev) > > > static int __init pci_sysfs_init(void) > > > { > > > struct pci_dev *pdev = NULL; > > > + struct pci_bus *pbus = NULL; > > > int retval; > > > > > > sysfs_initialized = 1; > > > @@ -1459,6 +1463,9 @@ static int __init pci_sysfs_init(void) > > > } > > > } > > > > > > + while ((pbus = pci_find_next_bus(pbus))) > > > + pci_create_legacy_files(pbus); > > > + > > > return 0; > > > } > > > late_initcall(pci_sysfs_init); > > > -- > > > 2.30.0 > > > > > > > > > _______________________________________________ > > > linux-arm-kernel mailing list > > > linux-arm-kernel@lists.infradead.org > > > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Friday 05 February 2021 10:59:50 Daniel Vetter wrote: > On Thu, Feb 4, 2021 at 11:24 PM Pali Rohár <pali@kernel.org> wrote: > > > > On Thursday 04 February 2021 15:50:19 Bjorn Helgaas wrote: > > > [+cc Oliver, Pali, Krzysztof] > > > > Just to note that extending or using sysfs_initialized introduces > > another race condition into kernel code which results in PCI fatal > > errors. Details are in email discussion which Bjorn already sent. > > Yeah I wondered why this doesn't race. It races, but with smaller probability. I have not seen this race condition on x86. But I was able to reproduce it with native PCIe drivers on ARM64 (Marvell Armada 3720; pci-aardvark). In mentioned discussion I wrote when this race condition happen. But I understand that it is hard to simulate it. > but since the history goes back > to pre-git times I figured it would have been addressed somehow > already if it indeed does race. > -Daniel > > > > s/also/Also/ in subject > > > > > > On Thu, Feb 04, 2021 at 05:58:30PM +0100, Daniel Vetter wrote: > > > > We are already doing this for all the regular sysfs files on PCI > > > > devices, but not yet on the legacy io files on the PCI buses. Thus far > > > > now problem, but in the next patch I want to wire up iomem revoke > > > > support. That needs the vfs up an running already to make so that > > > > iomem_get_mapping() works. > > > > > > s/now problem/no problem/ > > > s/an running/and running/ > > > s/so that/sure that/ ? > > > > > > iomem_get_mapping() doesn't exist; I don't know what that should be. > > > > > > > Wire it up exactly like the existing code. Note that > > > > pci_remove_legacy_files() doesn't need a check since the one for > > > > pci_bus->legacy_io is sufficient. > > > > > > I'm not sure exactly what you mean by "the existing code." I could > > > probably figure it out, but it would save time to mention the existing > > > function here. > > > > > > This looks like another instance where we should really apply Oliver's > > > idea of converting these to attribute_groups [1]. > > > > > > The cover letter mentions options discussed with Greg in [2], but I > > > don't think the "sysfs_initialized" hack vs attribute_groups was part > > > of that discussion. > > > > > > It's not absolutely a show-stopper, but it *is* a shame to extend the > > > sysfs_initialized hack if attribute_groups could do this more cleanly > > > and help solve more than one issue. > > > > > > Bjorn > > > > > > [1] https://lore.kernel.org/r/CAOSf1CHss03DBSDO4PmTtMp0tCEu5kScn704ZEwLKGXQzBfqaA@mail.gmail.com > > > [2] https://lore.kernel.org/dri-devel/CAKMK7uGrdDrbtj0OyzqQc0CGrQwc2F3tFJU9vLfm2jjufAZ5YQ@mail.gmail.com/ > > > > > > > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com> > > > > Cc: Stephen Rothwell <sfr@canb.auug.org.au> > > > > Cc: Jason Gunthorpe <jgg@ziepe.ca> > > > > Cc: Kees Cook <keescook@chromium.org> > > > > Cc: Dan Williams <dan.j.williams@intel.com> > > > > Cc: Andrew Morton <akpm@linux-foundation.org> > > > > Cc: John Hubbard <jhubbard@nvidia.com> > > > > Cc: Jérôme Glisse <jglisse@redhat.com> > > > > Cc: Jan Kara <jack@suse.cz> > > > > Cc: Dan Williams <dan.j.williams@intel.com> > > > > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> > > > > Cc: linux-mm@kvack.org > > > > Cc: linux-arm-kernel@lists.infradead.org > > > > Cc: linux-samsung-soc@vger.kernel.org > > > > Cc: linux-media@vger.kernel.org > > > > Cc: Bjorn Helgaas <bhelgaas@google.com> > > > > Cc: linux-pci@vger.kernel.org > > > > --- > > > > drivers/pci/pci-sysfs.c | 7 +++++++ > > > > 1 file changed, 7 insertions(+) > > > > > > > > diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c > > > > index fb072f4b3176..0c45b4f7b214 100644 > > > > --- a/drivers/pci/pci-sysfs.c > > > > +++ b/drivers/pci/pci-sysfs.c > > > > @@ -927,6 +927,9 @@ void pci_create_legacy_files(struct pci_bus *b) > > > > { > > > > int error; > > > > > > > > + if (!sysfs_initialized) > > > > + return; > > > > + > > > > b->legacy_io = kcalloc(2, sizeof(struct bin_attribute), > > > > GFP_ATOMIC); > > > > if (!b->legacy_io) > > > > @@ -1448,6 +1451,7 @@ void pci_remove_sysfs_dev_files(struct pci_dev *pdev) > > > > static int __init pci_sysfs_init(void) > > > > { > > > > struct pci_dev *pdev = NULL; > > > > + struct pci_bus *pbus = NULL; > > > > int retval; > > > > > > > > sysfs_initialized = 1; > > > > @@ -1459,6 +1463,9 @@ static int __init pci_sysfs_init(void) > > > > } > > > > } > > > > > > > > + while ((pbus = pci_find_next_bus(pbus))) > > > > + pci_create_legacy_files(pbus); > > > > + > > > > return 0; > > > > } > > > > late_initcall(pci_sysfs_init); > > > > -- > > > > 2.30.0 > > > > > > > > > > > > _______________________________________________ > > > > linux-arm-kernel mailing list > > > > linux-arm-kernel@lists.infradead.org > > > > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel > > > > -- > Daniel Vetter > Software Engineer, Intel Corporation > http://blog.ffwll.ch
On Fri, Feb 5, 2021 at 11:04 AM Pali Rohár <pali@kernel.org> wrote: > > On Friday 05 February 2021 10:59:50 Daniel Vetter wrote: > > On Thu, Feb 4, 2021 at 11:24 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > On Thursday 04 February 2021 15:50:19 Bjorn Helgaas wrote: > > > > [+cc Oliver, Pali, Krzysztof] > > > > > > Just to note that extending or using sysfs_initialized introduces > > > another race condition into kernel code which results in PCI fatal > > > errors. Details are in email discussion which Bjorn already sent. > > > > Yeah I wondered why this doesn't race. > > It races, but with smaller probability. I have not seen this race > condition on x86. But I was able to reproduce it with native PCIe > drivers on ARM64 (Marvell Armada 3720; pci-aardvark). In mentioned > discussion I wrote when this race condition happen. But I understand > that it is hard to simulate it. btw I looked at your patch, and isn't that just reducing the race window? I think we have a very similar problem in drm, where the drm_dev_register() for the overall device (which also registers all drm_connector) can race with the hotplug of an individual connector in drm_connector_register() which is hotplugged at runtime. I went with a per-connector registered boolean + a lock to make sure that really only one of the two call paths can end up registering the connector. Part of registering connectors is setting up sysfs files, so I think it's exactly the same problem as here. Cheers, Daniel > > > but since the history goes back > > to pre-git times I figured it would have been addressed somehow > > already if it indeed does race. > > -Daniel > > > > > > s/also/Also/ in subject > > > > > > > > On Thu, Feb 04, 2021 at 05:58:30PM +0100, Daniel Vetter wrote: > > > > > We are already doing this for all the regular sysfs files on PCI > > > > > devices, but not yet on the legacy io files on the PCI buses. Thus far > > > > > now problem, but in the next patch I want to wire up iomem revoke > > > > > support. That needs the vfs up an running already to make so that > > > > > iomem_get_mapping() works. > > > > > > > > s/now problem/no problem/ > > > > s/an running/and running/ > > > > s/so that/sure that/ ? > > > > > > > > iomem_get_mapping() doesn't exist; I don't know what that should be. > > > > > > > > > Wire it up exactly like the existing code. Note that > > > > > pci_remove_legacy_files() doesn't need a check since the one for > > > > > pci_bus->legacy_io is sufficient. > > > > > > > > I'm not sure exactly what you mean by "the existing code." I could > > > > probably figure it out, but it would save time to mention the existing > > > > function here. > > > > > > > > This looks like another instance where we should really apply Oliver's > > > > idea of converting these to attribute_groups [1]. > > > > > > > > The cover letter mentions options discussed with Greg in [2], but I > > > > don't think the "sysfs_initialized" hack vs attribute_groups was part > > > > of that discussion. > > > > > > > > It's not absolutely a show-stopper, but it *is* a shame to extend the > > > > sysfs_initialized hack if attribute_groups could do this more cleanly > > > > and help solve more than one issue. > > > > > > > > Bjorn > > > > > > > > [1] https://lore.kernel.org/r/CAOSf1CHss03DBSDO4PmTtMp0tCEu5kScn704ZEwLKGXQzBfqaA@mail.gmail.com > > > > [2] https://lore.kernel.org/dri-devel/CAKMK7uGrdDrbtj0OyzqQc0CGrQwc2F3tFJU9vLfm2jjufAZ5YQ@mail.gmail.com/ > > > > > > > > > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com> > > > > > Cc: Stephen Rothwell <sfr@canb.auug.org.au> > > > > > Cc: Jason Gunthorpe <jgg@ziepe.ca> > > > > > Cc: Kees Cook <keescook@chromium.org> > > > > > Cc: Dan Williams <dan.j.williams@intel.com> > > > > > Cc: Andrew Morton <akpm@linux-foundation.org> > > > > > Cc: John Hubbard <jhubbard@nvidia.com> > > > > > Cc: Jérôme Glisse <jglisse@redhat.com> > > > > > Cc: Jan Kara <jack@suse.cz> > > > > > Cc: Dan Williams <dan.j.williams@intel.com> > > > > > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> > > > > > Cc: linux-mm@kvack.org > > > > > Cc: linux-arm-kernel@lists.infradead.org > > > > > Cc: linux-samsung-soc@vger.kernel.org > > > > > Cc: linux-media@vger.kernel.org > > > > > Cc: Bjorn Helgaas <bhelgaas@google.com> > > > > > Cc: linux-pci@vger.kernel.org > > > > > --- > > > > > drivers/pci/pci-sysfs.c | 7 +++++++ > > > > > 1 file changed, 7 insertions(+) > > > > > > > > > > diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c > > > > > index fb072f4b3176..0c45b4f7b214 100644 > > > > > --- a/drivers/pci/pci-sysfs.c > > > > > +++ b/drivers/pci/pci-sysfs.c > > > > > @@ -927,6 +927,9 @@ void pci_create_legacy_files(struct pci_bus *b) > > > > > { > > > > > int error; > > > > > > > > > > + if (!sysfs_initialized) > > > > > + return; > > > > > + > > > > > b->legacy_io = kcalloc(2, sizeof(struct bin_attribute), > > > > > GFP_ATOMIC); > > > > > if (!b->legacy_io) > > > > > @@ -1448,6 +1451,7 @@ void pci_remove_sysfs_dev_files(struct pci_dev *pdev) > > > > > static int __init pci_sysfs_init(void) > > > > > { > > > > > struct pci_dev *pdev = NULL; > > > > > + struct pci_bus *pbus = NULL; > > > > > int retval; > > > > > > > > > > sysfs_initialized = 1; > > > > > @@ -1459,6 +1463,9 @@ static int __init pci_sysfs_init(void) > > > > > } > > > > > } > > > > > > > > > > + while ((pbus = pci_find_next_bus(pbus))) > > > > > + pci_create_legacy_files(pbus); > > > > > + > > > > > return 0; > > > > > } > > > > > late_initcall(pci_sysfs_init); > > > > > -- > > > > > 2.30.0 > > > > > > > > > > > > > > > _______________________________________________ > > > > > linux-arm-kernel mailing list > > > > > linux-arm-kernel@lists.infradead.org > > > > > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel > > > > > > > > -- > > Daniel Vetter > > Software Engineer, Intel Corporation > > http://blog.ffwll.ch
On Friday 05 February 2021 11:16:00 Daniel Vetter wrote: > On Fri, Feb 5, 2021 at 11:04 AM Pali Rohár <pali@kernel.org> wrote: > > > > On Friday 05 February 2021 10:59:50 Daniel Vetter wrote: > > > On Thu, Feb 4, 2021 at 11:24 PM Pali Rohár <pali@kernel.org> wrote: > > > > > > > > On Thursday 04 February 2021 15:50:19 Bjorn Helgaas wrote: > > > > > [+cc Oliver, Pali, Krzysztof] > > > > > > > > Just to note that extending or using sysfs_initialized introduces > > > > another race condition into kernel code which results in PCI fatal > > > > errors. Details are in email discussion which Bjorn already sent. > > > > > > Yeah I wondered why this doesn't race. > > > > It races, but with smaller probability. I have not seen this race > > condition on x86. But I was able to reproduce it with native PCIe > > drivers on ARM64 (Marvell Armada 3720; pci-aardvark). In mentioned > > discussion I wrote when this race condition happen. But I understand > > that it is hard to simulate it. > > btw I looked at your patch, and isn't that just reducing the race window? I probably have not wrote reply to that thread and only to Krzysztof on IRC, but my "hack" really does not solve that race condition. And as you wrote it only reduced occurrence on tested HW. Krzysztof wrote that would look at this issue and try to solve it properly. So I have not doing more investigation on that my "hack" patch, race conditions are hard to catch and solve... > I think we have a very similar problem in drm, where the > drm_dev_register() for the overall device (which also registers all > drm_connector) can race with the hotplug of an individual connector in > drm_connector_register() which is hotplugged at runtime. > > I went with a per-connector registered boolean + a lock to make sure > that really only one of the two call paths can end up registering the > connector. Part of registering connectors is setting up sysfs files, > so I think it's exactly the same problem as here. > > Cheers, Daniel > > > > > > but since the history goes back > > > to pre-git times I figured it would have been addressed somehow > > > already if it indeed does race. > > > -Daniel > > > > > > > > s/also/Also/ in subject > > > > > > > > > > On Thu, Feb 04, 2021 at 05:58:30PM +0100, Daniel Vetter wrote: > > > > > > We are already doing this for all the regular sysfs files on PCI > > > > > > devices, but not yet on the legacy io files on the PCI buses. Thus far > > > > > > now problem, but in the next patch I want to wire up iomem revoke > > > > > > support. That needs the vfs up an running already to make so that > > > > > > iomem_get_mapping() works. > > > > > > > > > > s/now problem/no problem/ > > > > > s/an running/and running/ > > > > > s/so that/sure that/ ? > > > > > > > > > > iomem_get_mapping() doesn't exist; I don't know what that should be. > > > > > > > > > > > Wire it up exactly like the existing code. Note that > > > > > > pci_remove_legacy_files() doesn't need a check since the one for > > > > > > pci_bus->legacy_io is sufficient. > > > > > > > > > > I'm not sure exactly what you mean by "the existing code." I could > > > > > probably figure it out, but it would save time to mention the existing > > > > > function here. > > > > > > > > > > This looks like another instance where we should really apply Oliver's > > > > > idea of converting these to attribute_groups [1]. > > > > > > > > > > The cover letter mentions options discussed with Greg in [2], but I > > > > > don't think the "sysfs_initialized" hack vs attribute_groups was part > > > > > of that discussion. > > > > > > > > > > It's not absolutely a show-stopper, but it *is* a shame to extend the > > > > > sysfs_initialized hack if attribute_groups could do this more cleanly > > > > > and help solve more than one issue. > > > > > > > > > > Bjorn > > > > > > > > > > [1] https://lore.kernel.org/r/CAOSf1CHss03DBSDO4PmTtMp0tCEu5kScn704ZEwLKGXQzBfqaA@mail.gmail.com > > > > > [2] https://lore.kernel.org/dri-devel/CAKMK7uGrdDrbtj0OyzqQc0CGrQwc2F3tFJU9vLfm2jjufAZ5YQ@mail.gmail.com/ > > > > > > > > > > > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com> > > > > > > Cc: Stephen Rothwell <sfr@canb.auug.org.au> > > > > > > Cc: Jason Gunthorpe <jgg@ziepe.ca> > > > > > > Cc: Kees Cook <keescook@chromium.org> > > > > > > Cc: Dan Williams <dan.j.williams@intel.com> > > > > > > Cc: Andrew Morton <akpm@linux-foundation.org> > > > > > > Cc: John Hubbard <jhubbard@nvidia.com> > > > > > > Cc: Jérôme Glisse <jglisse@redhat.com> > > > > > > Cc: Jan Kara <jack@suse.cz> > > > > > > Cc: Dan Williams <dan.j.williams@intel.com> > > > > > > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> > > > > > > Cc: linux-mm@kvack.org > > > > > > Cc: linux-arm-kernel@lists.infradead.org > > > > > > Cc: linux-samsung-soc@vger.kernel.org > > > > > > Cc: linux-media@vger.kernel.org > > > > > > Cc: Bjorn Helgaas <bhelgaas@google.com> > > > > > > Cc: linux-pci@vger.kernel.org > > > > > > --- > > > > > > drivers/pci/pci-sysfs.c | 7 +++++++ > > > > > > 1 file changed, 7 insertions(+) > > > > > > > > > > > > diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c > > > > > > index fb072f4b3176..0c45b4f7b214 100644 > > > > > > --- a/drivers/pci/pci-sysfs.c > > > > > > +++ b/drivers/pci/pci-sysfs.c > > > > > > @@ -927,6 +927,9 @@ void pci_create_legacy_files(struct pci_bus *b) > > > > > > { > > > > > > int error; > > > > > > > > > > > > + if (!sysfs_initialized) > > > > > > + return; > > > > > > + > > > > > > b->legacy_io = kcalloc(2, sizeof(struct bin_attribute), > > > > > > GFP_ATOMIC); > > > > > > if (!b->legacy_io) > > > > > > @@ -1448,6 +1451,7 @@ void pci_remove_sysfs_dev_files(struct pci_dev *pdev) > > > > > > static int __init pci_sysfs_init(void) > > > > > > { > > > > > > struct pci_dev *pdev = NULL; > > > > > > + struct pci_bus *pbus = NULL; > > > > > > int retval; > > > > > > > > > > > > sysfs_initialized = 1; > > > > > > @@ -1459,6 +1463,9 @@ static int __init pci_sysfs_init(void) > > > > > > } > > > > > > } > > > > > > > > > > > > + while ((pbus = pci_find_next_bus(pbus))) > > > > > > + pci_create_legacy_files(pbus); > > > > > > + > > > > > > return 0; > > > > > > } > > > > > > late_initcall(pci_sysfs_init); > > > > > > -- > > > > > > 2.30.0 > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > linux-arm-kernel mailing list > > > > > > linux-arm-kernel@lists.infradead.org > > > > > > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel > > > > > > > > > > > > -- > > > Daniel Vetter > > > Software Engineer, Intel Corporation > > > http://blog.ffwll.ch > > > > -- > Daniel Vetter > Software Engineer, Intel Corporation > http://blog.ffwll.ch
diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c index fb072f4b3176..0c45b4f7b214 100644 --- a/drivers/pci/pci-sysfs.c +++ b/drivers/pci/pci-sysfs.c @@ -927,6 +927,9 @@ void pci_create_legacy_files(struct pci_bus *b) { int error; + if (!sysfs_initialized) + return; + b->legacy_io = kcalloc(2, sizeof(struct bin_attribute), GFP_ATOMIC); if (!b->legacy_io) @@ -1448,6 +1451,7 @@ void pci_remove_sysfs_dev_files(struct pci_dev *pdev) static int __init pci_sysfs_init(void) { struct pci_dev *pdev = NULL; + struct pci_bus *pbus = NULL; int retval; sysfs_initialized = 1; @@ -1459,6 +1463,9 @@ static int __init pci_sysfs_init(void) } } + while ((pbus = pci_find_next_bus(pbus))) + pci_create_legacy_files(pbus); + return 0; } late_initcall(pci_sysfs_init);
We are already doing this for all the regular sysfs files on PCI devices, but not yet on the legacy io files on the PCI buses. Thus far now problem, but in the next patch I want to wire up iomem revoke support. That needs the vfs up an running already to make so that iomem_get_mapping() works. Wire it up exactly like the existing code. Note that pci_remove_legacy_files() doesn't need a check since the one for pci_bus->legacy_io is sufficient. Signed-off-by: Daniel Vetter <daniel.vetter@intel.com> Cc: Stephen Rothwell <sfr@canb.auug.org.au> Cc: Jason Gunthorpe <jgg@ziepe.ca> Cc: Kees Cook <keescook@chromium.org> Cc: Dan Williams <dan.j.williams@intel.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: John Hubbard <jhubbard@nvidia.com> Cc: Jérôme Glisse <jglisse@redhat.com> Cc: Jan Kara <jack@suse.cz> Cc: Dan Williams <dan.j.williams@intel.com> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Cc: Bjorn Helgaas <bhelgaas@google.com> Cc: linux-pci@vger.kernel.org --- drivers/pci/pci-sysfs.c | 7 +++++++ 1 file changed, 7 insertions(+)