Message ID | 20180511162028.20616-1-brgl@bgdev.pl |
---|---|
Headers | show |
Series | introduce support for early platform drivers | expand |
On Fri, May 11, 2018 at 11:20 AM, Bartosz Golaszewski <brgl@bgdev.pl> wrote: > This series is a follow-up to the RFC[1] posted a couple days ago. > > NOTE: this series applies on top of my recent patches[2] that move the previous > implementation of early platform devices to arch/sh. > > Problem: > > Certain class of devices, such as timers, certain clock drivers and irq chip > drivers need to be probed early in the boot sequence. The currently preferred > approach is using one of the OF_DECLARE() macros. This however does not create > a platform device which has many drawbacks - such as not being able to use > devres routines, dev_ log functions or no way of deferring the init OF function > if some other resources are missing. I skimmed though this and it doesn't look horrible (how's that for positive feedback? ;) ). But before going into the details, I think first there needs to be agreement this is the right direction. The question does remain though as to whether this class of devices should be platform drivers. They can't be modules. They can't be hotplugged. Can they be runtime-pm enabled? So the advantage is ... I assume that the clock maintainers had some reason to move clocks to be platform drivers. It's just not clear to me what that was. > For drivers that use both platform drivers and OF_DECLARE the situation is even > more complicated as the code needs to take into account that there can possibly > be no struct device present. For a specific use case that we're having problems > with, please refer to the recent DaVinci common-clock conversion patches and > the nasty workaround that this problem implies[3]. So devm_kzalloc will work with this solution? Why did we need devm_kzalloc in the first place? The clocks can never be removed and cleaning up on error paths is kind of pointless. The system would be hosed, right? > We also used to have an early platform drivers implementation but they were not > integrated with the linux device model at all - they merely used the same data > structures. The users could not use devres, defer probe and the early devices > never became actual platform devices later on. > > Proposed solution: > > This series aims at solving this problem by (re-)introducing the concept of > early platform drivers and devices - this time however in a way that seamlessly > integrates with the existing platform drivers and also offers device-tree > support. > > The idea is to provide a way for users to probe devices early, while already > being able to use devres, devices resources and properties and also deferred > probing. > > New structures are introduced: the early platform driver contains the > early_probe callback which has the same signature as regular platform_device > probe. This callback is called early on. The user can have both the early and > regular probe speficied or only one of them and they both receive the same > platform device object as argument. Any device data allocated early will be > carried over to the normal probe. > > The architecture code is responsible for calling early_platform_start() in > which the early drivers will be registered and devices populated from DT. Can we really do this in one spot for different devices (clk, timers, irq). The sequence is all very carefully crafted. Platform specific hooks is another thing to consider. > Once the device and kobject mechanisms are ready, all early drivers and devices > will be converted into real platform drivers and devices. Also: if any of the > early platform registration functions will be called once early initialization > is done, these functions will work like regular platform_device/driver ones. This could leave devices in a weird state. They've successfully probed early, but then are on the deferred list for normal probe for example. Rob -- To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
2018-05-11 22:13 GMT+02:00 Rob Herring <robh+dt@kernel.org>: > On Fri, May 11, 2018 at 11:20 AM, Bartosz Golaszewski <brgl@bgdev.pl> wrote: >> This series is a follow-up to the RFC[1] posted a couple days ago. >> >> NOTE: this series applies on top of my recent patches[2] that move the previous >> implementation of early platform devices to arch/sh. >> >> Problem: >> >> Certain class of devices, such as timers, certain clock drivers and irq chip >> drivers need to be probed early in the boot sequence. The currently preferred >> approach is using one of the OF_DECLARE() macros. This however does not create >> a platform device which has many drawbacks - such as not being able to use >> devres routines, dev_ log functions or no way of deferring the init OF function >> if some other resources are missing. > > I skimmed though this and it doesn't look horrible (how's that for > positive feedback? ;) ). But before going into the details, I think > first there needs to be agreement this is the right direction. > > The question does remain though as to whether this class of devices > should be platform drivers. They can't be modules. They can't be > hotplugged. Can they be runtime-pm enabled? So the advantage is ... > The main (but not the only) advantage for drivers that can both be platform drivers and OF_DECLARE drivers is that we get a single entry point and can reuse code without resorting to checking if (!dev). It results in more consistent code base. Another big advantage is consolidation of device tree and machine code for SoC drivers used in different boards of which some are still using board files and others are defined in DT (see: DaVinci). > I assume that the clock maintainers had some reason to move clocks to > be platform drivers. It's just not clear to me what that was. > >> For drivers that use both platform drivers and OF_DECLARE the situation is even >> more complicated as the code needs to take into account that there can possibly >> be no struct device present. For a specific use case that we're having problems >> with, please refer to the recent DaVinci common-clock conversion patches and >> the nasty workaround that this problem implies[3]. > > So devm_kzalloc will work with this solution? Why did we need > devm_kzalloc in the first place? The clocks can never be removed and > cleaning up on error paths is kind of pointless. The system would be > hosed, right? > It depends - not all clocks are necessary for system to boot. >> We also used to have an early platform drivers implementation but they were not >> integrated with the linux device model at all - they merely used the same data >> structures. The users could not use devres, defer probe and the early devices >> never became actual platform devices later on. >> >> Proposed solution: >> >> This series aims at solving this problem by (re-)introducing the concept of >> early platform drivers and devices - this time however in a way that seamlessly >> integrates with the existing platform drivers and also offers device-tree >> support. >> >> The idea is to provide a way for users to probe devices early, while already >> being able to use devres, devices resources and properties and also deferred >> probing. >> >> New structures are introduced: the early platform driver contains the >> early_probe callback which has the same signature as regular platform_device >> probe. This callback is called early on. The user can have both the early and >> regular probe speficied or only one of them and they both receive the same >> platform device object as argument. Any device data allocated early will be >> carried over to the normal probe. >> >> The architecture code is responsible for calling early_platform_start() in >> which the early drivers will be registered and devices populated from DT. > > Can we really do this in one spot for different devices (clk, timers, > irq). The sequence is all very carefully crafted. Platform specific > hooks is another thing to consider. > This is why I added support for early probe deferral - so that we can stop caring for the order as long as the drivers are aware of other resources they need and we call early_platform_start() the moment the earliest of the early devices is needed. >> Once the device and kobject mechanisms are ready, all early drivers and devices >> will be converted into real platform drivers and devices. Also: if any of the >> early platform registration functions will be called once early initialization >> is done, these functions will work like regular platform_device/driver ones. > > This could leave devices in a weird state. They've successfully probed > early, but then are on the deferred list for normal probe for example. > It's not really a weird state - once we're past the 'early' stage, the drivers' "earlyness" no longer plays any role. An early deferred device wouldn't be any different from other deferred devices. It's possible that we enabled some crucial resources in the early stage and then failed to fully probe the device later. I don't see a problem with that. Best regards, Bartosz Golaszewski -- To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, May 14, 2018 at 6:38 AM, Bartosz Golaszewski <brgl@bgdev.pl> wrote: > 2018-05-11 22:13 GMT+02:00 Rob Herring <robh+dt@kernel.org>: >> On Fri, May 11, 2018 at 11:20 AM, Bartosz Golaszewski <brgl@bgdev.pl> wrote: >>> This series is a follow-up to the RFC[1] posted a couple days ago. >>> >>> NOTE: this series applies on top of my recent patches[2] that move the previous >>> implementation of early platform devices to arch/sh. >>> >>> Problem: >>> >>> Certain class of devices, such as timers, certain clock drivers and irq chip >>> drivers need to be probed early in the boot sequence. The currently preferred >>> approach is using one of the OF_DECLARE() macros. This however does not create >>> a platform device which has many drawbacks - such as not being able to use >>> devres routines, dev_ log functions or no way of deferring the init OF function >>> if some other resources are missing. >> >> I skimmed though this and it doesn't look horrible (how's that for >> positive feedback? ;) ). But before going into the details, I think >> first there needs to be agreement this is the right direction. >> >> The question does remain though as to whether this class of devices >> should be platform drivers. They can't be modules. They can't be >> hotplugged. Can they be runtime-pm enabled? So the advantage is ... >> > > The main (but not the only) advantage for drivers that can both be > platform drivers and OF_DECLARE drivers is that we get a single entry > point and can reuse code without resorting to checking if (!dev). It > results in more consistent code base. Another big advantage is > consolidation of device tree and machine code for SoC drivers used in > different boards of which some are still using board files and others > are defined in DT (see: DaVinci). > >> I assume that the clock maintainers had some reason to move clocks to >> be platform drivers. It's just not clear to me what that was. >> >>> For drivers that use both platform drivers and OF_DECLARE the situation is even >>> more complicated as the code needs to take into account that there can possibly >>> be no struct device present. For a specific use case that we're having problems >>> with, please refer to the recent DaVinci common-clock conversion patches and >>> the nasty workaround that this problem implies[3]. >> >> So devm_kzalloc will work with this solution? Why did we need >> devm_kzalloc in the first place? The clocks can never be removed and >> cleaning up on error paths is kind of pointless. The system would be >> hosed, right? >> > > It depends - not all clocks are necessary for system to boot. That doesn't matter. You have a single driver for all/most of the clocks, so the driver can't be removed. >>> We also used to have an early platform drivers implementation but they were not >>> integrated with the linux device model at all - they merely used the same data >>> structures. The users could not use devres, defer probe and the early devices >>> never became actual platform devices later on. >>> >>> Proposed solution: >>> >>> This series aims at solving this problem by (re-)introducing the concept of >>> early platform drivers and devices - this time however in a way that seamlessly >>> integrates with the existing platform drivers and also offers device-tree >>> support. >>> >>> The idea is to provide a way for users to probe devices early, while already >>> being able to use devres, devices resources and properties and also deferred >>> probing. >>> >>> New structures are introduced: the early platform driver contains the >>> early_probe callback which has the same signature as regular platform_device >>> probe. This callback is called early on. The user can have both the early and >>> regular probe speficied or only one of them and they both receive the same >>> platform device object as argument. Any device data allocated early will be >>> carried over to the normal probe. >>> >>> The architecture code is responsible for calling early_platform_start() in >>> which the early drivers will be registered and devices populated from DT. >> >> Can we really do this in one spot for different devices (clk, timers, >> irq). The sequence is all very carefully crafted. Platform specific >> hooks is another thing to consider. >> > > This is why I added support for early probe deferral - so that we can > stop caring for the order as long as the drivers are aware of other > resources they need and we call early_platform_start() the moment the > earliest of the early devices is needed. Deferred probe helps for inter-device dependencies, but I am more concerned about timing of trying to register clocksources, irqchips, etc. What happens if we probe drivers before the infrastructure is initialized? Rob -- To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, May 11, 2018 at 11:20 AM, Bartosz Golaszewski <brgl@bgdev.pl> wrote: > From: Bartosz Golaszewski <bgolaszewski@baylibre.com> > > This introduces the core part of support for early platform drivers > and devices. > It looks like most of your prep patches are to separate the alloc and init of platform devices because you are essentially making early devices/drivers a sub-class. Maybe you could avoid doing that and simplify things a bit. Comments below based on doing that... > +/** > + * struct early_platform_driver > + * > + * @pdrv: real platform driver associated with this early platform driver > + * @list: list head for the list of early platform drivers > + * @early_probe: early probe callback > + */ > +struct early_platform_driver { > + struct platform_driver pdrv; > + struct list_head list; Couldn't you use an existing list in driver_private until you move over to the normal bus infra. > + int (*early_probe)(struct platform_device *); Just add this to platform_driver. > +}; > + > +/** > + * struct early_platform_device > + * > + * @pdev: real platform device associated with this early platform device > + * @list: list head for the list of early platform devices > + * @deferred: true if this device's early probe was deferred > + * @deferred_drv: early platform driver with which this device was matched > + */ > +struct early_platform_device { > + struct platform_device pdev; > + struct list_head list; Use a list in device_private? > + bool deferred; > + struct early_platform_driver *deferred_drv; Can't you use the existing deferred probe list? Rob -- To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, May 11, 2018 at 6:20 PM, Bartosz Golaszewski <brgl@bgdev.pl> wrote: > From: Bartosz Golaszewski <bgolaszewski@baylibre.com> > > Add a new single-bit field to struct device which will be used to > indicate that a device was probed early in the boot sequence. > > This does not affect the size of struct device on any architecture > I built on (ARM, ARM64, x86_64). > > Signed-off-by: Bartosz Golaszewski <bgolaszewski@baylibre.com> Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be> Gr{oetje,eeting}s, Geert
On Fri, May 11, 2018 at 6:20 PM, Bartosz Golaszewski <brgl@bgdev.pl> wrote: > From: Bartosz Golaszewski <bgolaszewski@baylibre.com> > > Early platform devices can have devres objects allocated in > early_probe(). This is not a bug so don't dump stack in this case. > > Signed-off-by: Bartosz Golaszewski <bgolaszewski@baylibre.com> Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be> Gr{oetje,eeting}s, Geert
On Fri, May 11, 2018 at 6:20 PM, Bartosz Golaszewski <brgl@bgdev.pl> wrote: > From: Bartosz Golaszewski <bgolaszewski@baylibre.com> > > We will use this function to match devices in the early platform core > code. Export it locally in drivers/base. > > Signed-off-by: Bartosz Golaszewski <bgolaszewski@baylibre.com> Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be> Gr{oetje,eeting}s, Geert
On Fri, May 11, 2018 at 6:20 PM, Bartosz Golaszewski <brgl@bgdev.pl> wrote: > From: Bartosz Golaszewski <bgolaszewski@baylibre.com> > > The early platform driver framework will allocate platform devices > using its own internal mechanisms. Provide a function that allows to > initialize a platform device object without allocating it. > > Signed-off-by: Bartosz Golaszewski <bgolaszewski@baylibre.com> Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be> Gr{oetje,eeting}s, Geert
On Fri, May 11, 2018 at 6:20 PM, Bartosz Golaszewski <brgl@bgdev.pl> wrote: > From: Bartosz Golaszewski <bgolaszewski@baylibre.com> > > The early platform driver mechanism will use this function as the > device release callback. Make it available in drivers/base. > > Signed-off-by: Bartosz Golaszewski <bgolaszewski@baylibre.com> Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be> Gr{oetje,eeting}s, Geert
Hi Bartosz, On Fri, May 11, 2018 at 6:20 PM, Bartosz Golaszewski <brgl@bgdev.pl> wrote: > From: Bartosz Golaszewski <bgolaszewski@baylibre.com> > > Provide a separate section in which pointers to early platform driver > structs will be stored. > > Signed-off-by: Bartosz Golaszewski <bgolaszewski@baylibre.com> Thanks for your patch! > --- a/include/asm-generic/vmlinux.lds.h > +++ b/include/asm-generic/vmlinux.lds.h > @@ -214,6 +214,16 @@ > #define CPU_METHOD_OF_TABLES() OF_TABLE(CONFIG_SMP, cpu_method) > #define CPUIDLE_METHOD_OF_TABLES() OF_TABLE(CONFIG_CPU_IDLE, cpuidle_method) > > +#ifdef CONFIG_EARLY_PLATFORM > +#define EARLY_PLATFORM_DRIVERS_TABLE() \ > + . = ALIGN(8); \ Should this use STRUCT_ALIGN() instead? > + VMLINUX_SYMBOL(__early_platform_drivers_table) = .; \ > + KEEP(*(__early_platform_drivers_table)) \ > + VMLINUX_SYMBOL(__early_platform_drivers_table_end) = .; > +#else > +#define EARLY_PLATFORM_DRIVERS_TABLE() > +#endif Gr{oetje,eeting}s, Geert
2018-05-14 23:29 GMT+02:00 Geert Uytterhoeven <geert@linux-m68k.org>: > Hi Bartosz, > > On Fri, May 11, 2018 at 6:20 PM, Bartosz Golaszewski <brgl@bgdev.pl> wrote: >> From: Bartosz Golaszewski <bgolaszewski@baylibre.com> >> >> Provide a separate section in which pointers to early platform driver >> structs will be stored. >> >> Signed-off-by: Bartosz Golaszewski <bgolaszewski@baylibre.com> > > Thanks for your patch! > >> --- a/include/asm-generic/vmlinux.lds.h >> +++ b/include/asm-generic/vmlinux.lds.h >> @@ -214,6 +214,16 @@ >> #define CPU_METHOD_OF_TABLES() OF_TABLE(CONFIG_SMP, cpu_method) >> #define CPUIDLE_METHOD_OF_TABLES() OF_TABLE(CONFIG_CPU_IDLE, cpuidle_method) >> >> +#ifdef CONFIG_EARLY_PLATFORM >> +#define EARLY_PLATFORM_DRIVERS_TABLE() \ >> + . = ALIGN(8); \ > > Should this use STRUCT_ALIGN() instead? > No, we're only using it to store pointers to structs, not the actual struct early_platform_driver objects. >> + VMLINUX_SYMBOL(__early_platform_drivers_table) = .; \ >> + KEEP(*(__early_platform_drivers_table)) \ >> + VMLINUX_SYMBOL(__early_platform_drivers_table_end) = .; >> +#else >> +#define EARLY_PLATFORM_DRIVERS_TABLE() >> +#endif > > Gr{oetje,eeting}s, > > Geert > > -- > Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org > > In personal conversations with technical people, I call myself a hacker. But > when I'm talking to journalists I just say "programmer" or something like that. > -- Linus Torvalds Thanks, Bartosz -- To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
2018-05-14 15:37 GMT+02:00 Rob Herring <robh+dt@kernel.org>: > On Fri, May 11, 2018 at 11:20 AM, Bartosz Golaszewski <brgl@bgdev.pl> wrote: >> From: Bartosz Golaszewski <bgolaszewski@baylibre.com> >> >> This introduces the core part of support for early platform drivers >> and devices. >> > > It looks like most of your prep patches are to separate the alloc and > init of platform devices because you are essentially making early > devices/drivers a sub-class. Maybe you could avoid doing that and > simplify things a bit. Comments below based on doing that... > My aim was to change as little as possible for everybody else while fixing our problem. These changes are already controversial enough without risky reusing of existing fields in common structures. I was just afraid that there are too many intricacies for it to be safe. >> +/** >> + * struct early_platform_driver >> + * >> + * @pdrv: real platform driver associated with this early platform driver >> + * @list: list head for the list of early platform drivers >> + * @early_probe: early probe callback >> + */ >> +struct early_platform_driver { >> + struct platform_driver pdrv; >> + struct list_head list; > > Couldn't you use an existing list in driver_private until you move > over to the normal bus infra. > This is something that the previous implementation did. It was quite unreadable, so I decided to go with a separate list. >> + int (*early_probe)(struct platform_device *); > > Just add this to platform_driver. > This would extend the structure for everybody else while there'll be very few such devices and not all systems would even require it. >> +}; >> + >> +/** >> + * struct early_platform_device >> + * >> + * @pdev: real platform device associated with this early platform device >> + * @list: list head for the list of early platform devices >> + * @deferred: true if this device's early probe was deferred >> + * @deferred_drv: early platform driver with which this device was matched >> + */ >> +struct early_platform_device { >> + struct platform_device pdev; >> + struct list_head list; > > Use a list in device_private? > >> + bool deferred; >> + struct early_platform_driver *deferred_drv; > > Can't you use the existing deferred probe list? > I thought about it, but I was afraid there could be some timing issues with that and decided against it. The early deferral also doesn't work in a workque, but is synchronous instead. Best regards, Bartosz Golaszewski -- To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, May 15, 2018 at 9:06 AM, Bartosz Golaszewski <brgl@bgdev.pl> wrote: > 2018-05-14 15:37 GMT+02:00 Rob Herring <robh+dt@kernel.org>: >> On Fri, May 11, 2018 at 11:20 AM, Bartosz Golaszewski <brgl@bgdev.pl> wrote: >>> From: Bartosz Golaszewski <bgolaszewski@baylibre.com> >>> >>> This introduces the core part of support for early platform drivers >>> and devices. >>> >> >> It looks like most of your prep patches are to separate the alloc and >> init of platform devices because you are essentially making early >> devices/drivers a sub-class. Maybe you could avoid doing that and >> simplify things a bit. Comments below based on doing that... >> > > My aim was to change as little as possible for everybody else while > fixing our problem. These changes are already controversial enough > without risky reusing of existing fields in common structures. I was > just afraid that there are too many intricacies for it to be safe. I don't think those intricacies would go away just by having separate fields. Perhaps it would make things fail more explicitly. After all, I think it needs to be a very atomic operation when a device is switched. >>> +/** >>> + * struct early_platform_driver >>> + * >>> + * @pdrv: real platform driver associated with this early platform driver >>> + * @list: list head for the list of early platform drivers >>> + * @early_probe: early probe callback >>> + */ >>> +struct early_platform_driver { >>> + struct platform_driver pdrv; >>> + struct list_head list; >> >> Couldn't you use an existing list in driver_private until you move >> over to the normal bus infra. >> > > This is something that the previous implementation did. It was quite > unreadable, so I decided to go with a separate list. > >>> + int (*early_probe)(struct platform_device *); >> >> Just add this to platform_driver. >> > > This would extend the structure for everybody else while there'll be > very few such devices and not all systems would even require it. > >>> +}; >>> + >>> +/** >>> + * struct early_platform_device >>> + * >>> + * @pdev: real platform device associated with this early platform device >>> + * @list: list head for the list of early platform devices >>> + * @deferred: true if this device's early probe was deferred >>> + * @deferred_drv: early platform driver with which this device was matched >>> + */ >>> +struct early_platform_device { >>> + struct platform_device pdev; >>> + struct list_head list; >> >> Use a list in device_private? >> >>> + bool deferred; >>> + struct early_platform_driver *deferred_drv; >> >> Can't you use the existing deferred probe list? >> > > I thought about it, but I was afraid there could be some timing issues > with that and decided against it. The early deferral also doesn't work > in a workque, but is synchronous instead. I didn't mean use the wq, but just the list fields. You'd still have the early list and normal list with different list heads. If you ever had a device wanting to be on both lists at the same time, then you've got major problems. Rob -- To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Rob, Quoting Rob Herring (2018-05-14 06:20:57) > On Mon, May 14, 2018 at 6:38 AM, Bartosz Golaszewski <brgl@bgdev.pl> wrote: > > 2018-05-11 22:13 GMT+02:00 Rob Herring <robh+dt@kernel.org>: > >> On Fri, May 11, 2018 at 11:20 AM, Bartosz Golaszewski <brgl@bgdev.pl> wrote: > >>> This series is a follow-up to the RFC[1] posted a couple days ago. > >>> > >>> NOTE: this series applies on top of my recent patches[2] that move the previous > >>> implementation of early platform devices to arch/sh. > >>> > >>> Problem: > >>> > >>> Certain class of devices, such as timers, certain clock drivers and irq chip > >>> drivers need to be probed early in the boot sequence. The currently preferred > >>> approach is using one of the OF_DECLARE() macros. This however does not create > >>> a platform device which has many drawbacks - such as not being able to use > >>> devres routines, dev_ log functions or no way of deferring the init OF function > >>> if some other resources are missing. > >> > >> I skimmed though this and it doesn't look horrible (how's that for > >> positive feedback? ;) ). But before going into the details, I think > >> first there needs to be agreement this is the right direction. > >> > >> The question does remain though as to whether this class of devices > >> should be platform drivers. They can't be modules. They can't be > >> hotplugged. Can they be runtime-pm enabled? So the advantage is ... > >> > > > > The main (but not the only) advantage for drivers that can both be > > platform drivers and OF_DECLARE drivers is that we get a single entry > > point and can reuse code without resorting to checking if (!dev). It > > results in more consistent code base. Another big advantage is > > consolidation of device tree and machine code for SoC drivers used in > > different boards of which some are still using board files and others > > are defined in DT (see: DaVinci). > > > >> I assume that the clock maintainers had some reason to move clocks to > >> be platform drivers. It's just not clear to me what that was. > >> > >>> For drivers that use both platform drivers and OF_DECLARE the situation is even > >>> more complicated as the code needs to take into account that there can possibly > >>> be no struct device present. For a specific use case that we're having problems > >>> with, please refer to the recent DaVinci common-clock conversion patches and > >>> the nasty workaround that this problem implies[3]. > >> > >> So devm_kzalloc will work with this solution? Why did we need > >> devm_kzalloc in the first place? The clocks can never be removed and > >> cleaning up on error paths is kind of pointless. The system would be > >> hosed, right? > >> > > > > It depends - not all clocks are necessary for system to boot. > > That doesn't matter. You have a single driver for all/most of the > clocks, so the driver can't be removed. -ECANOFWORMS A couple of quick rebuttals, but I imagine we're going to disagree on the platform_driver thing as a matter of taste no matter what... 1) There should be multiple clk drivers in a properly modeled system. Some folks still incorrectly put all clocks in a system into a single driver because That's How We Used To Do It, and some systems (simpler ones) really only have a single clock generator IP block. Excepting those two reasons above, we really should have separate drivers for clocks controlled by the PMIC, for the (one or more) clock generator blocks inside of the AP/SoC, and then even more for the drivers that map to IP blocks that have their own clock gens. Good examples of the latter are display controllers that generate their own PLL and pixel clock. Or MMC controllers that have a runtime-programmable clock divider. Examples of these are merged into mainline. 2) Stephen and I wanted clock drivers to actually be represented in the driver model. There were these gigantic clock drivers that exclusively used CLK_OF_DECLARE and they just sort of floated out there in the ether... no representation in sysfs, no struct device to map onto a clock controller struct, etc. I'd be happy to hear why you think that platform_driver is a bad fit for a device driver that generally manages memory-mapped system resources that are part of the system glue and not really tied to a specific bus. Sounds like a good fit to me. If platform_driver doesn't handle the early boot thing well, that tells me that we have a problem to solve in platform_driver, not in the clk subsystem or drivers. 3) Lots of clock controllers should be loadable modules. E.g. i2c clock expanders, potentially external PMIC-related drivers, external audio codecs, etc. Again, repeating my point #1 above, just because many platforms have a monolithic clock driver does not mean that this is the Right Way to do it. Best regards, Mike > > >>> We also used to have an early platform drivers implementation but they were not > >>> integrated with the linux device model at all - they merely used the same data > >>> structures. The users could not use devres, defer probe and the early devices > >>> never became actual platform devices later on. > >>> > >>> Proposed solution: > >>> > >>> This series aims at solving this problem by (re-)introducing the concept of > >>> early platform drivers and devices - this time however in a way that seamlessly > >>> integrates with the existing platform drivers and also offers device-tree > >>> support. > >>> > >>> The idea is to provide a way for users to probe devices early, while already > >>> being able to use devres, devices resources and properties and also deferred > >>> probing. > >>> > >>> New structures are introduced: the early platform driver contains the > >>> early_probe callback which has the same signature as regular platform_device > >>> probe. This callback is called early on. The user can have both the early and > >>> regular probe speficied or only one of them and they both receive the same > >>> platform device object as argument. Any device data allocated early will be > >>> carried over to the normal probe. > >>> > >>> The architecture code is responsible for calling early_platform_start() in > >>> which the early drivers will be registered and devices populated from DT. > >> > >> Can we really do this in one spot for different devices (clk, timers, > >> irq). The sequence is all very carefully crafted. Platform specific > >> hooks is another thing to consider. > >> > > > > This is why I added support for early probe deferral - so that we can > > stop caring for the order as long as the drivers are aware of other > > resources they need and we call early_platform_start() the moment the > > earliest of the early devices is needed. > > Deferred probe helps for inter-device dependencies, but I am more > concerned about timing of trying to register clocksources, irqchips, > etc. What happens if we probe drivers before the infrastructure is > initialized? > > Rob -- To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, May 30, 2018 at 2:40 PM, Michael Turquette <mturquette@baylibre.com> wrote: > Hi Rob, > > Quoting Rob Herring (2018-05-14 06:20:57) >> On Mon, May 14, 2018 at 6:38 AM, Bartosz Golaszewski <brgl@bgdev.pl> wrote: >> > 2018-05-11 22:13 GMT+02:00 Rob Herring <robh+dt@kernel.org>: >> >> On Fri, May 11, 2018 at 11:20 AM, Bartosz Golaszewski <brgl@bgdev.pl> wrote: >> >>> This series is a follow-up to the RFC[1] posted a couple days ago. >> >>> >> >>> NOTE: this series applies on top of my recent patches[2] that move the previous >> >>> implementation of early platform devices to arch/sh. >> >>> >> >>> Problem: >> >>> >> >>> Certain class of devices, such as timers, certain clock drivers and irq chip >> >>> drivers need to be probed early in the boot sequence. The currently preferred >> >>> approach is using one of the OF_DECLARE() macros. This however does not create >> >>> a platform device which has many drawbacks - such as not being able to use >> >>> devres routines, dev_ log functions or no way of deferring the init OF function >> >>> if some other resources are missing. >> >> >> >> I skimmed though this and it doesn't look horrible (how's that for >> >> positive feedback? ;) ). But before going into the details, I think >> >> first there needs to be agreement this is the right direction. >> >> >> >> The question does remain though as to whether this class of devices >> >> should be platform drivers. They can't be modules. They can't be >> >> hotplugged. Can they be runtime-pm enabled? So the advantage is ... >> >> >> > >> > The main (but not the only) advantage for drivers that can both be >> > platform drivers and OF_DECLARE drivers is that we get a single entry >> > point and can reuse code without resorting to checking if (!dev). It >> > results in more consistent code base. Another big advantage is >> > consolidation of device tree and machine code for SoC drivers used in >> > different boards of which some are still using board files and others >> > are defined in DT (see: DaVinci). >> > >> >> I assume that the clock maintainers had some reason to move clocks to >> >> be platform drivers. It's just not clear to me what that was. >> >> >> >>> For drivers that use both platform drivers and OF_DECLARE the situation is even >> >>> more complicated as the code needs to take into account that there can possibly >> >>> be no struct device present. For a specific use case that we're having problems >> >>> with, please refer to the recent DaVinci common-clock conversion patches and >> >>> the nasty workaround that this problem implies[3]. >> >> >> >> So devm_kzalloc will work with this solution? Why did we need >> >> devm_kzalloc in the first place? The clocks can never be removed and >> >> cleaning up on error paths is kind of pointless. The system would be >> >> hosed, right? >> >> >> > >> > It depends - not all clocks are necessary for system to boot. >> >> That doesn't matter. You have a single driver for all/most of the >> clocks, so the driver can't be removed. > > -ECANOFWORMS > > A couple of quick rebuttals, but I imagine we're going to disagree on > the platform_driver thing as a matter of taste no matter what... It's really more should the clocksource, clockevents and primary interrupt controller be drivers. Let's get agreement on that first. If yes, then it probably does make sense that their dependencies are drivers too. If not, then making only the dependencies drivers doesn't seem right to me. > 1) There should be multiple clk drivers in a properly modeled system. > Some folks still incorrectly put all clocks in a system into a single > driver because That's How We Used To Do It, and some systems (simpler > ones) really only have a single clock generator IP block. > > Excepting those two reasons above, we really should have separate > drivers for clocks controlled by the PMIC, for the (one or more) clock > generator blocks inside of the AP/SoC, and then even more for the > drivers that map to IP blocks that have their own clock gens. I agree those should be separate entities at least. But for a given h/w block, if you already have to use OF_DECLARE, why would you try to split that into OF_DECLARE and a driver? what advantage does putting non-boot essential clocks in a driver or transitioning to a driver get you? And I've seen PMIC clocks could be inputs to the SoC's clock controller(s), so the dependencies get more complicated. Then does the PMIC driver and its dependencies need to be early drivers too? > Good examples of the latter are display controllers that generate their > own PLL and pixel clock. Or MMC controllers that have a > runtime-programmable clock divider. Examples of these are merged into > mainline. But those are drivers of types other than a clock controller that happen to register some clocks as well. I wasn't saying these cases can't or shouldn't be part of the driver model. Look at irqchips. We have some that use the driver model (e.g. every GPIO driver) and some that don't because there's no need (e.g. GIC). > 2) Stephen and I wanted clock drivers to actually be represented in the > driver model. There were these gigantic clock drivers that exclusively > used CLK_OF_DECLARE and they just sort of floated out there in the > ether... no representation in sysfs, no struct device to map onto a > clock controller struct, etc. > > I'd be happy to hear why you think that platform_driver is a bad fit for > a device driver that generally manages memory-mapped system resources > that are part of the system glue and not really tied to a specific bus. > Sounds like a good fit to me. > > If platform_driver doesn't handle the early boot thing well, that tells > me that we have a problem to solve in platform_driver, not in the clk > subsystem or drivers. Doing things earlier is not the only way to solve the problems. Perhaps we need to figure out how to start things later. Maybe it's not feasible here, I don't know. > 3) Lots of clock controllers should be loadable modules. E.g. i2c clock > expanders, potentially external PMIC-related drivers, external audio > codecs, etc. > > Again, repeating my point #1 above, just because many platforms have a > monolithic clock driver does not mean that this is the Right Way to do > it. And in those cases, I completely agree they should be part of a driver. Rob -- To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Rob, On Thu, May 31, 2018 at 12:36 AM, Rob Herring <robh+dt@kernel.org> wrote: > On Wed, May 30, 2018 at 2:40 PM, Michael Turquette > <mturquette@baylibre.com> wrote: >> Quoting Rob Herring (2018-05-14 06:20:57) >>> On Mon, May 14, 2018 at 6:38 AM, Bartosz Golaszewski <brgl@bgdev.pl> wrote: >>> > 2018-05-11 22:13 GMT+02:00 Rob Herring <robh+dt@kernel.org>: >>> >> On Fri, May 11, 2018 at 11:20 AM, Bartosz Golaszewski <brgl@bgdev.pl> wrote: >>> >>> This series is a follow-up to the RFC[1] posted a couple days ago. >>> >>> >>> >>> NOTE: this series applies on top of my recent patches[2] that move the previous >>> >>> implementation of early platform devices to arch/sh. >>> >>> >>> >>> Problem: >>> >>> >>> >>> Certain class of devices, such as timers, certain clock drivers and irq chip >>> >>> drivers need to be probed early in the boot sequence. The currently preferred >>> >>> approach is using one of the OF_DECLARE() macros. This however does not create >>> >>> a platform device which has many drawbacks - such as not being able to use >>> >>> devres routines, dev_ log functions or no way of deferring the init OF function >>> >>> if some other resources are missing. >>> >> >>> >> I skimmed though this and it doesn't look horrible (how's that for >>> >> positive feedback? ;) ). But before going into the details, I think >>> >> first there needs to be agreement this is the right direction. >>> >> >>> >> The question does remain though as to whether this class of devices >>> >> should be platform drivers. They can't be modules. They can't be >>> >> hotplugged. Can they be runtime-pm enabled? So the advantage is ... >>> >> >>> > >>> > The main (but not the only) advantage for drivers that can both be >>> > platform drivers and OF_DECLARE drivers is that we get a single entry >>> > point and can reuse code without resorting to checking if (!dev). It >>> > results in more consistent code base. Another big advantage is >>> > consolidation of device tree and machine code for SoC drivers used in >>> > different boards of which some are still using board files and others >>> > are defined in DT (see: DaVinci). >>> > >>> >> I assume that the clock maintainers had some reason to move clocks to >>> >> be platform drivers. It's just not clear to me what that was. >>> >> >>> >>> For drivers that use both platform drivers and OF_DECLARE the situation is even >>> >>> more complicated as the code needs to take into account that there can possibly >>> >>> be no struct device present. For a specific use case that we're having problems >>> >>> with, please refer to the recent DaVinci common-clock conversion patches and >>> >>> the nasty workaround that this problem implies[3]. >>> >> >>> >> So devm_kzalloc will work with this solution? Why did we need >>> >> devm_kzalloc in the first place? The clocks can never be removed and >>> >> cleaning up on error paths is kind of pointless. The system would be >>> >> hosed, right? >>> >> >>> > >>> > It depends - not all clocks are necessary for system to boot. >>> >>> That doesn't matter. You have a single driver for all/most of the >>> clocks, so the driver can't be removed. >> >> -ECANOFWORMS >> >> A couple of quick rebuttals, but I imagine we're going to disagree on >> the platform_driver thing as a matter of taste no matter what... > > It's really more should the clocksource, clockevents and primary > interrupt controller be drivers. Let's get agreement on that first. If > yes, then it probably does make sense that their dependencies are > drivers too. If not, then making only the dependencies drivers doesn't > seem right to me. Yes, they should all be drivers. Assuming clocksources, clockevents, or primary interrupt controllers are special, and thus forcing everyone to use OF_DECLARE for the whole subsystem, doesn't fly everywhere. Clockevents and interrupt controllers can have a module clock. All three can be part of a PM Domain, which requires a struct device to be handled properly. >> 1) There should be multiple clk drivers in a properly modeled system. >> Some folks still incorrectly put all clocks in a system into a single >> driver because That's How We Used To Do It, and some systems (simpler >> ones) really only have a single clock generator IP block. >> >> Excepting those two reasons above, we really should have separate >> drivers for clocks controlled by the PMIC, for the (one or more) clock >> generator blocks inside of the AP/SoC, and then even more for the >> drivers that map to IP blocks that have their own clock gens. > > I agree those should be separate entities at least. But for a given > h/w block, if you already have to use OF_DECLARE, why would you try to > split that into OF_DECLARE and a driver? what advantage does putting > non-boot essential clocks in a driver or transitioning to a driver get > you? You may want to split it because of dependencies. OF_DECLARE doesn't handle EPROBE_DEFER, while some critical parts may be needed early. > And I've seen PMIC clocks could be inputs to the SoC's clock > controller(s), so the dependencies get more complicated. Then does the > PMIC driver and its dependencies need to be early drivers too? Especially if there are clock loops, but that's something different. Cfr. cs2000 in arch/arm64/boot/dts/renesas/salvator-common.dtsi, which provides a clock from the main SoC, but also consumes a clock from the main SoC. The latter is modeled in DT as a fixed-clock as a workaround. >> Good examples of the latter are display controllers that generate their >> own PLL and pixel clock. Or MMC controllers that have a >> runtime-programmable clock divider. Examples of these are merged into >> mainline. > > But those are drivers of types other than a clock controller that > happen to register some clocks as well. I wasn't saying these cases > can't or shouldn't be part of the driver model. Look at irqchips. We > have some that use the driver model (e.g. every GPIO driver) and some > that don't because there's no need (e.g. GIC). There is a need for using the driver model for the GIC. On some platforms, the GIC can be part of a clock and/or power domain. For secondary GICs, that got fixed in commit 9c8edddfc9924cb4 ("irqchip/gic: Add platform driver for non-root GICs that require RPM"). For primary GICs, it's still not fixed, so we have to live with CLK_IS_CRITICAL, and rely on other critical devices in the same power domain to avoid the power domain being powered off. >> 2) Stephen and I wanted clock drivers to actually be represented in the >> driver model. There were these gigantic clock drivers that exclusively >> used CLK_OF_DECLARE and they just sort of floated out there in the >> ether... no representation in sysfs, no struct device to map onto a >> clock controller struct, etc. >> >> I'd be happy to hear why you think that platform_driver is a bad fit for >> a device driver that generally manages memory-mapped system resources >> that are part of the system glue and not really tied to a specific bus. >> Sounds like a good fit to me. >> >> If platform_driver doesn't handle the early boot thing well, that tells >> me that we have a problem to solve in platform_driver, not in the clk >> subsystem or drivers. > > Doing things earlier is not the only way to solve the problems. > Perhaps we need to figure out how to start things later. Maybe it's > not feasible here, I don't know. The fixed probe order imposed by OF_DECLARE() limits this: if your OF_DECLARE() driver depends on something else, the latter must become an early device. If all subsystems would use real devices, EPROBE_DEFER would handle most of it automatically. Then, next step would be to avoid triggering EPROBE_DEFER, e.g. by analyzing dependencies in DT... Gr{oetje,eeting}s, Geert
* Geert Uytterhoeven <geert@linux-m68k.org> [180531 06:45]: > Yes, they should all be drivers. > > Assuming clocksources, clockevents, or primary interrupt controllers are > special, and thus forcing everyone to use OF_DECLARE for the whole > subsystem, doesn't fly everywhere. > > Clockevents and interrupt controllers can have a module clock. > All three can be part of a PM Domain, which requires a struct device to > be handled properly. I agree it sure would be nice to have all these as drivers. I believe SoC specific clockevent using SoC specific clocks is the reason why some clocks cannot currently be drivers. Regards, Tony -- To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
pt., 11 maj 2018 o 18:21 Bartosz Golaszewski <brgl@bgdev.pl> napisaĆ(a): > > This series is a follow-up to the RFC[1] posted a couple days ago. > > NOTE: this series applies on top of my recent patches[2] that move the previous > implementation of early platform devices to arch/sh. > > Problem: > > Certain class of devices, such as timers, certain clock drivers and irq chip > drivers need to be probed early in the boot sequence. The currently preferred > approach is using one of the OF_DECLARE() macros. This however does not create > a platform device which has many drawbacks - such as not being able to use > devres routines, dev_ log functions or no way of deferring the init OF function > if some other resources are missing. > > For drivers that use both platform drivers and OF_DECLARE the situation is even > more complicated as the code needs to take into account that there can possibly > be no struct device present. For a specific use case that we're having problems > with, please refer to the recent DaVinci common-clock conversion patches and > the nasty workaround that this problem implies[3]. > > We also used to have an early platform drivers implementation but they were not > integrated with the linux device model at all - they merely used the same data > structures. The users could not use devres, defer probe and the early devices > never became actual platform devices later on. > > Proposed solution: > > This series aims at solving this problem by (re-)introducing the concept of > early platform drivers and devices - this time however in a way that seamlessly > integrates with the existing platform drivers and also offers device-tree > support. > > The idea is to provide a way for users to probe devices early, while already > being able to use devres, devices resources and properties and also deferred > probing. > > New structures are introduced: the early platform driver contains the > early_probe callback which has the same signature as regular platform_device > probe. This callback is called early on. The user can have both the early and > regular probe speficied or only one of them and they both receive the same > platform device object as argument. Any device data allocated early will be > carried over to the normal probe. > > The architecture code is responsible for calling early_platform_start() in > which the early drivers will be registered and devices populated from DT. > > Once the device and kobject mechanisms are ready, all early drivers and devices > will be converted into real platform drivers and devices. Also: if any of the > early platform registration functions will be called once early initialization > is done, these functions will work like regular platform_device/driver ones. > > Patches 1-9/12 introduce changes to existing code that are necessary before > adding support for early drivers. Patch 10/12 contains the new framwork in an > isolated file which can be compiled only if needed by the architecture. > > Patch 11/12 contains a dummy early platform driver that serves as a code > example and can be used for simple testing. > > The last patch finally makes the of/platform code aware of early platform > drivers. > > If accepted, this new mechanism could potentially lead to consolidation of the > code currently used by users of OF_DECLARE, since they could be converted to > actual platform drivers. > > [1] https://lkml.org/lkml/2018/4/26/657 > [2] https://lkml.org/lkml/2018/4/30/547 > [3] https://lkml.org/lkml/2018/4/26/1094 > > Bartosz Golaszewski (12): > platform/early: add a new field to struct device > platform/early: don't WARN() on non-empty devres list for early > devices > platform/early: export platform_match() locally > platform: provide a separate function for initializing platform > devices > platform: export platform_device_release() locally > of: add a new flag for OF device nodes > of/platform: provide a separate routine for setting up device > resources > of/platform: provide a separate routine for device initialization > platform/early: add an init section for early driver data > platform/early: implement support for early platform drivers > misc: implement a dummy early platform driver > of/platform: make the OF code aware of early platform drivers > > drivers/base/Kconfig | 3 + > drivers/base/Makefile | 1 + > drivers/base/base.h | 4 + > drivers/base/dd.c | 2 +- > drivers/base/early.c | 332 ++++++++++++++++++++++++++++++ > drivers/base/platform.c | 28 ++- > drivers/misc/Kconfig | 8 + > drivers/misc/Makefile | 2 + > drivers/misc/dummy-early.c | 82 ++++++++ > drivers/of/platform.c | 85 +++++--- > include/asm-generic/vmlinux.lds.h | 11 + > include/linux/device.h | 1 + > include/linux/early_platform.h | 75 +++++++ > include/linux/of.h | 1 + > include/linux/of_platform.h | 2 + > include/linux/platform_device.h | 2 + > 16 files changed, 604 insertions(+), 35 deletions(-) > create mode 100644 drivers/base/early.c > create mode 100644 drivers/misc/dummy-early.c > create mode 100644 include/linux/early_platform.h > > -- > 2.17.0 > Hi For anyone interested: I'll be doing a BoF about early platform drivers next week on Monday at 6:00pm during the ELCE 2018 in Edinburgh. It would be great to discuss this topic personally. Best regards and hope to see you there Bartosz Golaszewski