diff mbox series

[OpenWrt-Devel] config: enable some useful features on !SMALL_FLASH devices

Message ID 20181229055126.GA9794@makrotopia.org
State Accepted
Delegated to: Daniel Golle
Headers show
Series [OpenWrt-Devel] config: enable some useful features on !SMALL_FLASH devices | expand

Commit Message

Daniel Golle Dec. 29, 2018, 5:51 a.m. UTC
enable kernel features needed for procd-ujail, procd-seccomp, lxc and
lvm2 on devices with big enough flash. Those packages are currently
useless in binary builds due to missing kernel features.
Enable the features on devices which can bare with the extra space
consumption.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
---
 config/Config-kernel.in | 32 ++++++++++++++++----------------
 1 file changed, 16 insertions(+), 16 deletions(-)

Comments

Daniel Golle Jan. 1, 2019, 4:56 p.m. UTC | #1
Hi!

On Sun, Dec 30, 2018 at 11:26:58AM +0100, Petr Štetiar wrote:
> Daniel Golle <daniel@makrotopia.org> [2018-12-29 06:51:32]:
> 
> >  config KERNEL_AIO
> >  config KERNEL_FHANDLE
> >  config KERNEL_FANOTIFY
> > +	default y if !SMALL_FLASH
> 
> What about `FEATURES += nas` to make it clear and don't abuse SMALL_FLASH.

This is not necessarily only used on NAS devices. systemd requires
FHANDLE and FANOTIFY (eg. when running inside LXC container), lvm2
needs AIO. Both could well run on a modern router or SBC having USB or
an SDCARD slot.
> 
> >  config KERNEL_CGROUPS
> >  config KERNEL_CPUSETS
> >  config KERNEL_CGROUP_CPUACCT
> >  config KERNEL_RESOURCE_COUNTERS
> >  config KERNEL_MEMCG
> >  config KERNEL_MEMCG_KMEM
> >  menuconfig KERNEL_CGROUP_SCHED
> >   config KERNEL_FAIR_GROUP_SCHED
> >   config KERNEL_RT_GROUP_SCHED
> >  config KERNEL_NAMESPACES
> >  config KERNEL_LXC_MISC
> >  config KERNEL_SECCOMP_FILTER
> >  config KERNEL_SECCOMP
> > -		default n
> > +		default y if !SMALL_FLASH
> 
> What about `FEATURES += containers` ?

From what I understood FEATURES is supposed to reflect hardware
capabilities -- all the above are generic software features useful on
any device having the capacity (ie. flash and RAM) to make use of them.

> 
> Daniel Engberg <daniel.engberg.lists@pyret.net> [2018-12-30 10:21:46]:
> 
> > however KERNEL_CGROUPS, config KERNEL_NAMESPACES, config KERNEL_LXC_MISC,
> > KERNEL_SECCOMP_FILTER are very limited use cases to my knowledge and more or
> > less only used on x86*?
> 
> There are other quite powerful platforms like mvebu, imx6, ipq etc. where you
> could use this as well.

I use LXC on oxnas/ox820 (ARM11mpcore) and ramips/mt7621 (MIPS1004Kc),
running Debian inside LXC containers (and it's annoying that I can't
use regular OpenWrt releases or buildbot-generated snapshots on the).


Cheers


Daniel
Petr Štetiar Jan. 1, 2019, 7:46 p.m. UTC | #2
Daniel Golle <daniel@makrotopia.org> [2019-01-01 17:56:25]:

Hi,

> On Sun, Dec 30, 2018 at 11:26:58AM +0100, Petr Štetiar wrote:
> > Daniel Golle <daniel@makrotopia.org> [2018-12-29 06:51:32]:
> > 
> > >  config KERNEL_AIO
> > >  config KERNEL_FHANDLE
> > >  config KERNEL_FANOTIFY
> > > +	default y if !SMALL_FLASH
> > 
> > What about `FEATURES += nas` to make it clear and don't abuse SMALL_FLASH.
> 
> This is not necessarily only used on NAS devices. systemd requires
> FHANDLE and FANOTIFY (eg. when running inside LXC container), lvm2
> needs AIO. Both could well run on a modern router or SBC having USB or
> an SDCARD slot.

to me it's still just NAS and container use cases. I'm afraid, that adding
more bloat to kernels for devices with USB and MMC/SD card slots would be
rejected also.

> > >  config KERNEL_CGROUPS
> > >  config KERNEL_CPUSETS
> > >  config KERNEL_CGROUP_CPUACCT
> > >  config KERNEL_RESOURCE_COUNTERS
> > >  config KERNEL_MEMCG
> > >  config KERNEL_MEMCG_KMEM
> > >  menuconfig KERNEL_CGROUP_SCHED
> > >   config KERNEL_FAIR_GROUP_SCHED
> > >   config KERNEL_RT_GROUP_SCHED
> > >  config KERNEL_NAMESPACES
> > >  config KERNEL_LXC_MISC
> > >  config KERNEL_SECCOMP_FILTER
> > >  config KERNEL_SECCOMP
> > > -		default n
> > > +		default y if !SMALL_FLASH
> > 
> > What about `FEATURES += containers` ?
> 
> From what I understood FEATURES is supposed to reflect hardware
> capabilities

Well, almost. I've found `squashfs` and `ext4` in there also.

>  -- all the above are generic software features useful on any device having
>  the capacity (ie. flash and RAM) to make use of them.

But as other Daniel already suggested, !SMALL_FLASH isn't proper group
selection either. Speaking about the capacity, did you measured how much those
features add to the kernel images?

> > Daniel Engberg <daniel.engberg.lists@pyret.net> [2018-12-30 10:21:46]:
> > 
> > > however KERNEL_CGROUPS, config KERNEL_NAMESPACES, config KERNEL_LXC_MISC,
> > > KERNEL_SECCOMP_FILTER are very limited use cases to my knowledge and more or
> > > less only used on x86*?
> > 
> > There are other quite powerful platforms like mvebu, imx6, ipq etc. where you
> > could use this as well.
> 
> I use LXC on oxnas/ox820 (ARM11mpcore) and ramips/mt7621 (MIPS1004Kc),

Ok, looking at IMAGE_SIZE values for mt7621 yields following results:

 IMAGE_SIZE := 6016k                         TP-LINK RE350 v1
 IMAGE_SIZE := 7798784                       I-O DATA WN-GX300GR
 IMAGE_SIZE := $(ralink_default_fw_size_4M)  MT7621 EVB
 IMAGE_SIZE := $(ralink_default_fw_size_8M)  AP-MT7621A-V60 EVB

So it wouldn't be wise to add more bloat into kernels for those devices.

> running Debian inside LXC containers (and it's annoying that I can't
> use regular OpenWrt releases or buildbot-generated snapshots on the).

I would like to do the same, but on the other hand I also understand, why your
patch as it is wouldn't be accepted. Our containers/NAS use cases are still
very rare, thus it makes no sense to enable those features by default to every
other !SMALL_FLASH target. As you can see on mt7621, it's not marked as
SMALL_FLASH, yet there are devices which might be considered SMALL_FLASH.

Now I realize, that it couldn't be handled with FEATURES anyway, as this is
always too broad group selection and ideally you want to enable those features
for specific devices only, meaning different kernels, images etc.

-- ynezz
Daniel Dickinson Jan. 7, 2019, 3:28 a.m. UTC | #3
On 2018-12-29 12:51 a.m., Daniel Golle wrote:
> enable kernel features needed for procd-ujail, procd-seccomp, lxc and
> lvm2 on devices with big enough flash. Those packages are currently
> useless in binary builds due to missing kernel features.
> Enable the features on devices which can bare with the extra space
> consumption.
>
On a related note: https://github.com/openwrt/openwrt/pull/1707 fixes a
missing symbol with LXC enabled.
Daniel Golle Jan. 7, 2019, 9:03 a.m. UTC | #4
On Tue, Jan 01, 2019 at 08:46:25PM +0100, Petr Štetiar wrote:
> Daniel Golle <daniel@makrotopia.org> [2019-01-01 17:56:25]:
> 
> Hi,
> 
> > On Sun, Dec 30, 2018 at 11:26:58AM +0100, Petr Štetiar wrote:
> > > Daniel Golle <daniel@makrotopia.org> [2018-12-29 06:51:32]:
> > > 
> > > >  config KERNEL_AIO
> > > >  config KERNEL_FHANDLE
> > > >  config KERNEL_FANOTIFY
> > > > +	default y if !SMALL_FLASH
> > > 
> > > What about `FEATURES += nas` to make it clear and don't abuse SMALL_FLASH.
> > 
> > This is not necessarily only used on NAS devices. systemd requires
> > FHANDLE and FANOTIFY (eg. when running inside LXC container), lvm2
> > needs AIO. Both could well run on a modern router or SBC having USB or
> > an SDCARD slot.
> 
> to me it's still just NAS and container use cases. I'm afraid, that adding
> more bloat to kernels for devices with USB and MMC/SD card slots would be
> rejected also.

Well, most modern routers come with USB and/or MMC/SD and Samba
installed in their stock-rom. Apart from that, a lot of useful things
can be done with network namespaces -- we are just not implementing
them because the kernel doesn't have that feature enabled.
(think: have some service connect over VPN, others directly over WAN)

> 
> > > >  config KERNEL_CGROUPS
> > > >  config KERNEL_CPUSETS
> > > >  config KERNEL_CGROUP_CPUACCT
> > > >  config KERNEL_RESOURCE_COUNTERS
> > > >  config KERNEL_MEMCG
> > > >  config KERNEL_MEMCG_KMEM
> > > >  menuconfig KERNEL_CGROUP_SCHED
> > > >   config KERNEL_FAIR_GROUP_SCHED
> > > >   config KERNEL_RT_GROUP_SCHED
> > > >  config KERNEL_NAMESPACES
> > > >  config KERNEL_LXC_MISC
> > > >  config KERNEL_SECCOMP_FILTER
> > > >  config KERNEL_SECCOMP
> > > > -		default n
> > > > +		default y if !SMALL_FLASH
> > > 
> > > What about `FEATURES += containers` ?
> > 
> > From what I understood FEATURES is supposed to reflect hardware
> > capabilities
> 
> Well, almost. I've found `squashfs` and `ext4` in there also.

True, and I don't think those two symbols make sense when talking
about FEATURES -- all devices do support squashfs without exception,
ext4 can only work on block-type storage, ie. devices booting from
(e)MMC or a SATA drive. So imho we could drop the squashfs symbol
and replace ext4 with a 'block_root' feature which imho would be
more meaningful.

> 
> >  -- all the above are generic software features useful on any device having
> >  the capacity (ie. flash and RAM) to make use of them.
> 
> But as other Daniel already suggested, !SMALL_FLASH isn't proper group
> selection either. Speaking about the capacity, did you measured how much those
> features add to the kernel images?
> 
> > > Daniel Engberg <daniel.engberg.lists@pyret.net> [2018-12-30 10:21:46]:
> > > 
> > > > however KERNEL_CGROUPS, config KERNEL_NAMESPACES, config KERNEL_LXC_MISC,
> > > > KERNEL_SECCOMP_FILTER are very limited use cases to my knowledge and more or
> > > > less only used on x86*?
> > > 
> > > There are other quite powerful platforms like mvebu, imx6, ipq etc. where you
> > > could use this as well.
> > 
> > I use LXC on oxnas/ox820 (ARM11mpcore) and ramips/mt7621 (MIPS1004Kc),
> 
> Ok, looking at IMAGE_SIZE values for mt7621 yields following results:
> 
>  IMAGE_SIZE := 6016k                         TP-LINK RE350 v1
>  IMAGE_SIZE := 7798784                       I-O DATA WN-GX300GR
>  IMAGE_SIZE := $(ralink_default_fw_size_4M)  MT7621 EVB
>  IMAGE_SIZE := $(ralink_default_fw_size_8M)  AP-MT7621A-V60 EVB
> 
> So it wouldn't be wise to add more bloat into kernels for those devices.

I don't think adding ~ 100kb to devices with 8MB of flash or more is a
problem. Regarding that evaluation board with only 4MB of NOR flash:
this is not a real-world device but rather an evalution platform, so
I wouldn't care much about users having to build their own stripped-
down version for that, because it's probably only a few hundred of
those EVBs ever made and most of them are by now collecting dust on
a shelf and will never be used for anything again.

> 
> > running Debian inside LXC containers (and it's annoying that I can't
> > use regular OpenWrt releases or buildbot-generated snapshots on the).
> 
> I would like to do the same, but on the other hand I also understand, why your
> patch as it is wouldn't be accepted. Our containers/NAS use cases are still
> very rare, thus it makes no sense to enable those features by default to every
> other !SMALL_FLASH target. As you can see on mt7621, it's not marked as
> SMALL_FLASH, yet there are devices which might be considered SMALL_FLASH.

One. The MT7621 EVB. The TP-LINK RE350 v1 can probably be converted to
a more sane flash partition layout to gain another megabyte or so.

> 
> Now I realize, that it couldn't be handled with FEATURES anyway, as this is
> always too broad group selection and ideally you want to enable those features
> for specific devices only, meaning different kernels, images etc.

Why specific devices? Wouldn't all devices with the resources (which
boils down to !SMALL_FLASH) be potentially more useful with those
kernel features enabled? And as a side-effect: the OpenWrt kernel
also becoming useful for non-OpenWrt userland...

But I'd really like to hear some more opinions on that and find a
solution which would allow using LVM2 and LXC with our release-
binaries -- containers with network-functions are becoming more
common and I know first hand that this is a reason for businesses
I have been working for to switch over to OpenEmbedded instead
of OpenWrt (and then hire people like me to port all the OpenWrt
stuff they need into their OE build...).

Anyone?


Cheers


Daniel

> 
> -- ynezz
Daniel Dickinson Jan. 7, 2019, 10:51 a.m. UTC | #5
On 2019-01-07 4:03 a.m., Daniel Golle wrote:
> On Tue, Jan 01, 2019 at 08:46:25PM +0100, Petr Štetiar wrote:
>> Daniel Golle <daniel@makrotopia.org> [2019-01-01 17:56:25]:
>>
>> Hi,
>>
>>> On Sun, Dec 30, 2018 at 11:26:58AM +0100, Petr Štetiar wrote:
>>>> Daniel Golle <daniel@makrotopia.org> [2018-12-29 06:51:32]:
>>>>
>>>>>  config KERNEL_AIO
>>>>>  config KERNEL_FHANDLE
>>>>>  config KERNEL_FANOTIFY
>>>>> +	default y if !SMALL_FLASH
>>>> What about `FEATURES += nas` to make it clear and don't abuse SMALL_FLASH.
>>> This is not necessarily only used on NAS devices. systemd requires
>>> FHANDLE and FANOTIFY (eg. when running inside LXC container), lvm2
>>> needs AIO. Both could well run on a modern router or SBC having USB or
>>> an SDCARD slot.
>> to me it's still just NAS and container use cases. I'm afraid, that adding
>> more bloat to kernels for devices with USB and MMC/SD card slots would be
>> rejected also.
> Well, most modern routers come with USB and/or MMC/SD and Samba
> installed in their stock-rom. Apart from that, a lot of useful things
> can be done with network namespaces -- we are just not implementing
> them because the kernel doesn't have that feature enabled.
> (think: have some service connect over VPN, others directly over WAN)
>
I think really small devices are rare now...and that's what SMALL_FLASH
is for (perhaps quantify SMALL would be better though).


>>>>>  config KERNEL_CGROUPS
>>>>>  config KERNEL_CPUSETS
>>>>>  config KERNEL_CGROUP_CPUACCT
>>>>>  config KERNEL_RESOURCE_COUNTERS
>>>>>  config KERNEL_MEMCG
>>>>>  config KERNEL_MEMCG_KMEM
>>>>>  menuconfig KERNEL_CGROUP_SCHED
>>>>>   config KERNEL_FAIR_GROUP_SCHED
>>>>>   config KERNEL_RT_GROUP_SCHED
>>>>>  config KERNEL_NAMESPACES
>>>>>  config KERNEL_LXC_MISC
>>>>>  config KERNEL_SECCOMP_FILTER
>>>>>  config KERNEL_SECCOMP
>>>>> -		default n
>>>>> +		default y if !SMALL_FLASH
>>>> What about `FEATURES += containers` ?
>>> From what I understood FEATURES is supposed to reflect hardware
>>> capabilities
>> Well, almost. I've found `squashfs` and `ext4` in there also.
> True, and I don't think those two symbols make sense when talking
> about FEATURES -- all devices do support squashfs without exception,
> ext4 can only work on block-type storage, ie. devices booting from
> (e)MMC or a SATA drive. So imho we could drop the squashfs symbol
> and replace ext4 with a 'block_root' feature which imho would be
> more meaningful.
+1
>>>  -- all the above are generic software features useful on any device having
>>>  the capacity (ie. flash and RAM) to make use of them.
>> But as other Daniel already suggested, !SMALL_FLASH isn't proper group
>> selection either. Speaking about the capacity, did you measured how much those
>> features add to the kernel images?
>>
>>>> Daniel Engberg <daniel.engberg.lists@pyret.net> [2018-12-30 10:21:46]:
>>>>
>>>>> however KERNEL_CGROUPS, config KERNEL_NAMESPACES, config KERNEL_LXC_MISC,
>>>>> KERNEL_SECCOMP_FILTER are very limited use cases to my knowledge and more or
>>>>> less only used on x86*?
>>>> There are other quite powerful platforms like mvebu, imx6, ipq etc. where you
>>>> could use this as well.
>>> I use LXC on oxnas/ox820 (ARM11mpcore) and ramips/mt7621 (MIPS1004Kc),
>> Ok, looking at IMAGE_SIZE values for mt7621 yields following results:
>>
>>  IMAGE_SIZE := 6016k                         TP-LINK RE350 v1
>>  IMAGE_SIZE := 7798784                       I-O DATA WN-GX300GR
>>  IMAGE_SIZE := $(ralink_default_fw_size_4M)  MT7621 EVB
>>  IMAGE_SIZE := $(ralink_default_fw_size_8M)  AP-MT7621A-V60 EVB
>>
>> So it wouldn't be wise to add more bloat into kernels for those devices.
> I don't think adding ~ 100kb to devices with 8MB of flash or more is a
> problem. Regarding that evaluation board with only 4MB of NOR flash:
> this is not a real-world device but rather an evalution platform, so
> I wouldn't care much about users having to build their own stripped-
> down version for that, because it's probably only a few hundred of
> those EVBs ever made and most of them are by now collecting dust on
> a shelf and will never be used for anything again.
+1
>>> running Debian inside LXC containers (and it's annoying that I can't
>>> use regular OpenWrt releases or buildbot-generated snapshots on the).
>> I would like to do the same, but on the other hand I also understand, why your
>> patch as it is wouldn't be accepted. Our containers/NAS use cases are still
>> very rare, thus it makes no sense to enable those features by default to every
>> other !SMALL_FLASH target. As you can see on mt7621, it's not marked as
>> SMALL_FLASH, yet there are devices which might be considered SMALL_FLASH.
> One. The MT7621 EVB. The TP-LINK RE350 v1 can probably be converted to
> a more sane flash partition layout to gain another megabyte or so.
>
>> Now I realize, that it couldn't be handled with FEATURES anyway, as this is
>> always too broad group selection and ideally you want to enable those features
>> for specific devices only, meaning different kernels, images etc.
> Why specific devices? Wouldn't all devices with the resources (which
> boils down to !SMALL_FLASH) be potentially more useful with those
> kernel features enabled? And as a side-effect: the OpenWrt kernel
> also becoming useful for non-OpenWrt userland...
>
> But I'd really like to hear some more opinions on that and find a
> solution which would allow using LVM2 and LXC with our release-
> binaries -- containers with network-functions are becoming more
> common and I know first hand that this is a reason for businesses
> I have been working for to switch over to OpenEmbedded instead
> of OpenWrt (and then hire people like me to port all the OpenWrt
> stuff they need into their OE build...).

I am definitely agreement and I can point to
https://github.com/openwrt/openwrt/pull/1713#issuecomment-451359902 (and
other comments in that PR) where there is a new contributor who
definitely echos the sentiment expressed here.

Regards,

Daniel
Petr Štetiar Jan. 7, 2019, 11 a.m. UTC | #6
Daniel Golle <daniel@makrotopia.org> [2019-01-07 10:03:09]:

Hi,

> One. The MT7621 EVB. The TP-LINK RE350 v1 can probably be converted to
> a more sane flash partition layout to gain another megabyte or so.

I've looked only at mt7621, so this was just example from one subtarget of
ramips target. So I tend to believe, that there's quite more such cases hidden
in the tree. Please correct me if I'm wrong.

> Why specific devices? Wouldn't all devices with the resources (which
> boils down to !SMALL_FLASH) be potentially more useful with those
> kernel features enabled? 

You currently can't use !SMALL_FLASH, because this is target/subtarget
specific feature, not per device feature. I think, that in order to use this
feature, you would need to convert/fix all devices like that TP-Link RE350
from all (sub)targets into tiny subtarget and then you could freely use
!SMALL_FLASH.

> I know first hand that this is a reason for businesses I have been working
> for to switch over to OpenEmbedded instead of OpenWrt (and then hire people
> like me to port all the OpenWrt stuff they need into their OE build...).

This is interesting, so instead of adding few config options and building own
images which is quite trivial task, someone is rather moving to completely
different {build,eco}system? Well, you can do that, so why not. And yes, I'm
aware about meta-openwrt OE layer.

-- ynezz
Jonas Gorski Jan. 7, 2019, 1:17 p.m. UTC | #7
On Mon, 7 Jan 2019 at 11:42, Petr Štetiar <ynezz@true.cz> wrote:
>
> Daniel Golle <daniel@makrotopia.org> [2019-01-07 10:03:09]:
>
> Hi,
>
> > One. The MT7621 EVB. The TP-LINK RE350 v1 can probably be converted to
> > a more sane flash partition layout to gain another megabyte or so.
>
> I've looked only at mt7621, so this was just example from one subtarget of
> ramips target. So I tend to believe, that there's quite more such cases hidden
> in the tree. Please correct me if I'm wrong.
>
> > Why specific devices? Wouldn't all devices with the resources (which
> > boils down to !SMALL_FLASH) be potentially more useful with those
> > kernel features enabled?
>
> You currently can't use !SMALL_FLASH, because this is target/subtarget
> specific feature, not per device feature. I think, that in order to use this
> feature, you would need to convert/fix all devices like that TP-Link RE350
> from all (sub)targets into tiny subtarget and then you could freely use
> !SMALL_FLASH.

I agree with not abusing small_flash for that. It has a clear defined
meaning, and shouldn't have unrelated side effects.

I think a new opt-in symbol for those targets with hardware
virtualization support and/or beefy enough cpus would make more sense.
Those virtualization options (probably) don't come for free, they will
have also a memory and performance impact even when not actively used.
How much that is (and if this assumption is true) would be nice to
have in the PR/patch for it.


Regards
Jonas
Daniel Golle Jan. 7, 2019, 2:20 p.m. UTC | #8
On Mon, Jan 07, 2019 at 01:17:34PM +0000, Jonas Gorski wrote:
> On Mon, 7 Jan 2019 at 11:42, Petr Štetiar <ynezz@true.cz> wrote:
> >
> > Daniel Golle <daniel@makrotopia.org> [2019-01-07 10:03:09]:
> >
> > Hi,
> >
> > > One. The MT7621 EVB. The TP-LINK RE350 v1 can probably be converted to
> > > a more sane flash partition layout to gain another megabyte or so.
> >
> > I've looked only at mt7621, so this was just example from one subtarget of
> > ramips target. So I tend to believe, that there's quite more such cases hidden
> > in the tree. Please correct me if I'm wrong.
> >
> > > Why specific devices? Wouldn't all devices with the resources (which
> > > boils down to !SMALL_FLASH) be potentially more useful with those
> > > kernel features enabled?
> >
> > You currently can't use !SMALL_FLASH, because this is target/subtarget
> > specific feature, not per device feature. I think, that in order to use this
> > feature, you would need to convert/fix all devices like that TP-Link RE350
> > from all (sub)targets into tiny subtarget and then you could freely use
> > !SMALL_FLASH.
> 
> I agree with not abusing small_flash for that. It has a clear defined
> meaning, and shouldn't have unrelated side effects.

So what else would the SMALL_FLASH symbol be used for then?
A quick grep reveals that currently already quite a few kernel config
defaults are set according to SMALL_FLASH, see

origin/master:Config-kernel.in-
origin/master:Config-kernel.in-config KERNEL_SWAP
origin/master:Config-kernel.in- bool "Support for paging of anonymous memory (swap)"
origin/master:Config-kernel.in: default y if !SMALL_FLASH
--
origin/master:Config-kernel.in-
origin/master:Config-kernel.in-config KERNEL_KALLSYMS
origin/master:Config-kernel.in- bool "Compile the kernel with symbol table information"
origin/master:Config-kernel.in: default y if !SMALL_FLASH
--
origin/master:Config-kernel.in-
origin/master:Config-kernel.in-config KERNEL_DEBUG_INFO
origin/master:Config-kernel.in- bool "Compile the kernel with debug information"
origin/master:Config-kernel.in: default y if !SMALL_FLASH
--
origin/master:Config-kernel.in-config KERNEL_ELF_CORE
origin/master:Config-kernel.in- bool "Enable process core dump support"
origin/master:Config-kernel.in- select KERNEL_COREDUMP
origin/master:Config-kernel.in: default y if !SMALL_FLASH
...

> 
> I think a new opt-in symbol for those targets with hardware
> virtualization support and/or beefy enough cpus would make more sense.
> Those virtualization options (probably) don't come for free, they will
> have also a memory and performance impact even when not actively used.
> How much that is (and if this assumption is true) would be nice to
> have in the PR/patch for it.

This is not about virtualization and none of the features selected
requires any special hardware support apart from the few extra
kilobytes of flash and memory. You are still right, it doesn't come
all for free at runtime in terms of CPU cycles, but the impact is
hardly measurable.

But sure, I understand that this can be opt-in, so lets call it
'full_kernel' or something like that and have target maintainers
decide themselves. In the picture I get after browsing through
all targets, it would still end up such that
full_kernel == !small_flash is true for all cases.


And sure, I can carry out some size measurements and compare memory
allocation after boot. I'm sure the run-time CPU impact is hardly
measurable at all.

> 
> 
> Regards
> Jonas
Jonas Gorski Jan. 7, 2019, 2:39 p.m. UTC | #9
On Mon, 7 Jan 2019 at 14:21, Daniel Golle <daniel@makrotopia.org> wrote:
>
> On Mon, Jan 07, 2019 at 01:17:34PM +0000, Jonas Gorski wrote:
> > On Mon, 7 Jan 2019 at 11:42, Petr Štetiar <ynezz@true.cz> wrote:
> > >
> > > Daniel Golle <daniel@makrotopia.org> [2019-01-07 10:03:09]:
> > >
> > > Hi,
> > >
> > > > One. The MT7621 EVB. The TP-LINK RE350 v1 can probably be converted to
> > > > a more sane flash partition layout to gain another megabyte or so.
> > >
> > > I've looked only at mt7621, so this was just example from one subtarget of
> > > ramips target. So I tend to believe, that there's quite more such cases hidden
> > > in the tree. Please correct me if I'm wrong.
> > >
> > > > Why specific devices? Wouldn't all devices with the resources (which
> > > > boils down to !SMALL_FLASH) be potentially more useful with those
> > > > kernel features enabled?
> > >
> > > You currently can't use !SMALL_FLASH, because this is target/subtarget
> > > specific feature, not per device feature. I think, that in order to use this
> > > feature, you would need to convert/fix all devices like that TP-Link RE350
> > > from all (sub)targets into tiny subtarget and then you could freely use
> > > !SMALL_FLASH.
> >
> > I agree with not abusing small_flash for that. It has a clear defined
> > meaning, and shouldn't have unrelated side effects.
>
> So what else would the SMALL_FLASH symbol be used for then?
> A quick grep reveals that currently already quite a few kernel config
> defaults are set according to SMALL_FLASH, see
>
> origin/master:Config-kernel.in-
> origin/master:Config-kernel.in-config KERNEL_SWAP
> origin/master:Config-kernel.in- bool "Support for paging of anonymous memory (swap)"
> origin/master:Config-kernel.in: default y if !SMALL_FLASH
> --
> origin/master:Config-kernel.in-
> origin/master:Config-kernel.in-config KERNEL_KALLSYMS
> origin/master:Config-kernel.in- bool "Compile the kernel with symbol table information"
> origin/master:Config-kernel.in: default y if !SMALL_FLASH
> --
> origin/master:Config-kernel.in-
> origin/master:Config-kernel.in-config KERNEL_DEBUG_INFO
> origin/master:Config-kernel.in- bool "Compile the kernel with debug information"
> origin/master:Config-kernel.in: default y if !SMALL_FLASH
> --
> origin/master:Config-kernel.in-config KERNEL_ELF_CORE
> origin/master:Config-kernel.in- bool "Enable process core dump support"
> origin/master:Config-kernel.in- select KERNEL_COREDUMP
> origin/master:Config-kernel.in: default y if !SMALL_FLASH
> ...

Most of these option only influence the size of the kernel, and have
no further runtime side effects. Also small_flash has impact on the
compression options used.

>
> >
> > I think a new opt-in symbol for those targets with hardware
> > virtualization support and/or beefy enough cpus would make more sense.
> > Those virtualization options (probably) don't come for free, they will
> > have also a memory and performance impact even when not actively used.
> > How much that is (and if this assumption is true) would be nice to
> > have in the PR/patch for it.
>
> This is not about virtualization and none of the features selected
> requires any special hardware support apart from the few extra
> kilobytes of flash and memory. You are still right, it doesn't come
> all for free at runtime in terms of CPU cycles, but the impact is
> hardly measurable.
>
> But sure, I understand that this can be opt-in, so lets call it
> 'full_kernel' or something like that and have target maintainers
> decide themselves. In the picture I get after browsing through
> all targets, it would still end up such that
> full_kernel == !small_flash is true for all cases.

"Full kernel" really has no real meaning and would describe
everything. The name should clearly describe the (non-default) feature
set it enables.

> And sure, I can carry out some size measurements and compare memory
> allocation after boot. I'm sure the run-time CPU impact is hardly
> measurable at all.

Please do so especially on low end devices. Maybe something like
routing throughput with/without.


Regards
Jonas
Daniel Golle Jan. 7, 2019, 2:48 p.m. UTC | #10
On Mon, Jan 07, 2019 at 02:39:26PM +0000, Jonas Gorski wrote:
> On Mon, 7 Jan 2019 at 14:21, Daniel Golle <daniel@makrotopia.org> wrote:
> >
> > On Mon, Jan 07, 2019 at 01:17:34PM +0000, Jonas Gorski wrote:
> > > On Mon, 7 Jan 2019 at 11:42, Petr Štetiar <ynezz@true.cz> wrote:
> > > >
> > > > Daniel Golle <daniel@makrotopia.org> [2019-01-07 10:03:09]:
> > > >
> > > > Hi,
> > > >
> > > > > One. The MT7621 EVB. The TP-LINK RE350 v1 can probably be converted to
> > > > > a more sane flash partition layout to gain another megabyte or so.
> > > >
> > > > I've looked only at mt7621, so this was just example from one subtarget of
> > > > ramips target. So I tend to believe, that there's quite more such cases hidden
> > > > in the tree. Please correct me if I'm wrong.
> > > >
> > > > > Why specific devices? Wouldn't all devices with the resources (which
> > > > > boils down to !SMALL_FLASH) be potentially more useful with those
> > > > > kernel features enabled?
> > > >
> > > > You currently can't use !SMALL_FLASH, because this is target/subtarget
> > > > specific feature, not per device feature. I think, that in order to use this
> > > > feature, you would need to convert/fix all devices like that TP-Link RE350
> > > > from all (sub)targets into tiny subtarget and then you could freely use
> > > > !SMALL_FLASH.
> > >
> > > I agree with not abusing small_flash for that. It has a clear defined
> > > meaning, and shouldn't have unrelated side effects.
> >
> > So what else would the SMALL_FLASH symbol be used for then?
> > A quick grep reveals that currently already quite a few kernel config
> > defaults are set according to SMALL_FLASH, see
> >
> > origin/master:Config-kernel.in-
> > origin/master:Config-kernel.in-config KERNEL_SWAP
> > origin/master:Config-kernel.in- bool "Support for paging of anonymous memory (swap)"
> > origin/master:Config-kernel.in: default y if !SMALL_FLASH
> > --
> > origin/master:Config-kernel.in-
> > origin/master:Config-kernel.in-config KERNEL_KALLSYMS
> > origin/master:Config-kernel.in- bool "Compile the kernel with symbol table information"
> > origin/master:Config-kernel.in: default y if !SMALL_FLASH
> > --
> > origin/master:Config-kernel.in-
> > origin/master:Config-kernel.in-config KERNEL_DEBUG_INFO
> > origin/master:Config-kernel.in- bool "Compile the kernel with debug information"
> > origin/master:Config-kernel.in: default y if !SMALL_FLASH
> > --
> > origin/master:Config-kernel.in-config KERNEL_ELF_CORE
> > origin/master:Config-kernel.in- bool "Enable process core dump support"
> > origin/master:Config-kernel.in- select KERNEL_COREDUMP
> > origin/master:Config-kernel.in: default y if !SMALL_FLASH
> > ...
> 
> Most of these option only influence the size of the kernel, and have
> no further runtime side effects. Also small_flash has impact on the
> compression options used.

They sure do, cache size on small CPUs is a very finite resource
and having a kernel with debug symbols will make things slower, of
course. SWAP also makes every single malloc call more expensive and
is just as well only useful on devices with block storage (ok, and
zramswap, but lets not talk about that).

> 
> >
> > >
> > > I think a new opt-in symbol for those targets with hardware
> > > virtualization support and/or beefy enough cpus would make more sense.
> > > Those virtualization options (probably) don't come for free, they will
> > > have also a memory and performance impact even when not actively used.
> > > How much that is (and if this assumption is true) would be nice to
> > > have in the PR/patch for it.
> >
> > This is not about virtualization and none of the features selected
> > requires any special hardware support apart from the few extra
> > kilobytes of flash and memory. You are still right, it doesn't come
> > all for free at runtime in terms of CPU cycles, but the impact is
> > hardly measurable.
> >
> > But sure, I understand that this can be opt-in, so lets call it
> > 'full_kernel' or something like that and have target maintainers
> > decide themselves. In the picture I get after browsing through
> > all targets, it would still end up such that
> > full_kernel == !small_flash is true for all cases.
> 
> "Full kernel" really has no real meaning and would describe
> everything. The name should clearly describe the (non-default) feature
> set it enables.

But they are not even necessarily related, just closer to the vanilla
default config which is used eg. by Debian and most other Linux distros
so projects like LVM2 started to rely on them.
My goal here is to bring modern generic kernel features into OpenWrt,
they are quite unrelated apart from being left out for space reasons
and because for a minimal router/AP they are unneeded.

> 
> > And sure, I can carry out some size measurements and compare memory
> > allocation after boot. I'm sure the run-time CPU impact is hardly
> > measurable at all.
> 
> Please do so especially on low end devices. Maybe something like
> routing throughput with/without.

I'll do within the next couple of days and post the results.


Cheers


Daniel

> 
> 
> Regards
> Jonas
> 
> _______________________________________________
> openwrt-devel mailing list
> openwrt-devel@lists.openwrt.org
> https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Daniel Golle Jan. 15, 2019, 2:42 p.m. UTC | #11
Hi Jonas,

On Mon, Jan 07, 2019 at 03:48:29PM +0100, Daniel Golle wrote:
> On Mon, Jan 07, 2019 at 02:39:26PM +0000, Jonas Gorski wrote:
> > On Mon, 7 Jan 2019 at 14:21, Daniel Golle <daniel@makrotopia.org> wrote:
> > >
> > > On Mon, Jan 07, 2019 at 01:17:34PM +0000, Jonas Gorski wrote:
> > > > On Mon, 7 Jan 2019 at 11:42, Petr Štetiar <ynezz@true.cz> wrote:
> > > > >
> > > > > Daniel Golle <daniel@makrotopia.org> [2019-01-07 10:03:09]:
> > > > >
> > > > > Hi,
> > > > >
> > > > > > One. The MT7621 EVB. The TP-LINK RE350 v1 can probably be converted to
> > > > > > a more sane flash partition layout to gain another megabyte or so.
> > > > >
> > > > > I've looked only at mt7621, so this was just example from one subtarget of
> > > > > ramips target. So I tend to believe, that there's quite more such cases hidden
> > > > > in the tree. Please correct me if I'm wrong.
> > > > >
> > > > > > Why specific devices? Wouldn't all devices with the resources (which
> > > > > > boils down to !SMALL_FLASH) be potentially more useful with those
> > > > > > kernel features enabled?
> > > > >
> > > > > You currently can't use !SMALL_FLASH, because this is target/subtarget
> > > > > specific feature, not per device feature. I think, that in order to use this
> > > > > feature, you would need to convert/fix all devices like that TP-Link RE350
> > > > > from all (sub)targets into tiny subtarget and then you could freely use
> > > > > !SMALL_FLASH.
> > > >
> > > > I agree with not abusing small_flash for that. It has a clear defined
> > > > meaning, and shouldn't have unrelated side effects.
> > >
> > > So what else would the SMALL_FLASH symbol be used for then?
> > > A quick grep reveals that currently already quite a few kernel config
> > > defaults are set according to SMALL_FLASH, see
> > >
> > > origin/master:Config-kernel.in-
> > > origin/master:Config-kernel.in-config KERNEL_SWAP
> > > origin/master:Config-kernel.in- bool "Support for paging of anonymous memory (swap)"
> > > origin/master:Config-kernel.in: default y if !SMALL_FLASH
> > > --
> > > origin/master:Config-kernel.in-
> > > origin/master:Config-kernel.in-config KERNEL_KALLSYMS
> > > origin/master:Config-kernel.in- bool "Compile the kernel with symbol table information"
> > > origin/master:Config-kernel.in: default y if !SMALL_FLASH
> > > --
> > > origin/master:Config-kernel.in-
> > > origin/master:Config-kernel.in-config KERNEL_DEBUG_INFO
> > > origin/master:Config-kernel.in- bool "Compile the kernel with debug information"
> > > origin/master:Config-kernel.in: default y if !SMALL_FLASH
> > > --
> > > origin/master:Config-kernel.in-config KERNEL_ELF_CORE
> > > origin/master:Config-kernel.in- bool "Enable process core dump support"
> > > origin/master:Config-kernel.in- select KERNEL_COREDUMP
> > > origin/master:Config-kernel.in: default y if !SMALL_FLASH
> > > ...
> > 
> > Most of these option only influence the size of the kernel, and have
> > no further runtime side effects. Also small_flash has impact on the
> > compression options used.
> 
> They sure do, cache size on small CPUs is a very finite resource
> and having a kernel with debug symbols will make things slower, of
> course. SWAP also makes every single malloc call more expensive and
> is just as well only useful on devices with block storage (ok, and
> zramswap, but lets not talk about that).
> 
> > 
> > >
> > > >
> > > > I think a new opt-in symbol for those targets with hardware
> > > > virtualization support and/or beefy enough cpus would make more sense.
> > > > Those virtualization options (probably) don't come for free, they will
> > > > have also a memory and performance impact even when not actively used.
> > > > How much that is (and if this assumption is true) would be nice to
> > > > have in the PR/patch for it.
> > >
> > > This is not about virtualization and none of the features selected
> > > requires any special hardware support apart from the few extra
> > > kilobytes of flash and memory. You are still right, it doesn't come
> > > all for free at runtime in terms of CPU cycles, but the impact is
> > > hardly measurable.
> > >
> > > But sure, I understand that this can be opt-in, so lets call it
> > > 'full_kernel' or something like that and have target maintainers
> > > decide themselves. In the picture I get after browsing through
> > > all targets, it would still end up such that
> > > full_kernel == !small_flash is true for all cases.
> > 
> > "Full kernel" really has no real meaning and would describe
> > everything. The name should clearly describe the (non-default) feature
> > set it enables.
> 
> But they are not even necessarily related, just closer to the vanilla
> default config which is used eg. by Debian and most other Linux distros
> so projects like LVM2 started to rely on them.
> My goal here is to bring modern generic kernel features into OpenWrt,
> they are quite unrelated apart from being left out for space reasons
> and because for a minimal router/AP they are unneeded.
> 
> > 
> > > And sure, I can carry out some size measurements and compare memory
> > > allocation after boot. I'm sure the run-time CPU impact is hardly
> > > measurable at all.
> > 
> > Please do so especially on low end devices. Maybe something like
> > routing throughput with/without.
> 
> I'll do within the next couple of days and post the results.

Kernel size increases by about 90kB (lzma compressed).
Memory consumption just after boot (ironically) seems to be lower with
those options enabled, however, it's all in the range of +/- 100kB.
Network performance seems to be exactly identical.

(tested on ar71xx based on latest master commit)


Cheers


Daniel
Jo-Philipp Wich Jan. 16, 2019, 6:54 a.m. UTC | #12
Hi,

I lean towards enabling these features by default.

With the eventual switch to 4.19, most 4M boards will fall of the cliff
anyway.


Regards,
Jo
Jonas Gorski Feb. 6, 2019, 12:12 p.m. UTC | #13
On Tue, 15 Jan 2019 at 15:42, Daniel Golle <daniel@makrotopia.org> wrote:
>
> Hi Jonas,
>
> On Mon, Jan 07, 2019 at 03:48:29PM +0100, Daniel Golle wrote:
> > On Mon, Jan 07, 2019 at 02:39:26PM +0000, Jonas Gorski wrote:
> > > On Mon, 7 Jan 2019 at 14:21, Daniel Golle <daniel@makrotopia.org> wrote:
> > > >
> > > > On Mon, Jan 07, 2019 at 01:17:34PM +0000, Jonas Gorski wrote:
> > > > > On Mon, 7 Jan 2019 at 11:42, Petr Štetiar <ynezz@true.cz> wrote:
> > > > > >
> > > > > > Daniel Golle <daniel@makrotopia.org> [2019-01-07 10:03:09]:
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > > One. The MT7621 EVB. The TP-LINK RE350 v1 can probably be converted to
> > > > > > > a more sane flash partition layout to gain another megabyte or so.
> > > > > >
> > > > > > I've looked only at mt7621, so this was just example from one subtarget of
> > > > > > ramips target. So I tend to believe, that there's quite more such cases hidden
> > > > > > in the tree. Please correct me if I'm wrong.
> > > > > >
> > > > > > > Why specific devices? Wouldn't all devices with the resources (which
> > > > > > > boils down to !SMALL_FLASH) be potentially more useful with those
> > > > > > > kernel features enabled?
> > > > > >
> > > > > > You currently can't use !SMALL_FLASH, because this is target/subtarget
> > > > > > specific feature, not per device feature. I think, that in order to use this
> > > > > > feature, you would need to convert/fix all devices like that TP-Link RE350
> > > > > > from all (sub)targets into tiny subtarget and then you could freely use
> > > > > > !SMALL_FLASH.
> > > > >
> > > > > I agree with not abusing small_flash for that. It has a clear defined
> > > > > meaning, and shouldn't have unrelated side effects.
> > > >
> > > > So what else would the SMALL_FLASH symbol be used for then?
> > > > A quick grep reveals that currently already quite a few kernel config
> > > > defaults are set according to SMALL_FLASH, see
> > > >
> > > > origin/master:Config-kernel.in-
> > > > origin/master:Config-kernel.in-config KERNEL_SWAP
> > > > origin/master:Config-kernel.in- bool "Support for paging of anonymous memory (swap)"
> > > > origin/master:Config-kernel.in: default y if !SMALL_FLASH
> > > > --
> > > > origin/master:Config-kernel.in-
> > > > origin/master:Config-kernel.in-config KERNEL_KALLSYMS
> > > > origin/master:Config-kernel.in- bool "Compile the kernel with symbol table information"
> > > > origin/master:Config-kernel.in: default y if !SMALL_FLASH
> > > > --
> > > > origin/master:Config-kernel.in-
> > > > origin/master:Config-kernel.in-config KERNEL_DEBUG_INFO
> > > > origin/master:Config-kernel.in- bool "Compile the kernel with debug information"
> > > > origin/master:Config-kernel.in: default y if !SMALL_FLASH
> > > > --
> > > > origin/master:Config-kernel.in-config KERNEL_ELF_CORE
> > > > origin/master:Config-kernel.in- bool "Enable process core dump support"
> > > > origin/master:Config-kernel.in- select KERNEL_COREDUMP
> > > > origin/master:Config-kernel.in: default y if !SMALL_FLASH
> > > > ...
> > >
> > > Most of these option only influence the size of the kernel, and have
> > > no further runtime side effects. Also small_flash has impact on the
> > > compression options used.
> >
> > They sure do, cache size on small CPUs is a very finite resource
> > and having a kernel with debug symbols will make things slower, of
> > course. SWAP also makes every single malloc call more expensive and
> > is just as well only useful on devices with block storage (ok, and
> > zramswap, but lets not talk about that).
> >
> > >
> > > >
> > > > >
> > > > > I think a new opt-in symbol for those targets with hardware
> > > > > virtualization support and/or beefy enough cpus would make more sense.
> > > > > Those virtualization options (probably) don't come for free, they will
> > > > > have also a memory and performance impact even when not actively used.
> > > > > How much that is (and if this assumption is true) would be nice to
> > > > > have in the PR/patch for it.
> > > >
> > > > This is not about virtualization and none of the features selected
> > > > requires any special hardware support apart from the few extra
> > > > kilobytes of flash and memory. You are still right, it doesn't come
> > > > all for free at runtime in terms of CPU cycles, but the impact is
> > > > hardly measurable.
> > > >
> > > > But sure, I understand that this can be opt-in, so lets call it
> > > > 'full_kernel' or something like that and have target maintainers
> > > > decide themselves. In the picture I get after browsing through
> > > > all targets, it would still end up such that
> > > > full_kernel == !small_flash is true for all cases.
> > >
> > > "Full kernel" really has no real meaning and would describe
> > > everything. The name should clearly describe the (non-default) feature
> > > set it enables.
> >
> > But they are not even necessarily related, just closer to the vanilla
> > default config which is used eg. by Debian and most other Linux distros
> > so projects like LVM2 started to rely on them.
> > My goal here is to bring modern generic kernel features into OpenWrt,
> > they are quite unrelated apart from being left out for space reasons
> > and because for a minimal router/AP they are unneeded.
> >
> > >
> > > > And sure, I can carry out some size measurements and compare memory
> > > > allocation after boot. I'm sure the run-time CPU impact is hardly
> > > > measurable at all.
> > >
> > > Please do so especially on low end devices. Maybe something like
> > > routing throughput with/without.
> >
> > I'll do within the next couple of days and post the results.
>
> Kernel size increases by about 90kB (lzma compressed).
> Memory consumption just after boot (ironically) seems to be lower with
> those options enabled, however, it's all in the range of +/- 100kB.
> Network performance seems to be exactly identical.
>
> (tested on ar71xx based on latest master commit)

I gave it my own test with a bcm6328 (320 MHz), and did see some
variance in network performance, but only a few percent. Might have
been differences from code changing alignments to cache lines or so.

I did see a more pronounced memory change though, in the range of ~500
kB less free memory. Might be relevant for 32 MiB devices.

Since growing kernels breaking boards with u-boot not loading enough
from flash is a common enough occurrence, I'd vote for deferring this
change until we branched for the upcoming release, to not introduce
additional board breakage shortly before that. After that we'll get
enough breakage there from the 4.19 switch anyway, so we might then
also up the official minimum requirement to 8 MiB flash.

Regards
Jonas
Jonas Gorski June 12, 2019, 12:46 p.m. UTC | #14
On Wed, 6 Feb 2019 at 13:12, Jonas Gorski <jonas.gorski@gmail.com> wrote:
>
> On Tue, 15 Jan 2019 at 15:42, Daniel Golle <daniel@makrotopia.org> wrote:
> >
> > Hi Jonas,
> >
> > On Mon, Jan 07, 2019 at 03:48:29PM +0100, Daniel Golle wrote:
> > > On Mon, Jan 07, 2019 at 02:39:26PM +0000, Jonas Gorski wrote:
> > > > On Mon, 7 Jan 2019 at 14:21, Daniel Golle <daniel@makrotopia.org> wrote:
> > > > >
> > > > > On Mon, Jan 07, 2019 at 01:17:34PM +0000, Jonas Gorski wrote:
> > > > > > On Mon, 7 Jan 2019 at 11:42, Petr Štetiar <ynezz@true.cz> wrote:
> > > > > > >
> > > > > > > Daniel Golle <daniel@makrotopia.org> [2019-01-07 10:03:09]:
> > > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > > One. The MT7621 EVB. The TP-LINK RE350 v1 can probably be converted to
> > > > > > > > a more sane flash partition layout to gain another megabyte or so.
> > > > > > >
> > > > > > > I've looked only at mt7621, so this was just example from one subtarget of
> > > > > > > ramips target. So I tend to believe, that there's quite more such cases hidden
> > > > > > > in the tree. Please correct me if I'm wrong.
> > > > > > >
> > > > > > > > Why specific devices? Wouldn't all devices with the resources (which
> > > > > > > > boils down to !SMALL_FLASH) be potentially more useful with those
> > > > > > > > kernel features enabled?
> > > > > > >
> > > > > > > You currently can't use !SMALL_FLASH, because this is target/subtarget
> > > > > > > specific feature, not per device feature. I think, that in order to use this
> > > > > > > feature, you would need to convert/fix all devices like that TP-Link RE350
> > > > > > > from all (sub)targets into tiny subtarget and then you could freely use
> > > > > > > !SMALL_FLASH.
> > > > > >
> > > > > > I agree with not abusing small_flash for that. It has a clear defined
> > > > > > meaning, and shouldn't have unrelated side effects.
> > > > >
> > > > > So what else would the SMALL_FLASH symbol be used for then?
> > > > > A quick grep reveals that currently already quite a few kernel config
> > > > > defaults are set according to SMALL_FLASH, see
> > > > >
> > > > > origin/master:Config-kernel.in-
> > > > > origin/master:Config-kernel.in-config KERNEL_SWAP
> > > > > origin/master:Config-kernel.in- bool "Support for paging of anonymous memory (swap)"
> > > > > origin/master:Config-kernel.in: default y if !SMALL_FLASH
> > > > > --
> > > > > origin/master:Config-kernel.in-
> > > > > origin/master:Config-kernel.in-config KERNEL_KALLSYMS
> > > > > origin/master:Config-kernel.in- bool "Compile the kernel with symbol table information"
> > > > > origin/master:Config-kernel.in: default y if !SMALL_FLASH
> > > > > --
> > > > > origin/master:Config-kernel.in-
> > > > > origin/master:Config-kernel.in-config KERNEL_DEBUG_INFO
> > > > > origin/master:Config-kernel.in- bool "Compile the kernel with debug information"
> > > > > origin/master:Config-kernel.in: default y if !SMALL_FLASH
> > > > > --
> > > > > origin/master:Config-kernel.in-config KERNEL_ELF_CORE
> > > > > origin/master:Config-kernel.in- bool "Enable process core dump support"
> > > > > origin/master:Config-kernel.in- select KERNEL_COREDUMP
> > > > > origin/master:Config-kernel.in: default y if !SMALL_FLASH
> > > > > ...
> > > >
> > > > Most of these option only influence the size of the kernel, and have
> > > > no further runtime side effects. Also small_flash has impact on the
> > > > compression options used.
> > >
> > > They sure do, cache size on small CPUs is a very finite resource
> > > and having a kernel with debug symbols will make things slower, of
> > > course. SWAP also makes every single malloc call more expensive and
> > > is just as well only useful on devices with block storage (ok, and
> > > zramswap, but lets not talk about that).
> > >
> > > >
> > > > >
> > > > > >
> > > > > > I think a new opt-in symbol for those targets with hardware
> > > > > > virtualization support and/or beefy enough cpus would make more sense.
> > > > > > Those virtualization options (probably) don't come for free, they will
> > > > > > have also a memory and performance impact even when not actively used.
> > > > > > How much that is (and if this assumption is true) would be nice to
> > > > > > have in the PR/patch for it.
> > > > >
> > > > > This is not about virtualization and none of the features selected
> > > > > requires any special hardware support apart from the few extra
> > > > > kilobytes of flash and memory. You are still right, it doesn't come
> > > > > all for free at runtime in terms of CPU cycles, but the impact is
> > > > > hardly measurable.
> > > > >
> > > > > But sure, I understand that this can be opt-in, so lets call it
> > > > > 'full_kernel' or something like that and have target maintainers
> > > > > decide themselves. In the picture I get after browsing through
> > > > > all targets, it would still end up such that
> > > > > full_kernel == !small_flash is true for all cases.
> > > >
> > > > "Full kernel" really has no real meaning and would describe
> > > > everything. The name should clearly describe the (non-default) feature
> > > > set it enables.
> > >
> > > But they are not even necessarily related, just closer to the vanilla
> > > default config which is used eg. by Debian and most other Linux distros
> > > so projects like LVM2 started to rely on them.
> > > My goal here is to bring modern generic kernel features into OpenWrt,
> > > they are quite unrelated apart from being left out for space reasons
> > > and because for a minimal router/AP they are unneeded.
> > >
> > > >
> > > > > And sure, I can carry out some size measurements and compare memory
> > > > > allocation after boot. I'm sure the run-time CPU impact is hardly
> > > > > measurable at all.
> > > >
> > > > Please do so especially on low end devices. Maybe something like
> > > > routing throughput with/without.
> > >
> > > I'll do within the next couple of days and post the results.
> >
> > Kernel size increases by about 90kB (lzma compressed).
> > Memory consumption just after boot (ironically) seems to be lower with
> > those options enabled, however, it's all in the range of +/- 100kB.
> > Network performance seems to be exactly identical.
> >
> > (tested on ar71xx based on latest master commit)
>
> I gave it my own test with a bcm6328 (320 MHz), and did see some
> variance in network performance, but only a few percent. Might have
> been differences from code changing alignments to cache lines or so.
>
> I did see a more pronounced memory change though, in the range of ~500
> kB less free memory. Might be relevant for 32 MiB devices.
>
> Since growing kernels breaking boards with u-boot not loading enough
> from flash is a common enough occurrence, I'd vote for deferring this
> change until we branched for the upcoming release, to not introduce
> additional board breakage shortly before that. After that we'll get
> enough breakage there from the 4.19 switch anyway, so we might then
> also up the official minimum requirement to 8 MiB flash.

Since 19.07 is now branched, I have no objections anymore to adding this patch.


Regards
Jonas
diff mbox series

Patch

diff --git a/config/Config-kernel.in b/config/Config-kernel.in
index f38cc792dd..f54a019267 100644
--- a/config/Config-kernel.in
+++ b/config/Config-kernel.in
@@ -194,15 +194,15 @@  config KERNEL_KPROBE_EVENT
 
 config KERNEL_AIO
 	bool "Compile the kernel with asynchronous IO support"
-	default n
+	default y if !SMALL_FLASH
 
 config KERNEL_FHANDLE
 	bool "Compile the kernel with support for fhandle syscalls"
-	default n
+	default y if !SMALL_FLASH
 
 config KERNEL_FANOTIFY
 	bool "Compile the kernel with modern file notification support"
-	default n
+	default y if !SMALL_FLASH
 
 config KERNEL_BLK_DEV_BSG
 	bool "Compile the kernel with SCSI generic v4 support for any block device"
@@ -316,7 +316,7 @@  config KERNEL_ENCRYPTED_KEYS
 
 config KERNEL_CGROUPS
 	bool "Enable kernel cgroups"
-	default n
+	default y if !SMALL_FLASH
 
 if KERNEL_CGROUPS
 
@@ -355,7 +355,7 @@  if KERNEL_CGROUPS
 
 	config KERNEL_CPUSETS
 		bool "Cpuset support"
-		default n
+		default y if !SMALL_FLASH
 		help
 		  This option will let you create and manage CPUSETs which
 		  allow dynamically partitioning a system into sets of CPUs and
@@ -369,14 +369,14 @@  if KERNEL_CGROUPS
 
 	config KERNEL_CGROUP_CPUACCT
 		bool "Simple CPU accounting cgroup subsystem"
-		default n
+		default y if !SMALL_FLASH
 		help
 		  Provides a simple Resource Controller for monitoring the
 		  total CPU consumed by the tasks in a cgroup.
 
 	config KERNEL_RESOURCE_COUNTERS
 		bool "Resource counters"
-		default n
+		default y if !SMALL_FLASH
 		help
 		  This option enables controller independent resource accounting
 		  infrastructure that works with cgroups.
@@ -387,7 +387,7 @@  if KERNEL_CGROUPS
 
 	config KERNEL_MEMCG
 		bool "Memory Resource Controller for Control Groups"
-		default n
+		default y if !SMALL_FLASH
 		depends on KERNEL_RESOURCE_COUNTERS || !LINUX_3_18
 		help
 		  Provides a memory resource controller that manages both anonymous
@@ -445,7 +445,7 @@  if KERNEL_CGROUPS
 
 	config KERNEL_MEMCG_KMEM
 		bool "Memory Resource Controller Kernel Memory accounting (EXPERIMENTAL)"
-		default n
+		default y if !SMALL_FLASH
 		depends on KERNEL_MEMCG
 		help
 		  The Kernel Memory extension for Memory Resource Controller can limit
@@ -466,7 +466,7 @@  if KERNEL_CGROUPS
 
 	menuconfig KERNEL_CGROUP_SCHED
 		bool "Group CPU scheduler"
-		default n
+		default y if !SMALL_FLASH
 		help
 		  This feature lets CPU scheduler recognize task groups and control CPU
 		  bandwidth allocation to such task groups. It uses cgroups to group
@@ -476,7 +476,7 @@  if KERNEL_CGROUPS
 
 		config KERNEL_FAIR_GROUP_SCHED
 			bool "Group scheduling for SCHED_OTHER"
-			default n
+			default y if !SMALL_FLASH
 
 		config KERNEL_CFS_BANDWIDTH
 			bool "CPU bandwidth provisioning for FAIR_GROUP_SCHED"
@@ -491,7 +491,7 @@  if KERNEL_CGROUPS
 
 		config KERNEL_RT_GROUP_SCHED
 			bool "Group scheduling for SCHED_RR/FIFO"
-			default n
+			default y if !SMALL_FLASH
 			help
 			  This feature lets you explicitly allocate real CPU bandwidth
 			  to task groups. If enabled, it will also make it impossible to
@@ -543,7 +543,7 @@  endif
 
 config KERNEL_NAMESPACES
 	bool "Enable kernel namespaces"
-	default n
+	default y if !SMALL_FLASH
 
 if KERNEL_NAMESPACES
 
@@ -591,7 +591,7 @@  endif
 
 config KERNEL_LXC_MISC
 	bool "Enable miscellaneous LXC related options"
-	default n
+	default y if !SMALL_FLASH
 
 if KERNEL_LXC_MISC
 
@@ -623,13 +623,13 @@  endif
 
 config KERNEL_SECCOMP_FILTER
 	bool
-	default n
+	default y if !SMALL_FLASH
 
 config KERNEL_SECCOMP
 	bool "Enable seccomp support"
 		depends on !(TARGET_uml)
 		select KERNEL_SECCOMP_FILTER
-		default n
+		default y if !SMALL_FLASH
 		help
 		  Build kernel with support for seccomp.