| Message ID | be244a81b5835217795cec0a4886e0958657c1b9.1682539911.git.ehem+openwrt@m5p.com |
|---|---|
| State | New |
| Headers | show |
| Series | (mostly) x86 kernel configuration adjustments | expand |
Hi On 2023-03-30, Elliott Mitchell wrote: > Full amd64 support isn't really appropriate for most situations > OpenWRT is deployed. Whereas x86-x32 seems extremely appropriate for > these situations. As such enable x86-x32 support. > > CONFIG_ARCH_MMAP_RND_COMPAT_BITS is required to follow along, > otherwise the kernel build breaks. > > Signed-off-by: Elliott Mitchell <ehem+openwrt@m5p.com> > --- > I suggest OpenWRT should be placing quite a bit of effort towards > x86-x32. x86-x32 seems a rather superior generic target for OpenWRT. > Only issue is it could be valuable to have at least minimal amd64 > userland support alongside the x86-x32 version. x86_32 is pretty much dead in the water, with almost zero deployment by general purpose distributions - apart from VM data centre environments doing their own thing (least amount of RAM usage possible, everything else being secondary at best). At least Debian did raise security concerns about the x86_32 ISA in the past. While I might understand (understand, not support) a desire for this as a dedicated subtarget (to appease the virtualization crowd), although I still don't see a reason or sufficient uptake in more conventional Linux environments. I would not be happy (at all) to lose 'normal' x86_64 support (on real hardware) for this exotic fringe hybrid. I can imagine that actually building for this environment (with a 32 bit userland) might lead to 'funny' results as well (as in major toolchain changes necessary to get it working as expected). Regards Stefan Lippers-Hollmann
On Thu, Apr 27, 2023 at 12:46:49AM +0200, Stefan Lippers-Hollmann wrote: > > On 2023-03-30, Elliott Mitchell wrote: > > Full amd64 support isn't really appropriate for most situations > > OpenWRT is deployed. Whereas x86-x32 seems extremely appropriate for > > these situations. As such enable x86-x32 support. > > > > CONFIG_ARCH_MMAP_RND_COMPAT_BITS is required to follow along, > > otherwise the kernel build breaks. > > > > Signed-off-by: Elliott Mitchell <ehem+openwrt@m5p.com> > > --- > > I suggest OpenWRT should be placing quite a bit of effort towards > > x86-x32. x86-x32 seems a rather superior generic target for OpenWRT. > > Only issue is it could be valuable to have at least minimal amd64 > > userland support alongside the x86-x32 version. > > x86_32 is pretty much dead in the water, with almost zero deployment > by general purpose distributions - apart from VM data centre > environments doing their own thing (least amount of RAM usage > possible, everything else being secondary at best). At least Debian > did raise security concerns about the x86_32 ISA in the past. Error: undefined symbol "x86_32" Without the extra context I would resolve that to "i386"/"ia32". I think "x32" or "x86_x32" are better strings for this case. According to what I had read that was in the past when x32 was sharing less of the i386 ABI. Notably x32 had been trying to pass time values in a distinct fashion. I will conceed I'm unsure whether x32 is ever truly going to get a seat at the table. On a different news front, Linux 5.10 has an option CONFIG_X86_X32_DISABLED which leaves x32 disabled by default (Debian's kernels were configured this way). Whereas 5.15 has removed the CONFIG_X86_X32_DISABLED option which seems to suggest the concerns may have been assuaged. > While I might understand (understand, not support) a desire for this > as a dedicated subtarget (to appease the virtualization crowd), > although I still don't see a reason or sufficient uptake in more > conventional Linux environments. I would not be happy (at all) to > lose 'normal' x86_64 support (on real hardware) for this exotic > fringe hybrid. I can imagine that actually building for this > environment (with a 32 bit userland) might lead to 'funny' results > as well (as in major toolchain changes necessary to get it working > as expected). I'm not proposing removing amd64 support, I'm proposing x32 is likely a more valuable target. Yet what you're describing reads like your desire is for OpenWRT/x86 to simply be yet another desktop Linux distribution. Unless you feel a networking device really needs 256GB of memory, virtual machines are precisely what OpenWRT/x86 *should* target. I think it is reasonable to also have a jumbo/desktop build, but using an entire x86 machine doesn't seem to match OpenWRT's main theme.
> Le 27 avr. 2023 à 02:00, Elliott Mitchell <ehem+openwrt@m5p.com> a écrit : > > On Thu, Apr 27, 2023 at 12:46:49AM +0200, Stefan Lippers-Hollmann wrote: > >> While I might understand (understand, not support) a desire for this >> as a dedicated subtarget (to appease the virtualization crowd), >> although I still don't see a reason or sufficient uptake in more >> conventional Linux environments. I would not be happy (at all) to >> lose 'normal' x86_64 support (on real hardware) for this exotic >> fringe hybrid. I can imagine that actually building for this >> environment (with a 32 bit userland) might lead to 'funny' results >> as well (as in major toolchain changes necessary to get it working >> as expected). > > I'm not proposing removing amd64 support, I'm proposing x32 is likely a > more valuable target. Do you mean to actually introduce an x86_x32 userspace target in OpenWrt? If so, I suggest you take a look at [1] to get an idea of the can of worms you might be opening there. I do not think OpenWrt has the resources to handle this level of breakage for such an exotic, barely upstream supported target. > Yet what you're describing reads like your desire > is for OpenWRT/x86 to simply be yet another desktop Linux distribution. > > Unless you feel a networking device really needs 256GB of memory, virtual > machines are precisely what OpenWRT/x86 *should* target. I think it is > reasonable to also have a jumbo/desktop build, but using an entire x86 > machine doesn't seem to match OpenWRT's main theme. You seem to ignore perfectly capable so-called « mini pc » routers which are in use out there. They don’t need a « jumbo/desktop » build and they don’t have 256GB RAM, yet they work perfectly well with the current OpenWrt image. Cheers, T [1] http://bugs.debian.org/cgi-bin/pkgreport.cgi?users=debian-x32@lists.debian.org&tags=port-x32
My own experience disagrees. I spent 17 months bringing up a Xeon-D based traffic shaper for 40Gb/s of traffic in a radio base station. And yes, when collected crunched traffic statistics, it did use more than 4GB of address space to do so. > On Mar 30, 2023, at 5:30 PM, Elliott Mitchell <ehem+openwrt@m5p.com> wrote: > > Full amd64 support isn't really appropriate for most situations > OpenWRT is deployed. Whereas x86-x32 seems extremely appropriate for > these situations. As such enable x86-x32 support. > > CONFIG_ARCH_MMAP_RND_COMPAT_BITS is required to follow along, > otherwise the kernel build breaks. > > Signed-off-by: Elliott Mitchell <ehem+openwrt@m5p.com> > --- > I suggest OpenWRT should be placing quite a bit of effort towards > x86-x32. x86-x32 seems a rather superior generic target for OpenWRT. > Only issue is it could be valuable to have at least minimal amd64 > userland support alongside the x86-x32 version. > --- > target/linux/x86/64/config-5.10 | 1 - > target/linux/x86/64/config-5.15 | 1 - > target/linux/x86/config-5.10 | 2 ++ > target/linux/x86/config-5.15 | 2 ++ > 4 files changed, 4 insertions(+), 2 deletions(-) > > diff --git a/target/linux/x86/64/config-5.10 b/target/linux/x86/64/config-5.10 > index 1515f90932..de93495fb1 100644 > --- a/target/linux/x86/64/config-5.10 > +++ b/target/linux/x86/64/config-5.10 > @@ -470,7 +470,6 @@ CONFIG_X86_PM_TIMER=y > # CONFIG_X86_POWERNOW_K8 is not set > # CONFIG_X86_VSYSCALL_EMULATION is not set > CONFIG_X86_X2APIC=y > -# CONFIG_X86_X32 is not set > CONFIG_XEN=y > CONFIG_XENFS=y > CONFIG_XEN_512GB=y > diff --git a/target/linux/x86/64/config-5.15 b/target/linux/x86/64/config-5.15 > index a20891ea55..39e5064e53 100644 > --- a/target/linux/x86/64/config-5.15 > +++ b/target/linux/x86/64/config-5.15 > @@ -493,7 +493,6 @@ CONFIG_X86_PM_TIMER=y > # CONFIG_X86_POWERNOW_K8 is not set > # CONFIG_X86_VSYSCALL_EMULATION is not set > CONFIG_X86_X2APIC=y > -# CONFIG_X86_X32 is not set > CONFIG_XEN=y > CONFIG_XENFS=y > CONFIG_XEN_512GB=y > diff --git a/target/linux/x86/config-5.10 b/target/linux/x86/config-5.10 > index afb7adc63a..8be829d549 100644 > --- a/target/linux/x86/config-5.10 > +++ b/target/linux/x86/config-5.10 > @@ -12,6 +12,7 @@ CONFIG_ARCH_HIBERNATION_POSSIBLE=y > CONFIG_ARCH_MAY_HAVE_PC_FDC=y > CONFIG_ARCH_MIGHT_HAVE_PC_PARPORT=y > CONFIG_ARCH_MIGHT_HAVE_PC_SERIO=y > +CONFIG_ARCH_MMAP_RND_COMPAT_BITS=8 > CONFIG_ARCH_RANDOM=y > CONFIG_ARCH_SELECT_MEMORY_MODEL=y > CONFIG_ARCH_SPARSEMEM_ENABLE=y > @@ -423,6 +424,7 @@ CONFIG_X86_UP_IOAPIC=y > CONFIG_X86_USE_PPRO_CHECKSUM=y > CONFIG_X86_VERBOSE_BOOTUP=y > CONFIG_X86_VMX_FEATURE_NAMES=y > +CONFIG_X86_X32=y > CONFIG_XZ_DEC_BCJ=y > CONFIG_XZ_DEC_X86=y > CONFIG_ZLIB_INFLATE=y > diff --git a/target/linux/x86/config-5.15 b/target/linux/x86/config-5.15 > index b59e809127..afe66b27b1 100644 > --- a/target/linux/x86/config-5.15 > +++ b/target/linux/x86/config-5.15 > @@ -13,6 +13,7 @@ CONFIG_ARCH_MAY_HAVE_PC_FDC=y > CONFIG_ARCH_MHP_MEMMAP_ON_MEMORY_ENABLE=y > CONFIG_ARCH_MIGHT_HAVE_PC_PARPORT=y > CONFIG_ARCH_MIGHT_HAVE_PC_SERIO=y > +CONFIG_ARCH_MMAP_RND_COMPAT_BITS=8 > CONFIG_ARCH_NR_GPIO=512 > CONFIG_ARCH_RANDOM=y > CONFIG_ARCH_SELECT_MEMORY_MODEL=y > @@ -428,6 +429,7 @@ CONFIG_X86_UP_IOAPIC=y > CONFIG_X86_USE_PPRO_CHECKSUM=y > CONFIG_X86_VERBOSE_BOOTUP=y > CONFIG_X86_VMX_FEATURE_NAMES=y > +CONFIG_X86_X32=y > CONFIG_XZ_DEC_BCJ=y > CONFIG_XZ_DEC_X86=y > CONFIG_ZLIB_INFLATE=y > -- > (\___(\___(\______ --=> 8-) EHM <=-- ______/)___/)___/) > \BS ( | ehem+openwrt@m5p.com PGP 87145445 | ) / > \_CS\ | _____ -O #include <stddisclaimer.h> O- _____ | / _/ > 8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445 > > > > > _______________________________________________ > openwrt-devel mailing list > openwrt-devel@lists.openwrt.org > https://lists.openwrt.org/mailman/listinfo/openwrt-devel
> On Apr 26, 2023, at 6:00 PM, Elliott Mitchell <ehem+openwrt@m5p.com> wrote: > > On Thu, Apr 27, 2023 at 12:46:49AM +0200, Stefan Lippers-Hollmann wrote: >> >> On 2023-03-30, Elliott Mitchell wrote: >>> Full amd64 support isn't really appropriate for most situations >>> OpenWRT is deployed. Whereas x86-x32 seems extremely appropriate for >>> these situations. As such enable x86-x32 support. >>> >>> CONFIG_ARCH_MMAP_RND_COMPAT_BITS is required to follow along, >>> otherwise the kernel build breaks. >>> >>> Signed-off-by: Elliott Mitchell <ehem+openwrt@m5p.com> >>> --- >>> I suggest OpenWRT should be placing quite a bit of effort towards >>> x86-x32. x86-x32 seems a rather superior generic target for OpenWRT. >>> Only issue is it could be valuable to have at least minimal amd64 >>> userland support alongside the x86-x32 version. >> >> x86_32 is pretty much dead in the water, with almost zero deployment >> by general purpose distributions - apart from VM data centre >> environments doing their own thing (least amount of RAM usage >> possible, everything else being secondary at best). At least Debian >> did raise security concerns about the x86_32 ISA in the past. > > Error: undefined symbol "x86_32" > > Without the extra context I would resolve that to "i386"/"ia32". I think > "x32" or "x86_x32" are better strings for this case. > > According to what I had read that was in the past when x32 was sharing > less of the i386 ABI. Notably x32 had been trying to pass time values > in a distinct fashion. I will conceed I'm unsure whether x32 is ever > truly going to get a seat at the table. > > On a different news front, Linux 5.10 has an option > CONFIG_X86_X32_DISABLED which leaves x32 disabled by default (Debian's > kernels were configured this way). Whereas 5.15 has removed the > CONFIG_X86_X32_DISABLED option which seems to suggest the concerns may > have been assuaged. > >> While I might understand (understand, not support) a desire for this >> as a dedicated subtarget (to appease the virtualization crowd), >> although I still don't see a reason or sufficient uptake in more >> conventional Linux environments. I would not be happy (at all) to >> lose 'normal' x86_64 support (on real hardware) for this exotic >> fringe hybrid. I can imagine that actually building for this >> environment (with a 32 bit userland) might lead to 'funny' results >> as well (as in major toolchain changes necessary to get it working >> as expected). > > I'm not proposing removing amd64 support, I'm proposing x32 is likely a > more valuable target. Yet what you're describing reads like your desire > is for OpenWRT/x86 to simply be yet another desktop Linux distribution. > > Unless you feel a networking device really needs 256GB of memory, virtual > machines are precisely what OpenWRT/x86 *should* target. I think it is > reasonable to also have a jumbo/desktop build, but using an entire x86 > machine doesn't seem to match OpenWRT's main theme. I personally know of companies running virtualized instances of AMD64 on m4.large or m5.large Amazon EC2 instances. I have three AMD64 firewalls... Two PC Engines APU6's (AMD GX-412TC "Jaguar" SoC based) running as a hot tandem pair for fail-over at the home office, and a dogfooding instance running on a KVM server that sandboxes my teenage kids and keeps them off the main household networks.
On Thu, Apr 27, 2023 at 11:27:30AM +0200, Thibaut wrote: > > > Le 27 avr. 2023 à 02:00, Elliott Mitchell <ehem+openwrt@m5p.com> a écrit : > > > > On Thu, Apr 27, 2023 at 12:46:49AM +0200, Stefan Lippers-Hollmann wrote: > > > >> While I might understand (understand, not support) a desire for this > >> as a dedicated subtarget (to appease the virtualization crowd), > >> although I still don't see a reason or sufficient uptake in more > >> conventional Linux environments. I would not be happy (at all) to > >> lose 'normal' x86_64 support (on real hardware) for this exotic > >> fringe hybrid. I can imagine that actually building for this > >> environment (with a 32 bit userland) might lead to 'funny' results > >> as well (as in major toolchain changes necessary to get it working > >> as expected). > > > > I'm not proposing removing amd64 support, I'm proposing x32 is likely a > > more valuable target. > > Do you mean to actually introduce an x86_x32 userspace target in OpenWrt? > If so, I suggest you take a look at [1] to get an idea of the can of worms you might be opening there. > > I do not think OpenWrt has the resources to handle this level of breakage for such an exotic, barely upstream supported target. It seems worthy experimention at least. Enabling the kernel option adds somewhere between 1-8191 bytes (kernel build grew 1 page). If this reduces memory usage by 10% and increases performance by 10% that seems notable. My hope is that others have flushed most of the bugs out by now. If there are still too many bugs at this point, it can be left unreleased. > > Yet what you're describing reads like your desire > > is for OpenWRT/x86 to simply be yet another desktop Linux distribution. > > > > Unless you feel a networking device really needs 256GB of memory, virtual > > machines are precisely what OpenWRT/x86 *should* target. I think it is > > reasonable to also have a jumbo/desktop build, but using an entire x86 > > machine doesn't seem to match OpenWRT's main theme. > > You seem to ignore perfectly capable so-called « mini pc » routers which are in use out there. They don’t need a « jumbo/desktop » build and they don’t have 256GB RAM, yet they work perfectly well with the current OpenWrt image. That is indeed a better choice for consideration. What type of installation do you think OpenWRT should target for such devices? I'm unsure how much memory such devices typically have. I was able to find listings for SO-DIMMs from 4GB to 32GB. Most recent devices will include an IOMMU and 4GB is more than enough to usefully use a hypervisor. In such a case it might well make sense to use 128MB for handling the network and devote the rest to other tasks. With a system that size, there is rather MORE urgency to keep VMs small. Alternatively do you feel a router needs 4GB of memory? Thing is this turns OpenWRT into simply another desktop Linux distribution. At which point why not go with a Linux distribution designed for the desktop and has all the desktop features?
On Fri, Apr 28, 2023 at 11:00:43AM -0600, Philip Prindeville wrote: > My own experience disagrees. > > I spent 17 months bringing up a Xeon-D based traffic shaper for 40Gb/s of traffic in a radio base station. > > And yes, when collected crunched traffic statistics, it did use more than 4GB of address space to do so. I can believe that. Though I kind of doubt this is the typical usage of OpenWRT. A kernel can have both amd64 and x32 enabled. My thinking was in typical usage most programs won't be that large and thus x32 is smaller and faster. Note, enabling CONFIG_X86_X32 does not force the use of x32 userspace, it merely allows it. The observed kernel runtime was 4KB larger which translates to +1-8191 bytes.
diff --git a/target/linux/x86/64/config-5.10 b/target/linux/x86/64/config-5.10 index 1515f90932..de93495fb1 100644 --- a/target/linux/x86/64/config-5.10 +++ b/target/linux/x86/64/config-5.10 @@ -470,7 +470,6 @@ CONFIG_X86_PM_TIMER=y # CONFIG_X86_POWERNOW_K8 is not set # CONFIG_X86_VSYSCALL_EMULATION is not set CONFIG_X86_X2APIC=y -# CONFIG_X86_X32 is not set CONFIG_XEN=y CONFIG_XENFS=y CONFIG_XEN_512GB=y diff --git a/target/linux/x86/64/config-5.15 b/target/linux/x86/64/config-5.15 index a20891ea55..39e5064e53 100644 --- a/target/linux/x86/64/config-5.15 +++ b/target/linux/x86/64/config-5.15 @@ -493,7 +493,6 @@ CONFIG_X86_PM_TIMER=y # CONFIG_X86_POWERNOW_K8 is not set # CONFIG_X86_VSYSCALL_EMULATION is not set CONFIG_X86_X2APIC=y -# CONFIG_X86_X32 is not set CONFIG_XEN=y CONFIG_XENFS=y CONFIG_XEN_512GB=y diff --git a/target/linux/x86/config-5.10 b/target/linux/x86/config-5.10 index afb7adc63a..8be829d549 100644 --- a/target/linux/x86/config-5.10 +++ b/target/linux/x86/config-5.10 @@ -12,6 +12,7 @@ CONFIG_ARCH_HIBERNATION_POSSIBLE=y CONFIG_ARCH_MAY_HAVE_PC_FDC=y CONFIG_ARCH_MIGHT_HAVE_PC_PARPORT=y CONFIG_ARCH_MIGHT_HAVE_PC_SERIO=y +CONFIG_ARCH_MMAP_RND_COMPAT_BITS=8 CONFIG_ARCH_RANDOM=y CONFIG_ARCH_SELECT_MEMORY_MODEL=y CONFIG_ARCH_SPARSEMEM_ENABLE=y @@ -423,6 +424,7 @@ CONFIG_X86_UP_IOAPIC=y CONFIG_X86_USE_PPRO_CHECKSUM=y CONFIG_X86_VERBOSE_BOOTUP=y CONFIG_X86_VMX_FEATURE_NAMES=y +CONFIG_X86_X32=y CONFIG_XZ_DEC_BCJ=y CONFIG_XZ_DEC_X86=y CONFIG_ZLIB_INFLATE=y diff --git a/target/linux/x86/config-5.15 b/target/linux/x86/config-5.15 index b59e809127..afe66b27b1 100644 --- a/target/linux/x86/config-5.15 +++ b/target/linux/x86/config-5.15 @@ -13,6 +13,7 @@ CONFIG_ARCH_MAY_HAVE_PC_FDC=y CONFIG_ARCH_MHP_MEMMAP_ON_MEMORY_ENABLE=y CONFIG_ARCH_MIGHT_HAVE_PC_PARPORT=y CONFIG_ARCH_MIGHT_HAVE_PC_SERIO=y +CONFIG_ARCH_MMAP_RND_COMPAT_BITS=8 CONFIG_ARCH_NR_GPIO=512 CONFIG_ARCH_RANDOM=y CONFIG_ARCH_SELECT_MEMORY_MODEL=y @@ -428,6 +429,7 @@ CONFIG_X86_UP_IOAPIC=y CONFIG_X86_USE_PPRO_CHECKSUM=y CONFIG_X86_VERBOSE_BOOTUP=y CONFIG_X86_VMX_FEATURE_NAMES=y +CONFIG_X86_X32=y CONFIG_XZ_DEC_BCJ=y CONFIG_XZ_DEC_X86=y CONFIG_ZLIB_INFLATE=y
Full amd64 support isn't really appropriate for most situations OpenWRT is deployed. Whereas x86-x32 seems extremely appropriate for these situations. As such enable x86-x32 support. CONFIG_ARCH_MMAP_RND_COMPAT_BITS is required to follow along, otherwise the kernel build breaks. Signed-off-by: Elliott Mitchell <ehem+openwrt@m5p.com> --- I suggest OpenWRT should be placing quite a bit of effort towards x86-x32. x86-x32 seems a rather superior generic target for OpenWRT. Only issue is it could be valuable to have at least minimal amd64 userland support alongside the x86-x32 version. --- target/linux/x86/64/config-5.10 | 1 - target/linux/x86/64/config-5.15 | 1 - target/linux/x86/config-5.10 | 2 ++ target/linux/x86/config-5.15 | 2 ++ 4 files changed, 4 insertions(+), 2 deletions(-)