| Message ID | 20240415192414.14155-2-palmer@rivosinc.com |
|---|---|
| State | New |
| Headers | show |
| Series | [RFC] build-many-glibcs: Add a rv64gcbv-on-rv64gc/lp64d sub-variant | expand |
On Mon, Apr 15, 2024 at 12:27 PM Palmer Dabbelt <palmer@rivosinc.com> wrote: > > Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com> > --- > So this is very much an RFC. I had written a commit message as > > This will hopefully be a widely used configuration in the future, but > it's not all that common right now. That said, let's be proactive and > at start testing it. > > but I even that's kind of a strong statement here so I'm just stashing > it off to the side to be sure. > > A few of us were talking about the GCC meeting last week, but I figured > I'd send a glibc patch to at least start some discussion on the lists as > something similar had come up in the TLSDESC thread and a while ago when > we talked about glibc-hwcaps. > > I'm not really sure what the right thing to do here is: we don't have > broadly available hardware that supports these new extensions, but > they're pretty much table stakes for hardware that's competitive with > arm64/x86. Without B and V we're going to end up piling up a ton of > IFUNC-based routines to try and work around the base ISA, but those > extensions are so fundamental to good codegen that it doesn't feel like > we're going to get where we want with just IFUNCs. > > Unfortunately I don't think we can drop support for the other base ISAs: > the distros appear to be targeting rv64gc and that's the only common > base for most hardware that's out there. I guess we could drop support > for the rv*ima-based base ISAs, but I'm not sure that buys us much: > we've got a lot of embedded users so we won't be able to drop the GCC > support for soft float, and dropping the glibc support puts us in a > clunky spot for testing. > > So we're probably stuck in a bit of a performance hole for a while. > That said, we could at least put a stake in the ground and start testing > some target with some newer extensions. That way distros that want to > try a larger base than rv64gc have something that's getting tested and > is thus likely to work. For GCC we ended up adding some test targets > with pretty much all the non-embedded, but at least B and V seem like > the bare minimum. > > I'm not sure if all that is worth the extra build/test resource burden > for everyone, though, particularly given the lack of hardware. for Android -- which is in a similar situation -- having the build system enable these extensions been pretty useful for finding toolchain and qemu bugs, which seems useful in its own right, if only for CI purposes... > --- > scripts/build-many-glibcs.py | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/scripts/build-many-glibcs.py b/scripts/build-many-glibcs.py > index ecc743e672..21da46dc4a 100755 > --- a/scripts/build-many-glibcs.py > +++ b/scripts/build-many-glibcs.py > @@ -411,7 +411,9 @@ class Context(object): > os_name='linux-gnu', > variant='rv64imafdc-lp64d', > gcc_cfg=['--with-arch=rv64imafdc', '--with-abi=lp64d', > - '--disable-multilib']) > + '--disable-multilib']), > + extra_glibcs=[{'variant': 'rv64gcbv-rv64gc-lp64d', > + 'ccopts': '-march=rv64gcv_zba_zbb_zbs'}] > self.add_config(arch='s390x', > os_name='linux-gnu', > glibcs=[{}, > -- > 2.44.0 >
On Mon, 15 Apr 2024 13:05:53 PDT (-0700), enh@google.com wrote: > On Mon, Apr 15, 2024 at 12:27 PM Palmer Dabbelt <palmer@rivosinc.com> wrote: >> >> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com> >> --- >> So this is very much an RFC. I had written a commit message as >> >> This will hopefully be a widely used configuration in the future, but >> it's not all that common right now. That said, let's be proactive and >> at start testing it. >> >> but I even that's kind of a strong statement here so I'm just stashing >> it off to the side to be sure. >> >> A few of us were talking about the GCC meeting last week, but I figured >> I'd send a glibc patch to at least start some discussion on the lists as >> something similar had come up in the TLSDESC thread and a while ago when >> we talked about glibc-hwcaps. >> >> I'm not really sure what the right thing to do here is: we don't have >> broadly available hardware that supports these new extensions, but >> they're pretty much table stakes for hardware that's competitive with >> arm64/x86. Without B and V we're going to end up piling up a ton of >> IFUNC-based routines to try and work around the base ISA, but those >> extensions are so fundamental to good codegen that it doesn't feel like >> we're going to get where we want with just IFUNCs. >> >> Unfortunately I don't think we can drop support for the other base ISAs: >> the distros appear to be targeting rv64gc and that's the only common >> base for most hardware that's out there. I guess we could drop support >> for the rv*ima-based base ISAs, but I'm not sure that buys us much: >> we've got a lot of embedded users so we won't be able to drop the GCC >> support for soft float, and dropping the glibc support puts us in a >> clunky spot for testing. >> >> So we're probably stuck in a bit of a performance hole for a while. >> That said, we could at least put a stake in the ground and start testing >> some target with some newer extensions. That way distros that want to >> try a larger base than rv64gc have something that's getting tested and >> is thus likely to work. For GCC we ended up adding some test targets >> with pretty much all the non-embedded, but at least B and V seem like >> the bare minimum. >> >> I'm not sure if all that is worth the extra build/test resource burden >> for everyone, though, particularly given the lack of hardware. > > for Android -- which is in a similar situation -- having the build > system enable these extensions been pretty useful for finding > toolchain and qemu bugs, which seems useful in its own right, if only > for CI purposes... Ya, that's kind of my theory here. Hopefully hardware vendors will eventually align to some interesting set of extensions, maybe split between a few markets, but at least we can do _something_ here. I'd be super surprised if we end up with any sort of long-term successful hardware that lacks B and V, so it seems like a safe target for testing. >> --- >> scripts/build-many-glibcs.py | 4 +++- >> 1 file changed, 3 insertions(+), 1 deletion(-) >> >> diff --git a/scripts/build-many-glibcs.py b/scripts/build-many-glibcs.py >> index ecc743e672..21da46dc4a 100755 >> --- a/scripts/build-many-glibcs.py >> +++ b/scripts/build-many-glibcs.py >> @@ -411,7 +411,9 @@ class Context(object): >> os_name='linux-gnu', >> variant='rv64imafdc-lp64d', >> gcc_cfg=['--with-arch=rv64imafdc', '--with-abi=lp64d', >> - '--disable-multilib']) >> + '--disable-multilib']), >> + extra_glibcs=[{'variant': 'rv64gcbv-rv64gc-lp64d', >> + 'ccopts': '-march=rv64gcv_zba_zbb_zbs'}] >> self.add_config(arch='s390x', >> os_name='linux-gnu', >> glibcs=[{}, >> -- >> 2.44.0 >>
Is this really lp64d? What is the status of the vector ABI? If we need to have (and test) more targets that are actually useful, so be it. But we don't want to be stuck testing targets nobody uses or that provide no testing value, and removing them is usually harder than adding them. On Mon, Apr 15, 2024 at 12:24:15PM -0700, Palmer Dabbelt wrote: > Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com> > --- > So this is very much an RFC. I had written a commit message as > > This will hopefully be a widely used configuration in the future, but > it's not all that common right now. That said, let's be proactive and > at start testing it. > > but I even that's kind of a strong statement here so I'm just stashing > it off to the side to be sure. > > A few of us were talking about the GCC meeting last week, but I figured > I'd send a glibc patch to at least start some discussion on the lists as > something similar had come up in the TLSDESC thread and a while ago when > we talked about glibc-hwcaps. > > I'm not really sure what the right thing to do here is: we don't have > broadly available hardware that supports these new extensions, but > they're pretty much table stakes for hardware that's competitive with > arm64/x86. Without B and V we're going to end up piling up a ton of > IFUNC-based routines to try and work around the base ISA, but those > extensions are so fundamental to good codegen that it doesn't feel like > we're going to get where we want with just IFUNCs. > > Unfortunately I don't think we can drop support for the other base ISAs: > the distros appear to be targeting rv64gc and that's the only common > base for most hardware that's out there. I guess we could drop support > for the rv*ima-based base ISAs, but I'm not sure that buys us much: > we've got a lot of embedded users so we won't be able to drop the GCC > support for soft float, and dropping the glibc support puts us in a > clunky spot for testing. > > So we're probably stuck in a bit of a performance hole for a while. > That said, we could at least put a stake in the ground and start testing > some target with some newer extensions. That way distros that want to > try a larger base than rv64gc have something that's getting tested and > is thus likely to work. For GCC we ended up adding some test targets > with pretty much all the non-embedded, but at least B and V seem like > the bare minimum. > > I'm not sure if all that is worth the extra build/test resource burden > for everyone, though, particularly given the lack of hardware. > --- > scripts/build-many-glibcs.py | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/scripts/build-many-glibcs.py b/scripts/build-many-glibcs.py > index ecc743e672..21da46dc4a 100755 > --- a/scripts/build-many-glibcs.py > +++ b/scripts/build-many-glibcs.py > @@ -411,7 +411,9 @@ class Context(object): > os_name='linux-gnu', > variant='rv64imafdc-lp64d', > gcc_cfg=['--with-arch=rv64imafdc', '--with-abi=lp64d', > - '--disable-multilib']) > + '--disable-multilib']), > + extra_glibcs=[{'variant': 'rv64gcbv-rv64gc-lp64d', > + 'ccopts': '-march=rv64gcv_zba_zbb_zbs'}] > self.add_config(arch='s390x', > os_name='linux-gnu', > glibcs=[{}, > -- > 2
On Mon, 15 Apr 2024 13:24:14 PDT (-0700), Darius Rad wrote: > Is this really lp64d? I guess that depend on what you mean by "ABI". Passing `-march=rv64gv -mabi=lp64d` will result in binaries that require the V extension to run, but can be linked with any other binary that is compiled with `-mabi=lp64d`. It's essentially the same as `-march=rv64g -mabi=lp64`, except for vector instead of float. > What is the status of the vector ABI? We don't have a global vector ABI switch for GCC (ie, `-mabi=lp64dv`) and we decided not to put on in for GCC-14. We've essentially got all the machinery there for it, as we've got the vector calling convention attribute, but that came in pretty late and we decided it'd be better to wait. I don't see any reason it'd miss GCC-15, but no promises as there's not even a patch on the lists yet... > If we need to > have (and test) more targets that are actually useful, so be it. But we > don't want to be stuck testing targets nobody uses or that provide no > testing value, and removing them is usually harder than adding them. Ya, that's kind of my worry here. We'd be going out on a limb here by targeting something without hardware availability, we've already got some baggage floating around from lightly-used targets and it'd be best to avoid too much of that. IMO it's really more of a question for users/distros -- and I'm definitely a crazy person on that front, I run Gentoo ;) > On Mon, Apr 15, 2024 at 12:24:15PM -0700, Palmer Dabbelt wrote: >> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com> >> --- >> So this is very much an RFC. I had written a commit message as >> >> This will hopefully be a widely used configuration in the future, but >> it's not all that common right now. That said, let's be proactive and >> at start testing it. >> >> but I even that's kind of a strong statement here so I'm just stashing >> it off to the side to be sure. >> >> A few of us were talking about the GCC meeting last week, but I figured >> I'd send a glibc patch to at least start some discussion on the lists as >> something similar had come up in the TLSDESC thread and a while ago when >> we talked about glibc-hwcaps. >> >> I'm not really sure what the right thing to do here is: we don't have >> broadly available hardware that supports these new extensions, but >> they're pretty much table stakes for hardware that's competitive with >> arm64/x86. Without B and V we're going to end up piling up a ton of >> IFUNC-based routines to try and work around the base ISA, but those >> extensions are so fundamental to good codegen that it doesn't feel like >> we're going to get where we want with just IFUNCs. >> >> Unfortunately I don't think we can drop support for the other base ISAs: >> the distros appear to be targeting rv64gc and that's the only common >> base for most hardware that's out there. I guess we could drop support >> for the rv*ima-based base ISAs, but I'm not sure that buys us much: >> we've got a lot of embedded users so we won't be able to drop the GCC >> support for soft float, and dropping the glibc support puts us in a >> clunky spot for testing. >> >> So we're probably stuck in a bit of a performance hole for a while. >> That said, we could at least put a stake in the ground and start testing >> some target with some newer extensions. That way distros that want to >> try a larger base than rv64gc have something that's getting tested and >> is thus likely to work. For GCC we ended up adding some test targets >> with pretty much all the non-embedded, but at least B and V seem like >> the bare minimum. >> >> I'm not sure if all that is worth the extra build/test resource burden >> for everyone, though, particularly given the lack of hardware. >> --- >> scripts/build-many-glibcs.py | 4 +++- >> 1 file changed, 3 insertions(+), 1 deletion(-) >> >> diff --git a/scripts/build-many-glibcs.py b/scripts/build-many-glibcs.py >> index ecc743e672..21da46dc4a 100755 >> --- a/scripts/build-many-glibcs.py >> +++ b/scripts/build-many-glibcs.py >> @@ -411,7 +411,9 @@ class Context(object): >> os_name='linux-gnu', >> variant='rv64imafdc-lp64d', >> gcc_cfg=['--with-arch=rv64imafdc', '--with-abi=lp64d', >> - '--disable-multilib']) >> + '--disable-multilib']), >> + extra_glibcs=[{'variant': 'rv64gcbv-rv64gc-lp64d', >> + 'ccopts': '-march=rv64gcv_zba_zbb_zbs'}] >> self.add_config(arch='s390x', >> os_name='linux-gnu', >> glibcs=[{}, >> -- >> 2
On Tue, 16 Apr 2024 at 07:28, Palmer Dabbelt <palmer@rivosinc.com> wrote: > Unfortunately I don't think we can drop support for the other base ISAs: > the distros appear to be targeting rv64gc and that's the only common > base for most hardware that's out there. > This is certainly the case for Ubuntu for now but I certainly hope that by 26.04 we are able to be a bit more aggressive here, fwiw. Cheers, mwh
On Mon, Apr 15, 2024 at 02:36:59PM -0700, Palmer Dabbelt wrote: > On Mon, 15 Apr 2024 13:24:14 PDT (-0700), Darius Rad wrote: > > Is this really lp64d? > > I guess that depend on what you mean by "ABI". Passing `-march=rv64gv > -mabi=lp64d` will result in binaries that require the V extension to run, > but can be linked with any other binary that is compiled with `-mabi=lp64d`. > It's essentially the same as `-march=rv64g -mabi=lp64`, except for vector > instead of float. > Got it. I thought that might be the case, but wasn't sure if the vector calling convention leaked in with the arch switch only (and not the abi). > > What is the status of the vector ABI? > > We don't have a global vector ABI switch for GCC (ie, `-mabi=lp64dv`) and we > decided not to put on in for GCC-14. We've essentially got all the > machinery there for it, as we've got the vector calling convention > attribute, but that came in pretty late and we decided it'd be better to > wait. > > I don't see any reason it'd miss GCC-15, but no promises as there's not even > a patch on the lists yet... > Understood. Though when this hits, won't it be the most likely target for vector? Meaning at that point, rv64gcbv/lp64d would be less useful or important. > > If we need to > > have (and test) more targets that are actually useful, so be it. But we > > don't want to be stuck testing targets nobody uses or that provide no > > testing value, and removing them is usually harder than adding them. > > Ya, that's kind of my worry here. We'd be going out on a limb here by > targeting something without hardware availability, we've already got some > baggage floating around from lightly-used targets and it'd be best to avoid > too much of that. > Maybe make an explicit note that if/when the vector ABI is in gcc, the target will be switched to that, rather than another target being added. Then there's some notice for people to not rely on rv64gcbv/lp64d, or at least put up support and/or evidence of a use case to retain the target. > IMO it's really more of a question for users/distros -- and I'm definitely a > crazy person on that front, I run Gentoo ;) >
On Tue, 16 Apr 2024 07:55:38 PDT (-0700), Darius Rad wrote: > On Mon, Apr 15, 2024 at 02:36:59PM -0700, Palmer Dabbelt wrote: >> On Mon, 15 Apr 2024 13:24:14 PDT (-0700), Darius Rad wrote: >> > Is this really lp64d? >> >> I guess that depend on what you mean by "ABI". Passing `-march=rv64gv >> -mabi=lp64d` will result in binaries that require the V extension to run, >> but can be linked with any other binary that is compiled with `-mabi=lp64d`. >> It's essentially the same as `-march=rv64g -mabi=lp64`, except for vector >> instead of float. >> > > Got it. I thought that might be the case, but wasn't sure if the vector > calling convention leaked in with the arch switch only (and not the abi). The intent was to avoid that, but GCC-14 will be the first release with any meaningful vector codegen so I guess we'll find out if we have any bugs ;) >> > What is the status of the vector ABI? >> >> We don't have a global vector ABI switch for GCC (ie, `-mabi=lp64dv`) and we >> decided not to put on in for GCC-14. We've essentially got all the >> machinery there for it, as we've got the vector calling convention >> attribute, but that came in pretty late and we decided it'd be better to >> wait. >> >> I don't see any reason it'd miss GCC-15, but no promises as there's not even >> a patch on the lists yet... >> > > Understood. Though when this hits, won't it be the most likely target for > vector? Meaning at that point, rv64gcbv/lp64d would be less useful or > important. I'm not really sure. If I was building a system I'd want an lp64dv-flavored ABI, but if distros are already rv64gc/lp64d then rv64gcv/lp64d would provide a binary compatible upgrade path for them. Assuming we end up with some sort of glibc-hwcaps the rv64gcv/lp64d builds might be common. It's kind of all just a guess until HW shows up, though... >> > If we need to >> > have (and test) more targets that are actually useful, so be it. But we >> > don't want to be stuck testing targets nobody uses or that provide no >> > testing value, and removing them is usually harder than adding them. >> >> Ya, that's kind of my worry here. We'd be going out on a limb here by >> targeting something without hardware availability, we've already got some >> baggage floating around from lightly-used targets and it'd be best to avoid >> too much of that. >> > > Maybe make an explicit note that if/when the vector ABI is in gcc, the > target will be switched to that, rather than another target being added. > Then there's some notice for people to not rely on rv64gcbv/lp64d, or at > least put up support and/or evidence of a use case to retain the target. Ya, I guess so far we've not really been explicit about what's supported in glibc and what's not -- there's some code in there to ban stuff like lv64f and some default ISAs in build-many-glibcs, but I don't know if any of that is really binding. So maybe we should do a wiki entry with what configurations are actively tested and thus likely to work? >> IMO it's really more of a question for users/distros -- and I'm definitely a >> crazy person on that front, I run Gentoo ;) >>
On Tue, 16 Apr 2024 00:01:31 PDT (-0700), michael.hudson@canonical.com wrote: > On Tue, 16 Apr 2024 at 07:28, Palmer Dabbelt <palmer@rivosinc.com> wrote: > >> Unfortunately I don't think we can drop support for the other base ISAs: >> the distros appear to be targeting rv64gc and that's the only common >> base for most hardware that's out there. >> > > This is certainly the case for Ubuntu for now but I certainly hope that by > 26.04 we are able to be a bit more aggressive here, fwiw. That'd be awesome. Any idea if something like this would help? I think we're all a bit worried about being stuck with a very small baseline and thus a bunch of vendor-specific behavior. That's certainly the way things are trending right now in RISC-V land, it's already super complicated to manage and it looks like it's going to get worse before it gets better. If we can pick up some extra testing burden now as a way to reduce fragmentation then it's probably a win in the long run, even if we have to go through a few generations of this before things calm down. If you're planning on moving to something completely different, though, then maybe it's not worth the extra builds. > Cheers, > mwh
On Tue, Apr 16, 2024 at 09:04:24AM -0700, Palmer Dabbelt wrote: > On Tue, 16 Apr 2024 07:55:38 PDT (-0700), Darius Rad wrote: > > On Mon, Apr 15, 2024 at 02:36:59PM -0700, Palmer Dabbelt wrote: > > > On Mon, 15 Apr 2024 13:24:14 PDT (-0700), Darius Rad wrote: > > > > Is this really lp64d? > > > > > > I guess that depend on what you mean by "ABI". Passing `-march=rv64gv > > > -mabi=lp64d` will result in binaries that require the V extension to run, > > > but can be linked with any other binary that is compiled with `-mabi=lp64d`. > > > It's essentially the same as `-march=rv64g -mabi=lp64`, except for vector > > > instead of float. > > > > > > > Got it. I thought that might be the case, but wasn't sure if the vector > > calling convention leaked in with the arch switch only (and not the abi). > > The intent was to avoid that, but GCC-14 will be the first release with any > meaningful vector codegen so I guess we'll find out if we have any bugs ;) > > > > > What is the status of the vector ABI? > > > > > > We don't have a global vector ABI switch for GCC (ie, `-mabi=lp64dv`) and we > > > decided not to put on in for GCC-14. We've essentially got all the > > > machinery there for it, as we've got the vector calling convention > > > attribute, but that came in pretty late and we decided it'd be better to > > > wait. > > > > > > I don't see any reason it'd miss GCC-15, but no promises as there's not even > > > a patch on the lists yet... > > > > > > > Understood. Though when this hits, won't it be the most likely target for > > vector? Meaning at that point, rv64gcbv/lp64d would be less useful or > > important. > > I'm not really sure. If I was building a system I'd want an lp64dv-flavored > ABI, but if distros are already rv64gc/lp64d then rv64gcv/lp64d would > provide a binary compatible upgrade path for them. Assuming we end up with > some sort of glibc-hwcaps the rv64gcv/lp64d builds might be common. > > It's kind of all just a guess until HW shows up, though... > > > > > If we need to > > > > have (and test) more targets that are actually useful, so be it. But we > > > > don't want to be stuck testing targets nobody uses or that provide no > > > > testing value, and removing them is usually harder than adding them. > > > > > > Ya, that's kind of my worry here. We'd be going out on a limb here by > > > targeting something without hardware availability, we've already got some > > > baggage floating around from lightly-used targets and it'd be best to avoid > > > too much of that. > > > > > > > Maybe make an explicit note that if/when the vector ABI is in gcc, the > > target will be switched to that, rather than another target being added. > > Then there's some notice for people to not rely on rv64gcbv/lp64d, or at > > least put up support and/or evidence of a use case to retain the target. > > Ya, I guess so far we've not really been explicit about what's supported in > glibc and what's not -- there's some code in there to ban stuff like lv64f > and some default ISAs in build-many-glibcs, but I don't know if any of that > is really binding. > At least build-many-glibcs, and more importantly the release notes, indicate what is actually tested, which says something. > So maybe we should do a wiki entry with what configurations are actively > tested and thus likely to work? > Sure, if we think that can be kept up to date. I think I would rather take an official position on what build-many-glibcs or the release testing means for supported targets (if there isn't something glibc-wide already), then we don't have another thing to maintain. > > > IMO it's really more of a question for users/distros -- and I'm definitely a > > > crazy person on that front, I run Gentoo
On Tue, 16 Apr 2024 13:41:15 PDT (-0700), Darius Rad wrote: > On Tue, Apr 16, 2024 at 09:04:24AM -0700, Palmer Dabbelt wrote: >> On Tue, 16 Apr 2024 07:55:38 PDT (-0700), Darius Rad wrote: >> > On Mon, Apr 15, 2024 at 02:36:59PM -0700, Palmer Dabbelt wrote: >> > > On Mon, 15 Apr 2024 13:24:14 PDT (-0700), Darius Rad wrote: >> > > > Is this really lp64d? >> > > >> > > I guess that depend on what you mean by "ABI". Passing `-march=rv64gv >> > > -mabi=lp64d` will result in binaries that require the V extension to run, >> > > but can be linked with any other binary that is compiled with `-mabi=lp64d`. >> > > It's essentially the same as `-march=rv64g -mabi=lp64`, except for vector >> > > instead of float. >> > > >> > >> > Got it. I thought that might be the case, but wasn't sure if the vector >> > calling convention leaked in with the arch switch only (and not the abi). >> >> The intent was to avoid that, but GCC-14 will be the first release with any >> meaningful vector codegen so I guess we'll find out if we have any bugs ;) >> >> > > > What is the status of the vector ABI? >> > > >> > > We don't have a global vector ABI switch for GCC (ie, `-mabi=lp64dv`) and we >> > > decided not to put on in for GCC-14. We've essentially got all the >> > > machinery there for it, as we've got the vector calling convention >> > > attribute, but that came in pretty late and we decided it'd be better to >> > > wait. >> > > >> > > I don't see any reason it'd miss GCC-15, but no promises as there's not even >> > > a patch on the lists yet... >> > > >> > >> > Understood. Though when this hits, won't it be the most likely target for >> > vector? Meaning at that point, rv64gcbv/lp64d would be less useful or >> > important. >> >> I'm not really sure. If I was building a system I'd want an lp64dv-flavored >> ABI, but if distros are already rv64gc/lp64d then rv64gcv/lp64d would >> provide a binary compatible upgrade path for them. Assuming we end up with >> some sort of glibc-hwcaps the rv64gcv/lp64d builds might be common. >> >> It's kind of all just a guess until HW shows up, though... >> >> > > > If we need to >> > > > have (and test) more targets that are actually useful, so be it. But we >> > > > don't want to be stuck testing targets nobody uses or that provide no >> > > > testing value, and removing them is usually harder than adding them. >> > > >> > > Ya, that's kind of my worry here. We'd be going out on a limb here by >> > > targeting something without hardware availability, we've already got some >> > > baggage floating around from lightly-used targets and it'd be best to avoid >> > > too much of that. >> > > >> > >> > Maybe make an explicit note that if/when the vector ABI is in gcc, the >> > target will be switched to that, rather than another target being added. >> > Then there's some notice for people to not rely on rv64gcbv/lp64d, or at >> > least put up support and/or evidence of a use case to retain the target. >> >> Ya, I guess so far we've not really been explicit about what's supported in >> glibc and what's not -- there's some code in there to ban stuff like lv64f >> and some default ISAs in build-many-glibcs, but I don't know if any of that >> is really binding. >> > > At least build-many-glibcs, and more importantly the release notes, > indicate what is actually tested, which says something. Certainly the list of tested targets per release is a pretty strong sign of what's actually working now, which is great (and thanks for testing basically every release...). >> So maybe we should do a wiki entry with what configurations are actively >> tested and thus likely to work? >> > > Sure, if we think that can be kept up to date. I think I would rather take > an official position on what build-many-glibcs or the release testing means > for supported targets (if there isn't something glibc-wide already), then > we don't have another thing to maintain. Ya, that's best. I guess that's more of a glibc-wide question, though, so we'll have to see if someone else chimes in? >> > > IMO it's really more of a question for users/distros -- and I'm definitely a >> > > crazy person on that front, I run Gentoo
* Palmer Dabbelt: > If we can pick up some extra testing burden now as a way to reduce > fragmentation then it's probably a win in the long run, even if we > have to go through a few generations of this before things calm down. > If you're planning on moving to something completely different, > though, then maybe it's not worth the extra builds. A run-time tester integrated with patchwork might be useful, assuming it's stable and produces clean results for the current tree. Thanks, Florian
* Michael Hudson-Doyle: > On Tue, 16 Apr 2024 at 07:28, Palmer Dabbelt <palmer@rivosinc.com> wrote: > >> Unfortunately I don't think we can drop support for the other base >> ISAs: the distros appear to be targeting rv64gc and that's the only >> common base for most hardware that's out there. > > This is certainly the case for Ubuntu for now but I certainly hope > that by 26.04 we are able to be a bit more aggressive here, fwiw. You might want to add an early check to ld.so to report CPU compatibilities, similar to what we do using gcc-macros.h in sysdeps/powerpc/powerpc64/le/dl-hwcap-check.h, sysdeps/s390/s390-64/dl-hwcap-check.h, sysdeps/x86/dl-get-cpu-features.c. This requires configure glibc withe the appropriate --with-rtld-early-cflags option, as GCC does not provide a way to force the compiler to use the baseline ISA once the defaults have changed (either within GCC itself, based on its configuration, or earlier on the command line). Thanks, Florian
On Wed, 17 Apr 2024 at 06:12, Palmer Dabbelt <palmer@rivosinc.com> wrote: > On Tue, 16 Apr 2024 00:01:31 PDT (-0700), michael.hudson@canonical.com > wrote: > > On Tue, 16 Apr 2024 at 07:28, Palmer Dabbelt <palmer@rivosinc.com> > wrote: > > > >> Unfortunately I don't think we can drop support for the other base ISAs: > >> the distros appear to be targeting rv64gc and that's the only common > >> base for most hardware that's out there. > >> > > > > This is certainly the case for Ubuntu for now but I certainly hope that > by > > 26.04 we are able to be a bit more aggressive here, fwiw. > > That'd be awesome. Any idea if something like this would help? Well having the non baseline builds tested and so not having everything explode as/when we shift the baseline would probably help, yes :-) > I think we're all a bit worried about being stuck with a very small > baseline and thus a bunch of vendor-specific behavior. Yes. That would be unfortunate. > That's certainly > the way things are trending right now in RISC-V land, it's already super > complicated to manage and it looks like it's going to get worse before > it gets better. > I think for Ubuntu we are obviously not interested in supporting every RISC-V core out there, we are only going to be aiming for what would be a "cortex a" CPU in the ARM world and I think the baseline for those is probably going to be a bit more than just rv64gc (like, we could almost certainly just copy Google's baseline for Android). > If we can pick up some extra testing burden now as a way to reduce > fragmentation then it's probably a win in the long run, even if we have > to go through a few generations of this before things calm down. If > you're planning on moving to something completely different, though, > then maybe it's not worth the extra builds. > We certainly don't have any concrete plans beyond "we don't want to be stuck on rv64gc forever". We also don't want to be building every package in Ubuntu for every core either! It's obviously hard to see where things are going but I can imagine a world where there are different baselines for different application domains in practice (e.g. iot, routers, servers) with different baselines in practice and it's worth building some core packages to target those. But who knows! Cheers, mwh
On 4/15/24 2:22 PM, Palmer Dabbelt wrote: > On Mon, 15 Apr 2024 13:05:53 PDT (-0700), enh@google.com wrote: >> On Mon, Apr 15, 2024 at 12:27 PM Palmer Dabbelt <palmer@rivosinc.com> >> wrote: >>> >>> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com> >>> --- >>> So this is very much an RFC. I had written a commit message as >>> >>> This will hopefully be a widely used configuration in the future, >>> but >>> it's not all that common right now. That said, let's be >>> proactive and >>> at start testing it. >>> >>> but I even that's kind of a strong statement here so I'm just stashing >>> it off to the side to be sure. >>> >>> A few of us were talking about the GCC meeting last week, but I figured >>> I'd send a glibc patch to at least start some discussion on the lists as >>> something similar had come up in the TLSDESC thread and a while ago when >>> we talked about glibc-hwcaps. >>> >>> I'm not really sure what the right thing to do here is: we don't have >>> broadly available hardware that supports these new extensions, but >>> they're pretty much table stakes for hardware that's competitive with >>> arm64/x86. Without B and V we're going to end up piling up a ton of >>> IFUNC-based routines to try and work around the base ISA, but those >>> extensions are so fundamental to good codegen that it doesn't feel like >>> we're going to get where we want with just IFUNCs. >>> >>> Unfortunately I don't think we can drop support for the other base ISAs: >>> the distros appear to be targeting rv64gc and that's the only common >>> base for most hardware that's out there. I guess we could drop support >>> for the rv*ima-based base ISAs, but I'm not sure that buys us much: >>> we've got a lot of embedded users so we won't be able to drop the GCC >>> support for soft float, and dropping the glibc support puts us in a >>> clunky spot for testing. >>> >>> So we're probably stuck in a bit of a performance hole for a while. >>> That said, we could at least put a stake in the ground and start testing >>> some target with some newer extensions. That way distros that want to >>> try a larger base than rv64gc have something that's getting tested and >>> is thus likely to work. For GCC we ended up adding some test targets >>> with pretty much all the non-embedded, but at least B and V seem like >>> the bare minimum. >>> >>> I'm not sure if all that is worth the extra build/test resource burden >>> for everyone, though, particularly given the lack of hardware. >> >> for Android -- which is in a similar situation -- having the build >> system enable these extensions been pretty useful for finding >> toolchain and qemu bugs, which seems useful in its own right, if only >> for CI purposes... > > Ya, that's kind of my theory here. Hopefully hardware vendors will > eventually align to some interesting set of extensions, maybe split > between a few markets, but at least we can do _something_ here. I'd be > super surprised if we end up with any sort of long-term successful > hardware that lacks B and V, so it seems like a safe target for testing. Definitely starting to see more B variants showing up. V variants are still quite scarce. I'm hoping the sweet spot is going to be rv64gcbv fairly soon, but I don't see it happening in time for the spring distro refreshes, obviously. jeff
diff --git a/scripts/build-many-glibcs.py b/scripts/build-many-glibcs.py index ecc743e672..21da46dc4a 100755 --- a/scripts/build-many-glibcs.py +++ b/scripts/build-many-glibcs.py @@ -411,7 +411,9 @@ class Context(object): os_name='linux-gnu', variant='rv64imafdc-lp64d', gcc_cfg=['--with-arch=rv64imafdc', '--with-abi=lp64d', - '--disable-multilib']) + '--disable-multilib']), + extra_glibcs=[{'variant': 'rv64gcbv-rv64gc-lp64d', + 'ccopts': '-march=rv64gcv_zba_zbb_zbs'}] self.add_config(arch='s390x', os_name='linux-gnu', glibcs=[{},
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com> --- So this is very much an RFC. I had written a commit message as This will hopefully be a widely used configuration in the future, but it's not all that common right now. That said, let's be proactive and at start testing it. but I even that's kind of a strong statement here so I'm just stashing it off to the side to be sure. A few of us were talking about the GCC meeting last week, but I figured I'd send a glibc patch to at least start some discussion on the lists as something similar had come up in the TLSDESC thread and a while ago when we talked about glibc-hwcaps. I'm not really sure what the right thing to do here is: we don't have broadly available hardware that supports these new extensions, but they're pretty much table stakes for hardware that's competitive with arm64/x86. Without B and V we're going to end up piling up a ton of IFUNC-based routines to try and work around the base ISA, but those extensions are so fundamental to good codegen that it doesn't feel like we're going to get where we want with just IFUNCs. Unfortunately I don't think we can drop support for the other base ISAs: the distros appear to be targeting rv64gc and that's the only common base for most hardware that's out there. I guess we could drop support for the rv*ima-based base ISAs, but I'm not sure that buys us much: we've got a lot of embedded users so we won't be able to drop the GCC support for soft float, and dropping the glibc support puts us in a clunky spot for testing. So we're probably stuck in a bit of a performance hole for a while. That said, we could at least put a stake in the ground and start testing some target with some newer extensions. That way distros that want to try a larger base than rv64gc have something that's getting tested and is thus likely to work. For GCC we ended up adding some test targets with pretty much all the non-embedded, but at least B and V seem like the bare minimum. I'm not sure if all that is worth the extra build/test resource burden for everyone, though, particularly given the lack of hardware. --- scripts/build-many-glibcs.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)