ncurses: make host-ncurses use host terminfo

Message ID 5f7fe697b92ac0145674.1531241163@cveaol6qa08.wv.mentorg.com
State Accepted
Headers show
Series
  • ncurses: make host-ncurses use host terminfo
Related show

Commit Message

Hollis Blanchard July 10, 2018, 4:46 p.m.
# HG changeset patch
# User Hollis Blanchard <hollis_blanchard@mentor.com>
# Date 1531239381 25200
#      Tue Jul 10 09:16:21 2018 -0700
# Node ID 5f7fe697b92ac0145674a6c96aad0f787b4bae32
# Parent  d71314cdccf1993ccdb05cddb16a5491f0bf723e
ncurses: make host-ncurses use host terminfo

Host GDB suffers a serious problem: pressing backspace (or ^W ^U or any other
"delete" key) results in a plain space being printed instead, making the
command prompt almost completely unusable.

That's because it's using host-ncurses, which embeds a path for the terminfo
database into the library itself. That path ends up being something like
/home/hollisb/buildroot.git/output/host/share/terminfo, which obviously doesn't
generally exist other hosts. ('relocate-sdk.sh' cannot and does not edit
binaries like libncurses.so.6, so doesn't resolve this problem.)

/usr/share/terminfo is a far better path to use, since it almost certainly
exists on the host. Theoretically, it could be from a different ncurses version
with incompatible terminfo database format, but this doesn't seem to be a
problem in practice. (Future patches could address the theoretical problem if
it actually appears in real life.)

This change allows buildroot's host gdb, which uses ncurses 6.x, to work on
RHEL5, RHEL6, and RHEL7, which all provide terminfo from ncurses 5.x.

Signed-off-by: Hollis Blanchard <hollis_blanchard@mentor.com>

Comments

Arnout Vandecappelle July 10, 2018, 7:52 p.m. | #1
On 10-07-18 18:46, Hollis Blanchard wrote:
> # HG changeset patch
> # User Hollis Blanchard <hollis_blanchard@mentor.com>
> # Date 1531239381 25200
> #      Tue Jul 10 09:16:21 2018 -0700
> # Node ID 5f7fe697b92ac0145674a6c96aad0f787b4bae32
> # Parent  d71314cdccf1993ccdb05cddb16a5491f0bf723e
> ncurses: make host-ncurses use host terminfo
> 
> Host GDB suffers a serious problem: pressing backspace (or ^W ^U or any other
> "delete" key) results in a plain space being printed instead, making the
> command prompt almost completely unusable.
> 
> That's because it's using host-ncurses, which embeds a path for the terminfo
> database into the library itself. That path ends up being something like
> /home/hollisb/buildroot.git/output/host/share/terminfo, which obviously doesn't
> generally exist other hosts. ('relocate-sdk.sh' cannot and does not edit
> binaries like libncurses.so.6, so doesn't resolve this problem.)
> 
> /usr/share/terminfo is a far better path to use, since it almost certainly
> exists on the host. Theoretically, it could be from a different ncurses version
> with incompatible terminfo database format, but this doesn't seem to be a
> problem in practice. (Future patches could address the theoretical problem if
> it actually appears in real life.)

 Indeed, the terminfo information is standardized AFAIU so this should indeed work.

Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

 Regards,
 Arnout

> This change allows buildroot's host gdb, which uses ncurses 6.x, to work on
> RHEL5, RHEL6, and RHEL7, which all provide terminfo from ncurses 5.x.
> 
> Signed-off-by: Hollis Blanchard <hollis_blanchard@mentor.com>
> 
> diff --git a/package/ncurses/ncurses.mk b/package/ncurses/ncurses.mk
> --- a/package/ncurses/ncurses.mk
> +++ b/package/ncurses/ncurses.mk
> @@ -153,6 +153,7 @@ HOST_NCURSES_CONF_OPTS = \
>  	--without-cxx \
>  	--without-cxx-binding \
>  	--without-ada \
> +	--with-default-terminfo-dir=/usr/share/terminfo \
>  	--without-normal
>  
>  $(eval $(autotools-package))
> _______________________________________________
> buildroot mailing list
> buildroot@busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
>
Thomas Petazzoni July 17, 2018, 2:40 p.m. | #2
Hello,

On Tue, 10 Jul 2018 09:46:03 -0700, Hollis Blanchard wrote:
> # HG changeset patch
> # User Hollis Blanchard <hollis_blanchard@mentor.com>
> # Date 1531239381 25200
> #      Tue Jul 10 09:16:21 2018 -0700
> # Node ID 5f7fe697b92ac0145674a6c96aad0f787b4bae32
> # Parent  d71314cdccf1993ccdb05cddb16a5491f0bf723e

This mercurial stuff is a bit annoying, as it becomes part of the
commit log. I removed it when applying the patch.

> ncurses: make host-ncurses use host terminfo
> 
> Host GDB suffers a serious problem: pressing backspace (or ^W ^U or any other
> "delete" key) results in a plain space being printed instead, making the
> command prompt almost completely unusable.
> 
> That's because it's using host-ncurses, which embeds a path for the terminfo
> database into the library itself. That path ends up being something like
> /home/hollisb/buildroot.git/output/host/share/terminfo, which obviously doesn't
> generally exist other hosts. ('relocate-sdk.sh' cannot and does not edit
> binaries like libncurses.so.6, so doesn't resolve this problem.)
> 
> /usr/share/terminfo is a far better path to use, since it almost certainly
> exists on the host. Theoretically, it could be from a different ncurses version
> with incompatible terminfo database format, but this doesn't seem to be a
> problem in practice. (Future patches could address the theoretical problem if
> it actually appears in real life.)
> 
> This change allows buildroot's host gdb, which uses ncurses 6.x, to work on
> RHEL5, RHEL6, and RHEL7, which all provide terminfo from ncurses 5.x.
> 
> Signed-off-by: Hollis Blanchard <hollis_blanchard@mentor.com>

Applied, thanks!

Thomas Petazzoni
Hollis Blanchard July 17, 2018, 4:53 p.m. | #3
On 07/17/2018 07:40 AM, Thomas Petazzoni wrote:
> Hello,
>
> On Tue, 10 Jul 2018 09:46:03 -0700, Hollis Blanchard wrote:
>> # HG changeset patch
>> # User Hollis Blanchard <hollis_blanchard@mentor.com>
>> # Date 1531239381 25200
>> #      Tue Jul 10 09:16:21 2018 -0700
>> # Node ID 5f7fe697b92ac0145674a6c96aad0f787b4bae32
>> # Parent  d71314cdccf1993ccdb05cddb16a5491f0bf723e
> This mercurial stuff is a bit annoying, as it becomes part of the
> commit log. I removed it when applying the patch.
You're right, and thank you for removing it.

Hollis Blanchard
Mentor Graphics Emulation Division
Peter Korsgaard July 19, 2018, 9:38 p.m. | #4
>>>>> "Hollis" == Hollis Blanchard <hollis_blanchard@mentor.com> writes:

 > # HG changeset patch
 > # User Hollis Blanchard <hollis_blanchard@mentor.com>
 > # Date 1531239381 25200
 > #      Tue Jul 10 09:16:21 2018 -0700
 > # Node ID 5f7fe697b92ac0145674a6c96aad0f787b4bae32
 > # Parent  d71314cdccf1993ccdb05cddb16a5491f0bf723e
 > ncurses: make host-ncurses use host terminfo

 > Host GDB suffers a serious problem: pressing backspace (or ^W ^U or any other
 > "delete" key) results in a plain space being printed instead, making the
 > command prompt almost completely unusable.

 > That's because it's using host-ncurses, which embeds a path for the terminfo
 > database into the library itself. That path ends up being something like
 > /home/hollisb/buildroot.git/output/host/share/terminfo, which obviously doesn't
 > generally exist other hosts. ('relocate-sdk.sh' cannot and does not edit
 > binaries like libncurses.so.6, so doesn't resolve this problem.)

 > /usr/share/terminfo is a far better path to use, since it almost certainly
 > exists on the host. Theoretically, it could be from a different ncurses version
 > with incompatible terminfo database format, but this doesn't seem to be a
 > problem in practice. (Future patches could address the theoretical problem if
 > it actually appears in real life.)

 > This change allows buildroot's host gdb, which uses ncurses 6.x, to work on
 > RHEL5, RHEL6, and RHEL7, which all provide terminfo from ncurses 5.x.

 > Signed-off-by: Hollis Blanchard <hollis_blanchard@mentor.com>

Committed to 2018.02.x and 2018.05.x, thanks.
Peter Korsgaard Aug. 8, 2018, 2:10 p.m. | #5
>>>>> "Peter" == Peter Korsgaard <peter@korsgaard.com> writes:

Hi,

 >> # HG changeset patch
 >> # User Hollis Blanchard <hollis_blanchard@mentor.com>
 >> # Date 1531239381 25200
 >> #      Tue Jul 10 09:16:21 2018 -0700
 >> # Node ID 5f7fe697b92ac0145674a6c96aad0f787b4bae32
 >> # Parent  d71314cdccf1993ccdb05cddb16a5491f0bf723e
 >> ncurses: make host-ncurses use host terminfo

 >> Host GDB suffers a serious problem: pressing backspace (or ^W ^U or any other
 >> "delete" key) results in a plain space being printed instead, making the
 >> command prompt almost completely unusable.

 >> That's because it's using host-ncurses, which embeds a path for the terminfo
 >> database into the library itself. That path ends up being something like
 >> /home/hollisb/buildroot.git/output/host/share/terminfo, which obviously doesn't
 >> generally exist other hosts. ('relocate-sdk.sh' cannot and does not edit
 >> binaries like libncurses.so.6, so doesn't resolve this problem.)

 >> /usr/share/terminfo is a far better path to use, since it almost certainly
 >> exists on the host. Theoretically, it could be from a different ncurses version
 >> with incompatible terminfo database format, but this doesn't seem to be a
 >> problem in practice. (Future patches could address the theoretical problem if
 >> it actually appears in real life.)

 >> This change allows buildroot's host gdb, which uses ncurses 6.x, to work on
 >> RHEL5, RHEL6, and RHEL7, which all provide terminfo from ncurses 5.x.

 >> Signed-off-by: Hollis Blanchard <hollis_blanchard@mentor.com>

 > Committed to 2018.02.x and 2018.05.x, thanks.

Hmm, this triggers an interesting issue with tic :/

** Building terminfo database, please wait...
Running sh ./shlib tic to install /usr/share/terminfo ...

        You may see messages regarding extended capabilities, e.g., AX.
        These are extended terminal capabilities which are compiled
        using
                tic -x
        If you have ncurses 4.2 applications, you should read the INSTALL
        document, and install the terminfo without the -x option.

"terminfo.tmp", line 21272, terminal 'v3220': /home/peko/.terminfo: permission denied (errno 30)

What happens here is that tic doesn't have write access to
/usr/share/terminfo (as we are not root), so instead it writes the db to
~/.terminfo.

We in general don't want buildroot to touch any files outside the
buildroot output directory (and $TMPDIR), but it normally isn't
enforced/checked by the autobuilders. For this specific setup with
building inside a container $HOME is mounted readonly, so it really
fails.

The question is how to fix it. The logical solution would be to set
TERMINFO to get it to put the database somewhere else, but that doesn't
work because run_tic.sh overwrites it:

# don't use user's TERMINFO variable
TERMINFO=${DESTDIR}$ticdir ; export TERMINFO

The only fix I can think of is:

HOST_NCURSES_MAKE_ENV += HOME=$(HOST_DIR)

But that isn't very nice. Any other ideas?
Arnout Vandecappelle Aug. 8, 2018, 8:37 p.m. | #6
On 08-08-18 16:10, Peter Korsgaard wrote:
>>>>>> "Peter" == Peter Korsgaard <peter@korsgaard.com> writes:
> 
> Hi,
> 
>  >> # HG changeset patch
>  >> # User Hollis Blanchard <hollis_blanchard@mentor.com>
>  >> # Date 1531239381 25200
>  >> #      Tue Jul 10 09:16:21 2018 -0700
>  >> # Node ID 5f7fe697b92ac0145674a6c96aad0f787b4bae32
>  >> # Parent  d71314cdccf1993ccdb05cddb16a5491f0bf723e
>  >> ncurses: make host-ncurses use host terminfo
> 
>  >> Host GDB suffers a serious problem: pressing backspace (or ^W ^U or any other
>  >> "delete" key) results in a plain space being printed instead, making the
>  >> command prompt almost completely unusable.
> 
>  >> That's because it's using host-ncurses, which embeds a path for the terminfo
>  >> database into the library itself. That path ends up being something like
>  >> /home/hollisb/buildroot.git/output/host/share/terminfo, which obviously doesn't
>  >> generally exist other hosts. ('relocate-sdk.sh' cannot and does not edit
>  >> binaries like libncurses.so.6, so doesn't resolve this problem.)
> 
>  >> /usr/share/terminfo is a far better path to use, since it almost certainly
>  >> exists on the host. Theoretically, it could be from a different ncurses version
>  >> with incompatible terminfo database format, but this doesn't seem to be a
>  >> problem in practice. (Future patches could address the theoretical problem if
>  >> it actually appears in real life.)
> 
>  >> This change allows buildroot's host gdb, which uses ncurses 6.x, to work on
>  >> RHEL5, RHEL6, and RHEL7, which all provide terminfo from ncurses 5.x.
> 
>  >> Signed-off-by: Hollis Blanchard <hollis_blanchard@mentor.com>
> 
>  > Committed to 2018.02.x and 2018.05.x, thanks.
> 
> Hmm, this triggers an interesting issue with tic :/
> 
> ** Building terminfo database, please wait...
> Running sh ./shlib tic to install /usr/share/terminfo ...
> 
>         You may see messages regarding extended capabilities, e.g., AX.
>         These are extended terminal capabilities which are compiled
>         using
>                 tic -x
>         If you have ncurses 4.2 applications, you should read the INSTALL
>         document, and install the terminfo without the -x option.
> 
> "terminfo.tmp", line 21272, terminal 'v3220': /home/peko/.terminfo: permission denied (errno 30)
> 
> What happens here is that tic doesn't have write access to
> /usr/share/terminfo (as we are not root), so instead it writes the db to
> ~/.terminfo.
> 
> We in general don't want buildroot to touch any files outside the
> buildroot output directory (and $TMPDIR), but it normally isn't
> enforced/checked by the autobuilders. For this specific setup with
> building inside a container $HOME is mounted readonly, so it really
> fails.

 Actually, sounds like a good idea to have an autobuilder that does exactly
this: nothing writeable outside of OUTPUT_DIR.

> 
> The question is how to fix it. The logical solution would be to set
> TERMINFO to get it to put the database somewhere else, but that doesn't
> work because run_tic.sh overwrites it:
> 
> # don't use user's TERMINFO variable
> TERMINFO=${DESTDIR}$ticdir ; export TERMINFO
           ^^^^^^^^^^
 So, set DESTDIR?

 Or actually, why is this script called at all? The whole point of Hollis's
patch was to use the system's terminfo, so why should we generate the terminfo
at all? In other words, can't we just pass --disable-db-install ?

 Regards,
 Arnout

> 
> The only fix I can think of is:
> 
> HOST_NCURSES_MAKE_ENV += HOME=$(HOST_DIR)
> 
> But that isn't very nice. Any other ideas?
>
Hollis Blanchard Aug. 9, 2018, 12:19 a.m. | #7
On 08/08/2018 01:37 PM, Arnout Vandecappelle wrote:
> On 08-08-18 16:10, Peter Korsgaard wrote:
>> The question is how to fix it. The logical solution would be to set
>> TERMINFO to get it to put the database somewhere else, but that doesn't
>> work because run_tic.sh overwrites it:
>>
>> # don't use user's TERMINFO variable
>> TERMINFO=${DESTDIR}$ticdir ; export TERMINFO
>             ^^^^^^^^^^
>   So, set DESTDIR?
>
>   Or actually, why is this script called at all? The whole point of Hollis's
> patch was to use the system's terminfo, so why should we generate the terminfo
> at all? In other words, can't we just pass --disable-db-install ?
I agree that we don't want to generate terminfo in the /host/ ncurses. I 
can confirm that adding --disable-db-install fixes the build behavior 
that I also see (building host-ncurses writes to my ~/.terminfo).

I can't imagine a problem, but I just haven't yet been able to rebuild a 
full toolchain with --disable-db-install.

Hollis Blanchard
Mentor Graphics Emulation Division
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    On 08/08/2018 01:37 PM, Arnout Vandecappelle wrote:
    <blockquote type="cite"
      cite="mid:4b28a0d6-dc95-4d3b-6ecb-a4607935f6e2@mind.be">
      <pre wrap="">
On 08-08-18 16:10, Peter Korsgaard wrote:
</pre>
      <blockquote type="cite">
        <pre wrap="">The question is how to fix it. The logical solution would be to set
TERMINFO to get it to put the database somewhere else, but that doesn't
work because run_tic.sh overwrites it:

# don't use user's TERMINFO variable
TERMINFO=${DESTDIR}$ticdir ; export TERMINFO
</pre>
      </blockquote>
      <pre wrap="">           ^^^^^^^^^^
 So, set DESTDIR?

 Or actually, why is this script called at all? The whole point of Hollis's
patch was to use the system's terminfo, so why should we generate the terminfo
at all? In other words, can't we just pass --disable-db-install ?
</pre>
    </blockquote>
    I agree that we don't want to generate terminfo in the <i>host</i>
    ncurses. I can confirm that adding --disable-db-install fixes the
    build behavior that I also see (building host-ncurses writes to my
    ~/.terminfo).<br>
    <br>
    I can't imagine a problem, but I just haven't yet been able to
    rebuild a full toolchain with --disable-db-install.<br>
    <pre class="moz-signature" cols="72">Hollis Blanchard
Mentor Graphics Emulation Division</pre>
  </body>
</html>
Peter Korsgaard Aug. 9, 2018, 2:28 p.m. | #8
>>>>> "Arnout" == Arnout Vandecappelle <arnout@mind.be> writes:

Hi,

 >> We in general don't want buildroot to touch any files outside the
 >> buildroot output directory (and $TMPDIR), but it normally isn't
 >> enforced/checked by the autobuilders. For this specific setup with
 >> building inside a container $HOME is mounted readonly, so it really
 >> fails.

 >  Actually, sounds like a good idea to have an autobuilder that does exactly
 > this: nothing writeable outside of OUTPUT_DIR.

Indeed. I will try to find (time and cpu) cycles to setup such a
builder.


 >> 
 >> The question is how to fix it. The logical solution would be to set
 >> TERMINFO to get it to put the database somewhere else, but that doesn't
 >> work because run_tic.sh overwrites it:
 >> 
 >> # don't use user's TERMINFO variable
 >> TERMINFO=${DESTDIR}$ticdir ; export TERMINFO
 >            ^^^^^^^^^^
 >  So, set DESTDIR?

That isn't really handy as DESTDIR is used in a bunch of other places
and this is built as part of make install.


 >  Or actually, why is this script called at all? The whole point of Hollis's
 > patch was to use the system's terminfo, so why should we generate the terminfo
 > at all? In other words, can't we just pass --disable-db-install ?

Ahh, I hadn't noticed that option. That indeed works, I'll send a patch
- Thanks!

Patch

diff --git a/package/ncurses/ncurses.mk b/package/ncurses/ncurses.mk
--- a/package/ncurses/ncurses.mk
+++ b/package/ncurses/ncurses.mk
@@ -153,6 +153,7 @@  HOST_NCURSES_CONF_OPTS = \
 	--without-cxx \
 	--without-cxx-binding \
 	--without-ada \
+	--with-default-terminfo-dir=/usr/share/terminfo \
 	--without-normal
 
 $(eval $(autotools-package))