| Message ID | 78b7917b-cadd-402c-a3c3-35fbcc337b81@www.fastmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | Support the new ("v0") mangling scheme in rust-demangle. | expand |
Ping: https://gcc.gnu.org/pipermail/gcc-patches/2020-March/542012.html Thanks, - Eddy B. On Fri, Mar 13, 2020, at 10:28 PM, Eduard-Mihai Burtescu wrote: > This is the libiberty (mainly for binutils/gdb) counterpart of > https://github.com/alexcrichton/rustc-demangle/pull/23. > > Relevant links for the new Rust mangling scheme (aka "v0"): > * Rust RFC: https://github.com/rust-lang/rfcs/pull/2603 > * tracking issue: https://github.com/rust-lang/rust/issues/60705 > * implementation: https://github.com/rust-lang/rust/pull/57967 > > This implementation includes full support for UTF-8 identifiers > via punycode, so I've included a testcase for that as well. > (Let me know if it causes any issues and I'll take it out) > > Last year I've submitted several small patches to rust-demangle > in preparation for upstreaming the entire new demangler, and > feedback from that has proven useful. > For example, I started with error-handling macros, but instead > the code now has "rdm->errored = 1;" before several returns/gotos. > > The patch is attached instead of inline, as it's over 1000 lines long. > > Bootstrapped and tested on x86_64-unknown-linux-gnu. > > Also, I have no commit access, so I'd be thankful if > someone would commit this for me if/once approved. > Attachments: > * 0001-Support-the-new-v0-mangling-scheme-in-rust-demangle.patch
Ping 2: https://gcc.gnu.org/pipermail/gcc-patches/2020-March/542012.html Thanks, - Eddy B. On Fri, Mar 13, 2020, at 10:28 PM, Eduard-Mihai Burtescu wrote: > This is the libiberty (mainly for binutils/gdb) counterpart of > https://github.com/alexcrichton/rustc-demangle/pull/23. > > Relevant links for the new Rust mangling scheme (aka "v0"): > * Rust RFC: https://github.com/rust-lang/rfcs/pull/2603 > * tracking issue: https://github.com/rust-lang/rust/issues/60705 > * implementation: https://github.com/rust-lang/rust/pull/57967 > > This implementation includes full support for UTF-8 identifiers > via punycode, so I've included a testcase for that as well. > (Let me know if it causes any issues and I'll take it out) > > Last year I've submitted several small patches to rust-demangle > in preparation for upstreaming the entire new demangler, and > feedback from that has proven useful. > For example, I started with error-handling macros, but instead > the code now has "rdm->errored = 1;" before several returns/gotos. > > The patch is attached instead of inline, as it's over 1000 lines long. > > Bootstrapped and tested on x86_64-unknown-linux-gnu. > > Also, I have no commit access, so I'd be thankful if > someone would commit this for me if/once approved. > Attachments: > * 0001-Support-the-new-v0-mangling-scheme-in-rust-demangle.patch
Ping 3: https://gcc.gnu.org/pipermail/gcc-patches/2020-March/542012.html Thanks, - Eddy B. On Tue, Apr 7, 2020, at 00:52, Eduard-Mihai Burtescu wrote: > Ping 2: https://gcc.gnu.org/pipermail/gcc-patches/2020-March/542012.html > > Thanks, > - Eddy B. > > On Fri, Mar 13, 2020, at 10:28 PM, Eduard-Mihai Burtescu wrote: > > This is the libiberty (mainly for binutils/gdb) counterpart of > > https://github.com/alexcrichton/rustc-demangle/pull/23. > > > > Relevant links for the new Rust mangling scheme (aka "v0"): > > * Rust RFC: https://github.com/rust-lang/rfcs/pull/2603 > > * tracking issue: https://github.com/rust-lang/rust/issues/60705 > > * implementation: https://github.com/rust-lang/rust/pull/57967 > > > > This implementation includes full support for UTF-8 identifiers > > via punycode, so I've included a testcase for that as well. > > (Let me know if it causes any issues and I'll take it out) > > > > Last year I've submitted several small patches to rust-demangle > > in preparation for upstreaming the entire new demangler, and > > feedback from that has proven useful. > > For example, I started with error-handling macros, but instead > > the code now has "rdm->errored = 1;" before several returns/gotos. > > > > The patch is attached instead of inline, as it's over 1000 lines long. > > > > Bootstrapped and tested on x86_64-unknown-linux-gnu. > > > > Also, I have no commit access, so I'd be thankful if > > someone would commit this for me if/once approved. > > Attachments: > > * 0001-Support-the-new-v0-mangling-scheme-in-rust-demangle.patch
Ping 4: https://gcc.gnu.org/pipermail/gcc-patches/2020-March/542012.html Thanks, - Eddy B. On Mon, Apr 13, 2020, at 05:52, Eduard-Mihai Burtescu wrote: > Ping 3: https://gcc.gnu.org/pipermail/gcc-patches/2020-March/542012.html > > Thanks, > - Eddy B. > > On Tue, Apr 7, 2020, at 00:52, Eduard-Mihai Burtescu wrote: > > Ping 2: https://gcc.gnu.org/pipermail/gcc-patches/2020-March/542012.html > > > > Thanks, > > - Eddy B. > > > > On Fri, Mar 13, 2020, at 10:28 PM, Eduard-Mihai Burtescu wrote: > > > This is the libiberty (mainly for binutils/gdb) counterpart of > > > https://github.com/alexcrichton/rustc-demangle/pull/23. > > > > > > Relevant links for the new Rust mangling scheme (aka "v0"): > > > * Rust RFC: https://github.com/rust-lang/rfcs/pull/2603 > > > * tracking issue: https://github.com/rust-lang/rust/issues/60705 > > > * implementation: https://github.com/rust-lang/rust/pull/57967 > > > > > > This implementation includes full support for UTF-8 identifiers > > > via punycode, so I've included a testcase for that as well. > > > (Let me know if it causes any issues and I'll take it out) > > > > > > Last year I've submitted several small patches to rust-demangle > > > in preparation for upstreaming the entire new demangler, and > > > feedback from that has proven useful. > > > For example, I started with error-handling macros, but instead > > > the code now has "rdm->errored = 1;" before several returns/gotos. > > > > > > The patch is attached instead of inline, as it's over 1000 lines long. > > > > > > Bootstrapped and tested on x86_64-unknown-linux-gnu. > > > > > > Also, I have no commit access, so I'd be thankful if > > > someone would commit this for me if/once approved. > > > Attachments: > > > * 0001-Support-the-new-v0-mangling-scheme-in-rust-demangle.patch
Ian, are you able to review this? I saw that you reviewed many of the prior changes to the Rust demangler. If not, can you suggest someone who can? Thanks very much. Nikhil
On 10/28/20 5:22 PM, Nikhil Benesch wrote: > Ian, are you able to review this? I saw that you reviewed many of the > prior changes to the Rust demangler. > > If not, can you suggest someone who can? > > Thanks very much. > Nikhil I seem to have failed to convince my email client to set the appropriate reply to headers. This is the patch to which I was referring: https://gcc.gnu.org/pipermail/gcc-patches/2020-March/542488.html
FWIW, the patch has become slightly outdated compared to the Rust upstream, so if someone wants to review it I should prepare a new version. The changes would be for the MVP version of "const generics" (Rust's equivalent to C++ templates with value parameters, enabling e.g. types like `CustomArray<T, 123>`), which supports a few different kinds of primitive values, whereas the original patch only handled unsigned integers. Thanks, - Eddy B. On Wed, Oct 28, 2020, at 23:25, Nikhil Benesch wrote: > On 10/28/20 5:22 PM, Nikhil Benesch wrote: > > Ian, are you able to review this? I saw that you reviewed many of the > > prior changes to the Rust demangler. > > > > If not, can you suggest someone who can? > > > > Thanks very much. > > Nikhil > > I seem to have failed to convince my email client to set the appropriate > reply to headers. This is the patch to which I was referring: > > https://gcc.gnu.org/pipermail/gcc-patches/2020-March/542488.html >
I think it is mostly a matter of snagging some of Ian's limited time. I suspect it is still worthwhile to try to get the original patch reviewed and merged, because then any follow-up changes for const generics support will be smaller and easier to review. On Wed, Oct 28, 2020 at 5:48 PM Eduard-Mihai Burtescu <eddyb@lyken.rs> wrote: > > FWIW, the patch has become slightly outdated compared to the Rust upstream, so if someone wants to review it I should prepare a new version. > > The changes would be for the MVP version of "const generics" (Rust's equivalent to C++ templates with value parameters, enabling e.g. types like `CustomArray<T, 123>`), which supports a few different kinds of primitive values, whereas the original patch only handled unsigned integers. > > Thanks, > - Eddy B. > > On Wed, Oct 28, 2020, at 23:25, Nikhil Benesch wrote: > > On 10/28/20 5:22 PM, Nikhil Benesch wrote: > > > Ian, are you able to review this? I saw that you reviewed many of the > > > prior changes to the Rust demangler. > > > > > > If not, can you suggest someone who can? > > > > > > Thanks very much. > > > Nikhil > > > > I seem to have failed to convince my email client to set the appropriate > > reply to headers. This is the patch to which I was referring: > > > > https://gcc.gnu.org/pipermail/gcc-patches/2020-March/542488.html > >
On Wed, Oct 28, 2020 at 3:22 PM Nikhil Benesch via Gcc-patches <gcc-patches@gcc.gnu.org> wrote: > > I think it is mostly a matter of snagging some of Ian's limited time. > I suspect it is still worthwhile to try to get the original patch > reviewed and merged, because then any follow-up changes for const > generics support will be smaller and easier to review. Yeah, it's not a good idea for me to be a blocker for changes to Rust code. I took a quick look at the original patch. The calls to sprintf should use snprintf instead. Other than that it seems fine though I have no idea whether it's correct. Ian > On Wed, Oct 28, 2020 at 5:48 PM Eduard-Mihai Burtescu <eddyb@lyken.rs> wrote: > > > > FWIW, the patch has become slightly outdated compared to the Rust upstream, so if someone wants to review it I should prepare a new version. > > > > The changes would be for the MVP version of "const generics" (Rust's equivalent to C++ templates with value parameters, enabling e.g. types like `CustomArray<T, 123>`), which supports a few different kinds of primitive values, whereas the original patch only handled unsigned integers. > > > > Thanks, > > - Eddy B. > > > > On Wed, Oct 28, 2020, at 23:25, Nikhil Benesch wrote: > > > On 10/28/20 5:22 PM, Nikhil Benesch wrote: > > > > Ian, are you able to review this? I saw that you reviewed many of the > > > > prior changes to the Rust demangler. > > > > > > > > If not, can you suggest someone who can? > > > > > > > > Thanks very much. > > > > Nikhil > > > > > > I seem to have failed to convince my email client to set the appropriate > > > reply to headers. This is the patch to which I was referring: > > > > > > https://gcc.gnu.org/pipermail/gcc-patches/2020-March/542488.html > > >
Hi Ian,
Thanks for replying! (Also I'm really sorry for the missing hard wrapping
in my earlier email, I shouldn't have sent that from my phone)
Regarding this being a blocker: we've already made the changes to Rust, and
the new ("v0") mangling format continues to remain opt-in and nightly-only,
as long as no distributions ship gdb & other tools, capable of demangling it.
I agree that landing the extra changes later on top should be fine anyway,
though when I make the sprintf -> snprintf change, I could provide the extra
changes as well (either as a combined patch, or as a second tiny patch).
Sadly I don't think I can get to either until next week, hope that's okay.
For correctness, I've been running this test after every change I make, both
with the standalone copy (in the same gist), and a local build of libiberty:
https://gist.github.com/eddyb/c41a69378750a433767cf53fe2316768#file-test-rs-L85-L116
The input is almost 1 million symbol names (collected from a full build of
the Rust compiler and Cargo), and the test ensures that the C implementation
produces the same result as the original "v0" demangler (written in Rust).
All that data is around 1GB but maybe I should try to upload it somewhere
public so that anyone can repeat this procedure, or I could even redo the
collection process (in order to gather even more / newer symbol names).
While this may not stress every aspect of the demangler, it is comprehensive
enough that it found several kinds of bugs in a 3rd party implementation
(which the IllumOS project wrote from scratch, instead of porting ours).
Thanks,
- Eddy B.
On Thu, Oct 29, 2020, at 00:45, Ian Lance Taylor wrote:
> On Wed, Oct 28, 2020 at 3:22 PM Nikhil Benesch via Gcc-patches
> <gcc-patches@gcc.gnu.org> wrote:
> >
> > I think it is mostly a matter of snagging some of Ian's limited time.
> > I suspect it is still worthwhile to try to get the original patch
> > reviewed and merged, because then any follow-up changes for const
> > generics support will be smaller and easier to review.
>
> Yeah, it's not a good idea for me to be a blocker for changes to Rust code.
>
> I took a quick look at the original patch. The calls to sprintf
> should use snprintf instead.
>
> Other than that it seems fine though I have no idea whether it's correct.
>
> Ian
>
>
>
> > On Wed, Oct 28, 2020 at 5:48 PM Eduard-Mihai Burtescu <eddyb@lyken.rs> wrote:
> > >
> > > FWIW, the patch has become slightly outdated compared to the Rust upstream, so if someone wants to review it I should prepare a new version.
> > >
> > > The changes would be for the MVP version of "const generics" (Rust's equivalent to C++ templates with value parameters, enabling e.g. types like `CustomArray<T, 123>`), which supports a few different kinds of primitive values, whereas the original patch only handled unsigned integers.
> > >
> > > Thanks,
> > > - Eddy B.
> > >
> > > On Wed, Oct 28, 2020, at 23:25, Nikhil Benesch wrote:
> > > > On 10/28/20 5:22 PM, Nikhil Benesch wrote:
> > > > > Ian, are you able to review this? I saw that you reviewed many of the
> > > > > prior changes to the Rust demangler.
> > > > >
> > > > > If not, can you suggest someone who can?
> > > > >
> > > > > Thanks very much.
> > > > > Nikhil
> > > >
> > > > I seem to have failed to convince my email client to set the appropriate
> > > > reply to headers. This is the patch to which I was referring:
> > > >
> > > > https://gcc.gnu.org/pipermail/gcc-patches/2020-March/542488.html
> > > >
>
On Wed, Oct 28, 2020 at 7:53 PM Eduard-Mihai Burtescu <eddyb@lyken.rs> wrote: > I agree that landing the extra changes later on top should be fine anyway, > though when I make the sprintf -> snprintf change, I could provide the extra > changes as well (either as a combined patch, or as a second tiny patch). > > Sadly I don't think I can get to either until next week, hope that's okay. I can make the sprintf -> snprintf change as early as tomorrow. I suspect I can also make the substantive const generics change, based on the Rust implementation, though that's less of a promise. > All that data is around 1GB but maybe I should try to upload it somewhere > public so that anyone can repeat this procedure, or I could even redo the > collection process (in order to gather even more / newer symbol names). Sharing that file would be very appreciated. I imagine sharing it publicly would be best, but if finding hosting is difficult, you could upload directly to my Dropbox: https://www.dropbox.com/request/PdJDiYQHs9poezwIHFV9 Nikhil
On 10/29/20 12:16 AM, Nikhil Benesch wrote: > On Wed, Oct 28, 2020 at 7:53 PM Eduard-Mihai Burtescu <eddyb@lyken.rs> wrote: >> I agree that landing the extra changes later on top should be fine anyway, >> though when I make the sprintf -> snprintf change, I could provide the extra >> changes as well (either as a combined patch, or as a second tiny patch). >> >> Sadly I don't think I can get to either until next week, hope that's okay. > > I can make the sprintf -> snprintf change as early as tomorrow. > > I suspect I can also make the substantive const generics change, based > on the Rust implementation, though that's less of a promise. Attached is an updated patch with both of the aforementioned changes. The demangling of constants matches the demangling in rustc-demangle library as best as I could manage. The strategy of demangling constant chars via `format!("{:?}", char)` in Rust makes life hard for us in C, because there is no easy way to replicate Rust's debug formatting for chars. (Unless GCC has a Unicode library handy somewhere.) In the course of this work I noticed several inconsistencies in how rustc-demangle and libiberty were demangling some legacy Rust symbols. I fixed those and added test cases, though the fixes required removing some of the fast checks for whether a given symbol is a valid Rust symbol. For ease of review, eddyb, I also attached the diff from your last diff to mine. Since I don't have your collection of symbols I generated my own by running nm on the materialized binary from https://github.com/MaterializeInc/materialize. Let me know how I can help. I'm happy to address review feedback myself, or I'm happy to hand things back over to you, eddyb. Nikhil diff --git a/rust-demangle.c b/rust-demangle.c index d604b3c..5c0a917 100644 --- a/rust-demangle.c +++ b/rust-demangle.c @@ -143,6 +143,35 @@ parse_disambiguator (struct rust_demangler *rdm) return parse_opt_integer_62 (rdm, 's'); } +static size_t +parse_hex_nibbles (struct rust_demangler *rdm, uint64_t *value) +{ + char c; + size_t hex_len; + + hex_len = 0; + *value = 0; + + while (!eat (rdm, '_')) + { + *value <<= 4; + + c = next (rdm); + if (ISDIGIT (c)) + *value |= c - '0'; + else if (c >= 'a' && c <= 'f') + *value |= 10 + (c - 'a'); + else + { + rdm->errored = 1; + return 0; + } + hex_len++; + } + + return hex_len; +} + struct rust_mangled_ident { /* ASCII part of the identifier. */ @@ -240,7 +269,7 @@ static void print_uint64 (struct rust_demangler *rdm, uint64_t x) { char s[21]; - sprintf (s, "%" PRIu64, x); + snprintf (s, 21, "%" PRIu64, x); PRINT (s); } @@ -248,7 +277,7 @@ static void print_uint64_hex (struct rust_demangler *rdm, uint64_t x) { char s[17]; - sprintf (s, "%" PRIx64, x); + snprintf (s, 17, "%" PRIx64, x); PRINT (s); } @@ -380,8 +409,7 @@ print_ident (struct rust_demangler *rdm, struct rust_mangled_ident ident) } else { - /* "." becomes "-" */ - PRINT ("-"); + PRINT ("."); len = 1; } } @@ -591,6 +619,9 @@ static int demangle_path_maybe_open_generics (struct rust_demangler *rdm); static void demangle_dyn_trait (struct rust_demangler *rdm); static void demangle_const (struct rust_demangler *rdm); static void demangle_const_uint (struct rust_demangler *rdm); +static void demangle_const_int (struct rust_demangler *rdm); +static void demangle_const_bool (struct rust_demangler *rdm); +static void demangle_const_char (struct rust_demangler *rdm); /* Optionally enter a binder ('G') for late-bound lifetimes, printing e.g. `for<'a, 'b> `, and make those lifetimes visible @@ -1089,6 +1120,11 @@ demangle_const (struct rust_demangler *rdm) ty_tag = next (rdm); switch (ty_tag) { + /* Placeholder. */ + case 'p': + PRINT ("_"); + return; + /* Unsigned integer types. */ case 'h': case 't': @@ -1096,6 +1132,27 @@ demangle_const (struct rust_demangler *rdm) case 'y': case 'o': case 'j': + demangle_const_uint (rdm); + break; + + /* Signed integer types. */ + case 'a': + case 's': + case 'l': + case 'x': + case 'n': + case 'i': + demangle_const_int (rdm); + break; + + /* Boolean. */ + case 'b': + demangle_const_bool (rdm); + break; + + /* Character. */ + case 'c': + demangle_const_char (rdm); break; default: @@ -1103,10 +1160,8 @@ demangle_const (struct rust_demangler *rdm) return; } - if (eat (rdm, 'p')) - PRINT ("_"); - else - demangle_const_uint (rdm); + if (rdm->errored) + return; if (rdm->verbose) { @@ -1118,74 +1173,121 @@ demangle_const (struct rust_demangler *rdm) static void demangle_const_uint (struct rust_demangler *rdm) { - char c; size_t hex_len; uint64_t value; if (rdm->errored) return; - value = 0; - hex_len = 0; - while (!eat (rdm, '_')) - { - value <<= 4; + hex_len = parse_hex_nibbles (rdm, &value); - c = next (rdm); - if (ISDIGIT (c)) - value |= c - '0'; - else if (c >= 'a' && c <= 'f') - value |= 10 + (c - 'a'); - else - { - rdm->errored = 1; - return; - } - hex_len++; - } - - /* Print anything that doesn't fit in `uint64_t` verbatim. */ if (hex_len > 16) { + /* Print anything that doesn't fit in `uint64_t` verbatim. */ PRINT ("0x"); print_str (rdm, rdm->sym + (rdm->next - hex_len), hex_len); + } + else if (hex_len > 0) + print_uint64 (rdm, value); + else + rdm->errored = 1; +} + +static void +demangle_const_int (struct rust_demangler *rdm) +{ + if (eat (rdm, 'n')) + PRINT ("-"); + demangle_const_uint (rdm); +} + +static void +demangle_const_bool (struct rust_demangler *rdm) +{ + uint64_t value; + + if (parse_hex_nibbles (rdm, &value) != 1) + { + rdm->errored = 1; + return; + } + + if (value == 0) + PRINT ("false"); + else if (value == 1) + PRINT ("true"); + else + rdm->errored = 1; +} + +static void +demangle_const_char (struct rust_demangler *rdm) +{ + size_t hex_len; + uint64_t value; + + hex_len = parse_hex_nibbles (rdm, &value); + + if (hex_len == 0 || hex_len > 8) + { + rdm->errored = 1; return; } - print_uint64 (rdm, value); + /* Match Rust's character "debug" output as best as we can. */ + PRINT ("'"); + if (value == '\t') + PRINT ("\\t"); + else if (value == '\r') + PRINT ("\\r"); + else if (value == '\n') + PRINT ("\\n"); + else if (value > ' ' && value < '~') + /* Rust also considers many non-ASCII codepoints to be printable, but + that logic is not easily ported to C. */ + print_str (rdm, (char *) &value, 1); + else + { + PRINT ("\\u{"); + print_uint64_hex (rdm, value); + PRINT ("}"); + } + PRINT ("'"); } -/* A legacy hash is the prefix "h" followed by 16 lowercase hex digits. - The hex digits must contain at least 5 distinct digits. */ +/* A legacy hash is the prefix "h" followed by any number of lowercase + hex digits. (The compiler will always generate 16 hex digits, but + demanglers are expected to accept any number of digits.) */ static int is_legacy_prefixed_hash (struct rust_mangled_ident ident) { - uint16_t seen; - int nibble; - size_t i, count; + size_t i; - if (ident.ascii_len != 17 || ident.ascii[0] != 'h') + if (ident.ascii[0] != 'h') return 0; - seen = 0; - for (i = 0; i < 16; i++) - { - nibble = decode_lower_hex_nibble (ident.ascii[1 + i]); - if (nibble < 0) - return 0; - seen |= (uint16_t)1 << nibble; - } + for (i = 1; i < ident.ascii_len; i++) + if (decode_lower_hex_nibble (ident.ascii[i]) < 0) + return 0; - /* Count how many distinct digits were seen. */ - count = 0; - while (seen) - { - if (seen & 1) - count++; - seen >>= 1; - } + return 1; +} - return count >= 5; +/* Consumes a suffix of the form ".llvm.XXXXXX". */ +static void +eat_llvm_suffix (struct rust_demangler *rdm) +{ + size_t i; + char c; + + if (strncmp (&rdm->sym[rdm->next], ".llvm.", 6) != 0) + return; + + for (i = rdm->next + 6; (c = rdm->sym[i]); i++) + if ((c < '0' || c > '9') && (c < 'A' || c > 'F') && c != '@') + return; + + rdm->next = i; } int @@ -1224,16 +1326,17 @@ rust_demangle_callback (const char *mangled, int options, if (rdm.version != -1 && !ISUPPER (rdm.sym[0])) return 0; - /* Rust symbols (v0) use only [_0-9a-zA-Z] characters. */ + /* Rust symbols (v0) use only [_0-9a-zA-Z] characters, but LLVM might add + a suffix containing [.@] characters. */ for (p = rdm.sym; *p; p++) { rdm.sym_len++; - if (*p == '_' || ISALNUM (*p)) + if (*p == '_' || *p == '.' || *p == '@' || ISALNUM (*p)) continue; - /* Legacy Rust symbols can also contain [.:$] characters. */ - if (rdm.version == -1 && (*p == '$' || *p == '.' || *p == ':')) + /* Legacy Rust symbols can also contain [:$] characters. */ + if (rdm.version == -1 && (*p == '$' || *p == ':')) continue; return 0; @@ -1242,64 +1345,55 @@ rust_demangle_callback (const char *mangled, int options, /* Legacy Rust symbols need to be handled separately. */ if (rdm.version == -1) { - /* Legacy Rust symbols always end with E. */ - if (!(rdm.sym_len > 0 && rdm.sym[rdm.sym_len - 1] == 'E')) + /* Print the first component of the path. */ + ident = parse_ident (&rdm); + if (rdm.errored || !ident.ascii) return 0; - rdm.sym_len--; + print_ident (&rdm, ident); - /* Legacy Rust symbols also always end with a path segment - that encodes a 16 hex digit hash, i.e. '17h[a-f0-9]{16}'. - This early check, before any parse_ident calls, should - quickly filter out most C++ symbols unrelated to Rust. */ - if (!(rdm.sym_len > 19 - && !memcmp (&rdm.sym[rdm.sym_len - 19], "17h", 3))) - return 0; - - do + while (!eat (&rdm, 'E')) { ident = parse_ident (&rdm); if (rdm.errored || !ident.ascii) return 0; - } - while (rdm.next < rdm.sym_len); - /* The last path segment should be the hash. */ - if (!is_legacy_prefixed_hash (ident)) - return 0; - - /* Reset the state for a second pass, to print the symbol. */ - rdm.next = 0; - if (!rdm.verbose && rdm.sym_len > 19) - { - /* Hide the last segment, containing the hash, if not verbose. */ - rdm.sym_len -= 19; - } - - do - { - if (rdm.next > 0) - print_str (&rdm, "::", 2); - - ident = parse_ident (&rdm); - print_ident (&rdm, ident); - } - while (rdm.next < rdm.sym_len); + /* Print the path component, unless it's the last component, we're + in verbose mode, and the component looks like a hash. */ + if (peek (&rdm) != 'E' + || rdm.verbose + || !is_legacy_prefixed_hash (ident)) + { + print_str (&rdm, "::", 2); + print_ident (&rdm, ident); + } + } } else { demangle_path (&rdm, 1); /* Skip instantiating crate. */ - if (!rdm.errored && rdm.next < rdm.sym_len) + if (!rdm.errored && peek (&rdm) >= 'A' && peek (&rdm) <= 'Z') { rdm.skipping_printing = 1; demangle_path (&rdm, 0); + rdm.skipping_printing = 0; } + } + + /* Suppress suffixes of the form ".llvm.XXXXXX". */ + eat_llvm_suffix (&rdm); - /* It's an error to not reach the end. */ - rdm.errored |= rdm.next != rdm.sym_len; + /* But output other suffixes that start with a dot. */ + if (peek (&rdm) == '.') + { + print_str (&rdm, &rdm.sym[rdm.next], rdm.sym_len - rdm.next); + rdm.next = rdm.sym_len; } + /* It's an error to not reach the end. */ + rdm.errored |= rdm.next != rdm.sym_len; + return !rdm.errored; } diff --git a/libiberty/rust-demangle.c b/libiberty/rust-demangle.c index b87365c85fe..9a457744b87 100644 --- a/libiberty/rust-demangle.c +++ b/libiberty/rust-demangle.c @@ -64,11 +64,16 @@ struct rust_demangler /* Non-zero if any error occurred. */ int errored; + /* Non-zero if nothing should be printed. */ + int skipping_printing; + /* Non-zero if printing should be verbose (e.g. include hashes). */ int verbose; /* Rust mangling version, with legacy mangling being -1. */ int version; + + uint64_t bound_lifetime_depth; }; /* Parsing functions. */ @@ -81,6 +86,18 @@ peek (const struct rust_demangler *rdm) return 0; } +static int +eat (struct rust_demangler *rdm, char c) +{ + if (peek (rdm) == c) + { + rdm->next++; + return 1; + } + else + return 0; +} + static char next (struct rust_demangler *rdm) { @@ -92,11 +109,87 @@ next (struct rust_demangler *rdm) return c; } +static uint64_t +parse_integer_62 (struct rust_demangler *rdm) +{ + char c; + uint64_t x; + + if (eat (rdm, '_')) + return 0; + + x = 0; + while (!eat (rdm, '_')) + { + c = next (rdm); + x *= 62; + if (ISDIGIT (c)) + x += c - '0'; + else if (ISLOWER (c)) + x += 10 + (c - 'a'); + else if (ISUPPER (c)) + x += 10 + 26 + (c - 'A'); + else + { + rdm->errored = 1; + return 0; + } + } + return x + 1; +} + +static uint64_t +parse_opt_integer_62 (struct rust_demangler *rdm, char tag) +{ + if (!eat (rdm, tag)) + return 0; + return 1 + parse_integer_62 (rdm); +} + +static uint64_t +parse_disambiguator (struct rust_demangler *rdm) +{ + return parse_opt_integer_62 (rdm, 's'); +} + +static size_t +parse_hex_nibbles (struct rust_demangler *rdm, uint64_t *value) +{ + char c; + size_t hex_len; + + hex_len = 0; + *value = 0; + + while (!eat (rdm, '_')) + { + *value <<= 4; + + c = next (rdm); + if (ISDIGIT (c)) + *value |= c - '0'; + else if (c >= 'a' && c <= 'f') + *value |= 10 + (c - 'a'); + else + { + rdm->errored = 1; + return 0; + } + hex_len++; + } + + return hex_len; +} + struct rust_mangled_ident { /* ASCII part of the identifier. */ const char *ascii; size_t ascii_len; + + /* Punycode insertion codes for Unicode codepoints, if any. */ + const char *punycode; + size_t punycode_len; }; static struct rust_mangled_ident @@ -104,10 +197,16 @@ parse_ident (struct rust_demangler *rdm) { char c; size_t start, len; + int is_punycode = 0; struct rust_mangled_ident ident; ident.ascii = NULL; ident.ascii_len = 0; + ident.punycode = NULL; + ident.punycode_len = 0; + + if (rdm->version != -1) + is_punycode = eat (rdm, 'u'); c = next (rdm); if (!ISDIGIT (c)) @@ -121,6 +220,10 @@ parse_ident (struct rust_demangler *rdm) while (ISDIGIT (peek (rdm))) len = len * 10 + (next (rdm) - '0'); + /* Skip past the optional `_` separator (v0). */ + if (rdm->version != -1) + eat (rdm, '_'); + start = rdm->next; rdm->next += len; /* Check for overflows. */ @@ -133,6 +236,27 @@ parse_ident (struct rust_demangler *rdm) ident.ascii = rdm->sym + start; ident.ascii_len = len; + if (is_punycode) + { + ident.punycode_len = 0; + while (ident.ascii_len > 0) + { + ident.ascii_len--; + + /* The last '_' is a separator between ascii & punycode. */ + if (ident.ascii[ident.ascii_len] == '_') + break; + + ident.punycode_len++; + } + if (!ident.punycode_len) + { + rdm->errored = 1; + return ident; + } + ident.punycode = ident.ascii + (len - ident.punycode_len); + } + if (ident.ascii_len == 0) ident.ascii = NULL; @@ -144,12 +268,28 @@ parse_ident (struct rust_demangler *rdm) static void print_str (struct rust_demangler *rdm, const char *data, size_t len) { - if (!rdm->errored) + if (!rdm->errored && !rdm->skipping_printing) rdm->callback (data, len, rdm->callback_opaque); } #define PRINT(s) print_str (rdm, s, strlen (s)) +static void +print_uint64 (struct rust_demangler *rdm, uint64_t x) +{ + char s[21]; + snprintf (s, 21, "%" PRIu64, x); + PRINT (s); +} + +static void +print_uint64_hex (struct rust_demangler *rdm, uint64_t x) +{ + char s[17]; + snprintf (s, 17, "%" PRIx64, x); + PRINT (s); +} + /* Return a 0x0-0xf value if the char is 0-9a-f, and -1 otherwise. */ static int decode_lower_hex_nibble (char nibble) @@ -230,9 +370,14 @@ static void print_ident (struct rust_demangler *rdm, struct rust_mangled_ident ident) { char unescaped; - size_t len; + uint8_t *out, *p, d; + size_t len, cap, punycode_pos, j; + /* Punycode parameters and state. */ + uint32_t c; + size_t base, t_min, t_max, skew, damp, bias, i; + size_t delta, w, k, t; - if (rdm->errored) + if (rdm->errored || rdm->skipping_printing) return; if (rdm->version == -1) @@ -273,8 +418,7 @@ print_ident (struct rust_demangler *rdm, struct rust_mangled_ident ident) } else { - /* "." becomes "-" */ - PRINT ("-"); + PRINT ("."); len = 1; } } @@ -294,39 +438,865 @@ print_ident (struct rust_demangler *rdm, struct rust_mangled_ident ident) return; } + + if (!ident.punycode) + { + print_str (rdm, ident.ascii, ident.ascii_len); + return; + } + + len = 0; + cap = 4; + while (cap < ident.ascii_len) + { + cap *= 2; + /* Check for overflows. */ + if ((cap * 4) / 4 != cap) + { + rdm->errored = 1; + return; + } + } + + /* Store the output codepoints as groups of 4 UTF-8 bytes. */ + out = (uint8_t *)malloc (cap * 4); + if (!out) + { + rdm->errored = 1; + return; + } + + /* Populate initial output from ASCII fragment. */ + for (len = 0; len < ident.ascii_len; len++) + { + p = out + 4 * len; + p[0] = 0; + p[1] = 0; + p[2] = 0; + p[3] = ident.ascii[len]; + } + + /* Punycode parameters and initial state. */ + base = 36; + t_min = 1; + t_max = 26; + skew = 38; + damp = 700; + bias = 72; + i = 0; + c = 0x80; + + punycode_pos = 0; + while (punycode_pos < ident.punycode_len) + { + /* Read one delta value. */ + delta = 0; + w = 1; + k = 0; + do + { + k += base; + t = k < bias ? 0 : (k - bias); + if (t < t_min) + t = t_min; + if (t > t_max) + t = t_max; + + if (punycode_pos >= ident.punycode_len) + goto cleanup; + d = ident.punycode[punycode_pos++]; + + if (ISLOWER (d)) + d = d - 'a'; + else if (ISDIGIT (d)) + d = 26 + (d - '0'); + else + { + rdm->errored = 1; + goto cleanup; + } + + delta += d * w; + w *= base - t; + } + while (d >= t); + + /* Compute the new insert position and character. */ + len++; + i += delta; + c += i / len; + i %= len; + + /* Ensure enough space is available. */ + if (cap < len) + { + cap *= 2; + /* Check for overflows. */ + if ((cap * 4) / 4 != cap || cap < len) + { + rdm->errored = 1; + goto cleanup; + } + } + p = (uint8_t *)realloc (out, cap * 4); + if (!p) + { + rdm->errored = 1; + goto cleanup; + } + out = p; + + /* Move the characters after the insert position. */ + p = out + i * 4; + memmove (p + 4, p, (len - i - 1) * 4); + + /* Insert the new character, as UTF-8 bytes. */ + p[0] = c >= 0x10000 ? 0xf0 | (c >> 18) : 0; + p[1] = c >= 0x800 ? (c < 0x10000 ? 0xe0 : 0x80) | ((c >> 12) & 0x3f) : 0; + p[2] = (c < 0x800 ? 0xc0 : 0x80) | ((c >> 6) & 0x3f); + p[3] = 0x80 | (c & 0x3f); + + /* If there are no more deltas, decoding is complete. */ + if (punycode_pos == ident.punycode_len) + break; + + i++; + + /* Perform bias adaptation. */ + delta /= damp; + damp = 2; + + delta += delta / len; + k = 0; + while (delta > ((base - t_min) * t_max) / 2) + { + delta /= base - t_min; + k += base; + } + bias = k + ((base - t_min + 1) * delta) / (delta + skew); + } + + /* Remove all the 0 bytes to leave behind an UTF-8 string. */ + for (i = 0, j = 0; i < len * 4; i++) + if (out[i] != 0) + out[j++] = out[i]; + + print_str (rdm, (const char *)out, j); + +cleanup: + free (out); +} + +/* Print the lifetime according to the previously decoded index. + An index of `0` always refers to `'_`, but starting with `1`, + indices refer to late-bound lifetimes introduced by a binder. */ +static void +print_lifetime_from_index (struct rust_demangler *rdm, uint64_t lt) +{ + char c; + uint64_t depth; + + PRINT ("'"); + if (lt == 0) + { + PRINT ("_"); + return; + } + + depth = rdm->bound_lifetime_depth - lt; + /* Try to print lifetimes alphabetically first. */ + if (depth < 26) + { + c = 'a' + depth; + print_str (rdm, &c, 1); + } + else + { + /* Use `'_123` after running out of letters. */ + PRINT ("_"); + print_uint64 (rdm, depth); + } +} + +/* Demangling functions. */ + +static void demangle_binder (struct rust_demangler *rdm); +static void demangle_path (struct rust_demangler *rdm, int in_value); +static void demangle_generic_arg (struct rust_demangler *rdm); +static void demangle_type (struct rust_demangler *rdm); +static int demangle_path_maybe_open_generics (struct rust_demangler *rdm); +static void demangle_dyn_trait (struct rust_demangler *rdm); +static void demangle_const (struct rust_demangler *rdm); +static void demangle_const_uint (struct rust_demangler *rdm); +static void demangle_const_int (struct rust_demangler *rdm); +static void demangle_const_bool (struct rust_demangler *rdm); +static void demangle_const_char (struct rust_demangler *rdm); + +/* Optionally enter a binder ('G') for late-bound lifetimes, + printing e.g. `for<'a, 'b> `, and make those lifetimes visible + to the caller (via depth level, which the caller should reset). */ +static void +demangle_binder (struct rust_demangler *rdm) +{ + uint64_t i, bound_lifetimes; + + if (rdm->errored) + return; + + bound_lifetimes = parse_opt_integer_62 (rdm, 'G'); + if (bound_lifetimes > 0) + { + PRINT ("for<"); + for (i = 0; i < bound_lifetimes; i++) + { + if (i > 0) + PRINT (", "); + rdm->bound_lifetime_depth++; + print_lifetime_from_index (rdm, 1); + } + PRINT ("> "); + } +} + +static void +demangle_path (struct rust_demangler *rdm, int in_value) +{ + char tag, ns; + int was_skipping_printing; + size_t i, backref, old_next; + uint64_t dis; + struct rust_mangled_ident name; + + if (rdm->errored) + return; + + switch (tag = next (rdm)) + { + case 'C': + dis = parse_disambiguator (rdm); + name = parse_ident (rdm); + + print_ident (rdm, name); + if (rdm->verbose) + { + PRINT ("["); + print_uint64_hex (rdm, dis); + PRINT ("]"); + } + break; + case 'N': + ns = next (rdm); + if (!ISLOWER (ns) && !ISUPPER (ns)) + { + rdm->errored = 1; + return; + } + + demangle_path (rdm, in_value); + + dis = parse_disambiguator (rdm); + name = parse_ident (rdm); + + if (ISUPPER (ns)) + { + /* Special namespaces, like closures and shims. */ + PRINT ("::{"); + switch (ns) + { + case 'C': + PRINT ("closure"); + break; + case 'S': + PRINT ("shim"); + break; + default: + print_str (rdm, &ns, 1); + } + if (name.ascii || name.punycode) + { + PRINT (":"); + print_ident (rdm, name); + } + PRINT ("#"); + print_uint64 (rdm, dis); + PRINT ("}"); + } + else + { + /* Implementation-specific/unspecified namespaces. */ + + if (name.ascii || name.punycode) + { + PRINT ("::"); + print_ident (rdm, name); + } + } + break; + case 'M': + case 'X': + /* Ignore the `impl`'s own path.*/ + parse_disambiguator (rdm); + was_skipping_printing = rdm->skipping_printing; + rdm->skipping_printing = 1; + demangle_path (rdm, in_value); + rdm->skipping_printing = was_skipping_printing; + /* fallthrough */ + case 'Y': + PRINT ("<"); + demangle_type (rdm); + if (tag != 'M') + { + PRINT (" as "); + demangle_path (rdm, 0); + } + PRINT (">"); + break; + case 'I': + demangle_path (rdm, in_value); + if (in_value) + PRINT ("::"); + PRINT ("<"); + for (i = 0; !rdm->errored && !eat (rdm, 'E'); i++) + { + if (i > 0) + PRINT (", "); + demangle_generic_arg (rdm); + } + PRINT (">"); + break; + case 'B': + backref = parse_integer_62 (rdm); + if (!rdm->skipping_printing) + { + old_next = rdm->next; + rdm->next = backref; + demangle_path (rdm, in_value); + rdm->next = old_next; + } + break; + default: + rdm->errored = 1; + return; + } +} + +static void +demangle_generic_arg (struct rust_demangler *rdm) +{ + uint64_t lt; + if (eat (rdm, 'L')) + { + lt = parse_integer_62 (rdm); + print_lifetime_from_index (rdm, lt); + } + else if (eat (rdm, 'K')) + demangle_const (rdm); + else + demangle_type (rdm); +} + +static const char * +basic_type (char tag) +{ + switch (tag) + { + case 'b': + return "bool"; + case 'c': + return "char"; + case 'e': + return "str"; + case 'u': + return "()"; + case 'a': + return "i8"; + case 's': + return "i16"; + case 'l': + return "i32"; + case 'x': + return "i64"; + case 'n': + return "i128"; + case 'i': + return "isize"; + case 'h': + return "u8"; + case 't': + return "u16"; + case 'm': + return "u32"; + case 'y': + return "u64"; + case 'o': + return "u128"; + case 'j': + return "usize"; + case 'f': + return "f32"; + case 'd': + return "f64"; + case 'z': + return "!"; + case 'p': + return "_"; + case 'v': + return "..."; + + default: + return NULL; + } +} + +static void +demangle_type (struct rust_demangler *rdm) +{ + char tag; + size_t i, old_next, backref; + uint64_t lt, old_bound_lifetime_depth; + const char *basic; + struct rust_mangled_ident abi; + + if (rdm->errored) + return; + + tag = next (rdm); + + basic = basic_type (tag); + if (basic) + { + PRINT (basic); + return; + } + + switch (tag) + { + case 'R': + case 'Q': + PRINT ("&"); + if (eat (rdm, 'L')) + { + lt = parse_integer_62 (rdm); + if (lt) + { + print_lifetime_from_index (rdm, lt); + PRINT (" "); + } + } + if (tag != 'R') + PRINT ("mut "); + demangle_type (rdm); + break; + case 'P': + case 'O': + PRINT ("*"); + if (tag != 'P') + PRINT ("mut "); + else + PRINT ("const "); + demangle_type (rdm); + break; + case 'A': + case 'S': + PRINT ("["); + demangle_type (rdm); + if (tag == 'A') + { + PRINT ("; "); + demangle_const (rdm); + } + PRINT ("]"); + break; + case 'T': + PRINT ("("); + for (i = 0; !rdm->errored && !eat (rdm, 'E'); i++) + { + if (i > 0) + PRINT (", "); + demangle_type (rdm); + } + if (i == 1) + PRINT (","); + PRINT (")"); + break; + case 'F': + old_bound_lifetime_depth = rdm->bound_lifetime_depth; + demangle_binder (rdm); + + if (eat (rdm, 'U')) + PRINT ("unsafe "); + + if (eat (rdm, 'K')) + { + if (eat (rdm, 'C')) + { + abi.ascii = "C"; + abi.ascii_len = 1; + } + else + { + abi = parse_ident (rdm); + if (!abi.ascii || abi.punycode) + { + rdm->errored = 1; + goto restore; + } + } + + PRINT ("extern \""); + + /* If the ABI had any `-`, they were replaced with `_`, + so the parts between `_` have to be re-joined with `-`. */ + for (i = 0; i < abi.ascii_len; i++) + { + if (abi.ascii[i] == '_') + { + print_str (rdm, abi.ascii, i); + PRINT ("-"); + abi.ascii += i + 1; + abi.ascii_len -= i + 1; + i = 0; + } + } + print_str (rdm, abi.ascii, abi.ascii_len); + + PRINT ("\" "); + } + + PRINT ("fn("); + for (i = 0; !rdm->errored && !eat (rdm, 'E'); i++) + { + if (i > 0) + PRINT (", "); + demangle_type (rdm); + } + PRINT (")"); + + if (eat (rdm, 'u')) + { + /* Skip printing the return type if it's 'u', i.e. `()`. */ + } + else + { + PRINT (" -> "); + demangle_type (rdm); + } + + /* Restore `bound_lifetime_depth` to outside the binder. */ + restore: + rdm->bound_lifetime_depth = old_bound_lifetime_depth; + break; + case 'D': + PRINT ("dyn "); + + old_bound_lifetime_depth = rdm->bound_lifetime_depth; + demangle_binder (rdm); + + for (i = 0; !rdm->errored && !eat (rdm, 'E'); i++) + { + if (i > 0) + PRINT (" + "); + demangle_dyn_trait (rdm); + } + + /* Restore `bound_lifetime_depth` to outside the binder. */ + rdm->bound_lifetime_depth = old_bound_lifetime_depth; + + if (!eat (rdm, 'L')) + { + rdm->errored = 1; + return; + } + lt = parse_integer_62 (rdm); + if (lt) + { + PRINT (" + "); + print_lifetime_from_index (rdm, lt); + } + break; + case 'B': + backref = parse_integer_62 (rdm); + if (!rdm->skipping_printing) + { + old_next = rdm->next; + rdm->next = backref; + demangle_type (rdm); + rdm->next = old_next; + } + break; + default: + /* Go back to the tag, so `demangle_path` also sees it. */ + rdm->next--; + demangle_path (rdm, 0); + } +} + +/* A trait in a trait object may have some "existential projections" + (i.e. associated type bindings) after it, which should be printed + in the `<...>` of the trait, e.g. `dyn Trait<T, U, Assoc=X>`. + To this end, this method will keep the `<...>` of an 'I' path + open, by omitting the `>`, and return `Ok(true)` in that case. */ +static int +demangle_path_maybe_open_generics (struct rust_demangler *rdm) +{ + int open; + size_t i, old_next, backref; + + open = 0; + + if (rdm->errored) + return open; + + if (eat (rdm, 'B')) + { + backref = parse_integer_62 (rdm); + if (!rdm->skipping_printing) + { + old_next = rdm->next; + rdm->next = backref; + open = demangle_path_maybe_open_generics (rdm); + rdm->next = old_next; + } + } + else if (eat (rdm, 'I')) + { + demangle_path (rdm, 0); + PRINT ("<"); + open = 1; + for (i = 0; !rdm->errored && !eat (rdm, 'E'); i++) + { + if (i > 0) + PRINT (", "); + demangle_generic_arg (rdm); + } + } + else + demangle_path (rdm, 0); + return open; +} + +static void +demangle_dyn_trait (struct rust_demangler *rdm) +{ + int open; + struct rust_mangled_ident name; + + if (rdm->errored) + return; + + open = demangle_path_maybe_open_generics (rdm); + + while (eat (rdm, 'p')) + { + if (!open) + PRINT ("<"); + else + PRINT (", "); + open = 1; + + name = parse_ident (rdm); + print_ident (rdm, name); + PRINT (" = "); + demangle_type (rdm); + } + + if (open) + PRINT (">"); +} + +static void +demangle_const (struct rust_demangler *rdm) +{ + char ty_tag; + size_t old_next, backref; + + if (rdm->errored) + return; + + if (eat (rdm, 'B')) + { + backref = parse_integer_62 (rdm); + if (!rdm->skipping_printing) + { + old_next = rdm->next; + rdm->next = backref; + demangle_const (rdm); + rdm->next = old_next; + } + return; + } + + ty_tag = next (rdm); + switch (ty_tag) + { + /* Placeholder. */ + case 'p': + PRINT ("_"); + return; + + /* Unsigned integer types. */ + case 'h': + case 't': + case 'm': + case 'y': + case 'o': + case 'j': + demangle_const_uint (rdm); + break; + + /* Signed integer types. */ + case 'a': + case 's': + case 'l': + case 'x': + case 'n': + case 'i': + demangle_const_int (rdm); + break; + + /* Boolean. */ + case 'b': + demangle_const_bool (rdm); + break; + + /* Character. */ + case 'c': + demangle_const_char (rdm); + break; + + default: + rdm->errored = 1; + return; + } + + if (rdm->errored) + return; + + if (rdm->verbose) + { + PRINT (": "); + PRINT (basic_type (ty_tag)); + } +} + +static void +demangle_const_uint (struct rust_demangler *rdm) +{ + size_t hex_len; + uint64_t value; + + if (rdm->errored) + return; + + hex_len = parse_hex_nibbles (rdm, &value); + + if (hex_len > 16) + { + /* Print anything that doesn't fit in `uint64_t` verbatim. */ + PRINT ("0x"); + print_str (rdm, rdm->sym + (rdm->next - hex_len), hex_len); + } + else if (hex_len > 0) + print_uint64 (rdm, value); + else + rdm->errored = 1; +} + +static void +demangle_const_int (struct rust_demangler *rdm) +{ + if (eat (rdm, 'n')) + PRINT ("-"); + demangle_const_uint (rdm); +} + +static void +demangle_const_bool (struct rust_demangler *rdm) +{ + uint64_t value; + + if (parse_hex_nibbles (rdm, &value) != 1) + { + rdm->errored = 1; + return; + } + + if (value == 0) + PRINT ("false"); + else if (value == 1) + PRINT ("true"); + else + rdm->errored = 1; +} + +static void +demangle_const_char (struct rust_demangler *rdm) +{ + size_t hex_len; + uint64_t value; + + hex_len = parse_hex_nibbles (rdm, &value); + + if (hex_len == 0 || hex_len > 8) + { + rdm->errored = 1; + return; + } + + /* Match Rust's character "debug" output as best as we can. */ + PRINT ("'"); + if (value == '\t') + PRINT ("\\t"); + else if (value == '\r') + PRINT ("\\r"); + else if (value == '\n') + PRINT ("\\n"); + else if (value > ' ' && value < '~') + /* Rust also considers many non-ASCII codepoints to be printable, but + that logic is not easily ported to C. */ + print_str (rdm, (char *) &value, 1); + else + { + PRINT ("\\u{"); + print_uint64_hex (rdm, value); + PRINT ("}"); + } + PRINT ("'"); } -/* A legacy hash is the prefix "h" followed by 16 lowercase hex digits. - The hex digits must contain at least 5 distinct digits. */ +/* A legacy hash is the prefix "h" followed by any number of lowercase + hex digits. (The compiler will always generate 16 hex digits, but + demanglers are expected to accept any number of digits.) */ static int is_legacy_prefixed_hash (struct rust_mangled_ident ident) { - uint16_t seen; - int nibble; - size_t i, count; + size_t i; - if (ident.ascii_len != 17 || ident.ascii[0] != 'h') + if (ident.ascii[0] != 'h') return 0; - seen = 0; - for (i = 0; i < 16; i++) - { - nibble = decode_lower_hex_nibble (ident.ascii[1 + i]); - if (nibble < 0) - return 0; - seen |= (uint16_t)1 << nibble; - } + for (i = 1; i < ident.ascii_len; i++) + if (decode_lower_hex_nibble (ident.ascii[i]) < 0) + return 0; - /* Count how many distinct digits were seen. */ - count = 0; - while (seen) - { - if (seen & 1) - count++; - seen >>= 1; - } + return 1; +} - return count >= 5; +/* Consumes a suffix of the form ".llvm.XXXXXX". */ +static void +eat_llvm_suffix (struct rust_demangler *rdm) +{ + size_t i; + char c; + + if (strncmp (&rdm->sym[rdm->next], ".llvm.", 6) != 0) + return; + + for (i = rdm->next + 6; (c = rdm->sym[i]); i++) + if ((c < '0' || c > '9') && (c < 'A' || c > 'F') && c != '@') + return; + + rdm->next = i; } int @@ -345,11 +1315,15 @@ rust_demangle_callback (const char *mangled, int options, rdm.next = 0; rdm.errored = 0; + rdm.skipping_printing = 0; rdm.verbose = (options & DMGL_VERBOSE) != 0; rdm.version = 0; + rdm.bound_lifetime_depth = 0; - /* Rust symbols always start with _ZN (legacy). */ - if (rdm.sym[0] == '_' && rdm.sym[1] == 'Z' && rdm.sym[2] == 'N') + /* Rust symbols always start with _R (v0) or _ZN (legacy). */ + if (rdm.sym[0] == '_' && rdm.sym[1] == 'R') + rdm.sym += 2; + else if (rdm.sym[0] == '_' && rdm.sym[1] == 'Z' && rdm.sym[2] == 'N') { rdm.sym += 3; rdm.version = -1; @@ -357,15 +1331,21 @@ rust_demangle_callback (const char *mangled, int options, else return 0; - /* Legacy Rust symbols use only [_0-9a-zA-Z.:$] characters. */ + /* Paths (v0) always start with uppercase characters. */ + if (rdm.version != -1 && !ISUPPER (rdm.sym[0])) + return 0; + + /* Rust symbols (v0) use only [_0-9a-zA-Z] characters, but LLVM might add + a suffix containing [.@] characters. */ for (p = rdm.sym; *p; p++) { rdm.sym_len++; - if (*p == '_' || ISALNUM (*p)) + if (*p == '_' || *p == '.' || *p == '@' || ISALNUM (*p)) continue; - if (rdm.version == -1 && (*p == '$' || *p == '.' || *p == ':')) + /* Legacy Rust symbols can also contain [:$] characters. */ + if (rdm.version == -1 && (*p == '$' || *p == ':')) continue; return 0; @@ -374,51 +1354,54 @@ rust_demangle_callback (const char *mangled, int options, /* Legacy Rust symbols need to be handled separately. */ if (rdm.version == -1) { - /* Legacy Rust symbols always end with E. */ - if (!(rdm.sym_len > 0 && rdm.sym[rdm.sym_len - 1] == 'E')) + /* Print the first component of the path. */ + ident = parse_ident (&rdm); + if (rdm.errored || !ident.ascii) return 0; - rdm.sym_len--; + print_ident (&rdm, ident); - /* Legacy Rust symbols also always end with a path segment - that encodes a 16 hex digit hash, i.e. '17h[a-f0-9]{16}'. - This early check, before any parse_ident calls, should - quickly filter out most C++ symbols unrelated to Rust. */ - if (!(rdm.sym_len > 19 - && !memcmp (&rdm.sym[rdm.sym_len - 19], "17h", 3))) - return 0; - - do + while (!eat (&rdm, 'E')) { ident = parse_ident (&rdm); if (rdm.errored || !ident.ascii) return 0; - } - while (rdm.next < rdm.sym_len); - /* The last path segment should be the hash. */ - if (!is_legacy_prefixed_hash (ident)) - return 0; - - /* Reset the state for a second pass, to print the symbol. */ - rdm.next = 0; - if (!rdm.verbose && rdm.sym_len > 19) - { - /* Hide the last segment, containing the hash, if not verbose. */ - rdm.sym_len -= 19; - } - - do - { - if (rdm.next > 0) - print_str (&rdm, "::", 2); - - ident = parse_ident (&rdm); - print_ident (&rdm, ident); - } - while (rdm.next < rdm.sym_len); + /* Print the path component, unless it's the last component, we're + in verbose mode, and the component looks like a hash. */ + if (peek (&rdm) != 'E' + || rdm.verbose + || !is_legacy_prefixed_hash (ident)) + { + print_str (&rdm, "::", 2); + print_ident (&rdm, ident); + } + } } else - return 0; + { + demangle_path (&rdm, 1); + + /* Skip instantiating crate. */ + if (!rdm.errored && peek (&rdm) >= 'A' && peek (&rdm) <= 'Z') + { + rdm.skipping_printing = 1; + demangle_path (&rdm, 0); + rdm.skipping_printing = 0; + } + } + + /* Suppress suffixes of the form ".llvm.XXXXXX". */ + eat_llvm_suffix (&rdm); + + /* But output other suffixes that start with a dot. */ + if (peek (&rdm) == '.') + { + print_str (&rdm, &rdm.sym[rdm.next], rdm.sym_len - rdm.next); + rdm.next = rdm.sym_len; + } + + /* It's an error to not reach the end. */ + rdm.errored |= rdm.next != rdm.sym_len; return !rdm.errored; } diff --git a/libiberty/testsuite/rust-demangle-expected b/libiberty/testsuite/rust-demangle-expected index 74774794736..24522d9b79e 100644 --- a/libiberty/testsuite/rust-demangle-expected +++ b/libiberty/testsuite/rust-demangle-expected @@ -26,15 +26,15 @@ main::main::he714a2e23ed7db23 --format=auto _ZN4main4main17he714a2e23ed7db23E main::main -# Hash is exactly 16 hex chars. Not more. +# The hash can be more than 16 digits, as long as all are valid hex digits. --format=auto _ZN4main4main18h1e714a2e23ed7db23E -main::main::h1e714a2e23ed7db23 -# Not less. +main::main +# Similarly, less than 16 hex digits in the hash is acceptable too. --format=auto _ZN4main4main16h714a2e23ed7db23E -main::main::h714a2e23ed7db23 -# And not non-hex. +main::main +# But not non-hex. --format=auto _ZN4main4main17he714a2e23ed7db2gE main::main::he714a2e23ed7db2g @@ -163,3 +163,87 @@ _ZN63_$LT$core..ptr..Unique$LT$T$GT$$u20$as$u20$core..ops..Deref$GT$5deref17h19f --format=rust _ZN11issue_609253foo37Foo$LT$issue_60925..llv$u6d$..Foo$GT$3foo17h059a991a004536adE issue_60925::foo::Foo<issue_60925::llvm::Foo>::foo +# +--format=rust +_ZN4core3ops8function6FnOnce40call_once$u7b$$u7b$vtable.shim$u7d$$u7d$17h000b1ad6c4f30bd6E +core::ops::function::FnOnce::call_once{{vtable.shim}} +# ThinLTO suffix 1 + legacy. +--format=rust +_ZN3fooE.llvm.9D1C9369 +foo +# Thin LTO suffix 2 + legacy. +--format=rust +_ZN3fooE.llvm.9D1C9369@@16 +foo +# ThinLTO suffix 3 + legacy. +--format=rust +_ZN9backtrace3foo17hbb467fcdaea5d79bE.llvm.A5310EB9 +backtrace::foo +# ThinLTO suffix 1 + v0. +--format=rust +_RC3foo.llvm.9D1C9369 +foo +# ThinLTO suffix 2 + v0. +--format=rust +_RC3foo.llvm.9D1C9369@@16 +foo +# ThinLTO suffix 3 + legacy. +--format=rust +_RNvC9backtrace3foo.llvm.A5310EB9 +backtrace::foo +# LLVM branch label + legacy. +--format=rust +_ZN4core5slice77_$LT$impl$u20$core..ops..index..IndexMut$LT$I$GT$$u20$for$u20$$u5b$T$u5d$$GT$9index_mut17haf9727c2edfbc47bE.exit.i.i +core::slice::<impl core::ops::index::IndexMut<I> for [T]>::index_mut.exit.i.i +# LLVM branch label + v0. +--format=rust +_RNvNtNtNtNtCs92dm3009vxr_4rand4rngs7adapter9reseeding4fork23FORK_HANDLER_REGISTERED.0.0 +rand::rngs::adapter::reseeding::fork::FORK_HANDLER_REGISTERED.0.0 +# Closure. +--format=rust +_RNCNCNgCs6DXkGYLi8lr_2cc5spawn00B5_ +cc::spawn::{closure#0}::{closure#0} +# UTF8 idents. +--format=rust +_RNqCs4fqI2P2rA04_11utf8_identsu30____7hkackfecea1cbdathfdh9hlq6y +utf8_idents::საჭმელად_გემრიელი_სადილი +# Crate with leading digit. +--format=rust +_RNvC6_123foo3bar +123foo::bar +# +--format=rust +_RMCs4fqI2P2rA04_13const_genericINtB0_8UnsignedKhb_E +<const_generic::Unsigned<11>> +# +--format=rust +_RMCs4fqI2P2rA04_13const_genericINtB0_6SignedKs98_E +<const_generic::Signed<152>> +# +--format=rust +_RMCs4fqI2P2rA04_13const_genericINtB0_6SignedKanb_E +<const_generic::Signed<-11>> +# +--format=rust +_RMCs4fqI2P2rA04_13const_genericINtB0_4BoolKb0_E +<const_generic::Bool<false>> +# +--format=rust +_RMCs4fqI2P2rA04_13const_genericINtB0_4BoolKb1_E +<const_generic::Bool<true>> +# +--format=rust +_RMCs4fqI2P2rA04_13const_genericINtB0_4CharKc76_E +<const_generic::Char<'v'>> +# +--format=rust +_RMCs4fqI2P2rA04_13const_genericINtB0_4CharKca_E +<const_generic::Char<'\n'>> +# +--format=rust +_RMCs4fqI2P2rA04_13const_genericINtB0_4CharKc2202_E +<const_generic::Char<'\u{2202}'>> +# +--format=rust +_RNvNvMCs4fqI2P2rA04_13const_genericINtB4_3FooKpE3foo3FOO +<const_generic::Foo<_>>::foo::FOO
Reading the diff patch, the v0 changes look great. I wouldn't be too worried about the "printable character" aspect, there are similar Unicode-related issues elsewhere, e.g. the "non-control ASCII" comment in decode_legacy_escape (I suppose we could make it also go through the "print a non-control ASCII character or some escape sequence" logic you added, if you think that helps). However, I'm not sure about the legacy changes. Or rather, the .llvm. one, it's not really Rust-specific, it's only in the rustc-demangle crate for convenience, but C++ code compiled with Clang could run into the same problem - ideally, stripping that suffix could be done uniformly in cplus-dem.c, but I decided against making that change myself, for now. I'm especially not comfortable removing the fast path, since that was the condition under which I was able to make Rust demangling be attempted first, before C++, in order to implement the Rust legacy demangling standalone, separately from C++ demangling, so that it could be together with the v0 one. It should be possible to keep the fast path if stripping .llvm.* suffixes is done before either Rust or C++ demangling is attempted, but even if that would be nice to have, IMO it should be a separate patch and not block v0 demangling. As for the dataset, it doesn't include .llvm. because it's collected by rustc itself, before LLVM had any chance to add any suffixes. This was done in order to have several different mangling formats dumped at once for every symbol. (It also contains symbols for e.g. functions that LLVM would've inlined away) I can test the patch and upload the dataset tomorrow, but if you want to get something committed sooner (is there a deadline for the next release?), feel free to land the v0 changes (snprintf + const values) without the legacy ones. Thanks, - Eddy B. On Sun, Nov 1, 2020, at 11:18, Nikhil Benesch wrote: > > > On 10/29/20 12:16 AM, Nikhil Benesch wrote: > > On Wed, Oct 28, 2020 at 7:53 PM Eduard-Mihai Burtescu <eddyb@lyken.rs> wrote: > >> I agree that landing the extra changes later on top should be fine anyway, > >> though when I make the sprintf -> snprintf change, I could provide the extra > >> changes as well (either as a combined patch, or as a second tiny patch). > >> > >> Sadly I don't think I can get to either until next week, hope that's okay. > > > > I can make the sprintf -> snprintf change as early as tomorrow. > > > > I suspect I can also make the substantive const generics change, based > > on the Rust implementation, though that's less of a promise. > > Attached is an updated patch with both of the aforementioned changes. > The demangling of constants matches the demangling in rustc-demangle > library as best as I could manage. The strategy of demangling constant > chars via `format!("{:?}", char)` in Rust makes life hard for us in C, > because there is no easy way to replicate Rust's debug formatting for > chars. (Unless GCC has a Unicode library handy somewhere.) > > In the course of this work I noticed several inconsistencies in how > rustc-demangle and libiberty were demangling some legacy Rust symbols. I > fixed those and added test cases, though the fixes required removing > some of the fast checks for whether a given symbol is a valid Rust symbol. > > For ease of review, eddyb, I also attached the diff from your last diff > to mine. Since I don't have your collection of symbols I generated my > own by running nm on the materialized binary from > https://github.com/MaterializeInc/materialize. > > Let me know how I can help. I'm happy to address review feedback myself, > or I'm happy to hand things back over to you, eddyb. > > Nikhil > > Attachments: > * rust-demangle-diff.patch > * rust-demangle.patch
On 11/1/20 6:57 AM, Eduard-Mihai Burtescu wrote: > Reading the diff patch, the v0 changes look great. I wouldn't be too worried > about the "printable character" aspect, there are similar Unicode-related > issues elsewhere, e.g. the "non-control ASCII" comment in decode_legacy_escape > (I suppose we could make it also go through the "print a non-control ASCII > character or some escape sequence" logic you added, if you think that helps). No, it's entirely fine with me! I just wasn't sure if the small deviations in output were acceptable. It sounds like they are. > However, I'm not sure about the legacy changes. Or rather, the .llvm. one, it's > not really Rust-specific, it's only in the rustc-demangle crate for convenience, > but C++ code compiled with Clang could run into the same problem - ideally, > stripping that suffix could be done uniformly in cplus-dem.c, but I decided > against making that change myself, for now. > > I'm especially not comfortable removing the fast path, since that was the > condition under which I was able to make Rust demangling be attempted first, > before C++, in order to implement the Rust legacy demangling standalone, > separately from C++ demangling, so that it could be together with the v0 one. > It should be possible to keep the fast path if stripping .llvm.* suffixes is > done before either Rust or C++ demangling is attempted, but even if that would > be nice to have, IMO it should be a separate patch and not block v0 demangling. That makes sense. I've attached updated patches (again generating a diff against both your original patch and trunk) without the changes to the legacy code. I did preserve one small hunk regarding the unescaping of a single '.' character in idents, as I believe that is just a straightforward bug in the existing code. > I can test the patch and upload the dataset tomorrow, but if you want to get > something committed sooner (is there a deadline for the next release?), feel > free to land the v0 changes (snprintf + const values) without the legacy ones. My understanding is that the GCC tree closes to new features on November 16 (for "GCC 11 Stage 3"), but I'm not sure whether that applies to libiberty or whether this patch would be classified as a feature or a bugfix. I don't have commit rights (nor am I even a GCC developer). Just wanted to tee things up for you and Ian this week. I'm very much looking forward to the new demangling scheme and didn't want to be just another +1 on the GitHub issue. So certainly no time pressure from me. But perhaps someone from the GCC side can confirm whether we are under a bit of time pressure here given the GCC 11 release. Cheers, Nikhil diff --git a/rust-demangle.c b/rust-demangle.c index d604b3c..9cd8f99 100644 --- a/rust-demangle.c +++ b/rust-demangle.c @@ -143,6 +143,35 @@ parse_disambiguator (struct rust_demangler *rdm) return parse_opt_integer_62 (rdm, 's'); } +static size_t +parse_hex_nibbles (struct rust_demangler *rdm, uint64_t *value) +{ + char c; + size_t hex_len; + + hex_len = 0; + *value = 0; + + while (!eat (rdm, '_')) + { + *value <<= 4; + + c = next (rdm); + if (ISDIGIT (c)) + *value |= c - '0'; + else if (c >= 'a' && c <= 'f') + *value |= 10 + (c - 'a'); + else + { + rdm->errored = 1; + return 0; + } + hex_len++; + } + + return hex_len; +} + struct rust_mangled_ident { /* ASCII part of the identifier. */ @@ -240,7 +269,7 @@ static void print_uint64 (struct rust_demangler *rdm, uint64_t x) { char s[21]; - sprintf (s, "%" PRIu64, x); + snprintf (s, 21, "%" PRIu64, x); PRINT (s); } @@ -248,7 +277,7 @@ static void print_uint64_hex (struct rust_demangler *rdm, uint64_t x) { char s[17]; - sprintf (s, "%" PRIx64, x); + snprintf (s, 17, "%" PRIx64, x); PRINT (s); } @@ -380,8 +409,7 @@ print_ident (struct rust_demangler *rdm, struct rust_mangled_ident ident) } else { - /* "." becomes "-" */ - PRINT ("-"); + PRINT ("."); len = 1; } } @@ -591,6 +619,9 @@ static int demangle_path_maybe_open_generics (struct rust_demangler *rdm); static void demangle_dyn_trait (struct rust_demangler *rdm); static void demangle_const (struct rust_demangler *rdm); static void demangle_const_uint (struct rust_demangler *rdm); +static void demangle_const_int (struct rust_demangler *rdm); +static void demangle_const_bool (struct rust_demangler *rdm); +static void demangle_const_char (struct rust_demangler *rdm); /* Optionally enter a binder ('G') for late-bound lifetimes, printing e.g. `for<'a, 'b> `, and make those lifetimes visible @@ -1089,6 +1120,11 @@ demangle_const (struct rust_demangler *rdm) ty_tag = next (rdm); switch (ty_tag) { + /* Placeholder. */ + case 'p': + PRINT ("_"); + return; + /* Unsigned integer types. */ case 'h': case 't': @@ -1096,6 +1132,27 @@ demangle_const (struct rust_demangler *rdm) case 'y': case 'o': case 'j': + demangle_const_uint (rdm); + break; + + /* Signed integer types. */ + case 'a': + case 's': + case 'l': + case 'x': + case 'n': + case 'i': + demangle_const_int (rdm); + break; + + /* Boolean. */ + case 'b': + demangle_const_bool (rdm); + break; + + /* Character. */ + case 'c': + demangle_const_char (rdm); break; default: @@ -1103,10 +1160,8 @@ demangle_const (struct rust_demangler *rdm) return; } - if (eat (rdm, 'p')) - PRINT ("_"); - else - demangle_const_uint (rdm); + if (rdm->errored) + return; if (rdm->verbose) { @@ -1118,41 +1173,86 @@ demangle_const (struct rust_demangler *rdm) static void demangle_const_uint (struct rust_demangler *rdm) { - char c; size_t hex_len; uint64_t value; if (rdm->errored) return; - value = 0; - hex_len = 0; - while (!eat (rdm, '_')) - { - value <<= 4; + hex_len = parse_hex_nibbles (rdm, &value); - c = next (rdm); - if (ISDIGIT (c)) - value |= c - '0'; - else if (c >= 'a' && c <= 'f') - value |= 10 + (c - 'a'); - else - { - rdm->errored = 1; - return; - } - hex_len++; - } - - /* Print anything that doesn't fit in `uint64_t` verbatim. */ if (hex_len > 16) { + /* Print anything that doesn't fit in `uint64_t` verbatim. */ PRINT ("0x"); print_str (rdm, rdm->sym + (rdm->next - hex_len), hex_len); + } + else if (hex_len > 0) + print_uint64 (rdm, value); + else + rdm->errored = 1; +} + +static void +demangle_const_int (struct rust_demangler *rdm) +{ + if (eat (rdm, 'n')) + PRINT ("-"); + demangle_const_uint (rdm); +} + +static void +demangle_const_bool (struct rust_demangler *rdm) +{ + uint64_t value; + + if (parse_hex_nibbles (rdm, &value) != 1) + { + rdm->errored = 1; return; } - print_uint64 (rdm, value); + if (value == 0) + PRINT ("false"); + else if (value == 1) + PRINT ("true"); + else + rdm->errored = 1; +} + +static void +demangle_const_char (struct rust_demangler *rdm) +{ + size_t hex_len; + uint64_t value; + + hex_len = parse_hex_nibbles (rdm, &value); + + if (hex_len == 0 || hex_len > 8) + { + rdm->errored = 1; + return; + } + + /* Match Rust's character "debug" output as best as we can. */ + PRINT ("'"); + if (value == '\t') + PRINT ("\\t"); + else if (value == '\r') + PRINT ("\\r"); + else if (value == '\n') + PRINT ("\\n"); + else if (value > ' ' && value < '~') + /* Rust also considers many non-ASCII codepoints to be printable, but + that logic is not easily ported to C. */ + print_str (rdm, (char *) &value, 1); + else + { + PRINT ("\\u{"); + print_uint64_hex (rdm, value); + PRINT ("}"); + } + PRINT ("'"); } /* A legacy hash is the prefix "h" followed by 16 lowercase hex digits. diff --git a/libiberty/rust-demangle.c b/libiberty/rust-demangle.c index b87365c85fe..f6b63a60652 100644 --- a/libiberty/rust-demangle.c +++ b/libiberty/rust-demangle.c @@ -64,11 +64,16 @@ struct rust_demangler /* Non-zero if any error occurred. */ int errored; + /* Non-zero if nothing should be printed. */ + int skipping_printing; + /* Non-zero if printing should be verbose (e.g. include hashes). */ int verbose; /* Rust mangling version, with legacy mangling being -1. */ int version; + + uint64_t bound_lifetime_depth; }; /* Parsing functions. */ @@ -81,6 +86,18 @@ peek (const struct rust_demangler *rdm) return 0; } +static int +eat (struct rust_demangler *rdm, char c) +{ + if (peek (rdm) == c) + { + rdm->next++; + return 1; + } + else + return 0; +} + static char next (struct rust_demangler *rdm) { @@ -92,11 +109,87 @@ next (struct rust_demangler *rdm) return c; } +static uint64_t +parse_integer_62 (struct rust_demangler *rdm) +{ + char c; + uint64_t x; + + if (eat (rdm, '_')) + return 0; + + x = 0; + while (!eat (rdm, '_')) + { + c = next (rdm); + x *= 62; + if (ISDIGIT (c)) + x += c - '0'; + else if (ISLOWER (c)) + x += 10 + (c - 'a'); + else if (ISUPPER (c)) + x += 10 + 26 + (c - 'A'); + else + { + rdm->errored = 1; + return 0; + } + } + return x + 1; +} + +static uint64_t +parse_opt_integer_62 (struct rust_demangler *rdm, char tag) +{ + if (!eat (rdm, tag)) + return 0; + return 1 + parse_integer_62 (rdm); +} + +static uint64_t +parse_disambiguator (struct rust_demangler *rdm) +{ + return parse_opt_integer_62 (rdm, 's'); +} + +static size_t +parse_hex_nibbles (struct rust_demangler *rdm, uint64_t *value) +{ + char c; + size_t hex_len; + + hex_len = 0; + *value = 0; + + while (!eat (rdm, '_')) + { + *value <<= 4; + + c = next (rdm); + if (ISDIGIT (c)) + *value |= c - '0'; + else if (c >= 'a' && c <= 'f') + *value |= 10 + (c - 'a'); + else + { + rdm->errored = 1; + return 0; + } + hex_len++; + } + + return hex_len; +} + struct rust_mangled_ident { /* ASCII part of the identifier. */ const char *ascii; size_t ascii_len; + + /* Punycode insertion codes for Unicode codepoints, if any. */ + const char *punycode; + size_t punycode_len; }; static struct rust_mangled_ident @@ -104,10 +197,16 @@ parse_ident (struct rust_demangler *rdm) { char c; size_t start, len; + int is_punycode = 0; struct rust_mangled_ident ident; ident.ascii = NULL; ident.ascii_len = 0; + ident.punycode = NULL; + ident.punycode_len = 0; + + if (rdm->version != -1) + is_punycode = eat (rdm, 'u'); c = next (rdm); if (!ISDIGIT (c)) @@ -121,6 +220,10 @@ parse_ident (struct rust_demangler *rdm) while (ISDIGIT (peek (rdm))) len = len * 10 + (next (rdm) - '0'); + /* Skip past the optional `_` separator (v0). */ + if (rdm->version != -1) + eat (rdm, '_'); + start = rdm->next; rdm->next += len; /* Check for overflows. */ @@ -133,6 +236,27 @@ parse_ident (struct rust_demangler *rdm) ident.ascii = rdm->sym + start; ident.ascii_len = len; + if (is_punycode) + { + ident.punycode_len = 0; + while (ident.ascii_len > 0) + { + ident.ascii_len--; + + /* The last '_' is a separator between ascii & punycode. */ + if (ident.ascii[ident.ascii_len] == '_') + break; + + ident.punycode_len++; + } + if (!ident.punycode_len) + { + rdm->errored = 1; + return ident; + } + ident.punycode = ident.ascii + (len - ident.punycode_len); + } + if (ident.ascii_len == 0) ident.ascii = NULL; @@ -144,12 +268,28 @@ parse_ident (struct rust_demangler *rdm) static void print_str (struct rust_demangler *rdm, const char *data, size_t len) { - if (!rdm->errored) + if (!rdm->errored && !rdm->skipping_printing) rdm->callback (data, len, rdm->callback_opaque); } #define PRINT(s) print_str (rdm, s, strlen (s)) +static void +print_uint64 (struct rust_demangler *rdm, uint64_t x) +{ + char s[21]; + snprintf (s, 21, "%" PRIu64, x); + PRINT (s); +} + +static void +print_uint64_hex (struct rust_demangler *rdm, uint64_t x) +{ + char s[17]; + snprintf (s, 17, "%" PRIx64, x); + PRINT (s); +} + /* Return a 0x0-0xf value if the char is 0-9a-f, and -1 otherwise. */ static int decode_lower_hex_nibble (char nibble) @@ -230,9 +370,14 @@ static void print_ident (struct rust_demangler *rdm, struct rust_mangled_ident ident) { char unescaped; - size_t len; + uint8_t *out, *p, d; + size_t len, cap, punycode_pos, j; + /* Punycode parameters and state. */ + uint32_t c; + size_t base, t_min, t_max, skew, damp, bias, i; + size_t delta, w, k, t; - if (rdm->errored) + if (rdm->errored || rdm->skipping_printing) return; if (rdm->version == -1) @@ -273,8 +418,7 @@ print_ident (struct rust_demangler *rdm, struct rust_mangled_ident ident) } else { - /* "." becomes "-" */ - PRINT ("-"); + PRINT ("."); len = 1; } } @@ -294,6 +438,830 @@ print_ident (struct rust_demangler *rdm, struct rust_mangled_ident ident) return; } + + if (!ident.punycode) + { + print_str (rdm, ident.ascii, ident.ascii_len); + return; + } + + len = 0; + cap = 4; + while (cap < ident.ascii_len) + { + cap *= 2; + /* Check for overflows. */ + if ((cap * 4) / 4 != cap) + { + rdm->errored = 1; + return; + } + } + + /* Store the output codepoints as groups of 4 UTF-8 bytes. */ + out = (uint8_t *)malloc (cap * 4); + if (!out) + { + rdm->errored = 1; + return; + } + + /* Populate initial output from ASCII fragment. */ + for (len = 0; len < ident.ascii_len; len++) + { + p = out + 4 * len; + p[0] = 0; + p[1] = 0; + p[2] = 0; + p[3] = ident.ascii[len]; + } + + /* Punycode parameters and initial state. */ + base = 36; + t_min = 1; + t_max = 26; + skew = 38; + damp = 700; + bias = 72; + i = 0; + c = 0x80; + + punycode_pos = 0; + while (punycode_pos < ident.punycode_len) + { + /* Read one delta value. */ + delta = 0; + w = 1; + k = 0; + do + { + k += base; + t = k < bias ? 0 : (k - bias); + if (t < t_min) + t = t_min; + if (t > t_max) + t = t_max; + + if (punycode_pos >= ident.punycode_len) + goto cleanup; + d = ident.punycode[punycode_pos++]; + + if (ISLOWER (d)) + d = d - 'a'; + else if (ISDIGIT (d)) + d = 26 + (d - '0'); + else + { + rdm->errored = 1; + goto cleanup; + } + + delta += d * w; + w *= base - t; + } + while (d >= t); + + /* Compute the new insert position and character. */ + len++; + i += delta; + c += i / len; + i %= len; + + /* Ensure enough space is available. */ + if (cap < len) + { + cap *= 2; + /* Check for overflows. */ + if ((cap * 4) / 4 != cap || cap < len) + { + rdm->errored = 1; + goto cleanup; + } + } + p = (uint8_t *)realloc (out, cap * 4); + if (!p) + { + rdm->errored = 1; + goto cleanup; + } + out = p; + + /* Move the characters after the insert position. */ + p = out + i * 4; + memmove (p + 4, p, (len - i - 1) * 4); + + /* Insert the new character, as UTF-8 bytes. */ + p[0] = c >= 0x10000 ? 0xf0 | (c >> 18) : 0; + p[1] = c >= 0x800 ? (c < 0x10000 ? 0xe0 : 0x80) | ((c >> 12) & 0x3f) : 0; + p[2] = (c < 0x800 ? 0xc0 : 0x80) | ((c >> 6) & 0x3f); + p[3] = 0x80 | (c & 0x3f); + + /* If there are no more deltas, decoding is complete. */ + if (punycode_pos == ident.punycode_len) + break; + + i++; + + /* Perform bias adaptation. */ + delta /= damp; + damp = 2; + + delta += delta / len; + k = 0; + while (delta > ((base - t_min) * t_max) / 2) + { + delta /= base - t_min; + k += base; + } + bias = k + ((base - t_min + 1) * delta) / (delta + skew); + } + + /* Remove all the 0 bytes to leave behind an UTF-8 string. */ + for (i = 0, j = 0; i < len * 4; i++) + if (out[i] != 0) + out[j++] = out[i]; + + print_str (rdm, (const char *)out, j); + +cleanup: + free (out); +} + +/* Print the lifetime according to the previously decoded index. + An index of `0` always refers to `'_`, but starting with `1`, + indices refer to late-bound lifetimes introduced by a binder. */ +static void +print_lifetime_from_index (struct rust_demangler *rdm, uint64_t lt) +{ + char c; + uint64_t depth; + + PRINT ("'"); + if (lt == 0) + { + PRINT ("_"); + return; + } + + depth = rdm->bound_lifetime_depth - lt; + /* Try to print lifetimes alphabetically first. */ + if (depth < 26) + { + c = 'a' + depth; + print_str (rdm, &c, 1); + } + else + { + /* Use `'_123` after running out of letters. */ + PRINT ("_"); + print_uint64 (rdm, depth); + } +} + +/* Demangling functions. */ + +static void demangle_binder (struct rust_demangler *rdm); +static void demangle_path (struct rust_demangler *rdm, int in_value); +static void demangle_generic_arg (struct rust_demangler *rdm); +static void demangle_type (struct rust_demangler *rdm); +static int demangle_path_maybe_open_generics (struct rust_demangler *rdm); +static void demangle_dyn_trait (struct rust_demangler *rdm); +static void demangle_const (struct rust_demangler *rdm); +static void demangle_const_uint (struct rust_demangler *rdm); +static void demangle_const_int (struct rust_demangler *rdm); +static void demangle_const_bool (struct rust_demangler *rdm); +static void demangle_const_char (struct rust_demangler *rdm); + +/* Optionally enter a binder ('G') for late-bound lifetimes, + printing e.g. `for<'a, 'b> `, and make those lifetimes visible + to the caller (via depth level, which the caller should reset). */ +static void +demangle_binder (struct rust_demangler *rdm) +{ + uint64_t i, bound_lifetimes; + + if (rdm->errored) + return; + + bound_lifetimes = parse_opt_integer_62 (rdm, 'G'); + if (bound_lifetimes > 0) + { + PRINT ("for<"); + for (i = 0; i < bound_lifetimes; i++) + { + if (i > 0) + PRINT (", "); + rdm->bound_lifetime_depth++; + print_lifetime_from_index (rdm, 1); + } + PRINT ("> "); + } +} + +static void +demangle_path (struct rust_demangler *rdm, int in_value) +{ + char tag, ns; + int was_skipping_printing; + size_t i, backref, old_next; + uint64_t dis; + struct rust_mangled_ident name; + + if (rdm->errored) + return; + + switch (tag = next (rdm)) + { + case 'C': + dis = parse_disambiguator (rdm); + name = parse_ident (rdm); + + print_ident (rdm, name); + if (rdm->verbose) + { + PRINT ("["); + print_uint64_hex (rdm, dis); + PRINT ("]"); + } + break; + case 'N': + ns = next (rdm); + if (!ISLOWER (ns) && !ISUPPER (ns)) + { + rdm->errored = 1; + return; + } + + demangle_path (rdm, in_value); + + dis = parse_disambiguator (rdm); + name = parse_ident (rdm); + + if (ISUPPER (ns)) + { + /* Special namespaces, like closures and shims. */ + PRINT ("::{"); + switch (ns) + { + case 'C': + PRINT ("closure"); + break; + case 'S': + PRINT ("shim"); + break; + default: + print_str (rdm, &ns, 1); + } + if (name.ascii || name.punycode) + { + PRINT (":"); + print_ident (rdm, name); + } + PRINT ("#"); + print_uint64 (rdm, dis); + PRINT ("}"); + } + else + { + /* Implementation-specific/unspecified namespaces. */ + + if (name.ascii || name.punycode) + { + PRINT ("::"); + print_ident (rdm, name); + } + } + break; + case 'M': + case 'X': + /* Ignore the `impl`'s own path.*/ + parse_disambiguator (rdm); + was_skipping_printing = rdm->skipping_printing; + rdm->skipping_printing = 1; + demangle_path (rdm, in_value); + rdm->skipping_printing = was_skipping_printing; + /* fallthrough */ + case 'Y': + PRINT ("<"); + demangle_type (rdm); + if (tag != 'M') + { + PRINT (" as "); + demangle_path (rdm, 0); + } + PRINT (">"); + break; + case 'I': + demangle_path (rdm, in_value); + if (in_value) + PRINT ("::"); + PRINT ("<"); + for (i = 0; !rdm->errored && !eat (rdm, 'E'); i++) + { + if (i > 0) + PRINT (", "); + demangle_generic_arg (rdm); + } + PRINT (">"); + break; + case 'B': + backref = parse_integer_62 (rdm); + if (!rdm->skipping_printing) + { + old_next = rdm->next; + rdm->next = backref; + demangle_path (rdm, in_value); + rdm->next = old_next; + } + break; + default: + rdm->errored = 1; + return; + } +} + +static void +demangle_generic_arg (struct rust_demangler *rdm) +{ + uint64_t lt; + if (eat (rdm, 'L')) + { + lt = parse_integer_62 (rdm); + print_lifetime_from_index (rdm, lt); + } + else if (eat (rdm, 'K')) + demangle_const (rdm); + else + demangle_type (rdm); +} + +static const char * +basic_type (char tag) +{ + switch (tag) + { + case 'b': + return "bool"; + case 'c': + return "char"; + case 'e': + return "str"; + case 'u': + return "()"; + case 'a': + return "i8"; + case 's': + return "i16"; + case 'l': + return "i32"; + case 'x': + return "i64"; + case 'n': + return "i128"; + case 'i': + return "isize"; + case 'h': + return "u8"; + case 't': + return "u16"; + case 'm': + return "u32"; + case 'y': + return "u64"; + case 'o': + return "u128"; + case 'j': + return "usize"; + case 'f': + return "f32"; + case 'd': + return "f64"; + case 'z': + return "!"; + case 'p': + return "_"; + case 'v': + return "..."; + + default: + return NULL; + } +} + +static void +demangle_type (struct rust_demangler *rdm) +{ + char tag; + size_t i, old_next, backref; + uint64_t lt, old_bound_lifetime_depth; + const char *basic; + struct rust_mangled_ident abi; + + if (rdm->errored) + return; + + tag = next (rdm); + + basic = basic_type (tag); + if (basic) + { + PRINT (basic); + return; + } + + switch (tag) + { + case 'R': + case 'Q': + PRINT ("&"); + if (eat (rdm, 'L')) + { + lt = parse_integer_62 (rdm); + if (lt) + { + print_lifetime_from_index (rdm, lt); + PRINT (" "); + } + } + if (tag != 'R') + PRINT ("mut "); + demangle_type (rdm); + break; + case 'P': + case 'O': + PRINT ("*"); + if (tag != 'P') + PRINT ("mut "); + else + PRINT ("const "); + demangle_type (rdm); + break; + case 'A': + case 'S': + PRINT ("["); + demangle_type (rdm); + if (tag == 'A') + { + PRINT ("; "); + demangle_const (rdm); + } + PRINT ("]"); + break; + case 'T': + PRINT ("("); + for (i = 0; !rdm->errored && !eat (rdm, 'E'); i++) + { + if (i > 0) + PRINT (", "); + demangle_type (rdm); + } + if (i == 1) + PRINT (","); + PRINT (")"); + break; + case 'F': + old_bound_lifetime_depth = rdm->bound_lifetime_depth; + demangle_binder (rdm); + + if (eat (rdm, 'U')) + PRINT ("unsafe "); + + if (eat (rdm, 'K')) + { + if (eat (rdm, 'C')) + { + abi.ascii = "C"; + abi.ascii_len = 1; + } + else + { + abi = parse_ident (rdm); + if (!abi.ascii || abi.punycode) + { + rdm->errored = 1; + goto restore; + } + } + + PRINT ("extern \""); + + /* If the ABI had any `-`, they were replaced with `_`, + so the parts between `_` have to be re-joined with `-`. */ + for (i = 0; i < abi.ascii_len; i++) + { + if (abi.ascii[i] == '_') + { + print_str (rdm, abi.ascii, i); + PRINT ("-"); + abi.ascii += i + 1; + abi.ascii_len -= i + 1; + i = 0; + } + } + print_str (rdm, abi.ascii, abi.ascii_len); + + PRINT ("\" "); + } + + PRINT ("fn("); + for (i = 0; !rdm->errored && !eat (rdm, 'E'); i++) + { + if (i > 0) + PRINT (", "); + demangle_type (rdm); + } + PRINT (")"); + + if (eat (rdm, 'u')) + { + /* Skip printing the return type if it's 'u', i.e. `()`. */ + } + else + { + PRINT (" -> "); + demangle_type (rdm); + } + + /* Restore `bound_lifetime_depth` to outside the binder. */ + restore: + rdm->bound_lifetime_depth = old_bound_lifetime_depth; + break; + case 'D': + PRINT ("dyn "); + + old_bound_lifetime_depth = rdm->bound_lifetime_depth; + demangle_binder (rdm); + + for (i = 0; !rdm->errored && !eat (rdm, 'E'); i++) + { + if (i > 0) + PRINT (" + "); + demangle_dyn_trait (rdm); + } + + /* Restore `bound_lifetime_depth` to outside the binder. */ + rdm->bound_lifetime_depth = old_bound_lifetime_depth; + + if (!eat (rdm, 'L')) + { + rdm->errored = 1; + return; + } + lt = parse_integer_62 (rdm); + if (lt) + { + PRINT (" + "); + print_lifetime_from_index (rdm, lt); + } + break; + case 'B': + backref = parse_integer_62 (rdm); + if (!rdm->skipping_printing) + { + old_next = rdm->next; + rdm->next = backref; + demangle_type (rdm); + rdm->next = old_next; + } + break; + default: + /* Go back to the tag, so `demangle_path` also sees it. */ + rdm->next--; + demangle_path (rdm, 0); + } +} + +/* A trait in a trait object may have some "existential projections" + (i.e. associated type bindings) after it, which should be printed + in the `<...>` of the trait, e.g. `dyn Trait<T, U, Assoc=X>`. + To this end, this method will keep the `<...>` of an 'I' path + open, by omitting the `>`, and return `Ok(true)` in that case. */ +static int +demangle_path_maybe_open_generics (struct rust_demangler *rdm) +{ + int open; + size_t i, old_next, backref; + + open = 0; + + if (rdm->errored) + return open; + + if (eat (rdm, 'B')) + { + backref = parse_integer_62 (rdm); + if (!rdm->skipping_printing) + { + old_next = rdm->next; + rdm->next = backref; + open = demangle_path_maybe_open_generics (rdm); + rdm->next = old_next; + } + } + else if (eat (rdm, 'I')) + { + demangle_path (rdm, 0); + PRINT ("<"); + open = 1; + for (i = 0; !rdm->errored && !eat (rdm, 'E'); i++) + { + if (i > 0) + PRINT (", "); + demangle_generic_arg (rdm); + } + } + else + demangle_path (rdm, 0); + return open; +} + +static void +demangle_dyn_trait (struct rust_demangler *rdm) +{ + int open; + struct rust_mangled_ident name; + + if (rdm->errored) + return; + + open = demangle_path_maybe_open_generics (rdm); + + while (eat (rdm, 'p')) + { + if (!open) + PRINT ("<"); + else + PRINT (", "); + open = 1; + + name = parse_ident (rdm); + print_ident (rdm, name); + PRINT (" = "); + demangle_type (rdm); + } + + if (open) + PRINT (">"); +} + +static void +demangle_const (struct rust_demangler *rdm) +{ + char ty_tag; + size_t old_next, backref; + + if (rdm->errored) + return; + + if (eat (rdm, 'B')) + { + backref = parse_integer_62 (rdm); + if (!rdm->skipping_printing) + { + old_next = rdm->next; + rdm->next = backref; + demangle_const (rdm); + rdm->next = old_next; + } + return; + } + + ty_tag = next (rdm); + switch (ty_tag) + { + /* Placeholder. */ + case 'p': + PRINT ("_"); + return; + + /* Unsigned integer types. */ + case 'h': + case 't': + case 'm': + case 'y': + case 'o': + case 'j': + demangle_const_uint (rdm); + break; + + /* Signed integer types. */ + case 'a': + case 's': + case 'l': + case 'x': + case 'n': + case 'i': + demangle_const_int (rdm); + break; + + /* Boolean. */ + case 'b': + demangle_const_bool (rdm); + break; + + /* Character. */ + case 'c': + demangle_const_char (rdm); + break; + + default: + rdm->errored = 1; + return; + } + + if (rdm->errored) + return; + + if (rdm->verbose) + { + PRINT (": "); + PRINT (basic_type (ty_tag)); + } +} + +static void +demangle_const_uint (struct rust_demangler *rdm) +{ + size_t hex_len; + uint64_t value; + + if (rdm->errored) + return; + + hex_len = parse_hex_nibbles (rdm, &value); + + if (hex_len > 16) + { + /* Print anything that doesn't fit in `uint64_t` verbatim. */ + PRINT ("0x"); + print_str (rdm, rdm->sym + (rdm->next - hex_len), hex_len); + } + else if (hex_len > 0) + print_uint64 (rdm, value); + else + rdm->errored = 1; +} + +static void +demangle_const_int (struct rust_demangler *rdm) +{ + if (eat (rdm, 'n')) + PRINT ("-"); + demangle_const_uint (rdm); +} + +static void +demangle_const_bool (struct rust_demangler *rdm) +{ + uint64_t value; + + if (parse_hex_nibbles (rdm, &value) != 1) + { + rdm->errored = 1; + return; + } + + if (value == 0) + PRINT ("false"); + else if (value == 1) + PRINT ("true"); + else + rdm->errored = 1; +} + +static void +demangle_const_char (struct rust_demangler *rdm) +{ + size_t hex_len; + uint64_t value; + + hex_len = parse_hex_nibbles (rdm, &value); + + if (hex_len == 0 || hex_len > 8) + { + rdm->errored = 1; + return; + } + + /* Match Rust's character "debug" output as best as we can. */ + PRINT ("'"); + if (value == '\t') + PRINT ("\\t"); + else if (value == '\r') + PRINT ("\\r"); + else if (value == '\n') + PRINT ("\\n"); + else if (value > ' ' && value < '~') + /* Rust also considers many non-ASCII codepoints to be printable, but + that logic is not easily ported to C. */ + print_str (rdm, (char *) &value, 1); + else + { + PRINT ("\\u{"); + print_uint64_hex (rdm, value); + PRINT ("}"); + } + PRINT ("'"); } /* A legacy hash is the prefix "h" followed by 16 lowercase hex digits. @@ -345,11 +1313,15 @@ rust_demangle_callback (const char *mangled, int options, rdm.next = 0; rdm.errored = 0; + rdm.skipping_printing = 0; rdm.verbose = (options & DMGL_VERBOSE) != 0; rdm.version = 0; + rdm.bound_lifetime_depth = 0; - /* Rust symbols always start with _ZN (legacy). */ - if (rdm.sym[0] == '_' && rdm.sym[1] == 'Z' && rdm.sym[2] == 'N') + /* Rust symbols always start with _R (v0) or _ZN (legacy). */ + if (rdm.sym[0] == '_' && rdm.sym[1] == 'R') + rdm.sym += 2; + else if (rdm.sym[0] == '_' && rdm.sym[1] == 'Z' && rdm.sym[2] == 'N') { rdm.sym += 3; rdm.version = -1; @@ -357,7 +1329,11 @@ rust_demangle_callback (const char *mangled, int options, else return 0; - /* Legacy Rust symbols use only [_0-9a-zA-Z.:$] characters. */ + /* Paths (v0) always start with uppercase characters. */ + if (rdm.version != -1 && !ISUPPER (rdm.sym[0])) + return 0; + + /* Rust symbols (v0) use only [_0-9a-zA-Z] characters. */ for (p = rdm.sym; *p; p++) { rdm.sym_len++; @@ -365,6 +1341,7 @@ rust_demangle_callback (const char *mangled, int options, if (*p == '_' || ISALNUM (*p)) continue; + /* Legacy Rust symbols can also contain [.:$] characters. */ if (rdm.version == -1 && (*p == '$' || *p == '.' || *p == ':')) continue; @@ -418,7 +1395,19 @@ rust_demangle_callback (const char *mangled, int options, while (rdm.next < rdm.sym_len); } else - return 0; + { + demangle_path (&rdm, 1); + + /* Skip instantiating crate. */ + if (!rdm.errored && rdm.next < rdm.sym_len) + { + rdm.skipping_printing = 1; + demangle_path (&rdm, 0); + } + + /* It's an error to not reach the end. */ + rdm.errored |= rdm.next != rdm.sym_len; + } return !rdm.errored; } diff --git a/libiberty/testsuite/rust-demangle-expected b/libiberty/testsuite/rust-demangle-expected index 74774794736..1351b9a2bd4 100644 --- a/libiberty/testsuite/rust-demangle-expected +++ b/libiberty/testsuite/rust-demangle-expected @@ -163,3 +163,51 @@ _ZN63_$LT$core..ptr..Unique$LT$T$GT$$u20$as$u20$core..ops..Deref$GT$5deref17h19f --format=rust _ZN11issue_609253foo37Foo$LT$issue_60925..llv$u6d$..Foo$GT$3foo17h059a991a004536adE issue_60925::foo::Foo<issue_60925::llvm::Foo>::foo +# +--format=rust +_ZN4core3ops8function6FnOnce40call_once$u7b$$u7b$vtable.shim$u7d$$u7d$17h000b1ad6c4f30bd6E +core::ops::function::FnOnce::call_once{{vtable.shim}} +# UTF8 idents. +--format=rust +_RNqCs4fqI2P2rA04_11utf8_identsu30____7hkackfecea1cbdathfdh9hlq6y +utf8_idents::საჭმელად_გემრიელი_სადილი +# Crate with leading digit. +--format=rust +_RNvC6_123foo3bar +123foo::bar +# +--format=rust +_RMCs4fqI2P2rA04_13const_genericINtB0_8UnsignedKhb_E +<const_generic::Unsigned<11>> +# +--format=rust +_RMCs4fqI2P2rA04_13const_genericINtB0_6SignedKs98_E +<const_generic::Signed<152>> +# +--format=rust +_RMCs4fqI2P2rA04_13const_genericINtB0_6SignedKanb_E +<const_generic::Signed<-11>> +# +--format=rust +_RMCs4fqI2P2rA04_13const_genericINtB0_4BoolKb0_E +<const_generic::Bool<false>> +# +--format=rust +_RMCs4fqI2P2rA04_13const_genericINtB0_4BoolKb1_E +<const_generic::Bool<true>> +# +--format=rust +_RMCs4fqI2P2rA04_13const_genericINtB0_4CharKc76_E +<const_generic::Char<'v'>> +# +--format=rust +_RMCs4fqI2P2rA04_13const_genericINtB0_4CharKca_E +<const_generic::Char<'\n'>> +# +--format=rust +_RMCs4fqI2P2rA04_13const_genericINtB0_4CharKc2202_E +<const_generic::Char<'\u{2202}'>> +# +--format=rust +_RNvNvMCs4fqI2P2rA04_13const_genericINtB4_3FooKpE3foo3FOO +<const_generic::Foo<_>>::foo::FOO
On 11/1/20 10:26 AM, Nikhil Benesch via Gcc-patches wrote: > > > On 11/1/20 6:57 AM, Eduard-Mihai Burtescu wrote: >> Reading the diff patch, the v0 changes look great. I wouldn't be too >> worried >> about the "printable character" aspect, there are similar >> Unicode-related >> issues elsewhere, e.g. the "non-control ASCII" comment in >> decode_legacy_escape >> (I suppose we could make it also go through the "print a non-control >> ASCII >> character or some escape sequence" logic you added, if you think that >> helps). > > No, it's entirely fine with me! I just wasn't sure if the small > deviations in output were acceptable. It sounds like they are. So I think the best path forward is to let you and Eduard-Mihai make the technical decisions about what bits are ready for the trunk. When y'all think something is ready, let's go ahead and get it installed and iterate on things that aren't quite ready yet. For bits y'all think are ready, ISTM that Eduard-Mihai should commit the changes. >> I can test the patch and upload the dataset tomorrow, but if you want >> to get >> something committed sooner (is there a deadline for the next >> release?), feel >> free to land the v0 changes (snprintf + const values) without the >> legacy ones. > > My understanding is that the GCC tree closes to new features on > November 16 (for "GCC 11 Stage 3"), but I'm not sure whether that > applies to libiberty or whether this patch would be classified as a > feature or a bugfix. > > I don't have commit rights (nor am I even a GCC developer). Just > wanted to tee things up for you and Ian this week. I'm very much > looking forward to the new demangling scheme and didn't want to be > just another +1 on the GitHub issue. > > So certainly no time pressure from me. But perhaps someone from the > GCC side can confirm whether we are under a bit of time pressure here > given the GCC 11 release. It's better to get it in sooner, but there is some degree of freedom depending on the impact of the changes. Changes in the rust demangler aren't likely to trigger codegen or ABI breakages in the compiler itself -- so with that in mind I think we should give this code a higher degree of freedom to land after the stage1 close deadline. jeff
On 11/6/20 12:09 PM, Jeff Law wrote: > So I think the best path forward is to let you and Eduard-Mihai make the > technical decisions about what bits are ready for the trunk. When y'all > think something is ready, let's go ahead and get it installed and > iterate on things that aren't quite ready yet. > > > For bits y'all think are ready, ISTM that Eduard-Mihai should commit the > changes. I've attached an updated version of the patch that contains some additional unit tests that eddyb noticed I lost. From my perspective, this is now ready for commit. Neither eddyb nor I have write access, so someone else will need to commit. (But please wait for eddyb to sign off too.) > It's better to get it in sooner, but there is some degree of freedom > depending on the impact of the changes. Changes in the rust demangler > aren't likely to trigger codegen or ABI breakages in the compiler itself > -- so with that in mind I think we should give this code a higher degree > of freedom to land after the stage1 close deadline. Got it. Thanks. That's very helpful context. Nikhil diff --git a/libiberty/rust-demangle.c b/libiberty/rust-demangle.c index b87365c85fe..08c615f6d8b 100644 --- a/libiberty/rust-demangle.c +++ b/libiberty/rust-demangle.c @@ -1,6 +1,7 @@ /* Demangler for the Rust programming language Copyright (C) 2016-2020 Free Software Foundation, Inc. Written by David Tolnay (dtolnay@gmail.com). + Rewritten by Eduard-Mihai Burtescu (eddyb@lyken.rs) for v0 support. This file is part of the libiberty library. Libiberty is free software; you can redistribute it and/or @@ -64,11 +65,16 @@ struct rust_demangler /* Non-zero if any error occurred. */ int errored; + /* Non-zero if nothing should be printed. */ + int skipping_printing; + /* Non-zero if printing should be verbose (e.g. include hashes). */ int verbose; /* Rust mangling version, with legacy mangling being -1. */ int version; + + uint64_t bound_lifetime_depth; }; /* Parsing functions. */ @@ -81,6 +87,18 @@ peek (const struct rust_demangler *rdm) return 0; } +static int +eat (struct rust_demangler *rdm, char c) +{ + if (peek (rdm) == c) + { + rdm->next++; + return 1; + } + else + return 0; +} + static char next (struct rust_demangler *rdm) { @@ -92,11 +110,87 @@ next (struct rust_demangler *rdm) return c; } +static uint64_t +parse_integer_62 (struct rust_demangler *rdm) +{ + char c; + uint64_t x; + + if (eat (rdm, '_')) + return 0; + + x = 0; + while (!eat (rdm, '_')) + { + c = next (rdm); + x *= 62; + if (ISDIGIT (c)) + x += c - '0'; + else if (ISLOWER (c)) + x += 10 + (c - 'a'); + else if (ISUPPER (c)) + x += 10 + 26 + (c - 'A'); + else + { + rdm->errored = 1; + return 0; + } + } + return x + 1; +} + +static uint64_t +parse_opt_integer_62 (struct rust_demangler *rdm, char tag) +{ + if (!eat (rdm, tag)) + return 0; + return 1 + parse_integer_62 (rdm); +} + +static uint64_t +parse_disambiguator (struct rust_demangler *rdm) +{ + return parse_opt_integer_62 (rdm, 's'); +} + +static size_t +parse_hex_nibbles (struct rust_demangler *rdm, uint64_t *value) +{ + char c; + size_t hex_len; + + hex_len = 0; + *value = 0; + + while (!eat (rdm, '_')) + { + *value <<= 4; + + c = next (rdm); + if (ISDIGIT (c)) + *value |= c - '0'; + else if (c >= 'a' && c <= 'f') + *value |= 10 + (c - 'a'); + else + { + rdm->errored = 1; + return 0; + } + hex_len++; + } + + return hex_len; +} + struct rust_mangled_ident { /* ASCII part of the identifier. */ const char *ascii; size_t ascii_len; + + /* Punycode insertion codes for Unicode codepoints, if any. */ + const char *punycode; + size_t punycode_len; }; static struct rust_mangled_ident @@ -104,10 +198,16 @@ parse_ident (struct rust_demangler *rdm) { char c; size_t start, len; + int is_punycode = 0; struct rust_mangled_ident ident; ident.ascii = NULL; ident.ascii_len = 0; + ident.punycode = NULL; + ident.punycode_len = 0; + + if (rdm->version != -1) + is_punycode = eat (rdm, 'u'); c = next (rdm); if (!ISDIGIT (c)) @@ -121,6 +221,10 @@ parse_ident (struct rust_demangler *rdm) while (ISDIGIT (peek (rdm))) len = len * 10 + (next (rdm) - '0'); + /* Skip past the optional `_` separator (v0). */ + if (rdm->version != -1) + eat (rdm, '_'); + start = rdm->next; rdm->next += len; /* Check for overflows. */ @@ -133,6 +237,27 @@ parse_ident (struct rust_demangler *rdm) ident.ascii = rdm->sym + start; ident.ascii_len = len; + if (is_punycode) + { + ident.punycode_len = 0; + while (ident.ascii_len > 0) + { + ident.ascii_len--; + + /* The last '_' is a separator between ascii & punycode. */ + if (ident.ascii[ident.ascii_len] == '_') + break; + + ident.punycode_len++; + } + if (!ident.punycode_len) + { + rdm->errored = 1; + return ident; + } + ident.punycode = ident.ascii + (len - ident.punycode_len); + } + if (ident.ascii_len == 0) ident.ascii = NULL; @@ -144,12 +269,28 @@ parse_ident (struct rust_demangler *rdm) static void print_str (struct rust_demangler *rdm, const char *data, size_t len) { - if (!rdm->errored) + if (!rdm->errored && !rdm->skipping_printing) rdm->callback (data, len, rdm->callback_opaque); } #define PRINT(s) print_str (rdm, s, strlen (s)) +static void +print_uint64 (struct rust_demangler *rdm, uint64_t x) +{ + char s[21]; + snprintf (s, 21, "%" PRIu64, x); + PRINT (s); +} + +static void +print_uint64_hex (struct rust_demangler *rdm, uint64_t x) +{ + char s[17]; + snprintf (s, 17, "%" PRIx64, x); + PRINT (s); +} + /* Return a 0x0-0xf value if the char is 0-9a-f, and -1 otherwise. */ static int decode_lower_hex_nibble (char nibble) @@ -230,9 +371,14 @@ static void print_ident (struct rust_demangler *rdm, struct rust_mangled_ident ident) { char unescaped; - size_t len; + uint8_t *out, *p, d; + size_t len, cap, punycode_pos, j; + /* Punycode parameters and state. */ + uint32_t c; + size_t base, t_min, t_max, skew, damp, bias, i; + size_t delta, w, k, t; - if (rdm->errored) + if (rdm->errored || rdm->skipping_printing) return; if (rdm->version == -1) @@ -273,8 +419,7 @@ print_ident (struct rust_demangler *rdm, struct rust_mangled_ident ident) } else { - /* "." becomes "-" */ - PRINT ("-"); + PRINT ("."); len = 1; } } @@ -294,6 +439,830 @@ print_ident (struct rust_demangler *rdm, struct rust_mangled_ident ident) return; } + + if (!ident.punycode) + { + print_str (rdm, ident.ascii, ident.ascii_len); + return; + } + + len = 0; + cap = 4; + while (cap < ident.ascii_len) + { + cap *= 2; + /* Check for overflows. */ + if ((cap * 4) / 4 != cap) + { + rdm->errored = 1; + return; + } + } + + /* Store the output codepoints as groups of 4 UTF-8 bytes. */ + out = (uint8_t *)malloc (cap * 4); + if (!out) + { + rdm->errored = 1; + return; + } + + /* Populate initial output from ASCII fragment. */ + for (len = 0; len < ident.ascii_len; len++) + { + p = out + 4 * len; + p[0] = 0; + p[1] = 0; + p[2] = 0; + p[3] = ident.ascii[len]; + } + + /* Punycode parameters and initial state. */ + base = 36; + t_min = 1; + t_max = 26; + skew = 38; + damp = 700; + bias = 72; + i = 0; + c = 0x80; + + punycode_pos = 0; + while (punycode_pos < ident.punycode_len) + { + /* Read one delta value. */ + delta = 0; + w = 1; + k = 0; + do + { + k += base; + t = k < bias ? 0 : (k - bias); + if (t < t_min) + t = t_min; + if (t > t_max) + t = t_max; + + if (punycode_pos >= ident.punycode_len) + goto cleanup; + d = ident.punycode[punycode_pos++]; + + if (ISLOWER (d)) + d = d - 'a'; + else if (ISDIGIT (d)) + d = 26 + (d - '0'); + else + { + rdm->errored = 1; + goto cleanup; + } + + delta += d * w; + w *= base - t; + } + while (d >= t); + + /* Compute the new insert position and character. */ + len++; + i += delta; + c += i / len; + i %= len; + + /* Ensure enough space is available. */ + if (cap < len) + { + cap *= 2; + /* Check for overflows. */ + if ((cap * 4) / 4 != cap || cap < len) + { + rdm->errored = 1; + goto cleanup; + } + } + p = (uint8_t *)realloc (out, cap * 4); + if (!p) + { + rdm->errored = 1; + goto cleanup; + } + out = p; + + /* Move the characters after the insert position. */ + p = out + i * 4; + memmove (p + 4, p, (len - i - 1) * 4); + + /* Insert the new character, as UTF-8 bytes. */ + p[0] = c >= 0x10000 ? 0xf0 | (c >> 18) : 0; + p[1] = c >= 0x800 ? (c < 0x10000 ? 0xe0 : 0x80) | ((c >> 12) & 0x3f) : 0; + p[2] = (c < 0x800 ? 0xc0 : 0x80) | ((c >> 6) & 0x3f); + p[3] = 0x80 | (c & 0x3f); + + /* If there are no more deltas, decoding is complete. */ + if (punycode_pos == ident.punycode_len) + break; + + i++; + + /* Perform bias adaptation. */ + delta /= damp; + damp = 2; + + delta += delta / len; + k = 0; + while (delta > ((base - t_min) * t_max) / 2) + { + delta /= base - t_min; + k += base; + } + bias = k + ((base - t_min + 1) * delta) / (delta + skew); + } + + /* Remove all the 0 bytes to leave behind an UTF-8 string. */ + for (i = 0, j = 0; i < len * 4; i++) + if (out[i] != 0) + out[j++] = out[i]; + + print_str (rdm, (const char *)out, j); + +cleanup: + free (out); +} + +/* Print the lifetime according to the previously decoded index. + An index of `0` always refers to `'_`, but starting with `1`, + indices refer to late-bound lifetimes introduced by a binder. */ +static void +print_lifetime_from_index (struct rust_demangler *rdm, uint64_t lt) +{ + char c; + uint64_t depth; + + PRINT ("'"); + if (lt == 0) + { + PRINT ("_"); + return; + } + + depth = rdm->bound_lifetime_depth - lt; + /* Try to print lifetimes alphabetically first. */ + if (depth < 26) + { + c = 'a' + depth; + print_str (rdm, &c, 1); + } + else + { + /* Use `'_123` after running out of letters. */ + PRINT ("_"); + print_uint64 (rdm, depth); + } +} + +/* Demangling functions. */ + +static void demangle_binder (struct rust_demangler *rdm); +static void demangle_path (struct rust_demangler *rdm, int in_value); +static void demangle_generic_arg (struct rust_demangler *rdm); +static void demangle_type (struct rust_demangler *rdm); +static int demangle_path_maybe_open_generics (struct rust_demangler *rdm); +static void demangle_dyn_trait (struct rust_demangler *rdm); +static void demangle_const (struct rust_demangler *rdm); +static void demangle_const_uint (struct rust_demangler *rdm); +static void demangle_const_int (struct rust_demangler *rdm); +static void demangle_const_bool (struct rust_demangler *rdm); +static void demangle_const_char (struct rust_demangler *rdm); + +/* Optionally enter a binder ('G') for late-bound lifetimes, + printing e.g. `for<'a, 'b> `, and make those lifetimes visible + to the caller (via depth level, which the caller should reset). */ +static void +demangle_binder (struct rust_demangler *rdm) +{ + uint64_t i, bound_lifetimes; + + if (rdm->errored) + return; + + bound_lifetimes = parse_opt_integer_62 (rdm, 'G'); + if (bound_lifetimes > 0) + { + PRINT ("for<"); + for (i = 0; i < bound_lifetimes; i++) + { + if (i > 0) + PRINT (", "); + rdm->bound_lifetime_depth++; + print_lifetime_from_index (rdm, 1); + } + PRINT ("> "); + } +} + +static void +demangle_path (struct rust_demangler *rdm, int in_value) +{ + char tag, ns; + int was_skipping_printing; + size_t i, backref, old_next; + uint64_t dis; + struct rust_mangled_ident name; + + if (rdm->errored) + return; + + switch (tag = next (rdm)) + { + case 'C': + dis = parse_disambiguator (rdm); + name = parse_ident (rdm); + + print_ident (rdm, name); + if (rdm->verbose) + { + PRINT ("["); + print_uint64_hex (rdm, dis); + PRINT ("]"); + } + break; + case 'N': + ns = next (rdm); + if (!ISLOWER (ns) && !ISUPPER (ns)) + { + rdm->errored = 1; + return; + } + + demangle_path (rdm, in_value); + + dis = parse_disambiguator (rdm); + name = parse_ident (rdm); + + if (ISUPPER (ns)) + { + /* Special namespaces, like closures and shims. */ + PRINT ("::{"); + switch (ns) + { + case 'C': + PRINT ("closure"); + break; + case 'S': + PRINT ("shim"); + break; + default: + print_str (rdm, &ns, 1); + } + if (name.ascii || name.punycode) + { + PRINT (":"); + print_ident (rdm, name); + } + PRINT ("#"); + print_uint64 (rdm, dis); + PRINT ("}"); + } + else + { + /* Implementation-specific/unspecified namespaces. */ + + if (name.ascii || name.punycode) + { + PRINT ("::"); + print_ident (rdm, name); + } + } + break; + case 'M': + case 'X': + /* Ignore the `impl`'s own path.*/ + parse_disambiguator (rdm); + was_skipping_printing = rdm->skipping_printing; + rdm->skipping_printing = 1; + demangle_path (rdm, in_value); + rdm->skipping_printing = was_skipping_printing; + /* fallthrough */ + case 'Y': + PRINT ("<"); + demangle_type (rdm); + if (tag != 'M') + { + PRINT (" as "); + demangle_path (rdm, 0); + } + PRINT (">"); + break; + case 'I': + demangle_path (rdm, in_value); + if (in_value) + PRINT ("::"); + PRINT ("<"); + for (i = 0; !rdm->errored && !eat (rdm, 'E'); i++) + { + if (i > 0) + PRINT (", "); + demangle_generic_arg (rdm); + } + PRINT (">"); + break; + case 'B': + backref = parse_integer_62 (rdm); + if (!rdm->skipping_printing) + { + old_next = rdm->next; + rdm->next = backref; + demangle_path (rdm, in_value); + rdm->next = old_next; + } + break; + default: + rdm->errored = 1; + return; + } +} + +static void +demangle_generic_arg (struct rust_demangler *rdm) +{ + uint64_t lt; + if (eat (rdm, 'L')) + { + lt = parse_integer_62 (rdm); + print_lifetime_from_index (rdm, lt); + } + else if (eat (rdm, 'K')) + demangle_const (rdm); + else + demangle_type (rdm); +} + +static const char * +basic_type (char tag) +{ + switch (tag) + { + case 'b': + return "bool"; + case 'c': + return "char"; + case 'e': + return "str"; + case 'u': + return "()"; + case 'a': + return "i8"; + case 's': + return "i16"; + case 'l': + return "i32"; + case 'x': + return "i64"; + case 'n': + return "i128"; + case 'i': + return "isize"; + case 'h': + return "u8"; + case 't': + return "u16"; + case 'm': + return "u32"; + case 'y': + return "u64"; + case 'o': + return "u128"; + case 'j': + return "usize"; + case 'f': + return "f32"; + case 'd': + return "f64"; + case 'z': + return "!"; + case 'p': + return "_"; + case 'v': + return "..."; + + default: + return NULL; + } +} + +static void +demangle_type (struct rust_demangler *rdm) +{ + char tag; + size_t i, old_next, backref; + uint64_t lt, old_bound_lifetime_depth; + const char *basic; + struct rust_mangled_ident abi; + + if (rdm->errored) + return; + + tag = next (rdm); + + basic = basic_type (tag); + if (basic) + { + PRINT (basic); + return; + } + + switch (tag) + { + case 'R': + case 'Q': + PRINT ("&"); + if (eat (rdm, 'L')) + { + lt = parse_integer_62 (rdm); + if (lt) + { + print_lifetime_from_index (rdm, lt); + PRINT (" "); + } + } + if (tag != 'R') + PRINT ("mut "); + demangle_type (rdm); + break; + case 'P': + case 'O': + PRINT ("*"); + if (tag != 'P') + PRINT ("mut "); + else + PRINT ("const "); + demangle_type (rdm); + break; + case 'A': + case 'S': + PRINT ("["); + demangle_type (rdm); + if (tag == 'A') + { + PRINT ("; "); + demangle_const (rdm); + } + PRINT ("]"); + break; + case 'T': + PRINT ("("); + for (i = 0; !rdm->errored && !eat (rdm, 'E'); i++) + { + if (i > 0) + PRINT (", "); + demangle_type (rdm); + } + if (i == 1) + PRINT (","); + PRINT (")"); + break; + case 'F': + old_bound_lifetime_depth = rdm->bound_lifetime_depth; + demangle_binder (rdm); + + if (eat (rdm, 'U')) + PRINT ("unsafe "); + + if (eat (rdm, 'K')) + { + if (eat (rdm, 'C')) + { + abi.ascii = "C"; + abi.ascii_len = 1; + } + else + { + abi = parse_ident (rdm); + if (!abi.ascii || abi.punycode) + { + rdm->errored = 1; + goto restore; + } + } + + PRINT ("extern \""); + + /* If the ABI had any `-`, they were replaced with `_`, + so the parts between `_` have to be re-joined with `-`. */ + for (i = 0; i < abi.ascii_len; i++) + { + if (abi.ascii[i] == '_') + { + print_str (rdm, abi.ascii, i); + PRINT ("-"); + abi.ascii += i + 1; + abi.ascii_len -= i + 1; + i = 0; + } + } + print_str (rdm, abi.ascii, abi.ascii_len); + + PRINT ("\" "); + } + + PRINT ("fn("); + for (i = 0; !rdm->errored && !eat (rdm, 'E'); i++) + { + if (i > 0) + PRINT (", "); + demangle_type (rdm); + } + PRINT (")"); + + if (eat (rdm, 'u')) + { + /* Skip printing the return type if it's 'u', i.e. `()`. */ + } + else + { + PRINT (" -> "); + demangle_type (rdm); + } + + /* Restore `bound_lifetime_depth` to outside the binder. */ + restore: + rdm->bound_lifetime_depth = old_bound_lifetime_depth; + break; + case 'D': + PRINT ("dyn "); + + old_bound_lifetime_depth = rdm->bound_lifetime_depth; + demangle_binder (rdm); + + for (i = 0; !rdm->errored && !eat (rdm, 'E'); i++) + { + if (i > 0) + PRINT (" + "); + demangle_dyn_trait (rdm); + } + + /* Restore `bound_lifetime_depth` to outside the binder. */ + rdm->bound_lifetime_depth = old_bound_lifetime_depth; + + if (!eat (rdm, 'L')) + { + rdm->errored = 1; + return; + } + lt = parse_integer_62 (rdm); + if (lt) + { + PRINT (" + "); + print_lifetime_from_index (rdm, lt); + } + break; + case 'B': + backref = parse_integer_62 (rdm); + if (!rdm->skipping_printing) + { + old_next = rdm->next; + rdm->next = backref; + demangle_type (rdm); + rdm->next = old_next; + } + break; + default: + /* Go back to the tag, so `demangle_path` also sees it. */ + rdm->next--; + demangle_path (rdm, 0); + } +} + +/* A trait in a trait object may have some "existential projections" + (i.e. associated type bindings) after it, which should be printed + in the `<...>` of the trait, e.g. `dyn Trait<T, U, Assoc=X>`. + To this end, this method will keep the `<...>` of an 'I' path + open, by omitting the `>`, and return `Ok(true)` in that case. */ +static int +demangle_path_maybe_open_generics (struct rust_demangler *rdm) +{ + int open; + size_t i, old_next, backref; + + open = 0; + + if (rdm->errored) + return open; + + if (eat (rdm, 'B')) + { + backref = parse_integer_62 (rdm); + if (!rdm->skipping_printing) + { + old_next = rdm->next; + rdm->next = backref; + open = demangle_path_maybe_open_generics (rdm); + rdm->next = old_next; + } + } + else if (eat (rdm, 'I')) + { + demangle_path (rdm, 0); + PRINT ("<"); + open = 1; + for (i = 0; !rdm->errored && !eat (rdm, 'E'); i++) + { + if (i > 0) + PRINT (", "); + demangle_generic_arg (rdm); + } + } + else + demangle_path (rdm, 0); + return open; +} + +static void +demangle_dyn_trait (struct rust_demangler *rdm) +{ + int open; + struct rust_mangled_ident name; + + if (rdm->errored) + return; + + open = demangle_path_maybe_open_generics (rdm); + + while (eat (rdm, 'p')) + { + if (!open) + PRINT ("<"); + else + PRINT (", "); + open = 1; + + name = parse_ident (rdm); + print_ident (rdm, name); + PRINT (" = "); + demangle_type (rdm); + } + + if (open) + PRINT (">"); +} + +static void +demangle_const (struct rust_demangler *rdm) +{ + char ty_tag; + size_t old_next, backref; + + if (rdm->errored) + return; + + if (eat (rdm, 'B')) + { + backref = parse_integer_62 (rdm); + if (!rdm->skipping_printing) + { + old_next = rdm->next; + rdm->next = backref; + demangle_const (rdm); + rdm->next = old_next; + } + return; + } + + ty_tag = next (rdm); + switch (ty_tag) + { + /* Placeholder. */ + case 'p': + PRINT ("_"); + return; + + /* Unsigned integer types. */ + case 'h': + case 't': + case 'm': + case 'y': + case 'o': + case 'j': + demangle_const_uint (rdm); + break; + + /* Signed integer types. */ + case 'a': + case 's': + case 'l': + case 'x': + case 'n': + case 'i': + demangle_const_int (rdm); + break; + + /* Boolean. */ + case 'b': + demangle_const_bool (rdm); + break; + + /* Character. */ + case 'c': + demangle_const_char (rdm); + break; + + default: + rdm->errored = 1; + return; + } + + if (rdm->errored) + return; + + if (rdm->verbose) + { + PRINT (": "); + PRINT (basic_type (ty_tag)); + } +} + +static void +demangle_const_uint (struct rust_demangler *rdm) +{ + size_t hex_len; + uint64_t value; + + if (rdm->errored) + return; + + hex_len = parse_hex_nibbles (rdm, &value); + + if (hex_len > 16) + { + /* Print anything that doesn't fit in `uint64_t` verbatim. */ + PRINT ("0x"); + print_str (rdm, rdm->sym + (rdm->next - hex_len), hex_len); + } + else if (hex_len > 0) + print_uint64 (rdm, value); + else + rdm->errored = 1; +} + +static void +demangle_const_int (struct rust_demangler *rdm) +{ + if (eat (rdm, 'n')) + PRINT ("-"); + demangle_const_uint (rdm); +} + +static void +demangle_const_bool (struct rust_demangler *rdm) +{ + uint64_t value; + + if (parse_hex_nibbles (rdm, &value) != 1) + { + rdm->errored = 1; + return; + } + + if (value == 0) + PRINT ("false"); + else if (value == 1) + PRINT ("true"); + else + rdm->errored = 1; +} + +static void +demangle_const_char (struct rust_demangler *rdm) +{ + size_t hex_len; + uint64_t value; + + hex_len = parse_hex_nibbles (rdm, &value); + + if (hex_len == 0 || hex_len > 8) + { + rdm->errored = 1; + return; + } + + /* Match Rust's character "debug" output as best as we can. */ + PRINT ("'"); + if (value == '\t') + PRINT ("\\t"); + else if (value == '\r') + PRINT ("\\r"); + else if (value == '\n') + PRINT ("\\n"); + else if (value > ' ' && value < '~') + /* Rust also considers many non-ASCII codepoints to be printable, but + that logic is not easily ported to C. */ + print_str (rdm, (char *) &value, 1); + else + { + PRINT ("\\u{"); + print_uint64_hex (rdm, value); + PRINT ("}"); + } + PRINT ("'"); } /* A legacy hash is the prefix "h" followed by 16 lowercase hex digits. @@ -345,11 +1314,15 @@ rust_demangle_callback (const char *mangled, int options, rdm.next = 0; rdm.errored = 0; + rdm.skipping_printing = 0; rdm.verbose = (options & DMGL_VERBOSE) != 0; rdm.version = 0; + rdm.bound_lifetime_depth = 0; - /* Rust symbols always start with _ZN (legacy). */ - if (rdm.sym[0] == '_' && rdm.sym[1] == 'Z' && rdm.sym[2] == 'N') + /* Rust symbols always start with _R (v0) or _ZN (legacy). */ + if (rdm.sym[0] == '_' && rdm.sym[1] == 'R') + rdm.sym += 2; + else if (rdm.sym[0] == '_' && rdm.sym[1] == 'Z' && rdm.sym[2] == 'N') { rdm.sym += 3; rdm.version = -1; @@ -357,7 +1330,11 @@ rust_demangle_callback (const char *mangled, int options, else return 0; - /* Legacy Rust symbols use only [_0-9a-zA-Z.:$] characters. */ + /* Paths (v0) always start with uppercase characters. */ + if (rdm.version != -1 && !ISUPPER (rdm.sym[0])) + return 0; + + /* Rust symbols (v0) use only [_0-9a-zA-Z] characters. */ for (p = rdm.sym; *p; p++) { rdm.sym_len++; @@ -365,6 +1342,7 @@ rust_demangle_callback (const char *mangled, int options, if (*p == '_' || ISALNUM (*p)) continue; + /* Legacy Rust symbols can also contain [.:$] characters. */ if (rdm.version == -1 && (*p == '$' || *p == '.' || *p == ':')) continue; @@ -418,7 +1396,19 @@ rust_demangle_callback (const char *mangled, int options, while (rdm.next < rdm.sym_len); } else - return 0; + { + demangle_path (&rdm, 1); + + /* Skip instantiating crate. */ + if (!rdm.errored && rdm.next < rdm.sym_len) + { + rdm.skipping_printing = 1; + demangle_path (&rdm, 0); + } + + /* It's an error to not reach the end. */ + rdm.errored |= rdm.next != rdm.sym_len; + } return !rdm.errored; } diff --git a/libiberty/testsuite/rust-demangle-expected b/libiberty/testsuite/rust-demangle-expected index 74774794736..7dca315d005 100644 --- a/libiberty/testsuite/rust-demangle-expected +++ b/libiberty/testsuite/rust-demangle-expected @@ -11,7 +11,7 @@ # ############ # -# Coverage Tests +# Coverage Tests (legacy) # # # Demangles as rust symbol. @@ -163,3 +163,135 @@ _ZN63_$LT$core..ptr..Unique$LT$T$GT$$u20$as$u20$core..ops..Deref$GT$5deref17h19f --format=rust _ZN11issue_609253foo37Foo$LT$issue_60925..llv$u6d$..Foo$GT$3foo17h059a991a004536adE issue_60925::foo::Foo<issue_60925::llvm::Foo>::foo +--format=rust +_ZN4core3ops8function6FnOnce40call_once$u7b$$u7b$vtable.shim$u7d$$u7d$17h000b1ad6c4f30bd6E +core::ops::function::FnOnce::call_once{{vtable.shim}} +# +############ +# +# Coverage Tests (v0) +# +# +# Crate with a leading digit. +--format=rust +_RNvC6_123foo3bar +123foo::bar +# UTF-8 identifiers. +--format=rust +_RNqCs4fqI2P2rA04_11utf8_identsu30____7hkackfecea1cbdathfdh9hlq6y +utf8_idents::საჭმელად_გემრიელი_სადილი +# Closure path elements. +--format=rust +_RNCNCNgCs6DXkGYLi8lr_2cc5spawn00B5_ +cc::spawn::{closure#0}::{closure#0} +# +--format=rust +_RNCINkXs25_NgCsbmNqQUJIY6D_4core5sliceINyB9_4IterhENuNgNoBb_4iter8iterator8Iterator9rpositionNCNgNpB9_6memchr7memrchrs_0E0Bb_ +<core::slice::Iter<u8> as core::iter::iterator::Iterator>::rposition::<core::slice::memchr::memrchr::{closure#1}>::{closure#0} +# dyn Trait ("trait object") types. +--format=rust +_RINbNbCskIICzLVDPPb_5alloc5alloc8box_freeDINbNiB4_5boxed5FnBoxuEp6OutputuEL_ECs1iopQbuBiw2_3std +alloc::alloc::box_free::<dyn alloc::boxed::FnBox<(), Output = ()>> +# Types with const generics parameters. +--format=rust +_RNvMC0INtC8arrayvec8ArrayVechKj7b_E3new +<arrayvec::ArrayVec<u8, 123>>::new +# +--format=rust +_RMCs4fqI2P2rA04_13const_genericINtB0_8UnsignedKhb_E +<const_generic::Unsigned<11>> +# +--format=rust +_RMCs4fqI2P2rA04_13const_genericINtB0_6SignedKs98_E +<const_generic::Signed<152>> +# +--format=rust +_RMCs4fqI2P2rA04_13const_genericINtB0_6SignedKanb_E +<const_generic::Signed<-11>> +# +--format=rust +_RMCs4fqI2P2rA04_13const_genericINtB0_4BoolKb0_E +<const_generic::Bool<false>> +# +--format=rust +_RMCs4fqI2P2rA04_13const_genericINtB0_4BoolKb1_E +<const_generic::Bool<true>> +# +--format=rust +_RMCs4fqI2P2rA04_13const_genericINtB0_4CharKc76_E +<const_generic::Char<'v'>> +# +--format=rust +_RMCs4fqI2P2rA04_13const_genericINtB0_4CharKca_E +<const_generic::Char<'\n'>> +# +--format=rust +_RMCs4fqI2P2rA04_13const_genericINtB0_4CharKc2202_E +<const_generic::Char<'\u{2202}'>> +# +--format=rust +_RNvNvMCs4fqI2P2rA04_13const_genericINtB4_3FooKpE3foo3FOO +<const_generic::Foo<_>>::foo::FOO +# +# All of the tests above but in auto mode instead: +# +# Crate with a leading digit. +--format=auto +_RNvC6_123foo3bar +123foo::bar +# UTF-8 identifiers. +--format=auto +_RNqCs4fqI2P2rA04_11utf8_identsu30____7hkackfecea1cbdathfdh9hlq6y +utf8_idents::საჭმელად_გემრიელი_სადილი +# Closure path elements. +--format=auto +_RNCNCNgCs6DXkGYLi8lr_2cc5spawn00B5_ +cc::spawn::{closure#0}::{closure#0} +# +--format=auto +_RNCINkXs25_NgCsbmNqQUJIY6D_4core5sliceINyB9_4IterhENuNgNoBb_4iter8iterator8Iterator9rpositionNCNgNpB9_6memchr7memrchrs_0E0Bb_ +<core::slice::Iter<u8> as core::iter::iterator::Iterator>::rposition::<core::slice::memchr::memrchr::{closure#1}>::{closure#0} +# dyn Trait ("trait object") types. +--format=auto +_RINbNbCskIICzLVDPPb_5alloc5alloc8box_freeDINbNiB4_5boxed5FnBoxuEp6OutputuEL_ECs1iopQbuBiw2_3std +alloc::alloc::box_free::<dyn alloc::boxed::FnBox<(), Output = ()>> +# Types with const generics parameters. +--format=auto +_RNvMC0INtC8arrayvec8ArrayVechKj7b_E3new +<arrayvec::ArrayVec<u8, 123>>::new +# +--format=auto +_RMCs4fqI2P2rA04_13const_genericINtB0_8UnsignedKhb_E +<const_generic::Unsigned<11>> +# +--format=auto +_RMCs4fqI2P2rA04_13const_genericINtB0_6SignedKs98_E +<const_generic::Signed<152>> +# +--format=auto +_RMCs4fqI2P2rA04_13const_genericINtB0_6SignedKanb_E +<const_generic::Signed<-11>> +# +--format=auto +_RMCs4fqI2P2rA04_13const_genericINtB0_4BoolKb0_E +<const_generic::Bool<false>> +# +--format=auto +_RMCs4fqI2P2rA04_13const_genericINtB0_4BoolKb1_E +<const_generic::Bool<true>> +# +--format=auto +_RMCs4fqI2P2rA04_13const_genericINtB0_4CharKc76_E +<const_generic::Char<'v'>> +# +--format=auto +_RMCs4fqI2P2rA04_13const_genericINtB0_4CharKca_E +<const_generic::Char<'\n'>> +# +--format=auto +_RMCs4fqI2P2rA04_13const_genericINtB0_4CharKc2202_E +<const_generic::Char<'\u{2202}'>> +# +--format=auto +_RNvNvMCs4fqI2P2rA04_13const_genericINtB4_3FooKpE3foo3FOO +<const_generic::Foo<_>>::foo::FOO
Hi everyone, Apologies again for the delay on my end, the past few weeks have been hectic and exhausting. The changes look good and pass the testing I was doing for my version of the patch. Feel free to commit Nikhil's latest patch for us. Thanks, - Eddy B. On Fri, Nov 13, 2020, at 08:42, Nikhil Benesch wrote: > On 11/6/20 12:09 PM, Jeff Law wrote: > > So I think the best path forward is to let you and Eduard-Mihai make the > > technical decisions about what bits are ready for the trunk. When y'all > > think something is ready, let's go ahead and get it installed and > > iterate on things that aren't quite ready yet. > > > > > > For bits y'all think are ready, ISTM that Eduard-Mihai should commit the > > changes. > > I've attached an updated version of the patch that contains some > additional unit tests that eddyb noticed I lost. From my perspective, > this is now ready for commit. > > Neither eddyb nor I have write access, so someone else will need to > commit. (But please wait for eddyb to sign off too.) > > > It's better to get it in sooner, but there is some degree of freedom > > depending on the impact of the changes. Changes in the rust demangler > > aren't likely to trigger codegen or ABI breakages in the compiler itself > > -- so with that in mind I think we should give this code a higher degree > > of freedom to land after the stage1 close deadline. > > Got it. Thanks. That's very helpful context. > > Nikhil > > Attachments: > * rust-demangle.patch
On Nov 13 2020, Nikhil Benesch via Gcc-patches wrote: > +static void > +demangle_const_char (struct rust_demangler *rdm) > +{ > + size_t hex_len; > + uint64_t value; > + > + hex_len = parse_hex_nibbles (rdm, &value); > + > + if (hex_len == 0 || hex_len > 8) > + { > + rdm->errored = 1; > + return; > + } > + > + /* Match Rust's character "debug" output as best as we can. */ > + PRINT ("'"); > + if (value == '\t') > + PRINT ("\\t"); > + else if (value == '\r') > + PRINT ("\\r"); > + else if (value == '\n') > + PRINT ("\\n"); > + else if (value > ' ' && value < '~') > + /* Rust also considers many non-ASCII codepoints to be printable, but > + that logic is not easily ported to C. */ > + print_str (rdm, (char *) &value, 1); * rust-demangle.c (demangle_const_char): Properly print the character value. diff --git a/libiberty/rust-demangle.c b/libiberty/rust-demangle.c index 6fd8f6a4db0..449941b56dc 100644 --- a/libiberty/rust-demangle.c +++ b/libiberty/rust-demangle.c @@ -1253,9 +1253,12 @@ demangle_const_char (struct rust_demangler *rdm) else if (value == '\n') PRINT ("\\n"); else if (value > ' ' && value < '~') - /* Rust also considers many non-ASCII codepoints to be printable, but - that logic is not easily ported to C. */ - print_str (rdm, (char *) &value, 1); + { + /* Rust also considers many non-ASCII codepoints to be printable, but + that logic is not easily ported to C. */ + char c = value; + print_str (rdm, &c, 1); + } else { PRINT ("\\u{");
On Tue, Apr 20, 2021 at 05:55:46PM +0200, Andreas Schwab wrote: > On Nov 13 2020, Nikhil Benesch via Gcc-patches wrote: > > > +static void > > +demangle_const_char (struct rust_demangler *rdm) > > +{ > > + size_t hex_len; > > + uint64_t value; > > + > > + hex_len = parse_hex_nibbles (rdm, &value); > > + > > + if (hex_len == 0 || hex_len > 8) > > + { > > + rdm->errored = 1; > > + return; > > + } > > + > > + /* Match Rust's character "debug" output as best as we can. */ > > + PRINT ("'"); > > + if (value == '\t') > > + PRINT ("\\t"); > > + else if (value == '\r') > > + PRINT ("\\r"); > > + else if (value == '\n') > > + PRINT ("\\n"); > > + else if (value > ' ' && value < '~') > > + /* Rust also considers many non-ASCII codepoints to be printable, but > > + that logic is not easily ported to C. */ > > + print_str (rdm, (char *) &value, 1); > > * rust-demangle.c (demangle_const_char): Properly print the > character value. Ok for trunk/11.2/10.4 . Jakub
From 689442d88b87453a697d2121e45a55071cb9056f Mon Sep 17 00:00:00 2001
From: Eduard-Mihai Burtescu <eddyb@lyken.rs>
Date: Fri, 13 Mar 2020 21:07:28 +0200
Subject: [PATCH] Support the new ("v0") mangling scheme in rust-demangle.
This is the libiberty (mainly for binutils/gdb) counterpart of
https://github.com/alexcrichton/rustc-demangle/pull/23.
Relevant links for the new Rust mangling scheme (aka "v0"):
* Rust RFC: https://github.com/rust-lang/rfcs/pull/2603
* tracking issue: https://github.com/rust-lang/rust/issues/60705
* implementation: https://github.com/rust-lang/rust/pull/57967
This implementation includes full support for UTF-8 identifiers
via punycode, so I've included a testcase for that as well.
2020-03-13 Eduard-Mihai Burtescu <eddyb@lyken.rs>
libiberty/ChangeLog:
* rust-demangle.c (struct rust_demangler): Add
skipping_printing and bound_lifetime_depth fields.
(eat): Add (v0-only).
(parse_integer_62): Add (v0-only).
(parse_opt_integer_62): Add (v0-only).
(parse_disambiguator): Add (v0-only).
(struct rust_mangled_ident): Add punycode{,_len} fields.
(parse_ident): Support v0 identifiers.
(print_str): Respect skipping_printing.
(print_uint64): Add (v0-only).
(print_uint64_hex): Add (v0-only).
(print_ident): Respect skipping_printing,
Support v0 identifiers.
(print_lifetime_from_index): Add (v0-only).
(demangle_binder): Add (v0-only).
(demangle_path): Add (v0-only).
(demangle_generic_arg): Add (v0-only).
(demangle_type): Add (v0-only).
(demangle_path_maybe_open_generics): Add (v0-only).
(demangle_dyn_trait): Add (v0-only).
(demangle_const): Add (v0-only).
(demangle_const_uint): Add (v0-only).
(basic_type): Add (v0-only).
(rust_demangle_callback): Support v0 symbols.
* testsuite/rust-demangle-expected: Add v0 testcases.
diff --git a/libiberty/rust-demangle.c b/libiberty/rust-demangle.c
index b87365c85fe9..9a31392458e0 100644
--- a/libiberty/rust-demangle.c
+++ b/libiberty/rust-demangle.c
@@ -1,6 +1,7 @@
/* Demangler for the Rust programming language
Copyright (C) 2016-2020 Free Software Foundation, Inc.
Written by David Tolnay (dtolnay@gmail.com).
+ Rewritten by Eduard-Mihai Burtescu (eddyb@lyken.rs) for v0 support.
This file is part of the libiberty library.
Libiberty is free software; you can redistribute it and/or
@@ -64,11 +65,16 @@ struct rust_demangler
/* Non-zero if any error occurred. */
int errored;
+ /* Non-zero if nothing should be printed. */
+ int skipping_printing;
+
/* Non-zero if printing should be verbose (e.g. include hashes). */
int verbose;
/* Rust mangling version, with legacy mangling being -1. */
int version;
+
+ uint64_t bound_lifetime_depth;
};
/* Parsing functions. */
@@ -81,6 +87,18 @@ peek (const struct rust_demangler *rdm)
return 0;
}
+static int
+eat (struct rust_demangler *rdm, char c)
+{
+ if (peek (rdm) == c)
+ {
+ rdm->next++;
+ return 1;
+ }
+ else
+ return 0;
+}
+
static char
next (struct rust_demangler *rdm)
{
@@ -92,11 +110,58 @@ next (struct rust_demangler *rdm)
return c;
}
+static uint64_t
+parse_integer_62 (struct rust_demangler *rdm)
+{
+ char c;
+ uint64_t x;
+
+ if (eat (rdm, '_'))
+ return 0;
+
+ x = 0;
+ while (!eat (rdm, '_'))
+ {
+ c = next (rdm);
+ x *= 62;
+ if (ISDIGIT (c))
+ x += c - '0';
+ else if (ISLOWER (c))
+ x += 10 + (c - 'a');
+ else if (ISUPPER (c))
+ x += 10 + 26 + (c - 'A');
+ else
+ {
+ rdm->errored = 1;
+ return 0;
+ }
+ }
+ return x + 1;
+}
+
+static uint64_t
+parse_opt_integer_62 (struct rust_demangler *rdm, char tag)
+{
+ if (!eat (rdm, tag))
+ return 0;
+ return 1 + parse_integer_62 (rdm);
+}
+
+static uint64_t
+parse_disambiguator (struct rust_demangler *rdm)
+{
+ return parse_opt_integer_62 (rdm, 's');
+}
+
struct rust_mangled_ident
{
/* ASCII part of the identifier. */
const char *ascii;
size_t ascii_len;
+
+ /* Punycode insertion codes for Unicode codepoints, if any. */
+ const char *punycode;
+ size_t punycode_len;
};
static struct rust_mangled_ident
@@ -104,10 +169,16 @@ parse_ident (struct rust_demangler *rdm)
{
char c;
size_t start, len;
+ int is_punycode = 0;
struct rust_mangled_ident ident;
ident.ascii = NULL;
ident.ascii_len = 0;
+ ident.punycode = NULL;
+ ident.punycode_len = 0;
+
+ if (rdm->version != -1)
+ is_punycode = eat (rdm, 'u');
c = next (rdm);
if (!ISDIGIT (c))
@@ -121,6 +192,10 @@ parse_ident (struct rust_demangler *rdm)
while (ISDIGIT (peek (rdm)))
len = len * 10 + (next (rdm) - '0');
+ /* Skip past the optional `_` separator (v0). */
+ if (rdm->version != -1)
+ eat (rdm, '_');
+
start = rdm->next;
rdm->next += len;
/* Check for overflows. */
@@ -133,6 +208,27 @@ parse_ident (struct rust_demangler *rdm)
ident.ascii = rdm->sym + start;
ident.ascii_len = len;
+ if (is_punycode)
+ {
+ ident.punycode_len = 0;
+ while (ident.ascii_len > 0)
+ {
+ ident.ascii_len--;
+
+ /* The last '_' is a separator between ascii & punycode. */
+ if (ident.ascii[ident.ascii_len] == '_')
+ break;
+
+ ident.punycode_len++;
+ }
+ if (!ident.punycode_len)
+ {
+ rdm->errored = 1;
+ return ident;
+ }
+ ident.punycode = ident.ascii + (len - ident.punycode_len);
+ }
+
if (ident.ascii_len == 0)
ident.ascii = NULL;
@@ -144,12 +240,28 @@ parse_ident (struct rust_demangler *rdm)
static void
print_str (struct rust_demangler *rdm, const char *data, size_t len)
{
- if (!rdm->errored)
+ if (!rdm->errored && !rdm->skipping_printing)
rdm->callback (data, len, rdm->callback_opaque);
}
#define PRINT(s) print_str (rdm, s, strlen (s))
+static void
+print_uint64 (struct rust_demangler *rdm, uint64_t x)
+{
+ char s[21];
+ sprintf (s, "%" PRIu64, x);
+ PRINT (s);
+}
+
+static void
+print_uint64_hex (struct rust_demangler *rdm, uint64_t x)
+{
+ char s[17];
+ sprintf (s, "%" PRIx64, x);
+ PRINT (s);
+}
+
/* Return a 0x0-0xf value if the char is 0-9a-f, and -1 otherwise. */
static int
decode_lower_hex_nibble (char nibble)
@@ -230,9 +342,14 @@ static void
print_ident (struct rust_demangler *rdm, struct rust_mangled_ident ident)
{
char unescaped;
- size_t len;
-
- if (rdm->errored)
+ uint8_t *out, *p, d;
+ size_t len, cap, punycode_pos, j;
+ /* Punycode parameters and state. */
+ uint32_t c;
+ size_t base, t_min, t_max, skew, damp, bias, i;
+ size_t delta, w, k, t;
+
+ if (rdm->errored || rdm->skipping_printing)
return;
if (rdm->version == -1)
@@ -294,6 +411,758 @@ print_ident (struct rust_demangler *rdm, struct rust_mangled_ident ident)
return;
}
+
+ if (!ident.punycode)
+ {
+ print_str (rdm, ident.ascii, ident.ascii_len);
+ return;
+ }
+
+ len = 0;
+ cap = 4;
+ while (cap < ident.ascii_len)
+ {
+ cap *= 2;
+ /* Check for overflows. */
+ if ((cap * 4) / 4 != cap)
+ {
+ rdm->errored = 1;
+ return;
+ }
+ }
+
+ /* Store the output codepoints as groups of 4 UTF-8 bytes. */
+ out = (uint8_t *)malloc (cap * 4);
+ if (!out)
+ {
+ rdm->errored = 1;
+ return;
+ }
+
+ /* Populate initial output from ASCII fragment. */
+ for (len = 0; len < ident.ascii_len; len++)
+ {
+ p = out + 4 * len;
+ p[0] = 0;
+ p[1] = 0;
+ p[2] = 0;
+ p[3] = ident.ascii[len];
+ }
+
+ /* Punycode parameters and initial state. */
+ base = 36;
+ t_min = 1;
+ t_max = 26;
+ skew = 38;
+ damp = 700;
+ bias = 72;
+ i = 0;
+ c = 0x80;
+
+ punycode_pos = 0;
+ while (punycode_pos < ident.punycode_len)
+ {
+ /* Read one delta value. */
+ delta = 0;
+ w = 1;
+ k = 0;
+ do
+ {
+ k += base;
+ t = k < bias ? 0 : (k - bias);
+ if (t < t_min)
+ t = t_min;
+ if (t > t_max)
+ t = t_max;
+
+ if (punycode_pos >= ident.punycode_len)
+ goto cleanup;
+ d = ident.punycode[punycode_pos++];
+
+ if (ISLOWER (d))
+ d = d - 'a';
+ else if (ISDIGIT (d))
+ d = 26 + (d - '0');
+ else
+ {
+ rdm->errored = 1;
+ goto cleanup;
+ }
+
+ delta += d * w;
+ w *= base - t;
+ }
+ while (d >= t);
+
+ /* Compute the new insert position and character. */
+ len++;
+ i += delta;
+ c += i / len;
+ i %= len;
+
+ /* Ensure enough space is available. */
+ if (cap < len)
+ {
+ cap *= 2;
+ /* Check for overflows. */
+ if ((cap * 4) / 4 != cap || cap < len)
+ {
+ rdm->errored = 1;
+ goto cleanup;
+ }
+ }
+ p = (uint8_t *)realloc (out, cap * 4);
+ if (!p)
+ {
+ rdm->errored = 1;
+ goto cleanup;
+ }
+ out = p;
+
+ /* Move the characters after the insert position. */
+ p = out + i * 4;
+ memmove (p + 4, p, (len - i - 1) * 4);
+
+ /* Insert the new character, as UTF-8 bytes. */
+ p[0] = c >= 0x10000 ? 0xf0 | (c >> 18) : 0;
+ p[1] = c >= 0x800 ? (c < 0x10000 ? 0xe0 : 0x80) | ((c >> 12) & 0x3f) : 0;
+ p[2] = (c < 0x800 ? 0xc0 : 0x80) | ((c >> 6) & 0x3f);
+ p[3] = 0x80 | (c & 0x3f);
+
+ /* If there are no more deltas, decoding is complete. */
+ if (punycode_pos == ident.punycode_len)
+ break;
+
+ i++;
+
+ /* Perform bias adaptation. */
+ delta /= damp;
+ damp = 2;
+
+ delta += delta / len;
+ k = 0;
+ while (delta > ((base - t_min) * t_max) / 2)
+ {
+ delta /= base - t_min;
+ k += base;
+ }
+ bias = k + ((base - t_min + 1) * delta) / (delta + skew);
+ }
+
+ /* Remove all the 0 bytes to leave behind an UTF-8 string. */
+ for (i = 0, j = 0; i < len * 4; i++)
+ if (out[i] != 0)
+ out[j++] = out[i];
+
+ print_str (rdm, (const char *)out, j);
+
+cleanup:
+ free (out);
+}
+
+/* Print the lifetime according to the previously decoded index.
+ An index of `0` always refers to `'_`, but starting with `1`,
+ indices refer to late-bound lifetimes introduced by a binder. */
+static void
+print_lifetime_from_index (struct rust_demangler *rdm, uint64_t lt)
+{
+ char c;
+ uint64_t depth;
+
+ PRINT ("'");
+ if (lt == 0)
+ {
+ PRINT ("_");
+ return;
+ }
+
+ depth = rdm->bound_lifetime_depth - lt;
+ /* Try to print lifetimes alphabetically first. */
+ if (depth < 26)
+ {
+ c = 'a' + depth;
+ print_str (rdm, &c, 1);
+ }
+ else
+ {
+ /* Use `'_123` after running out of letters. */
+ PRINT ("_");
+ print_uint64 (rdm, depth);
+ }
+}
+
+/* Demangling functions. */
+
+static void demangle_binder (struct rust_demangler *rdm);
+static void demangle_path (struct rust_demangler *rdm, int in_value);
+static void demangle_generic_arg (struct rust_demangler *rdm);
+static void demangle_type (struct rust_demangler *rdm);
+static int demangle_path_maybe_open_generics (struct rust_demangler *rdm);
+static void demangle_dyn_trait (struct rust_demangler *rdm);
+static void demangle_const (struct rust_demangler *rdm);
+static void demangle_const_uint (struct rust_demangler *rdm);
+
+/* Optionally enter a binder ('G') for late-bound lifetimes,
+ printing e.g. `for<'a, 'b> `, and make those lifetimes visible
+ to the caller (via depth level, which the caller should reset). */
+static void
+demangle_binder (struct rust_demangler *rdm)
+{
+ uint64_t i, bound_lifetimes;
+
+ if (rdm->errored)
+ return;
+
+ bound_lifetimes = parse_opt_integer_62 (rdm, 'G');
+ if (bound_lifetimes > 0)
+ {
+ PRINT ("for<");
+ for (i = 0; i < bound_lifetimes; i++)
+ {
+ if (i > 0)
+ PRINT (", ");
+ rdm->bound_lifetime_depth++;
+ print_lifetime_from_index (rdm, 1);
+ }
+ PRINT ("> ");
+ }
+}
+
+static void
+demangle_path (struct rust_demangler *rdm, int in_value)
+{
+ char tag, ns;
+ int was_skipping_printing;
+ size_t i, backref, old_next;
+ uint64_t dis;
+ struct rust_mangled_ident name;
+
+ if (rdm->errored)
+ return;
+
+ switch (tag = next (rdm))
+ {
+ case 'C':
+ dis = parse_disambiguator (rdm);
+ name = parse_ident (rdm);
+
+ print_ident (rdm, name);
+ if (rdm->verbose)
+ {
+ PRINT ("[");
+ print_uint64_hex (rdm, dis);
+ PRINT ("]");
+ }
+ break;
+ case 'N':
+ ns = next (rdm);
+ if (!ISLOWER (ns) && !ISUPPER (ns))
+ {
+ rdm->errored = 1;
+ return;
+ }
+
+ demangle_path (rdm, in_value);
+
+ dis = parse_disambiguator (rdm);
+ name = parse_ident (rdm);
+
+ if (ISUPPER (ns))
+ {
+ /* Special namespaces, like closures and shims. */
+ PRINT ("::{");
+ switch (ns)
+ {
+ case 'C':
+ PRINT ("closure");
+ break;
+ case 'S':
+ PRINT ("shim");
+ break;
+ default:
+ print_str (rdm, &ns, 1);
+ }
+ if (name.ascii || name.punycode)
+ {
+ PRINT (":");
+ print_ident (rdm, name);
+ }
+ PRINT ("#");
+ print_uint64 (rdm, dis);
+ PRINT ("}");
+ }
+ else
+ {
+ /* Implementation-specific/unspecified namespaces. */
+
+ if (name.ascii || name.punycode)
+ {
+ PRINT ("::");
+ print_ident (rdm, name);
+ }
+ }
+ break;
+ case 'M':
+ case 'X':
+ /* Ignore the `impl`'s own path.*/
+ parse_disambiguator (rdm);
+ was_skipping_printing = rdm->skipping_printing;
+ rdm->skipping_printing = 1;
+ demangle_path (rdm, in_value);
+ rdm->skipping_printing = was_skipping_printing;
+ /* fallthrough */
+ case 'Y':
+ PRINT ("<");
+ demangle_type (rdm);
+ if (tag != 'M')
+ {
+ PRINT (" as ");
+ demangle_path (rdm, 0);
+ }
+ PRINT (">");
+ break;
+ case 'I':
+ demangle_path (rdm, in_value);
+ if (in_value)
+ PRINT ("::");
+ PRINT ("<");
+ for (i = 0; !rdm->errored && !eat (rdm, 'E'); i++)
+ {
+ if (i > 0)
+ PRINT (", ");
+ demangle_generic_arg (rdm);
+ }
+ PRINT (">");
+ break;
+ case 'B':
+ backref = parse_integer_62 (rdm);
+ if (!rdm->skipping_printing)
+ {
+ old_next = rdm->next;
+ rdm->next = backref;
+ demangle_path (rdm, in_value);
+ rdm->next = old_next;
+ }
+ break;
+ default:
+ rdm->errored = 1;
+ return;
+ }
+}
+
+static void
+demangle_generic_arg (struct rust_demangler *rdm)
+{
+ uint64_t lt;
+ if (eat (rdm, 'L'))
+ {
+ lt = parse_integer_62 (rdm);
+ print_lifetime_from_index (rdm, lt);
+ }
+ else if (eat (rdm, 'K'))
+ demangle_const (rdm);
+ else
+ demangle_type (rdm);
+}
+
+static const char *
+basic_type (char tag)
+{
+ switch (tag)
+ {
+ case 'b':
+ return "bool";
+ case 'c':
+ return "char";
+ case 'e':
+ return "str";
+ case 'u':
+ return "()";
+ case 'a':
+ return "i8";
+ case 's':
+ return "i16";
+ case 'l':
+ return "i32";
+ case 'x':
+ return "i64";
+ case 'n':
+ return "i128";
+ case 'i':
+ return "isize";
+ case 'h':
+ return "u8";
+ case 't':
+ return "u16";
+ case 'm':
+ return "u32";
+ case 'y':
+ return "u64";
+ case 'o':
+ return "u128";
+ case 'j':
+ return "usize";
+ case 'f':
+ return "f32";
+ case 'd':
+ return "f64";
+ case 'z':
+ return "!";
+ case 'p':
+ return "_";
+ case 'v':
+ return "...";
+
+ default:
+ return NULL;
+ }
+}
+
+static void
+demangle_type (struct rust_demangler *rdm)
+{
+ char tag;
+ size_t i, old_next, backref;
+ uint64_t lt, old_bound_lifetime_depth;
+ const char *basic;
+ struct rust_mangled_ident abi;
+
+ if (rdm->errored)
+ return;
+
+ tag = next (rdm);
+
+ basic = basic_type (tag);
+ if (basic)
+ {
+ PRINT (basic);
+ return;
+ }
+
+ switch (tag)
+ {
+ case 'R':
+ case 'Q':
+ PRINT ("&");
+ if (eat (rdm, 'L'))
+ {
+ lt = parse_integer_62 (rdm);
+ if (lt)
+ {
+ print_lifetime_from_index (rdm, lt);
+ PRINT (" ");
+ }
+ }
+ if (tag != 'R')
+ PRINT ("mut ");
+ demangle_type (rdm);
+ break;
+ case 'P':
+ case 'O':
+ PRINT ("*");
+ if (tag != 'P')
+ PRINT ("mut ");
+ else
+ PRINT ("const ");
+ demangle_type (rdm);
+ break;
+ case 'A':
+ case 'S':
+ PRINT ("[");
+ demangle_type (rdm);
+ if (tag == 'A')
+ {
+ PRINT ("; ");
+ demangle_const (rdm);
+ }
+ PRINT ("]");
+ break;
+ case 'T':
+ PRINT ("(");
+ for (i = 0; !rdm->errored && !eat (rdm, 'E'); i++)
+ {
+ if (i > 0)
+ PRINT (", ");
+ demangle_type (rdm);
+ }
+ if (i == 1)
+ PRINT (",");
+ PRINT (")");
+ break;
+ case 'F':
+ old_bound_lifetime_depth = rdm->bound_lifetime_depth;
+ demangle_binder (rdm);
+
+ if (eat (rdm, 'U'))
+ PRINT ("unsafe ");
+
+ if (eat (rdm, 'K'))
+ {
+ if (eat (rdm, 'C'))
+ {
+ abi.ascii = "C";
+ abi.ascii_len = 1;
+ }
+ else
+ {
+ abi = parse_ident (rdm);
+ if (!abi.ascii || abi.punycode)
+ {
+ rdm->errored = 1;
+ goto restore;
+ }
+ }
+
+ PRINT ("extern \"");
+
+ /* If the ABI had any `-`, they were replaced with `_`,
+ so the parts between `_` have to be re-joined with `-`. */
+ for (i = 0; i < abi.ascii_len; i++)
+ {
+ if (abi.ascii[i] == '_')
+ {
+ print_str (rdm, abi.ascii, i);
+ PRINT ("-");
+ abi.ascii += i + 1;
+ abi.ascii_len -= i + 1;
+ i = 0;
+ }
+ }
+ print_str (rdm, abi.ascii, abi.ascii_len);
+
+ PRINT ("\" ");
+ }
+
+ PRINT ("fn(");
+ for (i = 0; !rdm->errored && !eat (rdm, 'E'); i++)
+ {
+ if (i > 0)
+ PRINT (", ");
+ demangle_type (rdm);
+ }
+ PRINT (")");
+
+ if (eat (rdm, 'u'))
+ {
+ /* Skip printing the return type if it's 'u', i.e. `()`. */
+ }
+ else
+ {
+ PRINT (" -> ");
+ demangle_type (rdm);
+ }
+
+ /* Restore `bound_lifetime_depth` to outside the binder. */
+ restore:
+ rdm->bound_lifetime_depth = old_bound_lifetime_depth;
+ break;
+ case 'D':
+ PRINT ("dyn ");
+
+ old_bound_lifetime_depth = rdm->bound_lifetime_depth;
+ demangle_binder (rdm);
+
+ for (i = 0; !rdm->errored && !eat (rdm, 'E'); i++)
+ {
+ if (i > 0)
+ PRINT (" + ");
+ demangle_dyn_trait (rdm);
+ }
+
+ /* Restore `bound_lifetime_depth` to outside the binder. */
+ rdm->bound_lifetime_depth = old_bound_lifetime_depth;
+
+ if (!eat (rdm, 'L'))
+ {
+ rdm->errored = 1;
+ return;
+ }
+ lt = parse_integer_62 (rdm);
+ if (lt)
+ {
+ PRINT (" + ");
+ print_lifetime_from_index (rdm, lt);
+ }
+ break;
+ case 'B':
+ backref = parse_integer_62 (rdm);
+ if (!rdm->skipping_printing)
+ {
+ old_next = rdm->next;
+ rdm->next = backref;
+ demangle_type (rdm);
+ rdm->next = old_next;
+ }
+ break;
+ default:
+ /* Go back to the tag, so `demangle_path` also sees it. */
+ rdm->next--;
+ demangle_path (rdm, 0);
+ }
+}
+
+/* A trait in a trait object may have some "existential projections"
+ (i.e. associated type bindings) after it, which should be printed
+ in the `<...>` of the trait, e.g. `dyn Trait<T, U, Assoc=X>`.
+ To this end, this method will keep the `<...>` of an 'I' path
+ open, by omitting the `>`, and return `Ok(true)` in that case. */
+static int
+demangle_path_maybe_open_generics (struct rust_demangler *rdm)
+{
+ int open;
+ size_t i, old_next, backref;
+
+ open = 0;
+
+ if (rdm->errored)
+ return open;
+
+ if (eat (rdm, 'B'))
+ {
+ backref = parse_integer_62 (rdm);
+ if (!rdm->skipping_printing)
+ {
+ old_next = rdm->next;
+ rdm->next = backref;
+ open = demangle_path_maybe_open_generics (rdm);
+ rdm->next = old_next;
+ }
+ }
+ else if (eat (rdm, 'I'))
+ {
+ demangle_path (rdm, 0);
+ PRINT ("<");
+ open = 1;
+ for (i = 0; !rdm->errored && !eat (rdm, 'E'); i++)
+ {
+ if (i > 0)
+ PRINT (", ");
+ demangle_generic_arg (rdm);
+ }
+ }
+ else
+ demangle_path (rdm, 0);
+ return open;
+}
+
+static void
+demangle_dyn_trait (struct rust_demangler *rdm)
+{
+ int open;
+ struct rust_mangled_ident name;
+
+ if (rdm->errored)
+ return;
+
+ open = demangle_path_maybe_open_generics (rdm);
+
+ while (eat (rdm, 'p'))
+ {
+ if (!open)
+ PRINT ("<");
+ else
+ PRINT (", ");
+ open = 1;
+
+ name = parse_ident (rdm);
+ print_ident (rdm, name);
+ PRINT (" = ");
+ demangle_type (rdm);
+ }
+
+ if (open)
+ PRINT (">");
+}
+
+static void
+demangle_const (struct rust_demangler *rdm)
+{
+ char ty_tag;
+ size_t old_next, backref;
+
+ if (rdm->errored)
+ return;
+
+ if (eat (rdm, 'B'))
+ {
+ backref = parse_integer_62 (rdm);
+ if (!rdm->skipping_printing)
+ {
+ old_next = rdm->next;
+ rdm->next = backref;
+ demangle_const (rdm);
+ rdm->next = old_next;
+ }
+ return;
+ }
+
+ ty_tag = next (rdm);
+ switch (ty_tag)
+ {
+ /* Unsigned integer types. */
+ case 'h':
+ case 't':
+ case 'm':
+ case 'y':
+ case 'o':
+ case 'j':
+ break;
+
+ default:
+ rdm->errored = 1;
+ return;
+ }
+
+ if (eat (rdm, 'p'))
+ PRINT ("_");
+ else
+ demangle_const_uint (rdm);
+
+ if (rdm->verbose)
+ {
+ PRINT (": ");
+ PRINT (basic_type (ty_tag));
+ }
+}
+
+static void
+demangle_const_uint (struct rust_demangler *rdm)
+{
+ char c;
+ size_t hex_len;
+ uint64_t value;
+
+ if (rdm->errored)
+ return;
+
+ value = 0;
+ hex_len = 0;
+ while (!eat (rdm, '_'))
+ {
+ value <<= 4;
+
+ c = next (rdm);
+ if (ISDIGIT (c))
+ value |= c - '0';
+ else if (c >= 'a' && c <= 'f')
+ value |= 10 + (c - 'a');
+ else
+ {
+ rdm->errored = 1;
+ return;
+ }
+ hex_len++;
+ }
+
+ /* Print anything that doesn't fit in `uint64_t` verbatim. */
+ if (hex_len > 16)
+ {
+ PRINT ("0x");
+ print_str (rdm, rdm->sym + (rdm->next - hex_len), hex_len);
+ return;
+ }
+
+ print_uint64 (rdm, value);
}
/* A legacy hash is the prefix "h" followed by 16 lowercase hex digits.
@@ -345,11 +1214,15 @@ rust_demangle_callback (const char *mangled, int options,
rdm.next = 0;
rdm.errored = 0;
+ rdm.skipping_printing = 0;
rdm.verbose = (options & DMGL_VERBOSE) != 0;
rdm.version = 0;
+ rdm.bound_lifetime_depth = 0;
- /* Rust symbols always start with _ZN (legacy). */
- if (rdm.sym[0] == '_' && rdm.sym[1] == 'Z' && rdm.sym[2] == 'N')
+ /* Rust symbols always start with _R (v0) or _ZN (legacy). */
+ if (rdm.sym[0] == '_' && rdm.sym[1] == 'R')
+ rdm.sym += 2;
+ else if (rdm.sym[0] == '_' && rdm.sym[1] == 'Z' && rdm.sym[2] == 'N')
{
rdm.sym += 3;
rdm.version = -1;
@@ -357,7 +1230,11 @@ rust_demangle_callback (const char *mangled, int options,
else
return 0;
- /* Legacy Rust symbols use only [_0-9a-zA-Z.:$] characters. */
+ /* Paths (v0) always start with uppercase characters. */
+ if (rdm.version != -1 && !ISUPPER (rdm.sym[0]))
+ return 0;
+
+ /* Rust symbols (v0) use only [_0-9a-zA-Z] characters. */
for (p = rdm.sym; *p; p++)
{
rdm.sym_len++;
@@ -365,6 +1242,7 @@ rust_demangle_callback (const char *mangled, int options,
if (*p == '_' || ISALNUM (*p))
continue;
+ /* Legacy Rust symbols can also contain [.:$] characters. */
if (rdm.version == -1 && (*p == '$' || *p == '.' || *p == ':'))
continue;
@@ -418,7 +1296,19 @@ rust_demangle_callback (const char *mangled, int options,
while (rdm.next < rdm.sym_len);
}
else
- return 0;
+ {
+ demangle_path (&rdm, 1);
+
+ /* Skip instantiating crate. */
+ if (!rdm.errored && rdm.next < rdm.sym_len)
+ {
+ rdm.skipping_printing = 1;
+ demangle_path (&rdm, 0);
+ }
+
+ /* It's an error to not reach the end. */
+ rdm.errored |= rdm.next != rdm.sym_len;
+ }
return !rdm.errored;
}
diff --git a/libiberty/testsuite/rust-demangle-expected b/libiberty/testsuite/rust-demangle-expected
index 74774794736a..6a6f4b141c09 100644
--- a/libiberty/testsuite/rust-demangle-expected
+++ b/libiberty/testsuite/rust-demangle-expected
@@ -11,7 +11,7 @@
#
############
#
-# Coverage Tests
+# Coverage Tests (legacy)
#
#
# Demangles as rust symbol.
@@ -163,3 +163,60 @@ _ZN63_$LT$core..ptr..Unique$LT$T$GT$$u20$as$u20$core..ops..Deref$GT$5deref17h19f
--format=rust
_ZN11issue_609253foo37Foo$LT$issue_60925..llv$u6d$..Foo$GT$3foo17h059a991a004536adE
issue_60925::foo::Foo<issue_60925::llvm::Foo>::foo
+#
+############
+#
+# Coverage Tests (v0)
+#
+#
+# Crate with a leading digit.
+--format=rust
+_RNvC6_123foo3bar
+123foo::bar
+# UTF-8 identifiers.
+--format=rust
+_RNqCs4fqI2P2rA04_11utf8_identsu30____7hkackfecea1cbdathfdh9hlq6y
+utf8_idents::საჭმელად_გემრიელი_სადილი
+# Closure path elements.
+--format=rust
+_RNCNCNgCs6DXkGYLi8lr_2cc5spawn00B5_
+cc::spawn::{closure#0}::{closure#0}
+#
+--format=rust
+_RNCINkXs25_NgCsbmNqQUJIY6D_4core5sliceINyB9_4IterhENuNgNoBb_4iter8iterator8Iterator9rpositionNCNgNpB9_6memchr7memrchrs_0E0Bb_
+<core::slice::Iter<u8> as core::iter::iterator::Iterator>::rposition::<core::slice::memchr::memrchr::{closure#1}>::{closure#0}
+# dyn Trait ("trait object") types.
+--format=rust
+_RINbNbCskIICzLVDPPb_5alloc5alloc8box_freeDINbNiB4_5boxed5FnBoxuEp6OutputuEL_ECs1iopQbuBiw2_3std
+alloc::alloc::box_free::<dyn alloc::boxed::FnBox<(), Output = ()>>
+# Type with a const generic parameter.
+--format=rust
+_RNvMC0INtC8arrayvec8ArrayVechKj7b_E3new
+<arrayvec::ArrayVec<u8, 123>>::new
+#
+# All of the tests above but in auto mode instead:
+#
+# Crate with a leading digit.
+--format=auto
+_RNvC6_123foo3bar
+123foo::bar
+# UTF-8 identifiers.
+--format=auto
+_RNqCs4fqI2P2rA04_11utf8_identsu30____7hkackfecea1cbdathfdh9hlq6y
+utf8_idents::საჭმელად_გემრიელი_სადილი
+# Closure path elements.
+--format=auto
+_RNCNCNgCs6DXkGYLi8lr_2cc5spawn00B5_
+cc::spawn::{closure#0}::{closure#0}
+#
+--format=auto
+_RNCINkXs25_NgCsbmNqQUJIY6D_4core5sliceINyB9_4IterhENuNgNoBb_4iter8iterator8Iterator9rpositionNCNgNpB9_6memchr7memrchrs_0E0Bb_
+<core::slice::Iter<u8> as core::iter::iterator::Iterator>::rposition::<core::slice::memchr::memrchr::{closure#1}>::{closure#0}
+# dyn Trait ("trait object") types.
+--format=auto
+_RINbNbCskIICzLVDPPb_5alloc5alloc8box_freeDINbNiB4_5boxed5FnBoxuEp6OutputuEL_ECs1iopQbuBiw2_3std
+alloc::alloc::box_free::<dyn alloc::boxed::FnBox<(), Output = ()>>
+# Type with a const generic parameter.
+--format=auto
+_RNvMC0INtC8arrayvec8ArrayVechKj7b_E3new
+<arrayvec::ArrayVec<u8, 123>>::new
--
2.24.1