[V2,3/3] dwarf2out: Emit BTF in dwarf2out_finish for BPF CO-RE usecase

DWARF generation is split between early and late phases when LTO is in effect.
This poses challenges for CTF/BTF generation especially if late debug info
generation is desirable, as turns out to be the case for BPF CO-RE.

In case of BPF CO-RE, the BPF backend adds information about CO-RE relocations
to the CTF container. This information is what needs to be emitted as a
separate .BTF.ext section when -more is in effect. Further, each CO-RE
relocation record holds an offset to a string specifying the access to the
structure's field. This means that .BTF string table needs to be modified
"late" in the compilation process. In other words, this implies that the BTF
sections cannot be finalized in dwarf2out_early_finish when -mcore for the BPF
backend is in effect.

Now, the emission of CTF/BTF debug info cannot be moved unconditionally to
dwarf2out_finish because dwarf2out_finish is not invoked at all for the LTO
compile phase for slim LTO objects, thus breaking CTF/BTF generation for other
targets when used with LTO.

The approach taken here in this patch is that -

1. LTO is disabled for BPF CO-RE
The reason to disable LTO for BPF CO-RE is that if LTO is in effect, BPF CO-RE
relocations need to be generated in the LTO link phase _after_ the optimizations
are done. This means we need to devise way to combine early and late BTF. At
this time, in absence of linker support for BTF sections, it makes sense to
steer clear of LTO for BPF CO-RE and bypass the issue.

2. Use a target hook to allow BPF backend to cleanly convey the case when late
finalization of the CTF container is desirable.

So, in other words,

dwarf2out_early_finish
  - Always emit CTF here.
  - if (BTF && ctfc_debuginfo_early_finish_p), emit BTF now.

dwarf2out_finish
  - if (BTF && !ctfc_debuginfo_early_finish_p && !in_lto_p) emit BTF now.
  - Use of in_lto_p to make sure LTO link phase does not affect BTF sections
for other targets.

gcc/ChangeLog:

	* dwarf2ctf.c (ctf_debug_finalize): Make it static.
	(ctf_debug_early_finish): New definition.
	(ctf_debug_finish): Likewise.
	* dwarf2ctf.h (ctf_debug_finalize): Remove declaration.
	(ctf_debug_early_finish): New declaration.
	(ctf_debug_finish): Likewise.
	* dwarf2out.c (dwarf2out_finish): Invoke ctf_debug_finish.
	(dwarf2out_early_finish): Invoke ctf_debug_early_finish.
---
 gcc/dwarf2ctf.c | 55 +++++++++++++++++++++++++++++++++++++++++++------------
 gcc/dwarf2ctf.h |  4 +++-
 gcc/dwarf2out.c |  9 +++++++--
 3 files changed, 53 insertions(+), 15 deletions(-)

On Thu, Aug 5, 2021 at 2:53 AM Indu Bhagat via Gcc-patches
<gcc-patches@gcc.gnu.org> wrote:
>
> DWARF generation is split between early and late phases when LTO is in effect.
> This poses challenges for CTF/BTF generation especially if late debug info
> generation is desirable, as turns out to be the case for BPF CO-RE.
>
> In case of BPF CO-RE, the BPF backend adds information about CO-RE relocations
> to the CTF container. This information is what needs to be emitted as a
> separate .BTF.ext section when -more is in effect. Further, each CO-RE
> relocation record holds an offset to a string specifying the access to the
> structure's field. This means that .BTF string table needs to be modified
> "late" in the compilation process. In other words, this implies that the BTF
> sections cannot be finalized in dwarf2out_early_finish when -mcore for the BPF
> backend is in effect.

OK, it didn't really get any clearer as to why the late annotation
cannot be done
after the early info was output.  Something to do with the BTF string table,
but all structure field names must be already present there so I must be missing
something.

ISTR the CO-RE info is fully contained in a new section .BTF.ext and the
"early" .BTF section is not altered?

> Now, the emission of CTF/BTF debug info cannot be moved unconditionally to
> dwarf2out_finish because dwarf2out_finish is not invoked at all for the LTO
> compile phase for slim LTO objects, thus breaking CTF/BTF generation for other
> targets when used with LTO.
>
> The approach taken here in this patch is that -
>
> 1. LTO is disabled for BPF CO-RE
> The reason to disable LTO for BPF CO-RE is that if LTO is in effect, BPF CO-RE
> relocations need to be generated in the LTO link phase _after_ the optimizations
> are done. This means we need to devise way to combine early and late BTF. At
> this time, in absence of linker support for BTF sections, it makes sense to
> steer clear of LTO for BPF CO-RE and bypass the issue.
>
> 2. Use a target hook to allow BPF backend to cleanly convey the case when late
> finalization of the CTF container is desirable.
>
> So, in other words,
>
> dwarf2out_early_finish
>   - Always emit CTF here.
>   - if (BTF && ctfc_debuginfo_early_finish_p), emit BTF now.
>
> dwarf2out_finish
>   - if (BTF && !ctfc_debuginfo_early_finish_p && !in_lto_p) emit BTF now.
>   - Use of in_lto_p to make sure LTO link phase does not affect BTF sections
> for other targets.
>
> gcc/ChangeLog:
>
>         * dwarf2ctf.c (ctf_debug_finalize): Make it static.
>         (ctf_debug_early_finish): New definition.
>         (ctf_debug_finish): Likewise.
>         * dwarf2ctf.h (ctf_debug_finalize): Remove declaration.
>         (ctf_debug_early_finish): New declaration.
>         (ctf_debug_finish): Likewise.
>         * dwarf2out.c (dwarf2out_finish): Invoke ctf_debug_finish.
>         (dwarf2out_early_finish): Invoke ctf_debug_early_finish.
> ---
>  gcc/dwarf2ctf.c | 55 +++++++++++++++++++++++++++++++++++++++++++------------
>  gcc/dwarf2ctf.h |  4 +++-
>  gcc/dwarf2out.c |  9 +++++++--
>  3 files changed, 53 insertions(+), 15 deletions(-)
>
> diff --git a/gcc/dwarf2ctf.c b/gcc/dwarf2ctf.c
> index 5e8a725..0fa429c 100644
> --- a/gcc/dwarf2ctf.c
> +++ b/gcc/dwarf2ctf.c
> @@ -917,6 +917,27 @@ gen_ctf_type (ctf_container_ref ctfc, dw_die_ref die)
>    return type_id;
>  }
>
> +/* Prepare for output and write out the CTF debug information.  */
> +
> +static void
> +ctf_debug_finalize (const char *filename, bool btf)
> +{
> +  if (btf)
> +    {
> +      btf_output (filename);
> +      btf_finalize ();
> +    }
> +
> +  else
> +    {
> +      /* Emit the collected CTF information.  */
> +      ctf_output (filename);
> +
> +      /* Reset the CTF state.  */
> +      ctf_finalize ();
> +    }
> +}
> +
>  bool
>  ctf_do_die (dw_die_ref die)
>  {
> @@ -966,25 +987,35 @@ ctf_debug_init_postprocess (bool btf)
>      btf_init_postprocess ();
>  }
>
> -/* Prepare for output and write out the CTF debug information.  */
> +/* Early finish CTF/BTF debug info.  */
>
>  void
> -ctf_debug_finalize (const char *filename, bool btf)
> +ctf_debug_early_finish (const char * filename)
>  {
> -  if (btf)
> +  /* Emit CTF debug info early always.  */
> +  if (ctf_debug_info_level > CTFINFO_LEVEL_NONE
> +      /* Emit BTF debug info early if the target does not require late
> +        emission.  */
> +       || (btf_debuginfo_p ()
> +          && targetm.ctfc_debuginfo_early_finish_p ()))
>      {
> -      btf_output (filename);
> -      btf_finalize ();
> +      /* Emit CTF/BTF debug info.  */
> +      ctf_debug_finalize (filename, btf_debuginfo_p ());
>      }
> +}
>
> -  else
> -    {
> -      /* Emit the collected CTF information.  */
> -      ctf_output (filename);
> +/* Finish CTF/BTF debug info emission.  */
>
> -      /* Reset the CTF state.  */
> -      ctf_finalize ();
> -    }
> +void
> +ctf_debug_finish (const char * filename)
> +{
> +  /* Emit BTF debug info here when the target needs to update the CTF container
> +     (ctfc) in the backend.  An example of this, at this time is the BPF CO-RE
> +     usecase.  */
> +  if (btf_debuginfo_p ()
> +      && (!in_lto_p && !targetm.ctfc_debuginfo_early_finish_p ()))
> +    /* Emit BTF debug info.  */
> +    ctf_debug_finalize (filename, btf_debuginfo_p ());
>  }
>
>  #include "gt-dwarf2ctf.h"
> diff --git a/gcc/dwarf2ctf.h b/gcc/dwarf2ctf.h
> index a3cf567..9edbde0 100644
> --- a/gcc/dwarf2ctf.h
> +++ b/gcc/dwarf2ctf.h
> @@ -24,13 +24,15 @@ along with GCC; see the file COPYING3.  If not see
>  #define GCC_DWARF2CTF_H 1
>
>  #include "dwarf2out.h"
> +#include "flags.h"
>
>  /* Debug Format Interface.  Used in dwarf2out.c.  */
>
>  extern void ctf_debug_init (void);
>  extern void ctf_debug_init_postprocess (bool);
>  extern bool ctf_do_die (dw_die_ref);
> -extern void ctf_debug_finalize (const char *, bool);
> +extern void ctf_debug_early_finish (const char *);
> +extern void ctf_debug_finish (const char *);
>
>  /* Wrappers for CTF/BTF to fetch information from GCC DWARF DIE.  Used in
>     ctfc.c.
> diff --git a/gcc/dwarf2out.c b/gcc/dwarf2out.c
> index b91a9b5..708cd1f 100644
> --- a/gcc/dwarf2out.c
> +++ b/gcc/dwarf2out.c
> @@ -31895,6 +31895,11 @@ dwarf2out_finish (const char *filename)
>    unsigned char checksum[16];
>    char dl_section_ref[MAX_ARTIFICIAL_LABEL_BYTES];
>
> +  /* Generate CTF/BTF debug info.  */
> +  if ((ctf_debug_info_level > CTFINFO_LEVEL_NONE
> +       || btf_debuginfo_p ()) && lang_GNU_C ())
> +    ctf_debug_finish (filename);
> +
>    /* Skip emitting DWARF if not required.  */
>    if (!dwarf_debuginfo_p ())
>      return;
> @@ -32799,8 +32804,8 @@ dwarf2out_early_finish (const char *filename)
>         ctf_debug_do_cu (node->die);
>        /* Post process the debug data in the CTF container if necessary.  */
>        ctf_debug_init_postprocess (btf_debuginfo_p ());
> -      /* Emit CTF/BTF debug info.  */
> -      ctf_debug_finalize (filename, btf_debuginfo_p ());
> +
> +      ctf_debug_early_finish (filename);
>      }
>
>    /* Do not generate DWARF assembler now when not producing LTO bytecode.  */
> --
> 1.8.3.1
>

On 8/10/21 5:00 AM, Richard Biener via Gcc-patches wrote:
> On Thu, Aug 5, 2021 at 2:53 AM Indu Bhagat via Gcc-patches
> <gcc-patches@gcc.gnu.org> wrote:
>>
>> DWARF generation is split between early and late phases when LTO is in effect.
>> This poses challenges for CTF/BTF generation especially if late debug info
>> generation is desirable, as turns out to be the case for BPF CO-RE.
>>
>> In case of BPF CO-RE, the BPF backend adds information about CO-RE relocations
>> to the CTF container. This information is what needs to be emitted as a
>> separate .BTF.ext section when -more is in effect. Further, each CO-RE
>> relocation record holds an offset to a string specifying the access to the
>> structure's field. This means that .BTF string table needs to be modified
>> "late" in the compilation process. In other words, this implies that the BTF
>> sections cannot be finalized in dwarf2out_early_finish when -mcore for the BPF
>> backend is in effect.
> 
> OK, it didn't really get any clearer as to why the late annotation
> cannot be done
> after the early info was output.  Something to do with the BTF string table,
> but all structure field names must be already present there so I must be missing
> something.
> 
> ISTR the CO-RE info is fully contained in a new section .BTF.ext and the
> "early" .BTF section is not altered?

This is unfortunately not quite the case. The CO-RE relocation records 
themselves are placed in the .BTF.ext section. But a key component of 
each record is an "access string," a string which encodes the accessed 
field via a sequence of field or array indices.

The .BTF.ext section does not have a string table of its own, so these 
"access strings" are placed in the .BTF section string table. The CO-RE 
relocation records refer to them by offset into the .BTF string table.

As a result, the CO-RE information spills into the regular .BTF section,
and we need to delay writing out the .BTF section until after these 
strings can be added.

Personally I don't think this way of splitting the CO-RE information is 
ideal, but hopefully now the issue is more clear?

Thank you for your reviews!


Example of access string encoding:

struct S {
   int a;
   union {
     int _unused;
     int b;
     char c;
   } u[4];
};

struct S *foo = ...;
int x  = foo->a;          /* encoded as "0:0"     */
int y  = foo[4]->u[2].b   /* encoded as "4:1:2:1" */
char z = foo->u[3].c      /* encoded as "0:1:3:2" */


> 
>> Now, the emission of CTF/BTF debug info cannot be moved unconditionally to
>> dwarf2out_finish because dwarf2out_finish is not invoked at all for the LTO
>> compile phase for slim LTO objects, thus breaking CTF/BTF generation for other
>> targets when used with LTO.
>>
>> The approach taken here in this patch is that -
>>
>> 1. LTO is disabled for BPF CO-RE
>> The reason to disable LTO for BPF CO-RE is that if LTO is in effect, BPF CO-RE
>> relocations need to be generated in the LTO link phase _after_ the optimizations
>> are done. This means we need to devise way to combine early and late BTF. At
>> this time, in absence of linker support for BTF sections, it makes sense to
>> steer clear of LTO for BPF CO-RE and bypass the issue.
>>
>> 2. Use a target hook to allow BPF backend to cleanly convey the case when late
>> finalization of the CTF container is desirable.
>>
>> So, in other words,
>>
>> dwarf2out_early_finish
>>    - Always emit CTF here.
>>    - if (BTF && ctfc_debuginfo_early_finish_p), emit BTF now.
>>
>> dwarf2out_finish
>>    - if (BTF && !ctfc_debuginfo_early_finish_p && !in_lto_p) emit BTF now.
>>    - Use of in_lto_p to make sure LTO link phase does not affect BTF sections
>> for other targets.
>>
>> gcc/ChangeLog:
>>
>>          * dwarf2ctf.c (ctf_debug_finalize): Make it static.
>>          (ctf_debug_early_finish): New definition.
>>          (ctf_debug_finish): Likewise.
>>          * dwarf2ctf.h (ctf_debug_finalize): Remove declaration.
>>          (ctf_debug_early_finish): New declaration.
>>          (ctf_debug_finish): Likewise.
>>          * dwarf2out.c (dwarf2out_finish): Invoke ctf_debug_finish.
>>          (dwarf2out_early_finish): Invoke ctf_debug_early_finish.
>> ---
>>   gcc/dwarf2ctf.c | 55 +++++++++++++++++++++++++++++++++++++++++++------------
>>   gcc/dwarf2ctf.h |  4 +++-
>>   gcc/dwarf2out.c |  9 +++++++--
>>   3 files changed, 53 insertions(+), 15 deletions(-)
>>
>> diff --git a/gcc/dwarf2ctf.c b/gcc/dwarf2ctf.c
>> index 5e8a725..0fa429c 100644
>> --- a/gcc/dwarf2ctf.c
>> +++ b/gcc/dwarf2ctf.c
>> @@ -917,6 +917,27 @@ gen_ctf_type (ctf_container_ref ctfc, dw_die_ref die)
>>     return type_id;
>>   }
>>
>> +/* Prepare for output and write out the CTF debug information.  */
>> +
>> +static void
>> +ctf_debug_finalize (const char *filename, bool btf)
>> +{
>> +  if (btf)
>> +    {
>> +      btf_output (filename);
>> +      btf_finalize ();
>> +    }
>> +
>> +  else
>> +    {
>> +      /* Emit the collected CTF information.  */
>> +      ctf_output (filename);
>> +
>> +      /* Reset the CTF state.  */
>> +      ctf_finalize ();
>> +    }
>> +}
>> +
>>   bool
>>   ctf_do_die (dw_die_ref die)
>>   {
>> @@ -966,25 +987,35 @@ ctf_debug_init_postprocess (bool btf)
>>       btf_init_postprocess ();
>>   }
>>
>> -/* Prepare for output and write out the CTF debug information.  */
>> +/* Early finish CTF/BTF debug info.  */
>>
>>   void
>> -ctf_debug_finalize (const char *filename, bool btf)
>> +ctf_debug_early_finish (const char * filename)
>>   {
>> -  if (btf)
>> +  /* Emit CTF debug info early always.  */
>> +  if (ctf_debug_info_level > CTFINFO_LEVEL_NONE
>> +      /* Emit BTF debug info early if the target does not require late
>> +        emission.  */
>> +       || (btf_debuginfo_p ()
>> +          && targetm.ctfc_debuginfo_early_finish_p ()))
>>       {
>> -      btf_output (filename);
>> -      btf_finalize ();
>> +      /* Emit CTF/BTF debug info.  */
>> +      ctf_debug_finalize (filename, btf_debuginfo_p ());
>>       }
>> +}
>>
>> -  else
>> -    {
>> -      /* Emit the collected CTF information.  */
>> -      ctf_output (filename);
>> +/* Finish CTF/BTF debug info emission.  */
>>
>> -      /* Reset the CTF state.  */
>> -      ctf_finalize ();
>> -    }
>> +void
>> +ctf_debug_finish (const char * filename)
>> +{
>> +  /* Emit BTF debug info here when the target needs to update the CTF container
>> +     (ctfc) in the backend.  An example of this, at this time is the BPF CO-RE
>> +     usecase.  */
>> +  if (btf_debuginfo_p ()
>> +      && (!in_lto_p && !targetm.ctfc_debuginfo_early_finish_p ()))
>> +    /* Emit BTF debug info.  */
>> +    ctf_debug_finalize (filename, btf_debuginfo_p ());
>>   }
>>
>>   #include "gt-dwarf2ctf.h"
>> diff --git a/gcc/dwarf2ctf.h b/gcc/dwarf2ctf.h
>> index a3cf567..9edbde0 100644
>> --- a/gcc/dwarf2ctf.h
>> +++ b/gcc/dwarf2ctf.h
>> @@ -24,13 +24,15 @@ along with GCC; see the file COPYING3.  If not see
>>   #define GCC_DWARF2CTF_H 1
>>
>>   #include "dwarf2out.h"
>> +#include "flags.h"
>>
>>   /* Debug Format Interface.  Used in dwarf2out.c.  */
>>
>>   extern void ctf_debug_init (void);
>>   extern void ctf_debug_init_postprocess (bool);
>>   extern bool ctf_do_die (dw_die_ref);
>> -extern void ctf_debug_finalize (const char *, bool);
>> +extern void ctf_debug_early_finish (const char *);
>> +extern void ctf_debug_finish (const char *);
>>
>>   /* Wrappers for CTF/BTF to fetch information from GCC DWARF DIE.  Used in
>>      ctfc.c.
>> diff --git a/gcc/dwarf2out.c b/gcc/dwarf2out.c
>> index b91a9b5..708cd1f 100644
>> --- a/gcc/dwarf2out.c
>> +++ b/gcc/dwarf2out.c
>> @@ -31895,6 +31895,11 @@ dwarf2out_finish (const char *filename)
>>     unsigned char checksum[16];
>>     char dl_section_ref[MAX_ARTIFICIAL_LABEL_BYTES];
>>
>> +  /* Generate CTF/BTF debug info.  */
>> +  if ((ctf_debug_info_level > CTFINFO_LEVEL_NONE
>> +       || btf_debuginfo_p ()) && lang_GNU_C ())
>> +    ctf_debug_finish (filename);
>> +
>>     /* Skip emitting DWARF if not required.  */
>>     if (!dwarf_debuginfo_p ())
>>       return;
>> @@ -32799,8 +32804,8 @@ dwarf2out_early_finish (const char *filename)
>>          ctf_debug_do_cu (node->die);
>>         /* Post process the debug data in the CTF container if necessary.  */
>>         ctf_debug_init_postprocess (btf_debuginfo_p ());
>> -      /* Emit CTF/BTF debug info.  */
>> -      ctf_debug_finalize (filename, btf_debuginfo_p ());
>> +
>> +      ctf_debug_early_finish (filename);
>>       }
>>
>>     /* Do not generate DWARF assembler now when not producing LTO bytecode.  */
>> --
>> 1.8.3.1
>>