Message ID | 20240502171132.95601-5-david.faust@oracle.com |
---|---|
State | New |
Headers | show |
Series | btf: refactor and add pruning option | expand |
On 5/2/24 10:11, David Faust wrote: > This patch heavily refactors btfout.cc to take advantage of the > structural changes in the prior commits. > > Now that inter-type references are internally stored as simply pointers, > all the painful, brittle, confusing infrastructure that was used in the > process of converting CTF type IDs to BTF type IDs can be thrown out. > This greatly simplifies the entire process of converting from CTF to > BTF, making the code cleaner, easier to read, and easier to maintain. > > In addition, we no longer need to worry about destructive changes in > internal data structures used commonly by CTF and BTF, which allows > deleting several ancillary data structures previously used in btfout.cc. > > This is nearly transparent, but a few improvements have also been made: > > 1) BTF_KIND_FUNC records are now _always_ constructed at early_finish, > allowing us to construct records even for functions which are later > inlined by optimizations. DATASEC entries for functions are only > constructed at late_finish, to avoid incorrectly generating entries > for functions which get inlined. > > 2) BTF_KIND_VAR records and DATASEC entries for them are now always > constructed at (late) finish, which avoids cases where we could > incorrectly create records for variables which were completely > optimized away. This fixes PR debug/113566. > I am a bit unsure ATM, how this commit will look like once we revisit where the BTF is generated for BPF and non-BPF backends. Since patch 1/6 moved everything to (late)finish, but that will be problematic, I would like to defer reviewing this until the approach for afore-mentioned patch is pinned. > 3) Some additional assembler comments have been added with more > information for debugging. > > gcc/ > * btfout.cc (struct btf_datasec_entry): New. > (struct btf_datasec): Add `id' member. Change `entries' to use > new struct btf_datasec_entry. > (func_map): New hash_map. > (max_translated_id): New. > (btf_var_ids, btf_id_map, holes, voids, num_vars_added) > (num_types_added, num_types_created): Delete. > (btf_absolute_var_id, btf_relative_var_id, btf_absolute_func_id) > (btf_relative_func_id, btf_absolute_datasec_id, init_btf_id_map) > (get_btf_id, set_btf_id, btf_emit_id_p): Delete. > (btf_removed_type_p): Delete. > (btf_dtd_kind, btf_emit_type_p): New helpers. > (btf_fwd_to_enum_p, btf_calc_num_vbytes): Use them. > (btf_collect_datasec): Delete. > (btf_dtd_postprocess_cb, btf_dvd_emit_preprocess_cb) > (btf_dtd_emit_preprocess_cb, btf_emit_preprocess): Delete. > (btf_dmd_representable_bitfield_p): Adapt to type reference changes > and delete now-unused ctfc argument. > (btf_asm_datasec_type_ref): Delete. > (btf_asm_type_ref): Adapt to type reference changes, simplify. > (btf_asm_type): Likewise. Mark struct/union types with bitfield > members. > (btf_asm_array): Adapt to data structure changes. > (btf_asm_varent): Likewise. > (btf_asm_sou_member): Likewise. Ensure non-bitfield members are > correctly re-encoded if struct or union contains any bitfield. > (btf_asm_func_arg, btf_asm_func_type, btf_asm_datasec_entry) > (btf_asm_datasec_type): Adapt to data structure changes. > (output_btf_header): Adapt to other changes, simplify type > length calculation, add info to assembler comments. > (output_btf_vars): Adapt to other changes. > (output_btf_strs): Fix overlong lines. > (output_asm_btf_sou_fields, output_asm_btf_enum_list) > (output_asm_btf_func_args_list, output_asm_btf_vlen_bytes) > (output_asm_btf_type, output_btf_types, output_btf_func_types) > (output_btf_datasec_types): Adapt to other changes. > (btf_init_postprocess): Delete. > (btf_output): Change to only perform output. > (btf_early_add_const_void, btf_early_add_func_records): New. > (btf_early_finish): Use them here. New. > (btf_datasec_push_entry): Adapt to data structure changes. > (btf_datasec_add_func, btf_datasec_add_var): New. > (btf_late_add_func_datasec_entries): New. > (btf_emit_variable_p): New helper. > (btf_late_add_vars): Use it here. New. > (btf_type_list_cb, btf_late_collect_translated_types): New. > (btf_late_assign_func_ids, btf_late_assign_var_ids) > (btf_late_assign_datasec_ids): New. > (btf_finish): Remove unused argument. Call new btf_late* > functions and btf_output. > (btf_finalize): Adapt to data structure changes. > * ctfc.h (struct ctf_dtdef): Convert existing boolean flags to > BOOL_BITFIELD and reorder. > (struct ctf_dvdef): Add dvd_id member. > (btf_finish): Remove argument from prototype. > (get_btf_id): Delete prototype. > (funcs_traverse_callback, traverse_btf_func_types): Add an > explanatory comment. > * dwarf2ctf.cc (ctf_debug_finish): Remove unused argument. > * dwarf2ctf.h: Analogous change. > * dwarf2out.cc: Likewise. > --- > gcc/btfout.cc | 1256 +++++++++++++++++++--------------------------- > gcc/ctfc.h | 17 +- > gcc/dwarf2ctf.cc | 4 +- > gcc/dwarf2ctf.h | 2 +- > gcc/dwarf2out.cc | 2 +- > 5 files changed, 539 insertions(+), 742 deletions(-) > > diff --git a/gcc/btfout.cc b/gcc/btfout.cc > index 14a503a4f80..0af0bd39fc7 100644 > --- a/gcc/btfout.cc > +++ b/gcc/btfout.cc > @@ -68,53 +68,44 @@ static char btf_info_section_label[MAX_BTF_LABEL_BYTES]; > > #define BTF_INVALID_TYPEID 0xFFFFFFFF > > -/* Mapping of CTF variables to the IDs they will be assigned when they are > - converted to BTF_KIND_VAR type records. Strictly accounts for the index > - from the start of the variable type entries, does not include the number > - of types emitted prior to the variable records. */ > -static GTY (()) hash_map <ctf_dvdef_ref, unsigned> *btf_var_ids; > - > -/* Mapping of type IDs from original CTF ID to BTF ID. Types do not map > - 1-to-1 from CTF to BTF. To avoid polluting the CTF container when updating > - type references-by-ID, we use this map instead. */ > -static ctf_id_t * btf_id_map = NULL; > - > -/* Information for creating the BTF_KIND_DATASEC records. */ > +/* Internal representation of an entry in a BTF_KIND_DATASEC record. */ > +struct btf_datasec_entry > +{ > + union { > + ctf_dvdef_ref dvd; /* Reference to the underlying variable represented. */ > + ctf_dtdef_ref dtd; /* Reference to the underlying type represented. */ > + }; > + bool is_var; /* True iff this entry represents a variable. */ > + uint32_t size; /* Size of variable or function, in bytes. > + For functions, always zero at compile time. */ > +}; > + > +/* Internal representation of a BTF_KIND_DATASEC record. */ > typedef struct btf_datasec > { > - const char *name; /* Section name, e.g. ".bss". */ > - uint32_t name_offset; /* Offset to name in string table. */ > - vec<struct btf_var_secinfo> entries; /* Variable entries in this section. */ > + ctf_id_t id; /* BTF type ID of this record. */ > + const char *name; /* Section name, e.g. ".bss". */ > + uint32_t name_offset; /* Offset to name in string table. */ > + vec<struct btf_datasec_entry> entries; /* Entries in this section. */ > } btf_datasec_t; > > /* One BTF_KIND_DATASEC record is created for each output data section which > will hold at least one variable. */ > static vec<btf_datasec_t> datasecs; > > -/* Holes occur for types which are present in the CTF container, but are either > - non-representable or redundant in BTF. */ > -static vec<ctf_id_t> holes; > - > -/* CTF definition(s) of void. Only one definition of void should be generated. > - We should not encounter more than one definition of void, but use a vector > - to be safe. */ > -static vec<ctf_id_t> voids; > - > /* Functions in BTF have two separate type records - one for the prototype > (BTF_KIND_FUNC_PROTO), as well as a BTF_KIND_FUNC. CTF_K_FUNCTION types > map closely to BTF_KIND_FUNC_PROTO, but the BTF_KIND_FUNC records must be > created. This vector holds them. */ > static GTY (()) vec<ctf_dtdef_ref, va_gc> *funcs; > > -/* The number of BTF variables added to the TU CTF container. */ > -static unsigned int num_vars_added = 0; > - > -/* The number of BTF types added to the TU CTF container. */ > -static unsigned int num_types_added = 0; > +/* Maps BTF_KIND_FUNC_PROTO to the BTF_KIND_FUNC record for it. Used when > + creating DATASEC entries. */ > +static GTY (()) hash_map<ctf_dtdef_ref, ctf_dtdef_ref> *func_map; > > -/* The number of types synthesized for BTF that do not correspond to > - CTF types. */ > -static unsigned int num_types_created = 0; > +/* Highest BTF ID assigned to any regular type translated from CTF. > + Does not include BTF_KIND_{VAR,FUNC,DATASEC} types. */ > +static ctf_id_t max_translated_id = 0; > > /* Name strings for BTF kinds. > Note: the indices here must match the type defines in btf.h. */ > @@ -160,6 +151,16 @@ get_btf_kind (uint32_t ctf_kind) > return BTF_KIND_UNKN; > } > > +/* Convenience wrapper around get_btf_kind for the common case. */ > + > +static uint32_t > +btf_dtd_kind (ctf_dtdef_ref dtd) > +{ > + if (!dtd) > + return BTF_KIND_UNKN; > + return get_btf_kind (CTF_V2_INFO_KIND (dtd->dtd_data.ctti_info)); > +} > + > /* Some BTF types, like BTF_KIND_FUNC_PROTO, are anonymous. The machinery > in btfout to emit BTF, may reset dtd_data->ctti_name, but does not update > the name in the ctf_dtdef_ref type object (deliberate choice). This > @@ -173,101 +174,20 @@ get_btf_type_name (ctf_dtdef_ref dtd) > return (dtd->dtd_data.ctti_name) ? dtd->dtd_name : anon; > } > > -/* Helper routines to map between 'relative' and 'absolute' IDs. > - > - In BTF all records (including variables) are output in one long list, and all > - inter-type references are via index into that list. But internally since we > - a) translate from CTF, which separates variable records from regular types > - and b) create some additional types after the fact, things like VAR and FUNC > - records are stored in separate vectors with their own indices. These > - functions map between the 'relative' IDs (i.e. indices in their respective > - containers) and 'absolute' IDs (i.e. indices in the final contiguous > - output list), which goes in order: > - all normal type records translated from CTF > - all BTF_KIND_VAR records > - all BTF_KIND_FUNC records (synthesized split function records) > - all BTF_KIND_DATASEC records (synthesized) > - > - The extra '+ 1's below are to account for the implicit "void" record, which > - has index 0 but isn't actually contained in the type list. */ > - > -/* Return the final BTF ID of the variable at relative index REL. */ > - > -static ctf_id_t > -btf_absolute_var_id (ctf_id_t rel) > -{ > - return rel + (num_types_added + 1); > -} > - > -/* Return the relative index of the variable with final BTF ID ABS. */ > - > -static ctf_id_t > -btf_relative_var_id (ctf_id_t abs) > -{ > - return abs - (num_types_added + 1); > -} > - > -/* Return the final BTF ID of the func record at relative index REL. */ > - > -static ctf_id_t > -btf_absolute_func_id (ctf_id_t rel) > -{ > - return rel + (num_types_added + 1) + num_vars_added; > -} > - > -/* Return the relative index of the func record with final BTF ID ABS. */ > - > -static ctf_id_t > -btf_relative_func_id (ctf_id_t abs) > -{ > - return abs - ((num_types_added + 1) + num_vars_added); > -} > - > -/* Return the final BTF ID of the datasec record at relative index REL. */ > - > -static ctf_id_t > -btf_absolute_datasec_id (ctf_id_t rel) > +static bool > +btf_emit_type_p (ctf_dtdef_ref dtd) > { > - return rel + (num_types_added + 1) + num_vars_added + funcs->length (); > -} > - > + uint32_t kind = btf_dtd_kind (dtd); > > -/* Allocate the btf_id_map, and initialize elements to BTF_INVALID_TYPEID. */ > + if (kind == BTF_KIND_UNKN) > + /* This type is not representable in BTF. */ > + return false; > > -static void > -init_btf_id_map (size_t len) > -{ > - btf_id_map = XNEWVEC (ctf_id_t, len); > + if (kind == BTF_KIND_INT && dtd->dtd_data.ctti_size == 0) > + /* This is a (redundant) definition of void. */ > + return false; > > - btf_id_map[0] = BTF_VOID_TYPEID; > - for (size_t i = 1; i < len; i++) > - btf_id_map[i] = BTF_INVALID_TYPEID; > -} > - > -/* Return the BTF type ID of CTF type ID KEY, or BTF_INVALID_TYPEID if the CTF > - type with ID KEY does not map to a BTF type. */ > - > -ctf_id_t > -get_btf_id (ctf_id_t key) > -{ > - return btf_id_map[key]; > -} > - > -/* Set the CTF type ID KEY to map to BTF type ID VAL. */ > - > -static inline void > -set_btf_id (ctf_id_t key, ctf_id_t val) > -{ > - btf_id_map[key] = val; > -} > - > -/* Return TRUE iff the given CTF type ID maps to a BTF type which will > - be emitted. */ > -static inline bool > -btf_emit_id_p (ctf_id_t id) > -{ > - return ((btf_id_map[id] != BTF_VOID_TYPEID) > - && (btf_id_map[id] <= BTF_MAX_TYPE)); > + return true; > } > > /* Return true if DTD is a forward-declared enum. The BTF representation > @@ -276,9 +196,8 @@ btf_emit_id_p (ctf_id_t id) > static bool > btf_fwd_to_enum_p (ctf_dtdef_ref dtd) > { > - uint32_t btf_kind = get_btf_kind (CTF_V2_INFO_KIND (dtd->dtd_data.ctti_info)); > - > - return (btf_kind == BTF_KIND_FWD && dtd->dtd_data.ctti_type == CTF_K_ENUM); > + uint32_t kind = btf_dtd_kind (dtd); > + return (kind == BTF_KIND_FWD && dtd->dtd_data.ctti_type == CTF_K_ENUM); > } > > /* Each BTF type can be followed additional, variable-length information > @@ -290,7 +209,7 @@ btf_calc_num_vbytes (ctf_dtdef_ref dtd) > { > uint64_t vlen_bytes = 0; > > - uint32_t kind = get_btf_kind (CTF_V2_INFO_KIND (dtd->dtd_data.ctti_info)); > + uint32_t kind = btf_dtd_kind (dtd); > uint32_t vlen = CTF_V2_INFO_VLEN (dtd->dtd_data.ctti_info); > > switch (kind) > @@ -360,41 +279,6 @@ init_btf_sections (void) > BTF_INFO_SECTION_LABEL, btf_label_num++); > } > > -/* Push a BTF datasec variable entry INFO into the datasec named SECNAME, > - creating the datasec if it does not already exist. */ > - > -static void > -btf_datasec_push_entry (ctf_container_ref ctfc, const char *secname, > - struct btf_var_secinfo info) > -{ > - if (secname == NULL) > - return; > - > - for (size_t i = 0; i < datasecs.length (); i++) > - if (strcmp (datasecs[i].name, secname) == 0) > - { > - datasecs[i].entries.safe_push (info); > - return; > - } > - > - /* If we don't already have a datasec record for secname, make one. */ > - > - uint32_t str_off; > - ctf_add_string (ctfc, secname, &str_off, CTF_AUX_STRTAB); > - if (strcmp (secname, "")) > - ctfc->ctfc_aux_strlen += strlen (secname) + 1; > - > - btf_datasec_t ds; > - ds.name = secname; > - ds.name_offset = str_off; > - > - ds.entries.create (0); > - ds.entries.safe_push (info); > - > - datasecs.safe_push (ds); > -} > - > - > /* Return the section name, as of interest to btf_collect_datasec, for the > given symtab node. Note that this deliberately returns NULL for objects > which do not go in a section btf_collect_datasec cares about. */ > @@ -423,301 +307,15 @@ get_section_name (symtab_node *node) > return section_name; > } > > -/* Construct all BTF_KIND_DATASEC records for CTFC. One such record is created > - for each non-empty data-containing section in the output. Each record is > - followed by a variable number of entries describing the variables stored > - in that section. */ > - > -static void > -btf_collect_datasec (ctf_container_ref ctfc) > -{ > - cgraph_node *func; > - FOR_EACH_FUNCTION (func) > - { > - dw_die_ref die = lookup_decl_die (func->decl); > - if (die == NULL) > - continue; > - > - ctf_dtdef_ref dtd = ctf_dtd_lookup (ctfc, die); > - if (dtd == NULL) > - continue; > - > - if (DECL_EXTERNAL (func->decl) > - && (lookup_attribute ("kernel_helper", > - DECL_ATTRIBUTES (func->decl))) != NULL_TREE) > - continue; > - > - /* Functions actually get two types: a BTF_KIND_FUNC_PROTO, and > - also a BTF_KIND_FUNC. But the CTF container only allocates one > - type per function, which matches closely with BTF_KIND_FUNC_PROTO. > - For each such function, also allocate a BTF_KIND_FUNC entry. > - These will be output later. */ > - ctf_dtdef_ref func_dtd = ggc_cleared_alloc<ctf_dtdef_t> (); > - func_dtd->dtd_data = dtd->dtd_data; > - func_dtd->dtd_data.ctti_type = dtd->dtd_type; > - func_dtd->linkage = dtd->linkage; > - func_dtd->dtd_name = dtd->dtd_name; > - /* +1 for the sentinel type not in the types map. */ > - func_dtd->dtd_type = num_types_added + num_types_created + 1; > - > - /* Only the BTF_KIND_FUNC type actually references the name. The > - BTF_KIND_FUNC_PROTO is always anonymous. */ > - dtd->dtd_data.ctti_name = 0; > - > - vec_safe_push (funcs, func_dtd); > - num_types_created++; > - > - /* Mark any 'extern' funcs and add DATASEC entries for them. */ > - if (DECL_EXTERNAL (func->decl)) > - { > - func_dtd->linkage = BTF_FUNC_EXTERN; > - > - const char *section_name = get_section_name (func); > - /* Note: get_section_name () returns NULL for functions in text > - section. This is intentional, since we do not want to generate > - DATASEC entries for them. */ > - if (section_name == NULL) > - continue; > - > - struct btf_var_secinfo info; > - > - info.type = func_dtd->dtd_type; > - > - /* Both zero at compile time. */ > - info.size = 0; > - info.offset = 0; > - > - btf_datasec_push_entry (ctfc, section_name, info); > - } > - } > - > - varpool_node *node; > - FOR_EACH_VARIABLE (node) > - { > - dw_die_ref die = lookup_decl_die (node->decl); > - if (die == NULL) > - continue; > - > - ctf_dvdef_ref dvd = ctf_dvd_lookup (ctfc, die); > - if (dvd == NULL) > - continue; > - > - /* Mark extern variables. */ > - if (DECL_EXTERNAL (node->decl)) > - { > - dvd->dvd_visibility = BTF_VAR_GLOBAL_EXTERN; > - > - /* PR112849: avoid assuming a section for extern decls without > - an explicit section, which would result in incorrectly > - emitting a BTF_KIND_DATASEC entry for them. */ > - if (node->get_section () == NULL) > - continue; > - } > - > - const char *section_name = get_section_name (node); > - if (section_name == NULL) > - continue; > - > - struct btf_var_secinfo info; > - > - info.type = 0; > - unsigned int *var_id = btf_var_ids->get (dvd); > - if (var_id) > - info.type = btf_absolute_var_id (*var_id); > - else > - continue; > - > - info.size = 0; > - tree size = DECL_SIZE_UNIT (node->decl); > - if (tree_fits_uhwi_p (size)) > - info.size = tree_to_uhwi (size); > - else if (VOID_TYPE_P (TREE_TYPE (node->decl))) > - info.size = 1; > - > - /* Offset is left as 0 at compile time, to be filled in by loaders such > - as libbpf. */ > - info.offset = 0; > - > - btf_datasec_push_entry (ctfc, section_name, info); > - } > - > - num_types_created += datasecs.length (); > -} > - > -/* Return true if the type ID is that of a type which will not be emitted (for > - example, if it is not representable in BTF). */ > - > -static bool > -btf_removed_type_p (ctf_id_t id) > -{ > - return holes.contains (id); > -} > - > -/* Adjust the given type ID to account for holes and duplicate definitions of > - void. */ > - > -static ctf_id_t > -btf_adjust_type_id (ctf_id_t id) > -{ > - size_t n; > - ctf_id_t i = 0; > - > - /* Do not adjust invalid type markers. */ > - if (id == BTF_INVALID_TYPEID) > - return id; > - > - for (n = 0; n < voids.length (); n++) > - if (id == voids[n]) > - return BTF_VOID_TYPEID; > - > - for (n = 0; n < holes.length (); n++) > - { > - if (holes[n] < id) > - i++; > - else if (holes[n] == id) > - return BTF_VOID_TYPEID; > - } > - > - return id - i; > -} > - > -/* Postprocessing callback routine for types. */ > - > -int > -btf_dtd_postprocess_cb (ctf_dtdef_ref *slot, ctf_container_ref arg_ctfc) > -{ > - ctf_dtdef_ref ctftype = (ctf_dtdef_ref) * slot; > - > - size_t index = ctftype->dtd_type; > - gcc_assert (index <= arg_ctfc->ctfc_types->elements ()); > - > - uint32_t ctf_kind, btf_kind; > - > - ctf_kind = CTF_V2_INFO_KIND (ctftype->dtd_data.ctti_info); > - btf_kind = get_btf_kind (ctf_kind); > - > - if (btf_kind == BTF_KIND_UNKN) > - /* This type is not representable in BTF. Create a hole. */ > - holes.safe_push (ctftype->dtd_type); > - > - else if (btf_kind == BTF_KIND_INT && ctftype->dtd_data.ctti_size == 0) > - { > - /* This is a (redundant) definition of void. */ > - voids.safe_push (ctftype->dtd_type); > - holes.safe_push (ctftype->dtd_type); > - } > - > - arg_ctfc->ctfc_types_list[index] = ctftype; > - > - return 1; > -} > - > -/* Preprocessing callback routine for variables. */ > - > -int > -btf_dvd_emit_preprocess_cb (ctf_dvdef_ref *slot, ctf_container_ref arg_ctfc) > -{ > - ctf_dvdef_ref var = (ctf_dvdef_ref) * slot; > - > - /* If this is an extern variable declaration with a defining declaration > - later, skip it so that only the defining declaration is emitted. > - This is the same case, fix and reasoning as in CTF; see PR105089. */ > - if (ctf_dvd_ignore_lookup (arg_ctfc, var->dvd_key)) > - return 1; > - > - /* Do not add variables which refer to unsupported types. */ > - if (!voids.contains (var->dvd_type->dtd_type) > - && btf_removed_type_p (var->dvd_type->dtd_type)) > - return 1; > - > - arg_ctfc->ctfc_vars_list[num_vars_added] = var; > - btf_var_ids->put (var, num_vars_added); > - > - num_vars_added++; > - num_types_created++; > - > - return 1; > -} > - > -/* Preprocessing callback routine for types. */ > - > -static void > -btf_dtd_emit_preprocess_cb (ctf_container_ref ctfc, ctf_dtdef_ref dtd) > -{ > - if (!btf_emit_id_p (dtd->dtd_type)) > - return; > - > - ctfc->ctfc_num_vlen_bytes += btf_calc_num_vbytes (dtd); > -} > - > -/* Preprocess the CTF information to prepare for BTF output. BTF is almost a > - subset of CTF, with many small differences in encoding, and lacking support > - for some types (notably floating point formats). > - > - During the preprocessing pass: > - - Ascertain that the sorted list of types has been prepared. For the BTF > - generation process, this is taken care of by the btf_init_postprocess (). > - > - - BTF_KIND_FUNC and BTF_KIND_DATASEC records are constructed. These types do > - not have analogues in CTF (the analogous type to CTF_K_FUNCTION is > - BTF_KIND_FUNC_PROTO), but can be relatively easily deduced from CTF > - information. > - > - - Construct BTF_KIND_VAR records, representing variables. > - > - - Calculate the total size in bytes of variable-length information following > - BTF type records. This is used for outputting the BTF header. > - > - After preprocessing, all BTF information is ready to be output: > - - ctfc->ctfc_types_list holdstypes converted from CTF types. This does not > - include KIND_VAR, KIND_FUNC, nor KIND_DATASEC types. These types have been > - re-encoded to the appropriate representation in BTF. > - - ctfc->ctfc_vars_list holds all variables which should be output. > - Variables of unsupported types are not present in this list. > - - Vector 'funcs' holds all BTF_KIND_FUNC types, one to match each > - BTF_KIND_FUNC_PROTO. > - - Vector 'datasecs' holds all BTF_KIND_DATASEC types. */ > - > -static void > -btf_emit_preprocess (ctf_container_ref ctfc) > -{ > - size_t num_ctf_types = ctfc->ctfc_types->elements (); > - size_t num_ctf_vars = ctfc->ctfc_vars->elements (); > - size_t i; > - > - if (num_ctf_types) > - { > - gcc_assert (ctfc->ctfc_types_list); > - /* Preprocess the types. */ > - for (i = 1; i <= num_ctf_types; i++) > - btf_dtd_emit_preprocess_cb (ctfc, ctfc->ctfc_types_list[i]); > - } > - > - btf_var_ids = hash_map<ctf_dvdef_ref, unsigned int>::create_ggc (100); > - > - if (num_ctf_vars) > - { > - /* Allocate and construct the list of variables. While BTF variables are > - not distinct from types (in that variables are simply types with > - BTF_KIND_VAR), it is simpler to maintain a separate list of variables > - and append them to the types list during output. */ > - ctfc->ctfc_vars_list = ggc_vec_alloc<ctf_dvdef_ref>(num_ctf_vars); > - ctfc->ctfc_vars->traverse<ctf_container_ref, btf_dvd_emit_preprocess_cb> > - (ctfc); > - > - ctfc->ctfc_num_vlen_bytes += (num_vars_added * sizeof (struct btf_var)); > - } > - > - btf_collect_datasec (ctfc); > -} > - > /* Return true iff DMD is a member description of a bit-field which can be > validly represented in BTF. */ > > static bool > -btf_dmd_representable_bitfield_p (ctf_container_ref ctfc, ctf_dmdef_t *dmd) > +btf_dmd_representable_bitfield_p (ctf_dmdef_t *dmd) > { > - ctf_dtdef_ref ref_type = ctfc->ctfc_types_list[dmd->dmd_type->dtd_type]; > + ctf_dtdef_ref ref_type = dmd->dmd_type; > + if (!ref_type) > + return false; > > if (CTF_V2_INFO_KIND (ref_type->dtd_data.ctti_info) == CTF_K_SLICE) > { > @@ -739,76 +337,34 @@ btf_dmd_representable_bitfield_p (ctf_container_ref ctfc, ctf_dmdef_t *dmd) > /* Asm'out a reference to another BTF type. */ > > static void > -btf_asm_type_ref (const char *prefix, ctf_container_ref ctfc, ctf_id_t ctf_id) > +btf_asm_type_ref (const char *prefix, ctf_dtdef_ref dtd) > { > - ctf_id_t btf_id = get_btf_id (ctf_id); > - if (btf_id == BTF_VOID_TYPEID || btf_id == BTF_INVALID_TYPEID) > - { > - /* There is no explicit void type. > - Also handle any invalid refs that made it this far, just in case. */ > - dw2_asm_output_data (4, btf_id, "%s: void", prefix); > - } > + if (!dtd || !btf_emit_type_p (dtd)) > + dw2_asm_output_data (4, BTF_VOID_TYPEID, "%s: void", prefix); > else > { > - gcc_assert (btf_id <= num_types_added); > - > - /* Ref to a standard type in the types list. Note: take care that we > - must index the type list by the original CTF id, not the BTF id. */ > - ctf_dtdef_ref ref_type = ctfc->ctfc_types_list[ctf_id]; > - uint32_t ref_kind > - = get_btf_kind (CTF_V2_INFO_KIND (ref_type->dtd_data.ctti_info)); > - > - const char *kind_name = btf_fwd_to_enum_p (ref_type) > - ? btf_kind_name (BTF_KIND_ENUM) > - : btf_kind_name (ref_kind); > - > - dw2_asm_output_data (4, btf_id, "%s: (BTF_KIND_%s '%s')", > - prefix, kind_name, > - get_btf_type_name (ref_type)); > + uint32_t kind = btf_dtd_kind (dtd); > + if (btf_fwd_to_enum_p (dtd)) > + kind = BTF_KIND_ENUM; > + else if (kind == BTF_KIND_FUNC_PROTO && dtd->dtd_type > max_translated_id) > + kind = BTF_KIND_FUNC; > + > + dw2_asm_output_data (4, dtd->dtd_type, "%s: (BTF_KIND_%s '%s')", > + prefix, btf_kind_name (kind), > + get_btf_type_name (dtd)); > } > } > > -/* Asm'out a reference to a BTF_KIND_VAR or BTF_KIND_FUNC type. These type > - kinds are BTF-specific, and should only be referred to by entries in > - BTF_KIND_DATASEC records. */ > - > -static void > -btf_asm_datasec_type_ref (const char *prefix, ctf_container_ref ctfc, > - ctf_id_t btf_id) > -{ > - if (btf_id >= num_types_added + 1 > - && btf_id < num_types_added + num_vars_added + 1) > - { > - /* Ref to a variable. Should only appear in DATASEC entries. */ > - ctf_id_t var_id = btf_relative_var_id (btf_id); > - ctf_dvdef_ref dvd = ctfc->ctfc_vars_list[var_id]; > - dw2_asm_output_data (4, btf_id, "%s: (BTF_KIND_VAR '%s')", > - prefix, dvd->dvd_name); > - > - } > - else if (btf_id >= num_types_added + num_vars_added + 1) > - { > - /* Ref to a FUNC record. */ > - size_t func_id = btf_relative_func_id (btf_id); > - ctf_dtdef_ref ref_type = (*funcs)[func_id]; > - dw2_asm_output_data (4, btf_id, "%s: (BTF_KIND_FUNC '%s')", > - prefix, get_btf_type_name (ref_type)); > - } > - else > - /* The caller should not be calling this. */ > - gcc_unreachable (); > -} > - > /* Asm'out a BTF type. This routine is responsible for the bulk of the task > of converting CTF types to their BTF representation. */ > > static void > -btf_asm_type (ctf_container_ref ctfc, ctf_dtdef_ref dtd) > +btf_asm_type (ctf_dtdef_ref dtd) > { > uint32_t btf_kind, btf_kflag, btf_vlen, btf_size; > uint32_t ctf_info = dtd->dtd_data.ctti_info; > > - btf_kind = get_btf_kind (CTF_V2_INFO_KIND (ctf_info)); > + btf_kind = btf_dtd_kind (dtd); > btf_size = dtd->dtd_data.ctti_size; > btf_vlen = CTF_V2_INFO_VLEN (ctf_info); > > @@ -827,17 +383,17 @@ btf_asm_type (ctf_container_ref ctfc, ctf_dtdef_ref dtd) > > if (btf_kind == BTF_KIND_STRUCT || btf_kind == BTF_KIND_UNION) > { > - /* If a struct/union has ANY bitfield members, set kflag=1. > - Note that we must also change the encoding of every member to encode > - both member bitfield size (stealing most-significant 8 bits) and bit > - offset (LS 24 bits). This is done during preprocessing. */ > + /* If a struct/union has ANY bitfield members, set kflag=1. */ > ctf_dmdef_t *dmd; > for (dmd = dtd->dtd_u.dtu_members; > dmd != NULL; dmd = (ctf_dmdef_t *) ctf_dmd_list_next (dmd)) > { > /* Set kflag if this member is a representable bitfield. */ > - if (btf_dmd_representable_bitfield_p (ctfc, dmd)) > - btf_kflag = 1; > + if (btf_dmd_representable_bitfield_p (dmd)) > + { > + btf_kflag = 1; > + break; > + } > } > } > > @@ -871,7 +427,7 @@ btf_asm_type (ctf_container_ref ctfc, ctf_dtdef_ref dtd) > : BTF_KF_ENUM_SIGNED; > if (dtd->dtd_data.ctti_size == 0x8) > btf_kind = BTF_KIND_ENUM64; > - } > + } > > /* PR debug/112656. BTF_KIND_FUNC_PROTO is always anonymous. */ > else if (btf_kind == BTF_KIND_FUNC_PROTO) > @@ -879,7 +435,7 @@ btf_asm_type (ctf_container_ref ctfc, ctf_dtdef_ref dtd) > > dw2_asm_output_data (4, dtd->dtd_data.ctti_name, > "TYPE %" PRIu64 " BTF_KIND_%s '%s'", > - get_btf_id (dtd->dtd_type), btf_kind_name (btf_kind), > + dtd->dtd_type, btf_kind_name (btf_kind), > get_btf_type_name (dtd)); > dw2_asm_output_data (4, BTF_TYPE_INFO (btf_kind, btf_kflag, btf_vlen), > "btt_info: kind=%u, kflag=%u, vlen=%u", > @@ -905,30 +461,29 @@ btf_asm_type (ctf_container_ref ctfc, ctf_dtdef_ref dtd) > break; > } > > - ctf_id_t ref_id = dtd->dtd_data.ctti_type; > - btf_asm_type_ref ("btt_type", ctfc, ref_id); > + btf_asm_type_ref ("btt_type", dtd->ref_type); > } > > /* Asm'out the variable information following a BTF_KIND_ARRAY. */ > > static void > -btf_asm_array (ctf_container_ref ctfc, ctf_arinfo_t arr) > +btf_asm_array (ctf_arinfo_t arr) > { > - btf_asm_type_ref ("bta_elem_type", ctfc, arr.ctr_contents->dtd_type); > - btf_asm_type_ref ("bta_index_type", ctfc, arr.ctr_index->dtd_type); > + btf_asm_type_ref ("bta_elem_type", arr.ctr_contents); > + btf_asm_type_ref ("bta_index_type", arr.ctr_index); > dw2_asm_output_data (4, arr.ctr_nelems, "bta_nelems"); > } > > /* Asm'out a BTF_KIND_VAR. */ > > static void > -btf_asm_varent (ctf_container_ref ctfc, ctf_dvdef_ref var) > +btf_asm_varent (ctf_dvdef_ref var) > { > - dw2_asm_output_data (4, var->dvd_name_offset, "TYPE %u BTF_KIND_VAR '%s'", > - (*(btf_var_ids->get (var)) + num_types_added + 1), > - var->dvd_name); > + dw2_asm_output_data (4, var->dvd_name_offset, > + "TYPE %" PRIu64 " BTF_KIND_VAR '%s'", > + var->dvd_id, var->dvd_name); > dw2_asm_output_data (4, BTF_TYPE_INFO (BTF_KIND_VAR, 0, 0), "btv_info"); > - btf_asm_type_ref ("btv_type", ctfc, var->dvd_type->dtd_type); > + btf_asm_type_ref ("btv_type", var->dvd_type); > dw2_asm_output_data (4, var->dvd_visibility, "btv_linkage"); > } > > @@ -936,23 +491,22 @@ btf_asm_varent (ctf_container_ref ctfc, ctf_dvdef_ref var) > BTF_KIND_UNION. */ > > static void > -btf_asm_sou_member (ctf_container_ref ctfc, ctf_dmdef_t * dmd, unsigned int idx) > +btf_asm_sou_member (ctf_dmdef_t * dmd, unsigned int idx) > { > - ctf_dtdef_ref ref_type = ctfc->ctfc_types_list[dmd->dmd_type->dtd_type]; > - ctf_id_t base_type = dmd->dmd_type->dtd_type; > + ctf_dtdef_ref base_type = dmd->dmd_type; > uint64_t sou_offset = dmd->dmd_offset; > > dw2_asm_output_data (4, dmd->dmd_name_offset, > "MEMBER '%s' idx=%u", > dmd->dmd_name, idx); > > - /* Re-encode bitfields to BTF representation. */ > - if (CTF_V2_INFO_KIND (ref_type->dtd_data.ctti_info) == CTF_K_SLICE) > + if (base_type > + && CTF_V2_INFO_KIND (base_type->dtd_data.ctti_info) == CTF_K_SLICE) > { > - if (btf_dmd_representable_bitfield_p (ctfc, dmd)) > + if (btf_dmd_representable_bitfield_p (dmd)) > { > - unsigned short word_offset = ref_type->dtd_u.dtu_slice.cts_offset; > - unsigned short bits = ref_type->dtd_u.dtu_slice.cts_bits; > + unsigned short word_offset = base_type->dtd_u.dtu_slice.cts_offset; > + unsigned short bits = base_type->dtd_u.dtu_slice.cts_bits; > > /* Pack the bit offset and bitfield size together. */ > sou_offset += word_offset; > @@ -960,17 +514,17 @@ btf_asm_sou_member (ctf_container_ref ctfc, ctf_dmdef_t * dmd, unsigned int idx) > sou_offset |= ((bits & 0xff) << 24); > > /* Refer to the base type of the slice. */ > - base_type = ref_type->dtd_u.dtu_slice.cts_type->dtd_type; > + base_type = base_type->dtd_u.dtu_slice.cts_type; > } > else > { > /* Bitfield cannot be represented in BTF. Emit the member as having > 'void' type. */ > - base_type = BTF_VOID_TYPEID; > + base_type = NULL; > } > } > > - btf_asm_type_ref ("btm_type", ctfc, base_type); > + btf_asm_type_ref ("btm_type", base_type); > dw2_asm_output_data (4, sou_offset, "btm_offset"); > } > > @@ -993,86 +547,68 @@ btf_asm_enum_const (unsigned int size, ctf_dmdef_t * dmd, unsigned int idx) > /* Asm'out a function parameter description following a BTF_KIND_FUNC_PROTO. */ > > static void > -btf_asm_func_arg (ctf_container_ref ctfc, ctf_func_arg_t * farg, > - size_t stroffset) > +btf_asm_func_arg (ctf_func_arg_t * farg, size_t stroffset) > { > /* If the function arg does not have a name, refer to the null string at > the start of the string table. This ensures correct encoding for varargs > '...' arguments. */ > if ((farg->farg_name != NULL) && strcmp (farg->farg_name, "")) > - dw2_asm_output_data (4, farg->farg_name_offset + stroffset, "farg_name"); > + dw2_asm_output_data (4, farg->farg_name_offset + stroffset, > + "farg_name '%s'", farg->farg_name); > else > - dw2_asm_output_data (4, 0, "farg_name"); > - > - ctf_id_t ref_id = BTF_VOID_TYPEID; > - if (farg->farg_type && !btf_removed_type_p (farg->farg_type->dtd_type)) > - ref_id = farg->farg_type->dtd_type; > + dw2_asm_output_data (4, 0, "farg_name ''"); > > - btf_asm_type_ref ("farg_type", ctfc, ref_id); > + btf_asm_type_ref ("farg_type", farg->farg_type); > } > > /* Asm'out a BTF_KIND_FUNC type. */ > > static void > -btf_asm_func_type (ctf_container_ref ctfc, ctf_dtdef_ref dtd, ctf_id_t id) > +btf_asm_func_type (ctf_dtdef_ref dtd) > { > - ctf_id_t ref_id = dtd->dtd_data.ctti_type; > dw2_asm_output_data (4, dtd->dtd_data.ctti_name, > "TYPE %" PRIu64 " BTF_KIND_FUNC '%s'", > - btf_absolute_func_id (id), get_btf_type_name (dtd)); > + dtd->dtd_type, get_btf_type_name (dtd)); > dw2_asm_output_data (4, BTF_TYPE_INFO (BTF_KIND_FUNC, 0, dtd->linkage), > "btt_info: kind=%u, kflag=%u, linkage=%u", > BTF_KIND_FUNC, 0, dtd->linkage); > - btf_asm_type_ref ("btt_type", ctfc, ref_id); > + btf_asm_type_ref ("btt_type", dtd->ref_type); > } > > -/* Collect the name for the DATASEC reference required to be output as a > - symbol. */ > +/* Asm'out a variable entry following a BTF_KIND_DATASEC. */ > > -static const char * > -get_name_for_datasec_entry (ctf_container_ref ctfc, ctf_id_t ref_id) > +static void > +btf_asm_datasec_entry (struct btf_datasec_entry entry) > { > - if (ref_id >= num_types_added + 1 > - && ref_id < num_types_added + num_vars_added + 1) > + const char *symbol_name = NULL; > + if (entry.is_var) > { > - /* Ref to a variable. Should only appear in DATASEC entries. */ > - ctf_id_t var_id = btf_relative_var_id (ref_id); > - ctf_dvdef_ref dvd = ctfc->ctfc_vars_list[var_id]; > - return dvd->dvd_name; > + symbol_name = entry.dvd->dvd_name; > + dw2_asm_output_data (4, entry.dvd->dvd_id, > + "bts_type: (BTF_KIND_VAR '%s')", symbol_name); > } > - else if (ref_id >= num_types_added + num_vars_added + 1) > + else > { > - /* Ref to a FUNC record. */ > - size_t func_id = btf_relative_func_id (ref_id); > - ctf_dtdef_ref ref_type = (*funcs)[func_id]; > - return get_btf_type_name (ref_type); > + symbol_name = entry.dtd->dtd_name; > + btf_asm_type_ref ("bts_type", entry.dtd); > } > - return NULL; > -} > - > -/* Asm'out a variable entry following a BTF_KIND_DATASEC. */ > > -static void > -btf_asm_datasec_entry (ctf_container_ref ctfc, struct btf_var_secinfo info) > -{ > - const char *symbol_name = get_name_for_datasec_entry (ctfc, info.type); > - btf_asm_datasec_type_ref ("bts_type", ctfc, info.type); > if (!btf_with_core_debuginfo_p () || symbol_name == NULL) > - dw2_asm_output_data (4, info.offset, "bts_offset"); > + dw2_asm_output_data (4, 0, "bts_offset"); > else > dw2_asm_output_offset (4, symbol_name, NULL, "bts_offset"); > - dw2_asm_output_data (4, info.size, "bts_size"); > + > + dw2_asm_output_data (4, entry.size, "bts_size"); > } > > /* Asm'out a whole BTF_KIND_DATASEC, including its variable entries. */ > > static void > -btf_asm_datasec_type (ctf_container_ref ctfc, btf_datasec_t ds, ctf_id_t id, > - size_t stroffset) > +btf_asm_datasec_type (btf_datasec_t ds) > { > - dw2_asm_output_data (4, ds.name_offset + stroffset, > + dw2_asm_output_data (4, ds.name_offset, > "TYPE %" PRIu64 " BTF_KIND_DATASEC '%s'", > - btf_absolute_datasec_id (id), ds.name); > + ds.id, ds.name); > dw2_asm_output_data (4, BTF_TYPE_INFO (BTF_KIND_DATASEC, 0, > ds.entries.length ()), > "btt_info: n_entries=%u", ds.entries.length ()); > @@ -1080,7 +616,7 @@ btf_asm_datasec_type (ctf_container_ref ctfc, btf_datasec_t ds, ctf_id_t id, > loaders such as libbpf. */ > dw2_asm_output_data (4, 0, "btt_size"); > for (size_t i = 0; i < ds.entries.length (); i++) > - btf_asm_datasec_entry (ctfc, ds.entries[i]); > + btf_asm_datasec_entry (ds.entries[i]); > } > > /* Compute and output the header information for a .BTF section. */ > @@ -1099,20 +635,11 @@ output_btf_header (ctf_container_ref ctfc) > > uint32_t type_off = 0, type_len = 0; > uint32_t str_off = 0, str_len = 0; > - uint32_t datasec_vlen_bytes = 0; > > if (!ctfc_is_empty_container (ctfc)) > { > - for (size_t i = 0; i < datasecs.length (); i++) > - { > - datasec_vlen_bytes += ((datasecs[i].entries.length ()) > - * sizeof (struct btf_var_secinfo)); > - } > - > /* Total length (bytes) of the types section. */ > - type_len = (num_types_added * sizeof (struct btf_type)) > - + (num_types_created * sizeof (struct btf_type)) > - + datasec_vlen_bytes > + type_len = ctfc->ctfc_num_types * sizeof (struct btf_type) > + ctfc->ctfc_num_vlen_bytes; > > str_off = type_off + type_len; > @@ -1124,7 +651,9 @@ output_btf_header (ctf_container_ref ctfc) > /* Offset of type section. */ > dw2_asm_output_data (4, type_off, "type_off"); > /* Length of type section in bytes. */ > - dw2_asm_output_data (4, type_len, "type_len"); > + dw2_asm_output_data (4, type_len, "type_len: ntypes=%u, vlen=%u", > + (uint32_t) ctfc->ctfc_num_types, > + (uint32_t) ctfc->ctfc_num_vlen_bytes); > /* Offset of string section. */ > dw2_asm_output_data (4, str_off, "str_off"); > /* Length of string section in bytes. */ > @@ -1137,11 +666,11 @@ static void > output_btf_vars (ctf_container_ref ctfc) > { > size_t i; > - size_t num_ctf_vars = num_vars_added; > + size_t num_ctf_vars = ctfc->ctfc_vars_list_count; > if (num_ctf_vars) > { > for (i = 0; i < num_ctf_vars; i++) > - btf_asm_varent (ctfc, ctfc->ctfc_vars_list[i]); > + btf_asm_varent (ctfc->ctfc_vars_list[i]); > } > } > > @@ -1156,7 +685,8 @@ output_btf_strs (ctf_container_ref ctfc) > > while (ctf_string) > { > - dw2_asm_output_nstring (ctf_string->cts_str, -1, "btf_string, str_pos = 0x%x", str_pos); > + dw2_asm_output_nstring (ctf_string->cts_str, -1, > + "btf_string, str_pos = 0x%x", str_pos); > str_pos += strlen(ctf_string->cts_str) + 1; > ctf_string = ctf_string->cts_next; > } > @@ -1164,7 +694,8 @@ output_btf_strs (ctf_container_ref ctfc) > ctf_string = ctfc->ctfc_aux_strtable.ctstab_head; > while (ctf_string) > { > - dw2_asm_output_nstring (ctf_string->cts_str, -1, "btf_aux_string, str_pos = 0x%x", str_pos); > + dw2_asm_output_nstring (ctf_string->cts_str, -1, > + "btf_aux_string, str_pos = 0x%x", str_pos); > str_pos += strlen(ctf_string->cts_str) + 1; > ctf_string = ctf_string->cts_next; > } > @@ -1174,7 +705,7 @@ output_btf_strs (ctf_container_ref ctfc) > BTF_KIND_UNION type. */ > > static void > -output_asm_btf_sou_fields (ctf_container_ref ctfc, ctf_dtdef_ref dtd) > +output_asm_btf_sou_fields (ctf_dtdef_ref dtd) > { > ctf_dmdef_t * dmd; > > @@ -1182,7 +713,7 @@ output_asm_btf_sou_fields (ctf_container_ref ctfc, ctf_dtdef_ref dtd) > for (dmd = dtd->dtd_u.dtu_members; > dmd != NULL; dmd = (ctf_dmdef_t *) ctf_dmd_list_next (dmd)) > { > - btf_asm_sou_member (ctfc, dmd, idx); > + btf_asm_sou_member (dmd, idx); > idx++; > } > } > @@ -1190,8 +721,7 @@ output_asm_btf_sou_fields (ctf_container_ref ctfc, ctf_dtdef_ref dtd) > /* Output all enumerator constants following a BTF_KIND_ENUM{,64}. */ > > static void > -output_asm_btf_enum_list (ctf_container_ref ARG_UNUSED (ctfc), > - ctf_dtdef_ref dtd) > +output_asm_btf_enum_list (ctf_dtdef_ref dtd) > { > ctf_dmdef_t * dmd; > > @@ -1214,7 +744,7 @@ output_asm_btf_func_args_list (ctf_container_ref ctfc, > ctf_func_arg_t * farg; > for (farg = dtd->dtd_u.dtu_argv; > farg != NULL; farg = (ctf_func_arg_t *) ctf_farg_list_next (farg)) > - btf_asm_func_arg (ctfc, farg, farg_name_offset); > + btf_asm_func_arg (farg, farg_name_offset); > } > > /* Output the variable portion of a BTF type record. The information depends > @@ -1225,7 +755,7 @@ output_asm_btf_vlen_bytes (ctf_container_ref ctfc, ctf_dtdef_ref dtd) > { > uint32_t btf_kind, encoding; > > - btf_kind = get_btf_kind (CTF_V2_INFO_KIND (dtd->dtd_data.ctti_info)); > + btf_kind = btf_dtd_kind (dtd); > > if (btf_kind == BTF_KIND_UNKN) > return; > @@ -1238,8 +768,7 @@ output_asm_btf_vlen_bytes (ctf_container_ref ctfc, ctf_dtdef_ref dtd) > if (dtd->dtd_data.ctti_size < 1) > break; > > - /* In BTF the CHAR `encoding' seems to not be used, so clear it > - here. */ > + /* In BTF the CHAR `encoding' seems to not be used, so clear it here. */ > dtd->dtd_u.dtu_enc.cte_format &= ~BTF_INT_CHAR; > > encoding = BTF_INT_DATA (dtd->dtd_u.dtu_enc.cte_format, > @@ -1250,16 +779,16 @@ output_asm_btf_vlen_bytes (ctf_container_ref ctfc, ctf_dtdef_ref dtd) > break; > > case BTF_KIND_ARRAY: > - btf_asm_array (ctfc, dtd->dtd_u.dtu_arr); > + btf_asm_array (dtd->dtd_u.dtu_arr); > break; > > case BTF_KIND_STRUCT: > case BTF_KIND_UNION: > - output_asm_btf_sou_fields (ctfc, dtd); > + output_asm_btf_sou_fields (dtd); > break; > > case BTF_KIND_ENUM: > - output_asm_btf_enum_list (ctfc, dtd); > + output_asm_btf_enum_list (dtd); > break; > > case BTF_KIND_FUNC_PROTO: > @@ -1289,9 +818,9 @@ output_asm_btf_vlen_bytes (ctf_container_ref ctfc, ctf_dtdef_ref dtd) > static void > output_asm_btf_type (ctf_container_ref ctfc, ctf_dtdef_ref type) > { > - if (btf_emit_id_p (type->dtd_type)) > + if (btf_emit_type_p (type)) > { > - btf_asm_type (ctfc, type); > + btf_asm_type (type); > output_asm_btf_vlen_bytes (ctfc, type); > } > } > @@ -1303,7 +832,9 @@ static void > output_btf_types (ctf_container_ref ctfc) > { > size_t i; > - size_t num_types = ctfc->ctfc_types->elements (); > + size_t num_types; > + num_types = ctfc->ctfc_types->elements (); > + > if (num_types) > { > for (i = 1; i <= num_types; i++) > @@ -1314,76 +845,45 @@ output_btf_types (ctf_container_ref ctfc) > /* Output all BTF_KIND_FUNC type records. */ > > static void > -output_btf_func_types (ctf_container_ref ctfc) > +output_btf_func_types (void) > { > ctf_dtdef_ref ref; > unsigned i; > FOR_EACH_VEC_ELT (*funcs, i, ref) > - btf_asm_func_type (ctfc, ref, i); > + btf_asm_func_type (ref); > } > > /* Output all BTF_KIND_DATASEC records. */ > > static void > -output_btf_datasec_types (ctf_container_ref ctfc) > +output_btf_datasec_types (void) > { > - size_t name_offset = ctfc_get_strtab_len (ctfc, CTF_STRTAB); > - > - for (size_t i = 0; i < datasecs.length(); i++) > - btf_asm_datasec_type (ctfc, datasecs[i], i, name_offset); > + for (size_t i = 0; i < datasecs.length (); i++) > + btf_asm_datasec_type (datasecs[i]); > } > > -/* Postprocess the CTF debug data post initialization. > - > - During the postprocess pass: > - > - - Prepare the sorted list of BTF types. > - > - The sorted list of BTF types is, firstly, used for lookup (during the BTF > - generation process) of CTF/BTF types given a typeID. > - > - Secondly, in the emitted BTF section, BTF Types need to be in the sorted > - order of their type IDs. The BTF types section is viewed as an array, > - with type IDs used to index into that array. It is essential that every > - type be placed at the exact index corresponding to its ID, or else > - references to that type from other types will no longer be correct. > - > - - References to void types are converted to reference BTF_VOID_TYPEID. In > - CTF, a distinct type is used to encode void. > - > - - Bitfield struct/union members are converted to BTF encoding. CTF uses > - slices to encode bitfields, but BTF does not have slices and encodes > - bitfield information directly in the variable-length btf_member > - descriptions following the struct or union type. > - > - - Unrepresentable types are removed. We cannot have any invalid BTF types > - appearing in the output so they must be removed, and type ids of other > - types and references adjust accordingly. This also involves ensuring that > - BTF descriptions of struct members referring to unrepresentable types are > - not emitted, as they would be nonsensical. > - > - - Adjust inner- and inter-type references-by-ID to account for removed > - types, and construct the types list. */ > +/* Write out all BTF debug info. */ > > void > -btf_init_postprocess (void) > +btf_output (ctf_container_ref ctfc) > { > - ctf_container_ref tu_ctfc = ctf_get_tu_ctfc (); > - > - holes.create (0); > - voids.create (0); > + output_btf_header (ctfc); > + output_btf_types (ctfc); > + output_btf_vars (ctfc); > + output_btf_func_types (); > + output_btf_datasec_types (); > + output_btf_strs (ctfc); > +} > > - num_types_added = 0; > - num_types_created = 0; > +/* Workaround for 'const void' variables. These variables are sometimes used > + in eBPF programs to address kernel symbols. DWARF does not generate const > + qualifier on void type, so we would incorrectly emit these variables > + without the const qualifier. Find any such variables, and update them to > + refer to a new 'const' modifier type for void. */ > > - /* Workaround for 'const void' variables. These variables are sometimes used > - in eBPF programs to address kernel symbols. DWARF does not generate const > - qualifier on void type, so we would incorrectly emit these variables > - without the const qualifier. > - Unfortunately we need the TREE node to know it was const, and we need > - to create the const modifier type (if needed) now, before making the types > - list. So we can't avoid iterating with FOR_EACH_VARIABLE here, and then > - again when creating the DATASEC entries. */ > +static void > +btf_early_add_const_void (ctf_container_ref ctfc) > +{ > ctf_dtdef_ref constvoid_dtd = NULL; > varpool_node *var; > FOR_EACH_VARIABLE (var) > @@ -1398,120 +898,389 @@ btf_init_postprocess (void) > if (die == NULL) > continue; > > - ctf_dvdef_ref dvd = ctf_dvd_lookup (tu_ctfc, die); > + ctf_dvdef_ref dvd = ctf_dvd_lookup (ctfc, die); > if (dvd == NULL) > continue; > > /* Create the 'const' modifier type for void. */ > if (constvoid_dtd == NULL) > - constvoid_dtd = ctf_add_reftype (tu_ctfc, CTF_ADD_ROOT, > - dvd->dvd_type, CTF_K_CONST, NULL); > + constvoid_dtd = ctf_add_reftype (ctfc, CTF_ADD_ROOT, > + dvd->dvd_type, CTF_K_CONST, NULL); > dvd->dvd_type = constvoid_dtd; > } > } > +} > > - size_t i; > - size_t num_ctf_types = tu_ctfc->ctfc_types->elements (); > +/* Functions actually get two type records: a BTF_KIND_FUNC_PROTO, and also a > + BTF_KIND_FUNC. But the CTF container only allocates one type per function, > + which matches closely with BTF_KIND_FUNC_PROTO. For each such function, > + construct a BTF_KIND_FUNC entry. This is done early, because we want FUNC > + records even for functions which are later inlined by optimizations. */ > > - if (num_ctf_types) > +static void > +btf_early_add_func_records (ctf_container_ref ctfc) > +{ > + cgraph_node *func; > + FOR_EACH_FUNCTION (func) > { > - init_btf_id_map (num_ctf_types + 1); > - > - /* Allocate the types list and traverse all types, placing each type > - at the index according to its ID. Add 1 because type ID 0 always > - represents VOID. */ > - tu_ctfc->ctfc_types_list > - = ggc_vec_alloc<ctf_dtdef_ref>(num_ctf_types + 1); > - tu_ctfc->ctfc_types->traverse<ctf_container_ref, btf_dtd_postprocess_cb> > - (tu_ctfc); > - > - /* Build mapping of CTF type ID -> BTF type ID, and count total number > - of valid BTF types added. */ > - for (i = 1; i <= num_ctf_types; i++) > - { > - ctf_dtdef_ref dtd = tu_ctfc->ctfc_types_list[i]; > - ctf_id_t btfid = btf_adjust_type_id (dtd->dtd_type); > - set_btf_id (dtd->dtd_type, btfid); > - if (btfid < BTF_MAX_TYPE && (btfid != BTF_VOID_TYPEID)) > - num_types_added ++; > - } > + dw_die_ref die = lookup_decl_die (func->decl); > + if (die == NULL) > + continue; > + > + ctf_dtdef_ref dtd = ctf_dtd_lookup (ctfc, die); > + if (dtd == NULL) > + continue; > + > + /* Do not add FUNC records for kernel helpers. */ > + if (DECL_EXTERNAL (func->decl) > + && (lookup_attribute ("kernel_helper", > + DECL_ATTRIBUTES (func->decl))) != NULL_TREE) > + continue; > + > + ctf_dtdef_ref func_dtd = ggc_cleared_alloc<ctf_dtdef_t> (); > + func_dtd->dtd_data = dtd->dtd_data; > + func_dtd->dtd_data.ctti_type = dtd->dtd_type; > + func_dtd->ref_type = dtd; > + func_dtd->linkage = dtd->linkage; > + func_dtd->dtd_name = dtd->dtd_name; > + /* Type ID will be assigned just before output. */ > + > + /* Only the BTF_KIND_FUNC type actually references the name. The > + BTF_KIND_FUNC_PROTO is always anonymous. */ > + dtd->dtd_data.ctti_name = 0; > + > + /* Mark 'extern' funcs. */ > + if (DECL_EXTERNAL (func->decl)) > + func_dtd->linkage = BTF_FUNC_EXTERN; > + > + /* Buffer newly created FUNC records. We cannot simply insert them > + into the types map, because types are keyed by their DWARF DIE, > + and we have no unique DIE to use as a key since the FUNC_PROTOs > + are already present in the map. */ > + vec_safe_push (funcs, func_dtd); > + func_map->put (dtd, func_dtd); > } > } > > -/* Process and output all BTF data. Entry point of btfout. */ > +/* Initial entry point of BTF generation, called at early_finish () after > + CTF information has possibly been output. Translate all CTF information > + to BTF, and do any processing that must be done early, such as creating > + BTF_KIND_FUNC records. */ > > void > -btf_output (const char * filename) > +btf_early_finish (void) > { > ctf_container_ref tu_ctfc = ctf_get_tu_ctfc (); > > - init_btf_sections (); > - > - datasecs.create (0); > vec_alloc (funcs, 16); > + func_map = hash_map<ctf_dtdef_ref, ctf_dtdef_ref>::create_ggc (16); > + > + btf_early_add_const_void (tu_ctfc); > + btf_early_add_func_records (tu_ctfc); > +} > + > +/* Push a BTF datasec entry ENTRY into the datasec named SECNAME, > + creating the datasec record if it does not already exist. */ > + > +static void > +btf_datasec_push_entry (ctf_container_ref ctfc, const char *secname, > + struct btf_datasec_entry entry) > +{ > + if (secname == NULL) > + return; > + > + /* If we already have a datasec record for the appropriate section, > + append the new entry to it. */ > + for (size_t i = 0; i < datasecs.length (); i++) > + if (strcmp (datasecs[i].name, secname) == 0) > + { > + datasecs[i].entries.safe_push (entry); > + return; > + } > + > + /* If we don't already have a datasec record for secname, make one. */ > + uint32_t str_off; > + ctf_add_string (ctfc, secname, &str_off, CTF_AUX_STRTAB); > + if (strcmp (secname, "")) > + ctfc->ctfc_aux_strlen += strlen (secname) + 1; > > - ctf_add_cuname (tu_ctfc, filename); > + /* Note: ID will be assigned just before output. */ > + btf_datasec_t ds; > + ds.name = secname; > + ds.name_offset = str_off; > > - btf_emit_preprocess (tu_ctfc); > + /* Insert the entry into the new datasec record. */ > + ds.entries.create (1); > + ds.entries.quick_push (entry); > > - output_btf_header (tu_ctfc); > - output_btf_types (tu_ctfc); > - output_btf_vars (tu_ctfc); > - output_btf_func_types (tu_ctfc); > - output_btf_datasec_types (tu_ctfc); > - output_btf_strs (tu_ctfc); > + /* Insert the datasec record itself. */ > + datasecs.safe_push (ds); > } > > -/* Reset all state for BTF generation so that we can rerun the compiler within > - the same process. */ > +/* Create a datasec entry for a function, and insert it into the datasec > + record for the appropriate section. Create the record if it does not > + yet exist. */ > > -void > -btf_finalize (void) > +static void > +btf_datasec_add_func (ctf_container_ref ctfc, cgraph_node *func, > + ctf_dtdef_ref func_dtd) > { > - btf_info_section = NULL; > + const char *section_name = get_section_name (func); > > - /* Clear preprocessing state. */ > - num_vars_added = 0; > - num_types_added = 0; > - num_types_created = 0; > + /* Note: get_section_name () returns NULL for functions in text > + section. This is intentional, since we do not want to generate > + DATASEC entries for them. */ > + if (section_name == NULL) > + return; > > - holes.release (); > - voids.release (); > - for (size_t i = 0; i < datasecs.length (); i++) > - datasecs[i].entries.release (); > - datasecs.release (); > + struct btf_datasec_entry entry; > + gcc_assert (func_dtd); > + entry.dtd = func_dtd; > + entry.is_var = false; > > - funcs = NULL; > + /* Size is left as zero at compile time, to be filled in by loaders > + such as libbpf. */ > + entry.size = 0; > > - btf_var_ids->empty (); > - btf_var_ids = NULL; > + btf_datasec_push_entry (ctfc, section_name, entry); > +} > > - free (btf_id_map); > - btf_id_map = NULL; > +/* Create a datasec entry for a variable, and insert it into the datasec > + record for the appropriate section. Create the record if it does not > + yet exist. */ > > - ctf_container_ref tu_ctfc = ctf_get_tu_ctfc (); > - ctfc_delete_container (tu_ctfc); > - tu_ctfc = NULL; > +static void > +btf_datasec_add_var (ctf_container_ref ctfc, varpool_node *var, > + ctf_dvdef_ref dvd) > +{ > + /* PR112849: avoid assuming a section for extern decls without > + an explicit section, which would result in incorrectly > + emitting a BTF_KIND_DATASEC entry for them. */ > + if (DECL_EXTERNAL (var->decl) && var->get_section () == NULL) > + return; > + > + const char *section_name = get_section_name (var); > + if (section_name == NULL) > + return; > + > + gcc_assert (dvd); > + struct btf_datasec_entry entry; > + entry.dvd = dvd; > + entry.is_var = true; > + entry.size = 0; > + > + tree size = DECL_SIZE_UNIT (var->decl); > + if (tree_fits_uhwi_p (size)) > + entry.size = tree_to_uhwi (size); > + else if (VOID_TYPE_P (TREE_TYPE (var->decl))) > + entry.size = 1; > + > + btf_datasec_push_entry (ctfc, section_name, entry); > } > > -/* Initial entry point of BTF generation, called at early_finish () after > - CTF information has possibly been output. Translate all CTF information > - to BTF, and do any processing that must be done early, such as creating > - BTF_KIND_FUNC records. */ > +/* Add datasec entries for functions to CTFC. */ > > -void > -btf_early_finish (void) > +static void > +btf_late_add_func_datasec_entries (ctf_container_ref ctfc) > +{ > + /* We need to create FUNC records at early_finish, so that we have them > + even for functions which are later inlined by optimization passes. > + But on the other hand, we do not want datasec entries for such functions, > + so only create the datasec entries for them late. This loop will not > + hit functions which have already been inlined. */ > + cgraph_node *func; > + FOR_EACH_FUNCTION (func) > + { > + dw_die_ref die = lookup_decl_die (func->decl); > + if (die == NULL) > + continue; > + > + ctf_dtdef_ref dtd = ctf_dtd_lookup (ctfc, die); > + if (dtd == NULL) > + continue; > + > + ctf_dtdef_ref *pdtd = func_map->get (dtd); > + if (pdtd && DECL_EXTERNAL (func->decl)) > + btf_datasec_add_func (ctfc, func, *pdtd); > + } > +} > + > +/* Helper function used to determine whether or not a BTF_KIND_VAR record > + for the variable VAR shall be emitted. */ > + > +static bool > +btf_emit_variable_p (ctf_container_ref ctfc, varpool_node *var, > + ctf_dvdef_ref *pdvd) > +{ > + dw_die_ref die = lookup_decl_die (var->decl); > + if (die == NULL) > + return false; > + > + ctf_dvdef_ref dvd = ctf_dvd_lookup (ctfc, die); > + if (dvd == NULL) > + return false; > + > + /* If this is an extern variable declaration with a defining declaration > + later, skip it so that only the defining declaration is emitted. > + This is the same case, fix and reasoning as in CTF; see PR105089. */ > + if (ctf_dvd_ignore_lookup (ctfc, dvd->dvd_key)) > + return false; > + > + /* Skip variables with unrepresentable types. */ > + if (!btf_emit_type_p (dvd->dvd_type)) > + return false; > + > + *pdvd = dvd; > + return true; > +} > + > +/* Add BTF_KIND_VAR records for variables. */ > + > +static void > +btf_late_add_vars (ctf_container_ref ctfc) > +{ > + size_t num_ctf_vars = ctfc->ctfc_vars->elements (); > + > + ctfc->ctfc_vars_list = ggc_vec_alloc<ctf_dvdef_ref>(num_ctf_vars); > + > + varpool_node *var; > + ctf_dvdef_ref dvd; > + FOR_EACH_VARIABLE (var) > + { > + if (!btf_emit_variable_p (ctfc, var, &dvd)) > + continue; > + > + /* Mark 'extern' variables. */ > + if (DECL_EXTERNAL (var->decl)) > + dvd->dvd_visibility = BTF_VAR_GLOBAL_EXTERN; > + > + /* Add the variable to the vars list. */ > + ctfc->ctfc_vars_list[ctfc->ctfc_vars_list_count++] = dvd; > + > + /* Add a BTF_KIND_DATASEC entry for the variable. */ > + btf_datasec_add_var (ctfc, var, dvd); > + } > +} > + > +/* Callback used by btf_late_assign_type_ids to insert types into their initial > + positions in the type list. */ > + > +static int > +btf_type_list_cb (ctf_dtdef_ref *slot, ctf_container_ref ctfc) > +{ > + ctf_dtdef_ref dtd = *slot; > + ctfc->ctfc_types_list[dtd->dtd_type] = dtd; > + return 1; > +} > + > +/* Construct the initial type list and assign BTF IDs for all types translated > + from CTF. */ > + > +static void > +btf_late_collect_translated_types (ctf_container_ref ctfc) > +{ > + size_t num_ctf_types = ctfc->ctfc_types->elements (); > + > + /* First, place each type at its CTF-assigned index in the list. > + The '+1' here and below is to account for the implicit void type with > + ID 0. There is no real type at index 0 in the list. */ > + ctfc->ctfc_types_list = ggc_vec_alloc<ctf_dtdef_ref>(num_ctf_types + 1); > + ctfc->ctfc_types->traverse<ctf_container_ref, btf_type_list_cb> (ctfc); > + > + /* Now, pass through the list and adjust IDs to account for types which will > + not be emitted. This results in each type that will be emitted in BTF > + being assigned an appropriate ID. Note that types which will not be > + emitted remain in the list; they are skipped at output time. */ > + unsigned int skip = 0; > + for (size_t i = 1; i <= num_ctf_types; i++) > + { > + ctf_dtdef_ref dtd = ctfc->ctfc_types_list[i]; > + if (!btf_emit_type_p (dtd)) > + { > + dtd->dtd_type = BTF_INVALID_TYPEID; > + skip += 1; > + continue; > + } > + > + dtd->dtd_type -= skip; > + ctfc->ctfc_num_types++; > + ctfc->ctfc_num_vlen_bytes += btf_calc_num_vbytes (dtd); > + } > + > + max_translated_id = ctfc->ctfc_num_types; > + ctfc->ctfc_nextid = ctfc->ctfc_num_types + 1; > +} > + > +/* Assign BTF IDs for FUNC records and account for their size. */ > + > +static void > +btf_late_assign_func_ids (ctf_container_ref ctfc) > +{ > + ctf_dtdef_ref dtd; > + unsigned int i; > + FOR_EACH_VEC_ELT (*funcs, i, dtd) > + { > + dtd->dtd_type = ctfc->ctfc_nextid++; > + ctfc->ctfc_num_types++; > + } > +} > + > +/* Assign BTF IDs for variables and account for their size. */ > + > +static void > +btf_late_assign_var_ids (ctf_container_ref ctfc) > +{ > + for (size_t i = 0; i < ctfc->ctfc_vars_list_count; i++) > + { > + ctf_dvdef_ref dvd = ctfc->ctfc_vars_list[i]; > + ctf_id_t id = ctfc->ctfc_nextid++; > + gcc_assert (id <= BTF_MAX_TYPE); > + dvd->dvd_id = id; > + > + ctfc->ctfc_num_types++; > + ctfc->ctfc_num_vlen_bytes += sizeof (struct btf_var); > + } > +} > + > +/* Assign BTF IDs for datasec records and account for their size. */ > + > +static void > +btf_late_assign_datasec_ids (ctf_container_ref ctfc) > { > - btf_init_postprocess (); > + for (size_t i = 0; i < datasecs.length (); i++) > + { > + datasecs[i].id = ctfc->ctfc_nextid++; > + datasecs[i].name_offset += ctfc_get_strtab_len (ctfc, CTF_STRTAB); > + ctfc->ctfc_num_types++; > + ctfc->ctfc_num_vlen_bytes += (datasecs[i].entries.length () > + * sizeof (struct btf_var_secinfo)); > + } > } > > /* Late entry point for BTF generation, called from dwarf2out_finish (). > Complete and emit BTF information. */ > > void > -btf_finish (const char * filename) > +btf_finish (void) > { > - btf_output (filename); > + ctf_container_ref tu_ctfc = ctf_get_tu_ctfc (); > + init_btf_sections (); > + > + datasecs.create (0); > + > + tu_ctfc->ctfc_num_types = 0; > + tu_ctfc->ctfc_num_vlen_bytes = 0; > + tu_ctfc->ctfc_vars_list_count = 0; > + > + btf_late_add_vars (tu_ctfc); > + btf_late_collect_translated_types (tu_ctfc); > + btf_late_add_func_datasec_entries (tu_ctfc); > + btf_late_assign_var_ids (tu_ctfc); > + btf_late_assign_func_ids (tu_ctfc); > + btf_late_assign_datasec_ids (tu_ctfc); > + > + /* Finally, write out the complete .BTF section. */ > + btf_output (tu_ctfc); > > /* If compiling for BPF with CO-RE info, we cannot deallocate until after > CO-RE information is created, which happens very late in BPF backend. > @@ -1521,6 +1290,27 @@ btf_finish (const char * filename) > btf_finalize (); > } > > +/* Reset all state for BTF generation so that we can rerun the compiler within > + the same process. */ > + > +void > +btf_finalize (void) > +{ > + btf_info_section = NULL; > + max_translated_id = 0; > + > + for (size_t i = 0; i < datasecs.length (); i++) > + datasecs[i].entries.release (); > + datasecs.release (); > + > + funcs = NULL; > + func_map->empty (); > + func_map = NULL; > + > + ctf_container_ref tu_ctfc = ctf_get_tu_ctfc (); > + ctfc_delete_container (tu_ctfc); > + tu_ctfc = NULL; > +} > > /* Traversal function for all BTF_KIND_FUNC type records. */ > > diff --git a/gcc/ctfc.h b/gcc/ctfc.h > index cfc805db7b5..90421c72c09 100644 > --- a/gcc/ctfc.h > +++ b/gcc/ctfc.h > @@ -162,10 +162,14 @@ struct GTY ((for_user)) ctf_dtdef > ctf_id_t dtd_type; /* Type identifier for this definition. */ > struct ctf_dtdef *ref_type; /* Type referred to by this type (if any). */ > ctf_itype_t dtd_data; /* Type node. */ > - bool from_global_func; /* Whether this type was added from a global > - function. */ > uint32_t linkage; /* Used in function types. 0=local, 1=global. */ > - bool dtd_enum_unsigned; /* Enum signedness. */ > + > + /* Whether this type was added from a global function. */ > + BOOL_BITFIELD from_global_func : 1; > + /* Enum signedness. */ > + BOOL_BITFIELD dtd_enum_unsigned : 1; > + /* Lots of spare bits. */ > + > union GTY ((desc ("ctf_dtu_d_union_selector (&%1)"))) > { > /* struct, union, or enum. */ > @@ -192,6 +196,7 @@ struct GTY ((for_user)) ctf_dvdef > uint32_t dvd_name_offset; /* Offset of the name in str table. */ > unsigned int dvd_visibility; /* External visibility. 0=static,1=global. */ > struct ctf_dtdef * dvd_type; /* Type of variable. */ > + ctf_id_t dvd_id; /* ID of this variable. Only used for BTF. */ > }; > > typedef struct ctf_dvdef ctf_dvdef_t; > @@ -388,7 +393,7 @@ extern void ctf_output (const char * filename); > extern void ctf_finalize (void); > > extern void btf_early_finish (void); > -extern void btf_finish (const char * filename); > +extern void btf_finish (void); > extern void btf_finalize (void); > > extern ctf_container_ref ctf_get_tu_ctfc (void); > @@ -442,7 +447,9 @@ extern int ctf_add_variable (ctf_container_ref, const char *, ctf_dtdef_ref, > dw_die_ref, unsigned int, dw_die_ref); > > extern ctf_dtdef_ref ctf_lookup_tree_type (ctf_container_ref, const tree); > -extern ctf_id_t get_btf_id (ctf_id_t); > + > +/* Callback and traversal function for BTF_KIND_FUNC records. Used by BPF > + target for BPF CO-RE implementation. */ > > typedef bool (*funcs_traverse_callback) (ctf_dtdef_ref, void *); > bool traverse_btf_func_types (funcs_traverse_callback, void *); > diff --git a/gcc/dwarf2ctf.cc b/gcc/dwarf2ctf.cc > index f16b5ceee74..ac195c1a862 100644 > --- a/gcc/dwarf2ctf.cc > +++ b/gcc/dwarf2ctf.cc > @@ -980,13 +980,13 @@ ctf_debug_early_finish (const char * filename) > /* Finish CTF/BTF debug info emission. */ > > void > -ctf_debug_finish (const char * filename) > +ctf_debug_finish () > { > /* Emit BTF debug info here when CO-RE relocations need to be generated. > BTF with CO-RE relocations needs to be generated when CO-RE is in effect > for the BPF target. */ > if (btf_debuginfo_p ()) > - btf_finish (filename); > + btf_finish (); > } > > #include "gt-dwarf2ctf.h" > diff --git a/gcc/dwarf2ctf.h b/gcc/dwarf2ctf.h > index 46184325bae..f8a181a9762 100644 > --- a/gcc/dwarf2ctf.h > +++ b/gcc/dwarf2ctf.h > @@ -32,7 +32,7 @@ extern void ctf_debug_init (void); > extern void ctf_debug_init_postprocess (bool); > extern bool ctf_do_die (dw_die_ref); > extern void ctf_debug_early_finish (const char *); > -extern void ctf_debug_finish (const char *); > +extern void ctf_debug_finish (void); > > /* Wrappers for CTF/BTF to fetch information from GCC DWARF DIE. Used in > ctfc.cc. > diff --git a/gcc/dwarf2out.cc b/gcc/dwarf2out.cc > index 1664934ccc3..794b94ca80b 100644 > --- a/gcc/dwarf2out.cc > +++ b/gcc/dwarf2out.cc > @@ -32276,7 +32276,7 @@ dwarf2out_finish (const char *filename) > /* Generate CTF/BTF debug info. */ > if ((ctf_debug_info_level > CTFINFO_LEVEL_NONE > || btf_debuginfo_p ()) && lang_GNU_C ()) > - ctf_debug_finish (filename); > + ctf_debug_finish (); > > /* Skip emitting DWARF if not required. */ > if (!dwarf_debuginfo_p ())
diff --git a/gcc/btfout.cc b/gcc/btfout.cc index 14a503a4f80..0af0bd39fc7 100644 --- a/gcc/btfout.cc +++ b/gcc/btfout.cc @@ -68,53 +68,44 @@ static char btf_info_section_label[MAX_BTF_LABEL_BYTES]; #define BTF_INVALID_TYPEID 0xFFFFFFFF -/* Mapping of CTF variables to the IDs they will be assigned when they are - converted to BTF_KIND_VAR type records. Strictly accounts for the index - from the start of the variable type entries, does not include the number - of types emitted prior to the variable records. */ -static GTY (()) hash_map <ctf_dvdef_ref, unsigned> *btf_var_ids; - -/* Mapping of type IDs from original CTF ID to BTF ID. Types do not map - 1-to-1 from CTF to BTF. To avoid polluting the CTF container when updating - type references-by-ID, we use this map instead. */ -static ctf_id_t * btf_id_map = NULL; - -/* Information for creating the BTF_KIND_DATASEC records. */ +/* Internal representation of an entry in a BTF_KIND_DATASEC record. */ +struct btf_datasec_entry +{ + union { + ctf_dvdef_ref dvd; /* Reference to the underlying variable represented. */ + ctf_dtdef_ref dtd; /* Reference to the underlying type represented. */ + }; + bool is_var; /* True iff this entry represents a variable. */ + uint32_t size; /* Size of variable or function, in bytes. + For functions, always zero at compile time. */ +}; + +/* Internal representation of a BTF_KIND_DATASEC record. */ typedef struct btf_datasec { - const char *name; /* Section name, e.g. ".bss". */ - uint32_t name_offset; /* Offset to name in string table. */ - vec<struct btf_var_secinfo> entries; /* Variable entries in this section. */ + ctf_id_t id; /* BTF type ID of this record. */ + const char *name; /* Section name, e.g. ".bss". */ + uint32_t name_offset; /* Offset to name in string table. */ + vec<struct btf_datasec_entry> entries; /* Entries in this section. */ } btf_datasec_t; /* One BTF_KIND_DATASEC record is created for each output data section which will hold at least one variable. */ static vec<btf_datasec_t> datasecs; -/* Holes occur for types which are present in the CTF container, but are either - non-representable or redundant in BTF. */ -static vec<ctf_id_t> holes; - -/* CTF definition(s) of void. Only one definition of void should be generated. - We should not encounter more than one definition of void, but use a vector - to be safe. */ -static vec<ctf_id_t> voids; - /* Functions in BTF have two separate type records - one for the prototype (BTF_KIND_FUNC_PROTO), as well as a BTF_KIND_FUNC. CTF_K_FUNCTION types map closely to BTF_KIND_FUNC_PROTO, but the BTF_KIND_FUNC records must be created. This vector holds them. */ static GTY (()) vec<ctf_dtdef_ref, va_gc> *funcs; -/* The number of BTF variables added to the TU CTF container. */ -static unsigned int num_vars_added = 0; - -/* The number of BTF types added to the TU CTF container. */ -static unsigned int num_types_added = 0; +/* Maps BTF_KIND_FUNC_PROTO to the BTF_KIND_FUNC record for it. Used when + creating DATASEC entries. */ +static GTY (()) hash_map<ctf_dtdef_ref, ctf_dtdef_ref> *func_map; -/* The number of types synthesized for BTF that do not correspond to - CTF types. */ -static unsigned int num_types_created = 0; +/* Highest BTF ID assigned to any regular type translated from CTF. + Does not include BTF_KIND_{VAR,FUNC,DATASEC} types. */ +static ctf_id_t max_translated_id = 0; /* Name strings for BTF kinds. Note: the indices here must match the type defines in btf.h. */ @@ -160,6 +151,16 @@ get_btf_kind (uint32_t ctf_kind) return BTF_KIND_UNKN; } +/* Convenience wrapper around get_btf_kind for the common case. */ + +static uint32_t +btf_dtd_kind (ctf_dtdef_ref dtd) +{ + if (!dtd) + return BTF_KIND_UNKN; + return get_btf_kind (CTF_V2_INFO_KIND (dtd->dtd_data.ctti_info)); +} + /* Some BTF types, like BTF_KIND_FUNC_PROTO, are anonymous. The machinery in btfout to emit BTF, may reset dtd_data->ctti_name, but does not update the name in the ctf_dtdef_ref type object (deliberate choice). This @@ -173,101 +174,20 @@ get_btf_type_name (ctf_dtdef_ref dtd) return (dtd->dtd_data.ctti_name) ? dtd->dtd_name : anon; } -/* Helper routines to map between 'relative' and 'absolute' IDs. - - In BTF all records (including variables) are output in one long list, and all - inter-type references are via index into that list. But internally since we - a) translate from CTF, which separates variable records from regular types - and b) create some additional types after the fact, things like VAR and FUNC - records are stored in separate vectors with their own indices. These - functions map between the 'relative' IDs (i.e. indices in their respective - containers) and 'absolute' IDs (i.e. indices in the final contiguous - output list), which goes in order: - all normal type records translated from CTF - all BTF_KIND_VAR records - all BTF_KIND_FUNC records (synthesized split function records) - all BTF_KIND_DATASEC records (synthesized) - - The extra '+ 1's below are to account for the implicit "void" record, which - has index 0 but isn't actually contained in the type list. */ - -/* Return the final BTF ID of the variable at relative index REL. */ - -static ctf_id_t -btf_absolute_var_id (ctf_id_t rel) -{ - return rel + (num_types_added + 1); -} - -/* Return the relative index of the variable with final BTF ID ABS. */ - -static ctf_id_t -btf_relative_var_id (ctf_id_t abs) -{ - return abs - (num_types_added + 1); -} - -/* Return the final BTF ID of the func record at relative index REL. */ - -static ctf_id_t -btf_absolute_func_id (ctf_id_t rel) -{ - return rel + (num_types_added + 1) + num_vars_added; -} - -/* Return the relative index of the func record with final BTF ID ABS. */ - -static ctf_id_t -btf_relative_func_id (ctf_id_t abs) -{ - return abs - ((num_types_added + 1) + num_vars_added); -} - -/* Return the final BTF ID of the datasec record at relative index REL. */ - -static ctf_id_t -btf_absolute_datasec_id (ctf_id_t rel) +static bool +btf_emit_type_p (ctf_dtdef_ref dtd) { - return rel + (num_types_added + 1) + num_vars_added + funcs->length (); -} - + uint32_t kind = btf_dtd_kind (dtd); -/* Allocate the btf_id_map, and initialize elements to BTF_INVALID_TYPEID. */ + if (kind == BTF_KIND_UNKN) + /* This type is not representable in BTF. */ + return false; -static void -init_btf_id_map (size_t len) -{ - btf_id_map = XNEWVEC (ctf_id_t, len); + if (kind == BTF_KIND_INT && dtd->dtd_data.ctti_size == 0) + /* This is a (redundant) definition of void. */ + return false; - btf_id_map[0] = BTF_VOID_TYPEID; - for (size_t i = 1; i < len; i++) - btf_id_map[i] = BTF_INVALID_TYPEID; -} - -/* Return the BTF type ID of CTF type ID KEY, or BTF_INVALID_TYPEID if the CTF - type with ID KEY does not map to a BTF type. */ - -ctf_id_t -get_btf_id (ctf_id_t key) -{ - return btf_id_map[key]; -} - -/* Set the CTF type ID KEY to map to BTF type ID VAL. */ - -static inline void -set_btf_id (ctf_id_t key, ctf_id_t val) -{ - btf_id_map[key] = val; -} - -/* Return TRUE iff the given CTF type ID maps to a BTF type which will - be emitted. */ -static inline bool -btf_emit_id_p (ctf_id_t id) -{ - return ((btf_id_map[id] != BTF_VOID_TYPEID) - && (btf_id_map[id] <= BTF_MAX_TYPE)); + return true; } /* Return true if DTD is a forward-declared enum. The BTF representation @@ -276,9 +196,8 @@ btf_emit_id_p (ctf_id_t id) static bool btf_fwd_to_enum_p (ctf_dtdef_ref dtd) { - uint32_t btf_kind = get_btf_kind (CTF_V2_INFO_KIND (dtd->dtd_data.ctti_info)); - - return (btf_kind == BTF_KIND_FWD && dtd->dtd_data.ctti_type == CTF_K_ENUM); + uint32_t kind = btf_dtd_kind (dtd); + return (kind == BTF_KIND_FWD && dtd->dtd_data.ctti_type == CTF_K_ENUM); } /* Each BTF type can be followed additional, variable-length information @@ -290,7 +209,7 @@ btf_calc_num_vbytes (ctf_dtdef_ref dtd) { uint64_t vlen_bytes = 0; - uint32_t kind = get_btf_kind (CTF_V2_INFO_KIND (dtd->dtd_data.ctti_info)); + uint32_t kind = btf_dtd_kind (dtd); uint32_t vlen = CTF_V2_INFO_VLEN (dtd->dtd_data.ctti_info); switch (kind) @@ -360,41 +279,6 @@ init_btf_sections (void) BTF_INFO_SECTION_LABEL, btf_label_num++); } -/* Push a BTF datasec variable entry INFO into the datasec named SECNAME, - creating the datasec if it does not already exist. */ - -static void -btf_datasec_push_entry (ctf_container_ref ctfc, const char *secname, - struct btf_var_secinfo info) -{ - if (secname == NULL) - return; - - for (size_t i = 0; i < datasecs.length (); i++) - if (strcmp (datasecs[i].name, secname) == 0) - { - datasecs[i].entries.safe_push (info); - return; - } - - /* If we don't already have a datasec record for secname, make one. */ - - uint32_t str_off; - ctf_add_string (ctfc, secname, &str_off, CTF_AUX_STRTAB); - if (strcmp (secname, "")) - ctfc->ctfc_aux_strlen += strlen (secname) + 1; - - btf_datasec_t ds; - ds.name = secname; - ds.name_offset = str_off; - - ds.entries.create (0); - ds.entries.safe_push (info); - - datasecs.safe_push (ds); -} - - /* Return the section name, as of interest to btf_collect_datasec, for the given symtab node. Note that this deliberately returns NULL for objects which do not go in a section btf_collect_datasec cares about. */ @@ -423,301 +307,15 @@ get_section_name (symtab_node *node) return section_name; } -/* Construct all BTF_KIND_DATASEC records for CTFC. One such record is created - for each non-empty data-containing section in the output. Each record is - followed by a variable number of entries describing the variables stored - in that section. */ - -static void -btf_collect_datasec (ctf_container_ref ctfc) -{ - cgraph_node *func; - FOR_EACH_FUNCTION (func) - { - dw_die_ref die = lookup_decl_die (func->decl); - if (die == NULL) - continue; - - ctf_dtdef_ref dtd = ctf_dtd_lookup (ctfc, die); - if (dtd == NULL) - continue; - - if (DECL_EXTERNAL (func->decl) - && (lookup_attribute ("kernel_helper", - DECL_ATTRIBUTES (func->decl))) != NULL_TREE) - continue; - - /* Functions actually get two types: a BTF_KIND_FUNC_PROTO, and - also a BTF_KIND_FUNC. But the CTF container only allocates one - type per function, which matches closely with BTF_KIND_FUNC_PROTO. - For each such function, also allocate a BTF_KIND_FUNC entry. - These will be output later. */ - ctf_dtdef_ref func_dtd = ggc_cleared_alloc<ctf_dtdef_t> (); - func_dtd->dtd_data = dtd->dtd_data; - func_dtd->dtd_data.ctti_type = dtd->dtd_type; - func_dtd->linkage = dtd->linkage; - func_dtd->dtd_name = dtd->dtd_name; - /* +1 for the sentinel type not in the types map. */ - func_dtd->dtd_type = num_types_added + num_types_created + 1; - - /* Only the BTF_KIND_FUNC type actually references the name. The - BTF_KIND_FUNC_PROTO is always anonymous. */ - dtd->dtd_data.ctti_name = 0; - - vec_safe_push (funcs, func_dtd); - num_types_created++; - - /* Mark any 'extern' funcs and add DATASEC entries for them. */ - if (DECL_EXTERNAL (func->decl)) - { - func_dtd->linkage = BTF_FUNC_EXTERN; - - const char *section_name = get_section_name (func); - /* Note: get_section_name () returns NULL for functions in text - section. This is intentional, since we do not want to generate - DATASEC entries for them. */ - if (section_name == NULL) - continue; - - struct btf_var_secinfo info; - - info.type = func_dtd->dtd_type; - - /* Both zero at compile time. */ - info.size = 0; - info.offset = 0; - - btf_datasec_push_entry (ctfc, section_name, info); - } - } - - varpool_node *node; - FOR_EACH_VARIABLE (node) - { - dw_die_ref die = lookup_decl_die (node->decl); - if (die == NULL) - continue; - - ctf_dvdef_ref dvd = ctf_dvd_lookup (ctfc, die); - if (dvd == NULL) - continue; - - /* Mark extern variables. */ - if (DECL_EXTERNAL (node->decl)) - { - dvd->dvd_visibility = BTF_VAR_GLOBAL_EXTERN; - - /* PR112849: avoid assuming a section for extern decls without - an explicit section, which would result in incorrectly - emitting a BTF_KIND_DATASEC entry for them. */ - if (node->get_section () == NULL) - continue; - } - - const char *section_name = get_section_name (node); - if (section_name == NULL) - continue; - - struct btf_var_secinfo info; - - info.type = 0; - unsigned int *var_id = btf_var_ids->get (dvd); - if (var_id) - info.type = btf_absolute_var_id (*var_id); - else - continue; - - info.size = 0; - tree size = DECL_SIZE_UNIT (node->decl); - if (tree_fits_uhwi_p (size)) - info.size = tree_to_uhwi (size); - else if (VOID_TYPE_P (TREE_TYPE (node->decl))) - info.size = 1; - - /* Offset is left as 0 at compile time, to be filled in by loaders such - as libbpf. */ - info.offset = 0; - - btf_datasec_push_entry (ctfc, section_name, info); - } - - num_types_created += datasecs.length (); -} - -/* Return true if the type ID is that of a type which will not be emitted (for - example, if it is not representable in BTF). */ - -static bool -btf_removed_type_p (ctf_id_t id) -{ - return holes.contains (id); -} - -/* Adjust the given type ID to account for holes and duplicate definitions of - void. */ - -static ctf_id_t -btf_adjust_type_id (ctf_id_t id) -{ - size_t n; - ctf_id_t i = 0; - - /* Do not adjust invalid type markers. */ - if (id == BTF_INVALID_TYPEID) - return id; - - for (n = 0; n < voids.length (); n++) - if (id == voids[n]) - return BTF_VOID_TYPEID; - - for (n = 0; n < holes.length (); n++) - { - if (holes[n] < id) - i++; - else if (holes[n] == id) - return BTF_VOID_TYPEID; - } - - return id - i; -} - -/* Postprocessing callback routine for types. */ - -int -btf_dtd_postprocess_cb (ctf_dtdef_ref *slot, ctf_container_ref arg_ctfc) -{ - ctf_dtdef_ref ctftype = (ctf_dtdef_ref) * slot; - - size_t index = ctftype->dtd_type; - gcc_assert (index <= arg_ctfc->ctfc_types->elements ()); - - uint32_t ctf_kind, btf_kind; - - ctf_kind = CTF_V2_INFO_KIND (ctftype->dtd_data.ctti_info); - btf_kind = get_btf_kind (ctf_kind); - - if (btf_kind == BTF_KIND_UNKN) - /* This type is not representable in BTF. Create a hole. */ - holes.safe_push (ctftype->dtd_type); - - else if (btf_kind == BTF_KIND_INT && ctftype->dtd_data.ctti_size == 0) - { - /* This is a (redundant) definition of void. */ - voids.safe_push (ctftype->dtd_type); - holes.safe_push (ctftype->dtd_type); - } - - arg_ctfc->ctfc_types_list[index] = ctftype; - - return 1; -} - -/* Preprocessing callback routine for variables. */ - -int -btf_dvd_emit_preprocess_cb (ctf_dvdef_ref *slot, ctf_container_ref arg_ctfc) -{ - ctf_dvdef_ref var = (ctf_dvdef_ref) * slot; - - /* If this is an extern variable declaration with a defining declaration - later, skip it so that only the defining declaration is emitted. - This is the same case, fix and reasoning as in CTF; see PR105089. */ - if (ctf_dvd_ignore_lookup (arg_ctfc, var->dvd_key)) - return 1; - - /* Do not add variables which refer to unsupported types. */ - if (!voids.contains (var->dvd_type->dtd_type) - && btf_removed_type_p (var->dvd_type->dtd_type)) - return 1; - - arg_ctfc->ctfc_vars_list[num_vars_added] = var; - btf_var_ids->put (var, num_vars_added); - - num_vars_added++; - num_types_created++; - - return 1; -} - -/* Preprocessing callback routine for types. */ - -static void -btf_dtd_emit_preprocess_cb (ctf_container_ref ctfc, ctf_dtdef_ref dtd) -{ - if (!btf_emit_id_p (dtd->dtd_type)) - return; - - ctfc->ctfc_num_vlen_bytes += btf_calc_num_vbytes (dtd); -} - -/* Preprocess the CTF information to prepare for BTF output. BTF is almost a - subset of CTF, with many small differences in encoding, and lacking support - for some types (notably floating point formats). - - During the preprocessing pass: - - Ascertain that the sorted list of types has been prepared. For the BTF - generation process, this is taken care of by the btf_init_postprocess (). - - - BTF_KIND_FUNC and BTF_KIND_DATASEC records are constructed. These types do - not have analogues in CTF (the analogous type to CTF_K_FUNCTION is - BTF_KIND_FUNC_PROTO), but can be relatively easily deduced from CTF - information. - - - Construct BTF_KIND_VAR records, representing variables. - - - Calculate the total size in bytes of variable-length information following - BTF type records. This is used for outputting the BTF header. - - After preprocessing, all BTF information is ready to be output: - - ctfc->ctfc_types_list holdstypes converted from CTF types. This does not - include KIND_VAR, KIND_FUNC, nor KIND_DATASEC types. These types have been - re-encoded to the appropriate representation in BTF. - - ctfc->ctfc_vars_list holds all variables which should be output. - Variables of unsupported types are not present in this list. - - Vector 'funcs' holds all BTF_KIND_FUNC types, one to match each - BTF_KIND_FUNC_PROTO. - - Vector 'datasecs' holds all BTF_KIND_DATASEC types. */ - -static void -btf_emit_preprocess (ctf_container_ref ctfc) -{ - size_t num_ctf_types = ctfc->ctfc_types->elements (); - size_t num_ctf_vars = ctfc->ctfc_vars->elements (); - size_t i; - - if (num_ctf_types) - { - gcc_assert (ctfc->ctfc_types_list); - /* Preprocess the types. */ - for (i = 1; i <= num_ctf_types; i++) - btf_dtd_emit_preprocess_cb (ctfc, ctfc->ctfc_types_list[i]); - } - - btf_var_ids = hash_map<ctf_dvdef_ref, unsigned int>::create_ggc (100); - - if (num_ctf_vars) - { - /* Allocate and construct the list of variables. While BTF variables are - not distinct from types (in that variables are simply types with - BTF_KIND_VAR), it is simpler to maintain a separate list of variables - and append them to the types list during output. */ - ctfc->ctfc_vars_list = ggc_vec_alloc<ctf_dvdef_ref>(num_ctf_vars); - ctfc->ctfc_vars->traverse<ctf_container_ref, btf_dvd_emit_preprocess_cb> - (ctfc); - - ctfc->ctfc_num_vlen_bytes += (num_vars_added * sizeof (struct btf_var)); - } - - btf_collect_datasec (ctfc); -} - /* Return true iff DMD is a member description of a bit-field which can be validly represented in BTF. */ static bool -btf_dmd_representable_bitfield_p (ctf_container_ref ctfc, ctf_dmdef_t *dmd) +btf_dmd_representable_bitfield_p (ctf_dmdef_t *dmd) { - ctf_dtdef_ref ref_type = ctfc->ctfc_types_list[dmd->dmd_type->dtd_type]; + ctf_dtdef_ref ref_type = dmd->dmd_type; + if (!ref_type) + return false; if (CTF_V2_INFO_KIND (ref_type->dtd_data.ctti_info) == CTF_K_SLICE) { @@ -739,76 +337,34 @@ btf_dmd_representable_bitfield_p (ctf_container_ref ctfc, ctf_dmdef_t *dmd) /* Asm'out a reference to another BTF type. */ static void -btf_asm_type_ref (const char *prefix, ctf_container_ref ctfc, ctf_id_t ctf_id) +btf_asm_type_ref (const char *prefix, ctf_dtdef_ref dtd) { - ctf_id_t btf_id = get_btf_id (ctf_id); - if (btf_id == BTF_VOID_TYPEID || btf_id == BTF_INVALID_TYPEID) - { - /* There is no explicit void type. - Also handle any invalid refs that made it this far, just in case. */ - dw2_asm_output_data (4, btf_id, "%s: void", prefix); - } + if (!dtd || !btf_emit_type_p (dtd)) + dw2_asm_output_data (4, BTF_VOID_TYPEID, "%s: void", prefix); else { - gcc_assert (btf_id <= num_types_added); - - /* Ref to a standard type in the types list. Note: take care that we - must index the type list by the original CTF id, not the BTF id. */ - ctf_dtdef_ref ref_type = ctfc->ctfc_types_list[ctf_id]; - uint32_t ref_kind - = get_btf_kind (CTF_V2_INFO_KIND (ref_type->dtd_data.ctti_info)); - - const char *kind_name = btf_fwd_to_enum_p (ref_type) - ? btf_kind_name (BTF_KIND_ENUM) - : btf_kind_name (ref_kind); - - dw2_asm_output_data (4, btf_id, "%s: (BTF_KIND_%s '%s')", - prefix, kind_name, - get_btf_type_name (ref_type)); + uint32_t kind = btf_dtd_kind (dtd); + if (btf_fwd_to_enum_p (dtd)) + kind = BTF_KIND_ENUM; + else if (kind == BTF_KIND_FUNC_PROTO && dtd->dtd_type > max_translated_id) + kind = BTF_KIND_FUNC; + + dw2_asm_output_data (4, dtd->dtd_type, "%s: (BTF_KIND_%s '%s')", + prefix, btf_kind_name (kind), + get_btf_type_name (dtd)); } } -/* Asm'out a reference to a BTF_KIND_VAR or BTF_KIND_FUNC type. These type - kinds are BTF-specific, and should only be referred to by entries in - BTF_KIND_DATASEC records. */ - -static void -btf_asm_datasec_type_ref (const char *prefix, ctf_container_ref ctfc, - ctf_id_t btf_id) -{ - if (btf_id >= num_types_added + 1 - && btf_id < num_types_added + num_vars_added + 1) - { - /* Ref to a variable. Should only appear in DATASEC entries. */ - ctf_id_t var_id = btf_relative_var_id (btf_id); - ctf_dvdef_ref dvd = ctfc->ctfc_vars_list[var_id]; - dw2_asm_output_data (4, btf_id, "%s: (BTF_KIND_VAR '%s')", - prefix, dvd->dvd_name); - - } - else if (btf_id >= num_types_added + num_vars_added + 1) - { - /* Ref to a FUNC record. */ - size_t func_id = btf_relative_func_id (btf_id); - ctf_dtdef_ref ref_type = (*funcs)[func_id]; - dw2_asm_output_data (4, btf_id, "%s: (BTF_KIND_FUNC '%s')", - prefix, get_btf_type_name (ref_type)); - } - else - /* The caller should not be calling this. */ - gcc_unreachable (); -} - /* Asm'out a BTF type. This routine is responsible for the bulk of the task of converting CTF types to their BTF representation. */ static void -btf_asm_type (ctf_container_ref ctfc, ctf_dtdef_ref dtd) +btf_asm_type (ctf_dtdef_ref dtd) { uint32_t btf_kind, btf_kflag, btf_vlen, btf_size; uint32_t ctf_info = dtd->dtd_data.ctti_info; - btf_kind = get_btf_kind (CTF_V2_INFO_KIND (ctf_info)); + btf_kind = btf_dtd_kind (dtd); btf_size = dtd->dtd_data.ctti_size; btf_vlen = CTF_V2_INFO_VLEN (ctf_info); @@ -827,17 +383,17 @@ btf_asm_type (ctf_container_ref ctfc, ctf_dtdef_ref dtd) if (btf_kind == BTF_KIND_STRUCT || btf_kind == BTF_KIND_UNION) { - /* If a struct/union has ANY bitfield members, set kflag=1. - Note that we must also change the encoding of every member to encode - both member bitfield size (stealing most-significant 8 bits) and bit - offset (LS 24 bits). This is done during preprocessing. */ + /* If a struct/union has ANY bitfield members, set kflag=1. */ ctf_dmdef_t *dmd; for (dmd = dtd->dtd_u.dtu_members; dmd != NULL; dmd = (ctf_dmdef_t *) ctf_dmd_list_next (dmd)) { /* Set kflag if this member is a representable bitfield. */ - if (btf_dmd_representable_bitfield_p (ctfc, dmd)) - btf_kflag = 1; + if (btf_dmd_representable_bitfield_p (dmd)) + { + btf_kflag = 1; + break; + } } } @@ -871,7 +427,7 @@ btf_asm_type (ctf_container_ref ctfc, ctf_dtdef_ref dtd) : BTF_KF_ENUM_SIGNED; if (dtd->dtd_data.ctti_size == 0x8) btf_kind = BTF_KIND_ENUM64; - } + } /* PR debug/112656. BTF_KIND_FUNC_PROTO is always anonymous. */ else if (btf_kind == BTF_KIND_FUNC_PROTO) @@ -879,7 +435,7 @@ btf_asm_type (ctf_container_ref ctfc, ctf_dtdef_ref dtd) dw2_asm_output_data (4, dtd->dtd_data.ctti_name, "TYPE %" PRIu64 " BTF_KIND_%s '%s'", - get_btf_id (dtd->dtd_type), btf_kind_name (btf_kind), + dtd->dtd_type, btf_kind_name (btf_kind), get_btf_type_name (dtd)); dw2_asm_output_data (4, BTF_TYPE_INFO (btf_kind, btf_kflag, btf_vlen), "btt_info: kind=%u, kflag=%u, vlen=%u", @@ -905,30 +461,29 @@ btf_asm_type (ctf_container_ref ctfc, ctf_dtdef_ref dtd) break; } - ctf_id_t ref_id = dtd->dtd_data.ctti_type; - btf_asm_type_ref ("btt_type", ctfc, ref_id); + btf_asm_type_ref ("btt_type", dtd->ref_type); } /* Asm'out the variable information following a BTF_KIND_ARRAY. */ static void -btf_asm_array (ctf_container_ref ctfc, ctf_arinfo_t arr) +btf_asm_array (ctf_arinfo_t arr) { - btf_asm_type_ref ("bta_elem_type", ctfc, arr.ctr_contents->dtd_type); - btf_asm_type_ref ("bta_index_type", ctfc, arr.ctr_index->dtd_type); + btf_asm_type_ref ("bta_elem_type", arr.ctr_contents); + btf_asm_type_ref ("bta_index_type", arr.ctr_index); dw2_asm_output_data (4, arr.ctr_nelems, "bta_nelems"); } /* Asm'out a BTF_KIND_VAR. */ static void -btf_asm_varent (ctf_container_ref ctfc, ctf_dvdef_ref var) +btf_asm_varent (ctf_dvdef_ref var) { - dw2_asm_output_data (4, var->dvd_name_offset, "TYPE %u BTF_KIND_VAR '%s'", - (*(btf_var_ids->get (var)) + num_types_added + 1), - var->dvd_name); + dw2_asm_output_data (4, var->dvd_name_offset, + "TYPE %" PRIu64 " BTF_KIND_VAR '%s'", + var->dvd_id, var->dvd_name); dw2_asm_output_data (4, BTF_TYPE_INFO (BTF_KIND_VAR, 0, 0), "btv_info"); - btf_asm_type_ref ("btv_type", ctfc, var->dvd_type->dtd_type); + btf_asm_type_ref ("btv_type", var->dvd_type); dw2_asm_output_data (4, var->dvd_visibility, "btv_linkage"); } @@ -936,23 +491,22 @@ btf_asm_varent (ctf_container_ref ctfc, ctf_dvdef_ref var) BTF_KIND_UNION. */ static void -btf_asm_sou_member (ctf_container_ref ctfc, ctf_dmdef_t * dmd, unsigned int idx) +btf_asm_sou_member (ctf_dmdef_t * dmd, unsigned int idx) { - ctf_dtdef_ref ref_type = ctfc->ctfc_types_list[dmd->dmd_type->dtd_type]; - ctf_id_t base_type = dmd->dmd_type->dtd_type; + ctf_dtdef_ref base_type = dmd->dmd_type; uint64_t sou_offset = dmd->dmd_offset; dw2_asm_output_data (4, dmd->dmd_name_offset, "MEMBER '%s' idx=%u", dmd->dmd_name, idx); - /* Re-encode bitfields to BTF representation. */ - if (CTF_V2_INFO_KIND (ref_type->dtd_data.ctti_info) == CTF_K_SLICE) + if (base_type + && CTF_V2_INFO_KIND (base_type->dtd_data.ctti_info) == CTF_K_SLICE) { - if (btf_dmd_representable_bitfield_p (ctfc, dmd)) + if (btf_dmd_representable_bitfield_p (dmd)) { - unsigned short word_offset = ref_type->dtd_u.dtu_slice.cts_offset; - unsigned short bits = ref_type->dtd_u.dtu_slice.cts_bits; + unsigned short word_offset = base_type->dtd_u.dtu_slice.cts_offset; + unsigned short bits = base_type->dtd_u.dtu_slice.cts_bits; /* Pack the bit offset and bitfield size together. */ sou_offset += word_offset; @@ -960,17 +514,17 @@ btf_asm_sou_member (ctf_container_ref ctfc, ctf_dmdef_t * dmd, unsigned int idx) sou_offset |= ((bits & 0xff) << 24); /* Refer to the base type of the slice. */ - base_type = ref_type->dtd_u.dtu_slice.cts_type->dtd_type; + base_type = base_type->dtd_u.dtu_slice.cts_type; } else { /* Bitfield cannot be represented in BTF. Emit the member as having 'void' type. */ - base_type = BTF_VOID_TYPEID; + base_type = NULL; } } - btf_asm_type_ref ("btm_type", ctfc, base_type); + btf_asm_type_ref ("btm_type", base_type); dw2_asm_output_data (4, sou_offset, "btm_offset"); } @@ -993,86 +547,68 @@ btf_asm_enum_const (unsigned int size, ctf_dmdef_t * dmd, unsigned int idx) /* Asm'out a function parameter description following a BTF_KIND_FUNC_PROTO. */ static void -btf_asm_func_arg (ctf_container_ref ctfc, ctf_func_arg_t * farg, - size_t stroffset) +btf_asm_func_arg (ctf_func_arg_t * farg, size_t stroffset) { /* If the function arg does not have a name, refer to the null string at the start of the string table. This ensures correct encoding for varargs '...' arguments. */ if ((farg->farg_name != NULL) && strcmp (farg->farg_name, "")) - dw2_asm_output_data (4, farg->farg_name_offset + stroffset, "farg_name"); + dw2_asm_output_data (4, farg->farg_name_offset + stroffset, + "farg_name '%s'", farg->farg_name); else - dw2_asm_output_data (4, 0, "farg_name"); - - ctf_id_t ref_id = BTF_VOID_TYPEID; - if (farg->farg_type && !btf_removed_type_p (farg->farg_type->dtd_type)) - ref_id = farg->farg_type->dtd_type; + dw2_asm_output_data (4, 0, "farg_name ''"); - btf_asm_type_ref ("farg_type", ctfc, ref_id); + btf_asm_type_ref ("farg_type", farg->farg_type); } /* Asm'out a BTF_KIND_FUNC type. */ static void -btf_asm_func_type (ctf_container_ref ctfc, ctf_dtdef_ref dtd, ctf_id_t id) +btf_asm_func_type (ctf_dtdef_ref dtd) { - ctf_id_t ref_id = dtd->dtd_data.ctti_type; dw2_asm_output_data (4, dtd->dtd_data.ctti_name, "TYPE %" PRIu64 " BTF_KIND_FUNC '%s'", - btf_absolute_func_id (id), get_btf_type_name (dtd)); + dtd->dtd_type, get_btf_type_name (dtd)); dw2_asm_output_data (4, BTF_TYPE_INFO (BTF_KIND_FUNC, 0, dtd->linkage), "btt_info: kind=%u, kflag=%u, linkage=%u", BTF_KIND_FUNC, 0, dtd->linkage); - btf_asm_type_ref ("btt_type", ctfc, ref_id); + btf_asm_type_ref ("btt_type", dtd->ref_type); } -/* Collect the name for the DATASEC reference required to be output as a - symbol. */ +/* Asm'out a variable entry following a BTF_KIND_DATASEC. */ -static const char * -get_name_for_datasec_entry (ctf_container_ref ctfc, ctf_id_t ref_id) +static void +btf_asm_datasec_entry (struct btf_datasec_entry entry) { - if (ref_id >= num_types_added + 1 - && ref_id < num_types_added + num_vars_added + 1) + const char *symbol_name = NULL; + if (entry.is_var) { - /* Ref to a variable. Should only appear in DATASEC entries. */ - ctf_id_t var_id = btf_relative_var_id (ref_id); - ctf_dvdef_ref dvd = ctfc->ctfc_vars_list[var_id]; - return dvd->dvd_name; + symbol_name = entry.dvd->dvd_name; + dw2_asm_output_data (4, entry.dvd->dvd_id, + "bts_type: (BTF_KIND_VAR '%s')", symbol_name); } - else if (ref_id >= num_types_added + num_vars_added + 1) + else { - /* Ref to a FUNC record. */ - size_t func_id = btf_relative_func_id (ref_id); - ctf_dtdef_ref ref_type = (*funcs)[func_id]; - return get_btf_type_name (ref_type); + symbol_name = entry.dtd->dtd_name; + btf_asm_type_ref ("bts_type", entry.dtd); } - return NULL; -} - -/* Asm'out a variable entry following a BTF_KIND_DATASEC. */ -static void -btf_asm_datasec_entry (ctf_container_ref ctfc, struct btf_var_secinfo info) -{ - const char *symbol_name = get_name_for_datasec_entry (ctfc, info.type); - btf_asm_datasec_type_ref ("bts_type", ctfc, info.type); if (!btf_with_core_debuginfo_p () || symbol_name == NULL) - dw2_asm_output_data (4, info.offset, "bts_offset"); + dw2_asm_output_data (4, 0, "bts_offset"); else dw2_asm_output_offset (4, symbol_name, NULL, "bts_offset"); - dw2_asm_output_data (4, info.size, "bts_size"); + + dw2_asm_output_data (4, entry.size, "bts_size"); } /* Asm'out a whole BTF_KIND_DATASEC, including its variable entries. */ static void -btf_asm_datasec_type (ctf_container_ref ctfc, btf_datasec_t ds, ctf_id_t id, - size_t stroffset) +btf_asm_datasec_type (btf_datasec_t ds) { - dw2_asm_output_data (4, ds.name_offset + stroffset, + dw2_asm_output_data (4, ds.name_offset, "TYPE %" PRIu64 " BTF_KIND_DATASEC '%s'", - btf_absolute_datasec_id (id), ds.name); + ds.id, ds.name); dw2_asm_output_data (4, BTF_TYPE_INFO (BTF_KIND_DATASEC, 0, ds.entries.length ()), "btt_info: n_entries=%u", ds.entries.length ()); @@ -1080,7 +616,7 @@ btf_asm_datasec_type (ctf_container_ref ctfc, btf_datasec_t ds, ctf_id_t id, loaders such as libbpf. */ dw2_asm_output_data (4, 0, "btt_size"); for (size_t i = 0; i < ds.entries.length (); i++) - btf_asm_datasec_entry (ctfc, ds.entries[i]); + btf_asm_datasec_entry (ds.entries[i]); } /* Compute and output the header information for a .BTF section. */ @@ -1099,20 +635,11 @@ output_btf_header (ctf_container_ref ctfc) uint32_t type_off = 0, type_len = 0; uint32_t str_off = 0, str_len = 0; - uint32_t datasec_vlen_bytes = 0; if (!ctfc_is_empty_container (ctfc)) { - for (size_t i = 0; i < datasecs.length (); i++) - { - datasec_vlen_bytes += ((datasecs[i].entries.length ()) - * sizeof (struct btf_var_secinfo)); - } - /* Total length (bytes) of the types section. */ - type_len = (num_types_added * sizeof (struct btf_type)) - + (num_types_created * sizeof (struct btf_type)) - + datasec_vlen_bytes + type_len = ctfc->ctfc_num_types * sizeof (struct btf_type) + ctfc->ctfc_num_vlen_bytes; str_off = type_off + type_len; @@ -1124,7 +651,9 @@ output_btf_header (ctf_container_ref ctfc) /* Offset of type section. */ dw2_asm_output_data (4, type_off, "type_off"); /* Length of type section in bytes. */ - dw2_asm_output_data (4, type_len, "type_len"); + dw2_asm_output_data (4, type_len, "type_len: ntypes=%u, vlen=%u", + (uint32_t) ctfc->ctfc_num_types, + (uint32_t) ctfc->ctfc_num_vlen_bytes); /* Offset of string section. */ dw2_asm_output_data (4, str_off, "str_off"); /* Length of string section in bytes. */ @@ -1137,11 +666,11 @@ static void output_btf_vars (ctf_container_ref ctfc) { size_t i; - size_t num_ctf_vars = num_vars_added; + size_t num_ctf_vars = ctfc->ctfc_vars_list_count; if (num_ctf_vars) { for (i = 0; i < num_ctf_vars; i++) - btf_asm_varent (ctfc, ctfc->ctfc_vars_list[i]); + btf_asm_varent (ctfc->ctfc_vars_list[i]); } } @@ -1156,7 +685,8 @@ output_btf_strs (ctf_container_ref ctfc) while (ctf_string) { - dw2_asm_output_nstring (ctf_string->cts_str, -1, "btf_string, str_pos = 0x%x", str_pos); + dw2_asm_output_nstring (ctf_string->cts_str, -1, + "btf_string, str_pos = 0x%x", str_pos); str_pos += strlen(ctf_string->cts_str) + 1; ctf_string = ctf_string->cts_next; } @@ -1164,7 +694,8 @@ output_btf_strs (ctf_container_ref ctfc) ctf_string = ctfc->ctfc_aux_strtable.ctstab_head; while (ctf_string) { - dw2_asm_output_nstring (ctf_string->cts_str, -1, "btf_aux_string, str_pos = 0x%x", str_pos); + dw2_asm_output_nstring (ctf_string->cts_str, -1, + "btf_aux_string, str_pos = 0x%x", str_pos); str_pos += strlen(ctf_string->cts_str) + 1; ctf_string = ctf_string->cts_next; } @@ -1174,7 +705,7 @@ output_btf_strs (ctf_container_ref ctfc) BTF_KIND_UNION type. */ static void -output_asm_btf_sou_fields (ctf_container_ref ctfc, ctf_dtdef_ref dtd) +output_asm_btf_sou_fields (ctf_dtdef_ref dtd) { ctf_dmdef_t * dmd; @@ -1182,7 +713,7 @@ output_asm_btf_sou_fields (ctf_container_ref ctfc, ctf_dtdef_ref dtd) for (dmd = dtd->dtd_u.dtu_members; dmd != NULL; dmd = (ctf_dmdef_t *) ctf_dmd_list_next (dmd)) { - btf_asm_sou_member (ctfc, dmd, idx); + btf_asm_sou_member (dmd, idx); idx++; } } @@ -1190,8 +721,7 @@ output_asm_btf_sou_fields (ctf_container_ref ctfc, ctf_dtdef_ref dtd) /* Output all enumerator constants following a BTF_KIND_ENUM{,64}. */ static void -output_asm_btf_enum_list (ctf_container_ref ARG_UNUSED (ctfc), - ctf_dtdef_ref dtd) +output_asm_btf_enum_list (ctf_dtdef_ref dtd) { ctf_dmdef_t * dmd; @@ -1214,7 +744,7 @@ output_asm_btf_func_args_list (ctf_container_ref ctfc, ctf_func_arg_t * farg; for (farg = dtd->dtd_u.dtu_argv; farg != NULL; farg = (ctf_func_arg_t *) ctf_farg_list_next (farg)) - btf_asm_func_arg (ctfc, farg, farg_name_offset); + btf_asm_func_arg (farg, farg_name_offset); } /* Output the variable portion of a BTF type record. The information depends @@ -1225,7 +755,7 @@ output_asm_btf_vlen_bytes (ctf_container_ref ctfc, ctf_dtdef_ref dtd) { uint32_t btf_kind, encoding; - btf_kind = get_btf_kind (CTF_V2_INFO_KIND (dtd->dtd_data.ctti_info)); + btf_kind = btf_dtd_kind (dtd); if (btf_kind == BTF_KIND_UNKN) return; @@ -1238,8 +768,7 @@ output_asm_btf_vlen_bytes (ctf_container_ref ctfc, ctf_dtdef_ref dtd) if (dtd->dtd_data.ctti_size < 1) break; - /* In BTF the CHAR `encoding' seems to not be used, so clear it - here. */ + /* In BTF the CHAR `encoding' seems to not be used, so clear it here. */ dtd->dtd_u.dtu_enc.cte_format &= ~BTF_INT_CHAR; encoding = BTF_INT_DATA (dtd->dtd_u.dtu_enc.cte_format, @@ -1250,16 +779,16 @@ output_asm_btf_vlen_bytes (ctf_container_ref ctfc, ctf_dtdef_ref dtd) break; case BTF_KIND_ARRAY: - btf_asm_array (ctfc, dtd->dtd_u.dtu_arr); + btf_asm_array (dtd->dtd_u.dtu_arr); break; case BTF_KIND_STRUCT: case BTF_KIND_UNION: - output_asm_btf_sou_fields (ctfc, dtd); + output_asm_btf_sou_fields (dtd); break; case BTF_KIND_ENUM: - output_asm_btf_enum_list (ctfc, dtd); + output_asm_btf_enum_list (dtd); break; case BTF_KIND_FUNC_PROTO: @@ -1289,9 +818,9 @@ output_asm_btf_vlen_bytes (ctf_container_ref ctfc, ctf_dtdef_ref dtd) static void output_asm_btf_type (ctf_container_ref ctfc, ctf_dtdef_ref type) { - if (btf_emit_id_p (type->dtd_type)) + if (btf_emit_type_p (type)) { - btf_asm_type (ctfc, type); + btf_asm_type (type); output_asm_btf_vlen_bytes (ctfc, type); } } @@ -1303,7 +832,9 @@ static void output_btf_types (ctf_container_ref ctfc) { size_t i; - size_t num_types = ctfc->ctfc_types->elements (); + size_t num_types; + num_types = ctfc->ctfc_types->elements (); + if (num_types) { for (i = 1; i <= num_types; i++) @@ -1314,76 +845,45 @@ output_btf_types (ctf_container_ref ctfc) /* Output all BTF_KIND_FUNC type records. */ static void -output_btf_func_types (ctf_container_ref ctfc) +output_btf_func_types (void) { ctf_dtdef_ref ref; unsigned i; FOR_EACH_VEC_ELT (*funcs, i, ref) - btf_asm_func_type (ctfc, ref, i); + btf_asm_func_type (ref); } /* Output all BTF_KIND_DATASEC records. */ static void -output_btf_datasec_types (ctf_container_ref ctfc) +output_btf_datasec_types (void) { - size_t name_offset = ctfc_get_strtab_len (ctfc, CTF_STRTAB); - - for (size_t i = 0; i < datasecs.length(); i++) - btf_asm_datasec_type (ctfc, datasecs[i], i, name_offset); + for (size_t i = 0; i < datasecs.length (); i++) + btf_asm_datasec_type (datasecs[i]); } -/* Postprocess the CTF debug data post initialization. - - During the postprocess pass: - - - Prepare the sorted list of BTF types. - - The sorted list of BTF types is, firstly, used for lookup (during the BTF - generation process) of CTF/BTF types given a typeID. - - Secondly, in the emitted BTF section, BTF Types need to be in the sorted - order of their type IDs. The BTF types section is viewed as an array, - with type IDs used to index into that array. It is essential that every - type be placed at the exact index corresponding to its ID, or else - references to that type from other types will no longer be correct. - - - References to void types are converted to reference BTF_VOID_TYPEID. In - CTF, a distinct type is used to encode void. - - - Bitfield struct/union members are converted to BTF encoding. CTF uses - slices to encode bitfields, but BTF does not have slices and encodes - bitfield information directly in the variable-length btf_member - descriptions following the struct or union type. - - - Unrepresentable types are removed. We cannot have any invalid BTF types - appearing in the output so they must be removed, and type ids of other - types and references adjust accordingly. This also involves ensuring that - BTF descriptions of struct members referring to unrepresentable types are - not emitted, as they would be nonsensical. - - - Adjust inner- and inter-type references-by-ID to account for removed - types, and construct the types list. */ +/* Write out all BTF debug info. */ void -btf_init_postprocess (void) +btf_output (ctf_container_ref ctfc) { - ctf_container_ref tu_ctfc = ctf_get_tu_ctfc (); - - holes.create (0); - voids.create (0); + output_btf_header (ctfc); + output_btf_types (ctfc); + output_btf_vars (ctfc); + output_btf_func_types (); + output_btf_datasec_types (); + output_btf_strs (ctfc); +} - num_types_added = 0; - num_types_created = 0; +/* Workaround for 'const void' variables. These variables are sometimes used + in eBPF programs to address kernel symbols. DWARF does not generate const + qualifier on void type, so we would incorrectly emit these variables + without the const qualifier. Find any such variables, and update them to + refer to a new 'const' modifier type for void. */ - /* Workaround for 'const void' variables. These variables are sometimes used - in eBPF programs to address kernel symbols. DWARF does not generate const - qualifier on void type, so we would incorrectly emit these variables - without the const qualifier. - Unfortunately we need the TREE node to know it was const, and we need - to create the const modifier type (if needed) now, before making the types - list. So we can't avoid iterating with FOR_EACH_VARIABLE here, and then - again when creating the DATASEC entries. */ +static void +btf_early_add_const_void (ctf_container_ref ctfc) +{ ctf_dtdef_ref constvoid_dtd = NULL; varpool_node *var; FOR_EACH_VARIABLE (var) @@ -1398,120 +898,389 @@ btf_init_postprocess (void) if (die == NULL) continue; - ctf_dvdef_ref dvd = ctf_dvd_lookup (tu_ctfc, die); + ctf_dvdef_ref dvd = ctf_dvd_lookup (ctfc, die); if (dvd == NULL) continue; /* Create the 'const' modifier type for void. */ if (constvoid_dtd == NULL) - constvoid_dtd = ctf_add_reftype (tu_ctfc, CTF_ADD_ROOT, - dvd->dvd_type, CTF_K_CONST, NULL); + constvoid_dtd = ctf_add_reftype (ctfc, CTF_ADD_ROOT, + dvd->dvd_type, CTF_K_CONST, NULL); dvd->dvd_type = constvoid_dtd; } } +} - size_t i; - size_t num_ctf_types = tu_ctfc->ctfc_types->elements (); +/* Functions actually get two type records: a BTF_KIND_FUNC_PROTO, and also a + BTF_KIND_FUNC. But the CTF container only allocates one type per function, + which matches closely with BTF_KIND_FUNC_PROTO. For each such function, + construct a BTF_KIND_FUNC entry. This is done early, because we want FUNC + records even for functions which are later inlined by optimizations. */ - if (num_ctf_types) +static void +btf_early_add_func_records (ctf_container_ref ctfc) +{ + cgraph_node *func; + FOR_EACH_FUNCTION (func) { - init_btf_id_map (num_ctf_types + 1); - - /* Allocate the types list and traverse all types, placing each type - at the index according to its ID. Add 1 because type ID 0 always - represents VOID. */ - tu_ctfc->ctfc_types_list - = ggc_vec_alloc<ctf_dtdef_ref>(num_ctf_types + 1); - tu_ctfc->ctfc_types->traverse<ctf_container_ref, btf_dtd_postprocess_cb> - (tu_ctfc); - - /* Build mapping of CTF type ID -> BTF type ID, and count total number - of valid BTF types added. */ - for (i = 1; i <= num_ctf_types; i++) - { - ctf_dtdef_ref dtd = tu_ctfc->ctfc_types_list[i]; - ctf_id_t btfid = btf_adjust_type_id (dtd->dtd_type); - set_btf_id (dtd->dtd_type, btfid); - if (btfid < BTF_MAX_TYPE && (btfid != BTF_VOID_TYPEID)) - num_types_added ++; - } + dw_die_ref die = lookup_decl_die (func->decl); + if (die == NULL) + continue; + + ctf_dtdef_ref dtd = ctf_dtd_lookup (ctfc, die); + if (dtd == NULL) + continue; + + /* Do not add FUNC records for kernel helpers. */ + if (DECL_EXTERNAL (func->decl) + && (lookup_attribute ("kernel_helper", + DECL_ATTRIBUTES (func->decl))) != NULL_TREE) + continue; + + ctf_dtdef_ref func_dtd = ggc_cleared_alloc<ctf_dtdef_t> (); + func_dtd->dtd_data = dtd->dtd_data; + func_dtd->dtd_data.ctti_type = dtd->dtd_type; + func_dtd->ref_type = dtd; + func_dtd->linkage = dtd->linkage; + func_dtd->dtd_name = dtd->dtd_name; + /* Type ID will be assigned just before output. */ + + /* Only the BTF_KIND_FUNC type actually references the name. The + BTF_KIND_FUNC_PROTO is always anonymous. */ + dtd->dtd_data.ctti_name = 0; + + /* Mark 'extern' funcs. */ + if (DECL_EXTERNAL (func->decl)) + func_dtd->linkage = BTF_FUNC_EXTERN; + + /* Buffer newly created FUNC records. We cannot simply insert them + into the types map, because types are keyed by their DWARF DIE, + and we have no unique DIE to use as a key since the FUNC_PROTOs + are already present in the map. */ + vec_safe_push (funcs, func_dtd); + func_map->put (dtd, func_dtd); } } -/* Process and output all BTF data. Entry point of btfout. */ +/* Initial entry point of BTF generation, called at early_finish () after + CTF information has possibly been output. Translate all CTF information + to BTF, and do any processing that must be done early, such as creating + BTF_KIND_FUNC records. */ void -btf_output (const char * filename) +btf_early_finish (void) { ctf_container_ref tu_ctfc = ctf_get_tu_ctfc (); - init_btf_sections (); - - datasecs.create (0); vec_alloc (funcs, 16); + func_map = hash_map<ctf_dtdef_ref, ctf_dtdef_ref>::create_ggc (16); + + btf_early_add_const_void (tu_ctfc); + btf_early_add_func_records (tu_ctfc); +} + +/* Push a BTF datasec entry ENTRY into the datasec named SECNAME, + creating the datasec record if it does not already exist. */ + +static void +btf_datasec_push_entry (ctf_container_ref ctfc, const char *secname, + struct btf_datasec_entry entry) +{ + if (secname == NULL) + return; + + /* If we already have a datasec record for the appropriate section, + append the new entry to it. */ + for (size_t i = 0; i < datasecs.length (); i++) + if (strcmp (datasecs[i].name, secname) == 0) + { + datasecs[i].entries.safe_push (entry); + return; + } + + /* If we don't already have a datasec record for secname, make one. */ + uint32_t str_off; + ctf_add_string (ctfc, secname, &str_off, CTF_AUX_STRTAB); + if (strcmp (secname, "")) + ctfc->ctfc_aux_strlen += strlen (secname) + 1; - ctf_add_cuname (tu_ctfc, filename); + /* Note: ID will be assigned just before output. */ + btf_datasec_t ds; + ds.name = secname; + ds.name_offset = str_off; - btf_emit_preprocess (tu_ctfc); + /* Insert the entry into the new datasec record. */ + ds.entries.create (1); + ds.entries.quick_push (entry); - output_btf_header (tu_ctfc); - output_btf_types (tu_ctfc); - output_btf_vars (tu_ctfc); - output_btf_func_types (tu_ctfc); - output_btf_datasec_types (tu_ctfc); - output_btf_strs (tu_ctfc); + /* Insert the datasec record itself. */ + datasecs.safe_push (ds); } -/* Reset all state for BTF generation so that we can rerun the compiler within - the same process. */ +/* Create a datasec entry for a function, and insert it into the datasec + record for the appropriate section. Create the record if it does not + yet exist. */ -void -btf_finalize (void) +static void +btf_datasec_add_func (ctf_container_ref ctfc, cgraph_node *func, + ctf_dtdef_ref func_dtd) { - btf_info_section = NULL; + const char *section_name = get_section_name (func); - /* Clear preprocessing state. */ - num_vars_added = 0; - num_types_added = 0; - num_types_created = 0; + /* Note: get_section_name () returns NULL for functions in text + section. This is intentional, since we do not want to generate + DATASEC entries for them. */ + if (section_name == NULL) + return; - holes.release (); - voids.release (); - for (size_t i = 0; i < datasecs.length (); i++) - datasecs[i].entries.release (); - datasecs.release (); + struct btf_datasec_entry entry; + gcc_assert (func_dtd); + entry.dtd = func_dtd; + entry.is_var = false; - funcs = NULL; + /* Size is left as zero at compile time, to be filled in by loaders + such as libbpf. */ + entry.size = 0; - btf_var_ids->empty (); - btf_var_ids = NULL; + btf_datasec_push_entry (ctfc, section_name, entry); +} - free (btf_id_map); - btf_id_map = NULL; +/* Create a datasec entry for a variable, and insert it into the datasec + record for the appropriate section. Create the record if it does not + yet exist. */ - ctf_container_ref tu_ctfc = ctf_get_tu_ctfc (); - ctfc_delete_container (tu_ctfc); - tu_ctfc = NULL; +static void +btf_datasec_add_var (ctf_container_ref ctfc, varpool_node *var, + ctf_dvdef_ref dvd) +{ + /* PR112849: avoid assuming a section for extern decls without + an explicit section, which would result in incorrectly + emitting a BTF_KIND_DATASEC entry for them. */ + if (DECL_EXTERNAL (var->decl) && var->get_section () == NULL) + return; + + const char *section_name = get_section_name (var); + if (section_name == NULL) + return; + + gcc_assert (dvd); + struct btf_datasec_entry entry; + entry.dvd = dvd; + entry.is_var = true; + entry.size = 0; + + tree size = DECL_SIZE_UNIT (var->decl); + if (tree_fits_uhwi_p (size)) + entry.size = tree_to_uhwi (size); + else if (VOID_TYPE_P (TREE_TYPE (var->decl))) + entry.size = 1; + + btf_datasec_push_entry (ctfc, section_name, entry); } -/* Initial entry point of BTF generation, called at early_finish () after - CTF information has possibly been output. Translate all CTF information - to BTF, and do any processing that must be done early, such as creating - BTF_KIND_FUNC records. */ +/* Add datasec entries for functions to CTFC. */ -void -btf_early_finish (void) +static void +btf_late_add_func_datasec_entries (ctf_container_ref ctfc) +{ + /* We need to create FUNC records at early_finish, so that we have them + even for functions which are later inlined by optimization passes. + But on the other hand, we do not want datasec entries for such functions, + so only create the datasec entries for them late. This loop will not + hit functions which have already been inlined. */ + cgraph_node *func; + FOR_EACH_FUNCTION (func) + { + dw_die_ref die = lookup_decl_die (func->decl); + if (die == NULL) + continue; + + ctf_dtdef_ref dtd = ctf_dtd_lookup (ctfc, die); + if (dtd == NULL) + continue; + + ctf_dtdef_ref *pdtd = func_map->get (dtd); + if (pdtd && DECL_EXTERNAL (func->decl)) + btf_datasec_add_func (ctfc, func, *pdtd); + } +} + +/* Helper function used to determine whether or not a BTF_KIND_VAR record + for the variable VAR shall be emitted. */ + +static bool +btf_emit_variable_p (ctf_container_ref ctfc, varpool_node *var, + ctf_dvdef_ref *pdvd) +{ + dw_die_ref die = lookup_decl_die (var->decl); + if (die == NULL) + return false; + + ctf_dvdef_ref dvd = ctf_dvd_lookup (ctfc, die); + if (dvd == NULL) + return false; + + /* If this is an extern variable declaration with a defining declaration + later, skip it so that only the defining declaration is emitted. + This is the same case, fix and reasoning as in CTF; see PR105089. */ + if (ctf_dvd_ignore_lookup (ctfc, dvd->dvd_key)) + return false; + + /* Skip variables with unrepresentable types. */ + if (!btf_emit_type_p (dvd->dvd_type)) + return false; + + *pdvd = dvd; + return true; +} + +/* Add BTF_KIND_VAR records for variables. */ + +static void +btf_late_add_vars (ctf_container_ref ctfc) +{ + size_t num_ctf_vars = ctfc->ctfc_vars->elements (); + + ctfc->ctfc_vars_list = ggc_vec_alloc<ctf_dvdef_ref>(num_ctf_vars); + + varpool_node *var; + ctf_dvdef_ref dvd; + FOR_EACH_VARIABLE (var) + { + if (!btf_emit_variable_p (ctfc, var, &dvd)) + continue; + + /* Mark 'extern' variables. */ + if (DECL_EXTERNAL (var->decl)) + dvd->dvd_visibility = BTF_VAR_GLOBAL_EXTERN; + + /* Add the variable to the vars list. */ + ctfc->ctfc_vars_list[ctfc->ctfc_vars_list_count++] = dvd; + + /* Add a BTF_KIND_DATASEC entry for the variable. */ + btf_datasec_add_var (ctfc, var, dvd); + } +} + +/* Callback used by btf_late_assign_type_ids to insert types into their initial + positions in the type list. */ + +static int +btf_type_list_cb (ctf_dtdef_ref *slot, ctf_container_ref ctfc) +{ + ctf_dtdef_ref dtd = *slot; + ctfc->ctfc_types_list[dtd->dtd_type] = dtd; + return 1; +} + +/* Construct the initial type list and assign BTF IDs for all types translated + from CTF. */ + +static void +btf_late_collect_translated_types (ctf_container_ref ctfc) +{ + size_t num_ctf_types = ctfc->ctfc_types->elements (); + + /* First, place each type at its CTF-assigned index in the list. + The '+1' here and below is to account for the implicit void type with + ID 0. There is no real type at index 0 in the list. */ + ctfc->ctfc_types_list = ggc_vec_alloc<ctf_dtdef_ref>(num_ctf_types + 1); + ctfc->ctfc_types->traverse<ctf_container_ref, btf_type_list_cb> (ctfc); + + /* Now, pass through the list and adjust IDs to account for types which will + not be emitted. This results in each type that will be emitted in BTF + being assigned an appropriate ID. Note that types which will not be + emitted remain in the list; they are skipped at output time. */ + unsigned int skip = 0; + for (size_t i = 1; i <= num_ctf_types; i++) + { + ctf_dtdef_ref dtd = ctfc->ctfc_types_list[i]; + if (!btf_emit_type_p (dtd)) + { + dtd->dtd_type = BTF_INVALID_TYPEID; + skip += 1; + continue; + } + + dtd->dtd_type -= skip; + ctfc->ctfc_num_types++; + ctfc->ctfc_num_vlen_bytes += btf_calc_num_vbytes (dtd); + } + + max_translated_id = ctfc->ctfc_num_types; + ctfc->ctfc_nextid = ctfc->ctfc_num_types + 1; +} + +/* Assign BTF IDs for FUNC records and account for their size. */ + +static void +btf_late_assign_func_ids (ctf_container_ref ctfc) +{ + ctf_dtdef_ref dtd; + unsigned int i; + FOR_EACH_VEC_ELT (*funcs, i, dtd) + { + dtd->dtd_type = ctfc->ctfc_nextid++; + ctfc->ctfc_num_types++; + } +} + +/* Assign BTF IDs for variables and account for their size. */ + +static void +btf_late_assign_var_ids (ctf_container_ref ctfc) +{ + for (size_t i = 0; i < ctfc->ctfc_vars_list_count; i++) + { + ctf_dvdef_ref dvd = ctfc->ctfc_vars_list[i]; + ctf_id_t id = ctfc->ctfc_nextid++; + gcc_assert (id <= BTF_MAX_TYPE); + dvd->dvd_id = id; + + ctfc->ctfc_num_types++; + ctfc->ctfc_num_vlen_bytes += sizeof (struct btf_var); + } +} + +/* Assign BTF IDs for datasec records and account for their size. */ + +static void +btf_late_assign_datasec_ids (ctf_container_ref ctfc) { - btf_init_postprocess (); + for (size_t i = 0; i < datasecs.length (); i++) + { + datasecs[i].id = ctfc->ctfc_nextid++; + datasecs[i].name_offset += ctfc_get_strtab_len (ctfc, CTF_STRTAB); + ctfc->ctfc_num_types++; + ctfc->ctfc_num_vlen_bytes += (datasecs[i].entries.length () + * sizeof (struct btf_var_secinfo)); + } } /* Late entry point for BTF generation, called from dwarf2out_finish (). Complete and emit BTF information. */ void -btf_finish (const char * filename) +btf_finish (void) { - btf_output (filename); + ctf_container_ref tu_ctfc = ctf_get_tu_ctfc (); + init_btf_sections (); + + datasecs.create (0); + + tu_ctfc->ctfc_num_types = 0; + tu_ctfc->ctfc_num_vlen_bytes = 0; + tu_ctfc->ctfc_vars_list_count = 0; + + btf_late_add_vars (tu_ctfc); + btf_late_collect_translated_types (tu_ctfc); + btf_late_add_func_datasec_entries (tu_ctfc); + btf_late_assign_var_ids (tu_ctfc); + btf_late_assign_func_ids (tu_ctfc); + btf_late_assign_datasec_ids (tu_ctfc); + + /* Finally, write out the complete .BTF section. */ + btf_output (tu_ctfc); /* If compiling for BPF with CO-RE info, we cannot deallocate until after CO-RE information is created, which happens very late in BPF backend. @@ -1521,6 +1290,27 @@ btf_finish (const char * filename) btf_finalize (); } +/* Reset all state for BTF generation so that we can rerun the compiler within + the same process. */ + +void +btf_finalize (void) +{ + btf_info_section = NULL; + max_translated_id = 0; + + for (size_t i = 0; i < datasecs.length (); i++) + datasecs[i].entries.release (); + datasecs.release (); + + funcs = NULL; + func_map->empty (); + func_map = NULL; + + ctf_container_ref tu_ctfc = ctf_get_tu_ctfc (); + ctfc_delete_container (tu_ctfc); + tu_ctfc = NULL; +} /* Traversal function for all BTF_KIND_FUNC type records. */ diff --git a/gcc/ctfc.h b/gcc/ctfc.h index cfc805db7b5..90421c72c09 100644 --- a/gcc/ctfc.h +++ b/gcc/ctfc.h @@ -162,10 +162,14 @@ struct GTY ((for_user)) ctf_dtdef ctf_id_t dtd_type; /* Type identifier for this definition. */ struct ctf_dtdef *ref_type; /* Type referred to by this type (if any). */ ctf_itype_t dtd_data; /* Type node. */ - bool from_global_func; /* Whether this type was added from a global - function. */ uint32_t linkage; /* Used in function types. 0=local, 1=global. */ - bool dtd_enum_unsigned; /* Enum signedness. */ + + /* Whether this type was added from a global function. */ + BOOL_BITFIELD from_global_func : 1; + /* Enum signedness. */ + BOOL_BITFIELD dtd_enum_unsigned : 1; + /* Lots of spare bits. */ + union GTY ((desc ("ctf_dtu_d_union_selector (&%1)"))) { /* struct, union, or enum. */ @@ -192,6 +196,7 @@ struct GTY ((for_user)) ctf_dvdef uint32_t dvd_name_offset; /* Offset of the name in str table. */ unsigned int dvd_visibility; /* External visibility. 0=static,1=global. */ struct ctf_dtdef * dvd_type; /* Type of variable. */ + ctf_id_t dvd_id; /* ID of this variable. Only used for BTF. */ }; typedef struct ctf_dvdef ctf_dvdef_t; @@ -388,7 +393,7 @@ extern void ctf_output (const char * filename); extern void ctf_finalize (void); extern void btf_early_finish (void); -extern void btf_finish (const char * filename); +extern void btf_finish (void); extern void btf_finalize (void); extern ctf_container_ref ctf_get_tu_ctfc (void); @@ -442,7 +447,9 @@ extern int ctf_add_variable (ctf_container_ref, const char *, ctf_dtdef_ref, dw_die_ref, unsigned int, dw_die_ref); extern ctf_dtdef_ref ctf_lookup_tree_type (ctf_container_ref, const tree); -extern ctf_id_t get_btf_id (ctf_id_t); + +/* Callback and traversal function for BTF_KIND_FUNC records. Used by BPF + target for BPF CO-RE implementation. */ typedef bool (*funcs_traverse_callback) (ctf_dtdef_ref, void *); bool traverse_btf_func_types (funcs_traverse_callback, void *); diff --git a/gcc/dwarf2ctf.cc b/gcc/dwarf2ctf.cc index f16b5ceee74..ac195c1a862 100644 --- a/gcc/dwarf2ctf.cc +++ b/gcc/dwarf2ctf.cc @@ -980,13 +980,13 @@ ctf_debug_early_finish (const char * filename) /* Finish CTF/BTF debug info emission. */ void -ctf_debug_finish (const char * filename) +ctf_debug_finish () { /* Emit BTF debug info here when CO-RE relocations need to be generated. BTF with CO-RE relocations needs to be generated when CO-RE is in effect for the BPF target. */ if (btf_debuginfo_p ()) - btf_finish (filename); + btf_finish (); } #include "gt-dwarf2ctf.h" diff --git a/gcc/dwarf2ctf.h b/gcc/dwarf2ctf.h index 46184325bae..f8a181a9762 100644 --- a/gcc/dwarf2ctf.h +++ b/gcc/dwarf2ctf.h @@ -32,7 +32,7 @@ extern void ctf_debug_init (void); extern void ctf_debug_init_postprocess (bool); extern bool ctf_do_die (dw_die_ref); extern void ctf_debug_early_finish (const char *); -extern void ctf_debug_finish (const char *); +extern void ctf_debug_finish (void); /* Wrappers for CTF/BTF to fetch information from GCC DWARF DIE. Used in ctfc.cc. diff --git a/gcc/dwarf2out.cc b/gcc/dwarf2out.cc index 1664934ccc3..794b94ca80b 100644 --- a/gcc/dwarf2out.cc +++ b/gcc/dwarf2out.cc @@ -32276,7 +32276,7 @@ dwarf2out_finish (const char *filename) /* Generate CTF/BTF debug info. */ if ((ctf_debug_info_level > CTFINFO_LEVEL_NONE || btf_debuginfo_p ()) && lang_GNU_C ()) - ctf_debug_finish (filename); + ctf_debug_finish (); /* Skip emitting DWARF if not required. */ if (!dwarf_debuginfo_p ())