[BPF] `BTFDebug` doesn't generate BTF for all structs, if BPF map type is wrapped in Rust wrapper types

[vadorovsky]

This issue was detected during an attempt of supporting BTF maps in Aya (aya-rs/aya#1117).

BTF map definitions have the following format in C:

struct my_key {
  int a;
};

struct my_value {
  int a;
};

struct {
  int (*type)[BPF_MAP_TYPE_HASH];
  typeof(struct my_key) *key;
  typeof(struct my_value) *value;
  int (*max_entries)[10];
} map_1 __attribute__((section(".maps")));

The map_1 instance is then used as *void in libbpf functions like bpf_map_lookup_elem, bpf_map_update_elem etc..

The key and value structs can be anything as long as they hold primitive/POD types and as long as they are aligned.

The program above produces the following BTF:

#0: <VOID>
#1: <PTR> --> [3]
#2: <INT> 'int' bits:32 off:0 enc:signed
#3: <ARRAY> n:1 idx-->[4] val-->[2]
#4: <INT> '__ARRAY_SIZE_TYPE__' bits:32 off:0
#5: <PTR> --> [6]
#6: <STRUCT> 'my_key' sz:4 n:1
        #00 'a' off:0 --> [2]
#7: <PTR> --> [8]
#8: <STRUCT> 'my_value' sz:4 n:1
        #00 'a' off:0 --> [2]
#9: <PTR> --> [10]
#10: <ARRAY> n:10 idx-->[4] val-->[2]
#11: <STRUCT> '<anon>' sz:32 n:4
        #00 'type' off:0 --> [1]
        #01 'key' off:64 --> [5]
        #02 'value' off:128 --> [7]
        #03 'max_entries' off:192 --> [9]
#12: <VAR> 'map_1' kind:global-alloc --> [11]
#13: <DATASEC> '.maps' sz:0 n:1
        #00 off:0 sz:32 --> [12]

We can see both the map struct (#11) and the types used as key (#6) and value (#8).

However, in Rust, we want to wrap such map definitions in two wrapper types:

• A wrapper type representing a specific map type (e.g. HashMap, RingBuf), which provide methods (get, update), so people interact with those wrapper types instead of working with void pointers.
• Another wrapper type, which wraps the type above in UnsafeCell, so the Rust compiler doesn't complain about concurrent mutability and doesn't consider such action unsafe. It's basically a way of telling compiler, that we (Aya) guarantee that this type provides a thread-safe mutabiity (and it does out of the box, because of RCU in Linux kernel).

This ends up looking like:

#![no_std]
#![no_main]

pub const BPF_MAP_TYPE_HASH: usize = 1;

// The real map definition.
pub struct HashMapDef<K, V, const M: usize, const F: usize> {
    r#type: *const [i32; BPF_MAP_TYPE_HASH],
    key: *const K,
    value: *const V,
    max_entries: *const [i32; M],
    map_flags: *const [i32; F],
}
impl<K, V, const M: usize, const F: usize> HashMapDef<K, V, M, F> {
    pub const fn new() -> Self {
        Self {
            r#type: &[0i32; BPF_MAP_TYPE_HASH],
            key: ::core::ptr::null(),
            value: ::core::ptr::null(),
            max_entries: &[0i32; M],
            map_flags: &[0i32; F],
        }
    }
}
// Use `UnsafeCell` to allow the mutability by multiple threads.
pub struct HashMap<K, V, const M: usize, const F: usize = 0>(
    core::cell::UnsafeCell<HashMapDef<K, V, M, F>>,
);
impl<K, V, const M: usize, const F: usize> HashMap<K, V, M, F> {
    pub const fn new() -> Self {
        Self(core::cell::UnsafeCell::new(HashMapDef::new()))
    }
}
/// Tell Rust that `HashMap` is thread-safe.
unsafe impl<K: Sync, V: Sync, const M: usize, const F: usize> Sync for HashMap<K, V, M, F> {}

// Define custom structs for key and values.
pub struct MyKey(u32);
pub struct MyValue(u32);

#[link_section = ".maps"]
#[export_name = "HASH_MAP"]
pub static HASH_MAP: HashMap<MyKey, MyValue, 10> = HashMap::new();

The BTF for this program looks like:

#0: <VOID>
#1: <STRUCT> 'UnsafeCell_3C_map_def_3A__3A_HashMapDef_3C_map_def_3A__3A_MyKey_2C__20_map_def_3A__3A_MyValue_2C__20_10_2C__20_0_3E__3E_' sz:40 n:1
        #00 'value' off:0 --> [2]
#2: <STRUCT> 'HashMapDef_3C_map_def_3A__3A_MyKey_2C__20_map_def_3A__3A_MyValue_2C__20_10_2C__20_0_3E_' sz:40 n:5
        #00 'type' off:0 --> [3]
        #01 'key' off:64 --> [7]
        #02 'value' off:128 --> [8]
        #03 'max_entries' off:192 --> [9]
        #04 'map_flags' off:256 --> [11]
#3: <PTR> --> [5]
#4: <INT> 'i32' bits:32 off:0 enc:signed
#5: <ARRAY> n:1 idx-->[6] val-->[4]
#6: <INT> '__ARRAY_SIZE_TYPE__' bits:32 off:0
#7: <PTR> --> [30]
#8: <PTR> --> [31]
#9: <PTR> --> [10]
#10: <ARRAY> n:10 idx-->[6] val-->[4]
#11: <PTR> --> [12]
#12: <ARRAY> n:0 idx-->[6] val-->[4]
#13: <STRUCT> 'HashMap_3C_map_def_3A__3A_MyKey_2C__20_map_def_3A__3A_MyValue_2C__20_10_2C__20_0_3E_' sz:40 n:1
        #00 '__0' off:0 --> [1]
#14: <VAR> 'HASH_MAP' kind:global-alloc --> [13]
#15: <PTR> --> [16]
#16: <INT> 'u8' bits:8 off:0
#17: <INT> 'usize' bits:64 off:0
#18: <FUNC_PROTO> r-->[4] n:3
        #00 's1' --> [15]
        #01 's2' --> [15]
        #02 'n' --> [17]
#19: <FUNC> 'bcmp' --> global [18]
#20: <FUNC_PROTO> r-->[4] n:3
        #00 's1' --> [15]
        #01 's2' --> [15]
        #02 'n' --> [17]
#21: <FUNC> 'memcmp' --> global [20]
#22: <PTR> --> [16]
#23: <FUNC_PROTO> r-->[22] n:3
        #00 's' --> [22]
        #01 'c' --> [4]
        #02 'n' --> [17]
#24: <FUNC> 'memset' --> global [23]
#25: <FUNC_PROTO> r-->[22] n:3
        #00 'dest' --> [22]
        #01 'src' --> [15]
        #02 'n' --> [17]
#26: <FUNC> 'memcpy' --> global [25]
#27: <FUNC_PROTO> r-->[22] n:3
        #00 'dest' --> [22]
        #01 'src' --> [15]
        #02 'n' --> [17]
#28: <FUNC> 'memmove' --> global [27]
#29: <DATASEC> '.maps' sz:0 n:1
        #00 off:0 sz:40 --> [14]
#30: <FWD> 'MyKey' kind:struct
#31: <FWD> 'MyValue' kind:struct

We can see that MyKey and MyValue are <FWD> types and do not contain the actual <STRUCT> definition.

The BTFDebug module should be able to dig through the nested map definitions and produce the <STRUCT> definitions for types used there. The problem is reproducible only if the key and/or value type are custom structs.

I'm working on the fix, which is mostly ready, apart from llvm-lit test which will make sure it doesn't regress in the future.