| Message ID | 1544620189-26759-2-git-send-email-magnus.karlsson@intel.com |
|---|---|
| State | Changes Requested, archived |
| Delegated to: | BPF Maintainers |
| Headers | show |
| Series | libbpf: adding AF_XDP support | expand |
On Wed, Dec 12, 2018 at 02:09:48PM +0100, Magnus Karlsson wrote: > diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map > index cd02cd4..ae4cc0d 100644 > --- a/tools/lib/bpf/libbpf.map > +++ b/tools/lib/bpf/libbpf.map > @@ -121,6 +121,15 @@ LIBBPF_0.0.1 { > libbpf_prog_type_by_name; > libbpf_set_print; > libbpf_strerror; > + xsk__peek_cons; > + xsk__release_cons; > + xsk__reserve_prod; > + xsk__submit_prod; > + xsk__get_data; > + xsk__create_umem; > + xsk__create_xdp_socket; > + xsk__delete_umem; > + xsk__delete_xdp_socket; > local: I fully support the idea to provide common library for AF_XDP that is easily available in the distros. The main question is whether AF_XDP warrants its own lib or piggy back on libbpf effort is acceptable. Do you think above set of APIs will be enough for foreseeable future or this is just a beginning? If above is enough then it falls into XDP category. libbpf already has minimal support for XDP and AF_XDP fits right in. But if AF_XDP will keep growing a lot then it would make sense to keep the functionality in a separate library that minimally depends on libbpf.
On Thu, Dec 13, 2018 at 7:24 AM Alexei Starovoitov <alexei.starovoitov@gmail.com> wrote: > > On Wed, Dec 12, 2018 at 02:09:48PM +0100, Magnus Karlsson wrote: > > diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map > > index cd02cd4..ae4cc0d 100644 > > --- a/tools/lib/bpf/libbpf.map > > +++ b/tools/lib/bpf/libbpf.map > > @@ -121,6 +121,15 @@ LIBBPF_0.0.1 { > > libbpf_prog_type_by_name; > > libbpf_set_print; > > libbpf_strerror; > > + xsk__peek_cons; > > + xsk__release_cons; > > + xsk__reserve_prod; > > + xsk__submit_prod; > > + xsk__get_data; > > + xsk__create_umem; > > + xsk__create_xdp_socket; > > + xsk__delete_umem; > > + xsk__delete_xdp_socket; > > local: > > I fully support the idea to provide common library for AF_XDP > that is easily available in the distros. > > The main question is whether AF_XDP warrants its own lib or > piggy back on libbpf effort is acceptable. > > Do you think above set of APIs will be enough for foreseeable > future or this is just a beginning? This should be enough for the foreseeable future, maybe with the addition of the two higher level data plane functions xsk__recvmsg and xsl__sendmsg that was mentioned in the cover letter. My intention with this functionality is not to create another DPDK (there is already one, so no reason to reinvent it). I just want to lower the bar of entry for using AF_XDP and to stop people copying the code in the sample application. To use AF_XDP you need libbpf anyway, so I think it is a good fit for it. The intention is to keep this functionality lean and mean. > If above is enough then it falls into XDP category. libbpf > already has minimal support for XDP and AF_XDP fits right in. > > But if AF_XDP will keep growing a lot then it would make > sense to keep the functionality in a separate library > that minimally depends on libbpf. >
On 12/13/2018 10:06 AM, Magnus Karlsson wrote: > On Thu, Dec 13, 2018 at 7:24 AM Alexei Starovoitov > <alexei.starovoitov@gmail.com> wrote: >> On Wed, Dec 12, 2018 at 02:09:48PM +0100, Magnus Karlsson wrote: >>> diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map >>> index cd02cd4..ae4cc0d 100644 >>> --- a/tools/lib/bpf/libbpf.map >>> +++ b/tools/lib/bpf/libbpf.map >>> @@ -121,6 +121,15 @@ LIBBPF_0.0.1 { >>> libbpf_prog_type_by_name; >>> libbpf_set_print; >>> libbpf_strerror; >>> + xsk__peek_cons; >>> + xsk__release_cons; >>> + xsk__reserve_prod; >>> + xsk__submit_prod; >>> + xsk__get_data; >>> + xsk__create_umem; >>> + xsk__create_xdp_socket; >>> + xsk__delete_umem; >>> + xsk__delete_xdp_socket; >>> local: >> >> I fully support the idea to provide common library for AF_XDP >> that is easily available in the distros. +1 >> The main question is whether AF_XDP warrants its own lib or >> piggy back on libbpf effort is acceptable. >> >> Do you think above set of APIs will be enough for foreseeable >> future or this is just a beginning? > > This should be enough for the foreseeable future, maybe with the > addition of the two higher level data plane functions xsk__recvmsg and > xsl__sendmsg that was mentioned in the cover letter. My intention with > this functionality is not to create another DPDK (there is already > one, so no reason to reinvent it). I just want to lower the bar of > entry for using AF_XDP and to stop people copying the code in the > sample application. To use AF_XDP you need libbpf anyway, so I think > it is a good fit for it. The intention is to keep this functionality > lean and mean. +1 >> If above is enough then it falls into XDP category. libbpf >> already has minimal support for XDP and AF_XDP fits right in. Agree, I think given we have XDP support in there already, it would fit to complement the lib with AF_XDP helpers to set up and get raw access to the pkt data. Any framework on top of this providing helper functions to develop applications should be out of scope here and subject to other libraries, DPDK and whatnot. >> But if AF_XDP will keep growing a lot then it would make >> sense to keep the functionality in a separate library >> that minimally depends on libbpf.
On Wed, Dec 12, 2018 at 02:09:48PM +0100, Magnus Karlsson wrote: > This commit adds AF_XDP support to libbpf. The main reason for > this is to facilitate writing applications that use AF_XDP by offering > higher-level APIs that hide many of the details of the AF_XDP > uapi. This is in the same vein as libbpf facilitates XDP adoption by > offering easy-to-use higher level interfaces of XDP > functionality. Hopefully this will facilitate adoption of AF_XDP, make > applications using it simpler and smaller, and finally also make it > possible for applications to benefit from optimizations in the AF_XDP > user space access code. Previously, people just copied and pasted the > code from the sample application into their application, which is not > desirable. > > The interface is composed of two parts: > > * Low-level access interface to the four rings and the packet > * High-level control plane interface for creating and setting > up umems and af_xdp sockets. > > Signed-off-by: Magnus Karlsson <magnus.karlsson@intel.com> ... > diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h > index 5f68d7b..da99203 100644 > --- a/tools/lib/bpf/libbpf.h > +++ b/tools/lib/bpf/libbpf.h may be instead of lib/bpf/libbpf.h the xsk stuff should go to lib/bpf/xsk.h ? > @@ -15,6 +15,7 @@ > #include <stdbool.h> > #include <sys/types.h> // for size_t > #include <linux/bpf.h> > +#include <linux/if_xdp.h> > > #ifdef __cplusplus > extern "C" { > @@ -355,6 +356,98 @@ LIBBPF_API const struct bpf_line_info * > bpf_prog_linfo__lfind(const struct bpf_prog_linfo *prog_linfo, > __u32 insn_off, __u32 nr_skip); > > +/* Do not access these members directly. Use the functions below. */ > +struct xsk_prod_ring { > + __u32 cached_prod; > + __u32 cached_cons; > + __u32 mask; > + __u32 size; > + __u32 *producer; > + __u32 *consumer; > + void *ring; > +}; > + > +/* Do not access these members directly. Use the functions below. */ > +struct xsk_cons_ring { > + __u32 cached_prod; > + __u32 cached_cons; > + __u32 mask; > + __u32 size; > + __u32 *producer; > + __u32 *consumer; > + void *ring; > +}; xsk_prod_ring and xsk_cons_ring have exactly the same members, but they're two different structs? why? May be have one 'struct xsk_ring' ? > + > +static inline __u64 *xsk__get_fill_desc(struct xsk_prod_ring *fill, > + __u64 idx) see tools/lib/bpf/README.rst the main idea is for "objects" use __ to separate class vs method. In this case 'struct xsk_ring' would be an object and the name of the method would be: static inline __u64 *xsk_ring__get_fill_desc(struct xsk_ring *fill, __u64 idx) > +{ > + __u64 *descs = (__u64 *)fill->ring; > + > + return &descs[idx & fill->mask]; > +} > + > +static inline __u64 *xsk__get_completion_desc(struct xsk_cons_ring *comp, > + __u64 idx) > +{ > + __u64 *descs = (__u64 *)comp->ring; > + > + return &descs[idx & comp->mask]; > +} > + > +static inline struct xdp_desc *xsk__get_tx_desc(struct xsk_prod_ring *tx, > + __u64 idx) > +{ > + struct xdp_desc *descs = (struct xdp_desc *)tx->ring; > + > + return &descs[idx & tx->mask]; > +} > + > +static inline struct xdp_desc *xsk__get_rx_desc(struct xsk_cons_ring *rx, > + __u64 idx) > +{ > + struct xdp_desc *descs = (struct xdp_desc *)rx->ring; > + > + return &descs[idx & rx->mask]; > +} > + > +LIBBPF_API size_t xsk__peek_cons(struct xsk_cons_ring *ring, size_t nb, > + __u32 *idx); > +LIBBPF_API void xsk__release_cons(struct xsk_cons_ring *ring); > +LIBBPF_API size_t xsk__reserve_prod(struct xsk_prod_ring *ring, size_t nb, > + __u32 *idx); > +LIBBPF_API void xsk__submit_prod(struct xsk_prod_ring *ring); if we combine the struct names then above could be: LIBBPF_API size_t xsk_ring__reserve(struct xsk_ring *ring, size_t nb, __u32 *idx); LIBBPF_API void xsk_ring__submit(struct xsk_ring *ring); ? > + > +LIBBPF_API void *xsk__get_data(void *umem_area, __u64 addr); > + > +#define XSK__DEFAULT_NUM_DESCS 2048 > +#define XSK__DEFAULT_FRAME_SHIFT 11 /* 2048 bytes */ > +#define XSK__DEFAULT_FRAME_SIZE (1 << XSK__DEFAULT_FRAME_SHIFT) > +#define XSK__DEFAULT_FRAME_HEADROOM 0 > + > +struct xsk_umem_config { > + __u32 fq_size; > + __u32 cq_size; > + __u32 frame_size; > + __u32 frame_headroom; > +}; > + > +struct xsk_xdp_socket_config { > + __u32 rx_size; > + __u32 tx_size; > +}; > + > +/* Set config to XSK_DEFAULT_CONFIG to get the default configuration. */ > +LIBBPF_API int xsk__create_umem(void *umem_area, __u64 size, > + struct xsk_prod_ring *fq, > + struct xsk_cons_ring *cq, > + struct xsk_umem_config *config); this one looks too low level. espcially considering it's usage: umem->fd = xsk__create_umem(buffer, size, &umem->fq, &umem->cq, NULL); may be create an object "struct xsk_umem" ? then api will be: err = xsk_umem__create(buffer, size, NULL) ? > +LIBBPF_API int xsk__create_xdp_socket(int umem_fd, struct xsk_cons_ring *rx, > + struct xsk_prod_ring *tx, > + struct xsk_xdp_socket_config *config); similar concern here. feels that implementation details are leaking into api. The usage of it is: xsk->fd = xsk__create_xdp_socket(umem->fd, &xsk->rx, &xsk->tx, NULL); may be create an object "struct xsk_socket" ?
On Fri, Dec 14, 2018 at 9:25 PM Alexei Starovoitov <alexei.starovoitov@gmail.com> wrote: > > On Wed, Dec 12, 2018 at 02:09:48PM +0100, Magnus Karlsson wrote: > > This commit adds AF_XDP support to libbpf. The main reason for > > this is to facilitate writing applications that use AF_XDP by offering > > higher-level APIs that hide many of the details of the AF_XDP > > uapi. This is in the same vein as libbpf facilitates XDP adoption by > > offering easy-to-use higher level interfaces of XDP > > functionality. Hopefully this will facilitate adoption of AF_XDP, make > > applications using it simpler and smaller, and finally also make it > > possible for applications to benefit from optimizations in the AF_XDP > > user space access code. Previously, people just copied and pasted the > > code from the sample application into their application, which is not > > desirable. > > > > The interface is composed of two parts: > > > > * Low-level access interface to the four rings and the packet > > * High-level control plane interface for creating and setting > > up umems and af_xdp sockets. > > > > Signed-off-by: Magnus Karlsson <magnus.karlsson@intel.com> > ... > > diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h > > index 5f68d7b..da99203 100644 > > --- a/tools/lib/bpf/libbpf.h > > +++ b/tools/lib/bpf/libbpf.h > > may be instead of lib/bpf/libbpf.h the xsk stuff should go to lib/bpf/xsk.h ? Yes. Good idea. > > @@ -15,6 +15,7 @@ > > #include <stdbool.h> > > #include <sys/types.h> // for size_t > > #include <linux/bpf.h> > > +#include <linux/if_xdp.h> > > > > #ifdef __cplusplus > > extern "C" { > > @@ -355,6 +356,98 @@ LIBBPF_API const struct bpf_line_info * > > bpf_prog_linfo__lfind(const struct bpf_prog_linfo *prog_linfo, > > __u32 insn_off, __u32 nr_skip); > > > > +/* Do not access these members directly. Use the functions below. */ > > +struct xsk_prod_ring { > > + __u32 cached_prod; > > + __u32 cached_cons; > > + __u32 mask; > > + __u32 size; > > + __u32 *producer; > > + __u32 *consumer; > > + void *ring; > > +}; > > + > > +/* Do not access these members directly. Use the functions below. */ > > +struct xsk_cons_ring { > > + __u32 cached_prod; > > + __u32 cached_cons; > > + __u32 mask; > > + __u32 size; > > + __u32 *producer; > > + __u32 *consumer; > > + void *ring; > > +}; > > xsk_prod_ring and xsk_cons_ring have exactly the same members, > but they're two different structs? why? > May be have one 'struct xsk_ring' ? They operate on the same ring but represents either the producer or the consumer of that same ring. The reason for this is that I want to make sure that the user gets a compile time error if he or she tries to use for example the function xsk__reserve_prod when user space is a consumer of that ring (and the same kind of argument for xsk__submit_prod, xsk__peek_cons, and xsk__release_cons). If we move to a xsk_ring, we lose this compile time check, or is there another better way of doing this in C without getting double definitions or using hard to read #defines? The only benefit I see with going to xsk_ring is that we get rid of two small inline functions and one struct definition in the header file, but the code size will be the same. Personally I prefer compile time error checking. But let me know what to proceed with. > > + > > +static inline __u64 *xsk__get_fill_desc(struct xsk_prod_ring *fill, > > + __u64 idx) > > see tools/lib/bpf/README.rst > the main idea is for "objects" use __ to separate class vs method. > In this case 'struct xsk_ring' would be an object and > the name of the method would be: > static inline __u64 *xsk_ring__get_fill_desc(struct xsk_ring *fill, __u64 idx) Got it. Will fix. > > +{ > > + __u64 *descs = (__u64 *)fill->ring; > > + > > + return &descs[idx & fill->mask]; > > +} > > + > > +static inline __u64 *xsk__get_completion_desc(struct xsk_cons_ring *comp, > > + __u64 idx) > > +{ > > + __u64 *descs = (__u64 *)comp->ring; > > + > > + return &descs[idx & comp->mask]; > > +} > > + > > +static inline struct xdp_desc *xsk__get_tx_desc(struct xsk_prod_ring *tx, > > + __u64 idx) > > +{ > > + struct xdp_desc *descs = (struct xdp_desc *)tx->ring; > > + > > + return &descs[idx & tx->mask]; > > +} > > + > > +static inline struct xdp_desc *xsk__get_rx_desc(struct xsk_cons_ring *rx, > > + __u64 idx) > > +{ > > + struct xdp_desc *descs = (struct xdp_desc *)rx->ring; > > + > > + return &descs[idx & rx->mask]; > > +} > > + > > +LIBBPF_API size_t xsk__peek_cons(struct xsk_cons_ring *ring, size_t nb, > > + __u32 *idx); > > +LIBBPF_API void xsk__release_cons(struct xsk_cons_ring *ring); > > +LIBBPF_API size_t xsk__reserve_prod(struct xsk_prod_ring *ring, size_t nb, > > + __u32 *idx); > > +LIBBPF_API void xsk__submit_prod(struct xsk_prod_ring *ring); > > if we combine the struct names then above could be: > > LIBBPF_API size_t xsk_ring__reserve(struct xsk_ring *ring, size_t nb, __u32 *idx); > LIBBPF_API void xsk_ring__submit(struct xsk_ring *ring); The implementations of xsk__peek_cons and and xsk__reserve_prod are different because one performs producer operations and the other consumer ones, so they cannot be combined (well at least not without some added if statements and state that would impact performance). Same for xsk__release_cons and xsk__submit_prod. But the static inline functions could be reduced from four to two with an xsk_ring struct. > > > + > > +LIBBPF_API void *xsk__get_data(void *umem_area, __u64 addr); > > + > > +#define XSK__DEFAULT_NUM_DESCS 2048 > > +#define XSK__DEFAULT_FRAME_SHIFT 11 /* 2048 bytes */ > > +#define XSK__DEFAULT_FRAME_SIZE (1 << XSK__DEFAULT_FRAME_SHIFT) > > +#define XSK__DEFAULT_FRAME_HEADROOM 0 > > + > > +struct xsk_umem_config { > > + __u32 fq_size; > > + __u32 cq_size; > > + __u32 frame_size; > > + __u32 frame_headroom; > > +}; > > + > > +struct xsk_xdp_socket_config { > > + __u32 rx_size; > > + __u32 tx_size; > > +}; > > + > > +/* Set config to XSK_DEFAULT_CONFIG to get the default configuration. */ > > +LIBBPF_API int xsk__create_umem(void *umem_area, __u64 size, > > + struct xsk_prod_ring *fq, > > + struct xsk_cons_ring *cq, > > + struct xsk_umem_config *config); > > this one looks too low level. > espcially considering it's usage: > umem->fd = xsk__create_umem(buffer, size, &umem->fq, &umem->cq, NULL); > > may be create an object "struct xsk_umem" ? > then api will be: > err = xsk_umem__create(buffer, size, NULL) ? Makes sense considering the other functions in the library. Will do. > > +LIBBPF_API int xsk__create_xdp_socket(int umem_fd, struct xsk_cons_ring *rx, > > + struct xsk_prod_ring *tx, > > + struct xsk_xdp_socket_config *config); > > similar concern here. feels that implementation details are leaking into api. > The usage of it is: > xsk->fd = xsk__create_xdp_socket(umem->fd, &xsk->rx, &xsk->tx, NULL); > > may be create an object "struct xsk_socket" ? Yes to this also. I will add a function xsk_socket__fd(struct xsk_socket) that returns the fd when the user needs it (for bind, poll and others), in the same spirit as other functionality in the library. Protest if you do not agree. Thanks: Magnus
On Mon, Dec 17, 2018 at 10:12:33AM +0100, Magnus Karlsson wrote: > On Fri, Dec 14, 2018 at 9:25 PM Alexei Starovoitov > <alexei.starovoitov@gmail.com> wrote: > > > > On Wed, Dec 12, 2018 at 02:09:48PM +0100, Magnus Karlsson wrote: > > > This commit adds AF_XDP support to libbpf. The main reason for > > > this is to facilitate writing applications that use AF_XDP by offering > > > higher-level APIs that hide many of the details of the AF_XDP > > > uapi. This is in the same vein as libbpf facilitates XDP adoption by > > > offering easy-to-use higher level interfaces of XDP > > > functionality. Hopefully this will facilitate adoption of AF_XDP, make > > > applications using it simpler and smaller, and finally also make it > > > possible for applications to benefit from optimizations in the AF_XDP > > > user space access code. Previously, people just copied and pasted the > > > code from the sample application into their application, which is not > > > desirable. > > > > > > The interface is composed of two parts: > > > > > > * Low-level access interface to the four rings and the packet > > > * High-level control plane interface for creating and setting > > > up umems and af_xdp sockets. > > > > > > Signed-off-by: Magnus Karlsson <magnus.karlsson@intel.com> > > ... > > > diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h > > > index 5f68d7b..da99203 100644 > > > --- a/tools/lib/bpf/libbpf.h > > > +++ b/tools/lib/bpf/libbpf.h > > > > may be instead of lib/bpf/libbpf.h the xsk stuff should go to lib/bpf/xsk.h ? > > Yes. Good idea. > > > > @@ -15,6 +15,7 @@ > > > #include <stdbool.h> > > > #include <sys/types.h> // for size_t > > > #include <linux/bpf.h> > > > +#include <linux/if_xdp.h> > > > > > > #ifdef __cplusplus > > > extern "C" { > > > @@ -355,6 +356,98 @@ LIBBPF_API const struct bpf_line_info * > > > bpf_prog_linfo__lfind(const struct bpf_prog_linfo *prog_linfo, > > > __u32 insn_off, __u32 nr_skip); > > > > > > +/* Do not access these members directly. Use the functions below. */ > > > +struct xsk_prod_ring { > > > + __u32 cached_prod; > > > + __u32 cached_cons; > > > + __u32 mask; > > > + __u32 size; > > > + __u32 *producer; > > > + __u32 *consumer; > > > + void *ring; > > > +}; > > > + > > > +/* Do not access these members directly. Use the functions below. */ > > > +struct xsk_cons_ring { > > > + __u32 cached_prod; > > > + __u32 cached_cons; > > > + __u32 mask; > > > + __u32 size; > > > + __u32 *producer; > > > + __u32 *consumer; > > > + void *ring; > > > +}; > > > > xsk_prod_ring and xsk_cons_ring have exactly the same members, > > but they're two different structs? why? > > May be have one 'struct xsk_ring' ? > > They operate on the same ring but represents either the producer or > the consumer of that same ring. The reason for this is that I want to > make sure that the user gets a compile time error if he or she tries > to use for example the function xsk__reserve_prod when user space is a > consumer of that ring (and the same kind of argument for > xsk__submit_prod, xsk__peek_cons, and xsk__release_cons). If we move > to a xsk_ring, we lose this compile time check, or is there another > better way of doing this in C without getting double definitions or > using hard to read #defines? The only benefit I see with going to > xsk_ring is that we get rid of two small inline functions and one > struct definition in the header file, but the code size will be the > same. Personally I prefer compile time error checking. But let me know > what to proceed with. how about the following? struct xsk_ring {...}; struct xsk_prod { struct xsk_ring r; }; struct xsk_cons { struct xsk_ring r; }; and compiler will warn when xsk_prod is used instead of xsk_cons. The methods can be called xsk_prod__* and xsk_cons__* Also do 'producer' and 'consumer' names really fit? Typically producer and consumer are two sides of the single ring.
diff --git a/tools/include/uapi/linux/if_xdp.h b/tools/include/uapi/linux/if_xdp.h new file mode 100644 index 0000000..caed8b1 --- /dev/null +++ b/tools/include/uapi/linux/if_xdp.h @@ -0,0 +1,78 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* + * if_xdp: XDP socket user-space interface + * Copyright(c) 2018 Intel Corporation. + * + * Author(s): Björn Töpel <bjorn.topel@intel.com> + * Magnus Karlsson <magnus.karlsson@intel.com> + */ + +#ifndef _LINUX_IF_XDP_H +#define _LINUX_IF_XDP_H + +#include <linux/types.h> + +/* Options for the sxdp_flags field */ +#define XDP_SHARED_UMEM (1 << 0) +#define XDP_COPY (1 << 1) /* Force copy-mode */ +#define XDP_ZEROCOPY (1 << 2) /* Force zero-copy mode */ + +struct sockaddr_xdp { + __u16 sxdp_family; + __u16 sxdp_flags; + __u32 sxdp_ifindex; + __u32 sxdp_queue_id; + __u32 sxdp_shared_umem_fd; +}; + +struct xdp_ring_offset { + __u64 producer; + __u64 consumer; + __u64 desc; +}; + +struct xdp_mmap_offsets { + struct xdp_ring_offset rx; + struct xdp_ring_offset tx; + struct xdp_ring_offset fr; /* Fill */ + struct xdp_ring_offset cr; /* Completion */ +}; + +/* XDP socket options */ +#define XDP_MMAP_OFFSETS 1 +#define XDP_RX_RING 2 +#define XDP_TX_RING 3 +#define XDP_UMEM_REG 4 +#define XDP_UMEM_FILL_RING 5 +#define XDP_UMEM_COMPLETION_RING 6 +#define XDP_STATISTICS 7 + +struct xdp_umem_reg { + __u64 addr; /* Start of packet data area */ + __u64 len; /* Length of packet data area */ + __u32 chunk_size; + __u32 headroom; +}; + +struct xdp_statistics { + __u64 rx_dropped; /* Dropped for reasons other than invalid desc */ + __u64 rx_invalid_descs; /* Dropped due to invalid descriptor */ + __u64 tx_invalid_descs; /* Dropped due to invalid descriptor */ +}; + +/* Pgoff for mmaping the rings */ +#define XDP_PGOFF_RX_RING 0 +#define XDP_PGOFF_TX_RING 0x80000000 +#define XDP_UMEM_PGOFF_FILL_RING 0x100000000ULL +#define XDP_UMEM_PGOFF_COMPLETION_RING 0x180000000ULL + +/* Rx/Tx descriptor */ +struct xdp_desc { + __u64 addr; + __u32 len; + __u32 options; +}; + +/* UMEM descriptor is __u64 */ + +#endif /* _LINUX_IF_XDP_H */ diff --git a/tools/lib/bpf/Build b/tools/lib/bpf/Build index 197b40f..91780e8 100644 --- a/tools/lib/bpf/Build +++ b/tools/lib/bpf/Build @@ -1 +1 @@ -libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o str_error.o netlink.o bpf_prog_linfo.o +libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o str_error.o netlink.o bpf_prog_linfo.o xsk.o diff --git a/tools/lib/bpf/Makefile b/tools/lib/bpf/Makefile index 34d9c36..ddaa147 100644 --- a/tools/lib/bpf/Makefile +++ b/tools/lib/bpf/Makefile @@ -179,6 +179,9 @@ $(BPF_IN): force elfdep bpfdep @(test -f ../../include/uapi/linux/if_link.h -a -f ../../../include/uapi/linux/if_link.h && ( \ (diff -B ../../include/uapi/linux/if_link.h ../../../include/uapi/linux/if_link.h >/dev/null) || \ echo "Warning: Kernel ABI header at 'tools/include/uapi/linux/if_link.h' differs from latest version at 'include/uapi/linux/if_link.h'" >&2 )) || true + @(test -f ../../include/uapi/linux/if_xdp.h -a -f ../../../include/uapi/linux/if_xdp.h && ( \ + (diff -B ../../include/uapi/linux/if_xdp.h ../../../include/uapi/linux/if_xdp.h >/dev/null) || \ + echo "Warning: Kernel ABI header at 'tools/include/uapi/linux/if_xdp.h' differs from latest version at 'include/uapi/linux/if_xdp.h'" >&2 )) || true $(Q)$(MAKE) $(build)=libbpf $(OUTPUT)libbpf.so: $(BPF_IN) @@ -189,7 +192,7 @@ $(OUTPUT)libbpf.a: $(BPF_IN) $(QUIET_LINK)$(RM) $@; $(AR) rcs $@ $^ $(OUTPUT)test_libbpf: test_libbpf.cpp $(OUTPUT)libbpf.a - $(QUIET_LINK)$(CXX) $^ -lelf -o $@ + $(QUIET_LINK)$(CXX) $(INCLUDES) $^ -lelf -o $@ check: check_abi diff --git a/tools/lib/bpf/README.rst b/tools/lib/bpf/README.rst index 056f383..20241cc 100644 --- a/tools/lib/bpf/README.rst +++ b/tools/lib/bpf/README.rst @@ -9,7 +9,7 @@ described here. It's recommended to follow these conventions whenever a new function or type is added to keep libbpf API clean and consistent. All types and functions provided by libbpf API should have one of the -following prefixes: ``bpf_``, ``btf_``, ``libbpf_``. +following prefixes: ``bpf_``, ``btf_``, ``libbpf_``, ``xsk_``. System call wrappers -------------------- @@ -62,6 +62,15 @@ Auxiliary functions and types that don't fit well in any of categories described above should have ``libbpf_`` prefix, e.g. ``libbpf_get_error`` or ``libbpf_prog_type_by_name``. +AF_XDP functions +------------------- + +AF_XDP functions should have ``xsk_`` prefix, e.g. ``xsk__get_data`` +or ``xsk__create_umem``. The interface consists of both low-level ring +access functions and high-level configuration functions. These can be +mixed and matched. Note that these functions are not reentrant for +performance reasons. + libbpf ABI ========== diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h index 5f68d7b..da99203 100644 --- a/tools/lib/bpf/libbpf.h +++ b/tools/lib/bpf/libbpf.h @@ -15,6 +15,7 @@ #include <stdbool.h> #include <sys/types.h> // for size_t #include <linux/bpf.h> +#include <linux/if_xdp.h> #ifdef __cplusplus extern "C" { @@ -355,6 +356,98 @@ LIBBPF_API const struct bpf_line_info * bpf_prog_linfo__lfind(const struct bpf_prog_linfo *prog_linfo, __u32 insn_off, __u32 nr_skip); +/* Do not access these members directly. Use the functions below. */ +struct xsk_prod_ring { + __u32 cached_prod; + __u32 cached_cons; + __u32 mask; + __u32 size; + __u32 *producer; + __u32 *consumer; + void *ring; +}; + +/* Do not access these members directly. Use the functions below. */ +struct xsk_cons_ring { + __u32 cached_prod; + __u32 cached_cons; + __u32 mask; + __u32 size; + __u32 *producer; + __u32 *consumer; + void *ring; +}; + +static inline __u64 *xsk__get_fill_desc(struct xsk_prod_ring *fill, + __u64 idx) +{ + __u64 *descs = (__u64 *)fill->ring; + + return &descs[idx & fill->mask]; +} + +static inline __u64 *xsk__get_completion_desc(struct xsk_cons_ring *comp, + __u64 idx) +{ + __u64 *descs = (__u64 *)comp->ring; + + return &descs[idx & comp->mask]; +} + +static inline struct xdp_desc *xsk__get_tx_desc(struct xsk_prod_ring *tx, + __u64 idx) +{ + struct xdp_desc *descs = (struct xdp_desc *)tx->ring; + + return &descs[idx & tx->mask]; +} + +static inline struct xdp_desc *xsk__get_rx_desc(struct xsk_cons_ring *rx, + __u64 idx) +{ + struct xdp_desc *descs = (struct xdp_desc *)rx->ring; + + return &descs[idx & rx->mask]; +} + +LIBBPF_API size_t xsk__peek_cons(struct xsk_cons_ring *ring, size_t nb, + __u32 *idx); +LIBBPF_API void xsk__release_cons(struct xsk_cons_ring *ring); +LIBBPF_API size_t xsk__reserve_prod(struct xsk_prod_ring *ring, size_t nb, + __u32 *idx); +LIBBPF_API void xsk__submit_prod(struct xsk_prod_ring *ring); + +LIBBPF_API void *xsk__get_data(void *umem_area, __u64 addr); + +#define XSK__DEFAULT_NUM_DESCS 2048 +#define XSK__DEFAULT_FRAME_SHIFT 11 /* 2048 bytes */ +#define XSK__DEFAULT_FRAME_SIZE (1 << XSK__DEFAULT_FRAME_SHIFT) +#define XSK__DEFAULT_FRAME_HEADROOM 0 + +struct xsk_umem_config { + __u32 fq_size; + __u32 cq_size; + __u32 frame_size; + __u32 frame_headroom; +}; + +struct xsk_xdp_socket_config { + __u32 rx_size; + __u32 tx_size; +}; + +/* Set config to XSK_DEFAULT_CONFIG to get the default configuration. */ +LIBBPF_API int xsk__create_umem(void *umem_area, __u64 size, + struct xsk_prod_ring *fq, + struct xsk_cons_ring *cq, + struct xsk_umem_config *config); +LIBBPF_API int xsk__create_xdp_socket(int umem_fd, struct xsk_cons_ring *rx, + struct xsk_prod_ring *tx, + struct xsk_xdp_socket_config *config); +/* Returns 0 for success. */ +LIBBPF_API int xsk__delete_umem(int fd); +LIBBPF_API int xsk__delete_xdp_socket(int fd); + #ifdef __cplusplus } /* extern "C" */ #endif diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map index cd02cd4..ae4cc0d 100644 --- a/tools/lib/bpf/libbpf.map +++ b/tools/lib/bpf/libbpf.map @@ -121,6 +121,15 @@ LIBBPF_0.0.1 { libbpf_prog_type_by_name; libbpf_set_print; libbpf_strerror; + xsk__peek_cons; + xsk__release_cons; + xsk__reserve_prod; + xsk__submit_prod; + xsk__get_data; + xsk__create_umem; + xsk__create_xdp_socket; + xsk__delete_umem; + xsk__delete_xdp_socket; local: *; }; diff --git a/tools/lib/bpf/xsk.c b/tools/lib/bpf/xsk.c new file mode 100644 index 0000000..60cd896 --- /dev/null +++ b/tools/lib/bpf/xsk.c @@ -0,0 +1,568 @@ +// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) + +/* + * AF_XDP user-space access library. + * + * Copyright(c) 2018 Intel Corporation. + */ + +#include <errno.h> +#include <stdlib.h> +#include <unistd.h> +#include <asm/barrier.h> +#include <linux/compiler.h> +#include <linux/if_xdp.h> +#include <linux/list.h> +#include <sys/mman.h> +#include <sys/socket.h> +#include <sys/types.h> + +#include "libbpf.h" + +#ifndef SOL_XDP + #define SOL_XDP 283 +#endif + +#ifndef AF_XDP + #define AF_XDP 44 +#endif + +#ifndef PF_XDP + #define PF_XDP AF_XDP +#endif + +/* This has to be a power of 2 for performance reasons. */ +#define HASH_TABLE_ENTRIES 128 + +struct xsk_umem_info { + struct xsk_prod_ring *fq; + struct xsk_cons_ring *cq; + char *umem_area; + struct list_head list; + struct xsk_umem_config config; + int fd; + int refcount; +}; + +struct xsk_xdp_socket_info { + struct xsk_cons_ring *rx; + struct xsk_prod_ring *tx; + __u64 outstanding_tx; + struct list_head list; + struct xsk_umem_info *umem; + struct xsk_xdp_socket_config config; + int fd; +}; + +static struct xsk_xdp_socket_info *xsk_hash_table[HASH_TABLE_ENTRIES]; +static struct xsk_umem_info *umem_hash_table[HASH_TABLE_ENTRIES]; + +/* For 32-bit systems, we need to use mmap2 as the offsets are 64-bit. + * Unfortunately, it is not part of glibc. + */ +static inline void *xsk_mmap(void *addr, size_t length, int prot, int flags, + int fd, __u64 offset) +{ +#ifdef __NR_mmap2 + unsigned int page_shift = __builtin_ffs(getpagesize()) - 1; + long ret = syscall(__NR_mmap2, addr, length, prot, flags, fd, + (off_t)(offset >> page_shift)); + + return (void *)ret; +#else + return mmap(addr, length, prot, flags, fd, offset); +#endif +} + +static __u32 xsk_prod_nb_free(struct xsk_prod_ring *r, __u32 nb) +{ + __u32 free_entries = r->cached_cons - r->cached_prod; + + if (free_entries >= nb) + return free_entries; + + /* Refresh the local tail pointer. + * cached_cons is r->size bigger than the real consumer pointer so + * that this addition can be avoided in the more frequently + * executed code that computs free_entries in the beginning of + * this function. Without this optimization it whould have been + * free_entries = r->cached_prod - r->cached_cons + r->size. + */ + r->cached_cons = *r->consumer + r->size; + + return r->cached_cons - r->cached_prod; +} + +static __u32 xsk_cons_nb_avail(struct xsk_cons_ring *r, __u32 nb) +{ + __u32 entries = r->cached_prod - r->cached_cons; + + if (entries == 0) { + r->cached_prod = *r->producer; + entries = r->cached_prod - r->cached_cons; + } + + return (entries > nb) ? nb : entries; +} + +size_t xsk__reserve_prod(struct xsk_prod_ring *prod, size_t nb, __u32 *idx) +{ + if (unlikely(xsk_prod_nb_free(prod, nb) < nb)) + return 0; + + *idx = prod->cached_prod; + prod->cached_prod += nb; + + return nb; +} + +void xsk__submit_prod(struct xsk_prod_ring *prod) +{ + /* Make sure everything has been written to the ring before signalling + * this to the kernel. + */ + smp_wmb(); + + *prod->producer = prod->cached_prod; +} + +size_t xsk__peek_cons(struct xsk_cons_ring *cons, size_t nb, __u32 *idx) +{ + size_t entries = xsk_cons_nb_avail(cons, nb); + + if (likely(entries > 0)) { + /* Make sure we do not speculatively read the data before + * we have received the packet buffers from the ring. + */ + smp_rmb(); + + *idx = cons->cached_cons; + cons->cached_cons += entries; + } + + return entries; +} + +void xsk__release_cons(struct xsk_cons_ring *cons) +{ + *cons->consumer = cons->cached_cons; +} + +void *xsk__get_data(void *umem_area, __u64 addr) +{ + return &((char *)umem_area)[addr]; +} + +static bool xsk_page_aligned(void *buffer) +{ + unsigned long addr = (unsigned long)buffer; + + return !(addr & (getpagesize() - 1)); +} + +/* Since the file descriptors are generally allocated sequentially, and also + * for performance reasons, we pick the simplest possible hash function: + * just a single "and" operation (from the modulo operator). + */ +static void xsk_hash_insert_umem(int fd, struct xsk_umem_info *umem) +{ + struct xsk_umem_info *umem_in_hash = + umem_hash_table[fd % HASH_TABLE_ENTRIES]; + + if (umem_in_hash) { + list_add_tail(&umem->list, &umem_in_hash->list); + return; + } + + INIT_LIST_HEAD(&umem->list); + umem_hash_table[fd % HASH_TABLE_ENTRIES] = umem; +} + +static struct xsk_umem_info *xsk_hash_find_umem(int fd) +{ + struct xsk_umem_info *umem = umem_hash_table[fd % HASH_TABLE_ENTRIES]; + + while (umem && umem->fd != fd) + umem = list_next_entry(umem, list); + + return umem; +} + +static void xsk_hash_remove_umem(int fd) +{ + struct xsk_umem_info *umem = umem_hash_table[fd % HASH_TABLE_ENTRIES]; + + while (umem && umem->fd != fd) + umem = list_next_entry(umem, list); + + if (umem) { + if (list_empty(&umem->list)) { + umem_hash_table[fd % HASH_TABLE_ENTRIES] = NULL; + return; + } + + if (umem == umem_hash_table[fd % HASH_TABLE_ENTRIES]) + umem_hash_table[fd % HASH_TABLE_ENTRIES] = + list_next_entry(umem, list); + list_del(&umem->list); + } +} + +static void xsk_hash_insert_xdp_socket(int fd, struct xsk_xdp_socket_info *xsk) +{ + struct xsk_xdp_socket_info *xsk_in_hash = + xsk_hash_table[fd % HASH_TABLE_ENTRIES]; + + if (xsk_in_hash) { + list_add_tail(&xsk->list, &xsk_in_hash->list); + return; + } + + INIT_LIST_HEAD(&xsk->list); + xsk_hash_table[fd % HASH_TABLE_ENTRIES] = xsk; +} + +static struct xsk_xdp_socket_info *xsk_hash_find_xdp_socket(int fd) +{ + struct xsk_xdp_socket_info *xsk = + xsk_hash_table[fd % HASH_TABLE_ENTRIES]; + + while (xsk && xsk->fd != fd) + xsk = list_next_entry(xsk, list); + + return xsk; +} + +static void xsk_hash_remove_xdp_socket(int fd) +{ + struct xsk_xdp_socket_info *xsk = + xsk_hash_table[fd % HASH_TABLE_ENTRIES]; + + while (xsk && xsk->fd != fd) + xsk = list_next_entry(xsk, list); + + if (xsk) { + if (list_empty(&xsk->list)) { + xsk_hash_table[fd % HASH_TABLE_ENTRIES] = NULL; + return; + } + + if (xsk == xsk_hash_table[fd % HASH_TABLE_ENTRIES]) + xsk_hash_table[fd % HASH_TABLE_ENTRIES] = + list_next_entry(xsk, list); + list_del(&xsk->list); + } +} + +static void xsk_set_umem_config(struct xsk_umem_config *config, + struct xsk_umem_config *usr_config) +{ + if (!usr_config) { + config->fq_size = XSK__DEFAULT_NUM_DESCS; + config->cq_size = XSK__DEFAULT_NUM_DESCS; + config->frame_size = XSK__DEFAULT_FRAME_SIZE; + config->frame_headroom = XSK__DEFAULT_FRAME_HEADROOM; + return; + } + + config->fq_size = usr_config->fq_size; + config->cq_size = usr_config->cq_size; + config->frame_size = usr_config->frame_size; + config->frame_headroom = usr_config->frame_headroom; +} + +static void xsk_set_xdp_socket_config(struct xsk_xdp_socket_config *config, + struct xsk_xdp_socket_config *usr_config) +{ + if (!usr_config) { + config->rx_size = XSK__DEFAULT_NUM_DESCS; + config->tx_size = XSK__DEFAULT_NUM_DESCS; + return; + } + + config->rx_size = usr_config->rx_size; + config->tx_size = usr_config->tx_size; +} + +int xsk__create_umem(void *umem_area, __u64 size, struct xsk_prod_ring *fq, + struct xsk_cons_ring *cq, + struct xsk_umem_config *usr_config) +{ + struct xdp_mmap_offsets off; + struct xsk_umem_info *umem; + struct xdp_umem_reg mr; + socklen_t optlen; + int err, fd; + void *map; + + if (!umem_area) + return -EFAULT; + if (!size && !xsk_page_aligned(umem_area)) + return -EINVAL; + + fd = socket(AF_XDP, SOCK_RAW, 0); + if (fd < 0) + return -errno; + + umem = calloc(1, sizeof(*umem)); + if (!umem) { + err = -ENOMEM; + goto out_socket; + } + + umem->umem_area = umem_area; + umem->fd = fd; + xsk_set_umem_config(&umem->config, usr_config); + + mr.addr = (uintptr_t)umem_area; + mr.len = size; + mr.chunk_size = umem->config.frame_size; + mr.headroom = umem->config.frame_headroom; + + err = setsockopt(fd, SOL_XDP, XDP_UMEM_REG, &mr, sizeof(mr)); + if (err) { + err = -errno; + goto out_umem_alloc; + } + err = setsockopt(fd, SOL_XDP, XDP_UMEM_FILL_RING, + &umem->config.fq_size, sizeof(umem->config.fq_size)); + if (err) { + err = -errno; + goto out_umem_alloc; + } + err = setsockopt(fd, SOL_XDP, XDP_UMEM_COMPLETION_RING, + &umem->config.cq_size, sizeof(umem->config.cq_size)); + if (err) { + err = -errno; + goto out_umem_alloc; + } + + optlen = sizeof(off); + err = getsockopt(fd, SOL_XDP, XDP_MMAP_OFFSETS, &off, &optlen); + if (err) { + err = -errno; + goto out_umem_alloc; + } + + map = xsk_mmap(NULL, off.fr.desc + umem->config.fq_size * sizeof(__u64), + PROT_READ | PROT_WRITE, MAP_SHARED | MAP_POPULATE, + fd, XDP_UMEM_PGOFF_FILL_RING); + if (map == MAP_FAILED) { + err = -errno; + goto out_umem_alloc; + } + + umem->fq = fq; + fq->mask = umem->config.fq_size - 1; + fq->size = umem->config.fq_size; + fq->producer = map + off.fr.producer; + fq->consumer = map + off.fr.consumer; + fq->ring = map + off.fr.desc; + fq->cached_cons = umem->config.fq_size; + + map = xsk_mmap(NULL, off.cr.desc + umem->config.cq_size * sizeof(__u64), + PROT_READ | PROT_WRITE, MAP_SHARED | MAP_POPULATE, + fd, XDP_UMEM_PGOFF_COMPLETION_RING); + if (map == MAP_FAILED) { + err = -errno; + goto out_mmap; + } + + umem->cq = cq; + cq->mask = umem->config.cq_size - 1; + cq->size = umem->config.cq_size; + cq->producer = map + off.cr.producer; + cq->consumer = map + off.cr.consumer; + cq->ring = map + off.cr.desc; + + xsk_hash_insert_umem(fd, umem); + return fd; + +out_mmap: + munmap(umem->fq, off.fr.desc + umem->config.fq_size * sizeof(__u64)); +out_umem_alloc: + free(umem); +out_socket: + close(fd); + return err; +} + +int xsk__create_xdp_socket(int umem_fd, struct xsk_cons_ring *rx, + struct xsk_prod_ring *tx, + struct xsk_xdp_socket_config *usr_config) +{ + struct xsk_xdp_socket_info *xsk; + struct xdp_mmap_offsets off; + struct xsk_umem_info *umem; + socklen_t optlen; + int err, fd; + void *map; + + umem = xsk_hash_find_umem(umem_fd); + if (!umem) + return -EBADF; + + if (umem->refcount++ == 0) { + fd = umem_fd; + } else { + fd = socket(AF_XDP, SOCK_RAW, 0); + if (fd < 0) + return -errno; + } + + xsk = calloc(1, sizeof(*xsk)); + if (!xsk) { + err = -ENOMEM; + goto out_socket; + } + + xsk->fd = fd; + xsk->outstanding_tx = 0; + xsk_set_xdp_socket_config(&xsk->config, usr_config); + + if (rx) { + err = setsockopt(fd, SOL_XDP, XDP_RX_RING, + &xsk->config.rx_size, + sizeof(xsk->config.rx_size)); + if (err) { + err = -errno; + goto out_xsk_alloc; + } + } + if (tx) { + err = setsockopt(fd, SOL_XDP, XDP_TX_RING, + &xsk->config.tx_size, + sizeof(xsk->config.tx_size)); + if (err) { + err = -errno; + goto out_xsk_alloc; + } + } + + optlen = sizeof(off); + err = getsockopt(fd, SOL_XDP, XDP_MMAP_OFFSETS, &off, &optlen); + if (err) { + err = -errno; + goto out_xsk_alloc; + } + + if (rx) { + map = xsk_mmap(NULL, off.rx.desc + + xsk->config.rx_size * sizeof(struct xdp_desc), + PROT_READ | PROT_WRITE, + MAP_SHARED | MAP_POPULATE, + fd, XDP_PGOFF_RX_RING); + if (map == MAP_FAILED) { + err = -errno; + goto out_xsk_alloc; + } + + rx->mask = xsk->config.rx_size - 1; + rx->size = xsk->config.rx_size; + rx->producer = map + off.rx.producer; + rx->consumer = map + off.rx.consumer; + rx->ring = map + off.rx.desc; + } + xsk->rx = rx; + + if (tx) { + map = xsk_mmap(NULL, off.tx.desc + + xsk->config.tx_size * sizeof(struct xdp_desc), + PROT_READ | PROT_WRITE, + MAP_SHARED | MAP_POPULATE, + fd, XDP_PGOFF_TX_RING); + if (map == MAP_FAILED) { + err = -errno; + goto out_mmap; + } + + tx->mask = xsk->config.tx_size - 1; + tx->size = xsk->config.tx_size; + tx->producer = map + off.tx.producer; + tx->consumer = map + off.tx.consumer; + tx->ring = map + off.tx.desc; + tx->cached_cons = xsk->config.tx_size; + } + xsk->tx = tx; + + xsk_hash_insert_xdp_socket(fd, xsk); + return fd; + +out_mmap: + if (rx) + munmap(xsk->rx, + off.rx.desc + + xsk->config.rx_size * sizeof(struct xdp_desc)); +out_xsk_alloc: + free(xsk); +out_socket: + if (--umem->refcount) + close(fd); + return err; +} + +int xsk__delete_umem(int fd) +{ + struct xdp_mmap_offsets off; + struct xsk_umem_info *umem; + socklen_t optlen; + int err; + + umem = xsk_hash_find_umem(fd); + if (!umem) + return -EBADF; + + if (umem->refcount) + return -EBUSY; + + optlen = sizeof(off); + err = getsockopt(fd, SOL_XDP, XDP_MMAP_OFFSETS, &off, &optlen); + if (!err) { + munmap(umem->fq->ring, + off.fr.desc + umem->config.fq_size * sizeof(__u64)); + munmap(umem->cq->ring, + off.cr.desc + umem->config.cq_size * sizeof(__u64)); + } + + xsk_hash_remove_umem(fd); + close(fd); + free(umem); + + return 0; +} + +int xsk__delete_xdp_socket(int fd) +{ + struct xsk_xdp_socket_info *xsk; + struct xdp_mmap_offsets off; + socklen_t optlen; + int err; + + xsk = xsk_hash_find_xdp_socket(fd); + if (!xsk) + return -EBADF; + + optlen = sizeof(off); + err = getsockopt(fd, SOL_XDP, XDP_MMAP_OFFSETS, &off, &optlen); + if (!err) { + if (xsk->rx) + munmap(xsk->rx->ring, + off.rx.desc + + xsk->config.rx_size * sizeof(struct xdp_desc)); + if (xsk->tx) + munmap(xsk->tx->ring, + off.tx.desc + + xsk->config.tx_size * sizeof(struct xdp_desc)); + } + + xsk->umem->refcount--; + xsk_hash_remove_xdp_socket(fd); + /* Do not close the fd that also has an associated umem connected + * to it. + */ + if (xsk->fd != xsk->umem->fd) + close(fd); + free(xsk); + + return 0; +}
This commit adds AF_XDP support to libbpf. The main reason for this is to facilitate writing applications that use AF_XDP by offering higher-level APIs that hide many of the details of the AF_XDP uapi. This is in the same vein as libbpf facilitates XDP adoption by offering easy-to-use higher level interfaces of XDP functionality. Hopefully this will facilitate adoption of AF_XDP, make applications using it simpler and smaller, and finally also make it possible for applications to benefit from optimizations in the AF_XDP user space access code. Previously, people just copied and pasted the code from the sample application into their application, which is not desirable. The interface is composed of two parts: * Low-level access interface to the four rings and the packet * High-level control plane interface for creating and setting up umems and af_xdp sockets. Signed-off-by: Magnus Karlsson <magnus.karlsson@intel.com> --- tools/include/uapi/linux/if_xdp.h | 78 ++++++ tools/lib/bpf/Build | 2 +- tools/lib/bpf/Makefile | 5 +- tools/lib/bpf/README.rst | 11 +- tools/lib/bpf/libbpf.h | 93 +++++++ tools/lib/bpf/libbpf.map | 9 + tools/lib/bpf/xsk.c | 568 ++++++++++++++++++++++++++++++++++++++ 7 files changed, 763 insertions(+), 3 deletions(-) create mode 100644 tools/include/uapi/linux/if_xdp.h create mode 100644 tools/lib/bpf/xsk.c