[ovs-dev] netdev-afxdp: Update memory locking limits unconditionally.
diff mbox series

Message ID 20191009142331.12891-1-i.maximets@ovn.org
State Accepted
Headers show
Series
  • [ovs-dev] netdev-afxdp: Update memory locking limits unconditionally.
Related show

Commit Message

Ilya Maximets Oct. 9, 2019, 2:23 p.m. UTC
Any type of AF_XDP socket in all modes implies creation of BPF map of
type BPF_MAP_TYPE_XSKMAP.  This leads to BPF_MAP_CREATE syscall and
subsequently 'xsk_map_alloc()' function that will charge required
memory from the memlock limit and fail with EPERM if we're trying
to allocate more.

On my system with 64K bytes of max locked memory by default, OVS
frequently starts to fail after addition of 3rd afxdp port in SKB
mode:

  netdev_afxdp|ERR|xsk_socket__create failed (Operation not permitted)
                   mode: SKB qid: 0

Fixes: 0de1b425962d ("netdev-afxdp: add new netdev type for AF_XDP.")
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
---
 lib/netdev-afxdp.c | 19 ++++++-------------
 1 file changed, 6 insertions(+), 13 deletions(-)

Comments

William Tu Oct. 10, 2019, 6:29 p.m. UTC | #1
On Wed, Oct 09, 2019 at 04:23:31PM +0200, Ilya Maximets wrote:
> Any type of AF_XDP socket in all modes implies creation of BPF map of
> type BPF_MAP_TYPE_XSKMAP.  This leads to BPF_MAP_CREATE syscall and
> subsequently 'xsk_map_alloc()' function that will charge required
> memory from the memlock limit and fail with EPERM if we're trying
> to allocate more.
> 
> On my system with 64K bytes of max locked memory by default, OVS
> frequently starts to fail after addition of 3rd afxdp port in SKB
> mode:
> 
>   netdev_afxdp|ERR|xsk_socket__create failed (Operation not permitted)
>                    mode: SKB qid: 0
> 
> Fixes: 0de1b425962d ("netdev-afxdp: add new netdev type for AF_XDP.")
> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
> ---
Good catch!

I applied to master.

--William
Ilya Maximets Oct. 11, 2019, 11:17 a.m. UTC | #2
On 10.10.2019 20:29, William Tu wrote:
> On Wed, Oct 09, 2019 at 04:23:31PM +0200, Ilya Maximets wrote:
>> Any type of AF_XDP socket in all modes implies creation of BPF map of
>> type BPF_MAP_TYPE_XSKMAP.  This leads to BPF_MAP_CREATE syscall and
>> subsequently 'xsk_map_alloc()' function that will charge required
>> memory from the memlock limit and fail with EPERM if we're trying
>> to allocate more.
>>
>> On my system with 64K bytes of max locked memory by default, OVS
>> frequently starts to fail after addition of 3rd afxdp port in SKB
>> mode:
>>
>>    netdev_afxdp|ERR|xsk_socket__create failed (Operation not permitted)
>>                     mode: SKB qid: 0
>>
>> Fixes: 0de1b425962d ("netdev-afxdp: add new netdev type for AF_XDP.")
>> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
>> ---
> Good catch!
> 
> I applied to master.
> 
> --William
> 

Thanks!
This also needs to be backported to 2.12.

Best regards, Ilya Maximets.
William Tu Oct. 11, 2019, 4:13 p.m. UTC | #3
On Fri, Oct 11, 2019 at 01:17:59PM +0200, Ilya Maximets wrote:
> On 10.10.2019 20:29, William Tu wrote:
> >On Wed, Oct 09, 2019 at 04:23:31PM +0200, Ilya Maximets wrote:
> >>Any type of AF_XDP socket in all modes implies creation of BPF map of
> >>type BPF_MAP_TYPE_XSKMAP.  This leads to BPF_MAP_CREATE syscall and
> >>subsequently 'xsk_map_alloc()' function that will charge required
> >>memory from the memlock limit and fail with EPERM if we're trying
> >>to allocate more.
> >>
> >>On my system with 64K bytes of max locked memory by default, OVS
> >>frequently starts to fail after addition of 3rd afxdp port in SKB
> >>mode:
> >>
> >>   netdev_afxdp|ERR|xsk_socket__create failed (Operation not permitted)
> >>                    mode: SKB qid: 0
> >>
> >>Fixes: 0de1b425962d ("netdev-afxdp: add new netdev type for AF_XDP.")
> >>Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
> >>---
> >Good catch!
> >
> >I applied to master.
> >
> >--William
> >
> 
> Thanks!
> This also needs to be backported to 2.12.
> 
> Best regards, Ilya Maximets.

Done, thanks
William

Patch
diff mbox series

diff --git a/lib/netdev-afxdp.c b/lib/netdev-afxdp.c
index 6e0180327..50ac1c5a5 100644
--- a/lib/netdev-afxdp.c
+++ b/lib/netdev-afxdp.c
@@ -524,19 +524,12 @@  netdev_afxdp_reconfigure(struct netdev *netdev)
     netdev->n_rxq = dev->requested_n_rxq;
     netdev->n_txq = netdev->n_rxq;
 
-    if (dev->requested_xdpmode == XDP_ZEROCOPY) {
-        dev->xdpmode = XDP_ZEROCOPY;
-        VLOG_INFO("AF_XDP device %s in DRV mode.", netdev_get_name(netdev));
-        if (setrlimit(RLIMIT_MEMLOCK, &r)) {
-            VLOG_ERR("ERROR: setrlimit(RLIMIT_MEMLOCK): %s",
-                      ovs_strerror(errno));
-        }
-    } else {
-        dev->xdpmode = XDP_COPY;
-        VLOG_INFO("AF_XDP device %s in SKB mode.", netdev_get_name(netdev));
-        /* TODO: set rlimit back to previous value
-         * when no device is in DRV mode.
-         */
+    dev->xdpmode = dev->requested_xdpmode;
+    VLOG_INFO("%s: Setting XDP mode to %s.", netdev_get_name(netdev),
+              dev->xdpmode == XDP_ZEROCOPY ? "DRV" : "SKB");
+
+    if (setrlimit(RLIMIT_MEMLOCK, &r)) {
+        VLOG_ERR("setrlimit(RLIMIT_MEMLOCK) failed: %s", ovs_strerror(errno));
     }
 
     err = xsk_configure_all(netdev);