diff mbox series

[ovs-dev,v2,1/3] system-dpdk: cleanup stale hugepage files after tests

Message ID 3435decd0cc431e438bedfbcb92c0619e15e7489.1589605823.git.gmuthukr@redhat.com
State New
Headers show
Series system-dpdk: add userspace-tso tests | expand

Commit Message

Gowrishankar Muthukrishnan May 16, 2020, 5:53 a.m. UTC
After dpdk tests completes, cleaning up hugepage map files
created by tests is helpful to release used memory into
hugepage memory allocator.

Signed-off-by: Gowrishankar Muthukrishnan <gmuthukr@redhat.com>
---
 tests/system-dpdk-macros.at | 13 +++++++++++++
 tests/system-dpdk.at        |  7 +++++++
 2 files changed, 20 insertions(+)

Comments

Flavio Leitner May 18, 2020, 6:14 p.m. UTC | #1
On Sat, May 16, 2020 at 11:23:51AM +0530, Gowrishankar Muthukrishnan wrote:
> After dpdk tests completes, cleaning up hugepage map files
> created by tests is helpful to release used memory into
> hugepage memory allocator.
> 
> Signed-off-by: Gowrishankar Muthukrishnan <gmuthukr@redhat.com>
> ---
>  tests/system-dpdk-macros.at | 13 +++++++++++++
>  tests/system-dpdk.at        |  7 +++++++
>  2 files changed, 20 insertions(+)
> 
> diff --git a/tests/system-dpdk-macros.at b/tests/system-dpdk-macros.at
> index c6708ca..d3a3aea 100644
> --- a/tests/system-dpdk-macros.at
> +++ b/tests/system-dpdk-macros.at
> @@ -63,3 +63,16 @@ m4_define([OVS_DPDK_START],
>     AT_CAPTURE_FILE([ovs-vswitchd.log])
>     on_exit "kill_ovs_vswitchd `cat ovs-vswitchd.pid`"
>  ])
> +
> +
> +# OVS_DPDK_HUGEPAGE_CLEANUP([file])
> +#
> +# Cleanup system for stale hugepages.
> +#
> +m4_define([OVS_DPDK_HUGEPAGE_CLEANUP],
> +  [dnl Cleanup mapping files in hugetlbfs mount point
> +   AT_CHECK([cat /proc/mounts | grep hugetlbfs], [], [stdout])

This could be:
   AT_CHECK([grep hugetlbfs /proc/mounts], [], [stdout])


> +   AT_CHECK([cut -d ' ' -f 2  stdout], [], [stdout])
> +   AT_CHECK([rm -f $(cat stdout)/$1], [], [])

Or combine all the above in a single line:

   rm -f $(awk '/hugetlbfs/ { print $2 }' /proc/mounts)/$1

> +
> +])
> diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at
> index a015d52..01ac970 100644
> --- a/tests/system-dpdk.at
> +++ b/tests/system-dpdk.at
> @@ -16,6 +16,7 @@ OVS_VSWITCHD_STOP(["/Global register is changed during/d
>  /EAL:   Invalid NUMA socket, default to 0/d
>  /EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !/d
>  /EAL: No free hugepages reported in hugepages-1048576kB/d"])
> +OVS_DPDK_HUGEPAGE_CLEANUP([rtemap_*])
>  AT_CLEANUP
>  dnl --------------------------------------------------------------------------
>  
> @@ -43,6 +44,7 @@ OVS_VSWITCHD_STOP("/does not exist. The Open vSwitch kernel module is probably n
>  /EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !/d
>  /EAL: No free hugepages reported in hugepages-1048576kB/d
>  ")
> +OVS_DPDK_HUGEPAGE_CLEANUP([rtemap_*])
>  AT_CLEANUP
>  dnl --------------------------------------------------------------------------
>  
> @@ -75,6 +77,7 @@ OVS_VSWITCHD_STOP(["\@does not exist. The Open vSwitch kernel module is probably
>  \@EAL:   Invalid NUMA socket, default to 0@d
>  \@EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !@d
>  \@EAL: No free hugepages reported in hugepages-1048576kB@d"])
> +OVS_DPDK_HUGEPAGE_CLEANUP([rtemap_*])
>  AT_CLEANUP
>  dnl --------------------------------------------------------------------------
>  
> @@ -154,6 +157,8 @@ OVS_VSWITCHD_STOP(["\@does not exist. The Open vSwitch kernel module is probably
>  \@EAL:   Invalid NUMA socket, default to 0@d
>  \@EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !@d
>  \@EAL: No free hugepages reported in hugepages-1048576kB@d"])
> +OVS_DPDK_HUGEPAGE_CLEANUP([rtemap_*])
> +OVS_DPDK_HUGEPAGE_CLEANUP([page0map_0])
>  AT_CLEANUP
>  dnl --------------------------------------------------------------------------
>  
> @@ -230,5 +235,7 @@ OVS_VSWITCHD_STOP(["\@does not exist. The Open vSwitch kernel module is probably
>  \@EAL:   Invalid NUMA socket, default to 0@d
>  \@EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !@d
>  \@EAL: No free hugepages reported in hugepages-1048576kB@d"])
> +OVS_DPDK_HUGEPAGE_CLEANUP([rtemap_*])
> +OVS_DPDK_HUGEPAGE_CLEANUP([page0map_0])
>  AT_CLEANUP
>  dnl --------------------------------------------------------------------------
> -- 
> 1.8.3.1
> 
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Ilya Maximets May 21, 2020, 2:24 p.m. UTC | #2
On 5/16/20 7:53 AM, Gowrishankar Muthukrishnan wrote:
> After dpdk tests completes, cleaning up hugepage map files
> created by tests is helpful to release used memory into
> hugepage memory allocator.
> 
> Signed-off-by: Gowrishankar Muthukrishnan <gmuthukr@redhat.com>
> ---
>  tests/system-dpdk-macros.at | 13 +++++++++++++
>  tests/system-dpdk.at        |  7 +++++++
>  2 files changed, 20 insertions(+)
> 
> diff --git a/tests/system-dpdk-macros.at b/tests/system-dpdk-macros.at
> index c6708ca..d3a3aea 100644
> --- a/tests/system-dpdk-macros.at
> +++ b/tests/system-dpdk-macros.at
> @@ -63,3 +63,16 @@ m4_define([OVS_DPDK_START],
>     AT_CAPTURE_FILE([ovs-vswitchd.log])
>     on_exit "kill_ovs_vswitchd `cat ovs-vswitchd.pid`"
>  ])
> +
> +
> +# OVS_DPDK_HUGEPAGE_CLEANUP([file])
> +#
> +# Cleanup system for stale hugepages.
> +#
> +m4_define([OVS_DPDK_HUGEPAGE_CLEANUP],
> +  [dnl Cleanup mapping files in hugetlbfs mount point
> +   AT_CHECK([cat /proc/mounts | grep hugetlbfs], [], [stdout])
> +   AT_CHECK([cut -d ' ' -f 2  stdout], [], [stdout])
> +   AT_CHECK([rm -f $(cat stdout)/$1], [], [])
> +
> +])

Can we just use --in-memory option and avoid having files at all?

General note about sending patches: Please, don't send new versions in reply
to the previous one.  This messes up mailboxes.

Best regards, Ilya Maximets.
Gowrishankar Muthukrishnan May 27, 2020, 2:10 a.m. UTC | #3
I tried --in-memory but finding issues with existing tests unable to pass
mainly, being unable to ping between kernel veth and dpdk tap. Problem
could be because of implicit --no-shconf ? .Can we handle this separately
as it is blocking other patches ? Your opinions please.

Regards.

On Thu, May 21, 2020 at 7:54 PM Ilya Maximets <i.maximets@ovn.org> wrote:

> On 5/16/20 7:53 AM, Gowrishankar Muthukrishnan wrote:
> > After dpdk tests completes, cleaning up hugepage map files
> > created by tests is helpful to release used memory into
> > hugepage memory allocator.
> >
> > Signed-off-by: Gowrishankar Muthukrishnan <gmuthukr@redhat.com>
> > ---
> >  tests/system-dpdk-macros.at | 13 +++++++++++++
> >  tests/system-dpdk.at        |  7 +++++++
> >  2 files changed, 20 insertions(+)
> >
> > diff --git a/tests/system-dpdk-macros.at b/tests/system-dpdk-macros.at
> > index c6708ca..d3a3aea 100644
> > --- a/tests/system-dpdk-macros.at
> > +++ b/tests/system-dpdk-macros.at
> > @@ -63,3 +63,16 @@ m4_define([OVS_DPDK_START],
> >     AT_CAPTURE_FILE([ovs-vswitchd.log])
> >     on_exit "kill_ovs_vswitchd `cat ovs-vswitchd.pid`"
> >  ])
> > +
> > +
> > +# OVS_DPDK_HUGEPAGE_CLEANUP([file])
> > +#
> > +# Cleanup system for stale hugepages.
> > +#
> > +m4_define([OVS_DPDK_HUGEPAGE_CLEANUP],
> > +  [dnl Cleanup mapping files in hugetlbfs mount point
> > +   AT_CHECK([cat /proc/mounts | grep hugetlbfs], [], [stdout])
> > +   AT_CHECK([cut -d ' ' -f 2  stdout], [], [stdout])
> > +   AT_CHECK([rm -f $(cat stdout)/$1], [], [])
> > +
> > +])
>
> Can we just use --in-memory option and avoid having files at all?
>
> General note about sending patches: Please, don't send new versions in
> reply
> to the previous one.  This messes up mailboxes.
>
> Best regards, Ilya Maximets.
>
>
diff mbox series

Patch

diff --git a/tests/system-dpdk-macros.at b/tests/system-dpdk-macros.at
index c6708ca..d3a3aea 100644
--- a/tests/system-dpdk-macros.at
+++ b/tests/system-dpdk-macros.at
@@ -63,3 +63,16 @@  m4_define([OVS_DPDK_START],
    AT_CAPTURE_FILE([ovs-vswitchd.log])
    on_exit "kill_ovs_vswitchd `cat ovs-vswitchd.pid`"
 ])
+
+
+# OVS_DPDK_HUGEPAGE_CLEANUP([file])
+#
+# Cleanup system for stale hugepages.
+#
+m4_define([OVS_DPDK_HUGEPAGE_CLEANUP],
+  [dnl Cleanup mapping files in hugetlbfs mount point
+   AT_CHECK([cat /proc/mounts | grep hugetlbfs], [], [stdout])
+   AT_CHECK([cut -d ' ' -f 2  stdout], [], [stdout])
+   AT_CHECK([rm -f $(cat stdout)/$1], [], [])
+
+])
diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at
index a015d52..01ac970 100644
--- a/tests/system-dpdk.at
+++ b/tests/system-dpdk.at
@@ -16,6 +16,7 @@  OVS_VSWITCHD_STOP(["/Global register is changed during/d
 /EAL:   Invalid NUMA socket, default to 0/d
 /EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !/d
 /EAL: No free hugepages reported in hugepages-1048576kB/d"])
+OVS_DPDK_HUGEPAGE_CLEANUP([rtemap_*])
 AT_CLEANUP
 dnl --------------------------------------------------------------------------
 
@@ -43,6 +44,7 @@  OVS_VSWITCHD_STOP("/does not exist. The Open vSwitch kernel module is probably n
 /EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !/d
 /EAL: No free hugepages reported in hugepages-1048576kB/d
 ")
+OVS_DPDK_HUGEPAGE_CLEANUP([rtemap_*])
 AT_CLEANUP
 dnl --------------------------------------------------------------------------
 
@@ -75,6 +77,7 @@  OVS_VSWITCHD_STOP(["\@does not exist. The Open vSwitch kernel module is probably
 \@EAL:   Invalid NUMA socket, default to 0@d
 \@EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !@d
 \@EAL: No free hugepages reported in hugepages-1048576kB@d"])
+OVS_DPDK_HUGEPAGE_CLEANUP([rtemap_*])
 AT_CLEANUP
 dnl --------------------------------------------------------------------------
 
@@ -154,6 +157,8 @@  OVS_VSWITCHD_STOP(["\@does not exist. The Open vSwitch kernel module is probably
 \@EAL:   Invalid NUMA socket, default to 0@d
 \@EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !@d
 \@EAL: No free hugepages reported in hugepages-1048576kB@d"])
+OVS_DPDK_HUGEPAGE_CLEANUP([rtemap_*])
+OVS_DPDK_HUGEPAGE_CLEANUP([page0map_0])
 AT_CLEANUP
 dnl --------------------------------------------------------------------------
 
@@ -230,5 +235,7 @@  OVS_VSWITCHD_STOP(["\@does not exist. The Open vSwitch kernel module is probably
 \@EAL:   Invalid NUMA socket, default to 0@d
 \@EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !@d
 \@EAL: No free hugepages reported in hugepages-1048576kB@d"])
+OVS_DPDK_HUGEPAGE_CLEANUP([rtemap_*])
+OVS_DPDK_HUGEPAGE_CLEANUP([page0map_0])
 AT_CLEANUP
 dnl --------------------------------------------------------------------------