| Message ID | 20190715211501.88213-1-vdasari@gmail.com |
|---|---|
| State | Accepted |
| Commit | c99d14775f78cb38b2109add063f58201ba07652 |
| Headers | show |
| Series | [ovs-dev,v2] ovs-macros: An option to suspend test execution on error | expand |
Bleep bloop. Greetings Vasu Dasari, I am a robot and I have tried out your patch. Thanks for your contribution. I encountered some error that I wasn't expecting. See the details below. checkpatch: WARNING: Line is 82 characters long (recommended limit is 79) #71 FILE: Documentation/topics/testing.rst:97: export OVS_RUNDIR=<dir>/ovs/_build-gcc/tests/system-userspace-testsuite.dir/139 WARNING: Line is 99 characters long (recommended limit is 79) #76 FILE: Documentation/topics/testing.rst:102: export OVS_RUNDIR=/opt/vdasari/Developer/ovs/_build-gcc/tests/system-userspace-testsuite.dir/139 Lines checked: 133, Warnings: 2, Errors: 0 Please check this out. If you feel there has been an error, please email aconole@bytheb.org Thanks, 0-day Robot
Vasu Dasari <vdasari@gmail.com> writes: > Origins for this patch are captured at > https://mail.openvswitch.org/pipermail/ovs-discuss/2019-June/048923.html. > > Summarizing here, when a test fails, it would be good to pause test execution > and let the developer poke around the system to see current status of system. > > As part of this patch, made a small tweaks to ovs-macros.at, so that when test > suite fails, ovs_on_exit() function will be called. And in this function, a check > is made to see if an environment variable to OVS_PAUSE_TEST is set. If it is > set, then test suite is paused and will continue to wait for user input > Ctrl-D. Meanwhile user can poke around the system to see why test case has > failed. Once done with investigation, user can press ctrl-d to cleanup the > test suite. > > For example, to re-run test case 139: > > export OVS_PAUSE_TEST=1 > cd tests/system-userspace-testsuite.dir/139 > sudo -E ./run > > When error occurs, above command would display something like this: > ===================================================== > Set environment variable to use various ovs utilities > export OVS_RUNDIR=/opt/vdasari/Developer/ovs/_build-gcc/tests/system-userspace-testsuite.dir/139 > Press ENTER to continue: > > ===================================================== > And from another window, one can execute ovs-xxx commands like: > export OVS_RUNDIR=/opt/vdasari/Developer/ovs/_build-gcc/tests/system-userspace-testsuite.dir/139 > $ ovs-ofctl dump-ports br0 > . > . > > To be able to pause while performing `make check`, one can do: > $ OVS_PAUSE_TEST=1 make check TESTSUITEFLAGS='-v' > > Signed-off-by: Vasu Dasari <vdasari@gmail.com> > Reviewed-by: Ben Pfaff <blp@ovn.org> > Reviewed-by: Aaron Conole <aconole@redhat.com> > --- > v0 -> v1: > Discussion at: https://mail.openvswitch.org/pipermail/ovs-dev/2019-July/360589.html > 1. Incorporated review commanets from Ben. > 2. Changed topics/testing.rst to document this trick > 3. Made this trick work via "make check" as well, by issuing command: "OVS_PAUSE_TEST=1 > make check TESTSUITEFLAGS='-v'" > > v1 -> v2: > Discussion at: https://mail.openvswitch.org/pipermail/ovs-dev/2019-July/360705.html > 1. Added a subsection, "Debugging unit tests" > 2. Renamed $as_echo to just echo. AS_ECHO is not suported in autoconf 2.69.11. And, we are not gainin > g anything from AS_ECHO in this code, just using echo. > > --- Acked-by: Aaron Conole <aconole@redhat.com> Thanks for the cool feature, Vasu!
On Mon, Jul 15, 2019 at 05:15:01PM -0400, Vasu Dasari wrote: > Origins for this patch are captured at > https://mail.openvswitch.org/pipermail/ovs-discuss/2019-June/048923.html. > > Summarizing here, when a test fails, it would be good to pause test execution > and let the developer poke around the system to see current status of system. Thanks a lot! I updated the documentation a bit because it talked about the userspace datapath testsuite even though it was part of the documentation for the main set of unit tests. I applied this to master, thanks again!
Thanks Ben and Aaron for your review and comments. *Vasu Dasari* On Tue, Jul 16, 2019 at 12:57 PM Ben Pfaff <blp@ovn.org> wrote: > On Mon, Jul 15, 2019 at 05:15:01PM -0400, Vasu Dasari wrote: > > Origins for this patch are captured at > > https://mail.openvswitch.org/pipermail/ovs-discuss/2019-June/048923.html > . > > > > Summarizing here, when a test fails, it would be good to pause test > execution > > and let the developer poke around the system to see current status of > system. > > Thanks a lot! > > I updated the documentation a bit because it talked about the userspace > datapath testsuite even though it was part of the documentation for the > main set of unit tests. > > I applied this to master, thanks again! >
diff --git a/Documentation/topics/testing.rst b/Documentation/topics/testing.rst index a8892e1c1..73077042c 100644 --- a/Documentation/topics/testing.rst +++ b/Documentation/topics/testing.rst @@ -81,6 +81,37 @@ To see a complete list of test options, run:: The results of a testing run are reported in ``tests/testsuite.log``. Report report test failures as bugs and include the ``testsuite.log`` in your report. +Debugging unit tests +++++++++++++++++++++ + +To initiate debugging from artifacts generated from `make check` run, use +following instructions. For example, to re-run test case 139:: + + export OVS_PAUSE_TEST=1 + cd tests/system-userspace-testsuite.dir/139 + sudo -E ./run + +When error occurs, above command would display something like this:: + + Set environment variable to use various ovs utilities + export OVS_RUNDIR=<dir>/ovs/_build-gcc/tests/system-userspace-testsuite.dir/139 + Press ENTER to continue: + +And from another window, one can execute ovs-xxx commands like:: + + export OVS_RUNDIR=/opt/vdasari/Developer/ovs/_build-gcc/tests/system-userspace-testsuite.dir/139 + $ ovs-ofctl dump-ports br0 + . + . + +Once done with investigation, press ENTER to perform cleanup operation. + +To be able to pause on error while performing `make check`, one can do:: + + $ OVS_PAUSE_TEST=1 make check TESTSUITEFLAGS='-v' + +and then follow similar on screen instructions + .. note:: Sometimes a few tests may fail on some runs but not others. This is usually a bug in the testsuite, not a bug in Open vSwitch itself. If you find that a diff --git a/tests/ovs-macros.at b/tests/ovs-macros.at index 10593429d..b6add7fda 100644 --- a/tests/ovs-macros.at +++ b/tests/ovs-macros.at @@ -35,11 +35,33 @@ m4_divert_push([PREPARE_TESTS]) # directory. ovs_init() { ovs_base=`pwd` - trap '. "$ovs_base/cleanup"' 0 + trap ovs_on_exit 0 : > cleanup ovs_setenv } +# Catch testsuite error condition and cleanup test environment by tearing down +# all interfaces and processes spawned. +# User has an option to leave the test environment in error state so that system +# can be poked around to get more information. User can enable this option by setting +# environment variable OVS_PAUSE_TEST=1. User needs to press CTRL-D to resume the +# cleanup operation. +ovs_pause() { + echo "=====================================================" + echo "Set following environment variable to use various ovs utilities" + echo "export OVS_RUNDIR=$ovs_base" + echo "Press ENTER to continue: " + read +} + +ovs_on_exit () { + if [ ! -z "${OVS_PAUSE_TEST}" ] && [ -z $at_verbose ]; then + trap '' INT + ovs_pause + fi + . "$ovs_base/cleanup" +} + # With no parameter or an empty parameter, sets the OVS_*DIR # environment variables to point to $ovs_base, the base directory in # which the test is running.