Message ID | 1518701885-16545-1-git-send-email-yegorslists@googlemail.com |
---|---|
State | Superseded |
Headers | show |
Series | [RFC] testing: add python-crossbar tests | expand |
On Thu, Feb 15, 2018 at 2:38 PM, <yegorslists@googlemail.com> wrote: > From: Yegor Yefremov <yegorslists@googlemail.com> > > This test invokes "crossbar version" command, that checks > all dependencies found in setup.py files and prints some > system related information. > > Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com> > --- > .../testing/tests/package/test_python_crossbar.py | 27 ++++++++++++++++++++++ > 1 file changed, 27 insertions(+) > create mode 100644 support/testing/tests/package/test_python_crossbar.py > > diff --git a/support/testing/tests/package/test_python_crossbar.py b/support/testing/tests/package/test_python_crossbar.py > new file mode 100644 > index 0000000..a67bdea > --- /dev/null > +++ b/support/testing/tests/package/test_python_crossbar.py > @@ -0,0 +1,27 @@ > +from tests.package.test_python import TestPythonBase > + > +class TestPythonCrossbar(TestPythonBase): > + def version_test(self, timeout=-1): > + cmd = self.interpreter + " -m crossbar version" > + _, exit_code = self.emulator.run(cmd, timeout) > + self.assertEqual(exit_code, 0) > + > +class TestPythonPy2Crossbar(TestPythonCrossbar): > + config = TestPythonBase.config + \ > +""" > +BR2_PACKAGE_PYTHON=y > +BR2_PACKAGE_PYTHON_CROSSBAR=y > +""" > + def test_run(self): > + self.login() > + self.version_test(60) > + > +class TestPythonPy3Crossbar(TestPythonCrossbar): > + config = TestPythonBase.config + \ > +""" > +BR2_PACKAGE_PYTHON3=y > +BR2_PACKAGE_PYTHON_CROSSBAR=y > +""" > + def test_run(self): > + self.login() > + self.version_test(60) I've tried this test locally, but it always timeouts. Even if I specify an hour as a timeout. If a dependency is missing (like python-bitstring), then the test also fails with an expected message, but if all dependencies are met, the test doesn't return. Any idea? Yegor
Hello, On Thu, Feb 15, 2018 at 11:41 AM, Yegor Yefremov wrote: [snip] > > I've tried this test locally, but it always timeouts. Even if I > specify an hour as a timeout. I confirm that here, both test cases fail by timeout, so it should not be related to the host. > > If a dependency is missing (like python-bitstring), then the test also > fails with an expected message, but if all dependencies are met, the > test doesn't return. > > Any idea? I have no idea of what causes it or how to fix it. However I can give you some ideas how to try to debug using the test infra. I don't know how far you already explored these options, so please forgive me if I say only obvious things. 1) run the test case locally using -k 2) look at the first line of the -run.log to get the qemu command 3) execute the qemu command and get to the VM prompt 4) check that the import of the module is not broken: buildroot login: root # python Python 3.6.3 (default, Feb 15 2018, 20:46:37) [GCC 4.9.4] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import crossbar >>> crossbar.__version__ '17.6.1-3' >>> 5) check a command that should print the usage # python -m crossbar othervalue Traceback (most recent call last): File "usr/lib/python3.6/site-packages/watchdog/observers/inotify_c.py", line 46, in _load_libc File "/tmp/tmp2/o/TestPythonPy3Crossbar/target/usr/lib/python3.6/ctypes/__init__.py", line 348, in __init__ OSError: File not found (omitted lines) NOTICE the host path (/tmp/tmp2 is my buildroot dir) being displayed inside the target. 6) ctrl+C to kill the VM 7) enter the <test-output>/TestPythonPy3Crossbar dir - this is the build directory, so you can do what we know well, for example, enter menuconfig, select 'strace', and call make (or do a clean build) 8) run the qemu command again with the new image # strace python -m crossbar version Wait a few minutes for many messages and look at them. These ones called my attention, maybe unrelated to the problem: (omitted lines) open("/lib//libc.so", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usrlib//libc.so", O_RDONLY) = -1 ENOENT (No such file or directory) open("/lib/libc.so", O_RDONLY) = -1 ENOENT (No such file or directory) open("/lib/libc.so", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/lib/libc.so", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/X11R6/lib/libc.so", O_RDONLY) = -1 ENOENT (No such file or directory) open("/lib//libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usrlib//libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory) open("/lib/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory) open("/lib/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/lib/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/X11R6/lib/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory) write(2, "Traceback (most recent call last"..., 35Traceback (most recent call last): (omitted lines) See also: # find / -name 'libc.so*' /lib/libc.so.1 /lib/libc.so.0 # ls /lib ld-uClibc-1.0.25.so libatomic.so.1 libc.so.1 ld-uClibc.so.0 libatomic.so.1.1.0 libgcc_s.so.1 ld-uClibc.so.1 libc.so.0 libuClibc-1.0.25.so 9) repeat steps 6 to 8 with your favorite debug tools So... (wild guesses!) Could it be related to not finding libc? Could it be related to paths from host leaked to the target? Also, did you try to rebase the test case to the previous package version just to know if the behavior is new? Regards, Ricardo
Hello, On Thu, Feb 15, 2018 at 11:38 AM, Yegor Yefremov wrote: [snip] > --- > .../testing/tests/package/test_python_crossbar.py | 27 ++++++++++++++++++++++ > 1 file changed, 27 insertions(+) > create mode 100644 support/testing/tests/package/test_python_crossbar.py The update to .gitlab.yml can be added now that 'make .gitlab-ci.yml' is fixed (on master). [snip] > +++ b/support/testing/tests/package/test_python_crossbar.py My series fixing flake8 warnings are now mostly applied (a few items need v2). Can you run flake8 and fix the 7 warnings? You can see the updated test_python.py for reference. I applied this and also http://patchwork.ozlabs.org/patch/874421/ http://patchwork.ozlabs.org/patch/874422/ but the test still fails with timeout: https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/53345963 I did not debugged it. Maybe I missed something? Regards, Ricardo
Hello, On Thu, 15 Feb 2018 23:16:32 -0200, Ricardo Martincoski wrote: > On Thu, Feb 15, 2018 at 11:41 AM, Yegor Yefremov wrote: > > [snip] > > > > I've tried this test locally, but it always timeouts. Even if I > > specify an hour as a timeout. > > I confirm that here, both test cases fail by timeout, so it should not be > related to the host. I applied Yegor's patch, and ran he test. It doesn't time out here. Here is what I have in the TestPythonPy2Crossbar-run.log: Welcome to Buildroot buildroot login: root # dmesg -n 1 # echo $? 0 # python -m crossbar version Traceback (most recent call last): File "usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main File "usr/lib/python2.7/runpy.py", line 72, in _run_code File "usr/lib/python2.7/site-packages/crossbar/__main__.py", line 36, in <module> File "usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 479, in load_entry_point File "usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2703, in load_entry_point File "usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2320, in load File "usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2343, in require File "usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 782, in resolve pkg_resources.ContextualVersionConflict: (txaio 2.8.2 (/usr/lib/python2.7/site-packages), Requirement.parse('txaio>=18.8.1'), set(['autobahn'])) # echo $? 1 So it doesn't work due to some dependency issues. Is the timeout problem going to appear once those dependency issues are solved ? Or should I have already seen the timeout ? Best regards, Thomas
Hello, On Mon, Sep 17, 2018 at 10:55 AM, Thomas Petazzoni wrote: > On Thu, 15 Feb 2018 23:16:32 -0200, Ricardo Martincoski wrote: > >> On Thu, Feb 15, 2018 at 11:41 AM, Yegor Yefremov wrote: >> >> [snip] >> > >> > I've tried this test locally, but it always timeouts. Even if I >> > specify an hour as a timeout. >> >> I confirm that here, both test cases fail by timeout, so it should not be >> related to the host. At the time I confirmed with the patch applied on aa77030b8f and also on (aa77030b8f + http://patchwork.ozlabs.org/patch/874421/ + http://patchwork.ozlabs.org/patch/874422/ ) and I got the timeout in both cases. > > I applied Yegor's patch, and ran he test. It doesn't time out here. > Here is what I have in the TestPythonPy2Crossbar-run.log: > > Welcome to Buildroot > buildroot login: root > # dmesg -n 1 > # echo $? > 0 > # python -m crossbar version > Traceback (most recent call last): > File "usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main > File "usr/lib/python2.7/runpy.py", line 72, in _run_code > File "usr/lib/python2.7/site-packages/crossbar/__main__.py", line 36, in <module> > File "usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 479, in load_entry_point > File "usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2703, in load_entry_point > File "usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2320, in load > File "usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2343, in require > File "usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 782, in resolve > pkg_resources.ContextualVersionConflict: (txaio 2.8.2 (/usr/lib/python2.7/site-packages), Requirement.parse('txaio>=18.8.1'), set(['autobahn'])) > # echo $? > 1 > > So it doesn't work due to some dependency issues. Is the timeout > problem going to appear once those dependency issues are solved ? Or > should I have already seen the timeout ? Sorry, I don't know. Yegor, any idea? Could you bisect this? I already started: 2018.02-rc2-4-gaa77030b8f + patch = timeout 2018.08-425-gc5a33a11be + patch = dependency issue 2018.05-rc3-25-gd937f908f1 + patch = timeout 2018.05-756-g3d34f36806 + patch = (still running: https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/98611581 ) Regards, Ricardo
Hi Ricardo, On Tue, Sep 18, 2018 at 4:31 AM Ricardo Martincoski <ricardo.martincoski@gmail.com> wrote: > > Hello, > > On Mon, Sep 17, 2018 at 10:55 AM, Thomas Petazzoni wrote: > > > On Thu, 15 Feb 2018 23:16:32 -0200, Ricardo Martincoski wrote: > > > >> On Thu, Feb 15, 2018 at 11:41 AM, Yegor Yefremov wrote: > >> > >> [snip] > >> > > >> > I've tried this test locally, but it always timeouts. Even if I > >> > specify an hour as a timeout. > >> > >> I confirm that here, both test cases fail by timeout, so it should not be > >> related to the host. > > At the time I confirmed with the patch applied on aa77030b8f and also on > (aa77030b8f + > http://patchwork.ozlabs.org/patch/874421/ + > http://patchwork.ozlabs.org/patch/874422/ ) > and I got the timeout in both cases. > > > > > I applied Yegor's patch, and ran he test. It doesn't time out here. > > Here is what I have in the TestPythonPy2Crossbar-run.log: > > > > Welcome to Buildroot > > buildroot login: root > > # dmesg -n 1 > > # echo $? > > 0 > > # python -m crossbar version > > Traceback (most recent call last): > > File "usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main > > File "usr/lib/python2.7/runpy.py", line 72, in _run_code > > File "usr/lib/python2.7/site-packages/crossbar/__main__.py", line 36, in <module> > > File "usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 479, in load_entry_point > > File "usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2703, in load_entry_point > > File "usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2320, in load > > File "usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2343, in require > > File "usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 782, in resolve > > pkg_resources.ContextualVersionConflict: (txaio 2.8.2 (/usr/lib/python2.7/site-packages), Requirement.parse('txaio>=18.8.1'), set(['autobahn'])) > > # echo $? > > 1 > > > > So it doesn't work due to some dependency issues. Is the timeout > > problem going to appear once those dependency issues are solved ? Or > > should I have already seen the timeout ? > > Sorry, I don't know. > > Yegor, any idea? Could you bisect this? I already started: > 2018.02-rc2-4-gaa77030b8f + patch = timeout > 2018.08-425-gc5a33a11be + patch = dependency issue > 2018.05-rc3-25-gd937f908f1 + patch = timeout > 2018.05-756-g3d34f36806 + patch = (still running: > https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/98611581 ) Thanks for working on this issue. AFAIK crossbar abandoned Py2 support in the newest version, so there is less to take of, but there are still issues with Py3.7 and async keywords, that need to be resolved first. I don't know when I'll have time to update crossbar, so if you can take over it would be great. Thanks. Regards, Yegor
Hello, I just remembered I've submitted a patch for a new host package long time ago: http://patchwork.ozlabs.org/patch/896368/ Yegor mentioned that maybe it can help to reduce python-crossbar startup time on the tests. Thomas/Yegor, I see the patch was reviewed by Yegor but wasn't applied. If there's something missing, please let me know. I hope it will help. Regards, Asaf. On Tue, Sep 18, 2018 at 9:20 AM Yegor Yefremov <yegorslists@googlemail.com> wrote: > > Hi Ricardo, > > On Tue, Sep 18, 2018 at 4:31 AM Ricardo Martincoski > <ricardo.martincoski@gmail.com> wrote: > > > > Hello, > > > > On Mon, Sep 17, 2018 at 10:55 AM, Thomas Petazzoni wrote: > > > > > On Thu, 15 Feb 2018 23:16:32 -0200, Ricardo Martincoski wrote: > > > > > >> On Thu, Feb 15, 2018 at 11:41 AM, Yegor Yefremov wrote: > > >> > > >> [snip] > > >> > > > >> > I've tried this test locally, but it always timeouts. Even if I > > >> > specify an hour as a timeout. > > >> > > >> I confirm that here, both test cases fail by timeout, so it should not be > > >> related to the host. > > > > At the time I confirmed with the patch applied on aa77030b8f and also on > > (aa77030b8f + > > http://patchwork.ozlabs.org/patch/874421/ + > > http://patchwork.ozlabs.org/patch/874422/ ) > > and I got the timeout in both cases. > > > > > > > > I applied Yegor's patch, and ran he test. It doesn't time out here. > > > Here is what I have in the TestPythonPy2Crossbar-run.log: > > > > > > Welcome to Buildroot > > > buildroot login: root > > > # dmesg -n 1 > > > # echo $? > > > 0 > > > # python -m crossbar version > > > Traceback (most recent call last): > > > File "usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main > > > File "usr/lib/python2.7/runpy.py", line 72, in _run_code > > > File "usr/lib/python2.7/site-packages/crossbar/__main__.py", line 36, in <module> > > > File "usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 479, in load_entry_point > > > File "usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2703, in load_entry_point > > > File "usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2320, in load > > > File "usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2343, in require > > > File "usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 782, in resolve > > > pkg_resources.ContextualVersionConflict: (txaio 2.8.2 (/usr/lib/python2.7/site-packages), Requirement.parse('txaio>=18.8.1'), set(['autobahn'])) > > > # echo $? > > > 1 > > > > > > So it doesn't work due to some dependency issues. Is the timeout > > > problem going to appear once those dependency issues are solved ? Or > > > should I have already seen the timeout ? > > > > Sorry, I don't know. > > > > Yegor, any idea? Could you bisect this? I already started: > > 2018.02-rc2-4-gaa77030b8f + patch = timeout > > 2018.08-425-gc5a33a11be + patch = dependency issue > > 2018.05-rc3-25-gd937f908f1 + patch = timeout > > 2018.05-756-g3d34f36806 + patch = (still running: > > https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/98611581 ) > > Thanks for working on this issue. AFAIK crossbar abandoned Py2 support > in the newest version, so there is less to take of, but there are > still issues with Py3.7 and async keywords, that need to be resolved > first. > > I don't know when I'll have time to update crossbar, so if you can > take over it would be great. > > Thanks. > > Regards, > Yegor > _______________________________________________ > buildroot mailing list > buildroot@busybox.net > http://lists.busybox.net/mailman/listinfo/buildroot
Hello, On Sat, Oct 20, 2018 at 12:49 PM, Asaf Kahlon wrote: > I just remembered I've submitted a patch for a new host package long > time ago: http://patchwork.ozlabs.org/patch/896368/ > Yegor mentioned that maybe it can help to reduce python-crossbar > startup time on the tests. > Thomas/Yegor, I see the patch was reviewed by Yegor but wasn't > applied. If there's something missing, please let me know. > I hope it will help. I am not against that patch but I am afraid it won't fix the scenario tested in this patch. Turns out that the qemu image we use don't have enough entropy at boot time. I confirmed by running this command on the qemu: # cat /proc/sys/kernel/random/entropy_avail 0 Because of this, crossbar -> pynacl -> libsodium hangs waiting for /dev/random, see [1]. We could update the default kernel image used by the test infra (today 3.11.0), or even use a custom one (4.14.x for instance) compiled in this test case, to get the getrandom syscall. It would improve a lot the startup time but we would yet need to use a timeout of a few minutes. The most elegant solution IMO is to add haveged to the image built for this test case by the test infra. This is what people with a real target with low entropy probably would use, I guess. There is no need to update the kernel this way. I have locally a v2 of this test case that I will send, see [2] for the test passing on the current master, but before I send it I will wait for feedback in another patch [3]. This is the changelog for v2: - call 'crossbar version' instead of 'python -m "crossbar version"' because it is not supported anymore, giving this message: /usr/bin/python3: No module named crossbar.__main__; 'crossbar' is a package and cannot be directly executed - add haveged to the target to provide enough entropy, avoiding hanging forever (on our current kernel 3.11.0 default image) or for a long time (when the default kernel image is upgraded to a newer kernel); - remove test for python 2 because upstream dropped support; - do not pass timeout parameter since version_test is now called once; - rewrap commit message to 72 - update code-style (we adopted flake8 after v1 submission) - update .gitlab-ci.yml [1] https://github.com/pyca/pynacl/issues/327 [2] https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/110669652 [3] http://patchwork.ozlabs.org/patch/984425/ Regards, Ricardo
Hello, On Sun, 21 Oct 2018 02:01:37 -0300, Ricardo Martincoski wrote: > We could update the default kernel image used by the test infra (today 3.11.0), We can definitely update that pre-built image if needed. There is no special reason to have a 3.11 based image, except this being the kernel version I had around at the time I started playing with the test infrastructure. > The most elegant solution IMO is to add haveged to the image built for this test > case by the test infra. This is what people with a real target with low entropy > probably would use, I guess. There is no need to update the kernel this way. I also don't see any problem with adding haveged in the image. Best regards, Thomas
diff --git a/support/testing/tests/package/test_python_crossbar.py b/support/testing/tests/package/test_python_crossbar.py new file mode 100644 index 0000000..a67bdea --- /dev/null +++ b/support/testing/tests/package/test_python_crossbar.py @@ -0,0 +1,27 @@ +from tests.package.test_python import TestPythonBase + +class TestPythonCrossbar(TestPythonBase): + def version_test(self, timeout=-1): + cmd = self.interpreter + " -m crossbar version" + _, exit_code = self.emulator.run(cmd, timeout) + self.assertEqual(exit_code, 0) + +class TestPythonPy2Crossbar(TestPythonCrossbar): + config = TestPythonBase.config + \ +""" +BR2_PACKAGE_PYTHON=y +BR2_PACKAGE_PYTHON_CROSSBAR=y +""" + def test_run(self): + self.login() + self.version_test(60) + +class TestPythonPy3Crossbar(TestPythonCrossbar): + config = TestPythonBase.config + \ +""" +BR2_PACKAGE_PYTHON3=y +BR2_PACKAGE_PYTHON_CROSSBAR=y +""" + def test_run(self): + self.login() + self.version_test(60)