| Message ID | 20200225194446.20651-6-mrostecki@opensuse.org |
|---|---|
| State | Changes Requested |
| Delegated to: | BPF Maintainers |
| Headers | show |
| Series | bpftool: Make probes which emit dmesg warnings optional | expand |
On 2/25/20 8:44 PM, Michal Rostecki wrote: > Add Python module with tests for "bpftool feature" command, which mainly > wheck whether the "full" option is working properly. nit, typo: wheck > > Signed-off-by: Michal Rostecki <mrostecki@opensuse.org> Ptal, when running the test I'm getting the following error: root@tank:~/bpf-next/tools/testing/selftests/bpf# ./test_bpftool.sh test_feature_dev_json (test_bpftool.TestBpftool) ... ERROR test_feature_kernel (test_bpftool.TestBpftool) ... ERROR test_feature_kernel_full (test_bpftool.TestBpftool) ... ERROR test_feature_kernel_full_vs_not_full (test_bpftool.TestBpftool) ... ERROR test_feature_macros (test_bpftool.TestBpftool) ... ERROR ====================================================================== ERROR: test_feature_dev_json (test_bpftool.TestBpftool) ---------------------------------------------------------------------- Traceback (most recent call last): File "/root/bpf-next/tools/testing/selftests/bpf/test_bpftool.py", line 58, in wrapper return f(*args, iface, **kwargs) File "/root/bpf-next/tools/testing/selftests/bpf/test_bpftool.py", line 83, in test_feature_dev_json res = bpftool_json(["feature", "probe", "dev", iface]) File "/root/bpf-next/tools/testing/selftests/bpf/test_bpftool.py", line 43, in bpftool_json res = _bpftool(args) File "/root/bpf-next/tools/testing/selftests/bpf/test_bpftool.py", line 34, in _bpftool res = subprocess.run(_args, capture_output=True) File "/usr/lib/python3.6/subprocess.py", line 423, in run with Popen(*popenargs, **kwargs) as process: TypeError: __init__() got an unexpected keyword argument 'capture_output' ====================================================================== ERROR: test_feature_kernel (test_bpftool.TestBpftool) ---------------------------------------------------------------------- Traceback (most recent call last): File "/root/bpf-next/tools/testing/selftests/bpf/test_bpftool.py", line 94, in test_feature_kernel bpftool_json(["feature", "probe", "kernel"]), File "/root/bpf-next/tools/testing/selftests/bpf/test_bpftool.py", line 43, in bpftool_json res = _bpftool(args) File "/root/bpf-next/tools/testing/selftests/bpf/test_bpftool.py", line 34, in _bpftool res = subprocess.run(_args, capture_output=True) File "/usr/lib/python3.6/subprocess.py", line 423, in run with Popen(*popenargs, **kwargs) as process: TypeError: __init__() got an unexpected keyword argument 'capture_output' ====================================================================== ERROR: test_feature_kernel_full (test_bpftool.TestBpftool) ---------------------------------------------------------------------- Traceback (most recent call last): File "/root/bpf-next/tools/testing/selftests/bpf/test_bpftool.py", line 122, in test_feature_kernel_full bpftool_json(["feature", "probe", "kernel", "full"]), File "/root/bpf-next/tools/testing/selftests/bpf/test_bpftool.py", line 43, in bpftool_json res = _bpftool(args) File "/root/bpf-next/tools/testing/selftests/bpf/test_bpftool.py", line 34, in _bpftool res = subprocess.run(_args, capture_output=True) File "/usr/lib/python3.6/subprocess.py", line 423, in run with Popen(*popenargs, **kwargs) as process: TypeError: __init__() got an unexpected keyword argument 'capture_output' ====================================================================== ERROR: test_feature_kernel_full_vs_not_full (test_bpftool.TestBpftool) ---------------------------------------------------------------------- Traceback (most recent call last): File "/root/bpf-next/tools/testing/selftests/bpf/test_bpftool.py", line 147, in test_feature_kernel_full_vs_not_full full_res = bpftool_json(["feature", "probe", "full"]) File "/root/bpf-next/tools/testing/selftests/bpf/test_bpftool.py", line 43, in bpftool_json res = _bpftool(args) File "/root/bpf-next/tools/testing/selftests/bpf/test_bpftool.py", line 34, in _bpftool res = subprocess.run(_args, capture_output=True) File "/usr/lib/python3.6/subprocess.py", line 423, in run with Popen(*popenargs, **kwargs) as process: TypeError: __init__() got an unexpected keyword argument 'capture_output' ====================================================================== ERROR: test_feature_macros (test_bpftool.TestBpftool) ---------------------------------------------------------------------- Traceback (most recent call last): File "/root/bpf-next/tools/testing/selftests/bpf/test_bpftool.py", line 177, in test_feature_macros res = bpftool(["feature", "probe", "macros"]) File "/root/bpf-next/tools/testing/selftests/bpf/test_bpftool.py", line 39, in bpftool return _bpftool(args, json=False).decode("utf-8") File "/root/bpf-next/tools/testing/selftests/bpf/test_bpftool.py", line 34, in _bpftool res = subprocess.run(_args, capture_output=True) File "/usr/lib/python3.6/subprocess.py", line 423, in run with Popen(*popenargs, **kwargs) as process: TypeError: __init__() got an unexpected keyword argument 'capture_output' ---------------------------------------------------------------------- Ran 5 tests in 0.001s FAILED (errors=5)
On 2/26/20 4:34 PM, Daniel Borkmann wrote: > Ptal, when running the test I'm getting the following error: > > root@tank:~/bpf-next/tools/testing/selftests/bpf# ./test_bpftool.sh > test_feature_dev_json (test_bpftool.TestBpftool) ... ERROR > test_feature_kernel (test_bpftool.TestBpftool) ... ERROR > test_feature_kernel_full (test_bpftool.TestBpftool) ... ERROR > test_feature_kernel_full_vs_not_full (test_bpftool.TestBpftool) ... ERROR > test_feature_macros (test_bpftool.TestBpftool) ... ERROR > > ====================================================================== > ERROR: test_feature_dev_json (test_bpftool.TestBpftool) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/root/bpf-next/tools/testing/selftests/bpf/test_bpftool.py", > line 58, in wrapper > return f(*args, iface, **kwargs) > File "/root/bpf-next/tools/testing/selftests/bpf/test_bpftool.py", > line 83, in test_feature_dev_json > res = bpftool_json(["feature", "probe", "dev", iface]) > File "/root/bpf-next/tools/testing/selftests/bpf/test_bpftool.py", > line 43, in bpftool_json > res = _bpftool(args) > File "/root/bpf-next/tools/testing/selftests/bpf/test_bpftool.py", > line 34, in _bpftool > res = subprocess.run(_args, capture_output=True) > File "/usr/lib/python3.6/subprocess.py", line 423, in run > with Popen(*popenargs, **kwargs) as process: > TypeError: __init__() got an unexpected keyword argument 'capture_output' Seems like that kwarg in Popen was added in Python 3.7. I will drop it and use the older way of getting combined output. Thanks for pointing that out!
2020-02-26 16:34 UTC+0100 ~ Daniel Borkmann <daniel@iogearbox.net> > On 2/25/20 8:44 PM, Michal Rostecki wrote: >> Add Python module with tests for "bpftool feature" command, which mainly >> wheck whether the "full" option is working properly. > > nit, typo: wheck > >> >> Signed-off-by: Michal Rostecki <mrostecki@opensuse.org> > > Ptal, when running the test I'm getting the following error: > > root@tank:~/bpf-next/tools/testing/selftests/bpf# ./test_bpftool.sh > test_feature_dev_json (test_bpftool.TestBpftool) ... ERROR > test_feature_kernel (test_bpftool.TestBpftool) ... ERROR > test_feature_kernel_full (test_bpftool.TestBpftool) ... ERROR > test_feature_kernel_full_vs_not_full (test_bpftool.TestBpftool) ... ERROR > test_feature_macros (test_bpftool.TestBpftool) ... ERROR > > ====================================================================== > ERROR: test_feature_dev_json (test_bpftool.TestBpftool) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/root/bpf-next/tools/testing/selftests/bpf/test_bpftool.py", > line 58, in wrapper > return f(*args, iface, **kwargs) > File "/root/bpf-next/tools/testing/selftests/bpf/test_bpftool.py", > line 83, in test_feature_dev_json > res = bpftool_json(["feature", "probe", "dev", iface]) > File "/root/bpf-next/tools/testing/selftests/bpf/test_bpftool.py", > line 43, in bpftool_json > res = _bpftool(args) > File "/root/bpf-next/tools/testing/selftests/bpf/test_bpftool.py", > line 34, in _bpftool > res = subprocess.run(_args, capture_output=True) > File "/usr/lib/python3.6/subprocess.py", line 423, in run > with Popen(*popenargs, **kwargs) as process: > TypeError: __init__() got an unexpected keyword argument 'capture_output' Apparently the “capture_output” option for subprocess was added to python 3.7 [0]. It worked on my system (python 3.7.5) but didn't pass on yours with 3.6. Michal, can you change it to something less recent please, so that people don't have to upgrade python to test? Quentin [0] https://docs.python.org/3/whatsnew/3.7.html#subprocess
diff --git a/tools/testing/selftests/.gitignore b/tools/testing/selftests/.gitignore index 61df01cdf0b2..304fdf1a21dc 100644 --- a/tools/testing/selftests/.gitignore +++ b/tools/testing/selftests/.gitignore @@ -3,4 +3,7 @@ gpiogpio-hammer gpioinclude/ gpiolsgpio tpm2/SpaceTest.log -tpm2/*.pyc + +# Python bytecode and cache +__pycache__/ +*.py[cod] diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index 2a583196fa51..2dffce6cc429 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -62,7 +62,8 @@ TEST_PROGS := test_kmod.sh \ test_tc_tunnel.sh \ test_tc_edt.sh \ test_xdping.sh \ - test_bpftool_build.sh + test_bpftool_build.sh \ + test_bpftool.sh TEST_PROGS_EXTENDED := with_addr.sh \ with_tunnels.sh \ diff --git a/tools/testing/selftests/bpf/test_bpftool.py b/tools/testing/selftests/bpf/test_bpftool.py new file mode 100644 index 000000000000..c8e54843a487 --- /dev/null +++ b/tools/testing/selftests/bpf/test_bpftool.py @@ -0,0 +1,179 @@ +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2020 SUSE LLC. + +import collections +import functools +import json +import os +import socket +import subprocess +import unittest + + +# Add the source tree of bpftool and /usr/local/sbin to PATH +cur_dir = os.path.dirname(os.path.realpath(__file__)) +bpftool_dir = os.path.abspath(os.path.join(cur_dir, "..", "..", "..", "..", + "tools", "bpf", "bpftool")) +os.environ["PATH"] = bpftool_dir + ":/usr/local/sbin:" + os.environ["PATH"] + + +class IfaceNotFoundError(Exception): + pass + + +class UnprivilegedUserError(Exception): + pass + + +def _bpftool(args, json=True): + _args = ["bpftool"] + if json: + _args.append("-j") + _args.extend(args) + + res = subprocess.run(_args, capture_output=True) + return res.stdout + + +def bpftool(args): + return _bpftool(args, json=False).decode("utf-8") + + +def bpftool_json(args): + res = _bpftool(args) + return json.loads(res) + + +def get_default_iface(): + for iface in socket.if_nameindex(): + if iface[1] != "lo": + return iface[1] + raise IfaceNotFoundError("Could not find any network interface to probe") + + +def default_iface(f): + @functools.wraps(f) + def wrapper(*args, **kwargs): + iface = get_default_iface() + return f(*args, iface, **kwargs) + return wrapper + + +class TestBpftool(unittest.TestCase): + @classmethod + def setUpClass(cls): + if os.getuid() != 0: + raise UnprivilegedUserError( + "This test suite needs root privileges") + + @default_iface + def test_feature_dev_json(self, iface): + unexpected_helpers = [ + "bpf_probe_write_user", + "bpf_trace_printk", + ] + expected_keys = [ + "syscall_config", + "program_types", + "map_types", + "helpers", + "misc", + ] + + res = bpftool_json(["feature", "probe", "dev", iface]) + # Check if the result has all expected keys. + self.assertCountEqual(res.keys(), expected_keys) + # Check if unexpected helpers are not included in helpers probes + # result. + for helpers in res["helpers"].values(): + for unexpected_helper in unexpected_helpers: + self.assertNotIn(unexpected_helper, helpers) + + def test_feature_kernel(self): + test_cases = [ + bpftool_json(["feature", "probe", "kernel"]), + bpftool_json(["feature", "probe"]), + bpftool_json(["feature"]), + ] + unexpected_helpers = [ + "bpf_probe_write_user", + "bpf_trace_printk", + ] + expected_keys = [ + "syscall_config", + "system_config", + "program_types", + "map_types", + "helpers", + "misc", + ] + + for tc in test_cases: + # Check if the result has all expected keys. + self.assertCountEqual(tc.keys(), expected_keys) + # Check if unexpected helpers are not included in helpers probes + # result. + for helpers in tc["helpers"].values(): + for unexpected_helper in unexpected_helpers: + self.assertNotIn(unexpected_helper, helpers) + + def test_feature_kernel_full(self): + test_cases = [ + bpftool_json(["feature", "probe", "kernel", "full"]), + bpftool_json(["feature", "probe", "full"]), + ] + expected_helpers = [ + "bpf_probe_write_user", + "bpf_trace_printk", + ] + + for tc in test_cases: + # Check if expected helpers are included at least once in any + # helpers list for any program type. Unfortunately we cannot assume + # that they will be included in all program types or a specific + # subset of programs. It depends on the kernel version and + # configuration. + found_helpers = False + + for helpers in tc["helpers"].values(): + if all(expected_helper in helpers + for expected_helper in expected_helpers): + found_helpers = True + break + + self.assertTrue(found_helpers) + + def test_feature_kernel_full_vs_not_full(self): + full_res = bpftool_json(["feature", "probe", "full"]) + not_full_res = bpftool_json(["feature", "probe"]) + not_full_set = set() + full_set = set() + + for helpers in full_res["helpers"].values(): + for helper in helpers: + full_set.add(helper) + + for helpers in not_full_res["helpers"].values(): + for helper in helpers: + not_full_set.add(helper) + + self.assertCountEqual(full_set - not_full_set, + {"bpf_probe_write_user", "bpf_trace_printk"}) + self.assertCountEqual(not_full_set - full_set, set()) + + def test_feature_macros(self): + expected_patterns = [ + r"/\*\*\* System call availability \*\*\*/", + r"#define HAVE_BPF_SYSCALL", + r"/\*\*\* eBPF program types \*\*\*/", + r"#define HAVE.*PROG_TYPE", + r"/\*\*\* eBPF map types \*\*\*/", + r"#define HAVE.*MAP_TYPE", + r"/\*\*\* eBPF helper functions \*\*\*/", + r"#define HAVE.*HELPER", + r"/\*\*\* eBPF misc features \*\*\*/", + ] + + res = bpftool(["feature", "probe", "macros"]) + for pattern in expected_patterns: + self.assertRegex(res, pattern) diff --git a/tools/testing/selftests/bpf/test_bpftool.sh b/tools/testing/selftests/bpf/test_bpftool.sh new file mode 100755 index 000000000000..66690778e36d --- /dev/null +++ b/tools/testing/selftests/bpf/test_bpftool.sh @@ -0,0 +1,5 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2020 SUSE LLC. + +python3 -m unittest -v test_bpftool.TestBpftool
Add Python module with tests for "bpftool feature" command, which mainly wheck whether the "full" option is working properly. Signed-off-by: Michal Rostecki <mrostecki@opensuse.org> --- tools/testing/selftests/.gitignore | 5 +- tools/testing/selftests/bpf/Makefile | 3 +- tools/testing/selftests/bpf/test_bpftool.py | 179 ++++++++++++++++++++ tools/testing/selftests/bpf/test_bpftool.sh | 5 + 4 files changed, 190 insertions(+), 2 deletions(-) create mode 100644 tools/testing/selftests/bpf/test_bpftool.py create mode 100755 tools/testing/selftests/bpf/test_bpftool.sh