| Message ID | 20190131081617.5005-1-masashi.honma@gmail.com |
|---|---|
| Headers | show |
| Series | Use python3 compatible codes for hwsim tests | expand |
On Thu, Jan 31, 2019 at 05:15:39PM +0900, Masashi Honma wrote: > This patch set modifies hwsim test to support python3 because python2 is > deprecated from 2020. After applying this patch set, user can run hwsim > test both python2 and python3. We can run with python3 by modifying > run-tests.py. Thanks, applied with number of fixes and cleanup. I ended up splitting the patches differently to make it easier to review the changes and to be able to pick the parts that looked correct to me. I did replace some areas by not doing unnecessary hexlify/unhexlify conversions (i.e., using binary comparisons where appropriate and hex strings were more convenient). In addition, I did more changes with wpaspy support. > [failing tests] > Some test cases fails. This is not because of python3, it fails with python2 > also. This is the list of the test cases. In addition to this, it looks like number of test cases were being skipped in your setup. Those test cases missed changes needed for python3. I implemented those and the end result can now pass every single test case with both python2 and python3. While the coverage looks more or less identical, there are some differences in particular due to the way dict iteration works. python3 ended up processing the dict entries in different order and that ended up actually finding a real bug in the D-Bus handling.. I did not yet check whether there could be other differences in coverage, so I left the python2 as the default for now. Anyway, the goal from now on would be to maintain working python3 support and likely remove that python2 default eventually. > [known issue] > Writing un-printable code with logger.debug() causes exception. > This occurs only at vm environment and this does not interrupt the test. > The test end with PASS even thought this exception. > > Traceback (most recent call last): > File "/usr/lib/python3.6/logging/__init__.py", line 996, in emit > stream.write(msg) > UnicodeEncodeError: 'ascii' codec can't encode character '\x80' in position 68: ordinal not in range(128) It looks like the logging.FileHandler() is not able to handle these and at least with Python 3.5.2 it did not even handle utf-8 which was a bit surprising. I tried to get that working, but did not find any easy way of doing so. Should try to figure out what exactly is happening here and why utf-8 part did not work.. For non-printable-even-with-utf-8-working cases, those should probably be filtered out by the code that generates such output. It does not really belong in a log file that is supposed to be text.
On 2019/02/05 10:21, Jouni Malinen wrote: > Thanks, applied with number of fixes and cleanup. I ended up splitting > the patches differently to make it easier to review the changes and to > be able to pick the parts that looked correct to me. I did replace some > areas by not doing unnecessary hexlify/unhexlify conversions (i.e., > using binary comparisons where appropriate and hex strings were more > convenient). In addition, I did more changes with wpaspy support. Thank you so much and sorry so much ! I will take care to write more precise commit log and not to send many patches at on time to reduce your review and clean up efforts. >> [failing tests] >> Some test cases fails. This is not because of python3, it fails with python2 >> also. This is the list of the test cases. > > In addition to this, it looks like number of test cases were being > skipped in your setup. Those test cases missed changes needed for > python3. I implemented those and the end result can now pass every > single test case with both python2 and python3. > > While the coverage looks more or less identical, there are some > differences in particular due to the way dict iteration works. python3 > ended up processing the dict entries in different order and that ended > up actually finding a real bug in the D-Bus handling.. I did not yet > check whether there could be other differences in coverage, so I left > the python2 as the default for now. Anyway, the goal from now on would > be to maintain working python3 support and likely remove that python2 > default eventually. Yes. I have a list of skipped test cases. 225 test cases are skipped even though I am using latest hostapd/wpa_supplicant and wireless-testing and vm/kernel-config. I must study more about this. >> [known issue] >> Writing un-printable code with logger.debug() causes exception. >> This occurs only at vm environment and this does not interrupt the test. >> The test end with PASS even thought this exception. >> >> Traceback (most recent call last): >> File "/usr/lib/python3.6/logging/__init__.py", line 996, in emit >> stream.write(msg) >> UnicodeEncodeError: 'ascii' codec can't encode character '\x80' in position 68: ordinal not in range(128) > > It looks like the logging.FileHandler() is not able to handle these and > at least with Python 3.5.2 it did not even handle utf-8 which was a bit > surprising. I tried to get that working, but did not find any easy way > of doing so. Should try to figure out what exactly is happening here and > why utf-8 part did not work.. For non-printable-even-with-utf-8-working > cases, those should probably be filtered out by the code that generates > such output. It does not really belong in a log file that is supposed to > be text. Thank you and sorry for incomplete patch set. Masashi Honma.
[summary] This patch set modifies hwsim test to support python3 because python2 is deprecated from 2020. After applying this patch set, user can run hwsim test both python2 and python3. We can run with python3 by modifying run-tests.py. #!/usr/bin/env python2 | v #!/usr/bin/env python3 [testing environment] I have tested all the test cases with vm-run.sh. The vm-run.sh test is using this wireless-testing kernel. commit 24e1d9cd149ef4daed8d1fb457704cd03172d777 'Add localversion to identify builds from this tree'. hostap commit 0eb34f8f2859e8d51d52c4e7781bdb135ba1b360 'tests: WPA2-PSK AP with PSK from a file (keyid and reload)' host OS Ubuntu 18.04 x64 python version Python 2.7.15rc1 Python 3.6.7 [failing tests] Some test cases fails. This is not because of python3, it fails with python2 also. This is the list of the test cases. ap_acs(5) ap_acs_5ghz ap_acs_5ghz_40mhz ap_acs_vht ap_acs_vht40 ap_acs_vht160 ap_params(2) ap_country ap_spectrum_management_required ap_vht(11) ap_vht160 ap_vht160b ap_vht160_no_dfs ap_vht160_no_dfs_108_plus ap_vht160_no_dfs_100_plus ap_vht160_no_dfs_112_minus ap_vht160_no_dfs_116_plus ap_vht160_no_dfs_120_minus ap_vht160_no_dfs_124_plus ap_vht160_no_dfs_128_minus ap_vht80_pwr_constraint erp(1) erp_key_lifetime_in_memory rrm(2) rrm_beacon_req_passive_scan_vht rrm_beacon_req_passive_scan_vht160 wmediumd(1) wmediumd_path_ttl wnm(10) wnm_bss_tm wnm_bss_tm_nei_11a wnm_bss_tm_nei_11g wnm_bss_tm_nei_11b wnm_bss_tm_nei_vht wnm_bss_tm_reject wnm_bss_tm_rsn wnm_bss_tm_scan_needed wnm_bss_tm_scan_needed_e4 wnm_bss_tm_scan_not_needed wpas_mesh(2) wpas_mesh_peer_connected wpas_mesh_max_peering [known issue] Writing un-printable code with logger.debug() causes exception. This occurs only at vm environment and this does not interrupt the test. The test end with PASS even thought this exception. Traceback (most recent call last): File "/usr/lib/python3.6/logging/__init__.py", line 996, in emit stream.write(msg) UnicodeEncodeError: 'ascii' codec can't encode character '\x80' in position 68: ordinal not in range(128) Call stack: File "/home/honma/git/hostap/tests/hwsim/run-tests.py", line 649, in <module> main() File "/home/honma/git/hostap/tests/hwsim/run-tests.py", line 497, in main t(dev, apdev) File "/home/honma/git/hostap/tests/hwsim/test_ap_eap.py", line 5004, in test_ap_wpa2_eap_non_ascii_identity2 identity=u"a\x80", password="password", wait_connect=False) File "/home/honma/git/hostap/tests/hwsim/wpasupplicant.py", line 1021, in connect self.set_network_quoted(id, field, kwargs[field]) File "/home/honma/git/hostap/tests/hwsim/wpasupplicant.py", line 293, in set_network_quoted res = self.request("SET_NETWORK " + str(id) + " " + field + ' "' + value + '"') File "/home/honma/git/hostap/tests/hwsim/wpasupplicant.py", line 173, in request logger.debug(self.dbg + ": CTRL: " + cmd) Message: 'wlan1: CTRL: SET_NETWORK 0 identity "a\x80"' Arguments: () Masashi Honma (38): tests: python3 require raw string tests: Use python3 compatible liblaries tests: Decode subprocess.check_output return value for python3 tests: Modify core functions for python3 tests: Modify ap_eap test sets for python3 tests: Modify ap_ft test sets for python3 tests: Modify ap_hs20 test sets for python3 tests: Modify ap_psk test sets for python3 tests: Modify ap_wps test sets for python3 tests: Modify cfg80211 test sets for python3 tests: Modify dbus test sets for python3 tests: Modify dbus_old test sets for python3 tests: Modify dpp test sets for python3 tests: Modify eap_proto test sets for python3 tests: Modify erp test sets for python3 tests: Fix erp_home_realm_oom error on python3 tests: Modify fils test sets for python3 tests: Modify fst_config test sets for python3 tests: Modify gas test sets for python3 tests: Modify hs20_filter test sets for python3 tests: Modify ieee8021x test sets for python3 tests: Modify kernel test sets for python3 tests: Modify macsec test sets for python3 tests: Modify mbo test sets for python3 tests: Modify nfc_wps test sets for python3 tests: Modify p2p_grpform test sets for python3 tests: Modify p2p_messages test sets for python3 tests: Modify radius test sets for python3 tests: Modify rrm test sets for python3 tests: Modify sae test sets for python3 tests: Modify sigma_dut test sets for python3 tests: Modify ssid test sets for python3 tests: Modify wmediumd test sets for python3 tests: Modify wnm test sets for python3 tests: Fix failure for WPA-EAP-SUITE-B-192 key_mgmt tests: Modify wpas_ctrl test sets for python3 tests: Modify wpas_mesh test sets for python3 tests: Modify wpas_wmm_ac test sets for python3 tests/hwsim/fst_test_common.py | 2 +- tests/hwsim/hostapd.py | 9 +- tests/hwsim/hwsim.py | 2 +- tests/hwsim/netlink.py | 11 +- tests/hwsim/nl80211.py | 4 +- tests/hwsim/p2p_utils.py | 9 +- tests/hwsim/remotehost.py | 6 +- tests/hwsim/rfkill.py | 12 +- tests/hwsim/run-tests.py | 2 +- tests/hwsim/test_ap_eap.py | 73 +-- tests/hwsim/test_ap_ft.py | 86 ++-- tests/hwsim/test_ap_hs20.py | 191 ++++---- tests/hwsim/test_ap_psk.py | 46 +- tests/hwsim/test_ap_wps.py | 637 +++++++++++++++------------ tests/hwsim/test_cfg80211.py | 4 +- tests/hwsim/test_dbus.py | 377 ++++++++-------- tests/hwsim/test_dbus_old.py | 23 +- tests/hwsim/test_dpp.py | 71 +-- tests/hwsim/test_eap_proto.py | 20 +- tests/hwsim/test_erp.py | 6 +- tests/hwsim/test_fils.py | 138 +++--- tests/hwsim/test_gas.py | 50 +-- tests/hwsim/test_hs20_filter.py | 10 +- tests/hwsim/test_hs20_pps_mo.py | 2 +- tests/hwsim/test_ieee8021x.py | 2 +- tests/hwsim/test_kernel.py | 4 +- tests/hwsim/test_macsec.py | 8 +- tests/hwsim/test_mbo.py | 14 +- tests/hwsim/test_nfc_wps.py | 5 +- tests/hwsim/test_p2p_grpform.py | 5 +- tests/hwsim/test_p2p_messages.py | 95 ++-- tests/hwsim/test_p2p_wifi_display.py | 5 +- tests/hwsim/test_radius.py | 2 +- tests/hwsim/test_rrm.py | 10 +- tests/hwsim/test_sae.py | 12 +- tests/hwsim/test_sigma_dut.py | 2 +- tests/hwsim/test_ssid.py | 4 +- tests/hwsim/test_wmediumd.py | 10 +- tests/hwsim/test_wnm.py | 16 +- tests/hwsim/test_wpas_config.py | 4 + tests/hwsim/test_wpas_ctrl.py | 15 +- tests/hwsim/test_wpas_mesh.py | 47 +- tests/hwsim/test_wpas_wmm_ac.py | 8 +- tests/hwsim/tshark.py | 12 +- tests/hwsim/wlantest.py | 2 +- tests/hwsim/wpasupplicant.py | 11 +- wpaspy/wpaspy.py | 13 +- 47 files changed, 1149 insertions(+), 948 deletions(-)