| Message ID | 20181220022952.20493-8-jsnow@redhat.com |
|---|---|
| State | New |
| Headers | show |
| Series | bitmaps: remove x- prefix from QMP api | expand |
On 12/19/18 8:29 PM, John Snow wrote: > Python before 3.6 does not sort dictionaries (including kwargs). > Therefore, printing QMP objects involves sorting the keys to have > a predictable ordering in the iotests output. It may be worth also mentioning that sometimes this sorting results in the log showing things in a different order than the source command (with no ill effect, as long as the output order is deterministic). > > However, if we want to pretty-print QMP objects being sent to the > QEMU process, we need to build the entire command before logging it. > Ordinarily, this would then involve "arguments" being sorted above > "execute", which would necessitate a rather ugly and harder-to-read > change to many iotests outputs. > > To facilitate pretty-printing AND maintaining predictable output AND > having "arguments" sort before "execute", add a custom sort function s/before/after/ > that takes a dictionary and recursively builds an OrderedDict that > maintains the specific key order we wish to see in iotests output. namely, keys within subdicts are sorted by key name (even if that is not the order they were input), but the top-level struct with "execute" and "arguments" stays the way we want it. > > Signed-off-by: John Snow <jsnow@redhat.com> > --- > tests/qemu-iotests/iotests.py | 24 ++++++++++++++++++++---- > 1 file changed, 20 insertions(+), 4 deletions(-) Reviewed-by: Eric Blake <eblake@redhat.com>
20.12.2018 5:29, John Snow wrote: > Python before 3.6 does not sort dictionaries (including kwargs). > Therefore, printing QMP objects involves sorting the keys to have > a predictable ordering in the iotests output. > > However, if we want to pretty-print QMP objects being sent to the > QEMU process, we need to build the entire command before logging it. > Ordinarily, this would then involve "arguments" being sorted above > "execute", which would necessitate a rather ugly and harder-to-read > change to many iotests outputs. I'm unsure about what it means 'build the entire command before logging'. [upd, after a second] aha, it's about '{"execute":...' -> {'execute': ...} may be, build the entire command object to be passed to json.dumps, or like this would be better, if you want. > > To facilitate pretty-printing AND maintaining predictable output AND > having "arguments" sort before "execute", add a custom sort function > that takes a dictionary and recursively builds an OrderedDict that > maintains the specific key order we wish to see in iotests output. > > Signed-off-by: John Snow<jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 9595429fea..565eebb1ab 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -30,6 +30,7 @@ import signal import logging import atexit import io +from collections import OrderedDict sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'scripts')) import qtest @@ -75,6 +76,16 @@ def qemu_img(*args): sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' '.join(qemu_img_args + list(args)))) return exitcode +def ordered_kwargs(kwargs): + # kwargs prior to 3.6 are not ordered, so: + od = OrderedDict() + for k, v in sorted(kwargs.items()): + if isinstance(v, dict): + od[k] = ordered_kwargs(v) + else: + od[k] = v + return od + def qemu_img_create(*args): args = list(args) @@ -257,8 +268,10 @@ def filter_img_info(output, filename): def log(msg, filters=[]): for flt in filters: msg = flt(msg) - if type(msg) is dict or type(msg) is list: - print(json.dumps(msg, sort_keys=True)) + if isinstance(msg, dict) or isinstance(msg, list): + # Don't sort if it's already sorted + do_sort = not isinstance(msg, OrderedDict) + print(json.dumps(msg, sort_keys=do_sort)) else: print(msg) @@ -448,8 +461,11 @@ class VM(qtest.QEMUQtestMachine): return result def qmp_log(self, cmd, filters=[filter_testfiles], **kwargs): - logmsg = '{"execute": "%s", "arguments": %s}' % \ - (cmd, json.dumps(kwargs, sort_keys=True)) + full_cmd = OrderedDict(( + ("execute", cmd), + ("arguments", ordered_kwargs(kwargs)) + )) + logmsg = json.dumps(full_cmd) log(logmsg, filters) result = self.qmp(cmd, **kwargs) log(json.dumps(result, sort_keys=True), filters)
Python before 3.6 does not sort dictionaries (including kwargs). Therefore, printing QMP objects involves sorting the keys to have a predictable ordering in the iotests output. However, if we want to pretty-print QMP objects being sent to the QEMU process, we need to build the entire command before logging it. Ordinarily, this would then involve "arguments" being sorted above "execute", which would necessitate a rather ugly and harder-to-read change to many iotests outputs. To facilitate pretty-printing AND maintaining predictable output AND having "arguments" sort before "execute", add a custom sort function that takes a dictionary and recursively builds an OrderedDict that maintains the specific key order we wish to see in iotests output. Signed-off-by: John Snow <jsnow@redhat.com> --- tests/qemu-iotests/iotests.py | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-)