| Message ID | 20201028183554.18078-1-pusharma@akamai.com |
|---|---|
| State | Changes Requested |
| Delegated to: | David Ahern |
| Headers | show |
| Series | [iproute2] tc: add print options to fix json output | expand |
| Context | Check | Description |
|---|---|---|
| jkicinski/tree_selection | success | Not a local patch |
On Wed, 28 Oct 2020 14:35:54 -0400 Puneet Sharma <pusharma@akamai.com> wrote: > Currently, json for basic rules output does not produce correct json > syntax. The following fixes were done to correct it for extended > matches for use with "basic" filters. > > tc/f_basic.c: replace fprintf with print_uint to support json output. > fixing this prints "handle" tag correctly in json output. > > tc/m_ematch.c: replace various fprintf with correct print. > add new "ematch" tag for json output which represents > "tc filter add ... basic match '()'" string. Added print_raw_string > to print raw string instead of key value for json. > > lib/json_writer.c: add jsonw_raw_string to print raw text in json. > > lib/json_print.c: add print_color_raw_string to print string > depending on context. > > example: > $ tc -s -d -j filter show dev <eth_name> ingress > Before: > ... > "options": {handle 0x2 > ( > cmp(u8 at 9 layer 1 eq 6) > OR cmp(u8 at 9 layer 1 eq 17) > ) AND ipset(test-ipv4 src) > > "actions": [{ > ... > > After: > [{ > ... > "options": { > "handle": 1, > "ematch": "(cmp(u8 at 9 layer 1 eq 6)OR cmp(u8 at 9 layer 1 eq 17)) AND ipset(test-ipv4 src)", > ... > ] > > Signed-off-by: Puneet Sharma <pusharma@akamai.com> > --- What is the point of introducing raw string? The JSON standard says that string fields must use proper escapes. Please don't emit invalid JSON. It will break consumption by other libraries.
On Thu, 29 Oct 2020 21:20:55 +0000 "Sharma, Puneet" <pusharma@akamai.com> wrote: > I did provide an example to better explain what patch is doing. > > Sorry for long paste. > > So, with current implementation output of command: > $ tc -s -d -j filter show dev <eth_name> ingress > > would contain: > [{ > "protocol": "ip", > "pref": 20000, > "kind": "basic", > "chain": 0 > },{ > "protocol": "ip", > "pref": 20000, > "kind": "basic", > "chain": 0, > "options": {handle 0x1 > ( > cmp(u8 at 9 layer 1 eq 6) > OR cmp(u8 at 9 layer 1 eq 17) > ) AND ipset(sg-test-ipv4 src) > > "actions": [{ > "order": 1, > "kind": "gact", > "control_action": { > "type": "pass" > }, > "prob": { > "random_type": "none", > "control_action": { > "type": "pass" > }, > "val": 0 > }, > "index": 1, > "ref": 1, > "bind": 1, > "installed": 2633, > "last_used": 2633, > "stats": { > "bytes": 0, > "packets": 0, > "drops": 0, > "overlimits": 0, > "requeues": 0, > "backlog": 0, > "qlen": 0 > } > }] > } > } > ] > > Clearly this is an invalid JSON. Look at “options" > > > With patch it would look like: > [{ > "protocol": "ip", > "pref": 20000, > "kind": "basic", > "chain": 0 > },{ > "protocol": "ip", > "pref": 20000, > "kind": "basic", > "chain": 0, > "options": { > "handle": 1, > "ematch": "(cmp(u8 at 9 layer 1 eq 6)OR cmp(u8 at 9 layer 1 eq 17)) AND ipset(sg-test-ipv4 src)", > "actions": [{ > "order": 1, > "kind": "gact", > "control_action": { > "type": "pass" > }, > "prob": { > "random_type": "none", > "control_action": { > "type": "pass" > }, > "val": 0 > }, > "index": 1, > "ref": 1, > "bind": 1, > "installed": 297829723, > "last_used": 297829723, > "stats": { > "bytes": 0, > "packets": 0, > "drops": 0, > "overlimits": 0, > "requeues": 0, > "backlog": 0, > "qlen": 0 > } > }] > } > } > ] > > Now it’s handling the “handle” and “ematch” inside “options" depending on context. > > Hope it’s more clear now. > > Thanks, > ~Puneet. > > > On Oct 29, 2020, at 4:17 PM, Stephen Hemminger <stephen@networkplumber.org> wrote: > > > > On Wed, 28 Oct 2020 14:35:54 -0400 > > Puneet Sharma <pusharma@akamai.com> wrote: > > > >> Currently, json for basic rules output does not produce correct json > >> syntax. The following fixes were done to correct it for extended > >> matches for use with "basic" filters. > >> > >> tc/f_basic.c: replace fprintf with print_uint to support json output. > >> fixing this prints "handle" tag correctly in json output. > >> > >> tc/m_ematch.c: replace various fprintf with correct print. > >> add new "ematch" tag for json output which represents > >> "tc filter add ... basic match '()'" string. Added print_raw_string > >> to print raw string instead of key value for json. > >> > >> lib/json_writer.c: add jsonw_raw_string to print raw text in json. > >> > >> lib/json_print.c: add print_color_raw_string to print string > >> depending on context. > >> > >> example: > >> $ tc -s -d -j filter show dev <eth_name> ingress > >> Before: > >> ... > >> "options": {handle 0x2 > >> ( > >> cmp(u8 at 9 layer 1 eq 6) > >> OR cmp(u8 at 9 layer 1 eq 17) > >> ) AND ipset(test-ipv4 src) > >> > >> "actions": [{ > >> ... > >> > >> After: > >> [{ > >> ... > >> "options": { > >> "handle": 1, > >> "ematch": "(cmp(u8 at 9 layer 1 eq 6)OR cmp(u8 at 9 layer 1 eq 17)) AND ipset(test-ipv4 src)", > >> ... > >> ] > >> > >> Signed-off-by: Puneet Sharma <pusharma@akamai.com> > >> --- > > > > What is the point of introducing raw string? > > The JSON standard says that string fields must use proper escapes. > > > > Please don't emit invalid JSON. It will break consumption by other libraries. > I agree that the existing output is wrong. But your patch introduces +void jsonw_raw_string(json_writer_t *self, const char *value); Why? You should just use jsonw_string() which already handles things like special characters in the string. In theory, there could be quotes or other characters in the ematch string.
Because basic match is made of multiple keywords and parsed and handle differently
example:
$ tc filter add dev $eth_dev_name ingress priority 20000 protocol ipv4 basic match '(cmp(u8 at 9 layer network eq 6) or cmp(u8 at 9 layer network eq 17)) and ipset(sg-test-ipv4 src)' action pass
and if jsonw_string used then it will double-quote every string passed and
prints something like this:
"ematch": "("cmp(u8 at 9 layer 1 eq 6,")","OR "cmp(u8 at 9 layer 1 eq 17,")",") ","AND "ipset(sg-test-ipv4 src,")",
instead of like this:
"ematch": "(cmp(u8 at 9 layer 1 eq 6)OR cmp(u8 at 9 layer 1 eq 17)) AND ipset(sg-test-ipv4 src)”,
Hope I explained it right.
> On Oct 29, 2020, at 7:16 PM, Stephen Hemminger <stephen@networkplumber.org> wrote:
>
> On Thu, 29 Oct 2020 21:20:55 +0000
> "Sharma, Puneet" <pusharma@akamai.com> wrote:
>
>> I did provide an example to better explain what patch is doing.
>>
>> Sorry for long paste.
>>
>> So, with current implementation output of command:
>> $ tc -s -d -j filter show dev <eth_name> ingress
>>
>> would contain:
>> [{
>> "protocol": "ip",
>> "pref": 20000,
>> "kind": "basic",
>> "chain": 0
>> },{
>> "protocol": "ip",
>> "pref": 20000,
>> "kind": "basic",
>> "chain": 0,
>> "options": {handle 0x1
>> (
>> cmp(u8 at 9 layer 1 eq 6)
>> OR cmp(u8 at 9 layer 1 eq 17)
>> ) AND ipset(sg-test-ipv4 src)
>>
>> "actions": [{
>> "order": 1,
>> "kind": "gact",
>> "control_action": {
>> "type": "pass"
>> },
>> "prob": {
>> "random_type": "none",
>> "control_action": {
>> "type": "pass"
>> },
>> "val": 0
>> },
>> "index": 1,
>> "ref": 1,
>> "bind": 1,
>> "installed": 2633,
>> "last_used": 2633,
>> "stats": {
>> "bytes": 0,
>> "packets": 0,
>> "drops": 0,
>> "overlimits": 0,
>> "requeues": 0,
>> "backlog": 0,
>> "qlen": 0
>> }
>> }]
>> }
>> }
>> ]
>>
>> Clearly this is an invalid JSON. Look at “options"
>>
>>
>> With patch it would look like:
>> [{
>> "protocol": "ip",
>> "pref": 20000,
>> "kind": "basic",
>> "chain": 0
>> },{
>> "protocol": "ip",
>> "pref": 20000,
>> "kind": "basic",
>> "chain": 0,
>> "options": {
>> "handle": 1,
>> "ematch": "(cmp(u8 at 9 layer 1 eq 6)OR cmp(u8 at 9 layer 1 eq 17)) AND ipset(sg-test-ipv4 src)",
>> "actions": [{
>> "order": 1,
>> "kind": "gact",
>> "control_action": {
>> "type": "pass"
>> },
>> "prob": {
>> "random_type": "none",
>> "control_action": {
>> "type": "pass"
>> },
>> "val": 0
>> },
>> "index": 1,
>> "ref": 1,
>> "bind": 1,
>> "installed": 297829723,
>> "last_used": 297829723,
>> "stats": {
>> "bytes": 0,
>> "packets": 0,
>> "drops": 0,
>> "overlimits": 0,
>> "requeues": 0,
>> "backlog": 0,
>> "qlen": 0
>> }
>> }]
>> }
>> }
>> ]
>>
>> Now it’s handling the “handle” and “ematch” inside “options" depending on context.
>>
>> Hope it’s more clear now.
>>
>> Thanks,
>> ~Puneet.
>>
>>> On Oct 29, 2020, at 4:17 PM, Stephen Hemminger <stephen@networkplumber.org> wrote:
>>>
>>> On Wed, 28 Oct 2020 14:35:54 -0400
>>> Puneet Sharma <pusharma@akamai.com> wrote:
>>>
>>>> Currently, json for basic rules output does not produce correct json
>>>> syntax. The following fixes were done to correct it for extended
>>>> matches for use with "basic" filters.
>>>>
>>>> tc/f_basic.c: replace fprintf with print_uint to support json output.
>>>> fixing this prints "handle" tag correctly in json output.
>>>>
>>>> tc/m_ematch.c: replace various fprintf with correct print.
>>>> add new "ematch" tag for json output which represents
>>>> "tc filter add ... basic match '()'" string. Added print_raw_string
>>>> to print raw string instead of key value for json.
>>>>
>>>> lib/json_writer.c: add jsonw_raw_string to print raw text in json.
>>>>
>>>> lib/json_print.c: add print_color_raw_string to print string
>>>> depending on context.
>>>>
>>>> example:
>>>> $ tc -s -d -j filter show dev <eth_name> ingress
>>>> Before:
>>>> ...
>>>> "options": {handle 0x2
>>>> (
>>>> cmp(u8 at 9 layer 1 eq 6)
>>>> OR cmp(u8 at 9 layer 1 eq 17)
>>>> ) AND ipset(test-ipv4 src)
>>>>
>>>> "actions": [{
>>>> ...
>>>>
>>>> After:
>>>> [{
>>>> ...
>>>> "options": {
>>>> "handle": 1,
>>>> "ematch": "(cmp(u8 at 9 layer 1 eq 6)OR cmp(u8 at 9 layer 1 eq 17)) AND ipset(test-ipv4 src)",
>>>> ...
>>>> ]
>>>>
>>>> Signed-off-by: Puneet Sharma <pusharma@akamai.com>
>>>> ---
>>>
>>> What is the point of introducing raw string?
>>> The JSON standard says that string fields must use proper escapes.
>>>
>>> Please don't emit invalid JSON. It will break consumption by other libraries.
>>
>
>
> I agree that the existing output is wrong. But your patch introduces
>
> +void jsonw_raw_string(json_writer_t *self, const char *value);
>
> Why?
>
> You should just use jsonw_string() which already handles things like special
> characters in the string. In theory, there could be quotes or other characters
> in the ematch string.
On 10/29/20 6:42 PM, Sharma, Puneet wrote: > Because basic match is made of multiple keywords and parsed and handle differently > example: > $ tc filter add dev $eth_dev_name ingress priority 20000 protocol ipv4 basic match '(cmp(u8 at 9 layer network eq 6) or cmp(u8 at 9 layer network eq 17)) and ipset(sg-test-ipv4 src)' action pass > > and if jsonw_string used then it will double-quote every string passed and > > prints something like this: > "ematch": "("cmp(u8 at 9 layer 1 eq 6,")","OR "cmp(u8 at 9 layer 1 eq 17,")",") ","AND "ipset(sg-test-ipv4 src,")", > I think you need to print that string to a temp buffer to collect the complete expression and then just call print_string on it.
diff --git a/include/json_print.h b/include/json_print.h index 50e71de4..91af7295 100644 --- a/include/json_print.h +++ b/include/json_print.h @@ -67,6 +67,7 @@ _PRINT_FUNC(s64, int64_t) _PRINT_FUNC(bool, bool) _PRINT_FUNC(null, const char*) _PRINT_FUNC(string, const char*) +_PRINT_FUNC(raw_string, const char*) _PRINT_FUNC(uint, unsigned int) _PRINT_FUNC(u64, uint64_t) _PRINT_FUNC(hhu, unsigned char) diff --git a/include/json_writer.h b/include/json_writer.h index b52dc2d0..afb561a6 100644 --- a/include/json_writer.h +++ b/include/json_writer.h @@ -31,6 +31,7 @@ void jsonw_name(json_writer_t *self, const char *name); /* Add value */ __attribute__((format(printf, 2, 3))) void jsonw_printf(json_writer_t *self, const char *fmt, ...); +void jsonw_raw_string(json_writer_t *self, const char *value); void jsonw_string(json_writer_t *self, const char *value); void jsonw_bool(json_writer_t *self, bool value); void jsonw_float(json_writer_t *self, double number); diff --git a/lib/json_print.c b/lib/json_print.c index fe0705bf..ff76afba 100644 --- a/lib/json_print.c +++ b/lib/json_print.c @@ -186,6 +186,23 @@ int print_color_string(enum output_type type, return ret; } +int print_color_raw_string(enum output_type type, + enum color_attr color, + const char *key, + const char *fmt, + const char *value) +{ + int ret = 0; + + if (_IS_JSON_CONTEXT(type)) + jsonw_raw_string(_jw, fmt); + else if (_IS_FP_CONTEXT(type)) { + ret = color_fprintf(stdout, color, fmt, value); + } + + return ret; +} + /* * value's type is bool. When using this function in FP context you can't pass * a value to it, you will need to use "is_json_context()" to have different diff --git a/lib/json_writer.c b/lib/json_writer.c index 88c5eb88..80ab0a20 100644 --- a/lib/json_writer.c +++ b/lib/json_writer.c @@ -51,6 +51,11 @@ static void jsonw_eor(json_writer_t *self) self->sep = ','; } +void jsonw_raw_string(json_writer_t *self, const char *str) +{ + for (; *str; ++str) + putc(*str, self->out); +} /* Output JSON encoded string */ /* Handles C escapes, does not do Unicode */ diff --git a/tc/f_basic.c b/tc/f_basic.c index 7b19cea6..444d4297 100644 --- a/tc/f_basic.c +++ b/tc/f_basic.c @@ -119,7 +119,7 @@ static int basic_print_opt(struct filter_util *qu, FILE *f, parse_rtattr_nested(tb, TCA_BASIC_MAX, opt); if (handle) - fprintf(f, "handle 0x%x ", handle); + print_uint(PRINT_ANY, "handle", "handle 0x%x ", handle); if (tb[TCA_BASIC_CLASSID]) { SPRINT_BUF(b1); diff --git a/tc/m_ematch.c b/tc/m_ematch.c index 8840a0dc..eee3819f 100644 --- a/tc/m_ematch.c +++ b/tc/m_ematch.c @@ -408,7 +408,7 @@ static int print_ematch_seq(FILE *fd, struct rtattr **tb, int start, hdr = RTA_DATA(tb[i]); if (hdr->flags & TCF_EM_INVERT) - fprintf(fd, "NOT "); + print_raw_string(PRINT_ANY, NULL, "NOT ", NULL); if (hdr->kind == 0) { __u32 ref; @@ -417,14 +417,15 @@ static int print_ematch_seq(FILE *fd, struct rtattr **tb, int start, return -1; ref = *(__u32 *) data; - fprintf(fd, "(\n"); + print_raw_string(PRINT_ANY, NULL, "(", NULL); + print_string(PRINT_FP, NULL, "\n", NULL); for (n = 0; n <= prefix; n++) - fprintf(fd, " "); + print_string(PRINT_FP, NULL, " ", NULL); if (print_ematch_seq(fd, tb, ref + 1, prefix + 1) < 0) return -1; for (n = 0; n < prefix; n++) - fprintf(fd, " "); - fprintf(fd, ") "); + print_string(PRINT_FP, NULL, " ", NULL); + print_raw_string(PRINT_ANY, NULL, ") ", NULL); } else { struct ematch_util *e; @@ -437,20 +438,21 @@ static int print_ematch_seq(FILE *fd, struct rtattr **tb, int start, fprintf(fd, "%s(", e->kind); if (e->print_eopt(fd, hdr, data, dlen) < 0) return -1; - fprintf(fd, ")\n"); + print_raw_string(PRINT_ANY, NULL, ")", NULL); + print_string(PRINT_FP, NULL, "\n", NULL); } if (hdr->flags & TCF_EM_REL_MASK) for (n = 0; n < prefix; n++) - fprintf(fd, " "); + print_string(PRINT_FP, NULL, " ", NULL); } switch (hdr->flags & TCF_EM_REL_MASK) { case TCF_EM_REL_AND: - fprintf(fd, "AND "); + print_raw_string(PRINT_ANY, NULL, "AND ", NULL); break; case TCF_EM_REL_OR: - fprintf(fd, "OR "); + print_raw_string(PRINT_ANY, NULL, "OR ", NULL); break; default: @@ -477,9 +479,13 @@ static int print_ematch_list(FILE *fd, struct tcf_ematch_tree_hdr *hdr, if (parse_rtattr_nested(tb, hdr->nmatches, rta) < 0) goto errout; - fprintf(fd, "\n "); + print_string(PRINT_FP, NULL, "\n ", NULL); + print_string(PRINT_JSON, "ematch", NULL, NULL); + print_raw_string(PRINT_JSON, NULL, "\"", NULL); if (print_ematch_seq(fd, tb, 1, 1) < 0) goto errout; + + print_raw_string(PRINT_JSON, NULL, "\",", NULL); } err = 0;
Currently, json for basic rules output does not produce correct json syntax. The following fixes were done to correct it for extended matches for use with "basic" filters. tc/f_basic.c: replace fprintf with print_uint to support json output. fixing this prints "handle" tag correctly in json output. tc/m_ematch.c: replace various fprintf with correct print. add new "ematch" tag for json output which represents "tc filter add ... basic match '()'" string. Added print_raw_string to print raw string instead of key value for json. lib/json_writer.c: add jsonw_raw_string to print raw text in json. lib/json_print.c: add print_color_raw_string to print string depending on context. example: $ tc -s -d -j filter show dev <eth_name> ingress Before: ... "options": {handle 0x2 ( cmp(u8 at 9 layer 1 eq 6) OR cmp(u8 at 9 layer 1 eq 17) ) AND ipset(test-ipv4 src) "actions": [{ ... After: [{ ... "options": { "handle": 1, "ematch": "(cmp(u8 at 9 layer 1 eq 6)OR cmp(u8 at 9 layer 1 eq 17)) AND ipset(test-ipv4 src)", ... ] Signed-off-by: Puneet Sharma <pusharma@akamai.com> --- include/json_print.h | 1 + include/json_writer.h | 1 + lib/json_print.c | 17 +++++++++++++++++ lib/json_writer.c | 5 +++++ tc/f_basic.c | 2 +- tc/m_ematch.c | 26 ++++++++++++++++---------- 6 files changed, 41 insertions(+), 11 deletions(-)