{"id":2197697,"url":"http://patchwork.ozlabs.org/api/1.0/patches/2197697/?format=json","project":{"id":47,"url":"http://patchwork.ozlabs.org/api/1.0/projects/47/?format=json","name":"Open vSwitch","link_name":"openvswitch","list_id":"ovs-dev.openvswitch.org","list_email":"ovs-dev@openvswitch.org","web_url":"http://openvswitch.org/","scm_url":"git@github.com:openvswitch/ovs.git","webscm_url":"https://github.com/openvswitch/ovs"},"msgid":"<20260218103716.3135692-3-amorenoz@redhat.com>","date":"2026-02-18T10:37:16","name":"[ovs-dev,RFC,v1,2/2] ofproto-dpif_upcall: Implement upcall tracing.","commit_ref":null,"pull_url":null,"state":"rfc","archived":false,"hash":"2c761274b6d92b39441134442d8e3d0258770412","submitter":{"id":77477,"url":"http://patchwork.ozlabs.org/api/1.0/people/77477/?format=json","name":"Adrián Moreno","email":"amorenoz@redhat.com"},"delegate":{"id":57772,"url":"http://patchwork.ozlabs.org/api/1.0/users/57772/?format=json","username":"imaximets","first_name":"Ilya","last_name":"Maximets","email":"i.maximets@samsung.com"},"mbox":"http://patchwork.ozlabs.org/project/openvswitch/patch/20260218103716.3135692-3-amorenoz@redhat.com/mbox/","series":[{"id":492538,"url":"http://patchwork.ozlabs.org/api/1.0/series/492538/?format=json","date":"2026-02-18T10:37:14","name":"Introduce upcall (live) tracing.","version":1,"mbox":"http://patchwork.ozlabs.org/series/492538/mbox/"}],"check":"success","checks":"http://patchwork.ozlabs.org/api/patches/2197697/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","dev@openvswitch.org"],"Delivered-To":["patchwork-incoming@legolas.ozlabs.org","ovs-dev@lists.linuxfoundation.org"],"Authentication-Results":["legolas.ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256\n header.s=mimecast20190719 header.b=SqMdIUdz;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org\n (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org;\n envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org)","smtp4.osuosl.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key)\n header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256\n header.s=mimecast20190719 header.b=SqMdIUdz","smtp4.osuosl.org; dmarc=pass (p=quarantine dis=none)\n header.from=redhat.com"],"Received":["from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519 server-signature ECDSA (secp384r1) server-digest SHA384)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4fGCdG4ZJ8z1xpY\n\tfor ; Wed, 18 Feb 2026 21:37:38 +1100 (AEDT)","from localhost (localhost [127.0.0.1])\n\tby smtp4.osuosl.org (Postfix) with ESMTP id 995034081D;\n\tWed, 18 Feb 2026 10:37:34 +0000 (UTC)","from smtp4.osuosl.org ([127.0.0.1])\n by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP\n id fDDdhYdenXDJ; Wed, 18 Feb 2026 10:37:31 +0000 (UTC)","from lists.linuxfoundation.org (lf-lists.osuosl.org\n [IPv6:2605:bc80:3010:104::8cd3:938])\n\tby smtp4.osuosl.org (Postfix) with ESMTPS id 3D4D8407FF;\n\tWed, 18 Feb 2026 10:37:31 +0000 (UTC)","from lf-lists.osuosl.org (localhost [127.0.0.1])\n\tby lists.linuxfoundation.org (Postfix) with ESMTP id B4E86C0078;\n\tWed, 18 Feb 2026 10:37:30 +0000 (UTC)","from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137])\n by lists.linuxfoundation.org (Postfix) with ESMTP id 81711C0033\n for ; Wed, 18 Feb 2026 10:37:29 +0000 (UTC)","from localhost (localhost [127.0.0.1])\n by smtp4.osuosl.org (Postfix) with ESMTP id 510B040368\n for ; Wed, 18 Feb 2026 10:37:29 +0000 (UTC)","from smtp4.osuosl.org ([127.0.0.1])\n by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP\n id qPtiwhY3sXZF for ;\n Wed, 18 Feb 2026 10:37:28 +0000 (UTC)","from us-smtp-delivery-124.mimecast.com\n (us-smtp-delivery-124.mimecast.com [170.10.133.124])\n by smtp4.osuosl.org (Postfix) with ESMTPS id ED9324068F\n for ; Wed, 18 Feb 2026 10:37:27 +0000 (UTC)","from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com\n (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by\n relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3,\n cipher=TLS_AES_256_GCM_SHA384) id us-mta-658-ks2WI21RN1a58xQiefj38Q-1; Wed,\n 18 Feb 2026 05:37:25 -0500","from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com\n (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12])\n (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest\n SHA256)\n (No client certificate requested)\n by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS\n id 4371A18002C3\n for ; Wed, 18 Feb 2026 10:37:24 +0000 (UTC)","from antares.redhat.com (unknown [10.45.224.73])\n by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP\n id CE5BC19560AD; Wed, 18 Feb 2026 10:37:22 +0000 (UTC)"],"X-Virus-Scanned":["amavis at osuosl.org","amavis at osuosl.org"],"X-Comment":"SPF check N/A for local connections -\n client-ip=2605:bc80:3010:104::8cd3:938; helo=lists.linuxfoundation.org;\n envelope-from=ovs-dev-bounces@openvswitch.org; receiver= ","DKIM-Filter":["OpenDKIM Filter v2.11.0 smtp4.osuosl.org 3D4D8407FF","OpenDKIM Filter v2.11.0 smtp4.osuosl.org ED9324068F"],"Received-SPF":"Pass (mailfrom) identity=mailfrom; client-ip=170.10.133.124;\n helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com;\n receiver=","DMARC-Filter":"OpenDMARC Filter v1.4.2 smtp4.osuosl.org ED9324068F","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n s=mimecast20190719; t=1771411046;\n h=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n to:to:cc:cc:mime-version:mime-version:content-type:content-type:\n content-transfer-encoding:content-transfer-encoding:\n in-reply-to:in-reply-to:references:references;\n bh=BhPak/2MZTd79y1LYyeAWiu3AmtSpbm+AGfYrjkJkEw=;\n b=SqMdIUdz2KMTnNZ9YEYrTcqOPdCj61raQ4aviIYxSIS13s26y5ALYi4KcPlwaBcKfrrN/6\n acVirAfdj7rEJcchKpx8Ev5gu0FTvwpK4wIjv7m6JYwZ0B0iH50lvM+xcGry2fGlczUy4d\n nUkprGK8HjJxRCRDZya13s7HFbSMdz4=","X-MC-Unique":"ks2WI21RN1a58xQiefj38Q-1","X-Mimecast-MFC-AGG-ID":"ks2WI21RN1a58xQiefj38Q_1771411044","To":"dev@openvswitch.org","Date":"Wed, 18 Feb 2026 11:37:16 +0100","Message-ID":"<20260218103716.3135692-3-amorenoz@redhat.com>","In-Reply-To":"<20260218103716.3135692-1-amorenoz@redhat.com>","References":"<20260218103716.3135692-1-amorenoz@redhat.com>","MIME-Version":"1.0","X-Scanned-By":"MIMEDefang 3.0 on 10.30.177.12","X-Mimecast-Spam-Score":"0","X-Mimecast-MFC-PROC-ID":"3SvjWTdAW_0iw0v7LVazNy0Q4qdpAGaw4iIgCXid_Uo_1771411044","X-Mimecast-Originator":"redhat.com","Subject":"[ovs-dev] [RFC PATCH v1 2/2] ofproto-dpif_upcall: Implement upcall\n tracing.","X-BeenThere":"ovs-dev@openvswitch.org","X-Mailman-Version":"2.1.30","Precedence":"list","List-Id":"","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","From":"Adrian Moreno via dev ","Reply-To":"Adrian Moreno ","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"ovs-dev-bounces@openvswitch.org","Sender":"\"dev\" "},"content":"Upcall tracing is implemented using 3 objects.\n\n- upcall_tracing: Stores the current configuration provided by the user\n (e.g: filter), and a fixed-size list of \"upcall_trace_block\"s.\n- upcall_trace_block: Groups several \"upcall_trace\" objects together\n based on a \"trace_id\".\n- upcall_trace: Wraps a list of oftrace_node (the same current\n \"ofproto/trace\" uses)\n\nUpcall trace_id and trace grouping:\nTraces are grouped together based on their trace_id. When an upcall\ncomes in with recirc_id = 0 (and the filter is matched) a new trace_id is\nallocated for it. This trace_id is then preserved in the frozen state so\nwhen if it's recirculated, the next upcall will inherit the trace_id\nfrom the original.\n\nOwnership model:\nupcall_trace objects are reference-counted. The block it's linked to\nholds a reference and the upcall that adds oftrace_nodes to it holds\nanother. If the block gets evicted while the trace is being used by an\nupcall, all the information will be dropped when the upcall ends.\n\nNew unixctl_conn commands allow listing, printing and flushing trace\ngroups.\n\nSigned-off-by: Adrian Moreno \n---\n ofproto/ofproto-dpif-rid.h | 1 +\n ofproto/ofproto-dpif-trace.c | 4 +-\n ofproto/ofproto-dpif-trace.h | 4 +-\n ofproto/ofproto-dpif-upcall-trace.c | 352 ++++++++++++++++++++++++++++\n ofproto/ofproto-dpif-upcall-trace.h | 23 ++\n ofproto/ofproto-dpif-upcall.c | 122 ++++++++++\n ofproto/ofproto-dpif-xlate.c | 5 +-\n ofproto/ofproto-dpif-xlate.h | 4 +\n 8 files changed, 511 insertions(+), 4 deletions(-)","diff":"diff --git a/ofproto/ofproto-dpif-rid.h b/ofproto/ofproto-dpif-rid.h\nindex 21f32b935..220d59dac 100644\n--- a/ofproto/ofproto-dpif-rid.h\n+++ b/ofproto/ofproto-dpif-rid.h\n@@ -155,6 +155,7 @@ struct frozen_state {\n bool conntracked; /* Conntrack occurred prior to freeze. */\n bool was_mpls; /* MPLS packet */\n struct uuid xport_uuid; /* UUID of 1st port packet received on. */\n+ uint64_t trace_id; /* Non-zero if selected for tracing. */\n \n /* Actions to be translated when thawing. */\n struct ofpact *ofpacts;\ndiff --git a/ofproto/ofproto-dpif-trace.c b/ofproto/ofproto-dpif-trace.c\nindex e43d9f88c..a5710b267 100644\n--- a/ofproto/ofproto-dpif-trace.c\n+++ b/ofproto/ofproto-dpif-trace.c\n@@ -61,7 +61,7 @@ oftrace_node_type_is_terminal(enum oftrace_node_type type)\n OVS_NOT_REACHED();\n }\n \n-static void\n+void\n oftrace_node_list_destroy(struct ovs_list *nodes)\n {\n if (nodes) {\n@@ -140,7 +140,7 @@ oftrace_pop_ct_state(struct ovs_list *next_ct_states)\n OVS_NOT_REACHED();\n }\n \n-static void\n+void\n oftrace_node_print_details(struct ds *output,\n const struct ovs_list *nodes, int level)\n {\ndiff --git a/ofproto/ofproto-dpif-trace.h b/ofproto/ofproto-dpif-trace.h\nindex f023b10cd..c11236e13 100644\n--- a/ofproto/ofproto-dpif-trace.h\n+++ b/ofproto/ofproto-dpif-trace.h\n@@ -98,5 +98,7 @@ bool oftrace_add_recirc_node(struct ovs_list *recirc_queue,\n const uint16_t zone);\n \n void ofproto_append_ports_to_map(struct ofputil_port_map *, struct hmap ports);\n-\n+void oftrace_node_list_destroy(struct ovs_list *nodes);\n+void oftrace_node_print_details(struct ds *, const struct ovs_list *nodes,\n+ int level);\n #endif /* ofproto-dpif-trace.h */\ndiff --git a/ofproto/ofproto-dpif-upcall-trace.c b/ofproto/ofproto-dpif-upcall-trace.c\nindex 94f158cf9..e1b7382a6 100644\n--- a/ofproto/ofproto-dpif-upcall-trace.c\n+++ b/ofproto/ofproto-dpif-upcall-trace.c\n@@ -2,14 +2,182 @@\n \n #include \"ofproto-dpif-upcall-trace.h\"\n \n+#include \"ofproto/ofproto-dpif-xlate.h\"\n+#include \"ofproto/ofproto-dpif-trace.h\"\n #include \n #include \n #include \n #include \n #include \"ofproto-dpif.h\"\n \n+\n VLOG_DEFINE_THIS_MODULE(upcall_trace);\n \n+struct upcall_trace_block;\n+struct upcall_trace;\n+\n+static bool upcall_tracing_matches(const struct upcall_tracing *,\n+ const struct flow *,\n+ const ofp_port_t *);\n+\n+/* upcall_trace - A single xlate result (one recirculation pass).\n+ *\n+ * An upcall_trace can be listed inside an \"upcall_trace_block\" and/or\n+ * referenced by an in-flight upcall. \"refcount\" is used to track these two\n+ * references.\n+ *\n+ * \"list_node\" should only be accessed through the associated\n+ * \"upcall_trace_block\" as it will get invalidated if removed from the block\n+ * list. \"upcall\"s can take pointer to the internal \"xtrace_node\" and add nodes\n+ * to the list.\n+ */\n+struct upcall_trace {\n+ struct ovs_list list_node; /* In upcall_upcall_trace_block->traces. */\n+ struct ovs_refcount refcount;\n+\n+ long long int timestamp_msec;\n+ uint32_t recirc_id;\n+ uint64_t trace_id;\n+ struct flow initial_flow; /* Flow at start of this xlate. */\n+\n+ /* The oftrace_node tree - moved from upcall_trace. */\n+ struct ovs_list xtrace_nodes; /* List of struct xtrace_node. */\n+};\n+\n+static struct upcall_trace *\n+upcall_trace_create(uint32_t recirc_id,\n+ uint64_t trace_id,\n+ const struct flow *flow)\n+{\n+ struct upcall_trace *trace = xzalloc(sizeof *trace);\n+ trace->recirc_id = recirc_id;\n+ trace->trace_id = trace_id;\n+ trace->timestamp_msec = time_wall_msec();\n+ trace->initial_flow = *flow;\n+ ovs_list_init(&trace->xtrace_nodes);\n+ ovs_refcount_init(&trace->refcount);\n+ return trace;\n+}\n+\n+static void\n+upcall_trace_destroy(struct upcall_trace *trace)\n+{\n+ if (trace) {\n+ oftrace_node_list_destroy(&trace->xtrace_nodes);\n+ free(trace);\n+ }\n+}\n+\n+static struct upcall_trace *\n+upcall_trace_ref(const struct upcall_trace *trace_)\n+{\n+ struct upcall_trace *trace = CONST_CAST(struct upcall_trace *, trace_);\n+ if (trace) {\n+ ovs_refcount_ref(&trace->refcount);\n+ }\n+ return trace;\n+}\n+\n+void\n+upcall_trace_unref(struct upcall_trace *trace)\n+{\n+ if (trace && ovs_refcount_unref(&trace->refcount) == 1) {\n+ upcall_trace_destroy(trace);\n+ }\n+}\n+\n+uint64_t\n+upcall_trace_get_trace_id(const struct upcall_trace *utrace)\n+{\n+ return utrace ? utrace->trace_id : 0;\n+}\n+\n+struct ovs_list *\n+upcall_trace_xlate_start(struct upcall_trace *utrace,\n+ struct xlate_in *xin)\n+{\n+ if (OVS_UNLIKELY(utrace)) {\n+ /* Copy initial flow - it may change during xlate. */\n+ utrace->initial_flow = xin->flow;\n+ utrace->recirc_id = xin->flow.recirc_id;\n+ if (!ovs_list_is_empty(&utrace->xtrace_nodes)) {\n+ VLOG_ERR(\"Started new xlate tracing without consuming previous\");\n+ oftrace_node_list_destroy(&utrace->xtrace_nodes);\n+ ovs_list_init(&utrace->xtrace_nodes);\n+ }\n+ return &utrace->xtrace_nodes;\n+ }\n+ return NULL;\n+}\n+\n+/* upcall_trace_block - A group of upcall_traces with the same trace_id. */\n+struct upcall_trace_block {\n+ struct ovs_list list_node; /* In upcall_tracing->blocks. */\n+ uint64_t trace_id;\n+ struct ovs_list traces; /* List of struct upcall_trace. */\n+};\n+\n+static struct upcall_trace_block *\n+upcall_trace_block_create(uint64_t trace_id)\n+{\n+ struct upcall_trace_block *block = xzalloc(sizeof *block);\n+ block->trace_id = trace_id;\n+ ovs_list_init(&block->traces);\n+ return block;\n+}\n+\n+static void\n+upcall_trace_block_destroy(struct upcall_trace_block *block)\n+{\n+ if (block) {\n+ struct upcall_trace *trace;\n+ LIST_FOR_EACH_POP (trace, list_node, &block->traces) {\n+ upcall_trace_unref(trace);\n+ }\n+ free(block);\n+ }\n+}\n+\n+static void\n+upcall_trace_block_format_short(const struct upcall_trace_block *block,\n+ struct ds *output)\n+{\n+ struct upcall_trace *trace;\n+ ds_put_format(output, \"trace_id=0x%\"PRIx64, block->trace_id);\n+ if (!ovs_list_is_empty(&block->traces)) {\n+ trace = CONTAINER_OF(ovs_list_front(&block->traces),\n+ struct upcall_trace, list_node);\n+\n+ ds_put_strftime_msec(output, \" ts=%H:%M:%S.###\",\n+ trace->timestamp_msec, false);\n+\n+ ds_put_format(output, \" nodes=%\"PRIuSIZE\" flow=\",\n+ ovs_list_size(&block->traces));\n+ flow_format(output, &trace->initial_flow, NULL);\n+ }\n+}\n+\n+static void\n+upcall_trace_block_format(const struct upcall_trace_block *block,\n+ struct ds *output)\n+{\n+ struct upcall_trace *trace;\n+ ds_put_format(output, \"=== Trace 0x%\"PRIx64\" ===\\n\", block->trace_id);\n+ LIST_FOR_EACH (trace, list_node, &block->traces) {\n+ ds_put_format(output, \"--- recirc_id=0x%\"PRIx32\" [\",\n+ trace->recirc_id);\n+ ds_put_strftime_msec(output, \"%H:%M:%S.###\",\n+ trace->timestamp_msec, false);\n+ ds_put_cstr(output, \"] ---\\n\");\n+ ds_put_cstr(output, \"\\nInitial flow: \");\n+ flow_format(output, &trace->initial_flow, NULL);\n+ ds_put_cstr(output, \"\\n\");\n+\n+ /* Format the xtrace_traces. */\n+ oftrace_node_print_details(output, &trace->xtrace_nodes, 0);\n+ ds_put_char(output, '\\n');\n+ }\n+}\n char * OVS_WARN_UNUSED_RESULT\n upcall_tracing_create(int argc, const char *argv[],\n struct ofproto_dpif **ofprotop,\n@@ -69,6 +237,7 @@ void\n upcall_tracing_destroy(struct upcall_tracing *tracing)\n {\n if (tracing) {\n+ upcall_tracing_flush_all(tracing);\n ovs_mutex_destroy(&tracing->mutex);\n free(tracing);\n }\n@@ -88,3 +257,186 @@ upcall_tracing_format(const struct upcall_tracing *tracing, struct ds *output)\n ds_put_cstr(output, \"disabled\");\n }\n }\n+\n+static bool\n+upcall_tracing_matches(const struct upcall_tracing *tracing,\n+ const struct flow *flowp,\n+ const ofp_port_t *ofp_in_port)\n+{\n+ bool matches = false;\n+ struct minimatch minimatch;\n+ struct flow flow;\n+ if (ofp_in_port) {\n+ flow = *flowp;\n+ flow.in_port.ofp_port = *ofp_in_port;\n+ flowp = &flow;\n+ }\n+ minimatch_init(&minimatch, &tracing->filter);\n+ if (minimatch_matches_flow(&minimatch, flowp)) {\n+ matches = true;\n+ }\n+ minimatch_destroy(&minimatch);\n+ return matches;\n+}\n+\n+static struct upcall_trace_block *\n+upcall_tracing_find_block__(struct upcall_tracing *tracing,\n+ uint64_t trace_id)\n+OVS_REQUIRES(tracing->mutex)\n+{\n+ struct upcall_trace_block *block;\n+ LIST_FOR_EACH (block, list_node, &tracing->blocks) {\n+ if (block->trace_id == trace_id) {\n+ return block;\n+ }\n+ }\n+ return NULL;\n+}\n+\n+static void\n+upcall_tracing_evict_if_needed(struct upcall_tracing *tracing)\n+OVS_REQUIRES(tracing->mutex)\n+{\n+ struct upcall_trace_block *block;\n+ while (tracing->n_blocks >= tracing->max_blocks\n+ && !ovs_list_is_empty(&tracing->blocks)) {\n+ block = CONTAINER_OF(ovs_list_pop_front(&tracing->blocks),\n+ struct upcall_trace_block, list_node);\n+ upcall_trace_block_destroy(block);\n+ VLOG_DBG(\"Deleted oldest trace block\");\n+ tracing->n_blocks--;\n+ }\n+}\n+\n+static uint64_t\n+upcall_tracing_id_generate(struct upcall_tracing *tracing)\n+OVS_REQUIRES(tracing->mutex)\n+{\n+ uint64_t id;\n+ if (tracing->last_trace_id == UINT64_MAX) {\n+ tracing->last_trace_id = 0;\n+ }\n+ id = ++tracing->last_trace_id;\n+ return id;\n+}\n+\n+/* Create a new trace allocating a new block and trace_id for it.*/\n+struct upcall_trace *\n+upcall_tracing_trace_from_flow(struct upcall_tracing *tracing,\n+ const struct flow *flow,\n+ const ofp_port_t *ofp_in_port)\n+{\n+ struct upcall_trace *trace = NULL;\n+ struct upcall_trace_block *block;\n+ uint64_t trace_id;\n+\n+ ovs_mutex_lock(&tracing->mutex);\n+\n+ if (!upcall_tracing_matches(tracing, flow, ofp_in_port)) {\n+ goto out;\n+ }\n+\n+ trace_id = upcall_tracing_id_generate(tracing);\n+ trace = upcall_trace_create(0, trace_id, flow);\n+\n+ upcall_tracing_evict_if_needed(tracing);\n+ block = upcall_trace_block_create(trace_id);\n+ ovs_list_push_back(&tracing->blocks, &block->list_node);\n+ ovs_list_push_back(&block->traces, &trace->list_node);\n+ tracing->n_blocks++;\n+\n+out:\n+ ovs_mutex_unlock(&tracing->mutex);\n+ return upcall_trace_ref(trace);\n+}\n+\n+/* Create a new trace if a block with 'trace_id' is already present.*/\n+struct upcall_trace *\n+upcall_tracing_append_to_id(struct upcall_tracing *tracing,\n+ uint64_t trace_id,\n+ uint32_t recirc_id,\n+ const struct flow *flow)\n+{\n+ struct upcall_trace *trace = NULL;\n+ struct upcall_trace_block *block;\n+\n+ ovs_mutex_lock(&tracing->mutex);\n+ block = upcall_tracing_find_block__(tracing, trace_id);\n+ if (!block) {\n+ goto out;\n+ }\n+\n+ trace = upcall_trace_create(recirc_id, trace_id, flow);\n+ ovs_list_push_back(&block->traces, &trace->list_node);\n+out:\n+ ovs_mutex_unlock(&tracing->mutex);\n+ return upcall_trace_ref(trace);\n+}\n+\n+void\n+upcall_tracing_format_list(struct upcall_tracing *tracing, struct ds *output)\n+{\n+ ovs_mutex_lock(&tracing->mutex);\n+\n+ if (ovs_list_is_empty(&tracing->blocks)) {\n+ ds_put_cstr(output, \"No traces captured.\\n\");\n+ } else {\n+ struct upcall_trace_block *block;\n+ LIST_FOR_EACH (block, list_node, &tracing->blocks) {\n+ upcall_trace_block_format_short(block, output);\n+ if (&block->list_node != ovs_list_back(&tracing->blocks)) {\n+ ds_put_char(output, '\\n');\n+ }\n+ }\n+ }\n+\n+ ovs_mutex_unlock(&tracing->mutex);\n+}\n+\n+/* Thread-safe version that finds and formats a block by trace_id.\n+ * Returns true if block was found and formatted. */\n+void\n+upcall_tracing_format_id(struct upcall_tracing *tracing,\n+ uint64_t trace_id, struct ds *output)\n+{\n+ struct upcall_trace_block *block;\n+\n+ ovs_mutex_lock(&tracing->mutex);\n+\n+ block = upcall_tracing_find_block__(tracing, trace_id);\n+ if (block) {\n+ upcall_trace_block_format(block, output);\n+ }\n+\n+ ovs_mutex_unlock(&tracing->mutex);\n+}\n+\n+void\n+upcall_tracing_flush(struct upcall_tracing *tracing, uint64_t trace_id)\n+{\n+ struct upcall_trace_block *block;\n+\n+ ovs_mutex_lock(&tracing->mutex);\n+ block = upcall_tracing_find_block__(tracing, trace_id);\n+ if (block) {\n+ ovs_list_remove(&block->list_node);\n+ upcall_trace_block_destroy(block);\n+ tracing->n_blocks--;\n+ }\n+\n+ ovs_mutex_unlock(&tracing->mutex);\n+}\n+\n+void\n+upcall_tracing_flush_all(struct upcall_tracing *tracing)\n+{\n+ struct upcall_trace_block *block;\n+\n+ ovs_mutex_lock(&tracing->mutex);\n+ LIST_FOR_EACH_POP (block, list_node, &tracing->blocks) {\n+ upcall_trace_block_destroy(block);\n+ }\n+ tracing->n_blocks = 0;\n+ ovs_mutex_unlock(&tracing->mutex);\n+}\n+\ndiff --git a/ofproto/ofproto-dpif-upcall-trace.h b/ofproto/ofproto-dpif-upcall-trace.h\nindex 150d48d29..7e4d35399 100644\n--- a/ofproto/ofproto-dpif-upcall-trace.h\n+++ b/ofproto/ofproto-dpif-upcall-trace.h\n@@ -25,9 +25,32 @@ struct upcall_tracing {\n size_t max_blocks;\n uint64_t last_trace_id;\n };\n+struct upcall_trace;\n+\n char * upcall_tracing_create(int argc, const char *argv[],\n struct ofproto_dpif **ofprotop,\n struct upcall_tracing **tracingp);\n void upcall_tracing_destroy(struct upcall_tracing *);\n void upcall_tracing_format(const struct upcall_tracing *, struct ds *);\n+void upcall_tracing_format_list(struct upcall_tracing *, struct ds *);\n+void upcall_tracing_format_id(struct upcall_tracing *, uint64_t trace_id,\n+ struct ds *);\n+\n+void upcall_tracing_flush(struct upcall_tracing *, uint64_t trace_id);\n+void upcall_tracing_flush_all(struct upcall_tracing *);\n+\n+struct upcall_trace *\n+upcall_tracing_trace_from_flow(struct upcall_tracing *tracing,\n+ const struct flow *flow,\n+ const ofp_port_t *ofp_in_port);\n+struct upcall_trace *\n+upcall_tracing_append_to_id(struct upcall_tracing *tracing,\n+ uint64_t trace_id,\n+ uint32_t recirc_id,\n+ const struct flow *flow);\n+struct ovs_list *upcall_trace_xlate_start(struct upcall_trace *,\n+ struct xlate_in *);\n+uint64_t upcall_trace_get_trace_id(const struct upcall_trace *);\n+void upcall_trace_unref(struct upcall_trace *);\n+\n #endif /*OFPROTO_DPIF_UPCALL_TRACE_H */\ndiff --git a/ofproto/ofproto-dpif-upcall.c b/ofproto/ofproto-dpif-upcall.c\nindex 20fa302a5..6fc3d2c1f 100644\n--- a/ofproto/ofproto-dpif-upcall.c\n+++ b/ofproto/ofproto-dpif-upcall.c\n@@ -263,6 +263,8 @@ struct upcall {\n struct user_action_cookie cookie;\n \n uint64_t odp_actions_stub[1024 / 8]; /* Stub for odp_actions. */\n+\n+ struct upcall_trace *trace; /* Upcall trace. */\n };\n \n /* Ukeys must transition through these states using transition_ukey(). */\n@@ -377,6 +379,11 @@ static void udpif_pause_revalidators(struct udpif *);\n static void udpif_resume_revalidators(struct udpif *);\n static void *udpif_upcall_handler(void *);\n static void *udpif_revalidator(void *);\n+static struct upcall_trace *udpif_trace_from_upcall(const struct upcall *);\n+\n+static struct upcall_trace\n+*udpif_trace_from_frozen_upcall(const struct upcall *,\n+ const struct frozen_state *);\n static unsigned long udpif_get_n_flows(struct udpif *);\n static void revalidate(struct revalidator *);\n static void revalidator_pause(struct revalidator *);\n@@ -411,6 +418,10 @@ static void upcall_unixctl_trace_create(struct unixctl_conn *, int argc,\n const char *argv[], void *aux);\n static void upcall_unixctl_trace_show(struct unixctl_conn *, int argc,\n const char *argv[], void *aux);\n+static void upcall_unixctl_trace_list(struct unixctl_conn *, int argc,\n+ const char *argv[], void *aux);\n+static void upcall_unixctl_trace_get(struct unixctl_conn *, int argc,\n+ const char *argv[], void *aux);\n static void upcall_unixctl_trace_delete(struct unixctl_conn *, int argc,\n const char *argv[], void *aux);\n \n@@ -496,6 +507,10 @@ udpif_init(void)\n upcall_unixctl_trace_create, NULL);\n unixctl_command_register(\"upcall/trace/show\", \"\", 0, 0,\n upcall_unixctl_trace_show, NULL);\n+ unixctl_command_register(\"upcall/trace/list\", \"\", 0, 1,\n+ upcall_unixctl_trace_list, NULL);\n+ unixctl_command_register(\"upcall/trace/get\", \"bridge trace_id\", 1, 1,\n+ upcall_unixctl_trace_get, NULL);\n unixctl_command_register(\"upcall/trace/delete\", \"\", 0, 1,\n upcall_unixctl_trace_delete, NULL);\n ovsthread_once_done(&once);\n@@ -1287,6 +1302,7 @@ upcall_receive(struct upcall *upcall, const struct dpif_backer *backer,\n \n upcall->out_tun_key = NULL;\n upcall->actions = NULL;\n+ upcall->trace = udpif_trace_from_upcall(upcall);\n \n return 0;\n }\n@@ -1310,6 +1326,14 @@ upcall_xlate(struct udpif *udpif, struct upcall *upcall,\n upcall->flow, upcall->ofp_in_port, NULL,\n stats.tcp_flags, upcall->packet, wc, odp_actions);\n \n+ if (!upcall->trace && xin.frozen_state) {\n+ upcall->trace =\n+ udpif_trace_from_frozen_upcall(upcall, xin.frozen_state);\n+ }\n+\n+ /* Set trace_id in xlate_in so it propagates through freezes. */\n+ xin.trace_id = upcall_trace_get_trace_id(upcall->trace);\n+\n if (upcall->type == MISS_UPCALL) {\n xin.resubmit_stats = &stats;\n \n@@ -1332,6 +1356,8 @@ upcall_xlate(struct udpif *udpif, struct upcall *upcall,\n \n upcall->reval_seq = seq_read(udpif->reval_seq);\n \n+ xin.trace = upcall_trace_xlate_start(upcall->trace, &xin);\n+\n xerr = xlate_actions(&xin, &upcall->xout);\n \n /* Translate again and log the ofproto trace for\n@@ -1401,6 +1427,9 @@ upcall_uninit(struct upcall *upcall)\n /* The reference was transferred to the ukey if one was created. */\n recirc_id_node_unref(upcall->recirc);\n }\n+ if (upcall->trace) {\n+ upcall_trace_unref(upcall->trace);\n+ }\n }\n }\n \n@@ -3477,6 +3506,66 @@ static void upcall_unixctl_trace_show(struct unixctl_conn *conn,\n ds_destroy(&ds);\n }\n \n+static void upcall_unixctl_trace_list(struct unixctl_conn *conn,\n+ int argc OVS_UNUSED,\n+ const char *argv[] OVS_UNUSED,\n+ void *aux OVS_UNUSED)\n+{\n+ struct ds ds = DS_EMPTY_INITIALIZER;\n+ const struct shash_node **backers;\n+ int i;\n+\n+ backers = shash_sort(&all_dpif_backers);\n+ for (i = 0; i < shash_count(&all_dpif_backers); i++) {\n+ struct dpif_backer *backer = backers[i]->data;\n+ struct upcall_tracing *tracing =\n+ ovsrcu_get(struct upcall_tracing *, &backer->udpif->tracing);\n+ if (tracing) {\n+ ds_put_format(&ds, \"%s:\\n\", dpif_name(backer->dpif));\n+ upcall_tracing_format_list(tracing, &ds);\n+ }\n+ }\n+ free(backers);\n+ unixctl_command_reply(conn, ds_cstr(&ds));\n+ ds_destroy(&ds);\n+}\n+\n+static void upcall_unixctl_trace_get(struct unixctl_conn *conn,\n+ int argc OVS_UNUSED,\n+ const char *argv[] OVS_UNUSED,\n+ void *aux OVS_UNUSED)\n+{\n+ struct ds ds = DS_EMPTY_INITIALIZER;\n+ const struct shash_node **backers;\n+ uint64_t trace_id;\n+ int i;\n+\n+ if (!strncmp(argv[1], \"0x\", 2) || !strncmp(argv[1], \"0X\", 2)) {\n+ if (sscanf(argv[1], \"%\"SCNx64, &trace_id) != 1) {\n+ unixctl_command_reply_error(conn, \"invalid trace_id\");\n+ return;\n+ }\n+ } else {\n+ if (sscanf(argv[1], \"%\"SCNu64, &trace_id) != 1) {\n+ unixctl_command_reply_error(conn, \"invalid trace_id\");\n+ return;\n+ }\n+ }\n+\n+ backers = shash_sort(&all_dpif_backers);\n+ for (i = 0; i < shash_count(&all_dpif_backers); i++) {\n+ struct dpif_backer *backer = backers[i]->data;\n+ struct upcall_tracing *tracing =\n+ ovsrcu_get(struct upcall_tracing *, &backer->udpif->tracing);\n+ if (tracing) {\n+ upcall_tracing_format_id(tracing, trace_id, &ds);\n+ }\n+ }\n+\n+ unixctl_command_reply(conn, ds_cstr(&ds));\n+ ds_destroy(&ds);\n+}\n+\n static void upcall_unixctl_trace_delete(struct unixctl_conn *conn,\n int argc OVS_UNUSED,\n const char *argv[] OVS_UNUSED,\n@@ -3839,3 +3928,36 @@ udpif_flow_unprogram(struct udpif *udpif, struct udpif_key *ukey,\n \n return opsp->error;\n }\n+\n+static struct upcall_trace *\n+udpif_trace_from_upcall(const struct upcall *upcall)\n+{\n+ struct upcall_trace *trace = NULL;\n+ struct upcall_tracing *tracing = ovsrcu_get(struct upcall_tracing *,\n+ &upcall->ofproto->backer->udpif->tracing);\n+\n+ /* Only check the filter for non-recirculated flows. For recirculated\n+ * flows, the trace_id will be retrieved from frozen_state later in\n+ * udpif_tracer_from_frozen_upcall(). */\n+ if (OVS_UNLIKELY(tracing) && upcall->flow->recirc_id == 0) {\n+ trace = upcall_tracing_trace_from_flow(tracing, upcall->flow,\n+ &upcall->ofp_in_port);\n+ }\n+ return trace;\n+}\n+\n+static struct upcall_trace *\n+udpif_trace_from_frozen_upcall(const struct upcall *upcall,\n+ const struct frozen_state *state)\n+{\n+ struct upcall_trace *trace = NULL;\n+ struct upcall_tracing *tracing = ovsrcu_get(struct upcall_tracing *,\n+ &upcall->ofproto->backer->udpif->tracing);\n+\n+ if (OVS_UNLIKELY(tracing) && state && state->trace_id) {\n+ trace = upcall_tracing_append_to_id(tracing, state->trace_id,\n+ upcall->flow->recirc_id,\n+ upcall->flow);\n+ }\n+ return trace;\n+}\ndiff --git a/ofproto/ofproto-dpif-xlate.c b/ofproto/ofproto-dpif-xlate.c\nindex de82e2903..8c23d6035 100644\n--- a/ofproto/ofproto-dpif-xlate.c\n+++ b/ofproto/ofproto-dpif-xlate.c\n@@ -5265,6 +5265,7 @@ xlate_controller_action(struct xlate_ctx *ctx, int len,\n .mirrors = ctx->mirrors,\n .conntracked = ctx->conntracked,\n .was_mpls = ctx->was_mpls,\n+ .trace_id = ctx->xin->trace_id,\n .ofpacts = NULL,\n .ofpacts_len = 0,\n .action_set = NULL,\n@@ -5310,6 +5311,7 @@ finish_freezing__(struct xlate_ctx *ctx, uint8_t table)\n .conntracked = ctx->conntracked,\n .was_mpls = ctx->was_mpls,\n .xport_uuid = ctx->xin->xport_uuid,\n+ .trace_id = ctx->xin->trace_id,\n .ofpacts = ctx->frozen_actions.data,\n .ofpacts_len = ctx->frozen_actions.size,\n .action_set = ctx->action_set.data,\n@@ -5382,7 +5384,7 @@ compose_recirculate_and_fork(struct xlate_ctx *ctx, uint8_t table,\n ctx->freezing = true;\n recirc_id = finish_freezing__(ctx, table);\n \n- if (OVS_UNLIKELY(ctx->xin->trace) && recirc_id) {\n+ if (OVS_UNLIKELY(ctx->xin->trace) && recirc_id && ctx->xin->recirc_queue) {\n if (oftrace_add_recirc_node(ctx->xin->recirc_queue,\n OFT_RECIRC_CONNTRACK, &ctx->xin->flow,\n ctx->ct_nat_action, ctx->xin->packet,\n@@ -7918,6 +7920,7 @@ xlate_in_init(struct xlate_in *xin, struct ofproto_dpif *ofproto,\n xin->in_packet_out = false;\n xin->recirc_queue = NULL;\n xin->xport_uuid = UUID_ZERO;\n+ xin->trace_id = 0;\n \n /* Do recirc lookup. */\n xin->frozen_state = NULL;\ndiff --git a/ofproto/ofproto-dpif-xlate.h b/ofproto/ofproto-dpif-xlate.h\nindex d973a634a..90a66e76a 100644\n--- a/ofproto/ofproto-dpif-xlate.h\n+++ b/ofproto/ofproto-dpif-xlate.h\n@@ -174,6 +174,10 @@ struct xlate_in {\n /* If true, port names are displayed instead of port numbers in\n * tracing translation. */\n bool names;\n+\n+ /* If non-zero, this xlate is being traced and the trace_id should be\n+ * propagated through recirculations. */\n+ uint64_t trace_id;\n };\n \n void xlate_ofproto_set(struct ofproto_dpif *, const char *name, struct dpif *,\n","prefixes":["ovs-dev","RFC","v1","2/2"]}