From patchwork Mon Sep 6 08:53:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eelco Chaudron X-Patchwork-Id: 1524857 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=MneEbmNI; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4H32J32Lrwz9sRN for ; Mon, 6 Sep 2021 18:53:55 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id C28B481A47; Mon, 6 Sep 2021 08:53:52 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YHUEUqAqypZC; Mon, 6 Sep 2021 08:53:51 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id E00D181776; Mon, 6 Sep 2021 08:53:50 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id B71CBC001C; Mon, 6 Sep 2021 08:53:50 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2D7DEC001B for ; Mon, 6 Sep 2021 08:53:50 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 724EA81776 for ; Mon, 6 Sep 2021 08:53:49 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1Fbp5st48IBI for ; Mon, 6 Sep 2021 08:53:48 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id B8C6B81024 for ; Mon, 6 Sep 2021 08:53:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630918427; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=G2imxAPX+KgtM9EUae1D2tLHBMMrkvYzUTOznjssh/c=; b=MneEbmNITsC8xABSupvpaV/nBHHOYc2Ks+UtB298hJqZpCR4wFOK1FG5LryiKyN1vX0Lxx gk8inUwtYOBdUm3SwGF0Bc64RfehvPur7nx1Uu0I8Q+hzUlDWYpwBCcU0fr6gSIK94aoTy W4CdjAgMZX0Ov48zUltCQkIVmStn0qM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-412-sLwALNrUOSGSFZu3By6Mvw-1; Mon, 06 Sep 2021 04:53:46 -0400 X-MC-Unique: sLwALNrUOSGSFZu3By6Mvw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C376B84A5EA; Mon, 6 Sep 2021 08:53:45 +0000 (UTC) Received: from ebuild.redhat.com (unknown [10.39.194.213]) by smtp.corp.redhat.com (Postfix) with ESMTP id 09D075D9CA; Mon, 6 Sep 2021 08:53:44 +0000 (UTC) From: Eelco Chaudron To: dev@openvswitch.org Date: Mon, 6 Sep 2021 10:53:42 +0200 Message-Id: <163091841785.96839.4880012333775292574.stgit@ebuild> In-Reply-To: <163091839517.96839.8006181203263313854.stgit@ebuild> References: <163091839517.96839.8006181203263313854.stgit@ebuild> User-Agent: StGit/0.21 MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=echaudro@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: fbl@sysclose.org Subject: [ovs-dev] [PATCH v6 1/2] dpctl: dpif: add kernel datapath cache hit output X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This patch adds cache usage statistics to the output: $ ovs-dpctl show system@ovs-system: lookups: hit:24 missed:71 lost:0 flows: 0 masks: hit:334 total:0 hit/pkt:3.52 cache: hit:4 hit rate:4.21% port 0: ovs-system (internal) port 1: genev_sys_6081 (geneve: packet_type=ptap) port 2: br-int (internal) port 3: br-ex (internal) port 4: eth2 port 5: sw1p1 (internal) port 6: sw0p4 (internal) Signed-off-by: Eelco Chaudron Acked-by: Flavio Leitner Acked-by: Paolo Valerio v2: - Changed cache naming to use a hyphen instead of spaces - Some error message grammar changes - Update dpctl man page - Add self tests to for the new set/get commands v3: - Rebase on the latest master branch v4: - Fixed commit message and tab/spaces in datapath include v5: - No updates v6: - Add ACKs --- datapath/linux/compat/include/linux/openvswitch.h | 2 +- lib/dpctl.c | 9 +++++++++ lib/dpif-netdev.c | 1 + lib/dpif-netlink.c | 9 +++++++++ lib/dpif.h | 2 ++ 5 files changed, 22 insertions(+), 1 deletion(-) diff --git a/datapath/linux/compat/include/linux/openvswitch.h b/datapath/linux/compat/include/linux/openvswitch.h index f0595eeba..a3f5fb919 100644 --- a/datapath/linux/compat/include/linux/openvswitch.h +++ b/datapath/linux/compat/include/linux/openvswitch.h @@ -126,8 +126,8 @@ struct ovs_dp_megaflow_stats { __u64 n_mask_hit; /* Number of masks used for flow lookups. */ __u32 n_masks; /* Number of masks for the datapath. */ __u32 pad0; /* Pad for future expension. */ + __u64 n_cache_hit; /* Number of cache matches for flow lookups. */ __u64 pad1; /* Pad for future expension. */ - __u64 pad2; /* Pad for future expension. */ }; struct ovs_vport_stats { diff --git a/lib/dpctl.c b/lib/dpctl.c index ef8ae7402..acc677974 100644 --- a/lib/dpctl.c +++ b/lib/dpctl.c @@ -611,6 +611,15 @@ show_dpif(struct dpif *dpif, struct dpctl_params *dpctl_p) dpctl_print(dpctl_p, " masks: hit:%"PRIu64" total:%"PRIu32 " hit/pkt:%.2f\n", stats.n_mask_hit, stats.n_masks, avg); + + if (stats.n_cache_hit != UINT64_MAX) { + double avg_hits = n_pkts ? + (double) stats.n_cache_hit / n_pkts * 100 : 0.0; + + dpctl_print(dpctl_p, + " cache: hit:%"PRIu64" hit rate:%.2f%%\n", + stats.n_cache_hit, avg_hits); + } } } diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index b3e57bb95..0d0623013 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -1963,6 +1963,7 @@ dpif_netdev_get_stats(const struct dpif *dpif, struct dpif_dp_stats *stats) } stats->n_masks = UINT32_MAX; stats->n_mask_hit = UINT64_MAX; + stats->n_cache_hit = UINT64_MAX; return 0; } diff --git a/lib/dpif-netlink.c b/lib/dpif-netlink.c index 34fc04237..70751e634 100644 --- a/lib/dpif-netlink.c +++ b/lib/dpif-netlink.c @@ -749,9 +749,18 @@ dpif_netlink_get_stats(const struct dpif *dpif_, struct dpif_dp_stats *stats) stats->n_masks = dp.megaflow_stats->n_masks; stats->n_mask_hit = get_32aligned_u64( &dp.megaflow_stats->n_mask_hit); + stats->n_cache_hit = get_32aligned_u64( + &dp.megaflow_stats->n_cache_hit); + + if (!stats->n_cache_hit) { + /* Old kernels don't use this field and always + * report zero instead. Disable this stat. */ + stats->n_cache_hit = UINT64_MAX; + } } else { stats->n_masks = UINT32_MAX; stats->n_mask_hit = UINT64_MAX; + stats->n_cache_hit = UINT64_MAX; } ofpbuf_delete(buf); } diff --git a/lib/dpif.h b/lib/dpif.h index 7c322d20e..b32ae5fc7 100644 --- a/lib/dpif.h +++ b/lib/dpif.h @@ -429,6 +429,8 @@ struct dpif_dp_stats { uint64_t n_missed; /* Number of flow table misses. */ uint64_t n_lost; /* Number of misses not sent to userspace. */ uint64_t n_flows; /* Number of flows present. */ + uint64_t n_cache_hit; /* Number of mega flow mask cache hits for + flow table matches. */ uint64_t n_mask_hit; /* Number of mega flow masks visited for flow table matches. */ uint32_t n_masks; /* Number of mega flow masks. */ From patchwork Mon Sep 6 08:53:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eelco Chaudron X-Patchwork-Id: 1524858 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=a0I7i6nA; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4H32JY4hb9z9sRN for ; Mon, 6 Sep 2021 18:54:21 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id BF7E74041A; Mon, 6 Sep 2021 08:54:18 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8zjEhPFE7UuU; Mon, 6 Sep 2021 08:54:17 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 0ACE2403F0; Mon, 6 Sep 2021 08:54:16 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id CE49AC001C; Mon, 6 Sep 2021 08:54:15 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 6E891C001B for ; Mon, 6 Sep 2021 08:54:14 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id E4E5640445 for ; Mon, 6 Sep 2021 08:54:08 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jVZj_dqGctHS for ; Mon, 6 Sep 2021 08:54:07 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id 163324043C for ; Mon, 6 Sep 2021 08:54:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630918446; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fCdaO5IPmFdFSj4SZrJIhiqMIZCGewKi6ghQSuAYzAI=; b=a0I7i6nAGuNjZn2qIxY/pIoAs8nqugtlZzwRZI15j/JQBOvsUEckcoNQ9OMUIwzfoEQoU8 zYYvTaeY6w/xecOzwAJSGaEoemCZPkiuV3MM9DrCeePfZgUcpIqsFC1o0m3Xq71cu3srO+ Y/6Da9hYFtAy4zYuy9TQRl3wcCQM97E= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-244-yp0odMNaNv66wFPlUzRf1A-1; Mon, 06 Sep 2021 04:54:02 -0400 X-MC-Unique: yp0odMNaNv66wFPlUzRf1A-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 6A15710866B9; Mon, 6 Sep 2021 08:54:01 +0000 (UTC) Received: from ebuild.redhat.com (unknown [10.39.194.213]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8198B10016F4; Mon, 6 Sep 2021 08:54:00 +0000 (UTC) From: Eelco Chaudron To: dev@openvswitch.org Date: Mon, 6 Sep 2021 10:53:58 +0200 Message-Id: <163091843101.96839.16631709571000877173.stgit@ebuild> In-Reply-To: <163091839517.96839.8006181203263313854.stgit@ebuild> References: <163091839517.96839.8006181203263313854.stgit@ebuild> User-Agent: StGit/0.21 MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=echaudro@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: fbl@sysclose.org Subject: [ovs-dev] [PATCH v6 2/2] dpctl: dpif: allow viewing and configuring dp cache sizes X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This patch adds a general way of viewing/configuring datapath cache sizes. With an implementation for the netlink interface. The ovs-dpctl/ovs-appctl show commands will display the current cache sizes configured: ovs-dpctl show system@ovs-system: lookups: hit:25 missed:63 lost:0 flows: 0 masks: hit:282 total:0 hit/pkt:3.20 cache: hit:4 hit rate:4.54% caches: masks-cache: size: 256 port 0: ovs-system (internal) port 1: br-int (internal) port 2: genev_sys_6081 (geneve: packet_type=ptap) port 3: br-ex (internal) port 4: eth2 port 5: sw0p1 (internal) port 6: sw0p3 (internal) A specific cache can be configured as follows: ovs-appctl dpctl/cache-set-size DP CACHE SIZE ovs-dpctl cache-set-size DP CACHE SIZE For example to disable the cache do: $ ovs-dpctl cache-set-size system@ovs-system masks-cache 0 Setting cache size successful, new size 0. Signed-off-by: Eelco Chaudron Acked-by: Paolo Valerio Acked-by: Flavio Leitner v2: - Changed precision to 2 digits - Handle missing kernel feature at netlink level v3: - Rebase on the latest master branch v4: - Fixed commit message - Fix issue with resetting user_features v5: - Include the actual resetting user_features fix v6: - Rebase on the latest master branch - Add ACK's --- datapath/linux/compat/include/linux/openvswitch.h | 2 lib/dpctl.c | 120 +++++++++++++++++++++ lib/dpctl.man | 14 ++ lib/dpif-netdev.c | 4 + lib/dpif-netlink.c | 117 ++++++++++++++++++++ lib/dpif-provider.h | 20 ++++ lib/dpif.c | 32 ++++++ lib/dpif.h | 7 + tests/system-traffic.at | 36 ++++++ utilities/ovs-dpctl.c | 4 + 10 files changed, 355 insertions(+), 1 deletion(-) diff --git a/datapath/linux/compat/include/linux/openvswitch.h b/datapath/linux/compat/include/linux/openvswitch.h index a3f5fb919..936644192 100644 --- a/datapath/linux/compat/include/linux/openvswitch.h +++ b/datapath/linux/compat/include/linux/openvswitch.h @@ -107,7 +107,7 @@ enum ovs_datapath_attr { OVS_DP_ATTR_MEGAFLOW_STATS, /* struct ovs_dp_megaflow_stats */ OVS_DP_ATTR_USER_FEATURES, /* OVS_DP_F_* */ OVS_DP_ATTR_PAD, - OVS_DP_ATTR_PAD2, + OVS_DP_ATTR_MASKS_CACHE_SIZE, OVS_DP_ATTR_PER_CPU_PIDS, /* Netlink PIDS to receive upcalls */ __OVS_DP_ATTR_MAX }; diff --git a/lib/dpctl.c b/lib/dpctl.c index acc677974..6cba8db51 100644 --- a/lib/dpctl.c +++ b/lib/dpctl.c @@ -591,6 +591,36 @@ compare_port_nos(const void *a_, const void *b_) return a < b ? -1 : a > b; } +static void +show_dpif_cache__(struct dpif *dpif, struct dpctl_params *dpctl_p) +{ + uint32_t nr_caches; + int error = dpif_cache_get_supported_levels(dpif, &nr_caches); + + if (error || nr_caches == 0) { + return; + } + + dpctl_print(dpctl_p, " caches:\n"); + for (int i = 0; i < nr_caches; i++) { + const char *name; + uint32_t size; + + if (dpif_cache_get_name(dpif, i, &name) || + dpif_cache_get_size(dpif, i, &size)) { + continue; + } + dpctl_print(dpctl_p, " %s: size: %u\n", name, size); + } +} + +static void +show_dpif_cache(struct dpif *dpif, struct dpctl_params *dpctl_p) +{ + dpctl_print(dpctl_p, "%s:\n", dpif_name(dpif)); + show_dpif_cache__(dpif, dpctl_p); +} + static void show_dpif(struct dpif *dpif, struct dpctl_params *dpctl_p) { @@ -623,6 +653,8 @@ show_dpif(struct dpif *dpif, struct dpctl_params *dpctl_p) } } + show_dpif_cache__(dpif, dpctl_p); + odp_port_t *port_nos = NULL; size_t allocated_port_nos = 0, n_port_nos = 0; DPIF_PORT_FOR_EACH (&dpif_port, &dump, dpif) { @@ -2409,6 +2441,92 @@ dpctl_ct_ipf_get_status(int argc, const char *argv[], return error; } +static int +dpctl_cache_get_size(int argc, const char *argv[], + struct dpctl_params *dpctl_p) +{ + int error; + + if (argc > 1) { + struct dpif *dpif; + + error = parsed_dpif_open(argv[1], false, &dpif); + if (!error) { + show_dpif_cache(dpif, dpctl_p); + dpif_close(dpif); + } else { + dpctl_error(dpctl_p, error, "Opening datapath %s failed", argv[1]); + } + } else { + error = dps_for_each(dpctl_p, show_dpif_cache); + } + + return error; +} + +static int +dpctl_cache_set_size(int argc, const char *argv[], + struct dpctl_params *dpctl_p) +{ + int i, error = EINVAL; + uint32_t nr_caches, size; + struct dpif *dpif; + + if (argc != 4) { + dpctl_error(dpctl_p, error, "Invalid number of arguments"); + return error; + } + + if (!ovs_scan(argv[3], "%"SCNu32, &size)) { + dpctl_error(dpctl_p, error, "size is malformed"); + return error; + } + + error = parsed_dpif_open(argv[1], false, &dpif); + if (error) { + dpctl_error(dpctl_p, error, "Opening datapath %s failed", + argv[1]); + return error; + } + + error = dpif_cache_get_supported_levels(dpif, &nr_caches); + if (error || nr_caches == 0) { + dpctl_error(dpctl_p, error, "Setting caches not supported"); + goto exit; + } + + for (i = 0; i < nr_caches; i++) { + const char *name; + + if (dpif_cache_get_name(dpif, i, &name)) { + continue; + } + if (!strcmp(argv[2], name)) { + break; + } + } + + if (i == nr_caches) { + error = EINVAL; + dpctl_error(dpctl_p, error, "Cache name \"%s\" not found on dpif", + argv[2]); + goto exit; + } + + error = dpif_cache_set_size(dpif, i, size); + if (!error) { + dpif_cache_get_size(dpif, i, &size); + dpctl_print(dpctl_p, "Setting cache size successful, new size %u\n", + size); + } else { + dpctl_error(dpctl_p, error, "Setting cache size failed"); + } + +exit: + dpif_close(dpif); + return error; +} + /* Undocumented commands for unit testing. */ static int @@ -2710,6 +2828,8 @@ static const struct dpctl_command all_commands[] = { 0, 4, dpctl_dump_conntrack, DP_RO }, { "flush-conntrack", "[dp] [zone=N] [ct-tuple]", 0, 3, dpctl_flush_conntrack, DP_RW }, + { "cache-get-size", "[dp]", 0, 1, dpctl_cache_get_size, DP_RO }, + { "cache-set-size", "dp cache ", 3, 3, dpctl_cache_set_size, DP_RW }, { "ct-stats-show", "[dp] [zone=N]", 0, 3, dpctl_ct_stats_show, DP_RO }, { "ct-bkts", "[dp] [gt=N]", 0, 2, dpctl_ct_bkts, DP_RO }, diff --git a/lib/dpctl.man b/lib/dpctl.man index 81046ef39..c1361f217 100644 --- a/lib/dpctl.man +++ b/lib/dpctl.man @@ -226,6 +226,20 @@ Fetches the flow from \fIdp\fR's flow table with unique identifier \fIufid\fR. . .IP "\*(DX\fBdel\-flows\fR [\fIdp\fR]" Deletes all flow entries from datapath \fIdp\fR's flow table. +.SS "DATAPATH FLOW CACHE COMMANDS" +The following commands are useful for debugging and configuring +the datapath flow cache settings. +. +.TP +\*(DX\fBcache\-get\-size\fR [\fIdp\fR] +Prints the current cache setting to the console. +. +.TP +\*(DX\fBcache\-set\-size\fR \fIdp\fR \fIcache\fR \fIsize\fR +Set the \fIdp\fR's specific \fIcache\fR to the given \fIsize\fR. +The cache name can be found by using the \fBcache\-get\-size\fR +command. +. .SS "CONNECTION TRACKING TABLE COMMANDS" The following commands are useful for debugging and configuring the connection tracking table in the datapath. diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 0d0623013..642204b54 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -8791,6 +8791,10 @@ const struct dpif_class dpif_netdev_class = { dpif_netdev_bond_add, dpif_netdev_bond_del, dpif_netdev_bond_stats_get, + NULL, /* cache_get_supported_levels */ + NULL, /* cache_get_name */ + NULL, /* cache_get_size */ + NULL, /* cache_set_size */ }; static void diff --git a/lib/dpif-netlink.c b/lib/dpif-netlink.c index 70751e634..18bd1125a 100644 --- a/lib/dpif-netlink.c +++ b/lib/dpif-netlink.c @@ -98,6 +98,7 @@ struct dpif_netlink_dp { const char *name; /* OVS_DP_ATTR_NAME. */ const uint32_t *upcall_pid; /* OVS_DP_ATTR_UPCALL_PID. */ uint32_t user_features; /* OVS_DP_ATTR_USER_FEATURES */ + uint32_t cache_size; /* OVS_DP_ATTR_MASKS_CACHE_SIZE */ const struct ovs_dp_stats *stats; /* OVS_DP_ATTR_STATS. */ const struct ovs_dp_megaflow_stats *megaflow_stats; /* OVS_DP_ATTR_MEGAFLOW_STATS.*/ @@ -4270,6 +4271,104 @@ probe_broken_meters(struct dpif *dpif) } return broken_meters; } + + +static int +dpif_netlink_cache_get_supported_levels(struct dpif *dpif_, uint32_t *levels) +{ + int error; + struct ofpbuf *buf; + struct dpif_netlink_dp dp; + + /* If available, in the kernel we support one level of cache. + * Unfortunately, there is no way to detect if the older kernel module has + * the cache feature. For now, we only report the cache information if the + * kernel module reports the OVS_DP_ATTR_MASKS_CACHE_SIZE attribute. */ + + *levels = 0; + error = dpif_netlink_dp_get(dpif_, &dp, &buf); + if (!error) { + + if (dp.cache_size != UINT32_MAX) { + *levels = 1; + } + ofpbuf_delete(buf); + } + + return error; +} + +static int +dpif_netlink_cache_get_name(struct dpif *dpif_ OVS_UNUSED, uint32_t level, + const char **name) +{ + if (level != 0) { + return EINVAL; + } + + *name = "masks-cache"; + return 0; +} + +static int +dpif_netlink_cache_get_size(struct dpif *dpif_, uint32_t level, uint32_t *size) +{ + int error; + struct ofpbuf *buf; + struct dpif_netlink_dp dp; + + if (level != 0) { + return EINVAL; + } + + error = dpif_netlink_dp_get(dpif_, &dp, &buf); + if (!error) { + + ofpbuf_delete(buf); + + if (dp.cache_size == UINT32_MAX) { + return EOPNOTSUPP; + } + *size = dp.cache_size; + } + return error; +} + +static int +dpif_netlink_cache_set_size(struct dpif *dpif_, uint32_t level, uint32_t size) +{ + int error; + struct ofpbuf *bufp; + struct dpif_netlink_dp request, reply; + struct dpif_netlink *dpif = dpif_netlink_cast(dpif_); + + size = ROUND_UP_POW2(size); + + if (level != 0) { + return EINVAL; + } + + dpif_netlink_dp_init(&request); + request.cmd = OVS_DP_CMD_SET; + request.name = dpif_->base_name; + request.dp_ifindex = dpif->dp_ifindex; + request.cache_size = size; + /* We need to set the dpif user_features, as the kernel module assumes the + * OVS_DP_ATTR_USER_FEATURES attribute is always present. If not, it will + * reset all the features. */ + request.user_features = dpif->user_features; + + error = dpif_netlink_dp_transact(&request, &reply, &bufp); + if (!error) { + ofpbuf_delete(bufp); + if (reply.cache_size != size) { + return EINVAL; + } + } + + return error; +} + const struct dpif_class dpif_netlink_class = { "system", @@ -4349,6 +4448,10 @@ const struct dpif_class dpif_netlink_class = { NULL, /* bond_add */ NULL, /* bond_del */ NULL, /* bond_stats_get */ + dpif_netlink_cache_get_supported_levels, + dpif_netlink_cache_get_name, + dpif_netlink_cache_get_size, + dpif_netlink_cache_set_size, }; static int @@ -4612,6 +4715,9 @@ dpif_netlink_dp_from_ofpbuf(struct dpif_netlink_dp *dp, const struct ofpbuf *buf [OVS_DP_ATTR_USER_FEATURES] = { .type = NL_A_U32, .optional = true }, + [OVS_DP_ATTR_MASKS_CACHE_SIZE] = { + .type = NL_A_U32, + .optional = true }, }; dpif_netlink_dp_init(dp); @@ -4644,6 +4750,12 @@ dpif_netlink_dp_from_ofpbuf(struct dpif_netlink_dp *dp, const struct ofpbuf *buf dp->user_features = nl_attr_get_u32(a[OVS_DP_ATTR_USER_FEATURES]); } + if (a[OVS_DP_ATTR_MASKS_CACHE_SIZE]) { + dp->cache_size = nl_attr_get_u32(a[OVS_DP_ATTR_MASKS_CACHE_SIZE]); + } else { + dp->cache_size = UINT32_MAX; + } + return 0; } @@ -4677,6 +4789,10 @@ dpif_netlink_dp_to_ofpbuf(const struct dpif_netlink_dp *dp, struct ofpbuf *buf) sizeof *dp->upcall_pids * dp->n_upcall_pids); } + if (dp->cache_size != UINT32_MAX) { + nl_msg_put_u32(buf, OVS_DP_ATTR_MASKS_CACHE_SIZE, dp->cache_size); + } + /* Skip OVS_DP_ATTR_STATS since we never have a reason to serialize it. */ } @@ -4685,6 +4801,7 @@ static void dpif_netlink_dp_init(struct dpif_netlink_dp *dp) { memset(dp, 0, sizeof *dp); + dp->cache_size = UINT32_MAX; } static void diff --git a/lib/dpif-provider.h b/lib/dpif-provider.h index 7e11b9697..27e3a7658 100644 --- a/lib/dpif-provider.h +++ b/lib/dpif-provider.h @@ -635,6 +635,26 @@ struct dpif_class { * sufficient to store BOND_BUCKETS number of elements. */ int (*bond_stats_get)(struct dpif *dpif, uint32_t bond_id, uint64_t *n_bytes); + + /* Cache configuration + * + * Multiple levels of cache can exist in a given datapath implementation. + * An API has been provided to get the number of supported caches, which + * can then be used to get/set specific configuration. Cache level is 0 + * indexed, i.e. if 1 level is supported, the level value to use is 0. + * + * Get the number of cache levels supported. */ + int (*cache_get_supported_levels)(struct dpif *dpif, uint32_t *levels); + + /* Get the cache name for the given level. */ + int (*cache_get_name)(struct dpif *dpif, uint32_t level, + const char **name); + + /* Get currently configured cache size. */ + int (*cache_get_size)(struct dpif *dpif, uint32_t level, uint32_t *size); + + /* Set cache size. */ + int (*cache_set_size)(struct dpif *dpif, uint32_t level, uint32_t size); }; extern const struct dpif_class dpif_netlink_class; diff --git a/lib/dpif.c b/lib/dpif.c index 8c4aed47b..0b94d6ff0 100644 --- a/lib/dpif.c +++ b/lib/dpif.c @@ -2058,3 +2058,35 @@ dpif_get_n_offloaded_flows(struct dpif *dpif, uint64_t *n_flows) } return n_devs ? 0 : EOPNOTSUPP; } + +int +dpif_cache_get_supported_levels(struct dpif *dpif, uint32_t *levels) +{ + return dpif->dpif_class->cache_get_supported_levels + ? dpif->dpif_class->cache_get_supported_levels(dpif, levels) + : EOPNOTSUPP; +} + +int +dpif_cache_get_name(struct dpif *dpif, uint32_t level, const char **name) +{ + return dpif->dpif_class->cache_get_name + ? dpif->dpif_class->cache_get_name(dpif, level, name) + : EOPNOTSUPP; +} + +int +dpif_cache_get_size(struct dpif *dpif, uint32_t level, uint32_t *size) +{ + return dpif->dpif_class->cache_get_size + ? dpif->dpif_class->cache_get_size(dpif, level, size) + : EOPNOTSUPP; +} + +int +dpif_cache_set_size(struct dpif *dpif, uint32_t level, uint32_t size) +{ + return dpif->dpif_class->cache_set_size + ? dpif->dpif_class->cache_set_size(dpif, level, size) + : EOPNOTSUPP; +} diff --git a/lib/dpif.h b/lib/dpif.h index b32ae5fc7..8febfb9f6 100644 --- a/lib/dpif.h +++ b/lib/dpif.h @@ -908,6 +908,13 @@ int dpif_bond_del(struct dpif *, uint32_t bond_id); int dpif_bond_stats_get(struct dpif *, uint32_t bond_id, uint64_t *n_bytes); bool dpif_supports_lb_output_action(const struct dpif *); + +/* Cache */ +int dpif_cache_get_supported_levels(struct dpif *dpif, uint32_t *levels); +int dpif_cache_get_name(struct dpif *dpif, uint32_t level, const char **name); +int dpif_cache_get_size(struct dpif *dpif, uint32_t level, uint32_t *size); +int dpif_cache_set_size(struct dpif *dpif, uint32_t level, uint32_t size); + /* Miscellaneous. */ diff --git a/tests/system-traffic.at b/tests/system-traffic.at index f400cfabc..662ad2a8f 100644 --- a/tests/system-traffic.at +++ b/tests/system-traffic.at @@ -1454,6 +1454,42 @@ AT_CHECK([ovs-ofctl dump-flows br0 | grep "in_port=4" | ofctl_strip], [0], [dnl OVS_TRAFFIC_VSWITCHD_STOP AT_CLEANUP +AT_SETUP([datapath - configure cache size]) + +OVS_TRAFFIC_VSWITCHD_START() +OVS_CHECK_KERNEL_EXCL(3, 10, 5, 8) + +AT_CHECK([ovs-dpctl cache-get-size one-bad-dp], [1], [], [dnl +ovs-dpctl: Opening datapath one-bad-dp failed (No such device) +]) +AT_CHECK([ovs-dpctl cache-get-size | grep masks-cache | tr -d [[:blank:]]], [0], [dnl +masks-cache:size:256 +]) +AT_CHECK([ovs-dpctl cache-set-size one-bad-dp masks-cache 0], [1], [], [dnl +ovs-dpctl: Opening datapath one-bad-dp failed (No such device) +]) +AT_CHECK([ovs-dpctl cache-set-size system@ovs-system dummy-cache 0], [1], [], [dnl +ovs-dpctl: Cache name "dummy-cache" not found on dpif (Invalid argument) +]) +AT_CHECK([ovs-dpctl cache-set-size system@ovs-system masks-cache 80000], [1], [], [dnl +ovs-dpctl: Setting cache size failed (Numerical result out of range) +]) +AT_CHECK([ovs-dpctl cache-set-size system@ovs-system masks-cache 0], [0], [dnl +Setting cache size successful, new size 0 +]) +AT_CHECK([ovs-dpctl cache-get-size | grep masks-cache | tr -d [[:blank:]]], [0], [dnl +masks-cache:size:0 +]) +AT_CHECK([ovs-dpctl cache-set-size system@ovs-system masks-cache 256], [0], [dnl +Setting cache size successful, new size 256 +]) +AT_CHECK([ovs-dpctl cache-get-size | grep masks-cache | tr -d [[:blank:]]], [0], [dnl +masks-cache:size:256 +]) + +OVS_TRAFFIC_VSWITCHD_STOP +AT_CLEANUP + AT_BANNER([conntrack]) AT_SETUP([conntrack - controller]) diff --git a/utilities/ovs-dpctl.c b/utilities/ovs-dpctl.c index f616995c3..56d7a942b 100644 --- a/utilities/ovs-dpctl.c +++ b/utilities/ovs-dpctl.c @@ -198,6 +198,10 @@ usage(void *userdata OVS_UNUSED) " del-flow [DP] FLOW delete FLOW from DP\n" " del-flows [DP] [FILE] " \ "delete all or specified flows from DP\n" + " cache-get-size [DP] " \ + "Show the current size for all caches\n" + " cache-set-size DP CACHE SIZE " \ + "Set cache size for a specific cache\n" " dump-conntrack [DP] [zone=ZONE] " \ "display conntrack entries for ZONE\n" " flush-conntrack [DP] [zone=ZONE] [ct-tuple]" \