From patchwork Tue Jul 10 06:29:38 2018
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
X-Patchwork-Submitter: "Li,Rongqing via dev"
X-Patchwork-Id: 941812
Return-Path:
X-Original-To: incoming@patchwork.ozlabs.org
Delivered-To: patchwork-incoming@bilbo.ozlabs.org
Authentication-Results: ozlabs.org;
spf=pass (mailfrom) smtp.mailfrom=openvswitch.org
(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;
envelope-from=ovs-dev-bounces@openvswitch.org;
receiver=)
Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none)
header.from=openvswitch.org
Authentication-Results: ozlabs.org;
dkim=fail reason="signature verification failed" (1024-bit key;
unprotected) header.d=broadcom.com header.i=@broadcom.com
header.b="dc22PcSm"; dkim-atps=neutral
Received: from mail.linuxfoundation.org (mail.linuxfoundation.org
[140.211.169.12])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256
bits)) (No client certificate requested)
by ozlabs.org (Postfix) with ESMTPS id 41PsYC3MxCz9s37
for ;
Tue, 10 Jul 2018 16:21:51 +1000 (AEST)
Received: from mail.linux-foundation.org (localhost [127.0.0.1])
by mail.linuxfoundation.org (Postfix) with ESMTP id 83B05D14;
Tue, 10 Jul 2018 06:20:09 +0000 (UTC)
X-Original-To: dev@openvswitch.org
Delivered-To: ovs-dev@mail.linuxfoundation.org
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
[172.17.192.35])
by mail.linuxfoundation.org (Postfix) with ESMTPS id 16357D12
for ; Tue, 10 Jul 2018 06:20:09 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.7.6
Received: from mail-qk0-f196.google.com (mail-qk0-f196.google.com
[209.85.220.196])
by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 5BA7667E
for ; Tue, 10 Jul 2018 06:20:07 +0000 (UTC)
Received: by mail-qk0-f196.google.com with SMTP id c131-v6so10985291qkb.0
for ; Mon, 09 Jul 2018 23:20:07 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com;
s=google;
h=from:to:cc:subject:date:message-id:in-reply-to:references;
bh=BbfNuHaGyISvRuMTeT1BpUu+DtBMaSJ1/Sj8uiI77Jw=;
b=dc22PcSmbrcV0GRgnDkCgSuOO21XejaC21alY0Xd0L42VmJW93yhYENL/6Ppu0+UZ8
elZXx5pWtyxa08t9CPU/UoUbAVjBvJfkY4PkwFMP04dy2S1WLs3s9Z3wQutgBgHMrrtX
PGcjnonKxWRHcnOnyBw24xMZl5QNUe5D88xsI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
:references;
bh=BbfNuHaGyISvRuMTeT1BpUu+DtBMaSJ1/Sj8uiI77Jw=;
b=Ngp3aevGJMj5sikpauq6LX+2soe37z7vf2+9gnwwIAMrJ6SH+Nokdx5QzDUPbrlSHj
FU1ge9z2B2qBZPzmz192vg/ffzLICUQZorQybqClxByjdBsqzjUMHAMcA06SRWxy38Ps
wI5j+jd8q/5a4J5TtFJzDH+4+SoAvo8X46gg/lqI3HfEii/gHWVIFc42pSyxYKn3YEsT
RswLGm5Cgl/JahUakQ5Sp2Ja0r6vW8goBsXgwYFy+Kde+JwggsSnDVDIf9rJCX8sO0VE
9k32GsNOoHr4arzAtf2EnnIJw8uZ4eyIIffGdKr0CswKBU/Gb5dKISmFdYoK9+sawjLE
fcXQ==
X-Gm-Message-State: APt69E2eWb4SiZY9I6Q/4XcZDr7EZP/NlWGV7/KrKEQs/W/K1ISd+zud
NJBtbKrV8qhST7IaDMMAgHqHOeLDAzY=
X-Google-Smtp-Source:
AAOMgpcdcib27LRsUPpCFWePRkSSBG0tcWTTaesdxljzUB2qy27F9tngsFpuVtLG2dJLO9ttZhJSbA==
X-Received: by 2002:a37:1adb:: with SMTP id
l88-v6mr20251705qkh.413.1531203606181;
Mon, 09 Jul 2018 23:20:06 -0700 (PDT)
Received: from host-10-123-153-36.dhcp.broadcom.net ([192.19.234.250])
by smtp.gmail.com with ESMTPSA id
m51-v6sm2592203qtc.85.2018.07.09.23.20.03
(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
Mon, 09 Jul 2018 23:20:05 -0700 (PDT)
To: dev@openvswitch.org
Date: Tue, 10 Jul 2018 11:59:38 +0530
Message-Id: <20180710062938.18939-5-sriharsha.basavapatna@broadcom.com>
X-Mailer: git-send-email 2.18.0.rc1.1.g6f333ff
In-Reply-To: <20180710062938.18939-1-sriharsha.basavapatna@broadcom.com>
References: <20180710062938.18939-1-sriharsha.basavapatna@broadcom.com>
X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED,
DKIM_VALID, DKIM_VALID_AU,
RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
smtp1.linux-foundation.org
Subject: [ovs-dev] [PATCH v3 4/4] netdev: Add a configuration option to
enable dynamic rebalancing of flows
X-BeenThere: ovs-dev@openvswitch.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id:
List-Unsubscribe: ,
List-Archive:
List-Post:
List-Help:
List-Subscribe: ,
X-Patchwork-Original-From: Sriharsha Basavapatna via dev
From: "Li,Rongqing via dev"
Reply-To: Sriharsha Basavapatna
MIME-Version: 1.0
Sender: ovs-dev-bounces@openvswitch.org
Errors-To: ovs-dev-bounces@openvswitch.org
This is the fourth patch in the patch-set to support dynamic rebalancing
of offloaded flows.
A new OVS configuration parameter "offload-rebalance", is added to ovsdb.
The default value of this is "disable". To enable this feature, set the
value of this parameter to "pps-rate", which provides packets-per-second
rate based policy to dynamically offload and un-offload flows.
Note: This option can be enabled only when 'hw-offload' policy is enabled.
It also requires 'tc-policy' to be set to 'skip_sw'; otherwise, flow
offload errors (specifically ENOSPC error this feature depends on) reported
by an offloaded device are supressed by TC-Flower kernel module.
Signed-off-by: Sriharsha Basavapatna
Co-authored-by: Venkat Duvvuru
Signed-off-by: Venkat Duvvuru
Reviewed-by: Sathya Perla
---
lib/dpif-netlink.c | 3 ++-
lib/netdev.c | 43 +++++++++++++++++++++++++++++++++++
lib/netdev.h | 2 ++
ofproto/ofproto-dpif-upcall.c | 7 ++++--
vswitchd/vswitch.xml | 22 ++++++++++++++++++
5 files changed, 74 insertions(+), 3 deletions(-)
diff --git a/lib/dpif-netlink.c b/lib/dpif-netlink.c
index 3c4b9b69f..dfe21971d 100644
--- a/lib/dpif-netlink.c
+++ b/lib/dpif-netlink.c
@@ -2177,7 +2177,8 @@ parse_flow_put(struct dpif_netlink *dpif, struct dpif_flow_put *put)
VLOG_DBG("added flow");
} else if (err != EEXIST) {
- if (outdev && dev && (err == ENOSPC)) {
+ if (netdev_is_offload_rebalance_policy_enabled() && outdev &&
+ dev && (err == ENOSPC)) {
tunnel_netdev = flow_get_tunnel_netdev(&match.flow.tunnel);
if (tunnel_netdev) {
oor_netdev = tunnel_netdev;
diff --git a/lib/netdev.c b/lib/netdev.c
index b17f0563f..9992b846c 100644
--- a/lib/netdev.c
+++ b/lib/netdev.c
@@ -2474,6 +2474,43 @@ netdev_free_custom_stats_counters(struct netdev_custom_stats *custom_stats)
#ifdef __linux__
+enum offload_rebalance_policy {
+ OFFLOAD_REBALANCE_POLICY_DISABLE,
+ OFFLOAD_REBALANCE_POLICY_PPS_RATE
+};
+
+static enum
+offload_rebalance_policy netdev_offload_rebalance_policy =
+ OFFLOAD_REBALANCE_POLICY_DISABLE;
+
+static void
+netdev_offload_rebalance_set_policy(const char *policy)
+{
+ if (!policy) {
+ return;
+ }
+
+ if (!strcmp(policy, "pps-rate")) {
+ netdev_offload_rebalance_policy = OFFLOAD_REBALANCE_POLICY_PPS_RATE;
+ } else if (!strcmp(policy, "disable")) {
+ netdev_offload_rebalance_policy = OFFLOAD_REBALANCE_POLICY_DISABLE;
+ } else {
+ VLOG_WARN("netdev: Invalid policy '%s'", policy);
+ return;
+ }
+
+ VLOG_INFO("netdev: Using policy '%s'", policy);
+}
+
+bool
+netdev_is_offload_rebalance_policy_enabled(void)
+{
+ if (netdev_offload_rebalance_policy == OFFLOAD_REBALANCE_POLICY_PPS_RATE) {
+ return true;
+ }
+ return false;
+}
+
static void
netdev_ports_flow_init(void)
{
@@ -2494,12 +2531,18 @@ netdev_set_flow_api_enabled(const struct smap *ovs_other_config)
if (ovsthread_once_start(&once)) {
netdev_flow_api_enabled = true;
+ const char *offl_rebal_policy = NULL;
VLOG_INFO("netdev: Flow API Enabled");
tc_set_policy(smap_get_def(ovs_other_config, "tc-policy",
TC_POLICY_DEFAULT));
+ offl_rebal_policy = smap_get_def(ovs_other_config,
+ "offload-rebalance",
+ OFFLOAD_REBALANCE_POLICY_DEFAULT);
+ netdev_offload_rebalance_set_policy(offl_rebal_policy);
+
netdev_ports_flow_init();
ovsthread_once_done(&once);
diff --git a/lib/netdev.h b/lib/netdev.h
index c941f1e1e..0ac2774c9 100644
--- a/lib/netdev.h
+++ b/lib/netdev.h
@@ -223,6 +223,8 @@ int netdev_init_flow_api(struct netdev *);
uint32_t netdev_get_block_id(struct netdev *);
bool netdev_is_flow_api_enabled(void);
void netdev_set_flow_api_enabled(const struct smap *ovs_other_config);
+bool netdev_is_offload_rebalance_policy_enabled(void);
+#define OFFLOAD_REBALANCE_POLICY_DEFAULT "disable"
struct dpif_port;
int netdev_ports_insert(struct netdev *, const struct dpif_class *,
diff --git a/ofproto/ofproto-dpif-upcall.c b/ofproto/ofproto-dpif-upcall.c
index cf1524615..0910b66b1 100644
--- a/ofproto/ofproto-dpif-upcall.c
+++ b/ofproto/ofproto-dpif-upcall.c
@@ -967,7 +967,9 @@ udpif_revalidator(void *arg)
dpif_flow_dump_destroy(udpif->dump);
seq_change(udpif->dump_seq);
- udpif_run_flow_rebalance(udpif);
+ if (netdev_is_offload_rebalance_policy_enabled()) {
+ udpif_run_flow_rebalance(udpif);
+ }
duration = MAX(time_msec() - start_time, 1);
udpif->dump_duration = duration;
@@ -2741,7 +2743,8 @@ revalidate(struct revalidator *revalidator)
}
ukey->dump_seq = dump_seq;
- if (result != UKEY_DELETE) {
+ if (netdev_is_offload_rebalance_policy_enabled() &&
+ result != UKEY_DELETE) {
udpif_update_flow_pps(udpif, ukey, f);
}
diff --git a/vswitchd/vswitch.xml b/vswitchd/vswitch.xml
index 76094852d..a0aad0204 100644
--- a/vswitchd/vswitch.xml
+++ b/vswitchd/vswitch.xml
@@ -475,6 +475,28 @@
+
+
+ Configures HW offload rebalancing, that allows to dynamically
+ offload and un-offload flows while an offload-device is out of
+ resources (OOR). The value specifies the criteria used to
+ select flows for offloading. The default value of 'disable',
+ disables this policy. To enable packets-per-second rate based
+ offload policy, set the value to 'pps-rate'.
+ Options:
+ disable
- No offload rebalancing
+ pps-rate
- Packets-per-second rate based policy
+
+
+ This is only relevant if HW offloading is enabled (hw-offload).
+ When this policy is enabled, it also requires 'tc-policy' to
+ be set to 'skip_sw'.
+
+
+ The default value is disable
.
+
+