From patchwork Thu Sep 27 06:48:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li,Rongqing via dev" X-Patchwork-Id: 975549 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=openvswitch.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=broadcom.com header.i=@broadcom.com header.b="QQ14LiDr"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42LQB74g3Nz9s1x for ; Thu, 27 Sep 2018 16:38:39 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 700631028; Thu, 27 Sep 2018 06:38:16 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 53F181009 for ; Thu, 27 Sep 2018 06:38:15 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 836F4F8 for ; Thu, 27 Sep 2018 06:38:14 +0000 (UTC) Received: by mail-wr1-f66.google.com with SMTP id j15-v6so1241975wrt.8 for ; Wed, 26 Sep 2018 23:38:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=bOFxBYZBy7d7vp1sk3Xhg+u6t8ldxEcDab6Xar5IpFA=; b=QQ14LiDrcWh5UA1M6FScHZZ35JsAV+S6UYz2MEqfcSrbVXtQqJ4/38k83f3NuacHjm dI4uZpbD69z8GcvPSOD7Ifwpl0WHSqrW3WaRKWfkqnXg0aaNBn04n6N/6MXM0H6kh2YC 465O9zDG2at6HlzvYKcbTMfkyVx3jSPsavC2I= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=bOFxBYZBy7d7vp1sk3Xhg+u6t8ldxEcDab6Xar5IpFA=; b=sric6cXNiT7GYg6gvoecXAy+24OfW+rziQ39jQFfSYOtQtDq4v4LBdVzZ1IC/TZ0YB v38yRyCVLc03mvLtGmk+bLR3QghYn7rMKyQeE0aXqJFJti8gSBiM+RRlUAF/Agf+6XBW GkPQm9h+7ecbejdtWx1M2nGyPXsQSxsdvfexE3VjvYal33Ffdr58UZko6WSWyq35s4PZ TI8qlFIsagbZfqmTcZPR19Rxnjah9+QLqUz2CySlLY67ouw9PTgBlrFEQHWQZfbfYfJm 3f2dG2jdevulHlFiptjmkhbHiqRBplgIi806/6N1ulmhPBHND1YR5XGRr2Jds4SZd3Vb qEnA== X-Gm-Message-State: ABuFfoinIrqsVOdIaMt/nuw324dXjYK3SS3LDcRv7GyJJBegrR/pR8/Z 9m2k2+DNZwWLs9cP9P89KYY1OZ9kOHg= X-Google-Smtp-Source: ACcGV62yj18DGHdMk1a9B9gBtH9aqvKDYyLa572mxM9jsLW8jBgOv4tf0+isRhQ0zNskBW5l9yY0ag== X-Received: by 2002:adf:f301:: with SMTP id i1-v6mr5104590wro.161.1538030292567; Wed, 26 Sep 2018 23:38:12 -0700 (PDT) Received: from host-10-123-153-36.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id 64-v6sm1488547wrr.64.2018.09.26.23.38.09 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 26 Sep 2018 23:38:11 -0700 (PDT) To: dev@openvswitch.org Date: Thu, 27 Sep 2018 12:18:25 +0530 Message-Id: <20180927064827.26256-2-sriharsha.basavapatna@broadcom.com> X-Mailer: git-send-email 2.18.0.rc1.1.g6f333ff In-Reply-To: <20180927064827.26256-1-sriharsha.basavapatna@broadcom.com> References: <20180927064827.26256-1-sriharsha.basavapatna@broadcom.com> X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v7 1/3] dpif-netlink: Detect Out-Of-Resource condition on a netdev X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Sriharsha Basavapatna via dev From: "Li,Rongqing via dev" Reply-To: Sriharsha Basavapatna MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org This is the first patch in the patch-set to support dynamic rebalancing of offloaded flows. The patch detects OOR condition on a netdev port when ENOSPC error is returned by TC-Flower while adding a flow rule. A new structure is added to the netdev called "netdev_hw_info", to store OOR related information required to perform dynamic offload-rebalancing. Signed-off-by: Sriharsha Basavapatna Co-authored-by: Venkat Duvvuru Signed-off-by: Venkat Duvvuru Reviewed-by: Sathya Perla --- lib/dpif-netlink.c | 18 +++++++++++++++++- lib/flow.c | 25 +++++++++++++++++++++++++ lib/flow.h | 1 + lib/netdev-provider.h | 11 +++++++++++ lib/netdev.c | 35 +++++++++++++++++++++++++++++++++++ lib/netdev.h | 3 +++ 6 files changed, 92 insertions(+), 1 deletion(-) diff --git a/lib/dpif-netlink.c b/lib/dpif-netlink.c index e6d5a6ec5..b9ce9cbe2 100644 --- a/lib/dpif-netlink.c +++ b/lib/dpif-netlink.c @@ -2178,7 +2178,23 @@ parse_flow_put(struct dpif_netlink *dpif, struct dpif_flow_put *put) VLOG_DBG("added flow"); } else if (err != EEXIST) { - VLOG_ERR_RL(&rl, "failed to offload flow: %s", ovs_strerror(err)); + struct netdev *oor_netdev = NULL; + if (err == ENOSPC && netdev_is_offload_rebalance_policy_enabled()) { + /* + * We need to set OOR on the input netdev (i.e, 'dev') for the + * flow. But if the flow has a tunnel attribute (i.e, decap action, + * with a virtual device like a VxLAN interface as its in-port), + * then lookup and set OOR on the underlying tunnel (real) netdev. + */ + oor_netdev = flow_get_tunnel_netdev(&match.flow.tunnel); + if (!oor_netdev) { + /* Not a 'tunnel' flow */ + oor_netdev = dev; + } + netdev_set_hw_info(oor_netdev, HW_INFO_TYPE_OOR, true); + } + VLOG_ERR_RL(&rl, "failed to offload flow: %s: %s", ovs_strerror(err), + (oor_netdev ? oor_netdev->name : dev->name)); } out: diff --git a/lib/flow.c b/lib/flow.c index 77ed3d9df..a39807908 100644 --- a/lib/flow.c +++ b/lib/flow.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #include #include @@ -41,6 +42,8 @@ #include "unaligned.h" #include "util.h" #include "openvswitch/nsh.h" +#include "ovs-router.h" +#include "lib/netdev-provider.h" COVERAGE_DEFINE(flow_extract); COVERAGE_DEFINE(miniflow_malloc); @@ -3403,3 +3406,25 @@ flow_limit_vlans(int vlan_limit) flow_vlan_limit = MIN(vlan_limit, FLOW_MAX_VLAN_HEADERS); } } + +struct netdev * +flow_get_tunnel_netdev(struct flow_tnl *tunnel) +{ + char iface[IFNAMSIZ]; + struct in6_addr ip6; + struct in6_addr gw; + + if (tunnel->ip_src) { + in6_addr_set_mapped_ipv4(&ip6, tunnel->ip_src); + } else if (ipv6_addr_is_set(&tunnel->ipv6_src)) { + ip6 = tunnel->ipv6_src; + } else { + return NULL; + } + + if (!ovs_router_lookup(0, &ip6, iface, NULL, &gw)) { + return NULL; + } + + return netdev_from_name(iface); +} diff --git a/lib/flow.h b/lib/flow.h index d03f1ba9c..aca60c41a 100644 --- a/lib/flow.h +++ b/lib/flow.h @@ -73,6 +73,7 @@ void flow_extract(struct dp_packet *, struct flow *); void flow_zero_wildcards(struct flow *, const struct flow_wildcards *); void flow_unwildcard_tp_ports(const struct flow *, struct flow_wildcards *); void flow_get_metadata(const struct flow *, struct match *flow_metadata); +struct netdev *flow_get_tunnel_netdev(struct flow_tnl *tunnel); const char *ct_state_to_string(uint32_t state); uint32_t ct_state_from_string(const char *); diff --git a/lib/netdev-provider.h b/lib/netdev-provider.h index 5a7947351..e320dad61 100644 --- a/lib/netdev-provider.h +++ b/lib/netdev-provider.h @@ -35,6 +35,15 @@ extern "C" { struct netdev_tnl_build_header_params; #define NETDEV_NUMA_UNSPEC OVS_NUMA_UNSPEC +/* Offload-capable (HW) netdev information */ +struct netdev_hw_info { + bool oor; /* Out of Offload Resources ? */ +}; + +enum hw_info_type { + HW_INFO_TYPE_OOR = 1 /* OOR state */ +}; + /* A network device (e.g. an Ethernet device). * * Network device implementations may read these members but should not modify @@ -80,6 +89,8 @@ struct netdev { int n_rxq; struct shash_node *node; /* Pointer to element in global map. */ struct ovs_list saved_flags_list; /* Contains "struct netdev_saved_flags". */ + + struct netdev_hw_info hw_info; /* offload-capable netdev info */ }; static inline void diff --git a/lib/netdev.c b/lib/netdev.c index 690d28409..3e2ce7c09 100644 --- a/lib/netdev.c +++ b/lib/netdev.c @@ -415,6 +415,7 @@ netdev_open(const char *name, const char *type, struct netdev **netdevp) netdev->reconfigure_seq = seq_create(); netdev->last_reconfigure_seq = seq_read(netdev->reconfigure_seq); + netdev->hw_info.oor = false; netdev->node = shash_add(&netdev_shash, name, netdev); /* By default enable one tx and rx queue per netdev. */ @@ -2252,6 +2253,31 @@ netdev_get_block_id(struct netdev *netdev) : 0); } +/* + * Get the value of the hw info parameter specified by type. + * Returns the value on success (>= 0). Returns -1 on failure. + */ +int +netdev_get_hw_info(struct netdev *netdev, int type) +{ + if (type == HW_INFO_TYPE_OOR) { + return netdev->hw_info.oor; + } + + return -1; +} + +/* + * Set the value of the hw info parameter specified by type. + */ +void +netdev_set_hw_info(struct netdev *netdev, int type, int val) +{ + if (type == HW_INFO_TYPE_OOR) { + netdev->hw_info.oor = val; + } +} + bool netdev_is_flow_api_enabled(void) { @@ -2489,6 +2515,15 @@ netdev_free_custom_stats_counters(struct netdev_custom_stats *custom_stats) } #ifdef __linux__ + +static bool netdev_offload_rebalance_policy = false; + +bool +netdev_is_offload_rebalance_policy_enabled(void) +{ + return netdev_offload_rebalance_policy; +} + static void netdev_ports_flow_init(void) { diff --git a/lib/netdev.h b/lib/netdev.h index 556676046..b0e5c5b72 100644 --- a/lib/netdev.h +++ b/lib/netdev.h @@ -227,8 +227,11 @@ int netdev_flow_del(struct netdev *, const ovs_u128 *, struct dpif_flow_stats *); int netdev_init_flow_api(struct netdev *); uint32_t netdev_get_block_id(struct netdev *); +int netdev_get_hw_info(struct netdev *, int); +void netdev_set_hw_info(struct netdev *, int, int); bool netdev_is_flow_api_enabled(void); void netdev_set_flow_api_enabled(const struct smap *ovs_other_config); +bool netdev_is_offload_rebalance_policy_enabled(void); struct dpif_port; int netdev_ports_insert(struct netdev *, const struct dpif_class *,