From patchwork Thu Sep 10 03:09:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: yang_y_yi X-Patchwork-Id: 1361171 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=163.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=163.com header.i=@163.com header.a=rsa-sha256 header.s=s110527 header.b=gguGZKCv; dkim-atps=neutral Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Bn3ln1RDbz9sTd for ; Thu, 10 Sep 2020 13:09:57 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 84CB387642; Thu, 10 Sep 2020 03:09:53 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OKZpNVjGkJC7; Thu, 10 Sep 2020 03:09:49 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by hemlock.osuosl.org (Postfix) with ESMTP id 2177287638; Thu, 10 Sep 2020 03:09:49 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D929EC0893; Thu, 10 Sep 2020 03:09:48 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 62FEAC0051 for ; Thu, 10 Sep 2020 03:09:47 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 52F5D86DE0 for ; Thu, 10 Sep 2020 03:09:47 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wWxV5zT2cMgx for ; Thu, 10 Sep 2020 03:09:46 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-m971.mail.163.com (mail-m971.mail.163.com [123.126.97.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 6949986E00 for ; Thu, 10 Sep 2020 03:09:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:Subject:Date:Message-Id:MIME-Version; bh=ACrf2 yZUsYn1gDl+51B9mqyNXOYehuLKSuBD+m+VE9Y=; b=gguGZKCv+PsruQCneBT3s 1jvaBzgD3W1+7eiM8OdSHhhBInVN3+cOvRkwWiZmNgwzvWCcDDZkgQ1zMqXb7KO8 ohOko8mJ5kzUL5C17VegI0BKP3Uavxbg2Ew3leifH7V/of5e397XFIIN2F/HECLL cG+uDzgrl+drNKecyv5qyQ= Received: from yangyi0100.home.langchao.com (unknown [111.207.123.58]) by smtp1 (Coremail) with SMTP id GdxpCgDnTzDvmFlfaPxiBQ--.766S4; Thu, 10 Sep 2020 11:09:36 +0800 (CST) From: yang_y_yi@163.com To: ovs-dev@openvswitch.org Date: Thu, 10 Sep 2020 11:09:33 +0800 Message-Id: <20200910030934.116248-3-yang_y_yi@163.com> X-Mailer: git-send-email 2.19.2.windows.1 In-Reply-To: <20200910030934.116248-1-yang_y_yi@163.com> References: <20200910030934.116248-1-yang_y_yi@163.com> MIME-Version: 1.0 X-CM-TRANSID: GdxpCgDnTzDvmFlfaPxiBQ--.766S4 X-Coremail-Antispam: 1Uf129KBjvAXoW3uFWrXw1UWFWrZw4xtFWfGrg_yoW8Jry8Ao WxArnFqF18Gr1Uuryjgwn3JryUZ3y7GF4rAFs8Can0kF1Iqw4Fqr4xCw4YvFW5ZF4FqF47 urWIyayfKF4DWF93n29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7v73VFW2AGmfu7bjvjm3 AaLaJ3UbIYCTnIWIevJa73UjIFyTuYvjxUOKsjDUUUU X-Originating-IP: [111.207.123.58] X-CM-SenderInfo: 51dqwsp1b1xqqrwthudrp/1tbiEBGbi18YDCwJuQAAsG Cc: i.maximets@ovn.org, yang_y_yi@163.com, fbl@sysclose.org Subject: [ovs-dev] [PATCH v2 2/3] Fix tap interface statistics issue X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Yi Yang After tap interface is moved to network namespace, "ovs-vsctl list interface tapXXX" can get statistics info of tap interface, the root cause is OVS still gets statistics info in root namespace. With netns option help, OVS can get statistics info in tap interface netns. This patch added enter and exit netns helpers and change statistics-related functions for those tap interfaces which have been moved into netns and make sure "ovs-vsctl list interface tapXXX" can get statistics info correctly. Here is a result sample for reference: name : tap1 ofport : 4 ofport_request : [] options : {netns=ns01} other_config : {} statistics : {rx_bytes=6228, rx_packets=68, tx_bytes=8310, tx_packets=95} status : {} type : tap Signed-off-by: Yi Yang --- lib/dpif-netlink.c | 51 ++++++++++++++++++ lib/dpif-netlink.h | 3 ++ lib/netdev-linux.c | 63 +++++++++++++++++++++- lib/netlink-socket.c | 146 +++++++++++++++++++++++++++++++++++++++++++++++++++ lib/netlink-socket.h | 2 + 5 files changed, 263 insertions(+), 2 deletions(-) diff --git a/lib/dpif-netlink.c b/lib/dpif-netlink.c index 7da4fb5..8ed37ed 100644 --- a/lib/dpif-netlink.c +++ b/lib/dpif-netlink.c @@ -4282,6 +4282,43 @@ dpif_netlink_vport_transact(const struct dpif_netlink_vport *request, return error; } +static int +dpif_netlink_vport_transact_nopool(const struct dpif_netlink_vport *request, + struct dpif_netlink_vport *reply, + struct ofpbuf **bufp) +{ + struct ofpbuf *request_buf; + int error; + + ovs_assert((reply != NULL) == (bufp != NULL)); + + error = dpif_netlink_init(); + if (error) { + if (reply) { + *bufp = NULL; + dpif_netlink_vport_init(reply); + } + return error; + } + + request_buf = ofpbuf_new(1024); + dpif_netlink_vport_to_ofpbuf(request, request_buf); + error = nl_transact_nopool(NETLINK_GENERIC, request_buf, bufp); + ofpbuf_delete(request_buf); + + if (reply) { + if (!error) { + error = dpif_netlink_vport_from_ofpbuf(reply, *bufp); + } + if (error) { + dpif_netlink_vport_init(reply); + ofpbuf_delete(*bufp); + *bufp = NULL; + } + } + return error; +} + /* Obtains information about the kernel vport named 'name' and stores it into * '*reply' and '*bufp'. The caller must free '*bufp' when the reply is no * longer needed ('reply' will contain pointers into '*bufp'). */ @@ -4298,6 +4335,20 @@ dpif_netlink_vport_get(const char *name, struct dpif_netlink_vport *reply, return dpif_netlink_vport_transact(&request, reply, bufp); } +int +dpif_netlink_vport_get_nopool(const char *name, + struct dpif_netlink_vport *reply, + struct ofpbuf **bufp) +{ + struct dpif_netlink_vport request; + + dpif_netlink_vport_init(&request); + request.cmd = OVS_VPORT_CMD_GET; + request.name = name; + + return dpif_netlink_vport_transact_nopool(&request, reply, bufp); +} + /* Parses the contents of 'buf', which contains a "struct ovs_header" followed * by Netlink attributes, into 'dp'. Returns 0 if successful, otherwise a * positive errno value. diff --git a/lib/dpif-netlink.h b/lib/dpif-netlink.h index 24294bc..9372241 100644 --- a/lib/dpif-netlink.h +++ b/lib/dpif-netlink.h @@ -55,6 +55,9 @@ int dpif_netlink_vport_transact(const struct dpif_netlink_vport *request, struct ofpbuf **bufp); int dpif_netlink_vport_get(const char *name, struct dpif_netlink_vport *reply, struct ofpbuf **bufp); +int dpif_netlink_vport_get_nopool(const char *name, + struct dpif_netlink_vport *reply, + struct ofpbuf **bufp); bool dpif_netlink_is_internal_device(const char *name); diff --git a/lib/netdev-linux.c b/lib/netdev-linux.c index ebe4ddb..346dc94 100644 --- a/lib/netdev-linux.c +++ b/lib/netdev-linux.c @@ -520,8 +520,16 @@ static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 20); * changes in the device miimon status, so we can use atomic_count. */ static atomic_count miimon_cnt = ATOMIC_COUNT_INIT(0); +struct netns_knob { + int main_fd; + int netns_fd; + char nspath[128]; +}; + static int netdev_linux_parse_vnet_hdr(struct dp_packet *b); static void netdev_linux_prepend_vnet_hdr(struct dp_packet *b, int mtu); +static int enter_netns(struct netns_knob *netns_knob, const char *netns); +static void exit_netns(struct netns_knob *netns_knob); static int netdev_linux_do_ethtool(const char *name, struct ethtool_cmd *, int cmd, const char *cmd_name); static int get_flags(const struct netdev *, unsigned int *flags); @@ -2158,11 +2166,18 @@ netdev_stats_from_ovs_vport_stats(struct netdev_stats *dst, static int get_stats_via_vport__(const struct netdev *netdev, struct netdev_stats *stats) { + struct netdev_linux *netdev_linux = netdev_linux_cast(netdev); struct dpif_netlink_vport reply; struct ofpbuf *buf; int error; - error = dpif_netlink_vport_get(netdev_get_name(netdev), &reply, &buf); + if (is_tap_netdev(netdev) && (netdev_linux->netns != NULL)) { + error = dpif_netlink_vport_get_nopool(netdev_get_name(netdev), + &reply, &buf); + } else { + error = dpif_netlink_vport_get(netdev_get_name(netdev), + &reply, &buf); + } if (error) { return error; } else if (!reply.stats) { @@ -2237,6 +2252,33 @@ netdev_linux_get_stats(const struct netdev *netdev_, return error; } +static int +enter_netns(struct netns_knob *netns_knob, const char *netns) +{ + sprintf(netns_knob->nspath, "/proc/%d/ns/net", getpid()); + netns_knob->main_fd = open(netns_knob->nspath, O_RDONLY); + if (netns_knob->main_fd < 0) { + return errno; + } + + sprintf(netns_knob->nspath, "/var/run/netns/%s", netns); + netns_knob->netns_fd = open(netns_knob->nspath, O_RDONLY); + if (netns_knob->netns_fd < 0) { + close(netns_knob->main_fd); + return errno; + } + setns(netns_knob->netns_fd, CLONE_NEWNET); + return 0; +} + +static void +exit_netns(struct netns_knob *netns_knob) +{ + setns(netns_knob->main_fd, CLONE_NEWNET); + close(netns_knob->netns_fd); + close(netns_knob->main_fd); +} + /* Retrieves current device stats for 'netdev-tap' netdev or * netdev-internal. */ static int @@ -2245,6 +2287,14 @@ netdev_tap_get_stats(const struct netdev *netdev_, struct netdev_stats *stats) struct netdev_linux *netdev = netdev_linux_cast(netdev_); struct netdev_stats dev_stats; int error; + struct netns_knob netns_knob; + + if (is_tap_netdev(netdev_) && (netdev->netns != NULL)) { + error = enter_netns(&netns_knob, netdev->netns); + if (error) { + return error; + } + } ovs_mutex_lock(&netdev->mutex); get_stats_via_vport(netdev_, stats); @@ -2298,6 +2348,10 @@ netdev_tap_get_stats(const struct netdev *netdev_, struct netdev_stats *stats) stats->rx_dropped += netdev->rx_dropped; ovs_mutex_unlock(&netdev->mutex); + if (is_tap_netdev(netdev_) && (netdev->netns != NULL)) { + exit_netns(&netns_knob); + } + return error; } @@ -6245,6 +6299,7 @@ netdev_stats_from_rtnl_link_stats64(struct netdev_stats *dst, int get_stats_via_netlink(const struct netdev *netdev_, struct netdev_stats *stats) { + struct netdev_linux *netdev = netdev_linux_cast(netdev_); struct ofpbuf request; struct ofpbuf *reply; int error; @@ -6258,7 +6313,11 @@ get_stats_via_netlink(const struct netdev *netdev_, struct netdev_stats *stats) RTM_GETLINK, NLM_F_REQUEST); ofpbuf_put_zeros(&request, sizeof(struct ifinfomsg)); nl_msg_put_string(&request, IFLA_IFNAME, netdev_get_name(netdev_)); - error = nl_transact(NETLINK_ROUTE, &request, &reply); + if (is_tap_netdev(netdev_) && (netdev->netns != NULL)) { + error = nl_transact_nopool(NETLINK_ROUTE, &request, &reply); + } else { + error = nl_transact(NETLINK_ROUTE, &request, &reply); + } ofpbuf_uninit(&request); if (error) { return error; diff --git a/lib/netlink-socket.c b/lib/netlink-socket.c index 47077e9..2377ca7 100644 --- a/lib/netlink-socket.c +++ b/lib/netlink-socket.c @@ -1807,6 +1807,152 @@ nl_transact(int protocol, const struct ofpbuf *request, return error; } +static int +nl_sock_create_nopool(int protocol, struct nl_sock **sockp) +{ + struct nl_sock *sock; +#ifndef _WIN32 + struct sockaddr_nl local, remote; +#endif + socklen_t local_size; + int rcvbuf; + int retval = 0; + + *sockp = NULL; + sock = xmalloc(sizeof *sock); + +#ifdef _WIN32 + sock->overlapped.hEvent = NULL; + sock->handle = CreateFile(OVS_DEVICE_NAME_USER, + GENERIC_READ | GENERIC_WRITE, + FILE_SHARE_READ | FILE_SHARE_WRITE, + NULL, OPEN_EXISTING, + FILE_FLAG_OVERLAPPED, NULL); + + if (sock->handle == INVALID_HANDLE_VALUE) { + VLOG_ERR("fcntl: %s", ovs_lasterror_to_string()); + goto error; + } + + memset(&sock->overlapped, 0, sizeof sock->overlapped); + sock->overlapped.hEvent = CreateEvent(NULL, FALSE, FALSE, NULL); + if (sock->overlapped.hEvent == NULL) { + VLOG_ERR("fcntl: %s", ovs_lasterror_to_string()); + goto error; + } + /* Initialize the type/ioctl to Generic */ + sock->read_ioctl = OVS_IOCTL_READ; +#else + sock->fd = socket(AF_NETLINK, SOCK_RAW, protocol); + if (sock->fd < 0) { + VLOG_ERR("fcntl: %s", ovs_strerror(errno)); + goto error; + } +#endif + + sock->protocol = protocol; + sock->next_seq = 1; + + rcvbuf = 1024 * 1024; +#ifdef _WIN32 + sock->rcvbuf = rcvbuf; + retval = get_sock_pid_from_kernel(sock); + if (retval != 0) { + goto error; + } + retval = set_sock_property(sock); + if (retval != 0) { + goto error; + } +#else + if (setsockopt(sock->fd, SOL_SOCKET, SO_RCVBUFFORCE, + &rcvbuf, sizeof rcvbuf)) { + /* Only root can use SO_RCVBUFFORCE. Everyone else gets EPERM. + * Warn only if the failure is therefore unexpected. */ + if (errno != EPERM) { + VLOG_WARN_RL(&rl, "setting %d-byte socket receive buffer failed " + "(%s)", rcvbuf, ovs_strerror(errno)); + } + } + + retval = get_socket_rcvbuf(sock->fd); + if (retval < 0) { + retval = -retval; + goto error; + } + sock->rcvbuf = retval; + retval = 0; + + /* Connect to kernel (pid 0) as remote address. */ + memset(&remote, 0, sizeof remote); + remote.nl_family = AF_NETLINK; + remote.nl_pid = 0; + if (connect(sock->fd, (struct sockaddr *) &remote, sizeof remote) < 0) { + VLOG_ERR("connect(0): %s", ovs_strerror(errno)); + goto error; + } + + /* Obtain pid assigned by kernel. */ + local_size = sizeof local; + if (getsockname(sock->fd, (struct sockaddr *) &local, &local_size) < 0) { + VLOG_ERR("getsockname: %s", ovs_strerror(errno)); + goto error; + } + if (local_size < sizeof local || local.nl_family != AF_NETLINK) { + VLOG_ERR("getsockname returned bad Netlink name"); + retval = EINVAL; + goto error; + } + sock->pid = local.nl_pid; +#endif + + *sockp = sock; + return 0; + +error: + if (retval == 0) { + retval = errno; + if (retval == 0) { + retval = EINVAL; + } + } +#ifdef _WIN32 + if (sock->overlapped.hEvent) { + CloseHandle(sock->overlapped.hEvent); + } + if (sock->handle != INVALID_HANDLE_VALUE) { + CloseHandle(sock->handle); + } +#else + if (sock->fd >= 0) { + close(sock->fd); + } +#endif + free(sock); + return retval; +} + +int +nl_transact_nopool(int protocol, const struct ofpbuf *request, + struct ofpbuf **replyp) +{ + struct nl_sock *sock; + int error; + + error = nl_sock_create_nopool(protocol, &sock); + if (error) { + if (replyp) { + *replyp = NULL; + } + return error; + } + + error = nl_sock_transact(sock, request, replyp); + nl_sock_destroy(sock); + + return error; +} + /* Sends the 'request' member of the 'n' transactions in 'transactions' on a * Netlink socket for the given 'protocol' (e.g. NETLINK_ROUTE or * NETLINK_GENERIC), in order, and receives responses to all of them. Fills in diff --git a/lib/netlink-socket.h b/lib/netlink-socket.h index 7852ad0..08cbab8 100644 --- a/lib/netlink-socket.h +++ b/lib/netlink-socket.h @@ -254,6 +254,8 @@ struct nl_transaction { int nl_transact(int protocol, const struct ofpbuf *request, struct ofpbuf **replyp); void nl_transact_multiple(int protocol, struct nl_transaction **, size_t n); +int nl_transact_nopool(int protocol, const struct ofpbuf *request, + struct ofpbuf **replyp); /* Table dumping. */ #define NL_DUMP_BUFSIZE 4096