From patchwork Mon Nov 11 14:02:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eelco Chaudron X-Patchwork-Id: 1192944 X-Patchwork-Delegate: i.maximets@samsung.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="YWYrtlK2"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 47BXdS0yfgz9sPV for ; Tue, 12 Nov 2019 01:02:52 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 28471CC1; Mon, 11 Nov 2019 14:02:48 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id D9CAFCAD for ; Mon, 11 Nov 2019 14:02:46 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by smtp1.linuxfoundation.org (Postfix) with ESMTP id AAE928B4 for ; Mon, 11 Nov 2019 14:02:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573480964; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=zzGleomroT8sn0Kmv9CWDam44I9jdR0WlAnscq1xyW0=; b=YWYrtlK25irLzIGBFdtiaEB6RL5Q7FpXmBl6TgsZBY6u6vWJ/MG/QLAVK/lpgTbB6Efwrl Ro/SpSoYOdReZrPM8kqv81/LOO/R/MR1srm9xieZqkHlG62Nnhi5Wh35X6i0FXCrzteadP MnG0jhYUygfoN7HBRhDfQrgzPQ5/iEk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-233-0dArMgChMCS09PSJ485gww-1; Mon, 11 Nov 2019 09:02:41 -0500 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A52A9806056; Mon, 11 Nov 2019 14:02:39 +0000 (UTC) Received: from netdev64.ntdv.lab.eng.bos.redhat.com (wsfd-netdev64.ntdv.lab.eng.bos.redhat.com [10.19.188.127]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4FDE45D72C; Mon, 11 Nov 2019 14:02:39 +0000 (UTC) From: Eelco Chaudron To: dev@openvswitch.org Date: Mon, 11 Nov 2019 09:02:37 -0500 Message-Id: <20191111140205.115245.76451.stgit@netdev64> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-MC-Unique: 0dArMgChMCS09PSJ485gww-1 X-Mimecast-Spam-Score: 0 X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: i.maximets@ovn.org Subject: [ovs-dev] [PATCH v6] netdev-dpdk: add support for the RTE_ETH_EVENT_INTR_RESET event X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org Currently, OVS does not register and therefore not handle the interface reset event from the DPDK framework. This would cause a problem in cases where a VF is used as an interface, and its configuration changes. As an example in the following scenario the MAC change is not detected/acted upon until OVS is restarted without the patch applied: $ echo 1 > /sys/bus/pci/devices/0000:05:00.1/sriov_numvfs $ ovs-vsctl add-port ovs_pvp_br0 dpdk0 -- \ set Interface dpdk0 type=dpdk -- \ set Interface dpdk0 options:dpdk-devargs=0000:05:0a.0 $ ip link set p5p2 vf 0 mac 52:54:00:92:d3:33 Requires patch "bridge: Allow manual notifications about interfaces' updates." Signed-off-by: Eelco Chaudron --- v5 -> v6: Moved the if_notifier_manual_report() function till after the actual reset of the interface to make it more clear that it's supposed to happen AFTER the reset. Removed empty line in the code v4 -> v5: Using new if_notifier_manual_report() API v3 -> v4: Add support for if-notification to make sure set_config() gets called v2 -> v3: v1 -> v2: Fixed Ilya's comments lib/netdev-dpdk.c | 54 +++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 52 insertions(+), 2 deletions(-) diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 4805783..cb57b96 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -46,6 +46,7 @@ #include "dpdk.h" #include "dpif-netdev.h" #include "fatal-signal.h" +#include "if-notifier.h" #include "netdev-provider.h" #include "netdev-vport.h" #include "odp-util.h" @@ -396,6 +397,8 @@ struct netdev_dpdk { bool attached; /* If true, rte_eth_dev_start() was successfully called */ bool started; + bool reset_needed; + /* 1 pad byte here. */ struct eth_addr hwaddr; int mtu; int socket_id; @@ -1173,6 +1176,8 @@ common_construct(struct netdev *netdev, dpdk_port_t port_no, ovsrcu_index_init(&dev->vid, -1); dev->vhost_reconfigured = false; dev->attached = false; + dev->started = false; + dev->reset_needed = false; ovsrcu_init(&dev->qos_conf, NULL); @@ -1769,6 +1774,34 @@ netdev_dpdk_process_devargs(struct netdev_dpdk *dev, return new_port_id; } +static int +dpdk_eth_event_callback(dpdk_port_t port_id, enum rte_eth_event_type type, + void *param OVS_UNUSED, void *ret_param OVS_UNUSED) +{ + struct netdev_dpdk *dev; + + switch ((int) type) { + case RTE_ETH_EVENT_INTR_RESET: + ovs_mutex_lock(&dpdk_mutex); + dev = netdev_dpdk_lookup_by_port_id(port_id); + if (dev) { + ovs_mutex_lock(&dev->mutex); + dev->reset_needed = true; + netdev_request_reconfigure(&dev->up); + VLOG_DBG_RL(&rl, "%s: Device reset requested.", + netdev_get_name(&dev->up)); + ovs_mutex_unlock(&dev->mutex); + } + ovs_mutex_unlock(&dpdk_mutex); + break; + + default: + /* Ignore all other types. */ + break; + } + return 0; +} + static void dpdk_set_rxq_config(struct netdev_dpdk *dev, const struct smap *args) OVS_REQUIRES(dev->mutex) @@ -3807,6 +3840,8 @@ netdev_dpdk_class_init(void) /* This function can be called for different classes. The initialization * needs to be done only once */ if (ovsthread_once_start(&once)) { + int ret; + ovs_thread_create("dpdk_watchdog", dpdk_watchdog, NULL); unixctl_command_register("netdev-dpdk/set-admin-state", "[netdev] up|down", 1, 2, @@ -3820,6 +3855,14 @@ netdev_dpdk_class_init(void) "[netdev]", 0, 1, netdev_dpdk_get_mempool_info, NULL); + ret = rte_eth_dev_callback_register(RTE_ETH_ALL, + RTE_ETH_EVENT_INTR_RESET, + dpdk_eth_event_callback, NULL); + if (ret != 0) { + VLOG_ERR("Ethernet device callback register error: %s", + rte_strerror(-ret)); + } + ovsthread_once_done(&once); } @@ -4180,13 +4223,20 @@ netdev_dpdk_reconfigure(struct netdev *netdev) && dev->rxq_size == dev->requested_rxq_size && dev->txq_size == dev->requested_txq_size && dev->socket_id == dev->requested_socket_id - && dev->started) { + && dev->started && !dev->reset_needed) { /* Reconfiguration is unnecessary */ goto out; } - rte_eth_dev_stop(dev->port_id); + if (dev->reset_needed) { + rte_eth_dev_reset(dev->port_id); + if_notifier_manual_report(); + dev->reset_needed = false; + } else { + rte_eth_dev_stop(dev->port_id); + } + dev->started = false; err = netdev_dpdk_mempool_configure(dev);