From patchwork Wed Nov 6 13:43:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eelco Chaudron X-Patchwork-Id: 1190431 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="SNs5DCuE"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 477SRL2d7Wz9sPv for ; Thu, 7 Nov 2019 00:43:26 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 86157BA9; Wed, 6 Nov 2019 13:43:24 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id AE2C3B88 for ; Wed, 6 Nov 2019 13:43:23 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from us-smtp-delivery-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by smtp1.linuxfoundation.org (Postfix) with ESMTP id 14BF8844 for ; Wed, 6 Nov 2019 13:43:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573047802; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=uF4FvS0ZoHYrxb1oXoI2IbBqEs0ao4ZwTE23dVpbkXI=; b=SNs5DCuE/mF6tAD2fUVVxZyEIoFZ6piVTeBZgDBWVe5P2AQQRfCi23e1qfBszQFPPLppIs fout4rQ14nlq1ojTuai+oMvblzMHrL3SIuQ4q1PaM1x4agwNDA3I5VsbtwNE5M3Nc26o2u Rm0GByIcbVFeEWeb959eAs/Zylz4+JM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-348-hTAlIgTEOr-hHkgef2dqiQ-1; Wed, 06 Nov 2019 08:43:18 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5DA8A8017E0; Wed, 6 Nov 2019 13:43:17 +0000 (UTC) Received: from netdev64.ntdv.lab.eng.bos.redhat.com (wsfd-netdev64.ntdv.lab.eng.bos.redhat.com [10.19.188.127]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0801410018FF; Wed, 6 Nov 2019 13:43:16 +0000 (UTC) From: Eelco Chaudron To: dev@openvswitch.org Date: Wed, 6 Nov 2019 08:43:11 -0500 Message-Id: <20191106134245.165674.92584.stgit@netdev64> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: hTAlIgTEOr-hHkgef2dqiQ-1 X-Mimecast-Spam-Score: 0 X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: i.maximets@samsung.com Subject: [ovs-dev] [PATCH v5] netdev-dpdk: add support for the RTE_ETH_EVENT_INTR_RESET event X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org Currently, OVS does not register and therefore not handle the interface reset event from the DPDK framework. This would cause a problem in cases where a VF is used as an interface, and its configuration changes. As an example in the following scenario the MAC change is not detected/acted upon until OVS is restarted without the patch applied: $ echo 1 > /sys/bus/pci/devices/0000:05:00.1/sriov_numvfs $ ovs-vsctl add-port ovs_pvp_br0 dpdk0 -- \ set Interface dpdk0 type=dpdk -- \ set Interface dpdk0 options:dpdk-devargs=0000:05:0a.0 $ ip link set p5p2 vf 0 mac 52:54:00:92:d3:33 Requires patch "bridge: Allow manual notifications about interfaces' updates." Signed-off-by: Eelco Chaudron --- v4 -> v5: Using new if_notifier_manual_report() API v3 -> v4: Add support for if-notification to make sure set_config() gets called v2 -> v3: v1 -> v2: Fixed Ilya's comments lib/netdev-dpdk.c | 56 +++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 54 insertions(+), 2 deletions(-) diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 4805783..b744589 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -46,6 +46,7 @@ #include "dpdk.h" #include "dpif-netdev.h" #include "fatal-signal.h" +#include "if-notifier.h" #include "netdev-provider.h" #include "netdev-vport.h" #include "odp-util.h" @@ -396,6 +397,8 @@ struct netdev_dpdk { bool attached; /* If true, rte_eth_dev_start() was successfully called */ bool started; + bool reset_needed; + /* 1 pad byte here. */ struct eth_addr hwaddr; int mtu; int socket_id; @@ -1173,6 +1176,8 @@ common_construct(struct netdev *netdev, dpdk_port_t port_no, ovsrcu_index_init(&dev->vid, -1); dev->vhost_reconfigured = false; dev->attached = false; + dev->started = false; + dev->reset_needed = false; ovsrcu_init(&dev->qos_conf, NULL); @@ -1769,6 +1774,36 @@ netdev_dpdk_process_devargs(struct netdev_dpdk *dev, return new_port_id; } +static int +dpdk_eth_event_callback(dpdk_port_t port_id, enum rte_eth_event_type type, + void *param OVS_UNUSED, void *ret_param OVS_UNUSED) +{ + struct netdev_dpdk *dev; + + switch ((int) type) { + case RTE_ETH_EVENT_INTR_RESET: + ovs_mutex_lock(&dpdk_mutex); + dev = netdev_dpdk_lookup_by_port_id(port_id); + if (dev) { + ovs_mutex_lock(&dev->mutex); + dev->reset_needed = true; + netdev_request_reconfigure(&dev->up); + VLOG_DBG_RL(&rl, "%s: Device reset requested.", + netdev_get_name(&dev->up)); + ovs_mutex_unlock(&dev->mutex); + } + ovs_mutex_unlock(&dpdk_mutex); + + if_notifier_manual_report(); + break; + + default: + /* Ignore all other types. */ + break; + } + return 0; +} + static void dpdk_set_rxq_config(struct netdev_dpdk *dev, const struct smap *args) OVS_REQUIRES(dev->mutex) @@ -3807,6 +3842,8 @@ netdev_dpdk_class_init(void) /* This function can be called for different classes. The initialization * needs to be done only once */ if (ovsthread_once_start(&once)) { + int ret; + ovs_thread_create("dpdk_watchdog", dpdk_watchdog, NULL); unixctl_command_register("netdev-dpdk/set-admin-state", "[netdev] up|down", 1, 2, @@ -3820,6 +3857,15 @@ netdev_dpdk_class_init(void) "[netdev]", 0, 1, netdev_dpdk_get_mempool_info, NULL); + ret = rte_eth_dev_callback_register(RTE_ETH_ALL, + RTE_ETH_EVENT_INTR_RESET, + dpdk_eth_event_callback, NULL); + + if (ret != 0) { + VLOG_ERR("Ethernet device callback register error: %s", + rte_strerror(-ret)); + } + ovsthread_once_done(&once); } @@ -4180,13 +4226,19 @@ netdev_dpdk_reconfigure(struct netdev *netdev) && dev->rxq_size == dev->requested_rxq_size && dev->txq_size == dev->requested_txq_size && dev->socket_id == dev->requested_socket_id - && dev->started) { + && dev->started && !dev->reset_needed) { /* Reconfiguration is unnecessary */ goto out; } - rte_eth_dev_stop(dev->port_id); + if (dev->reset_needed) { + rte_eth_dev_reset(dev->port_id); + dev->reset_needed = false; + } else { + rte_eth_dev_stop(dev->port_id); + } + dev->started = false; err = netdev_dpdk_mempool_configure(dev);