From patchwork Wed Jul 5 12:27:11 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shachar Beiser X-Patchwork-Id: 784579 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3x2gHD5N7Nz9s7g for ; Wed, 5 Jul 2017 22:32:08 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id A33B9B35; Wed, 5 Jul 2017 12:28:26 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 54229AB5 for ; Wed, 5 Jul 2017 12:28:20 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by smtp1.linuxfoundation.org (Postfix) with ESMTP id 1FB4B3ED for ; Wed, 5 Jul 2017 12:28:17 +0000 (UTC) Received: from Internal Mail-Server by MTLPINE1 (envelope-from shacharbe@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Jul 2017 15:27:49 +0300 Received: from r-aa-dragon21.mtr.labs.mlnx (r-aa-dragon21.mtr.labs.mlnx [10.209.68.158]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id v65CRnqB009609; Wed, 5 Jul 2017 15:27:49 +0300 Received: from r-aa-dragon21.mtr.labs.mlnx (localhost [127.0.0.1]) by r-aa-dragon21.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id v65CRnMj025920; Wed, 5 Jul 2017 12:27:49 GMT Received: (from shacharbe@localhost) by r-aa-dragon21.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id v65CRnXu025919; Wed, 5 Jul 2017 12:27:49 GMT From: Shachar Beiser To: ovs-dev@openvswitch.org Date: Wed, 5 Jul 2017 12:27:11 +0000 Message-Id: X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Shachar Beiser Subject: [ovs-dev] [PATCH 04/11] ovs/dp-cls: initializing HW pipeline X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org The HW pipeline is made of 3 entites: dp-cls thread, a pool of flow tags and a message queue between the pmd context and dp-cls offload thread. This patch initializes those 3 entities Signed-off-by: Shachar Beiser --- lib/automake.mk | 1 + lib/dpif-netdev.c | 1 + lib/hw-pipeline.c | 183 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ lib/hw-pipeline.h | 3 + 4 files changed, 188 insertions(+) create mode 100644 lib/hw-pipeline.c diff --git a/lib/automake.mk b/lib/automake.mk index fa27aeb..f962a22 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -379,6 +379,7 @@ if DPDK_NETDEV lib_libopenvswitch_la_SOURCES += \ lib/dpdk.c \ lib/netdev-dpdk.c \ + lib/hw-pipeline.c \ lib/hw-pipeline.h else lib_libopenvswitch_la_SOURCES += \ diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 7a8739d..ef3083b 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -1015,6 +1015,7 @@ create_dp_netdev(const char *name, const struct dpif_class *class, ovs_mutex_lock(&dp->port_mutex); dp_netdev_set_nonpmd(dp); + hw_pipeline_init(dp); error = do_add_port(dp, name, dpif_netdev_port_open_type(dp->class, "internal"), diff --git a/lib/hw-pipeline.c b/lib/hw-pipeline.c new file mode 100644 index 0000000..1720c12 --- /dev/null +++ b/lib/hw-pipeline.c @@ -0,0 +1,183 @@ +/* + * Copyright (c) 2009, 2010, 2011, 2012, 2013, 2014, 2016 Nicira, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "dpif-netdev.h" +#include "include/openvswitch/vlog.h" +#include "hw-pipeline.h" + +VLOG_DEFINE_THIS_MODULE(hw_pipeline); + +// Internal functions Flow Tags Pool + +uint32_t hw_pipeline_ft_pool_init(flow_tag_pool *p,uint32_t pool_size); +// Internal functions Message Queue + +static int hw_pipeline_msg_queue_init(msg_queue *message_queue, + unsigned core_id); + +void *hw_pipeline_thread(void *pdp); + +uint32_t hw_pipeline_ft_pool_init(flow_tag_pool *p, + uint32_t pool_size) +{ + uint32_t ii=0; + + if (OVS_UNLIKELY( p == NULL )) { + VLOG_ERR("pool size is too big or pool is NULL \n"); + return -1; + } + if (OVS_UNLIKELY( pool_size > HW_MAX_FLOW_TAG )) { + pool_size = HW_MAX_FLOW_TAG; + } + p->ft_data = (flow_elem *)xmalloc(pool_size * sizeof(flow_elem)); + if (OVS_UNLIKELY( p->ft_data == NULL )) { + VLOG_ERR("No free memory for the pool \n"); + return -1; + } + memset(p->ft_data,0,(pool_size * sizeof(flow_elem))); + rte_spinlock_init(&p->lock); + rte_spinlock_lock(&p->lock); + p->head=0; + p->tail=0; + p->pool_size = pool_size; + for (ii=0;iift_data[ii].next = ii+1; + rte_spinlock_init(&p->ft_data[ii].lock); + } + p->ft_data[pool_size-1].next = HW_NO_FREE_FLOW_TAG; + rte_spinlock_unlock(&p->lock); + return 0; +} + +/*************************************************************************/ +// Msg Queue +// A queue that contains pairs : (flow , key ) +// The queue is used a communication channel between pmd_thread_main & +// hw_pipeline_thread . +// The hw_pipeline_thread dequeue (key,flow ) from the msg queue +// & calls emc_hw_insert that inserts classifier rules +// to hardware flow tables. +// The pmd_thread_main enqueue (key,flow) into the msg qeueue and continues. +/*************************************************************************/ +static int hw_pipeline_msg_queue_init(msg_queue *message_queue, + unsigned core_id) +{ + int ret; + const char dir[] = "/tmp"; + const char fifo[] = "/tmp/msgq_pipe"; + char fifo_pmd[20]; + + sprintf(fifo_pmd,"%s%d",fifo,core_id); + message_queue->tv.tv_sec = 0; + message_queue->tv.tv_usec = HW_PIPELINE_MSGQ_TO; + ovs_strlcpy(message_queue->pipeName,fifo_pmd,strlen(fifo_pmd)); + if (mkdir(dir, 0755) == -1 && errno != EEXIST) { + VLOG_ERR("Failed to create directory: "); + return -1; + } + ret = mkfifo(fifo_pmd,0666); + if (OVS_UNLIKELY(ret < 0)) { + if (errno==EEXIST) { + ret = unlink(fifo_pmd); + if (OVS_UNLIKELY(ret < 0)) { + VLOG_ERR("Remove fifo failed .\n"); + return -1; + } + ret = mkfifo(fifo_pmd,0666 ); + if (OVS_UNLIKELY(ret < 0)) { + if (errno==EEXIST) { + VLOG_ERR("That file already exists.\n"); + VLOG_ERR("(or we passed in a symbolic link,"); + VLOG_ERR(" which we did not.)\n"); + return -1; + } + } + } + else if (errno==EROFS) { + VLOG_ERR("The name file resides on a read-only file-system\n"); + return -1; + } + else { + VLOG_ERR("mkfifo failed %x \n",errno); + return -1; + } + } + message_queue->readFd = open(message_queue->pipeName, + O_RDONLY|O_NONBLOCK); + if (OVS_UNLIKELY(message_queue->readFd == -1)) { + VLOG_ERR("Error creating read file descriptor"); + return -1; + } + message_queue->writeFd = open(message_queue->pipeName, + O_WRONLY|O_NONBLOCK); + if (OVS_UNLIKELY(message_queue->writeFd == -1)) { + VLOG_ERR("Error creating write file descriptor"); + return -1; + } + return 0; +} + +void *hw_pipeline_thread(void *pdp) +{ + struct dp_netdev *dp= (struct dp_netdev *)pdp; + ovsrcu_quiesce_start(); + if (dp->ppl_md.id == HW_OFFLOAD_PIPELINE) { + VLOG_INFO(" HW_OFFLOAD_PIPELINE is set \n"); + } + else { + VLOG_INFO(" HW_OFFLOAD_PIPELINE is off \n"); + } + while (1) { + } + ovsrcu_quiesce_end(); + return NULL; +} +int hw_pipeline_init(struct dp_netdev *dp) +{ + int ret=0; + static uint32_t id=0; + + VLOG_INFO("hw_pipeline_init\n"); + ret = hw_pipeline_ft_pool_init(&dp->ft_pool,HW_MAX_FLOW_TAG); + if (OVS_UNLIKELY(ret != 0)) { + VLOG_ERR(" hw_pipeline_ft_pool_init failed \n"); + return ret; + } + ret = hw_pipeline_msg_queue_init(&dp->message_queue,id++); + if (OVS_UNLIKELY(ret != 0)) { + VLOG_ERR(" hw_pipeline_msg_queue_init failed \n"); + return ret; + } + dp->thread_ofload = ovs_thread_create("ft_offload",hw_pipeline_thread,dp); + dp->ppl_md.id = HW_OFFLOAD_PIPELINE; + return 0; +} diff --git a/lib/hw-pipeline.h b/lib/hw-pipeline.h index 4dcafa2..28b734e 100644 --- a/lib/hw-pipeline.h +++ b/lib/hw-pipeline.h @@ -17,6 +17,9 @@ #include "dpif-netdev.h" #define HW_NO_FREE_FLOW_TAG 0xffffffff +#define HW_PIPELINE_MSGQ_TO 10000 +#define HW_MAX_FLOW_TAG 65536 +#define MSG_QUEUE_MAX_SIZE 65536 enum pipeline_id { DEFAULT_SW_PIPELINE = 0,