From patchwork Sat Mar 14 10:03:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tonghao Zhang X-Patchwork-Id: 1254830 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=GlHHbsv9; dkim-atps=neutral Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 48fdSf60Y2z9sP7 for ; Sat, 14 Mar 2020 21:04:02 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 8CF16876B0; Sat, 14 Mar 2020 10:04:00 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nJ-q8bbulsXb; Sat, 14 Mar 2020 10:03:59 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by fraxinus.osuosl.org (Postfix) with ESMTP id 7DBE88764C; Sat, 14 Mar 2020 10:03:59 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 64B19C18D3; Sat, 14 Mar 2020 10:03:59 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0932FC0177 for ; Sat, 14 Mar 2020 10:03:58 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 05F96876B0 for ; Sat, 14 Mar 2020 10:03:58 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xi2T2nFK7kUz for ; Sat, 14 Mar 2020 10:03:56 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pg1-f194.google.com (mail-pg1-f194.google.com [209.85.215.194]) by fraxinus.osuosl.org (Postfix) with ESMTPS id DDB8A8764C for ; Sat, 14 Mar 2020 10:03:56 +0000 (UTC) Received: by mail-pg1-f194.google.com with SMTP id z4so748102pgu.3 for ; Sat, 14 Mar 2020 03:03:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=OJS/aJ+dBM80P6txwpXv8nqFWlwVutCrS71oISE9V20=; b=GlHHbsv9XgjpJYo9z0Foj8RGuaSnWJnUfxnSp/2U9Ty61SJu2nXHiwcaoVnP6Bb/zK 00ammU9ghpcmS91KfpH3PWW5rpXrUhob7Y9bH6H6a+K+sgQ8UoY0Rl/AaBYT1K5Bjj9P U9yoi5HGouBl6In9XOJVb99fXrquGg7u3r+GytHcx3UP+t1dm+JTJsg3v3MWkeEpjB2C hY9FseUwnpgJLBRjOSP2nJ/GHuzlBLyVWq8WRtM5fEq9TqaW113t5LYwtVcM5zPn1it6 sk+jT7FGbsNcQ9We0pcBEwFzT9TKfkiSSSB/72M3pcjkaRqC81trCOHY0+CUOzdpRG3p Es+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=OJS/aJ+dBM80P6txwpXv8nqFWlwVutCrS71oISE9V20=; b=LgY6GY/G6kq2+9pP0hfuTVIbIhQyjeJED7v5E3nuhXc3wnyYu4dQjPNZKv/vnmSpwj KGLOFFW0pnBBw3j5JeTMcEiNohYjTGUYXFr70w6B/XAiKYnEICBMWG6YF7Xm2p9LD1xu qP+rsau96uCEHpa9TzODEWOblPzeBtKIG41NA8YIHhBvOUMnhsh+CzX8k2VZ6xqvx7ZF abeeiauOo/U62YYJhTkGeztemR1OqKzRlzgZ+dIvV+fEGXPpftI2C28qUahULOcwjuxg 5Aq8ihWJo4cescl2EPvRIeERAVdjm8RcxcG/4R3tp8QcrRFW07oR1hggjmc30TqfwHCA ah1Q== X-Gm-Message-State: ANhLgQ0Qs8O9yZCDtm0MIOpTe7ezqYD5PdH8fWyLnZsl6o8riuXJSp+D w+FJk+rViYFa1oxFVdXi2FFbjiYn X-Google-Smtp-Source: ADFU+vsdtnzA62dVmcLkAhTwx7TisgoEeJ7rv+n47N7PU01alD3mwbhJIMRUuDkcAh0O+lkVadnqcw== X-Received: by 2002:a63:e551:: with SMTP id z17mr7460490pgj.21.1584180236021; Sat, 14 Mar 2020 03:03:56 -0700 (PDT) Received: from local.opencloud.tech.localdomain ([219.143.129.147]) by smtp.gmail.com with ESMTPSA id c207sm20809671pfb.47.2020.03.14.03.03.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 14 Mar 2020 03:03:55 -0700 (PDT) From: xiangxia.m.yue@gmail.com To: dev@openvswitch.org Date: Sat, 14 Mar 2020 18:03:49 +0800 Message-Id: <1584180230-89020-1-git-send-email-xiangxia.m.yue@gmail.com> X-Mailer: git-send-email 1.8.3.1 Cc: Ilya Maximets Subject: [ovs-dev] [PATCH ovs 1/2] dpif-netdev: Expand the meter capacity using cmap X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Tonghao Zhang For now, ovs-vswitchd use the array of the dp_meter struct to store meter's data, and at most, there are only 65536 (defined by MAX_METERS) meters that can be used. But in some case, for example, in the edge gateway, we should use 200,000+, at least, meters for IP address bandwidth limitation. Every one IP address will use two meters for its rx and tx path[1]. In other way, ovs-vswitchd should support meter-offload (rte_mtr_xxx api introduced by dpdk.), but there are more than 65536 meters in the hardware, such as Mellanox ConnectX-6. This patch use cmap to manage the meter, instead of the array. * Insertion performance, ovs-ofctl add-meter 1000+ meters, the cmap takes abount 4000ms, as same as previous implementation. * Lookup performance in datapath, we add 1000+ meter which rate is 10G (the NIC cards are 10Gb, so netdev-datapath will not drop the packets.), and a flow which only forwarding the packets from p0 to p1, with meter action[2]. On other server, the pktgen-dpdk will generate 64B packets to p0. The forwarding performance is 4,814,400 pps. Without this path, 4,935,584 pps. There are about 1% performance loss. For addressing this issue, next patch add a meter cache. [1]. $ in_port=p0,ip,ip_dst=1.1.1.x action=meter:n,output:p1 $ in_port=p1,ip,ip_src=1.1.1.x action=meter:m,output:p0 [2]. $ in_port=p0 action=meter:100,output:p1 Cc: Ben Pfaff Cc: Jarno Rajahalme Cc: Ilya Maximets Cc: Andy Zhou Signed-off-by: Tonghao Zhang --- lib/dpif-netdev.c | 199 +++++++++++++++++++++++++++++++++++------------------- 1 file changed, 130 insertions(+), 69 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index d393aab..5474d52 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -98,9 +98,11 @@ DEFINE_STATIC_PER_THREAD_DATA(uint32_t, recirc_depth, 0) /* Configuration parameters. */ enum { MAX_FLOWS = 65536 }; /* Maximum number of flows in flow table. */ -enum { MAX_METERS = 65536 }; /* Maximum number of meters. */ -enum { MAX_BANDS = 8 }; /* Maximum number of bands / meter. */ -enum { N_METER_LOCKS = 64 }; /* Maximum number of meters. */ + +/* Maximum number of meters in the table. */ +#define METER_ENTRY_MAX (1 << 19) +/* Maximum number of bands / meter. */ +#define METER_BAND_MAX (8) COVERAGE_DEFINE(datapath_drop_meter); COVERAGE_DEFINE(datapath_drop_upcall_error); @@ -280,6 +282,9 @@ struct dp_meter_band { }; struct dp_meter { + struct cmap_node node; + struct ovs_mutex lock; + uint32_t id; uint16_t flags; uint16_t n_bands; uint32_t max_delta_t; @@ -289,6 +294,12 @@ struct dp_meter { struct dp_meter_band bands[]; }; +struct dp_netdev_meter { + struct cmap table OVS_GUARDED; + struct ovs_mutex lock; /* Used for meter table. */ + uint32_t hash_basis; +}; + struct pmd_auto_lb { bool auto_lb_requested; /* Auto load balancing requested by user. */ bool is_enabled; /* Current status of Auto load balancing. */ @@ -329,8 +340,7 @@ struct dp_netdev { atomic_uint32_t tx_flush_interval; /* Meters. */ - struct ovs_mutex meter_locks[N_METER_LOCKS]; - struct dp_meter *meters[MAX_METERS]; /* Meter bands. */ + struct dp_netdev_meter *meter; /* Probability of EMC insertions is a factor of 'emc_insert_min'.*/ OVS_ALIGNED_VAR(CACHE_LINE_SIZE) atomic_uint32_t emc_insert_min; @@ -378,19 +388,6 @@ struct dp_netdev { struct pmd_auto_lb pmd_alb; }; -static void meter_lock(const struct dp_netdev *dp, uint32_t meter_id) - OVS_ACQUIRES(dp->meter_locks[meter_id % N_METER_LOCKS]) -{ - ovs_mutex_lock(&dp->meter_locks[meter_id % N_METER_LOCKS]); -} - -static void meter_unlock(const struct dp_netdev *dp, uint32_t meter_id) - OVS_RELEASES(dp->meter_locks[meter_id % N_METER_LOCKS]) -{ - ovs_mutex_unlock(&dp->meter_locks[meter_id % N_METER_LOCKS]); -} - - static struct dp_netdev_port *dp_netdev_lookup_port(const struct dp_netdev *dp, odp_port_t) OVS_REQUIRES(dp->port_mutex); @@ -1523,6 +1520,84 @@ choose_port(struct dp_netdev *dp, const char *name) return ODPP_NONE; } +static uint32_t +dp_meter_hash(uint32_t meter_id, uint32_t basis) +{ + return hash_bytes32(&meter_id, sizeof meter_id, basis); +} + +static void +dp_netdev_meter_init(struct dp_netdev *dp) +{ + struct dp_netdev_meter *dp_meter; + + dp_meter = xmalloc(sizeof *dp_meter); + + cmap_init(&dp_meter->table); + ovs_mutex_init(&dp_meter->lock); + dp_meter->hash_basis = random_uint32(); + + dp->meter = dp_meter; +} + +static void +dp_netdev_meter_destroy(struct dp_netdev *dp) +{ + struct dp_netdev_meter *dp_meter = dp->meter; + struct dp_meter *meter; + + ovs_mutex_lock(&dp_meter->lock); + CMAP_FOR_EACH (meter, node, &dp_meter->table) { + cmap_remove(&dp_meter->table, &meter->node, + dp_meter_hash(meter->id, dp_meter->hash_basis)); + + ovsrcu_postpone(free, meter); + } + + cmap_destroy(&dp_meter->table); + ovs_mutex_unlock(&dp_meter->lock); + + ovs_mutex_destroy(&dp_meter->lock); + free(dp_meter); +} + +static struct dp_meter* +dp_meter_lookup(struct dp_netdev_meter *dp_meter, uint32_t meter_id) +{ + uint32_t hash = dp_meter_hash(meter_id, dp_meter->hash_basis); + struct dp_meter *meter; + + CMAP_FOR_EACH_WITH_HASH (meter, node, hash, &dp_meter->table) { + if (meter->id == meter_id) { + return meter; + } + } + + return NULL; +} + +static void +dp_meter_detach_free(struct dp_netdev_meter *dp_meter, uint32_t meter_id) + OVS_REQUIRES(dp_meter->lock) +{ + struct dp_meter *meter; + + meter = dp_meter_lookup(dp_meter, meter_id); + if (meter) { + cmap_remove(&dp_meter->table, &meter->node, + dp_meter_hash(meter_id, dp_meter->hash_basis)); + ovsrcu_postpone(free, meter); + } +} + +static void +dp_meter_attach(struct dp_netdev_meter *dp_meter, struct dp_meter *meter) + OVS_REQUIRES(dp_meter->lock) +{ + cmap_insert(&dp_meter->table, &meter->node, + dp_meter_hash(meter->id, dp_meter->hash_basis)); +} + static int create_dp_netdev(const char *name, const struct dpif_class *class, struct dp_netdev **dpp) @@ -1556,9 +1631,8 @@ create_dp_netdev(const char *name, const struct dpif_class *class, dp->reconfigure_seq = seq_create(); dp->last_reconfigure_seq = seq_read(dp->reconfigure_seq); - for (int i = 0; i < N_METER_LOCKS; ++i) { - ovs_mutex_init_adaptive(&dp->meter_locks[i]); - } + /* Init meter resource. */ + dp_netdev_meter_init(dp); /* Disable upcalls by default. */ dp_netdev_disable_upcall(dp); @@ -1647,16 +1721,6 @@ dp_netdev_destroy_upcall_lock(struct dp_netdev *dp) fat_rwlock_destroy(&dp->upcall_rwlock); } -static void -dp_delete_meter(struct dp_netdev *dp, uint32_t meter_id) - OVS_REQUIRES(dp->meter_locks[meter_id % N_METER_LOCKS]) -{ - if (dp->meters[meter_id]) { - free(dp->meters[meter_id]); - dp->meters[meter_id] = NULL; - } -} - /* Requires dp_netdev_mutex so that we can't get a new reference to 'dp' * through the 'dp_netdevs' shash while freeing 'dp'. */ static void @@ -1694,16 +1758,7 @@ dp_netdev_free(struct dp_netdev *dp) /* Upcalls must be disabled at this point */ dp_netdev_destroy_upcall_lock(dp); - int i; - - for (i = 0; i < MAX_METERS; ++i) { - meter_lock(dp, i); - dp_delete_meter(dp, i); - meter_unlock(dp, i); - } - for (i = 0; i < N_METER_LOCKS; ++i) { - ovs_mutex_destroy(&dp->meter_locks[i]); - } + dp_netdev_meter_destroy(dp); free(dp->pmd_cmask); free(CONST_CAST(char *, dp->name)); @@ -5708,10 +5763,10 @@ static void dpif_netdev_meter_get_features(const struct dpif * dpif OVS_UNUSED, struct ofputil_meter_features *features) { - features->max_meters = MAX_METERS; + features->max_meters = METER_ENTRY_MAX; features->band_types = DP_SUPPORTED_METER_BAND_TYPES; features->capabilities = DP_SUPPORTED_METER_FLAGS_MASK; - features->max_bands = MAX_BANDS; + features->max_bands = METER_BAND_MAX; features->max_color = 0; } @@ -5732,14 +5787,13 @@ dp_netdev_run_meter(struct dp_netdev *dp, struct dp_packet_batch *packets_, uint32_t exceeded_rate[NETDEV_MAX_BURST]; int exceeded_pkt = cnt; /* First packet that exceeded a band rate. */ - if (meter_id >= MAX_METERS) { + if (meter_id >= METER_ENTRY_MAX) { return; } - meter_lock(dp, meter_id); - meter = dp->meters[meter_id]; + meter = dp_meter_lookup(dp->meter, meter_id); if (!meter) { - goto out; + return; } /* Initialize as negative values. */ @@ -5747,6 +5801,7 @@ dp_netdev_run_meter(struct dp_netdev *dp, struct dp_packet_batch *packets_, /* Initialize as zeroes. */ memset(exceeded_rate, 0, cnt * sizeof *exceeded_rate); + ovs_mutex_lock(&meter->lock); /* All packets will hit the meter at the same time. */ long_delta_t = now / 1000 - meter->used / 1000; /* msec */ @@ -5860,8 +5915,8 @@ dp_netdev_run_meter(struct dp_netdev *dp, struct dp_packet_batch *packets_, dp_packet_batch_refill(packets_, packet, j); } } - out: - meter_unlock(dp, meter_id); + + ovs_mutex_unlock(&meter->lock); } /* Meter set/get/del processing is still single-threaded. */ @@ -5870,11 +5925,12 @@ dpif_netdev_meter_set(struct dpif *dpif, ofproto_meter_id meter_id, struct ofputil_meter_config *config) { struct dp_netdev *dp = get_dp_netdev(dpif); + struct dp_netdev_meter *dp_meter = dp->meter; uint32_t mid = meter_id.uint32; struct dp_meter *meter; int i; - if (mid >= MAX_METERS) { + if (mid >= METER_ENTRY_MAX) { return EFBIG; /* Meter_id out of range. */ } @@ -5882,7 +5938,7 @@ dpif_netdev_meter_set(struct dpif *dpif, ofproto_meter_id meter_id, return EBADF; /* Unsupported flags set */ } - if (config->n_bands > MAX_BANDS) { + if (config->n_bands > METER_BAND_MAX) { return EINVAL; } @@ -5903,6 +5959,8 @@ dpif_netdev_meter_set(struct dpif *dpif, ofproto_meter_id meter_id, meter->n_bands = config->n_bands; meter->max_delta_t = 0; meter->used = time_usec(); + meter->id = mid; + ovs_mutex_init(&meter->lock); /* set up bands */ for (i = 0; i < config->n_bands; ++i) { @@ -5928,10 +5986,12 @@ dpif_netdev_meter_set(struct dpif *dpif, ofproto_meter_id meter_id, } } - meter_lock(dp, mid); - dp_delete_meter(dp, mid); /* Free existing meter, if any */ - dp->meters[mid] = meter; - meter_unlock(dp, mid); + ovs_mutex_lock(&dp_meter->lock); + + dp_meter_detach_free(dp_meter, mid); /* Free existing meter, if any */ + dp_meter_attach(dp_meter, meter); + + ovs_mutex_unlock(&dp_meter->lock); return 0; } @@ -5941,22 +6001,23 @@ dpif_netdev_meter_get(const struct dpif *dpif, ofproto_meter_id meter_id_, struct ofputil_meter_stats *stats, uint16_t n_bands) { - const struct dp_netdev *dp = get_dp_netdev(dpif); + struct dp_netdev *dp = get_dp_netdev(dpif); uint32_t meter_id = meter_id_.uint32; - int retval = 0; + const struct dp_meter *meter; - if (meter_id >= MAX_METERS) { + if (meter_id >= METER_ENTRY_MAX) { return EFBIG; } - meter_lock(dp, meter_id); - const struct dp_meter *meter = dp->meters[meter_id]; + meter = dp_meter_lookup(dp->meter, meter_id); if (!meter) { - retval = ENOENT; - goto done; + return ENOENT; } + if (stats) { - int i = 0; + int i; + + ovs_mutex_lock(&meter->lock); stats->packet_in_count = meter->packet_count; stats->byte_in_count = meter->byte_count; @@ -5966,12 +6027,11 @@ dpif_netdev_meter_get(const struct dpif *dpif, stats->bands[i].byte_count = meter->bands[i].byte_count; } + ovs_mutex_unlock(&meter->lock); stats->n_bands = i; } -done: - meter_unlock(dp, meter_id); - return retval; + return 0; } static int @@ -5980,15 +6040,16 @@ dpif_netdev_meter_del(struct dpif *dpif, struct ofputil_meter_stats *stats, uint16_t n_bands) { struct dp_netdev *dp = get_dp_netdev(dpif); + struct dp_netdev_meter *dp_meter = dp->meter; int error; error = dpif_netdev_meter_get(dpif, meter_id_, stats, n_bands); if (!error) { uint32_t meter_id = meter_id_.uint32; - meter_lock(dp, meter_id); - dp_delete_meter(dp, meter_id); - meter_unlock(dp, meter_id); + ovs_mutex_lock(&dp_meter->lock); + dp_meter_detach_free(dp_meter, meter_id); + ovs_mutex_unlock(&dp_meter->lock); } return error; } From patchwork Sat Mar 14 10:03:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tonghao Zhang X-Patchwork-Id: 1254831 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=jCgBkCxM; dkim-atps=neutral Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 48fdSl1zKbz9sP7 for ; Sat, 14 Mar 2020 21:04:06 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id BB41A8993F; Sat, 14 Mar 2020 10:04:04 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9+pb3cetxGy9; Sat, 14 Mar 2020 10:04:03 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by hemlock.osuosl.org (Postfix) with ESMTP id C3DC389960; Sat, 14 Mar 2020 10:04:03 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id A945EC1D8D; Sat, 14 Mar 2020 10:04:03 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id B7295C0177 for ; Sat, 14 Mar 2020 10:04:01 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id A374720509 for ; Sat, 14 Mar 2020 10:04:01 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9zeJMJak2j4G for ; Sat, 14 Mar 2020 10:04:00 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pj1-f65.google.com (mail-pj1-f65.google.com [209.85.216.65]) by silver.osuosl.org (Postfix) with ESMTPS id 1424E204D9 for ; Sat, 14 Mar 2020 10:03:59 +0000 (UTC) Received: by mail-pj1-f65.google.com with SMTP id l36so5430663pjb.3 for ; Sat, 14 Mar 2020 03:03:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Ljmf5PPRer3Rx5CW3QTn++wG0OxrHfaqxUiOatTOPM4=; b=jCgBkCxME6PlC8LI3CPvnFb8gY3gCCgivDh94xdatB8++rpPwoN+rEB0LSml55fRlo euHtQTYb8blrHRtwvYb0x8azZYbTPLwn+UyjXZa1l+RziRW+KOZkpgV8020xInqUlcKw G6lNync6rtBIw3WIL1UqPMXfrO7NQcqeLCucQOcJBIRnwXB6dLzRHjHSyVjxuUFUtFol V8904JxL0nobwxM5CLR6n7KcJlfEOp1piIS475E9Tx/e0xv4bug2ovOfAJaIQqh/8D8I yC1op3Jk16Gi9LnmoNXg5qsBg7ie1ZbV5lTVo4n6BsbYWLhmNdFucRBb5u7O2gnmZ2jU btHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Ljmf5PPRer3Rx5CW3QTn++wG0OxrHfaqxUiOatTOPM4=; b=nUhU5uIXnUJ7ZiW4BX1cif0scdH+HsIMTuhISg/YpZJnqDP10UTXX02zHKYBX0qJjI pPKiIEqfg7plvHOEHXt6UvnLKAVAc2Zk/fyII2GzDLmPW+DKuWwk2JePXUQgrDaRTOFv 0eQDmeSES17H058O6Fs09eHcqSM3KFnVPGlsW9mpBAuXyK/dBvSjpSM/1rwYvYBJT6uE UHxWqI465OT2CZf6K7eapB5XynRpNxD9p+ZQw/Q9t2HDZn+OCdciDZMOPMKFCjJiwClb w7Un1oGxspW7wtAruWsHE1zlvRw36KyeOROjPJZNnt1wpoh9NPb72Y5MkaQVDfqmNgEd wplw== X-Gm-Message-State: ANhLgQ2MQQJ0kaR4j8mGSuTacvLWp9HsPjT+tFtl3BTRoiCiiThJK9FI CLPNsF7p/UZKIPtvF4mjfomYw1Eb X-Google-Smtp-Source: ADFU+vtPdYsVKfNeq3Bifdgcc4zXYJPZOMoZFyehavbBEZQ/yYxBbW2S011A7w81/eMnFhgqquYBlA== X-Received: by 2002:a17:90a:8a97:: with SMTP id x23mr13995649pjn.151.1584180238469; Sat, 14 Mar 2020 03:03:58 -0700 (PDT) Received: from local.opencloud.tech.localdomain ([219.143.129.147]) by smtp.gmail.com with ESMTPSA id c207sm20809671pfb.47.2020.03.14.03.03.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 14 Mar 2020 03:03:58 -0700 (PDT) From: xiangxia.m.yue@gmail.com To: dev@openvswitch.org Date: Sat, 14 Mar 2020 18:03:50 +0800 Message-Id: <1584180230-89020-2-git-send-email-xiangxia.m.yue@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1584180230-89020-1-git-send-email-xiangxia.m.yue@gmail.com> References: <1584180230-89020-1-git-send-email-xiangxia.m.yue@gmail.com> Cc: Ilya Maximets Subject: [ovs-dev] [PATCH ovs 2/2] dpif-netdev: Add meter cache for performance X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Tonghao Zhang The cmap can address the meter capacity issue, but using cmap, there is 1% performance loss compared with previous array implementation. This patch add a meter cache in dp_netdev for fast lookup. With this patch, forwarding between p0 and p1, is 4,929,904 pps, the test method is same as first patch. Cc: Ben Pfaff Cc: Jarno Rajahalme Cc: Ilya Maximets Cc: Andy Zhou Signed-off-by: Tonghao Zhang --- lib/dpif-netdev.c | 108 ++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 104 insertions(+), 4 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 5474d52..3553fae 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -103,6 +103,11 @@ enum { MAX_FLOWS = 65536 }; /* Maximum number of flows in flow table. */ #define METER_ENTRY_MAX (1 << 19) /* Maximum number of bands / meter. */ #define METER_BAND_MAX (8) +/* A typical server system will have 32K L1d-cache per core. + * Set the number of meter cache entries to 8192. */ +#define METER_CACHE_SHIFT (13) +#define METER_CACHE_MAX (1 << METER_CACHE_SHIFT) +#define METER_CACHE_MASK (METER_CACHE_MAX -1) COVERAGE_DEFINE(datapath_drop_meter); COVERAGE_DEFINE(datapath_drop_upcall_error); @@ -284,6 +289,8 @@ struct dp_meter_band { struct dp_meter { struct cmap_node node; struct ovs_mutex lock; + struct ovs_refcount ref_cnt; + atomic_bool dead; uint32_t id; uint16_t flags; uint16_t n_bands; @@ -298,6 +305,9 @@ struct dp_netdev_meter { struct cmap table OVS_GUARDED; struct ovs_mutex lock; /* Used for meter table. */ uint32_t hash_basis; + + OVS_ALIGNED_VAR(CACHE_LINE_SIZE) + OVSRCU_TYPE(struct dp_meter *) cache[METER_CACHE_MAX]; }; struct pmd_auto_lb { @@ -1526,6 +1536,84 @@ dp_meter_hash(uint32_t meter_id, uint32_t basis) return hash_bytes32(&meter_id, sizeof meter_id, basis); } +static bool +dp_meter_ref(struct dp_meter *meter) +{ + return ovs_refcount_try_ref_rcu(&meter->ref_cnt); +} + +static void +dp_meter_unref(struct dp_meter *meter) +{ + if (ovs_refcount_unref_relaxed(&meter->ref_cnt) == 1) { + ovsrcu_postpone(free, meter); + } +} + +static void +dp_meter_cache_init(struct dp_netdev_meter *dp_meter) +{ + int i; + + for (i = 0; i < METER_CACHE_MAX; i++) { + ovsrcu_init(&dp_meter->cache[i], NULL); + } +} + +static void +dp_meter_cache_destroy(struct dp_netdev_meter *dp_meter) +{ + struct dp_meter *meter; + int i; + + for (i = 0; i < METER_CACHE_MAX; i++) { + meter = ovsrcu_get(struct dp_meter *, &dp_meter->cache[i]); + + if (meter) { + dp_meter_unref(meter); + ovsrcu_set(&dp_meter->cache[i], NULL); + } + } +} + +static struct dp_meter * +dp_meter_cache_lookup(struct dp_netdev_meter *dp_meter, + uint32_t meter_id) +{ + uint32_t hash = meter_id % METER_CACHE_MASK; + struct dp_meter *meter; + bool dead; + + meter = ovsrcu_get(struct dp_meter *, &dp_meter->cache[hash]); + if (likely(meter && meter->id == meter_id)) { + atomic_read_relaxed(&meter->dead, &dead); + if (dead) { + return NULL; + } + + return meter; + } + + return NULL; +} + +static void +dp_meter_cache_insert(struct dp_netdev_meter *dp_meter, + struct dp_meter *meter) +{ + uint32_t hash = meter->id % METER_CACHE_MASK; + struct dp_meter *old; + + old = ovsrcu_get(struct dp_meter *, &dp_meter->cache[hash]); + if (old) { + dp_meter_unref(old); + } + + if (dp_meter_ref(meter)) { + ovsrcu_set(&dp_meter->cache[hash], meter); + } +} + static void dp_netdev_meter_init(struct dp_netdev *dp) { @@ -1537,6 +1625,8 @@ dp_netdev_meter_init(struct dp_netdev *dp) ovs_mutex_init(&dp_meter->lock); dp_meter->hash_basis = random_uint32(); + dp_meter_cache_init(dp_meter); + dp->meter = dp_meter; } @@ -1558,6 +1648,7 @@ dp_netdev_meter_destroy(struct dp_netdev *dp) ovs_mutex_unlock(&dp_meter->lock); ovs_mutex_destroy(&dp_meter->lock); + dp_meter_cache_destroy(dp_meter); free(dp_meter); } @@ -1586,6 +1677,8 @@ dp_meter_detach_free(struct dp_netdev_meter *dp_meter, uint32_t meter_id) if (meter) { cmap_remove(&dp_meter->table, &meter->node, dp_meter_hash(meter_id, dp_meter->hash_basis)); + + atomic_store_relaxed(&meter->dead, true); ovsrcu_postpone(free, meter); } } @@ -5791,10 +5884,16 @@ dp_netdev_run_meter(struct dp_netdev *dp, struct dp_packet_batch *packets_, return; } - meter = dp_meter_lookup(dp->meter, meter_id); - if (!meter) { - return; - } + meter = dp_meter_cache_lookup(dp->meter, meter_id); + if (unlikely(!meter)) { + /* Missed in cache, lookup in slow path. */ + meter = dp_meter_lookup(dp->meter, meter_id); + if (!meter) { + return; + } + + dp_meter_cache_insert(dp->meter, meter); + } /* Initialize as negative values. */ memset(exceeded_band, 0xff, cnt * sizeof *exceeded_band); @@ -5961,6 +6060,7 @@ dpif_netdev_meter_set(struct dpif *dpif, ofproto_meter_id meter_id, meter->used = time_usec(); meter->id = mid; ovs_mutex_init(&meter->lock); + atomic_store_relaxed(&meter->dead, false); /* set up bands */ for (i = 0; i < config->n_bands; ++i) {