From patchwork Wed May 13 13:31:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tonghao Zhang X-Patchwork-Id: 1289295 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=JLApe1Gy; dkim-atps=neutral Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49MbFR282Kz9sRf for ; Wed, 13 May 2020 23:32:27 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 6178A2655A; Wed, 13 May 2020 13:32:23 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kznt0aaz3LdO; Wed, 13 May 2020 13:32:10 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by silver.osuosl.org (Postfix) with ESMTP id 197282047B; Wed, 13 May 2020 13:32:10 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0A712C016F; Wed, 13 May 2020 13:32:10 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id AAE8BC016F for ; Wed, 13 May 2020 13:32:08 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 8620722177 for ; Wed, 13 May 2020 13:32:08 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZSNS51Ze0tuj for ; Wed, 13 May 2020 13:32:02 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pg1-f193.google.com (mail-pg1-f193.google.com [209.85.215.193]) by silver.osuosl.org (Postfix) with ESMTPS id 7AA3F20495 for ; Wed, 13 May 2020 13:32:02 +0000 (UTC) Received: by mail-pg1-f193.google.com with SMTP id t11so7789451pgg.2 for ; Wed, 13 May 2020 06:32:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=3jdHi5STWC3pp/O8AWk+xsexkuflM0eAG9rQweUmuE0=; b=JLApe1Gyyb9kNcHCayb6moT/DUPGiBWc1/+0BqFzGGDWaQ6PiHILSj2QKfkQIW+DvG W6DF67/BLL1YSi08Y/2iO3axWmCV4srwq7FsxUcvqcurOQyoZnnxVdBfx9fWIJJFxAvi qbdxHIshXiIVybC1BpZLMVdQd1Rv8mbLA4P1D4dG5XeStZWef5H0ELvEj0VDxiMXw9Lb jZ0qsnEx71c03GahF/iNURSdqkV0e+dmmNZzO/eSOQAd/zqYxrP0KH5WslHGypvXTEr5 uU9z7a5naAJjmGWtvBJ/m0uHrBTXRAd5c5y3BXS/sSFdUVXX51BRLjgPs19LuHzHMCps wyRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=3jdHi5STWC3pp/O8AWk+xsexkuflM0eAG9rQweUmuE0=; b=gjWamQ3F02ClCA2BH09XkdHQEgEodn7N58FX5Z/+sDWwQa/c1HJA9GPr24gAVht/YL LwuNK9/jYeuvTkJ7HeQeFJ7ZyrfIg/JU7PtCKfImmALNdtWnUdxWGXEM2qOLlG/2QiOm Fl7LICmXR3gpcWw0lDWIyfhKAll3JfaUbXjrU0DJ18zJwzNz+2bbtq5KQ/LQoWWagc4W RBfjh/hKXWUGY9Mt2Y6AIQUqfMnjg6sWqdoDVLSHOg08DDAHzjwbR8NWkcylPSpfDY7q z65jvKIoQDbpovR1nhJECN4hVIdKq3/AI6Amprbk7Hv0OV3gj19YqQDZP9TOMb7bQ4uz zwsA== X-Gm-Message-State: AGi0PuY/zEve1K+/I8cD05UGQAL45ZiJ/ofK28mTFrsZ2KuRWaq9EQZS rCSkT7z6UWf+mSaWyOzSU8eJMhTfe7A= X-Google-Smtp-Source: APiQypJZv0A65NedjTJ4qe84WgIm36UxTxKNwrRgOLOjv1nxBJbTYKaG5xFysOPR7lYussuPxXRtGA== X-Received: by 2002:a63:f001:: with SMTP id k1mr21351828pgh.268.1589376721563; Wed, 13 May 2020 06:32:01 -0700 (PDT) Received: from localhost.localdomain ([203.100.54.194]) by smtp.gmail.com with ESMTPSA id h11sm14488745pfo.120.2020.05.13.06.31.58 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 13 May 2020 06:32:00 -0700 (PDT) From: xiangxia.m.yue@gmail.com To: dev@openvswitch.org, i.maximets@ovn.org, u9012063@gmail.com, jarno@ovn.org, blp@ovn.org, azhou@ovn.org, pshelar@ovn.org Date: Wed, 13 May 2020 21:31:32 +0800 Message-Id: <20200513133135.48474-2-xiangxia.m.yue@gmail.com> X-Mailer: git-send-email 2.15.0 In-Reply-To: <20200513133135.48474-1-xiangxia.m.yue@gmail.com> References: <1584180230-89020-1-git-send-email-xiangxia.m.yue@gmail.com> <20200513133135.48474-1-xiangxia.m.yue@gmail.com> Subject: [ovs-dev] [PATCH ovs v2 1/4] dpif-netdev: Expand the meters supported number X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Tonghao Zhang For now, ovs-vswitchd use the array of the dp_meter struct to store meter's data, and at most, there are only 65536 (defined by MAX_METERS) meters that can be used. But in some case, for example, in the edge gateway, we should use 200,000, at least, meters for IP address bandwidth limitation. Every one IP address will use two meters for its rx and tx path[1]. In other way, ovs-vswitchd should support meter-offload (rte_mtr_xxx api introduced by dpdk.), but there are more than 65536 meters in the hardware, such as Mellanox ConnectX-6. This patch use array to manage the meter, but it can ben expanded. [1]. $ in_port=p0,ip,ip_dst=1.1.1.x action=meter:n,output:p1 $ in_port=p1,ip,ip_src=1.1.1.x action=meter:m,output:p0 Cc: Ilya Maximets Cc: William Tu Cc: Jarno Rajahalme Cc: Ben Pfaff Cc: Andy Zhou Cc: Pravin Shelar Signed-off-by: Tonghao Zhang --- v2: * add comments for dp_meter_instance * change the log * remove extra newline * I don't move the dp_netdev_meter_init/destroy up. because them depends other meters function and put all meter function together may make the codes clean. --- lib/dpif-netdev.c | 319 ++++++++++++++++++++++++++++++++++++---------- 1 file changed, 250 insertions(+), 69 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index ef14e83b5f06..b5deaab31eb0 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -98,9 +98,12 @@ DEFINE_STATIC_PER_THREAD_DATA(uint32_t, recirc_depth, 0) /* Configuration parameters. */ enum { MAX_FLOWS = 65536 }; /* Maximum number of flows in flow table. */ -enum { MAX_METERS = 65536 }; /* Maximum number of meters. */ -enum { MAX_BANDS = 8 }; /* Maximum number of bands / meter. */ -enum { N_METER_LOCKS = 64 }; /* Maximum number of meters. */ + +/* Maximum number of meters in the table. */ +#define METER_ENTRY_MAX (200000ULL) +/* Maximum number of bands / meter. */ +#define METER_BAND_MAX (8) +#define DP_METER_ARRAY_SIZE_MIN (1ULL << 10) COVERAGE_DEFINE(datapath_drop_meter); COVERAGE_DEFINE(datapath_drop_upcall_error); @@ -283,12 +286,26 @@ struct dp_meter { uint16_t flags; uint16_t n_bands; uint32_t max_delta_t; + uint32_t id; + struct ovs_mutex lock; uint64_t used; uint64_t packet_count; uint64_t byte_count; struct dp_meter_band bands[]; }; +struct dp_meter_instance { + uint32_t n_meters; + /* Followed by struct dp_meter[n]; where n is the n_meters. */ + OVSRCU_TYPE(struct dp_meter *) dp_meters[]; +}; + +struct dp_meter_table { + OVSRCU_TYPE(struct dp_meter_instance *) ti; + uint32_t count; + struct ovs_mutex lock; +}; + struct pmd_auto_lb { bool auto_lb_requested; /* Auto load balancing requested by user. */ bool is_enabled; /* Current status of Auto load balancing. */ @@ -329,8 +346,7 @@ struct dp_netdev { atomic_uint32_t tx_flush_interval; /* Meters. */ - struct ovs_mutex meter_locks[N_METER_LOCKS]; - struct dp_meter *meters[MAX_METERS]; /* Meter bands. */ + struct dp_meter_table meter_tbl; /* Probability of EMC insertions is a factor of 'emc_insert_min'.*/ OVS_ALIGNED_VAR(CACHE_LINE_SIZE) atomic_uint32_t emc_insert_min; @@ -378,19 +394,6 @@ struct dp_netdev { struct pmd_auto_lb pmd_alb; }; -static void meter_lock(const struct dp_netdev *dp, uint32_t meter_id) - OVS_ACQUIRES(dp->meter_locks[meter_id % N_METER_LOCKS]) -{ - ovs_mutex_lock(&dp->meter_locks[meter_id % N_METER_LOCKS]); -} - -static void meter_unlock(const struct dp_netdev *dp, uint32_t meter_id) - OVS_RELEASES(dp->meter_locks[meter_id % N_METER_LOCKS]) -{ - ovs_mutex_unlock(&dp->meter_locks[meter_id % N_METER_LOCKS]); -} - - static struct dp_netdev_port *dp_netdev_lookup_port(const struct dp_netdev *dp, odp_port_t) OVS_REQUIRES(dp->port_mutex); @@ -1523,6 +1526,9 @@ choose_port(struct dp_netdev *dp, const char *name) return ODPP_NONE; } +static void dp_netdev_meter_init(struct dp_meter_table *tbl); +static void dp_netdev_meter_destroy(struct dp_meter_table *tbl); + static int create_dp_netdev(const char *name, const struct dpif_class *class, struct dp_netdev **dpp) @@ -1556,9 +1562,7 @@ create_dp_netdev(const char *name, const struct dpif_class *class, dp->reconfigure_seq = seq_create(); dp->last_reconfigure_seq = seq_read(dp->reconfigure_seq); - for (int i = 0; i < N_METER_LOCKS; ++i) { - ovs_mutex_init_adaptive(&dp->meter_locks[i]); - } + dp_netdev_meter_init(&dp->meter_tbl); /* Disable upcalls by default. */ dp_netdev_disable_upcall(dp); @@ -1647,16 +1651,6 @@ dp_netdev_destroy_upcall_lock(struct dp_netdev *dp) fat_rwlock_destroy(&dp->upcall_rwlock); } -static void -dp_delete_meter(struct dp_netdev *dp, uint32_t meter_id) - OVS_REQUIRES(dp->meter_locks[meter_id % N_METER_LOCKS]) -{ - if (dp->meters[meter_id]) { - free(dp->meters[meter_id]); - dp->meters[meter_id] = NULL; - } -} - /* Requires dp_netdev_mutex so that we can't get a new reference to 'dp' * through the 'dp_netdevs' shash while freeing 'dp'. */ static void @@ -1694,16 +1688,7 @@ dp_netdev_free(struct dp_netdev *dp) /* Upcalls must be disabled at this point */ dp_netdev_destroy_upcall_lock(dp); - int i; - - for (i = 0; i < MAX_METERS; ++i) { - meter_lock(dp, i); - dp_delete_meter(dp, i); - meter_unlock(dp, i); - } - for (i = 0; i < N_METER_LOCKS; ++i) { - ovs_mutex_destroy(&dp->meter_locks[i]); - } + dp_netdev_meter_destroy(&dp->meter_tbl); free(dp->pmd_cmask); free(CONST_CAST(char *, dp->name)); @@ -5713,14 +5698,197 @@ dp_netdev_disable_upcall(struct dp_netdev *dp) /* Meters */ +static uint32_t +meter_hash(struct dp_meter_instance *ti, uint32_t id) +{ + uint32_t n_meters = ti->n_meters; + + return id % n_meters; +} + +static void +dp_meter_free(struct dp_meter *meter) +{ + ovs_mutex_destroy(&meter->lock); + free(meter); +} + +static struct dp_meter_instance * +dp_meter_instance_alloc(const uint32_t size) +{ + struct dp_meter_instance *ti; + + ti = xzalloc(sizeof(*ti) + sizeof(struct dp_meter *) * size); + ti->n_meters = size; + + return ti; +} + +static void +dp_meter_instance_realloc(struct dp_meter_table *tbl, const uint32_t size) +{ + struct dp_meter_instance *new_ti; + struct dp_meter_instance *ti; + int n_meters; + int i; + + new_ti = dp_meter_instance_alloc(size); + + ti = ovsrcu_get(struct dp_meter_instance *, &tbl->ti); + n_meters = MIN(size, ti->n_meters); + + for (i = 0; i < n_meters; i++) { + if (ovsrcu_get(struct dp_meter *, &ti->dp_meters[i])) { + new_ti->dp_meters[i] = ti->dp_meters[i]; + } + } + + ovsrcu_set(&tbl->ti, new_ti); + ovsrcu_postpone(free, ti); +} + +static void +dp_meter_instance_insert(struct dp_meter_instance *ti, + struct dp_meter *meter) +{ + uint32_t hash; + + hash = meter_hash(ti, meter->id); + ovsrcu_set(&ti->dp_meters[hash], meter); +} + +static void +dp_meter_instance_remove(struct dp_meter_instance *ti, + struct dp_meter *meter) +{ + uint32_t hash; + + hash = meter_hash(ti, meter->id); + ovsrcu_set(&ti->dp_meters[hash], NULL); +} + +static void +dp_netdev_meter_init(struct dp_meter_table *tbl) +{ + struct dp_meter_instance *ti; + + ti = dp_meter_instance_alloc(DP_METER_ARRAY_SIZE_MIN); + ovsrcu_set(&tbl->ti, ti); + + ovs_mutex_init(&tbl->lock); + tbl->count = 0; +} + +static void +dp_netdev_meter_destroy(struct dp_meter_table *tbl) +{ + struct dp_meter_instance *ti; + int i; + + ti = ovsrcu_get(struct dp_meter_instance *, &tbl->ti); + for (i = 0; i < ti->n_meters; i++) { + struct dp_meter *meter; + + meter = ovsrcu_get(struct dp_meter *, &ti->dp_meters[i]); + if (meter) { + ovsrcu_postpone(dp_meter_free, meter); + } + } + + ovsrcu_postpone(free, ti); + ovs_mutex_destroy(&tbl->lock); +} + +static struct dp_meter * +dp_meter_lookup(struct dp_meter_table *meter_tbl, uint32_t meter_id) +{ + struct dp_meter_instance *ti; + struct dp_meter *meter; + uint32_t hash; + + ti = ovsrcu_get(struct dp_meter_instance *, &meter_tbl->ti); + hash = meter_hash(ti, meter_id); + + meter = ovsrcu_get(struct dp_meter *, &ti->dp_meters[hash]); + if (meter && meter->id == meter_id) { + return meter; + } + + return NULL; +} + +static void +dp_meter_detach_free(struct dp_meter_table *meter_tbl, uint32_t meter_id) + OVS_REQUIRES(meter_tbl->lock) +{ + struct dp_meter_instance *ti; + struct dp_meter *meter; + + meter = dp_meter_lookup(meter_tbl, meter_id); + if (!meter) { + return; + } + + ti = ovsrcu_get(struct dp_meter_instance *, &meter_tbl->ti); + dp_meter_instance_remove(ti, meter); + ovsrcu_postpone(dp_meter_free, meter); + + meter_tbl->count--; + /* Shrink the meter array if necessary. */ + if (ti->n_meters > DP_METER_ARRAY_SIZE_MIN && + meter_tbl->count <= (ti->n_meters / 4)) { + int half_size = ti->n_meters / 2; + int i; + + /* Avoid hash collision, don't move slots to other place. + * Make sure there are no references of meters in array + * which will be released. + */ + for (i = half_size; i < ti->n_meters; i++) { + if (ovsrcu_get(struct dp_meter *, &ti->dp_meters[i])) { + return; + } + } + + dp_meter_instance_realloc(meter_tbl, half_size); + } +} + +static int +dp_meter_attach(struct dp_meter_table *meter_tbl, struct dp_meter *meter) + OVS_REQUIRES(meter_tbl->lock) +{ + struct dp_meter_instance *ti; + uint32_t hash; + + ti = ovsrcu_get(struct dp_meter_instance *, &meter_tbl->ti); + hash = meter_hash(ti, meter->id); + + if (OVS_UNLIKELY(ovsrcu_get(struct dp_meter *, + &ti->dp_meters[hash]))) { + VLOG_WARN("Failed to attach meter id %u to slot %u/%u.\n", + meter->id, hash, ti->n_meters); + return EBUSY; + } + + dp_meter_instance_insert(ti, meter); + + meter_tbl->count++; + if (meter_tbl->count >= ti->n_meters) { + dp_meter_instance_realloc(meter_tbl, ti->n_meters * 2); + } + + return 0; +} + static void dpif_netdev_meter_get_features(const struct dpif * dpif OVS_UNUSED, struct ofputil_meter_features *features) { - features->max_meters = MAX_METERS; + features->max_meters = METER_ENTRY_MAX; features->band_types = DP_SUPPORTED_METER_BAND_TYPES; features->capabilities = DP_SUPPORTED_METER_FLAGS_MASK; - features->max_bands = MAX_BANDS; + features->max_bands = METER_BAND_MAX; features->max_color = 0; } @@ -5742,14 +5910,13 @@ dp_netdev_run_meter(struct dp_netdev *dp, struct dp_packet_batch *packets_, uint32_t exceeded_rate[NETDEV_MAX_BURST]; int exceeded_pkt = cnt; /* First packet that exceeded a band rate. */ - if (meter_id >= MAX_METERS) { + if (meter_id >= METER_ENTRY_MAX) { return; } - meter_lock(dp, meter_id); - meter = dp->meters[meter_id]; + meter = dp_meter_lookup(&dp->meter_tbl, meter_id); if (!meter) { - goto out; + return; } /* Initialize as negative values. */ @@ -5757,6 +5924,7 @@ dp_netdev_run_meter(struct dp_netdev *dp, struct dp_packet_batch *packets_, /* Initialize as zeroes. */ memset(exceeded_rate, 0, cnt * sizeof *exceeded_rate); + ovs_mutex_lock(&meter->lock); /* All packets will hit the meter at the same time. */ long_delta_t = now / 1000 - meter->used / 1000; /* msec */ @@ -5874,8 +6042,8 @@ dp_netdev_run_meter(struct dp_netdev *dp, struct dp_packet_batch *packets_, dp_packet_batch_refill(packets_, packet, j); } } - out: - meter_unlock(dp, meter_id); + + ovs_mutex_unlock(&meter->lock); } /* Meter set/get/del processing is still single-threaded. */ @@ -5884,11 +6052,12 @@ dpif_netdev_meter_set(struct dpif *dpif, ofproto_meter_id meter_id, struct ofputil_meter_config *config) { struct dp_netdev *dp = get_dp_netdev(dpif); + struct dp_meter_table *meter_tbl = &dp->meter_tbl; uint32_t mid = meter_id.uint32; struct dp_meter *meter; - int i; + int err, i; - if (mid >= MAX_METERS) { + if (mid >= METER_ENTRY_MAX) { return EFBIG; /* Meter_id out of range. */ } @@ -5896,7 +6065,7 @@ dpif_netdev_meter_set(struct dpif *dpif, ofproto_meter_id meter_id, return EBADF; /* Unsupported flags set */ } - if (config->n_bands > MAX_BANDS) { + if (config->n_bands > METER_BAND_MAX) { return EINVAL; } @@ -5917,6 +6086,8 @@ dpif_netdev_meter_set(struct dpif *dpif, ofproto_meter_id meter_id, meter->n_bands = config->n_bands; meter->max_delta_t = 0; meter->used = time_usec(); + meter->id = mid; + ovs_mutex_init(&meter->lock); /* set up bands */ for (i = 0; i < config->n_bands; ++i) { @@ -5942,12 +6113,22 @@ dpif_netdev_meter_set(struct dpif *dpif, ofproto_meter_id meter_id, } } - meter_lock(dp, mid); - dp_delete_meter(dp, mid); /* Free existing meter, if any */ - dp->meters[mid] = meter; - meter_unlock(dp, mid); + ovs_mutex_lock(&meter_tbl->lock); + + dp_meter_detach_free(meter_tbl, mid); /* Free existing meter, if any */ + err = dp_meter_attach(meter_tbl, meter); + if (err) { + goto unlock_out; + } + + ovs_mutex_unlock(&meter_tbl->lock); return 0; + +unlock_out: + ovs_mutex_unlock(&meter_tbl->lock); + dp_meter_free(meter); + return err; } static int @@ -5955,23 +6136,23 @@ dpif_netdev_meter_get(const struct dpif *dpif, ofproto_meter_id meter_id_, struct ofputil_meter_stats *stats, uint16_t n_bands) { - const struct dp_netdev *dp = get_dp_netdev(dpif); + struct dp_netdev *dp = get_dp_netdev(dpif); uint32_t meter_id = meter_id_.uint32; - int retval = 0; + const struct dp_meter *meter; - if (meter_id >= MAX_METERS) { + if (meter_id >= METER_ENTRY_MAX) { return EFBIG; } - meter_lock(dp, meter_id); - const struct dp_meter *meter = dp->meters[meter_id]; + meter = dp_meter_lookup(&dp->meter_tbl, meter_id); if (!meter) { - retval = ENOENT; - goto done; + return ENOENT; } + if (stats) { int i = 0; + ovs_mutex_lock(&meter->lock); stats->packet_in_count = meter->packet_count; stats->byte_in_count = meter->byte_count; @@ -5979,13 +6160,12 @@ dpif_netdev_meter_get(const struct dpif *dpif, stats->bands[i].packet_count = meter->bands[i].packet_count; stats->bands[i].byte_count = meter->bands[i].byte_count; } + ovs_mutex_unlock(&meter->lock); stats->n_bands = i; } -done: - meter_unlock(dp, meter_id); - return retval; + return 0; } static int @@ -5994,15 +6174,16 @@ dpif_netdev_meter_del(struct dpif *dpif, struct ofputil_meter_stats *stats, uint16_t n_bands) { struct dp_netdev *dp = get_dp_netdev(dpif); + struct dp_meter_table *meter_tbl = &dp->meter_tbl; int error; error = dpif_netdev_meter_get(dpif, meter_id_, stats, n_bands); if (!error) { uint32_t meter_id = meter_id_.uint32; - meter_lock(dp, meter_id); - dp_delete_meter(dp, meter_id); - meter_unlock(dp, meter_id); + ovs_mutex_lock(&meter_tbl->lock); + dp_meter_detach_free(meter_tbl, meter_id); + ovs_mutex_unlock(&meter_tbl->lock); } return error; } From patchwork Wed May 13 13:31:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tonghao Zhang X-Patchwork-Id: 1289296 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=lFC49Hh1; dkim-atps=neutral Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49MbFS2lZ6z9sSW for ; Wed, 13 May 2020 23:32:28 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 905DB20495; Wed, 13 May 2020 13:32:23 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Q6gaEDclXsh6; Wed, 13 May 2020 13:32:11 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by silver.osuosl.org (Postfix) with ESMTP id 2976B221D9; Wed, 13 May 2020 13:32:11 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0633EC0890; Wed, 13 May 2020 13:32:11 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 64207C016F for ; Wed, 13 May 2020 13:32:09 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 400DF20427 for ; Wed, 13 May 2020 13:32:09 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id i2sDRQgOEw+W for ; Wed, 13 May 2020 13:32:05 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pf1-f195.google.com (mail-pf1-f195.google.com [209.85.210.195]) by silver.osuosl.org (Postfix) with ESMTPS id 9093D2049E for ; Wed, 13 May 2020 13:32:05 +0000 (UTC) Received: by mail-pf1-f195.google.com with SMTP id y18so1201985pfl.9 for ; Wed, 13 May 2020 06:32:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=u/D6BgyBjbnMRDeGYUvKGl1NDCUo2ExShl2q7NyaY7E=; b=lFC49Hh1Danxq6U+Afr5pLrsdDLQeGAmdZqd+ANO/gRlZRcT0gXRW2zzSDtCDxBAz3 vj9ItE9dP9i821SQD2lvus2x4s5QP+0BNx1Re2TmKY0W+3tWotKR9tG3JUCBhLRHKcPS qeYw275oO4aTyJNM2TAG2YgEcIq50sRv6fAZqfaTddUKZxC3sbXTAEcHbEMaoYgAKw7H jGUTTz15vtU2Av2h5sWvw48PoKULYxGy4s4Zn4B5LZQJQ8DugWCotXFg1Dy/1w+0xIU2 tgJE8BpxK5TPx5PWYdyZ5XfhuHcYc5kIMx5vIDYqr4McX/ctYFeB3btQAxWS1Txb02/I 4/GA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=u/D6BgyBjbnMRDeGYUvKGl1NDCUo2ExShl2q7NyaY7E=; b=po3iX6vpv87yBR0h9UvaAWy8v7bSpns58pOYUDZad6bLix1s4oUIr+QRenGtFpCTSz aqFkFC1K7Uuma4NSBRABSsEVIfiZ8qDWTAnZKcR6fUaQ3wdsf/3DCGGUFMcESny9Gn/O OEYtKnoaFeG1qoRiTwVYbxfj190cJYHTVcLzu3HuV4UxgCZHv7eVtqnilpuizuFmlfub UIr+eSbp+gScs4wVHbx5cl4z5H7aE+SUQKB+e4UZtSvecAbD0dDDtLlEMqUr3AaGOzoz Ab+AYrqKomZJG0lzm6wl2iBA3kD0ILtG9pQPMiYYzStxaPQ22pBoffLhiU+uDVQFPo34 OrpA== X-Gm-Message-State: AGi0PuaOL1Jrwq4u4G/SC7bxFQ0XSpp/FaMWHOqzirjP9P0pCnez4hgc /MCEvuK3kuve9JO4lJV+IpgpEPN5SbM= X-Google-Smtp-Source: APiQypJioq5iXAm+pdFrXZpSJ+G0pqiUO77rlyCpG1OJTIskYfB59KvQ3W46MGMgZ7E/ypQkUUyUCQ== X-Received: by 2002:a63:7519:: with SMTP id q25mr18079114pgc.224.1589376724781; Wed, 13 May 2020 06:32:04 -0700 (PDT) Received: from localhost.localdomain ([203.100.54.194]) by smtp.gmail.com with ESMTPSA id h11sm14488745pfo.120.2020.05.13.06.32.01 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 13 May 2020 06:32:04 -0700 (PDT) From: xiangxia.m.yue@gmail.com To: dev@openvswitch.org, i.maximets@ovn.org, u9012063@gmail.com, jarno@ovn.org, blp@ovn.org, azhou@ovn.org, pshelar@ovn.org Date: Wed, 13 May 2020 21:31:33 +0800 Message-Id: <20200513133135.48474-3-xiangxia.m.yue@gmail.com> X-Mailer: git-send-email 2.15.0 In-Reply-To: <20200513133135.48474-1-xiangxia.m.yue@gmail.com> References: <1584180230-89020-1-git-send-email-xiangxia.m.yue@gmail.com> <20200513133135.48474-1-xiangxia.m.yue@gmail.com> Subject: [ovs-dev] [PATCH ovs v2 2/4] dpif-netdev: Add burst size to buckets X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Tonghao Zhang For now, the meter of the userspace datapath, don't include the bucket burst size to buckets. This patch includes it now. $ ovs-ofctl -O OpenFlow13 add-meter br0 \ 'meter=1 pktps burst stats bands=type=drop rate=10000 burst_size=2000' Cc: Ilya Maximets Cc: William Tu Cc: Jarno Rajahalme Cc: Ben Pfaff Cc: Andy Zhou Cc: Pravin Shelar Signed-off-by: Tonghao Zhang --- v2: * burst size unit is kilobits. should x1000. --- lib/dpif-netdev.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index b5deaab31eb0..36ecbfc728a9 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -6093,15 +6093,11 @@ dpif_netdev_meter_set(struct dpif *dpif, ofproto_meter_id meter_id, for (i = 0; i < config->n_bands; ++i) { uint32_t band_max_delta_t; - /* Set burst size to a workable value if none specified. */ - if (config->bands[i].burst_size == 0) { - config->bands[i].burst_size = config->bands[i].rate; - } - meter->bands[i].up = config->bands[i]; /* Convert burst size to the bucket units: */ /* pkts => 1/1000 packets, kilobits => bits. */ - meter->bands[i].up.burst_size *= 1000; + meter->bands[i].up.burst_size += config->bands[i].rate; + meter->bands[i].up.burst_size *= 1000ULL; /* Initialize bucket to empty. */ meter->bands[i].bucket = 0; From patchwork Wed May 13 13:31:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tonghao Zhang X-Patchwork-Id: 1289294 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=CXCa4q3a; dkim-atps=neutral Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49MbFP3dfwz9sSW for ; Wed, 13 May 2020 23:32:25 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 1D83D264BE; Wed, 13 May 2020 13:32:23 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id b94z0uRpF2Qs; Wed, 13 May 2020 13:32:18 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by silver.osuosl.org (Postfix) with ESMTP id 1D52125E31; Wed, 13 May 2020 13:32:14 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0F9B8C088F; Wed, 13 May 2020 13:32:14 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 3FC79C016F for ; Wed, 13 May 2020 13:32:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 2FB02881D2 for ; Wed, 13 May 2020 13:32:10 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id R+BSQ2ZmqZnB for ; Wed, 13 May 2020 13:32:09 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pl1-f196.google.com (mail-pl1-f196.google.com [209.85.214.196]) by whitealder.osuosl.org (Postfix) with ESMTPS id A5CA0881BB for ; Wed, 13 May 2020 13:32:09 +0000 (UTC) Received: by mail-pl1-f196.google.com with SMTP id t7so6833948plr.0 for ; Wed, 13 May 2020 06:32:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=S/Gx9qV5N3G6zU4fxDZNw3l8c0wctHBWjMgWzvejGu4=; b=CXCa4q3aMKB2VIIDUvW5YjKDKvVNi1Kwlx3PwaBiO0xnET/fdtrqRdVXNsoZl/3Dah 6gONRrRu5s8QqWpAnSJRhLT0wBvUZMiIxAoB6voJ1DjDjIdgPZihftcpI/gmR8ODWYLS Jap6HJLouhb/S4TR5p9hFqdN7fDWNqH8H6LzRHlgvhNgWbgj7lPJMsQxU9GuXqzjAFrA TnQ/UJ0rJoNsvpN0j83Cg6US/+Mi9MBcLAr1jFur1FjlP68ih6rVlr9D1x5Pgy9mXEF/ 9717zYZH1OmrTklyyU69jV3kgGl3ItHwqfIngbGY6P/nYkQESuaRvb8bE7J7+/x5xzl1 4xWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=S/Gx9qV5N3G6zU4fxDZNw3l8c0wctHBWjMgWzvejGu4=; b=NMKlMrb+d16a2mRkVrk677ESQA0N1h+J1jT355e/pQVHewP0YNRqCu/GbP/RMxQD8/ r0JujVSbtmA2xL8Bp0Z0NcONOWVwyIvOeXU1us5hQBqNL4Qdxs4ZN+J0lkmvdByWDnww 4+P3ZRyQvc2e3HIZlacb1JBzKCiM7IGdYU3+kWxCqg9kG3pn5Tlys0D4XrLOaxUZ1+9P idt0uaobXoLw5xW6q0rcZN0eRrr2EoyizbqyEje0Falnej7stFsVPwprX8GvQZdIMPoN rPoJv+gcypjbfvKbZI5ctiNB/sJ9VxxNcl8dYSI1dXASDqYR4IEZEnWBudba0IW9ZGHU cq0A== X-Gm-Message-State: AGi0Pua3kb2CTGfLgBTVSV39mJUbrPav+oN6HQms0uqWQwwNgh/DZltL yHiN1GQ99jRqiKEUqAVF/AcmpT+i4aM= X-Google-Smtp-Source: APiQypKOO+i6qDtXTfMtePEncPrntQyij9XYkcGiqB8EHQGGl1K9tFzYCIhei9RnIw/PcpRsR2O2yQ== X-Received: by 2002:a17:90a:8d12:: with SMTP id c18mr34001180pjo.144.1589376728966; Wed, 13 May 2020 06:32:08 -0700 (PDT) Received: from localhost.localdomain ([203.100.54.194]) by smtp.gmail.com with ESMTPSA id h11sm14488745pfo.120.2020.05.13.06.32.05 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 13 May 2020 06:32:08 -0700 (PDT) From: xiangxia.m.yue@gmail.com To: dev@openvswitch.org, i.maximets@ovn.org, u9012063@gmail.com, jarno@ovn.org, blp@ovn.org, azhou@ovn.org, pshelar@ovn.org Date: Wed, 13 May 2020 21:31:34 +0800 Message-Id: <20200513133135.48474-4-xiangxia.m.yue@gmail.com> X-Mailer: git-send-email 2.15.0 In-Reply-To: <20200513133135.48474-1-xiangxia.m.yue@gmail.com> References: <1584180230-89020-1-git-send-email-xiangxia.m.yue@gmail.com> <20200513133135.48474-1-xiangxia.m.yue@gmail.com> Subject: [ovs-dev] [PATCH ovs v2 3/4] dpif-netdev: Use the u64 instead of u32 for buckets X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Tonghao Zhang When setting the meter rate to 4.3+Gbps, there is an overflow, the meters don't work as expected. Cc: Ilya Maximets Cc: William Tu Cc: Jarno Rajahalme Cc: Ben Pfaff Cc: Andy Zhou Cc: Pravin Shelar Acked-by: William Tu Signed-off-by: Tonghao Zhang --- include/openvswitch/ofp-meter.h | 2 +- lib/dpif-netdev.c | 4 ++-- lib/ofp-meter.c | 4 ++-- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/include/openvswitch/ofp-meter.h b/include/openvswitch/ofp-meter.h index 6776eae87e26..f55f89ac1a71 100644 --- a/include/openvswitch/ofp-meter.h +++ b/include/openvswitch/ofp-meter.h @@ -37,7 +37,7 @@ struct ofputil_meter_band { uint16_t type; uint8_t prec_level; /* Non-zero if type == OFPMBT_DSCP_REMARK. */ uint32_t rate; - uint32_t burst_size; + uint64_t burst_size; }; void ofputil_format_meter_band(struct ds *, enum ofp13_meter_flags, diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 36ecbfc728a9..9dac33ee3fca 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -277,7 +277,7 @@ static bool dpcls_lookup(struct dpcls *cls, struct dp_meter_band { struct ofputil_meter_band up; /* type, prec_level, pad, rate, burst_size */ - uint32_t bucket; /* In 1/1000 packets (for PKTPS), or in bits (for KBPS) */ + uint64_t bucket; /* In 1/1000 packets (for PKTPS), or in bits (for KBPS) */ uint64_t packet_count; uint64_t byte_count; }; @@ -5971,7 +5971,7 @@ dp_netdev_run_meter(struct dp_netdev *dp, struct dp_packet_batch *packets_, band = &meter->bands[m]; /* Update band's bucket. */ - band->bucket += delta_t * band->up.rate; + band->bucket += (uint64_t)delta_t * band->up.rate; band->bucket += delta_in_us * band->up.rate / 1000; if (band->bucket > band->up.burst_size) { band->bucket = band->up.burst_size; diff --git a/lib/ofp-meter.c b/lib/ofp-meter.c index 9ea40a0bfb63..1ac993bb028b 100644 --- a/lib/ofp-meter.c +++ b/lib/ofp-meter.c @@ -72,7 +72,7 @@ ofputil_format_meter_band(struct ds *s, enum ofp13_meter_flags flags, ds_put_format(s, " rate=%"PRIu32, mb->rate); if (flags & OFPMF13_BURST) { - ds_put_format(s, " burst_size=%"PRIu32, mb->burst_size); + ds_put_format(s, " burst_size=%"PRIu64, mb->burst_size); } if (mb->type == OFPMBT13_DSCP_REMARK) { ds_put_format(s, " prec_level=%"PRIu8, mb->prec_level); @@ -703,7 +703,7 @@ parse_ofp_meter_mod_str__(struct ofputil_meter_mod *mm, char *string, return error; } } else if (!strcmp(name, "burst_size")) { - char *error = str_to_u32(value, &band->burst_size); + char *error = str_to_u64(value, &band->burst_size); if (error) { return error; } From patchwork Wed May 13 13:31:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tonghao Zhang X-Patchwork-Id: 1289293 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=fgF6QRO/; dkim-atps=neutral Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49MbFP1bPXz9sRf for ; Wed, 13 May 2020 23:32:25 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id A863689336; Wed, 13 May 2020 13:32:23 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aOi3NtvsXd-9; Wed, 13 May 2020 13:32:22 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by hemlock.osuosl.org (Postfix) with ESMTP id BFC27889C3; Wed, 13 May 2020 13:32:22 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id A029AC08A2; Wed, 13 May 2020 13:32:22 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id C3F06C0178 for ; Wed, 13 May 2020 13:32:20 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 9C5BC264BE for ; Wed, 13 May 2020 13:32:20 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mQsaXSYEBAzh for ; Wed, 13 May 2020 13:32:17 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by silver.osuosl.org (Postfix) with ESMTPS id 5BEEE25D6D for ; Wed, 13 May 2020 13:32:13 +0000 (UTC) Received: by mail-pl1-f177.google.com with SMTP id f15so6825639plr.3 for ; Wed, 13 May 2020 06:32:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=iBzVYftHLY8cKyHNXorOMselR/J+8mQsE6hKctZuyFk=; b=fgF6QRO/p+G/rFiFzOd4AqqfRLXmCFz8pYSRASOR5Wn07FLPjdpQcknmwfzb+f8JsS woomtZEe+ajqp9eQngsDiyXWj+OeVuqPnL34Soxt/6JyFJNJ5+JCnw2/P3MWpcqxadcJ budtGo6SJaArjeDl2/YkgFaSpuhF6tny45IbhgX0WzIh6LkGPYEkMIcxyQVfiBoB9atp CfHxpn3JhJC9JdVewOYV1ZBdcC/88K0e7IPoIg7EzBqIkdVAqIcspJp6CwFUGH8Hfbor x24tr7nm7pdcPeXxiI3KKgyw3qk1xZHxiQJM+EyoghOFRpLhP4ixNFpScN2GwS0mgyI+ rtdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=iBzVYftHLY8cKyHNXorOMselR/J+8mQsE6hKctZuyFk=; b=U1lGCTE3nKIcKAZ2zFMfr76XPoSEqR7B8XW9TsRp5vsw/ds1a+8HmxSwIqt+JHPKIz Rr1+OOdC4eq3Hufvo9j1SfdSFVyPnB3R13R6SnijLsIntJ3UO8OtPO28GjbtIMFAq7/z Ot8cbe04vD/AXw8tG++WiC+OdmJRJFcQ3zZeYwAHDax9FKl0nH5NDw1svAAH2bL3PGth 7OLKgTX2MG+PW5zU/9mJhyw6SMcl3yVdL5w8w2gFyLh9aDucHkcPfA+dQG5NGG2HbU7Z 7Uyu2a7YcsZkxFxpmgboQEQJnnYjTyywPAYVlje2nIqRxWQAzvzHmFf4kWBOvApgHUf4 aPzw== X-Gm-Message-State: AOAM5336JoBbL4kJTdHuDuKzwkR+fg0DFRqCCJNuwxt2DVHxFTYL435f Flx+LqM+td4r2PQVZ9IvO1tP7ZoFQfY= X-Google-Smtp-Source: ABdhPJx9Qiv4lHLl0V33wWk59FMCNxTopPng8PQbX1JzD4aruZcUP5mTfgD87Epx2Llf4VKPlfBONg== X-Received: by 2002:a17:90a:2610:: with SMTP id l16mr9359604pje.219.1589376732659; Wed, 13 May 2020 06:32:12 -0700 (PDT) Received: from localhost.localdomain ([203.100.54.194]) by smtp.gmail.com with ESMTPSA id h11sm14488745pfo.120.2020.05.13.06.32.09 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 13 May 2020 06:32:11 -0700 (PDT) From: xiangxia.m.yue@gmail.com To: dev@openvswitch.org, i.maximets@ovn.org, u9012063@gmail.com, jarno@ovn.org, blp@ovn.org, azhou@ovn.org, pshelar@ovn.org Date: Wed, 13 May 2020 21:31:35 +0800 Message-Id: <20200513133135.48474-5-xiangxia.m.yue@gmail.com> X-Mailer: git-send-email 2.15.0 In-Reply-To: <20200513133135.48474-1-xiangxia.m.yue@gmail.com> References: <1584180230-89020-1-git-send-email-xiangxia.m.yue@gmail.com> <20200513133135.48474-1-xiangxia.m.yue@gmail.com> Subject: [ovs-dev] [PATCH ovs v2 4/4] Revert "dpif-netdev: includes microsecond delta in meter bucket calculation" X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Tonghao Zhang This reverts commit 5c41c31ebd64fda821fb733a5784a7a440a794f8. Use the pktgen-dpdk to test the commit 5c41c31ebd64 ("dpif-netdev: includes microsecond delta in meter bucket calculation"), it does't work as expected. And it broken the meter function (e.g. set rate 200Mbps, the rate watched was 400Mbps). To reproduce it: $ ovs-vsctl add-br br-int -- set bridge br-int datapath_type=netdev $ ovs-ofctl -O OpenFlow13 add-meter br-int \ "meter=100 kbps burst stats bands=type=drop rate=200000 burst_size=200000" $ ovs-ofctl -O OpenFlow13 add-flow br-int \ "in_port=dpdk0 action=meter:100,output:dpdk1" $ pktgen -l 1,3,5,7,9,11,13,15,17,19 -n 8 --socket-mem 4096 --file-prefix pg1 \ -w 0000:82:00.0 -w 0000:82:00.1 -- -T -P -m "[3/5/7/9/11/13/15].[0-1]" -f meter-test.pkt meter-test.pkt: | set 0 count 0 | set 0 size 1500 | set 0 rate 100 | set 0 burst 64 | set 0 sport 1234 | set 0 dport 5678 | set 0 prime 1 | set 0 type ipv4 | set 0 proto udp | set 0 dst ip 1.1.1.2 | set 0 src ip 1.1.1.1/24 | set 0 dst mac ec:0d:9a:ab:54:0a | set 0 src mac ec:0d:9a:bf:df:bb | set 0 vlanid 0 | start 0 Note that 42697ca7757b ("dpif-netdev: fix meter at high packet rate.") has fixed it as 5c41c31ebd64. Cc: Ilya Maximets Cc: William Tu Cc: Jarno Rajahalme Cc: Ben Pfaff Cc: Andy Zhou Cc: Jiang Lidong Cc: Pravin Shelar Signed-off-by: Tonghao Zhang --- lib/dpif-netdev.c | 5 ----- 1 file changed, 5 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 9dac33ee3fca..b834b56d761b 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -5903,7 +5903,6 @@ dp_netdev_run_meter(struct dp_netdev *dp, struct dp_packet_batch *packets_, struct dp_packet *packet; long long int long_delta_t; /* msec */ uint32_t delta_t; /* msec */ - uint32_t delta_in_us; /* usec */ const size_t cnt = dp_packet_batch_size(packets_); uint32_t bytes, volume; int exceeded_band[NETDEV_MAX_BURST]; @@ -5934,9 +5933,6 @@ dp_netdev_run_meter(struct dp_netdev *dp, struct dp_packet_batch *packets_, Assuming that all racing threads received packets at the same time to avoid overflow. */ long_delta_t = 0; - delta_in_us = 0; - } else { - delta_in_us = (now - meter->used) % 1000; } /* Make sure delta_t will not be too large, so that bucket will not @@ -5972,7 +5968,6 @@ dp_netdev_run_meter(struct dp_netdev *dp, struct dp_packet_batch *packets_, /* Update band's bucket. */ band->bucket += (uint64_t)delta_t * band->up.rate; - band->bucket += delta_in_us * band->up.rate / 1000; if (band->bucket > band->up.burst_size) { band->bucket = band->up.burst_size; }