diff mbox

[ovs-dev,v3,2/4] dpif-netdev: Avoid reading RSS hash when EMC is disabled

Message ID 1502466766-17370-3-git-send-email-antonio.fischetti@intel.com
State Superseded
Delegated to: Darrell Ball
Headers show

Commit Message

Fischetti, Antonio Aug. 11, 2017, 3:52 p.m. UTC 
When EMC is disabled the reading of RSS hash is skipped.
Also, for packets that are not recirculated it retrieves the hash
value without considering the recirc id.

Signed-off-by: Antonio Fischetti <antonio.fischetti@intel.com>
---
This patch depends on previous patch in this series.

Port-to-Port Test

Comments

Darrell Ball Aug. 14, 2017, 6:32 a.m. UTC | #1 
I did not try it yet, but seems reasonable
If the hash is needed for something else, it will be read at that point. 

-----Original Message-----
From: <ovs-dev-bounces@openvswitch.org> on behalf of "antonio.fischetti@intel.com" <antonio.fischetti@intel.com>
Date: Friday, August 11, 2017 at 8:52 AM
To: "dev@openvswitch.org" <dev@openvswitch.org>
Subject: [ovs-dev] [PATCH v3 2/4] dpif-netdev: Avoid reading RSS hash when	EMC is disabled

    When EMC is disabled the reading of RSS hash is skipped.
    Also, for packets that are not recirculated it retrieves the hash
    value without considering the recirc id.
    
    Signed-off-by: Antonio Fischetti <antonio.fischetti@intel.com>
    ---
    This patch depends on previous patch in this series.
    
    Port-to-Port Test
    =================
    Software built with "-O2 -march=native -g".
    
    I measured the Rx rate regardless of pkt loss by sending 1 UDP flow,
    64B packets, at the line-rate.
    
    2 PMDs with 3 Tx queues.
    
    Flows setup:
      in_port=dpdk0 actions=output:dpdk1
      in_port=dpdk1 actions=output:dpdk0
    
    Results
    -------
    Values are for the Rx rate in Mpps, regardless of packet loss.
    
    RSS column:
       Yes: RSS hash is provided by the NIC
       No: RSS is disabled and the 5-tuple hash must be
           computed in software.
    
    EMC column:
       Yes: default probability insertion,
       No: EMC disabled.
    
    Orig OvS-DPDK means Commit ID:
      6b1babacc3ca0488e07596bf822fe356c9bab646
    
                +--------------+----------------+----------------+
    +-----+-----+    Orig      | Orig + patch 1 | Orig + patch 1 |
    | RSS | EMC |              |                |  this patch    |
    +-----+-----+--------------+----------------+----------------+
    | Yes | Yes | 11.99, 11.41 | 12.20, 11.31   | 12.20, 11.31   |
    | Yes |  No |  8.32,  8.42 |  8.35,  8.39   |  8.62, 8.62    |
    +-----+-----+--------------+----------------+----------------+
    |  No | Yes |  9.87, 11.15 |  9.79, 11.20   |  9.85, 11.09   |
    |  No |  No |  7.82,  7.84 |  7.84,  7.93   |  8.40,  8.38   |
    +-----+-----+--------------+----------------+----------------+
    ---
     lib/dpif-netdev.c | 20 +++++++++++++++++++-
     1 file changed, 19 insertions(+), 1 deletion(-)
    
    diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
    index 8f6b96b..0db6f83 100644
    --- a/lib/dpif-netdev.c
    +++ b/lib/dpif-netdev.c
    @@ -4578,6 +4578,22 @@ dp_netdev_upcall(struct dp_netdev_pmd_thread *pmd, struct dp_packet *packet_,
     }
     
     static inline uint32_t
    +dpif_netdev_packet_get_rss_hash_orig_pkt(struct dp_packet *packet,
    +                                const struct miniflow *mf)
    +{
    +    uint32_t hash;
    +
    +    if (OVS_LIKELY(dp_packet_rss_valid(packet))) {
    +        hash = dp_packet_get_rss_hash(packet);
    +    } else {
    +        hash = miniflow_hash_5tuple(mf, 0);
    +        dp_packet_set_rss_hash(packet, hash);
    +    }
    +
    +    return hash;
    +}
    +
    +static inline uint32_t
     dpif_netdev_packet_get_rss_hash(struct dp_packet *packet,
                                     const struct miniflow *mf)
     {
    @@ -4715,7 +4731,6 @@ emc_processing(struct dp_netdev_pmd_thread *pmd,
             }
             miniflow_extract(packet, &key->mf);
             key->len = 0; /* Not computed yet. */
    -        key->hash = dpif_netdev_packet_get_rss_hash(packet, &key->mf);
     
             /*
              * EMC lookup is skipped when one or both of the following
    @@ -4732,9 +4747,12 @@ emc_processing(struct dp_netdev_pmd_thread *pmd,
              */
             if (OVS_LIKELY(cur_min)) {
                 if (!md_is_valid) {
    +                key->hash = dpif_netdev_packet_get_rss_hash_orig_pkt(packet,
    +                        &key->mf);
                     flow = emc_lookup(flow_cache, key);
                 } else {
                     /* Recirculated packet. */
    +                key->hash = dpif_netdev_packet_get_rss_hash(packet, &key->mf);
                     if (flow_cache->n_entries & EMC_RECIRCT_NO_INSERT_THRESHOLD) {
                         /* EMC occupancy is over the threshold.  We skip EMC
                          * lookup for recirculated packets. */
    -- 
    2.4.11
    
    _______________________________________________
    dev mailing list
    dev@openvswitch.org
    https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs-2Ddev&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=Kp92W-efwcliK8l_PTI61aY9T1CPqdhzlAxSHGRMad8&s=NdrRKHn6b0aW6Gbj5wOH3_Rkon2dcVgoyPBzfzJs_E4&e=
Darrell Ball Aug. 25, 2017, 8:59 a.m. UTC | #2 
Hi Antonio

Can the dependency of this Patch 2 on Patch 1 be removed, while Patch 1 is being discussed ?

Thanks Darrell 

On 8/13/17, 11:32 PM, "ovs-dev-bounces@openvswitch.org on behalf of Darrell Ball" <ovs-dev-bounces@openvswitch.org on behalf of dball@vmware.com> wrote:

    I did not try it yet, but seems reasonable
    If the hash is needed for something else, it will be read at that point. 
    
    -----Original Message-----
    From: <ovs-dev-bounces@openvswitch.org> on behalf of "antonio.fischetti@intel.com" <antonio.fischetti@intel.com>
    Date: Friday, August 11, 2017 at 8:52 AM
    To: "dev@openvswitch.org" <dev@openvswitch.org>
    Subject: [ovs-dev] [PATCH v3 2/4] dpif-netdev: Avoid reading RSS hash when	EMC is disabled
    
        When EMC is disabled the reading of RSS hash is skipped.
        Also, for packets that are not recirculated it retrieves the hash
        value without considering the recirc id.
        
        Signed-off-by: Antonio Fischetti <antonio.fischetti@intel.com>
        ---
        This patch depends on previous patch in this series.
        
        Port-to-Port Test
        =================
        Software built with "-O2 -march=native -g".
        
        I measured the Rx rate regardless of pkt loss by sending 1 UDP flow,
        64B packets, at the line-rate.
        
        2 PMDs with 3 Tx queues.
        
        Flows setup:
          in_port=dpdk0 actions=output:dpdk1
          in_port=dpdk1 actions=output:dpdk0
        
        Results
        -------
        Values are for the Rx rate in Mpps, regardless of packet loss.
        
        RSS column:
           Yes: RSS hash is provided by the NIC
           No: RSS is disabled and the 5-tuple hash must be
               computed in software.
        
        EMC column:
           Yes: default probability insertion,
           No: EMC disabled.
        
        Orig OvS-DPDK means Commit ID:
          6b1babacc3ca0488e07596bf822fe356c9bab646
        
                    +--------------+----------------+----------------+
        +-----+-----+    Orig      | Orig + patch 1 | Orig + patch 1 |
        | RSS | EMC |              |                |  this patch    |
        +-----+-----+--------------+----------------+----------------+
        | Yes | Yes | 11.99, 11.41 | 12.20, 11.31   | 12.20, 11.31   |
        | Yes |  No |  8.32,  8.42 |  8.35,  8.39   |  8.62, 8.62    |
        +-----+-----+--------------+----------------+----------------+
        |  No | Yes |  9.87, 11.15 |  9.79, 11.20   |  9.85, 11.09   |
        |  No |  No |  7.82,  7.84 |  7.84,  7.93   |  8.40,  8.38   |
        +-----+-----+--------------+----------------+----------------+
        ---
         lib/dpif-netdev.c | 20 +++++++++++++++++++-
         1 file changed, 19 insertions(+), 1 deletion(-)
        
        diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
        index 8f6b96b..0db6f83 100644
        --- a/lib/dpif-netdev.c
        +++ b/lib/dpif-netdev.c
        @@ -4578,6 +4578,22 @@ dp_netdev_upcall(struct dp_netdev_pmd_thread *pmd, struct dp_packet *packet_,
         }
         
         static inline uint32_t
        +dpif_netdev_packet_get_rss_hash_orig_pkt(struct dp_packet *packet,
        +                                const struct miniflow *mf)
        +{
        +    uint32_t hash;
        +
        +    if (OVS_LIKELY(dp_packet_rss_valid(packet))) {
        +        hash = dp_packet_get_rss_hash(packet);
        +    } else {
        +        hash = miniflow_hash_5tuple(mf, 0);
        +        dp_packet_set_rss_hash(packet, hash);
        +    }
        +
        +    return hash;
        +}
        +
        +static inline uint32_t
         dpif_netdev_packet_get_rss_hash(struct dp_packet *packet,
                                         const struct miniflow *mf)
         {
        @@ -4715,7 +4731,6 @@ emc_processing(struct dp_netdev_pmd_thread *pmd,
                 }
                 miniflow_extract(packet, &key->mf);
                 key->len = 0; /* Not computed yet. */
        -        key->hash = dpif_netdev_packet_get_rss_hash(packet, &key->mf);
         
                 /*
                  * EMC lookup is skipped when one or both of the following
        @@ -4732,9 +4747,12 @@ emc_processing(struct dp_netdev_pmd_thread *pmd,
                  */
                 if (OVS_LIKELY(cur_min)) {
                     if (!md_is_valid) {
        +                key->hash = dpif_netdev_packet_get_rss_hash_orig_pkt(packet,
        +                        &key->mf);
                         flow = emc_lookup(flow_cache, key);
                     } else {
                         /* Recirculated packet. */
        +                key->hash = dpif_netdev_packet_get_rss_hash(packet, &key->mf);
                         if (flow_cache->n_entries & EMC_RECIRCT_NO_INSERT_THRESHOLD) {
                             /* EMC occupancy is over the threshold.  We skip EMC
                              * lookup for recirculated packets. */
        -- 
        2.4.11
        
        _______________________________________________
        dev mailing list
        dev@openvswitch.org
        https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs-2Ddev&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=Kp92W-efwcliK8l_PTI61aY9T1CPqdhzlAxSHGRMad8&s=NdrRKHn6b0aW6Gbj5wOH3_Rkon2dcVgoyPBzfzJs_E4&e= 
        
    
    _______________________________________________
    dev mailing list
    dev@openvswitch.org
    https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs-2Ddev&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=vhciWYIB9ZtYxl6vdpUtQE953Hx0s1E0Oz0DFmWOBug&s=10on8gc7nTUk8oyo3jwRYrlyQ6PFB6HSddFXL91iLSQ&e=
Fischetti, Antonio Aug. 25, 2017, 1:19 p.m. UTC | #3 
Sure Darrell, I'll do that.
Thanks!

Antonio

> -----Original Message-----
> From: Darrell Ball [mailto:dball@vmware.com]
> Sent: Friday, August 25, 2017 9:59 AM
> To: Fischetti, Antonio <antonio.fischetti@intel.com>; dev@openvswitch.org
> Subject: Re: [ovs-dev] [PATCH v3 2/4] dpif-netdev: Avoid reading RSS hash when
> EMC is disabled
> 
> Hi Antonio
> 
> Can the dependency of this Patch 2 on Patch 1 be removed, while Patch 1 is
> being discussed ?
> 
> Thanks Darrell
> 
> On 8/13/17, 11:32 PM, "ovs-dev-bounces@openvswitch.org on behalf of Darrell
> Ball" <ovs-dev-bounces@openvswitch.org on behalf of dball@vmware.com> wrote:
> 
>     I did not try it yet, but seems reasonable
>     If the hash is needed for something else, it will be read at that point.
> 
>     -----Original Message-----
>     From: <ovs-dev-bounces@openvswitch.org> on behalf of
> "antonio.fischetti@intel.com" <antonio.fischetti@intel.com>
>     Date: Friday, August 11, 2017 at 8:52 AM
>     To: "dev@openvswitch.org" <dev@openvswitch.org>
>     Subject: [ovs-dev] [PATCH v3 2/4] dpif-netdev: Avoid reading RSS hash when
> 	EMC is disabled
> 
>         When EMC is disabled the reading of RSS hash is skipped.
>         Also, for packets that are not recirculated it retrieves the hash
>         value without considering the recirc id.
> 
>         Signed-off-by: Antonio Fischetti <antonio.fischetti@intel.com>
>         ---
>         This patch depends on previous patch in this series.
> 
>         Port-to-Port Test
>         =================
>         Software built with "-O2 -march=native -g".
> 
>         I measured the Rx rate regardless of pkt loss by sending 1 UDP flow,
>         64B packets, at the line-rate.
> 
>         2 PMDs with 3 Tx queues.
> 
>         Flows setup:
>           in_port=dpdk0 actions=output:dpdk1
>           in_port=dpdk1 actions=output:dpdk0
> 
>         Results
>         -------
>         Values are for the Rx rate in Mpps, regardless of packet loss.
> 
>         RSS column:
>            Yes: RSS hash is provided by the NIC
>            No: RSS is disabled and the 5-tuple hash must be
>                computed in software.
> 
>         EMC column:
>            Yes: default probability insertion,
>            No: EMC disabled.
> 
>         Orig OvS-DPDK means Commit ID:
>           6b1babacc3ca0488e07596bf822fe356c9bab646
> 
>                     +--------------+----------------+----------------+
>         +-----+-----+    Orig      | Orig + patch 1 | Orig + patch 1 |
>         | RSS | EMC |              |                |  this patch    |
>         +-----+-----+--------------+----------------+----------------+
>         | Yes | Yes | 11.99, 11.41 | 12.20, 11.31   | 12.20, 11.31   |
>         | Yes |  No |  8.32,  8.42 |  8.35,  8.39   |  8.62, 8.62    |
>         +-----+-----+--------------+----------------+----------------+
>         |  No | Yes |  9.87, 11.15 |  9.79, 11.20   |  9.85, 11.09   |
>         |  No |  No |  7.82,  7.84 |  7.84,  7.93   |  8.40,  8.38   |
>         +-----+-----+--------------+----------------+----------------+
>         ---
>          lib/dpif-netdev.c | 20 +++++++++++++++++++-
>          1 file changed, 19 insertions(+), 1 deletion(-)
> 
>         diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
>         index 8f6b96b..0db6f83 100644
>         --- a/lib/dpif-netdev.c
>         +++ b/lib/dpif-netdev.c
>         @@ -4578,6 +4578,22 @@ dp_netdev_upcall(struct dp_netdev_pmd_thread
> *pmd, struct dp_packet *packet_,
>          }
> 
>          static inline uint32_t
>         +dpif_netdev_packet_get_rss_hash_orig_pkt(struct dp_packet *packet,
>         +                                const struct miniflow *mf)
>         +{
>         +    uint32_t hash;
>         +
>         +    if (OVS_LIKELY(dp_packet_rss_valid(packet))) {
>         +        hash = dp_packet_get_rss_hash(packet);
>         +    } else {
>         +        hash = miniflow_hash_5tuple(mf, 0);
>         +        dp_packet_set_rss_hash(packet, hash);
>         +    }
>         +
>         +    return hash;
>         +}
>         +
>         +static inline uint32_t
>          dpif_netdev_packet_get_rss_hash(struct dp_packet *packet,
>                                          const struct miniflow *mf)
>          {
>         @@ -4715,7 +4731,6 @@ emc_processing(struct dp_netdev_pmd_thread *pmd,
>                  }
>                  miniflow_extract(packet, &key->mf);
>                  key->len = 0; /* Not computed yet. */
>         -        key->hash = dpif_netdev_packet_get_rss_hash(packet, &key->mf);
> 
>                  /*
>                   * EMC lookup is skipped when one or both of the following
>         @@ -4732,9 +4747,12 @@ emc_processing(struct dp_netdev_pmd_thread *pmd,
>                   */
>                  if (OVS_LIKELY(cur_min)) {
>                      if (!md_is_valid) {
>         +                key->hash =
> dpif_netdev_packet_get_rss_hash_orig_pkt(packet,
>         +                        &key->mf);
>                          flow = emc_lookup(flow_cache, key);
>                      } else {
>                          /* Recirculated packet. */
>         +                key->hash = dpif_netdev_packet_get_rss_hash(packet,
> &key->mf);
>                          if (flow_cache->n_entries &
> EMC_RECIRCT_NO_INSERT_THRESHOLD) {
>                              /* EMC occupancy is over the threshold.  We skip
> EMC
>                               * lookup for recirculated packets. */
>         --
>         2.4.11
> 
>         _______________________________________________
>         dev mailing list
>         dev@openvswitch.org
>         https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__mail.openvswitch.org_mailman_listinfo_ovs-
> 2Ddev&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=Kp92W-
> efwcliK8l_PTI61aY9T1CPqdhzlAxSHGRMad8&s=NdrRKHn6b0aW6Gbj5wOH3_Rkon2dcVgoyPBzfzJ
> s_E4&e=
> 
> 
>     _______________________________________________
>     dev mailing list
>     dev@openvswitch.org
>     https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__mail.openvswitch.org_mailman_listinfo_ovs-
> 2Ddev&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-
> uZnsw&m=vhciWYIB9ZtYxl6vdpUtQE953Hx0s1E0Oz0DFmWOBug&s=10on8gc7nTUk8oyo3jwRYrlyQ
> 6PFB6HSddFXL91iLSQ&e=
>
diff mbox

Patch

=================
Software built with "-O2 -march=native -g".

I measured the Rx rate regardless of pkt loss by sending 1 UDP flow,
64B packets, at the line-rate.

2 PMDs with 3 Tx queues.

Flows setup:
  in_port=dpdk0 actions=output:dpdk1
  in_port=dpdk1 actions=output:dpdk0

Results
-------
Values are for the Rx rate in Mpps, regardless of packet loss.

RSS column:
   Yes: RSS hash is provided by the NIC
   No: RSS is disabled and the 5-tuple hash must be
       computed in software.

EMC column:
   Yes: default probability insertion,
   No: EMC disabled.

Orig OvS-DPDK means Commit ID:
  6b1babacc3ca0488e07596bf822fe356c9bab646

            +--------------+----------------+----------------+
+-----+-----+    Orig      | Orig + patch 1 | Orig + patch 1 |
| RSS | EMC |              |                |  this patch    |
+-----+-----+--------------+----------------+----------------+
| Yes | Yes | 11.99, 11.41 | 12.20, 11.31   | 12.20, 11.31   |
| Yes |  No |  8.32,  8.42 |  8.35,  8.39   |  8.62, 8.62    |
+-----+-----+--------------+----------------+----------------+
|  No | Yes |  9.87, 11.15 |  9.79, 11.20   |  9.85, 11.09   |
|  No |  No |  7.82,  7.84 |  7.84,  7.93   |  8.40,  8.38   |
+-----+-----+--------------+----------------+----------------+
---
 lib/dpif-netdev.c | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
index 8f6b96b..0db6f83 100644
--- a/lib/dpif-netdev.c
+++ b/lib/dpif-netdev.c
@@ -4578,6 +4578,22 @@  dp_netdev_upcall(struct dp_netdev_pmd_thread *pmd, struct dp_packet *packet_,
 }
 
 static inline uint32_t
+dpif_netdev_packet_get_rss_hash_orig_pkt(struct dp_packet *packet,
+                                const struct miniflow *mf)
+{
+    uint32_t hash;
+
+    if (OVS_LIKELY(dp_packet_rss_valid(packet))) {
+        hash = dp_packet_get_rss_hash(packet);
+    } else {
+        hash = miniflow_hash_5tuple(mf, 0);
+        dp_packet_set_rss_hash(packet, hash);
+    }
+
+    return hash;
+}
+
+static inline uint32_t
 dpif_netdev_packet_get_rss_hash(struct dp_packet *packet,
                                 const struct miniflow *mf)
 {
@@ -4715,7 +4731,6 @@  emc_processing(struct dp_netdev_pmd_thread *pmd,
         }
         miniflow_extract(packet, &key->mf);
         key->len = 0; /* Not computed yet. */
-        key->hash = dpif_netdev_packet_get_rss_hash(packet, &key->mf);
 
         /*
          * EMC lookup is skipped when one or both of the following
@@ -4732,9 +4747,12 @@  emc_processing(struct dp_netdev_pmd_thread *pmd,
          */
         if (OVS_LIKELY(cur_min)) {
             if (!md_is_valid) {
+                key->hash = dpif_netdev_packet_get_rss_hash_orig_pkt(packet,
+                        &key->mf);
                 flow = emc_lookup(flow_cache, key);
             } else {
                 /* Recirculated packet. */
+                key->hash = dpif_netdev_packet_get_rss_hash(packet, &key->mf);
                 if (flow_cache->n_entries & EMC_RECIRCT_NO_INSERT_THRESHOLD) {
                     /* EMC occupancy is over the threshold.  We skip EMC
                      * lookup for recirculated packets. */