[ovs-dev,v6,9/9] doc: Add meter offload topic document

Fow now, add introduction and the limitation of meter offload.

Signed-off-by: Jianbo Liu <jianbol@nvidia.com>
---
 Documentation/automake.mk          |  1 +
 Documentation/howto/index.rst      |  1 +
 Documentation/howto/tc-offload.rst | 75 ++++++++++++++++++++++++++++++
 3 files changed, 77 insertions(+)
 create mode 100644 Documentation/howto/tc-offload.rst

On 2 Jul 2022, at 5:18, Jianbo Liu wrote:

> Fow now, add introduction and the limitation of meter offload.
>
> Signed-off-by: Jianbo Liu <jianbol@nvidia.com>
> ---
>  Documentation/automake.mk          |  1 +
>  Documentation/howto/index.rst      |  1 +
>  Documentation/howto/tc-offload.rst | 75 ++++++++++++++++++++++++++++++
>  3 files changed, 77 insertions(+)
>  create mode 100644 Documentation/howto/tc-offload.rst
>
> diff --git a/Documentation/automake.mk b/Documentation/automake.mk
> index 6c2c57739..185349c14 100644
> --- a/Documentation/automake.mk
> +++ b/Documentation/automake.mk
> @@ -79,6 +79,7 @@ DOC_SOURCE = \
>  	Documentation/howto/vlan.png \
>  	Documentation/howto/vlan.rst \
>  	Documentation/howto/vtep.rst \
> +	Documentation/howto/tc-offload.rst \
>  	Documentation/ref/index.rst \
>  	Documentation/faq/index.rst \
>  	Documentation/faq/configuration.rst \
> diff --git a/Documentation/howto/index.rst b/Documentation/howto/index.rst
> index 60fb8a717..1812f6a11 100644
> --- a/Documentation/howto/index.rst
> +++ b/Documentation/howto/index.rst
> @@ -49,4 +49,5 @@ OVS
>     vtep
>     sflow
>     dpdk
> +   tc-offload
>
> diff --git a/Documentation/howto/tc-offload.rst b/Documentation/howto/tc-offload.rst
> new file mode 100644
> index 000000000..09a136264
> --- /dev/null
> +++ b/Documentation/howto/tc-offload.rst
> @@ -0,0 +1,75 @@
> +..
> +      Licensed under the Apache License, Version 2.0 (the "License"); you may
> +      not use this file except in compliance with the License. You may obtain
> +      a copy of the License at
> +
> +          http://www.apache.org/licenses/LICENSE-2.0
> +
> +      Unless required by applicable law or agreed to in writing, software
> +      distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
> +      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
> +      License for the specific language governing permissions and limitations
> +      under the License.
> +
> +      Convention for heading levels in Open vSwitch documentation:
> +
> +      =======  Heading 0 (reserved for the title in a document)
> +      -------  Heading 1
> +      ~~~~~~~  Heading 2
> +      +++++++  Heading 3
> +      '''''''  Heading 4
> +
> +      Avoid deeper levels because they do not render well.
> +
> +=====================================
> +Hardware offload with Linux TC flower
> +=====================================
> +
> +This document describes how to offload with TC flower. It is disabled by
> +default and can be enabled by::
> +


How about re-arranging the layout a bit. For example:

==========================================
Flow Hardware offload with Linux TC flower
==========================================

This document describes how to offload flows with TC flower.

Flow Hardware Offload
---------------------

The flow hardware offload is disabled by default and can be enabled by::

     $ ovs-vsctl set Open_vSwitch . other_config:hw-offload=true

TC flower has one additional configuration option caled ``tc-policy``.
For more details see ``man ovs-vswitchd.conf.db``.

> +     $ ovs-vsctl set Open_vSwitch . other_config:hw-offload=true
> +
> +Offload meter to TC
> +-------------------
> +
> +After hardware offload is enabled, meter offload is enabled as well.
> +

What about:

TC Meter Offload
----------------

Offloading meters to TC does not require any additional configuration and is
enabled automatically when possible. Offloading with meters does require the
tc-police action to be available in the Linux kernel. For more details on the
tc-police action, see ``man tc-police``.


> +Configuration
> +~~~~~~~~~~~~~
> +
> +There is no parameter change in ovs-ofctl command, to configue a meter and use
> +it for a flow in the offload way. Usually the commands are like::
> +
> +     $ ovs-ofctl -O OpenFlow13 add-meter br0 "meter=1 pktps bands=type=drop rate=1"
> +     $ ovs-ofctl -O OpenFlow13 add-flow br0 "priority=10,in_port=ovs-p0,udp actions=meter:1,normal"
> +

+
+ For more details, see ``man ovs-ofctl``.


> +.. note::
> +  Each meter is mapped to one TC police action. To avovid the conflicton, the
> +  police action index of 0x10000000-0x1fffffff are resevered for the mapping.
> +  You can check the police actions by the command "tc action ls action police"
> +  in Linux system.

``tc action ls action police``

> +
> +Dump meter statistics
> +---------------------
> +
> +You can use ovs-ofctl to display the statisticsof meter::
> +
> +     $ ovs-ofctl -O OpenFlow13 meter-stats br0
> +     OFPST_METER reply (OF1.3) (xid=0x2):
> +     meter:1 flow_count:1 packet_in_count:11 byte_in_count:377 duration:3.199s bands:
> +     0: packet_count:9 byte_count:0
> +
> +The stats of the dropped packets is stored to band 0.
> +
> +.. note::
> +  Meter in OVS DP is not synced with the mapped poloce action, and the first
> +  packet for DP learning does not pass TC police action, as the rule is not
> +  offloaded yet. So, you may notice that packet_count of band 0 (for dropped
> +  packets) is counted after the rule is offoalded, while packet_in_count is
> +  summed with the numbers of packets of both DP and TC police action.
> +
> +.. note::
> +  Linux TC does not support the dropped bytes count, so the byte_count of band
> +  0 is always 0 if meter is offloaded.
> +

I would remove the Dump meter statistics section and replace it with a known limitations section, i.e.:

Known TC flow offload limitations
---------------------------------

General
~~~~~~~

These sections describe limitations to the general TC flow offload
implementation.

Flow bytes count
++++++++++++++++

Flows that are offloaded with TC do not include the L2 bytes in the packet
byte count. Take the datapath flow dump below as an example. The first one
is from the none-offloaded case the second one is from a TC offloaded flow::

    in_port(2),eth(macs),eth_type(0x0800),ipv4(proto=17,frag=no), packets:10, bytes:470, used:0.001s, actions:outputmeter(0),3

    in_port(2),eth(macs),eth_type(0x0800),ipv4(proto=17,frag=no), packets:10, bytes:330, used:0.001s, actions:outputmeter(0),3

As you can see above the none-offload case reports 140 bytes more, which is 14
bytes per packet. This represents the L2 header, in this case, 2 * *Ethernet
address* + *Ethertype*.

TC Meter Offload
~~~~~~~~~~~~~~~~

These sections describe limitations related to the TC meter offload
implementation.

Missing byte count drop statistics
++++++++++++++++++++++++++++++++++

The kernel's TC infrastructure is only counting the number of dropped packet,
not their byte size. This results in the meter statistics always showing 0
for byte_count. Here is an example::

    $ ovs-ofctl -O OpenFlow13 meter-stats br0
    OFPST_METER reply (OF1.3) (xid=0x2):
    meter:1 flow_count:1 packet_in_count:11 byte_in_count:377 duration:3.199s bands:
    0: packet_count:9 byte_count:0

First flow packet not processed by meter
++++++++++++++++++++++++++++++++++++++++

Packets that are received by ovs-vswitchd through an upcall before the actual
meter flow is installed, are not passing TC police action and therefore are
not considered for policing.



Just to be compete here is a single diff with all my changes on top of you patch:


diff --git a/Documentation/howto/tc-offload.rst b/Documentation/howto/tc-offload.rst
index 09a136264..3105c8d21 100644
--- a/Documentation/howto/tc-offload.rst
+++ b/Documentation/howto/tc-offload.rst
@@ -21,19 +21,30 @@

       Avoid deeper levels because they do not render well.

-=====================================
-Hardware offload with Linux TC flower
-=====================================
+==========================================
+Flow Hardware offload with Linux TC flower
+==========================================

-This document describes how to offload with TC flower. It is disabled by
-default and can be enabled by::
+This document describes how to offload flows with TC flower.

-     $ ovs-vsctl set Open_vSwitch . other_config:hw-offload=true
+Flow Hardware Offload
+---------------------
+
+The flow hardware offload is disabled by default and can be enabled by::
+
+    $ ovs-vsctl set Open_vSwitch . other_config:hw-offload=true
+
+TC flower has one additional configuration option caled ``tc-policy``.
+For more details see ``man ovs-vswitchd.conf.db``.
+
+TC Meter Offload
+----------------

-Offload meter to TC
--------------------
+Offloading meters to TC does not require any additional configuration and is
+enabled automatically when possible. Offloading with meters does require the
+tc-police action to be available in the Linux kernel. For more details on the
+tc-police action, see ``man tc-police``.

-After hardware offload is enabled, meter offload is enabled as well.

 Configuration
 ~~~~~~~~~~~~~
@@ -41,35 +52,63 @@ Configuration
 There is no parameter change in ovs-ofctl command, to configue a meter and use
 it for a flow in the offload way. Usually the commands are like::

-     $ ovs-ofctl -O OpenFlow13 add-meter br0 "meter=1 pktps bands=type=drop rate=1"
-     $ ovs-ofctl -O OpenFlow13 add-flow br0 "priority=10,in_port=ovs-p0,udp actions=meter:1,normal"
+    $ ovs-ofctl -O OpenFlow13 add-meter br0 "meter=1 pktps bands=type=drop rate=1"
+    $ ovs-ofctl -O OpenFlow13 add-flow br0 "priority=10,in_port=ovs-p0,udp actions=meter:1,normal"
+
+For more details, see ``man ovs-ofctl``.

 .. note::
   Each meter is mapped to one TC police action. To avovid the conflicton, the
   police action index of 0x10000000-0x1fffffff are resevered for the mapping.
-  You can check the police actions by the command "tc action ls action police"
+  You can check the police actions by the command ``tc action ls action police``
   in Linux system.

-Dump meter statistics
----------------------

-You can use ovs-ofctl to display the statisticsof meter::
+Known TC flow offload limitations
+---------------------------------

-     $ ovs-ofctl -O OpenFlow13 meter-stats br0
-     OFPST_METER reply (OF1.3) (xid=0x2):
-     meter:1 flow_count:1 packet_in_count:11 byte_in_count:377 duration:3.199s bands:
-     0: packet_count:9 byte_count:0
+General
+~~~~~~~

-The stats of the dropped packets is stored to band 0.
+These sections describe limitations to the general TC flow offload
+implementation.

-.. note::
-  Meter in OVS DP is not synced with the mapped poloce action, and the first
-  packet for DP learning does not pass TC police action, as the rule is not
-  offloaded yet. So, you may notice that packet_count of band 0 (for dropped
-  packets) is counted after the rule is offoalded, while packet_in_count is
-  summed with the numbers of packets of both DP and TC police action.
+Flow bytes count
+++++++++++++++++

-.. note::
-  Linux TC does not support the dropped bytes count, so the byte_count of band
-  0 is always 0 if meter is offloaded.
+Flows that are offloaded with TC do not include the L2 bytes in the packet
+byte count. Take the datapath flow dump below as an example. The first one
+is from the none-offloaded case the second one is from a TC offloaded flow::
+
+    in_port(2),eth(macs),eth_type(0x0800),ipv4(proto=17,frag=no), packets:10, bytes:470, used:0.001s, actions:outputmeter(0),3
+
+    in_port(2),eth(macs),eth_type(0x0800),ipv4(proto=17,frag=no), packets:10, bytes:330, used:0.001s, actions:outputmeter(0),3
+
+As you can see above the none-offload case reports 140 bytes more, which is 14
+bytes per packet. This represents the L2 header, in this case, 2 * *Ethernet
+address* + *Ethertype*.
+
+TC Meter Offload
+~~~~~~~~~~~~~~~~
+
+These sections describe limitations related to the TC meter offload
+implementation.
+
+Missing byte count drop statistics
+++++++++++++++++++++++++++++++++++
+
+The kernel's TC infrastructure is only counting the number of dropped packet,
+not their byte size. This results in the meter statistics always showing 0
+for byte_count. Here is an example::
+
+    $ ovs-ofctl -O OpenFlow13 meter-stats br0
+    OFPST_METER reply (OF1.3) (xid=0x2):
+    meter:1 flow_count:1 packet_in_count:11 byte_in_count:377 duration:3.199s bands:
+    0: packet_count:9 byte_count:0
+
+First flow packet not processed by meter
+++++++++++++++++++++++++++++++++++++++++

+Packets that are received by ovs-vswitchd through an upcall before the actual
+meter flow is installed, are not passing TC police action and therefore are
+not considered for policing.

On Thu, 2022-07-07 at 13:53 +0200, Eelco Chaudron wrote:
> On 2 Jul 2022, at 5:18, Jianbo Liu wrote:
> 
> > Fow now, add introduction and the limitation of meter offload.
> > 
> > Signed-off-by: Jianbo Liu <jianbol@nvidia.com>
> > ---
> >  Documentation/automake.mk          |  1 +
> >  Documentation/howto/index.rst      |  1 +
> >  Documentation/howto/tc-offload.rst | 75
> > ++++++++++++++++++++++++++++++
> >  3 files changed, 77 insertions(+)
> >  create mode 100644 Documentation/howto/tc-offload.rst
> > 
> > diff --git a/Documentation/automake.mk b/Documentation/automake.mk
> > index 6c2c57739..185349c14 100644
> > --- a/Documentation/automake.mk
> > +++ b/Documentation/automake.mk
> > @@ -79,6 +79,7 @@ DOC_SOURCE = \
> >         Documentation/howto/vlan.png \
> >         Documentation/howto/vlan.rst \
> >         Documentation/howto/vtep.rst \
> > +       Documentation/howto/tc-offload.rst \
> >         Documentation/ref/index.rst \
> >         Documentation/faq/index.rst \
> >         Documentation/faq/configuration.rst \
> > diff --git a/Documentation/howto/index.rst
> > b/Documentation/howto/index.rst
> > index 60fb8a717..1812f6a11 100644
> > --- a/Documentation/howto/index.rst
> > +++ b/Documentation/howto/index.rst
> > @@ -49,4 +49,5 @@ OVS
> >     vtep
> >     sflow
> >     dpdk
> > +   tc-offload
> > 
> > diff --git a/Documentation/howto/tc-offload.rst
> > b/Documentation/howto/tc-offload.rst
> > new file mode 100644
> > index 000000000..09a136264
> > --- /dev/null
> > +++ b/Documentation/howto/tc-offload.rst
> > @@ -0,0 +1,75 @@
> > +..
> > +      Licensed under the Apache License, Version 2.0 (the
> > "License"); you may
> > +      not use this file except in compliance with the License. You
> > may obtain
> > +      a copy of the License at
> > +
> > +          http://www.apache.org/licenses/LICENSE-2.0
> > +
> > +      Unless required by applicable law or agreed to in writing,
> > software
> > +      distributed under the License is distributed on an "AS IS"
> > BASIS, WITHOUT
> > +      WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> > implied. See the
> > +      License for the specific language governing permissions and
> > limitations
> > +      under the License.
> > +
> > +      Convention for heading levels in Open vSwitch documentation:
> > +
> > +      =======  Heading 0 (reserved for the title in a document)
> > +      -------  Heading 1
> > +      ~~~~~~~  Heading 2
> > +      +++++++  Heading 3
> > +      '''''''  Heading 4
> > +
> > +      Avoid deeper levels because they do not render well.
> > +
> > +=====================================
> > +Hardware offload with Linux TC flower
> > +=====================================
> > +
> > +This document describes how to offload with TC flower. It is
> > disabled by
> > +default and can be enabled by::
> > +
> 
> 
> How about re-arranging the layout a bit. For example:
> 
> ==========================================
> Flow Hardware offload with Linux TC flower
> ==========================================
> 
> This document describes how to offload flows with TC flower.
> 
> Flow Hardware Offload
> ---------------------
> 
> The flow hardware offload is disabled by default and can be enabled
> by::
> 
>      $ ovs-vsctl set Open_vSwitch . other_config:hw-offload=true
> 
> TC flower has one additional configuration option caled ``tc-
> policy``.
> For more details see ``man ovs-vswitchd.conf.db``.
> 
> > +     $ ovs-vsctl set Open_vSwitch . other_config:hw-offload=true
> > +
> > +Offload meter to TC
> > +-------------------
> > +
> > +After hardware offload is enabled, meter offload is enabled as
> > well.
> > +
> 
> What about:
> 
> TC Meter Offload
> ----------------
> 
> Offloading meters to TC does not require any additional configuration
> and is
> enabled automatically when possible. Offloading with meters does
> require the
> tc-police action to be available in the Linux kernel. For more
> details on the
> tc-police action, see ``man tc-police``.
> 
> 
> > +Configuration
> > +~~~~~~~~~~~~~
> > +
> > +There is no parameter change in ovs-ofctl command, to configue a
> > meter and use
> > +it for a flow in the offload way. Usually the commands are like::
> > +
> > +     $ ovs-ofctl -O OpenFlow13 add-meter br0 "meter=1 pktps
> > bands=type=drop rate=1"
> > +     $ ovs-ofctl -O OpenFlow13 add-flow br0
> > "priority=10,in_port=ovs-p0,udp actions=meter:1,normal"
> > +
> 
> +
> + For more details, see ``man ovs-ofctl``.
> 
> 
> > +.. note::
> > +  Each meter is mapped to one TC police action. To avovid the
> > conflicton, the
> > +  police action index of 0x10000000-0x1fffffff are resevered for
> > the mapping.
> > +  You can check the police actions by the command "tc action ls
> > action police"
> > +  in Linux system.
> 
> ``tc action ls action police``
> 
> > +
> > +Dump meter statistics
> > +---------------------
> > +
> > +You can use ovs-ofctl to display the statisticsof meter::
> > +
> > +     $ ovs-ofctl -O OpenFlow13 meter-stats br0
> > +     OFPST_METER reply (OF1.3) (xid=0x2):
> > +     meter:1 flow_count:1 packet_in_count:11 byte_in_count:377
> > duration:3.199s bands:
> > +     0: packet_count:9 byte_count:0
> > +
> > +The stats of the dropped packets is stored to band 0.
> > +
> > +.. note::
> > +  Meter in OVS DP is not synced with the mapped poloce action, and
> > the first
> > +  packet for DP learning does not pass TC police action, as the
> > rule is not
> > +  offloaded yet. So, you may notice that packet_count of band 0
> > (for dropped
> > +  packets) is counted after the rule is offoalded, while
> > packet_in_count is
> > +  summed with the numbers of packets of both DP and TC police
> > action.
> > +
> > +.. note::
> > +  Linux TC does not support the dropped bytes count, so the
> > byte_count of band
> > +  0 is always 0 if meter is offloaded.
> > +
> 
> I would remove the Dump meter statistics section and replace it with
> a known limitations section, i.e.:
> 
> Known TC flow offload limitations
> ---------------------------------
> 
> General
> ~~~~~~~
> 
> These sections describe limitations to the general TC flow offload
> implementation.
> 
> Flow bytes count
> ++++++++++++++++
> 
> Flows that are offloaded with TC do not include the L2 bytes in the
> packet
> byte count. Take the datapath flow dump below as an example. The
> first one
> is from the none-offloaded case the second one is from a TC offloaded
> flow::
> 
>     in_port(2),eth(macs),eth_type(0x0800),ipv4(proto=17,frag=no),
> packets:10, bytes:470, used:0.001s, actions:outputmeter(0),3
> 
>     in_port(2),eth(macs),eth_type(0x0800),ipv4(proto=17,frag=no),
> packets:10, bytes:330, used:0.001s, actions:outputmeter(0),3
> 
> As you can see above the none-offload case reports 140 bytes more,
> which is 14
> bytes per packet. This represents the L2 header, in this case, 2 *
> *Ethernet
> address* + *Ethertype*.
> 
> TC Meter Offload
> ~~~~~~~~~~~~~~~~
> 
> These sections describe limitations related to the TC meter offload
> implementation.
> 
> Missing byte count drop statistics
> ++++++++++++++++++++++++++++++++++
> 
> The kernel's TC infrastructure is only counting the number of dropped
> packet,
> not their byte size. This results in the meter statistics always
> showing 0
> for byte_count. Here is an example::
> 
>     $ ovs-ofctl -O OpenFlow13 meter-stats br0
>     OFPST_METER reply (OF1.3) (xid=0x2):
>     meter:1 flow_count:1 packet_in_count:11 byte_in_count:377
> duration:3.199s bands:
>     0: packet_count:9 byte_count:0
> 
> First flow packet not processed by meter
> ++++++++++++++++++++++++++++++++++++++++
> 
> Packets that are received by ovs-vswitchd through an upcall before
> the actual
> meter flow is installed, are not passing TC police action and
> therefore are
> not considered for policing.
> 
> 
> 
> Just to be compete here is a single diff with all my changes on top
> of you patch:

I completely agree with your changes. It help me a lot.
Thanks you very much!

> 
> 
> diff --git a/Documentation/howto/tc-offload.rst
> b/Documentation/howto/tc-offload.rst
> index 09a136264..3105c8d21 100644
> --- a/Documentation/howto/tc-offload.rst
> +++ b/Documentation/howto/tc-offload.rst
> @@ -21,19 +21,30 @@
> 
>        Avoid deeper levels because they do not render well.
> 
> -=====================================
> -Hardware offload with Linux TC flower
> -=====================================
> +==========================================
> +Flow Hardware offload with Linux TC flower
> +==========================================
> 
> -This document describes how to offload with TC flower. It is
> disabled by
> -default and can be enabled by::
> +This document describes how to offload flows with TC flower.
> 
> -     $ ovs-vsctl set Open_vSwitch . other_config:hw-offload=true
> +Flow Hardware Offload
> +---------------------
> +
> +The flow hardware offload is disabled by default and can be enabled
> by::
> +
> +    $ ovs-vsctl set Open_vSwitch . other_config:hw-offload=true
> +
> +TC flower has one additional configuration option caled ``tc-
> policy``.
> +For more details see ``man ovs-vswitchd.conf.db``.
> +
> +TC Meter Offload
> +----------------
> 
> -Offload meter to TC
> --------------------
> +Offloading meters to TC does not require any additional
> configuration and is
> +enabled automatically when possible. Offloading with meters does
> require the
> +tc-police action to be available in the Linux kernel. For more
> details on the
> +tc-police action, see ``man tc-police``.
> 
> -After hardware offload is enabled, meter offload is enabled as well.
> 
>  Configuration
>  ~~~~~~~~~~~~~
> @@ -41,35 +52,63 @@ Configuration
>  There is no parameter change in ovs-ofctl command, to configue a
> meter and use
>  it for a flow in the offload way. Usually the commands are like::
> 
> -     $ ovs-ofctl -O OpenFlow13 add-meter br0 "meter=1 pktps
> bands=type=drop rate=1"
> -     $ ovs-ofctl -O OpenFlow13 add-flow br0
> "priority=10,in_port=ovs-p0,udp actions=meter:1,normal"
> +    $ ovs-ofctl -O OpenFlow13 add-meter br0 "meter=1 pktps
> bands=type=drop rate=1"
> +    $ ovs-ofctl -O OpenFlow13 add-flow br0 "priority=10,in_port=ovs-
> p0,udp actions=meter:1,normal"
> +
> +For more details, see ``man ovs-ofctl``.
> 
>  .. note::
>    Each meter is mapped to one TC police action. To avovid the
> conflicton, the
>    police action index of 0x10000000-0x1fffffff are resevered for the
> mapping.
> -  You can check the police actions by the command "tc action ls
> action police"
> +  You can check the police actions by the command ``tc action ls
> action police``
>    in Linux system.
> 
> -Dump meter statistics
> ----------------------
> 
> -You can use ovs-ofctl to display the statisticsof meter::
> +Known TC flow offload limitations
> +---------------------------------
> 
> -     $ ovs-ofctl -O OpenFlow13 meter-stats br0
> -     OFPST_METER reply (OF1.3) (xid=0x2):
> -     meter:1 flow_count:1 packet_in_count:11 byte_in_count:377
> duration:3.199s bands:
> -     0: packet_count:9 byte_count:0
> +General
> +~~~~~~~
> 
> -The stats of the dropped packets is stored to band 0.
> +These sections describe limitations to the general TC flow offload
> +implementation.
> 
> -.. note::
> -  Meter in OVS DP is not synced with the mapped poloce action, and
> the first
> -  packet for DP learning does not pass TC police action, as the rule
> is not
> -  offloaded yet. So, you may notice that packet_count of band 0 (for
> dropped
> -  packets) is counted after the rule is offoalded, while
> packet_in_count is
> -  summed with the numbers of packets of both DP and TC police
> action.
> +Flow bytes count
> +++++++++++++++++
> 
> -.. note::
> -  Linux TC does not support the dropped bytes count, so the
> byte_count of band
> -  0 is always 0 if meter is offloaded.
> +Flows that are offloaded with TC do not include the L2 bytes in the
> packet
> +byte count. Take the datapath flow dump below as an example. The
> first one
> +is from the none-offloaded case the second one is from a TC
> offloaded flow::
> +
> +    in_port(2),eth(macs),eth_type(0x0800),ipv4(proto=17,frag=no),
> packets:10, bytes:470, used:0.001s, actions:outputmeter(0),3
> +
> +    in_port(2),eth(macs),eth_type(0x0800),ipv4(proto=17,frag=no),
> packets:10, bytes:330, used:0.001s, actions:outputmeter(0),3
> +
> +As you can see above the none-offload case reports 140 bytes more,
> which is 14
> +bytes per packet. This represents the L2 header, in this case, 2 *
> *Ethernet
> +address* + *Ethertype*.
> +
> +TC Meter Offload
> +~~~~~~~~~~~~~~~~
> +
> +These sections describe limitations related to the TC meter offload
> +implementation.
> +
> +Missing byte count drop statistics
> +++++++++++++++++++++++++++++++++++
> +
> +The kernel's TC infrastructure is only counting the number of
> dropped packet,
> +not their byte size. This results in the meter statistics always
> showing 0
> +for byte_count. Here is an example::
> +
> +    $ ovs-ofctl -O OpenFlow13 meter-stats br0
> +    OFPST_METER reply (OF1.3) (xid=0x2):
> +    meter:1 flow_count:1 packet_in_count:11 byte_in_count:377
> duration:3.199s bands:
> +    0: packet_count:9 byte_count:0
> +
> +First flow packet not processed by meter
> +++++++++++++++++++++++++++++++++++++++++
> 
> +Packets that are received by ovs-vswitchd through an upcall before
> the actual
> +meter flow is installed, are not passing TC police action and
> therefore are
> +not considered for policing.
>