[net-next,v2] net: bcmgenet: collect Rx discarded packet count

Bits 31:16 of RDMA_PROD_INDEX contain Rx discarded packet count, which
are the Rx packets that had to be dropped by MAC hardware since there
was no room on the Rx queue. Add code to collect this information into
the netdev stats.

Signed-off-by: Petri Gynther <pgynther@google.com>
---
 drivers/net/ethernet/broadcom/genet/bcmgenet.c | 18 ++++++++++++++++++
 drivers/net/ethernet/broadcom/genet/bcmgenet.h |  1 +
 2 files changed, 19 insertions(+)

2015-03-10 15:55 GMT-07:00 Petri Gynther <pgynther@google.com>:
> Bits 31:16 of RDMA_PROD_INDEX contain Rx discarded packet count, which
> are the Rx packets that had to be dropped by MAC hardware since there
> was no room on the Rx queue. Add code to collect this information into
> the netdev stats.
>
> Signed-off-by: Petri Gynther <pgynther@google.com>
> ---
>  drivers/net/ethernet/broadcom/genet/bcmgenet.c | 18 ++++++++++++++++++
>  drivers/net/ethernet/broadcom/genet/bcmgenet.h |  1 +
>  2 files changed, 19 insertions(+)
>
> diff --git a/drivers/net/ethernet/broadcom/genet/bcmgenet.c b/drivers/net/ethernet/broadcom/genet/bcmgenet.c
> index 275be56..d3be1aeb 100644
> --- a/drivers/net/ethernet/broadcom/genet/bcmgenet.c
> +++ b/drivers/net/ethernet/broadcom/genet/bcmgenet.c
> @@ -1384,9 +1384,27 @@ static unsigned int bcmgenet_desc_rx(struct bcmgenet_priv *priv,
>         int len, err;
>         unsigned int rxpktprocessed = 0, rxpkttoprocess;
>         unsigned int p_index;
> +       unsigned int discards;
>         unsigned int chksum_ok = 0;
>
>         p_index = bcmgenet_rdma_ring_readl(priv, index, RDMA_PROD_INDEX);
> +
> +       discards = (p_index >> DMA_P_INDEX_DISCARD_CNT_SHIFT) &
> +                  DMA_P_INDEX_DISCARD_CNT_MASK;
> +       if (discards > ring->old_discards) {
> +               discards = discards - ring->old_discards;
> +               dev->stats.rx_missed_errors += discards;
> +               dev->stats.rx_errors += discards;
> +               ring->old_discards += discards;
> +
> +               /* Clear HW register when we reach 75% of maximum 0xFFFF */
> +               if (ring->old_discards >= 0xC000) {
> +                       ring->old_discards = 0;
> +                       bcmgenet_rdma_ring_writel(priv, index, 0,
> +                                                 RDMA_PROD_INDEX);
> +               }
> +       }

So this looks better, I am wondering if we should not actually do this
in the case where we run out of memory, because this is pretty much
when the discard counter is likely to be increased?

> +
>         p_index &= DMA_P_INDEX_MASK;
>
>         if (likely(p_index >= ring->c_index))
> diff --git a/drivers/net/ethernet/broadcom/genet/bcmgenet.h b/drivers/net/ethernet/broadcom/genet/bcmgenet.h
> index 17443db..2a81138 100644
> --- a/drivers/net/ethernet/broadcom/genet/bcmgenet.h
> +++ b/drivers/net/ethernet/broadcom/genet/bcmgenet.h
> @@ -548,6 +548,7 @@ struct bcmgenet_rx_ring {
>         unsigned int    read_ptr;       /* Rx ring read pointer */
>         unsigned int    cb_ptr;         /* Rx ring initial CB ptr */
>         unsigned int    end_ptr;        /* Rx ring end CB ptr */
> +       unsigned int    old_discards;
>  };
>
>  /* device context */
> --
> 2.2.0.rc0.207.ga3a616c
>

I find the Rx discard counter very useful, as it indicates HW dropped
packets, which directly affects media playback quality.

In our unit tests, we actually discovered a major interrupts-off
offender with this. Even with a moderate Rx traffic, we were seeing Rx
discards. It turned out that using a large dmesg log buffer and
dumping it with "cat /proc/kmsg" held interrupts off for long enough
that bcmgenet experienced discards. It was very useful to have the
discard data available when debugging this.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

2015-03-11 10:59 GMT-07:00 Petri Gynther <pgynther@google.com>:
> I find the Rx discard counter very useful, as it indicates HW dropped
> packets, which directly affects media playback quality.

I do not question the usefulness of that counter, I am wondering about
whether we could make sure we deal with this counter in a slow/error
path, I guess the answer to my previous question is no, because having
interrupts off for too long does not mean the CPU ran out of memory
for incoming packets.

In  that case, this patch is fine as-is.

>
> In our unit tests, we actually discovered a major interrupts-off
> offender with this. Even with a moderate Rx traffic, we were seeing Rx
> discards. It turned out that using a large dmesg log buffer and
> dumping it with "cat /proc/kmsg" held interrupts off for long enough
> that bcmgenet experienced discards. It was very useful to have the
> discard data available when debugging this.

That is completely fair, it is just that I did a lot of performance
optimization before, and any expensive register in the hot-path is a
big no-no because that impacts your small packet performance (and thus
larger packet performance as well).

From: Petri Gynther <pgynther@google.com>
Date: Tue, 10 Mar 2015 15:55:00 -0700 (PDT)

> Bits 31:16 of RDMA_PROD_INDEX contain Rx discarded packet count, which
> are the Rx packets that had to be dropped by MAC hardware since there
> was no room on the Rx queue. Add code to collect this information into
> the netdev stats.
> 
> Signed-off-by: Petri Gynther <pgynther@google.com>

Applied, thanks.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html