[ovs-dev,v4,1/2] Encap & Decap actions for MPLS packet type.

From: Martin Varghese <martin.varghese@nokia.com>

The encap & decap actions are extended to support MPLS packet type.
Encap & decap actions adds and removes MPLS header at start of the
packet.

Signed-off-by: Martin Varghese <martin.varghese@nokia.com>
---
 NEWS                                          |  2 +-
 .../linux/compat/include/linux/openvswitch.h  | 35 ++++++-
 include/openvswitch/ofp-ed-props.h            | 18 ++++
 lib/dpif-netdev.c                             |  1 +
 lib/dpif.c                                    |  1 +
 lib/odp-execute.c                             | 12 +++
 lib/odp-util.c                                | 58 +++++++++---
 lib/ofp-actions.c                             |  5 +
 lib/ofp-ed-props.c                            | 91 +++++++++++++++++++
 lib/ovs-actions.xml                           | 31 +++++--
 lib/packets.c                                 | 36 ++++++++
 lib/packets.h                                 |  2 +
 ofproto/ofproto-dpif-ipfix.c                  |  1 +
 ofproto/ofproto-dpif-sflow.c                  |  1 +
 ofproto/ofproto-dpif-xlate.c                  | 60 ++++++++++++
 ofproto/ofproto-dpif.c                        | 39 ++++++++
 ofproto/ofproto-dpif.h                        |  5 +-
 tests/system-traffic.at                       | 36 ++++++++
 18 files changed, 410 insertions(+), 24 deletions(-)

On 26 Mar 2021, at 7:21, Martin Varghese wrote:

> From: Martin Varghese <martin.varghese@nokia.com>
>
> The encap & decap actions are extended to support MPLS packet type.
> Encap & decap actions adds and removes MPLS header at start of the
> packet.

Hi Martin,

I’m trying to do some real-life testing, and I’m running into 
issues. This might be me setting it up wrongly but just wanting to 
confirm…

I’m sending an MPLS packet that contains an ARP packet into a physical 
port. This is the packet:

Frame 4: 64 bytes on wire (512 bits), 64 bytes captured (512 bits)
     Encapsulation type: Ethernet (1)
     [Protocols in frame: eth:ethertype:mpls:data]
Ethernet II, Src: 00:00:00_00:00:01 (00:00:00:00:00:01), Dst: 
00:00:00_00:00:02 (00:00:00:00:00:02)
     Destination: 00:00:00_00:00:02 (00:00:00:00:00:02)
         Address: 00:00:00_00:00:02 (00:00:00:00:00:02)
         .... ..0. .... .... .... .... = LG bit: Globally unique address 
(factory default)
         .... ...0 .... .... .... .... = IG bit: Individual address 
(unicast)
     Source: 00:00:00_00:00:01 (00:00:00:00:00:01)
         Address: 00:00:00_00:00:01 (00:00:00:00:00:01)
         .... ..0. .... .... .... .... = LG bit: Globally unique address 
(factory default)
         .... ...0 .... .... .... .... = IG bit: Individual address 
(unicast)
     Type: MPLS label switched packet (0x8847)
MultiProtocol Label Switching Header, Label: 100, Exp: 0, S: 1, TTL: 64
     0000 0000 0000 0110 0100 .... .... .... = MPLS Label: 100
     .... .... .... .... .... 000. .... .... = MPLS Experimental Bits: 0
     .... .... .... .... .... ...1 .... .... = MPLS Bottom Of Label 
Stack: 1
     .... .... .... .... .... .... 0100 0000 = MPLS TTL: 64
Data (46 bytes)

0000  ff ff ff ff ff ff 52 54 00 88 51 38 08 06 00 01   ......RT..Q8....
0010  08 00 06 04 00 01 52 54 00 88 51 38 01 01 01 65   ......RT..Q8...e
0020  00 00 00 00 00 00 01 01 01 64 27 98 a0 47         .........d'..G
     Data: 
ffffffffffff525400885138080600010800060400015254008851380101016500000000?


I’m trying to use the following rules:

   ovs-ofctl del-flows ovs_pvp_br0
   ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0 
"priority=100,dl_type=0x8847,mpls_label=100 
actions=decap(),decap(packet_type(ns=0,type=0x806)),resubmit(,3)"
   ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0 "table=3,priority=10 
actions=normal"

With these, I expect the packet to be sent to vnet0, but it’s not. 
Actually, the datapath rule looks odd, while the userspace rules seem to 
match:

   $ ovs-dpctl dump-flows
   recirc_id(0),in_port(1),eth(),eth_type(0x8847),mpls(label=100/0xfffff,tc=0/0,ttl=0/0x0,bos=1/1), 
packets:13, bytes:1118, used:0.322s, 
actions:pop_eth,pop_mpls(eth_type=0x806),recirc(0x19a)
   recirc_id(0x19a),in_port(1),eth_type(0x0806), packets:13, bytes:884, 
used:0.322s, actions:drop

   $ ovs-ofctl dump-flows ovs_pvp_br0 -O OpenFlow13
   cookie=0x0, duration=85.007s, table=0, n_packets=51, n_bytes=4386, 
priority=100,mpls,mpls_label=100 
actions=decap(),decap(packet_type(ns=0,type=0x806)),resubmit(,3)
   cookie=0x0, duration=84.990s, table=3, n_packets=51, n_bytes=3468, 
priority=10 actions=NORMAL


If I use the old way, doing pop_mpls, it works fine:


ovs-ofctl del-flows ovs_pvp_br0
ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0 
"priority=100,dl_type=0x8847,mpls_label=100 
actions=pop_mpls:0x0806,resubmit(,3)"
ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0 "table=3,priority=10 
actions=normal"


I also noticed (despite the test example) to make encap work, I had to 
set the ethernet MAC addresses, or else the packets were not getting 
out.
So something like:

   ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0 
"priority=100,in_port=vnet0,actions=encap(mpls(ether_type=0x8847)),set_mpls_label:100,encap(ethernet),,set_field:00:00:00:00:00:02->dl_dst,set_field:00:00:00:00:00:01->dl_src,output:enp5s0f0

Maybe the test case can be made more realistic? Once I understand the 
failure, I can continue with the review.


Cheers,

Eelco



> Signed-off-by: Martin Varghese <martin.varghese@nokia.com>
> ---
>  NEWS                                          |  2 +-
>  .../linux/compat/include/linux/openvswitch.h  | 35 ++++++-
>  include/openvswitch/ofp-ed-props.h            | 18 ++++
>  lib/dpif-netdev.c                             |  1 +
>  lib/dpif.c                                    |  1 +
>  lib/odp-execute.c                             | 12 +++
>  lib/odp-util.c                                | 58 +++++++++---
>  lib/ofp-actions.c                             |  5 +
>  lib/ofp-ed-props.c                            | 91 
> +++++++++++++++++++
>  lib/ovs-actions.xml                           | 31 +++++--
>  lib/packets.c                                 | 36 ++++++++
>  lib/packets.h                                 |  2 +
>  ofproto/ofproto-dpif-ipfix.c                  |  1 +
>  ofproto/ofproto-dpif-sflow.c                  |  1 +
>  ofproto/ofproto-dpif-xlate.c                  | 60 ++++++++++++
>  ofproto/ofproto-dpif.c                        | 39 ++++++++
>  ofproto/ofproto-dpif.h                        |  5 +-
>  tests/system-traffic.at                       | 36 ++++++++
>  18 files changed, 410 insertions(+), 24 deletions(-)
>
> diff --git a/NEWS b/NEWS
> index 95cf922aa..4bf4e9e7b 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -120,7 +120,7 @@ v2.14.0 - 17 Aug 2020
>     - GTP-U Tunnel Protocol
>       * Add two new fields: tun_gtpu_flags, tun_gtpu_msgtype.
>       * Only support for userspace datapath.
> -
> +   - Encap & Decap action support for MPLS packet type.
>
>  v2.13.0 - 14 Feb 2020
>  ---------------------
> diff --git a/datapath/linux/compat/include/linux/openvswitch.h 
> b/datapath/linux/compat/include/linux/openvswitch.h
> index 875de2025..8feea7dd4 100644
> --- a/datapath/linux/compat/include/linux/openvswitch.h
> +++ b/datapath/linux/compat/include/linux/openvswitch.h
> @@ -810,8 +810,32 @@ struct ovs_action_push_tnl {
>  };
>  #endif
>
> -/**
> - * enum ovs_ct_attr - Attributes for %OVS_ACTION_ATTR_CT action.
> +/* struct ovs_action_add_mpls - %OVS_ACTION_ATTR_ADD_MPLS action
> + * argument.
> + * @mpls_lse: MPLS label stack entry to push.
> + * @mpls_ethertype: Ethertype to set in the encapsulating ethernet 
> frame.
> + * @tun_flags: MPLS tunnel attributes.
> + *
> + * The only values @mpls_ethertype should ever be given are 
> %ETH_P_MPLS_UC and
> + * %ETH_P_MPLS_MC, indicating MPLS unicast or multicast. Other are 
> rejected.
> + */
> +struct ovs_action_add_mpls {
> +	__be32 mpls_lse;
> +	__be16 mpls_ethertype; /* Either %ETH_P_MPLS_UC or %ETH_P_MPLS_MC */
> +	__u16 tun_flags;
> +};
> +
> +#define OVS_MPLS_L3_TUNNEL_FLAG_MASK  (1 << 0) /* Flag to specify the 
> place of
> +						* insertion of MPLS header.
> +						* When false, the MPLS header
> +						* will be inserted at the start
> +						* of the packet.
> +						* When true, the MPLS header
> +						* will be inserted at the start
> +						* of the l3 header.
> +						*/
> +
> +/* enum ovs_ct_attr - Attributes for %OVS_ACTION_ATTR_CT action.
>   * @OVS_CT_ATTR_COMMIT: If present, commits the connection to the 
> conntrack
>   * table. This allows future packets for the same connection to be 
> identified
>   * as 'established' or 'related'. The flow key for the current packet 
> will
> @@ -1001,7 +1025,11 @@ struct check_pkt_len_arg {
>   * @OVS_ACTION_ATTR_CHECK_PKT_LEN: Check the packet length and 
> execute a set
>   * of actions if greater than the specified packet length, else 
> execute
>   * another set of actions.
> - * @OVS_ACTION_ATTR_DROP: Explicit drop action.
> + * @OVS_ACTION_ATTR_ADD_MPLS: Push a new MPLS label stack entry at 
> the
> + * start of the packet or at the start of the l3 header depending on 
> the value
> + * of l3 tunnel flag in the tun_flags field of 
> OVS_ACTION_ATTR_ADD_MPLS
> + * argument.
> +  * @OVS_ACTION_ATTR_DROP: Explicit drop action.
>   */
>
>  enum ovs_action_attr {
> @@ -1030,6 +1058,7 @@ enum ovs_action_attr {
>  	OVS_ACTION_ATTR_METER,        /* u32 meter number. */
>  	OVS_ACTION_ATTR_CLONE,        /* Nested OVS_CLONE_ATTR_*.  */
>  	OVS_ACTION_ATTR_CHECK_PKT_LEN, /* Nested OVS_CHECK_PKT_LEN_ATTR_*. 
> */
> +	OVS_ACTION_ATTR_ADD_MPLS,     /* struct ovs_action_add_mpls. */
>
>  #ifndef __KERNEL__
>  	OVS_ACTION_ATTR_TUNNEL_PUSH,   /* struct ovs_action_push_tnl*/
> diff --git a/include/openvswitch/ofp-ed-props.h 
> b/include/openvswitch/ofp-ed-props.h
> index 306c6fe73..c85f3c283 100644
> --- a/include/openvswitch/ofp-ed-props.h
> +++ b/include/openvswitch/ofp-ed-props.h
> @@ -46,6 +46,11 @@ enum ofp_ed_nsh_prop_type {
>      OFPPPT_PROP_NSH_TLV = 2,     /* property TLV in NSH */
>  };
>
> +enum ofp_ed_mpls_prop_type {
> +    OFPPPT_PROP_MPLS_NONE = 0,    /* unused */
> +    OFPPPT_PROP_MPLS_ETHERTYPE = 1,  /* MPLS Ethertype */
> +};
> +
>  /*
>   * External representation of encap/decap properties.
>   * These must be padded to a multiple of 8 bytes.
> @@ -72,6 +77,13 @@ struct ofp_ed_prop_nsh_tlv {
>      uint8_t data[0];
>  };
>
> +struct ofp_ed_prop_mpls_ethertype {
> +    struct ofp_ed_prop_header header;
> +    uint16_t ether_type;         /* MPLS ethertype .*/
> +    uint8_t pad[2];          /* Padding to 8 bytes. */
> +};
> +
> +
>  /*
>   * Internal representation of encap/decap properties
>   */
> @@ -96,6 +108,12 @@ struct ofpact_ed_prop_nsh_tlv {
>      /* tlv_len octets of metadata value, padded to a multiple of 8 
> bytes. */
>      uint8_t data[0];
>  };
> +
> +struct ofpact_ed_prop_mpls_ethertype {
> +    struct ofpact_ed_prop header;
> +    uint16_t ether_type;         /* MPLS ethertype .*/
> +    uint8_t pad[2];          /* Padding to 8 bytes. */
> +};
>  enum ofperr decode_ed_prop(const struct ofp_ed_prop_header 
> **ofp_prop,
>                             struct ofpbuf *out, size_t *remaining);
>  enum ofperr encode_ed_prop(const struct ofpact_ed_prop **prop,
> diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
> index 94cc9b80c..bbdea5603 100644
> --- a/lib/dpif-netdev.c
> +++ b/lib/dpif-netdev.c
> @@ -8044,6 +8044,7 @@ dp_execute_cb(void *aux_, struct dp_packet_batch 
> *packets_,
>      case OVS_ACTION_ATTR_CT_CLEAR:
>      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
>      case OVS_ACTION_ATTR_DROP:
> +    case OVS_ACTION_ATTR_ADD_MPLS:
>      case __OVS_ACTION_ATTR_MAX:
>          OVS_NOT_REACHED();
>      }
> diff --git a/lib/dpif.c b/lib/dpif.c
> index 56d0b4a65..bbd1296e3 100644
> --- a/lib/dpif.c
> +++ b/lib/dpif.c
> @@ -1273,6 +1273,7 @@ dpif_execute_helper_cb(void *aux_, struct 
> dp_packet_batch *packets_,
>      case OVS_ACTION_ATTR_UNSPEC:
>      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
>      case OVS_ACTION_ATTR_DROP:
> +    case OVS_ACTION_ATTR_ADD_MPLS:
>      case __OVS_ACTION_ATTR_MAX:
>          OVS_NOT_REACHED();
>      }
> diff --git a/lib/odp-execute.c b/lib/odp-execute.c
> index 6eeda2a61..2f4cdd92c 100644
> --- a/lib/odp-execute.c
> +++ b/lib/odp-execute.c
> @@ -819,6 +819,7 @@ requires_datapath_assistance(const struct nlattr 
> *a)
>      case OVS_ACTION_ATTR_POP_NSH:
>      case OVS_ACTION_ATTR_CT_CLEAR:
>      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
> +    case OVS_ACTION_ATTR_ADD_MPLS:
>      case OVS_ACTION_ATTR_DROP:
>          return false;
>
> @@ -1061,6 +1062,17 @@ odp_execute_actions(void *dp, struct 
> dp_packet_batch *batch, bool steal,
>              }
>              break;
>
> +        case OVS_ACTION_ATTR_ADD_MPLS: {
> +            const struct ovs_action_add_mpls *mpls = nl_attr_get(a);
> +            bool l3_flag =  mpls->tun_flags & 
> OVS_MPLS_L3_TUNNEL_FLAG_MASK;
> +
> +            DP_PACKET_BATCH_FOR_EACH (i, packet, batch) {
> +                add_mpls(packet, mpls->mpls_ethertype, 
> mpls->mpls_lse,
> +                         l3_flag);
> +            }
> +            break;
> +        }
> +
>          case OVS_ACTION_ATTR_DROP:{
>              const enum xlate_error *drop_reason = nl_attr_get(a);
>
> diff --git a/lib/odp-util.c b/lib/odp-util.c
> index a8598d52a..f24e16d08 100644
> --- a/lib/odp-util.c
> +++ b/lib/odp-util.c
> @@ -142,6 +142,8 @@ odp_action_len(uint16_t type)
>      case OVS_ACTION_ATTR_PUSH_NSH: return ATTR_LEN_VARIABLE;
>      case OVS_ACTION_ATTR_POP_NSH: return 0;
>      case OVS_ACTION_ATTR_CHECK_PKT_LEN: return ATTR_LEN_VARIABLE;
> +    case OVS_ACTION_ATTR_ADD_MPLS:
> +         return sizeof(struct ovs_action_add_mpls);
>      case OVS_ACTION_ATTR_DROP: return sizeof(uint32_t);
>
>      case OVS_ACTION_ATTR_UNSPEC:
> @@ -1254,6 +1256,14 @@ format_odp_action(struct ds *ds, const struct 
> nlattr *a,
>      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
>          format_odp_check_pkt_len_action(ds, a, portno_names);
>          break;
> +    case OVS_ACTION_ATTR_ADD_MPLS: {
> +        const struct ovs_action_push_mpls *mpls = nl_attr_get(a);
> +        ds_put_cstr(ds, "add_mpls(");
> +        format_mpls_lse(ds, mpls->mpls_lse);
> +        ds_put_format(ds, ",eth_type=0x%"PRIx16")",
> +                      ntohs(mpls->mpls_ethertype));
> +        break;
> +    }
>      case OVS_ACTION_ATTR_DROP:
>          ds_put_cstr(ds, "drop");
>          break;
> @@ -7876,7 +7886,8 @@ commit_vlan_action(const struct flow* flow, 
> struct flow *base,
>  /* Wildcarding already done at action translation time. */
>  static void
>  commit_mpls_action(const struct flow *flow, struct flow *base,
> -                   struct ofpbuf *odp_actions)
> +                   struct ofpbuf *odp_actions, bool pending_encap,
> +                   bool pending_decap)
>  {
>      int base_n = flow_count_mpls_labels(base, NULL);
>      int flow_n = flow_count_mpls_labels(flow, NULL);
> @@ -7913,7 +7924,11 @@ commit_mpls_action(const struct flow *flow, 
> struct flow *base,
>              if ((!eth_type_mpls(flow->dl_type)) && base_n > 1) {
>                  dl_type = htons(ETH_TYPE_MPLS);
>              } else {
> -                dl_type = flow->dl_type;
> +                if ((flow->packet_type == PT_ETH) && pending_decap) {
> +                    dl_type =  htons(ETH_TYPE_TEB);
> +                } else {
> +                    dl_type = flow->dl_type;
> +                }
>              }
>              nl_msg_put_be16(odp_actions, OVS_ACTION_ATTR_POP_MPLS, 
> dl_type);
>              ovs_assert(flow_pop_mpls(base, base_n, flow->dl_type, 
> NULL));
> @@ -7924,18 +7939,29 @@ commit_mpls_action(const struct flow *flow, 
> struct flow *base,
>      /* If, after the above popping and setting, there are more LSEs 
> in flow
>       * than base then some LSEs need to be pushed. */
>      while (base_n < flow_n) {
> -        struct ovs_action_push_mpls *mpls;
>
> -        mpls = nl_msg_put_unspec_zero(odp_actions,
> -                                      OVS_ACTION_ATTR_PUSH_MPLS,
> -                                      sizeof *mpls);
> -        mpls->mpls_ethertype = flow->dl_type;
> -        mpls->mpls_lse = flow->mpls_lse[flow_n - base_n - 1];
> +        if (pending_encap) {
> +             struct ovs_action_add_mpls *mpls;
> +
> +             mpls = nl_msg_put_unspec_zero(odp_actions,
> +                                           OVS_ACTION_ATTR_ADD_MPLS,
> +                                           sizeof *mpls);
> +             mpls->mpls_ethertype = flow->dl_type;
> +             mpls->mpls_lse = flow->mpls_lse[flow_n - base_n - 1];
> +        } else {
> +             struct ovs_action_push_mpls *mpls;
> +
> +             mpls = nl_msg_put_unspec_zero(odp_actions,
> +                                           OVS_ACTION_ATTR_PUSH_MPLS,
> +                                           sizeof *mpls);
> +             mpls->mpls_ethertype = flow->dl_type;
> +             mpls->mpls_lse = flow->mpls_lse[flow_n - base_n - 1];
> +        }
>          /* Update base flow's MPLS stack, but do not clear L3.  We 
> need the L3
>           * headers if the flow is restored later due to returning 
> from a patch
>           * port or group bucket. */
> -        flow_push_mpls(base, base_n, mpls->mpls_ethertype, NULL, 
> false);
> -        flow_set_mpls_lse(base, 0, mpls->mpls_lse);
> +        flow_push_mpls(base, base_n, flow->dl_type, NULL, false);
> +        flow_set_mpls_lse(base, 0, flow->mpls_lse[flow_n - base_n - 
> 1]);
>          base_n++;
>      }
>  }
> @@ -8586,6 +8612,10 @@ commit_encap_decap_action(const struct flow 
> *flow,
>              memcpy(&base_flow->dl_dst, &flow->dl_dst,
>                     sizeof(*flow) - offsetof(struct flow, dl_dst));
>              break;
> +        case PT_MPLS:
> +            commit_mpls_action(flow, base_flow, odp_actions, 
> pending_encap,
> +                               pending_decap);
> +            break;
>          default:
>              /* Only the above protocols are supported for encap.
>               * The check is done at action translation. */
> @@ -8608,6 +8638,10 @@ commit_encap_decap_action(const struct flow 
> *flow,
>                  /* pop_nsh. */
>                  odp_put_pop_nsh_action(odp_actions);
>                  break;
> +            case PT_MPLS:
> +                commit_mpls_action(flow, base_flow, odp_actions, 
> pending_encap,
> +                                   pending_decap);
> +                break;
>              default:
>                  /* Checks are done during translation. */
>                  OVS_NOT_REACHED();
> @@ -8653,7 +8687,7 @@ commit_odp_actions(const struct flow *flow, 
> struct flow *base,
>      /* Make packet a non-MPLS packet before committing L3/4 actions,
>       * which would otherwise do nothing. */
>      if (eth_type_mpls(base->dl_type) && 
> !eth_type_mpls(flow->dl_type)) {
> -        commit_mpls_action(flow, base, odp_actions);
> +        commit_mpls_action(flow, base, odp_actions, false, false);
>          mpls_done = true;
>      }
>      commit_set_nsh_action(flow, base, odp_actions, wc, use_masked);
> @@ -8661,7 +8695,7 @@ commit_odp_actions(const struct flow *flow, 
> struct flow *base,
>      commit_set_port_action(flow, base, odp_actions, wc, use_masked);
>      slow2 = commit_set_icmp_action(flow, base, odp_actions, wc);
>      if (!mpls_done) {
> -        commit_mpls_action(flow, base, odp_actions);
> +        commit_mpls_action(flow, base, odp_actions, false, false);
>      }
>      commit_vlan_action(flow, base, odp_actions, wc);
>      commit_set_priority_action(flow, base, odp_actions, wc, 
> use_masked);
> diff --git a/lib/ofp-actions.c b/lib/ofp-actions.c
> index 0342a228b..28a12a569 100644
> --- a/lib/ofp-actions.c
> +++ b/lib/ofp-actions.c
> @@ -4441,6 +4441,7 @@ decode_NXAST_RAW_ENCAP(const struct 
> nx_action_encap *nae,
>      switch (ntohl(nae->new_pkt_type)) {
>      case PT_ETH:
>      case PT_NSH:
> +    case PT_MPLS:
>          /* Add supported encap header types here. */
>          break;
>      default:
> @@ -4492,6 +4493,8 @@ parse_encap_header(const char *hdr, ovs_be32 
> *packet_type)
>          *packet_type = htonl(PT_ETH);
>      } else if (strcmp(hdr, "nsh") == 0) {
>          *packet_type = htonl(PT_NSH);
> +    } else if (strcmp(hdr, "mpls") == 0) {
> +        *packet_type = htonl(PT_MPLS);
>      } else {
>          return false;
>      }
> @@ -4573,6 +4576,8 @@ format_encap_pkt_type(const ovs_be32 pkt_type)
>          return "ethernet";
>      case PT_NSH:
>          return "nsh";
> +    case PT_MPLS:
> +        return "mpls";
>      default:
>          return "UNKNOWN";
>      }
> diff --git a/lib/ofp-ed-props.c b/lib/ofp-ed-props.c
> index 02a9235d5..fc261e4c6 100644
> --- a/lib/ofp-ed-props.c
> +++ b/lib/ofp-ed-props.c
> @@ -79,6 +79,27 @@ decode_ed_prop(const struct ofp_ed_prop_header 
> **ofp_prop,
>          }
>          break;
>      }
> +    case OFPPPC_MPLS: {
> +       switch (prop_type) {
> +        case OFPPPT_PROP_MPLS_ETHERTYPE: {
> +            struct ofp_ed_prop_mpls_ethertype *opnmt =
> +                ALIGNED_CAST(struct ofp_ed_prop_mpls_ethertype *, 
> *ofp_prop);
> +            if (len > sizeof(*opnmt) || len > *remaining) {
> +                return OFPERR_NXBAC_BAD_ED_PROP;
> +            }
> +            struct ofpact_ed_prop_mpls_ethertype *pnmt =
> +                    ofpbuf_put_uninit(out, sizeof(*pnmt));
> +            pnmt->header.prop_class = prop_class;
> +            pnmt->header.type = prop_type;
> +            pnmt->header.len = len;
> +            pnmt->ether_type = opnmt->ether_type;
> +            break;
> +        }
> +        default:
> +            return OFPERR_NXBAC_UNKNOWN_ED_PROP;
> +        }
> +        break;
> +    }
>      default:
>          return OFPERR_NXBAC_UNKNOWN_ED_PROP;
>      }
> @@ -134,6 +155,27 @@ encode_ed_prop(const struct ofpact_ed_prop 
> **prop,
>          }
>          break;
>      }
> +    case OFPPPC_MPLS: {
> +       switch ((*prop)->type) {
> +       case OFPPPT_PROP_MPLS_ETHERTYPE: {
> +           struct ofpact_ed_prop_mpls_ethertype *pnmt =
> +                ALIGNED_CAST(struct ofpact_ed_prop_mpls_ethertype *, 
> *prop);
> +            struct ofp_ed_prop_mpls_ethertype *opnmt =
> +                    ofpbuf_put_uninit(out, sizeof(*opnmt));
> +            opnmt->header.prop_class = htons((*prop)->prop_class);
> +            opnmt->header.type = (*prop)->type;
> +            opnmt->header.len =
> +                    offsetof(struct ofpact_ed_prop_mpls_ethertype, 
> pad);
> +            opnmt->ether_type = pnmt->ether_type;
> +            prop_len = sizeof(*pnmt);
> +            break;
> +
> +       }
> +       default:
> +            return OFPERR_OFPBAC_BAD_ARGUMENT;
> +       }
> +       break;
> +    }
>      default:
>          return OFPERR_OFPBAC_BAD_ARGUMENT;
>      }
> @@ -181,6 +223,13 @@ parse_ed_prop_type(uint16_t prop_class,
>          } else {
>              return false;
>          }
> +    case OFPPPC_MPLS:
> +        if (!strcmp(str, "ether_type")) {
> +            *type = OFPPPT_PROP_MPLS_ETHERTYPE;
> +            return true;
> +        } else {
> +            return false;
> +        }
>      default:
>          return false;
>      }
> @@ -259,6 +308,28 @@ parse_ed_prop_value(uint16_t prop_class, uint8_t 
> prop_type OVS_UNUSED,
>              OVS_NOT_REACHED();
>          }
>          break;
> +    case OFPPPC_MPLS:
> +        switch (prop_type) {
> +        case OFPPPT_PROP_MPLS_ETHERTYPE: {
> +            uint16_t ethertype;
> +            error = str_to_u16(value, "ether_type", &ethertype);
> +            if (error != NULL) {
> +                return error;
> +            }
> +            struct ofpact_ed_prop_mpls_ethertype *pnmt =
> +                    ofpbuf_put_uninit(out, sizeof(*pnmt));
> +            pnmt->header.prop_class = prop_class;
> +            pnmt->header.type = prop_type;
> +            pnmt->header.len =
> +                    offsetof(struct ofpact_ed_prop_mpls_ethertype, 
> pad);
> +            pnmt->ether_type = ethertype;
> +
> +            break;
> +        }
> +        default:
> +            break;
> +      }
> +      break;
>      default:
>          /* Unsupported property classes rejected before. */
>          OVS_NOT_REACHED();
> @@ -300,6 +371,14 @@ format_ed_prop_type(const struct ofpact_ed_prop 
> *prop)
>              OVS_NOT_REACHED();
>          }
>          break;
> +    case OFPPPC_MPLS:
> +         switch (prop->type) {
> +         case OFPPPT_PROP_MPLS_ETHERTYPE:
> +              return "ether_type";
> +         default:
> +               OVS_NOT_REACHED();
> +         }
> +         break;
>      default:
>          OVS_NOT_REACHED();
>      }
> @@ -332,6 +411,18 @@ format_ed_prop(struct ds *s OVS_UNUSED,
>          default:
>              OVS_NOT_REACHED();
>          }
> +     case OFPPPC_MPLS:
> +        switch (prop->type) {
> +        case OFPPPT_PROP_MPLS_ETHERTYPE: {
> +          struct ofpact_ed_prop_mpls_ethertype *pnmt =
> +                ALIGNED_CAST(struct ofpact_ed_prop_mpls_ethertype *, 
> prop);
> +            ds_put_format(s, "%s=%d", format_ed_prop_type(prop),
> +                          pnmt->ether_type);
> +            return;
> +        }
> +        default:
> +            OVS_NOT_REACHED();
> +        }
>      default:
>          OVS_NOT_REACHED();
>      }
> diff --git a/lib/ovs-actions.xml b/lib/ovs-actions.xml
> index a2778de4b..e97f818d9 100644
> --- a/lib/ovs-actions.xml
> +++ b/lib/ovs-actions.xml
> @@ -265,13 +265,13 @@
>        </p>
>
>        <p>
> -        When a <code>decap</code> action decapsulates a packet, Open 
> vSwitch
> -        raises this error if it does not support the type of inner 
> packet.
> -        <code>decap</code> of an Ethernet header raises this error if 
> a VLAN
> -        header is present, <code>decap</code> of a NSH packet raises 
> this error
> -        if the NSH inner packet is not Ethernet, IPv4, IPv6, or NSH, 
> and
> -        <code>decap</code> of other types of packets is unsupported 
> and also
> -        raises this error.
> +        The <code>decap</code> action is supported only for packet 
> types
> +        ethernet, NSH and MPLS. Openvswitch raises this error for 
> other
> +        packet types. When a <code>decap</code> action decapsulates a 
> packet,
> +        Open vSwitch raises this error if it does not support the 
> type of inner
> +        packet. <code>decap</code> of an Ethernet header raises this 
> error if a
> +        VLAN header is present, <code>decap</code> of a NSH packet 
> raises this
> +        error if the NSH inner packet is not Ethernet, IPv4, IPv6, or 
> NSH.
>        </p>
>
>        <p>
> @@ -1097,6 +1097,8 @@ for <var>i</var> in [1,<var>n_members</var>]:
>        <h2>The <code>encap</code> action</h2>
>        <syntax><code>encap(nsh(</code>[<code>md_type=<var>md_type</var></code>]<code>, 
> </code>[<code>tlv(<var>class</var>,<var>type</var>,<var>value</var>)</code>]...<code>))</code></syntax>
>        <syntax><code>encap(ethernet)</code></syntax>
> +      
> <syntax><code>encap(mpls(ether_type=<var>ether_type</var>))</code>
> +      </syntax>
>
>        <p>
>          The <code>encap</code> action encapsulates a packet with a 
> specified
> @@ -1135,6 +1137,12 @@ for <var>i</var> in [1,<var>n_members</var>]:
>          source and destination are initially zeroed.
>        </p>
>
> +      <p>
> +        The <code>encap(mpls(ethertype=....))</code> variant 
> encapsulates an
> +        ethernet or L3 packet with a MPLS header. The 
> <var>ethertype</var>
> +        could be MPLS unicast (0x8847) or multicast (0x8848) 
> ethertypes.
> +      </p>
> +
>        <conformance>
>          This action is an Open vSwitch extension to OpenFlow 1.3 and 
> later,
>          introduced in Open vSwitch 2.8.
> @@ -1144,6 +1152,9 @@ for <var>i</var> in [1,<var>n_members</var>]:
>      <action name="DECAP">
>        <h2>The <code>decap</code> action</h2>
>        <syntax><code>decap</code></syntax>
> +      <syntax><code>decap(packet_type(ns=<var>name_space</var>,
> +      type=<var>ethertype</var>))</code></syntax> for decapsulating 
> MPLS
> +      packets.
>
>        <p>
>          Removes an outermost encapsulation from the packet:
> @@ -1164,6 +1175,12 @@ for <var>i</var> in [1,<var>n_members</var>]:
>            packet type errors.
>          </li>
>
> +        <li>
> +          Otherwise, if the packet is a MPLS packet, removes the MPLS 
> header
> +          and classifies the inner packet as mentioned in the packet 
> type
> +          argument of the decap.
> +        </li>
> +
>          <li>
>            Otherwise, raises an unsupported packet type error.
>          </li>
> diff --git a/lib/packets.c b/lib/packets.c
> index 4a7643c5d..5e3c3900f 100644
> --- a/lib/packets.c
> +++ b/lib/packets.c
> @@ -418,6 +418,38 @@ push_mpls(struct dp_packet *packet, ovs_be16 
> ethtype, ovs_be32 lse)
>      pkt_metadata_init_conn(&packet->md);
>  }
>
> +void
> +add_mpls(struct dp_packet *packet, ovs_be16 ethtype, ovs_be32 lse, 
> bool l3)
> +{
> +    char * header;
> +
> +    if (!eth_type_mpls(ethtype)) {
> +        return;
> +    }
> +
> +    if (!l3) {
> +        header =  dp_packet_push_uninit(packet, MPLS_HLEN);
> +        memcpy(header, &lse, sizeof lse);
> +        packet->l2_5_ofs = 0;
> +        packet->packet_type = htonl(PT_MPLS);
> +    } else {
> +        size_t len;
> +
> +        if (!is_mpls(packet)) {
> +            /* Set MPLS label stack offset. */
> +            packet->l2_5_ofs = packet->l3_ofs;
> +        }
> +        set_ethertype(packet, ethtype);
> +
> +        /* Push new MPLS shim header onto packet. */
> +        len = packet->l2_5_ofs;
> +        header = dp_packet_resize_l2_5(packet, MPLS_HLEN);
> +        memmove(header, header + MPLS_HLEN, len);
> +        memcpy(header + len, &lse, sizeof lse);
> +    }
> +    pkt_metadata_init_conn(&packet->md);
> +}
> +
>  /* If 'packet' is an MPLS packet, removes its outermost MPLS label 
> stack entry.
>   * If the label that was removed was the only MPLS label, changes 
> 'packet''s
>   * Ethertype to 'ethtype' (which ordinarily should not be an MPLS
> @@ -429,6 +461,10 @@ pop_mpls(struct dp_packet *packet, ovs_be16 
> ethtype)
>          struct mpls_hdr *mh = dp_packet_l2_5(packet);
>          size_t len = packet->l2_5_ofs;
>
> +        if (ethtype == htons(ETH_TYPE_TEB)) {
> +             packet->packet_type = htonl(PT_ETH);
> +        }
> +
>          set_ethertype(packet, ethtype);
>          if (get_16aligned_be32(&mh->mpls_lse) & htonl(MPLS_BOS_MASK)) 
> {
>              dp_packet_set_l2_5(packet, NULL);
> diff --git a/lib/packets.h b/lib/packets.h
> index 481bc22fa..3f5862e08 100644
> --- a/lib/packets.h
> +++ b/lib/packets.h
> @@ -350,6 +350,8 @@ void set_mpls_lse_label(ovs_be32 *lse, ovs_be32 
> label);
>  void set_mpls_lse_bos(ovs_be32 *lse, uint8_t bos);
>  ovs_be32 set_mpls_lse_values(uint8_t ttl, uint8_t tc, uint8_t bos,
>                               ovs_be32 label);
> +void add_mpls(struct dp_packet *packet, ovs_be16 ethtype, ovs_be32 
> lse,
> +              bool l3_flag);
>
>  /* Example:
>   *
> diff --git a/ofproto/ofproto-dpif-ipfix.c 
> b/ofproto/ofproto-dpif-ipfix.c
> index 796eb6f88..9280e008e 100644
> --- a/ofproto/ofproto-dpif-ipfix.c
> +++ b/ofproto/ofproto-dpif-ipfix.c
> @@ -3018,6 +3018,7 @@ dpif_ipfix_read_actions(const struct flow *flow,
>          case OVS_ACTION_ATTR_CHECK_PKT_LEN:
>          case OVS_ACTION_ATTR_UNSPEC:
>          case OVS_ACTION_ATTR_DROP:
> +        case OVS_ACTION_ATTR_ADD_MPLS:
>          case __OVS_ACTION_ATTR_MAX:
>          default:
>              break;
> diff --git a/ofproto/ofproto-dpif-sflow.c 
> b/ofproto/ofproto-dpif-sflow.c
> index fdcb9eabb..ca46a9bc4 100644
> --- a/ofproto/ofproto-dpif-sflow.c
> +++ b/ofproto/ofproto-dpif-sflow.c
> @@ -1226,6 +1226,7 @@ dpif_sflow_read_actions(const struct flow *flow,
>          case OVS_ACTION_ATTR_UNSPEC:
>          case OVS_ACTION_ATTR_CHECK_PKT_LEN:
>          case OVS_ACTION_ATTR_DROP:
> +        case OVS_ACTION_ATTR_ADD_MPLS:
>          case __OVS_ACTION_ATTR_MAX:
>          default:
>              break;
> diff --git a/ofproto/ofproto-dpif-xlate.c 
> b/ofproto/ofproto-dpif-xlate.c
> index 7108c8a30..a97534233 100644
> --- a/ofproto/ofproto-dpif-xlate.c
> +++ b/ofproto/ofproto-dpif-xlate.c
> @@ -6381,6 +6381,45 @@ rewrite_flow_encap_ethernet(struct xlate_ctx 
> *ctx,
>          ctx->error = XLATE_UNSUPPORTED_PACKET_TYPE;
>      }
>  }
> +static void
> +rewrite_flow_encap_mpls(struct xlate_ctx *ctx,
> +                        const struct ofpact_encap *encap,
> +                        struct flow *flow,
> +                        struct flow_wildcards *wc)
> +{
> +    int n;
> +    uint32_t i;
> +    uint16_t ether_type;
> +    const char *ptr = (char *) encap->props;
> +
> +     for (i = 0; i < encap->n_props; i++) {
> +        struct ofpact_ed_prop *prop_ptr =
> +            ALIGNED_CAST(struct ofpact_ed_prop *, ptr);
> +        if (prop_ptr->prop_class == OFPPPC_MPLS) {
> +            switch (prop_ptr->type) {
> +                case OFPPPT_PROP_MPLS_ETHERTYPE: {
> +                     struct ofpact_ed_prop_mpls_ethertype 
> *prop_ether_type =
> +                        ALIGNED_CAST(struct 
> ofpact_ed_prop_mpls_ethertype *,
> +                                     prop_ptr);
> +                    ether_type = prop_ether_type->ether_type;
> +                    break;
> +                 }
> +            }
> +        }
> +     }
> +
> +    wc->masks.packet_type = OVS_BE32_MAX;
> +    if (flow->packet_type != htonl(PT_MPLS)) {
> +        memset(&ctx->wc->masks.mpls_lse, 0x0,
> +               sizeof *wc->masks.mpls_lse * FLOW_MAX_MPLS_LABELS);
> +        memset(&flow->mpls_lse, 0x0, sizeof *flow->mpls_lse *
> +               FLOW_MAX_MPLS_LABELS);
> +    }
> +    flow->packet_type = htonl(PT_MPLS);
> +    n = flow_count_mpls_labels(flow, ctx->wc);
> +    flow_push_mpls(flow, n, htons(ether_type), ctx->wc, true);
> +}
> +
>
>  /* For an MD2 NSH header returns a pointer to an ofpbuf with the 
> encoded
>   * MD2 TLVs provided as encap properties to the encap operation. This
> @@ -6513,6 +6552,12 @@ xlate_generic_encap_action(struct xlate_ctx 
> *ctx,
>          case PT_NSH:
>              encap_data = rewrite_flow_push_nsh(ctx, encap, flow, wc);
>              break;
> +        case PT_MPLS:
> +            rewrite_flow_encap_mpls(ctx, encap,  flow, wc);
> +            if (!ctx->xbridge->support.add_mpls) {
> +                ctx->xout->slow |= SLOW_ACTION;
> +            }
> +            break;
>          default:
>              /* New packet type was checked during decoding. */
>              OVS_NOT_REACHED();
> @@ -6582,6 +6627,21 @@ xlate_generic_decap_action(struct xlate_ctx 
> *ctx,
>              ctx->pending_decap = true;
>              /* Trigger recirculation. */
>              return true;
> +        case PT_MPLS: {
> +             int n;
> +             ovs_be16 ethertype;
> +
> +             flow->packet_type = decap->new_pkt_type;
> +             ethertype = pt_ns_type_be(flow->packet_type);
> +
> +             n = flow_count_mpls_labels(flow, ctx->wc);
> +             flow_pop_mpls(flow, n, ethertype, ctx->wc);
> +             if (!ctx->xbridge->support.add_mpls) {
> +                ctx->xout->slow |= SLOW_ACTION;
> +             }
> +             ctx->pending_decap = true;
> +             return true;
> +        }
>          default:
>              /* Error handling: drop packet. */
>              xlate_report_debug(
> diff --git a/ofproto/ofproto-dpif.c b/ofproto/ofproto-dpif.c
> index fd0b2fdea..d9a2922e7 100644
> --- a/ofproto/ofproto-dpif.c
> +++ b/ofproto/ofproto-dpif.c
> @@ -1520,6 +1520,44 @@ check_nd_extensions(struct dpif_backer *backer)
>
>      return !error;
>  }
> +/* Tests whether 'backer''s datapath supports the
> + * OVS_ACTION_ATTR_ADD_MPLS action. */
> +static bool
> +check_add_mpls(struct dpif_backer *backer)
> +{
> +    struct odputil_keybuf keybuf;
> +    struct ofpbuf actions;
> +    struct ofpbuf key;
> +    struct flow flow;
> +    bool supported;
> +
> +    struct odp_flow_key_parms odp_parms = {
> +        .flow = &flow,
> +        .probe = true,
> +    };
> +
> +    memset(&flow, 0, sizeof flow);
> +    ofpbuf_use_stack(&key, &keybuf, sizeof keybuf);
> +    odp_flow_key_from_flow(&odp_parms, &key);
> +    ofpbuf_init(&actions, 64);
> +
> +    struct ovs_action_add_mpls *mpls;
> +
> +    mpls = nl_msg_put_unspec_zero(&actions,
> +                                  OVS_ACTION_ATTR_ADD_MPLS,
> +                                  sizeof *mpls);
> +    mpls->mpls_ethertype = htons(ETH_TYPE_MPLS);
> +
> +    supported = dpif_probe_feature(backer->dpif, "add_mpls", &key,
> +                                   &actions, NULL);
> +    ofpbuf_uninit(&actions);
> +    VLOG_INFO("%s: Datapath %s add_mpls action",
> +              dpif_name(backer->dpif), supported ? "supports"
> +                                                 : "does not 
> support");
> +    return supported;
> +
> +}
> +
>
>  #define CHECK_FEATURE__(NAME, SUPPORT, FIELD, VALUE, ETHTYPE)         
>       \
>  static bool                                                           
>       \
> @@ -1590,6 +1628,7 @@ check_support(struct dpif_backer *backer)
>          dpif_supports_explicit_drop_action(backer->dpif);
>      backer->rt_support.lb_output_action=
>          dpif_supports_lb_output_action(backer->dpif);
> +    backer->rt_support.add_mpls = check_add_mpls(backer);
>
>      /* Flow fields. */
>      backer->rt_support.odp.ct_state = check_ct_state(backer);
> diff --git a/ofproto/ofproto-dpif.h b/ofproto/ofproto-dpif.h
> index b41c3d82a..c04bfff8d 100644
> --- a/ofproto/ofproto-dpif.h
> +++ b/ofproto/ofproto-dpif.h
> @@ -204,7 +204,10 @@ struct group_dpif *group_dpif_lookup(struct 
> ofproto_dpif *,
>      DPIF_SUPPORT_FIELD(bool, explicit_drop_action, "Explicit Drop 
> action")  \
>                                                                              \
>      /* True if the datapath supports balance_tcp optimization */      
>       \
> -    DPIF_SUPPORT_FIELD(bool, lb_output_action, "Optimized Balance TCP 
> mode")
> +    DPIF_SUPPORT_FIELD(bool, lb_output_action, "Optimized Balance TCP 
> mode")\
> +                                                                      
>       \
> +    /* True if the datapath supports layer 2 MPLS tunnelling */       
>       \
> +    DPIF_SUPPORT_FIELD(bool, add_mpls, "l2 MPLS tunnelling")
>
>
>  /* Stores the various features which the corresponding backer 
> supports. */
> diff --git a/tests/system-traffic.at b/tests/system-traffic.at
> index fb5b9a36d..b865b5210 100644
> --- a/tests/system-traffic.at
> +++ b/tests/system-traffic.at
> @@ -1027,9 +1027,45 @@ NS_CHECK_EXEC([at_ns0], [ping -q -c 3 -i 0.3 -w 
> 2 10.1.1.2 | FORMAT_PING], [0],
>  3 packets transmitted, 3 received, 0% packet loss, time 0ms
>  ])
>
> +OVS_TRAFFIC_VSWITCHD_STOP
> +AT_CLEANUP
> +
> +
> +AT_SETUP([datapath - ptap mpls actions])
> +OVS_TRAFFIC_VSWITCHD_START([_ADD_BR([br1])])
> +
> +ADD_NAMESPACES(at_ns0, at_ns1)
> +
> +ADD_VETH(p0, at_ns0, br0, "10.1.1.1/24")
> +ADD_VETH(p1, at_ns1, br1, "10.1.1.2/24")
> +
> +AT_CHECK([ip link add patch0 type veth peer name patch1])
> +on_exit 'ip link del patch0'
> +
> +AT_CHECK([ip link set dev patch0 up])
> +AT_CHECK([ip link set dev patch1 up])
> +AT_CHECK([ovs-vsctl add-port br0 patch0 -- set interface patch0 
> ofport_request=100])
> +AT_CHECK([ovs-vsctl add-port br1 patch1 -- set interface patch1 
> ofport_request=100])
> +
> +AT_DATA([flows.txt], [dnl
> +table=0,priority=100,dl_type=0x0800 
> actions=encap(mpls(ether_type=0x8847)),set_mpls_label:2,encap(ethernet),output:100
> +table=0,priority=100,dl_type=0x8847,mpls_label=2 
> actions=decap(),decap(packet_type(ns=0,type=0)),resubmit(,3)
> +table=0,priority=10 actions=resubmit(,3)
> +table=3,priority=10 actions=normal
> +])
> +
> +AT_CHECK([ovs-ofctl  -Oopenflow13 add-flows br0 flows.txt])
> +AT_CHECK([ovs-ofctl  -Oopenflow13 add-flows br1 flows.txt])
> +
> +NS_CHECK_EXEC([at_ns0], [ping -q -c 3 -i 0.3 -w 2 10.1.1.2 | 
> FORMAT_PING], [0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +
>  NS_CHECK_EXEC([at_ns1], [ping -q -c 3 -i 0.3 -w 2 10.1.1.1 | 
> FORMAT_PING], [0], [dnl
>  3 packets transmitted, 3 received, 0% packet loss, time 0ms
>  ])
> +
>  OVS_TRAFFIC_VSWITCHD_STOP
>  AT_CLEANUP
>
> -- 
> 2.18.4

On Wed, Mar 31, 2021 at 03:59:40PM +0200, Eelco Chaudron wrote:
> 
> 
> On 26 Mar 2021, at 7:21, Martin Varghese wrote:
> 
> > From: Martin Varghese <martin.varghese@nokia.com>
> > 
> > The encap & decap actions are extended to support MPLS packet type.
> > Encap & decap actions adds and removes MPLS header at start of the
> > packet.
> 
> Hi Martin,
> 
> I’m trying to do some real-life testing, and I’m running into issues. This
> might be me setting it up wrongly but just wanting to confirm…
> 
> I’m sending an MPLS packet that contains an ARP packet into a physical port.
> This is the packet:
> 
> Frame 4: 64 bytes on wire (512 bits), 64 bytes captured (512 bits)
>     Encapsulation type: Ethernet (1)
>     [Protocols in frame: eth:ethertype:mpls:data]
> Ethernet II, Src: 00:00:00_00:00:01 (00:00:00:00:00:01), Dst:
> 00:00:00_00:00:02 (00:00:00:00:00:02)
>     Destination: 00:00:00_00:00:02 (00:00:00:00:00:02)
>         Address: 00:00:00_00:00:02 (00:00:00:00:00:02)
>         .... ..0. .... .... .... .... = LG bit: Globally unique address
> (factory default)
>         .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
>     Source: 00:00:00_00:00:01 (00:00:00:00:00:01)
>         Address: 00:00:00_00:00:01 (00:00:00:00:00:01)
>         .... ..0. .... .... .... .... = LG bit: Globally unique address
> (factory default)
>         .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
>     Type: MPLS label switched packet (0x8847)
> MultiProtocol Label Switching Header, Label: 100, Exp: 0, S: 1, TTL: 64
>     0000 0000 0000 0110 0100 .... .... .... = MPLS Label: 100
>     .... .... .... .... .... 000. .... .... = MPLS Experimental Bits: 0
>     .... .... .... .... .... ...1 .... .... = MPLS Bottom Of Label Stack: 1
>     .... .... .... .... .... .... 0100 0000 = MPLS TTL: 64
> Data (46 bytes)
> 
> 0000  ff ff ff ff ff ff 52 54 00 88 51 38 08 06 00 01   ......RT..Q8....
> 0010  08 00 06 04 00 01 52 54 00 88 51 38 01 01 01 65   ......RT..Q8...e
> 0020  00 00 00 00 00 00 01 01 01 64 27 98 a0 47         .........d'..G
>     Data:
> ffffffffffff525400885138080600010800060400015254008851380101016500000000?
> 
> 
> I’m trying to use the following rules:
> 
>   ovs-ofctl del-flows ovs_pvp_br0
>   ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0
> "priority=100,dl_type=0x8847,mpls_label=100
> actions=decap(),decap(packet_type(ns=0,type=0x806)),resubmit(,3)"
>   ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0 "table=3,priority=10
> actions=normal"
> 
> With these, I expect the packet to be sent to vnet0, but it’s not. Actually,
> the datapath rule looks odd, while the userspace rules seem to match:
> 
>   $ ovs-dpctl dump-flows
>   recirc_id(0),in_port(1),eth(),eth_type(0x8847),mpls(label=100/0xfffff,tc=0/0,ttl=0/0x0,bos=1/1),
> packets:13, bytes:1118, used:0.322s,
> actions:pop_eth,pop_mpls(eth_type=0x806),recirc(0x19a)
>   recirc_id(0x19a),in_port(1),eth_type(0x0806), packets:13, bytes:884,
> used:0.322s, actions:drop
> 
>   $ ovs-ofctl dump-flows ovs_pvp_br0 -O OpenFlow13
>   cookie=0x0, duration=85.007s, table=0, n_packets=51, n_bytes=4386,
> priority=100,mpls,mpls_label=100
> actions=decap(),decap(packet_type(ns=0,type=0x806)),resubmit(,3)
>   cookie=0x0, duration=84.990s, table=3, n_packets=51, n_bytes=3468,
> priority=10 actions=NORMAL
>
The inner packet is ethernet. So the packet type should be (ns=0,type=0)
?

> 
> If I use the old way, doing pop_mpls, it works fine:
> 
> 
> ovs-ofctl del-flows ovs_pvp_br0
> ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0
> "priority=100,dl_type=0x8847,mpls_label=100
> actions=pop_mpls:0x0806,resubmit(,3)"
> ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0 "table=3,priority=10
> actions=normal"
> 
> 
> I also noticed (despite the test example) to make encap work, I had to set
> the ethernet MAC addresses, or else the packets were not getting out.
> So something like:
> 
>   ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0 "priority=100,in_port=vnet0,actions=encap(mpls(ether_type=0x8847)),set_mpls_label:100,encap(ethernet),,set_field:00:00:00:00:00:02->dl_dst,set_field:00:00:00:00:00:01->dl_src,output:enp5s0f0
> 

The packets are not going out because you are sending the packet on a
real nic and not on a virtual inerface (veth pair) ?

> Maybe the test case can be made more realistic? Once I understand the
> failure, I can continue with the review.
> 
> 
> Cheers,
> 
> Eelco
> 
> 
> 
> > Signed-off-by: Martin Varghese <martin.varghese@nokia.com>
> > ---
> >  NEWS                                          |  2 +-
> >  .../linux/compat/include/linux/openvswitch.h  | 35 ++++++-
> >  include/openvswitch/ofp-ed-props.h            | 18 ++++
> >  lib/dpif-netdev.c                             |  1 +
> >  lib/dpif.c                                    |  1 +
> >  lib/odp-execute.c                             | 12 +++
> >  lib/odp-util.c                                | 58 +++++++++---
> >  lib/ofp-actions.c                             |  5 +
> >  lib/ofp-ed-props.c                            | 91 +++++++++++++++++++
> >  lib/ovs-actions.xml                           | 31 +++++--
> >  lib/packets.c                                 | 36 ++++++++
> >  lib/packets.h                                 |  2 +
> >  ofproto/ofproto-dpif-ipfix.c                  |  1 +
> >  ofproto/ofproto-dpif-sflow.c                  |  1 +
> >  ofproto/ofproto-dpif-xlate.c                  | 60 ++++++++++++
> >  ofproto/ofproto-dpif.c                        | 39 ++++++++
> >  ofproto/ofproto-dpif.h                        |  5 +-
> >  tests/system-traffic.at                       | 36 ++++++++
> >  18 files changed, 410 insertions(+), 24 deletions(-)
> > 
> > diff --git a/NEWS b/NEWS
> > index 95cf922aa..4bf4e9e7b 100644
> > --- a/NEWS
> > +++ b/NEWS
> > @@ -120,7 +120,7 @@ v2.14.0 - 17 Aug 2020
> >     - GTP-U Tunnel Protocol
> >       * Add two new fields: tun_gtpu_flags, tun_gtpu_msgtype.
> >       * Only support for userspace datapath.
> > -
> > +   - Encap & Decap action support for MPLS packet type.
> > 
> >  v2.13.0 - 14 Feb 2020
> >  ---------------------
> > diff --git a/datapath/linux/compat/include/linux/openvswitch.h
> > b/datapath/linux/compat/include/linux/openvswitch.h
> > index 875de2025..8feea7dd4 100644
> > --- a/datapath/linux/compat/include/linux/openvswitch.h
> > +++ b/datapath/linux/compat/include/linux/openvswitch.h
> > @@ -810,8 +810,32 @@ struct ovs_action_push_tnl {
> >  };
> >  #endif
> > 
> > -/**
> > - * enum ovs_ct_attr - Attributes for %OVS_ACTION_ATTR_CT action.
> > +/* struct ovs_action_add_mpls - %OVS_ACTION_ATTR_ADD_MPLS action
> > + * argument.
> > + * @mpls_lse: MPLS label stack entry to push.
> > + * @mpls_ethertype: Ethertype to set in the encapsulating ethernet
> > frame.
> > + * @tun_flags: MPLS tunnel attributes.
> > + *
> > + * The only values @mpls_ethertype should ever be given are
> > %ETH_P_MPLS_UC and
> > + * %ETH_P_MPLS_MC, indicating MPLS unicast or multicast. Other are
> > rejected.
> > + */
> > +struct ovs_action_add_mpls {
> > +	__be32 mpls_lse;
> > +	__be16 mpls_ethertype; /* Either %ETH_P_MPLS_UC or %ETH_P_MPLS_MC */
> > +	__u16 tun_flags;
> > +};
> > +
> > +#define OVS_MPLS_L3_TUNNEL_FLAG_MASK  (1 << 0) /* Flag to specify the
> > place of
> > +						* insertion of MPLS header.
> > +						* When false, the MPLS header
> > +						* will be inserted at the start
> > +						* of the packet.
> > +						* When true, the MPLS header
> > +						* will be inserted at the start
> > +						* of the l3 header.
> > +						*/
> > +
> > +/* enum ovs_ct_attr - Attributes for %OVS_ACTION_ATTR_CT action.
> >   * @OVS_CT_ATTR_COMMIT: If present, commits the connection to the
> > conntrack
> >   * table. This allows future packets for the same connection to be
> > identified
> >   * as 'established' or 'related'. The flow key for the current packet
> > will
> > @@ -1001,7 +1025,11 @@ struct check_pkt_len_arg {
> >   * @OVS_ACTION_ATTR_CHECK_PKT_LEN: Check the packet length and execute
> > a set
> >   * of actions if greater than the specified packet length, else execute
> >   * another set of actions.
> > - * @OVS_ACTION_ATTR_DROP: Explicit drop action.
> > + * @OVS_ACTION_ATTR_ADD_MPLS: Push a new MPLS label stack entry at the
> > + * start of the packet or at the start of the l3 header depending on
> > the value
> > + * of l3 tunnel flag in the tun_flags field of OVS_ACTION_ATTR_ADD_MPLS
> > + * argument.
> > +  * @OVS_ACTION_ATTR_DROP: Explicit drop action.
> >   */
> > 
> >  enum ovs_action_attr {
> > @@ -1030,6 +1058,7 @@ enum ovs_action_attr {
> >  	OVS_ACTION_ATTR_METER,        /* u32 meter number. */
> >  	OVS_ACTION_ATTR_CLONE,        /* Nested OVS_CLONE_ATTR_*.  */
> >  	OVS_ACTION_ATTR_CHECK_PKT_LEN, /* Nested OVS_CHECK_PKT_LEN_ATTR_*. */
> > +	OVS_ACTION_ATTR_ADD_MPLS,     /* struct ovs_action_add_mpls. */
> > 
> >  #ifndef __KERNEL__
> >  	OVS_ACTION_ATTR_TUNNEL_PUSH,   /* struct ovs_action_push_tnl*/
> > diff --git a/include/openvswitch/ofp-ed-props.h
> > b/include/openvswitch/ofp-ed-props.h
> > index 306c6fe73..c85f3c283 100644
> > --- a/include/openvswitch/ofp-ed-props.h
> > +++ b/include/openvswitch/ofp-ed-props.h
> > @@ -46,6 +46,11 @@ enum ofp_ed_nsh_prop_type {
> >      OFPPPT_PROP_NSH_TLV = 2,     /* property TLV in NSH */
> >  };
> > 
> > +enum ofp_ed_mpls_prop_type {
> > +    OFPPPT_PROP_MPLS_NONE = 0,    /* unused */
> > +    OFPPPT_PROP_MPLS_ETHERTYPE = 1,  /* MPLS Ethertype */
> > +};
> > +
> >  /*
> >   * External representation of encap/decap properties.
> >   * These must be padded to a multiple of 8 bytes.
> > @@ -72,6 +77,13 @@ struct ofp_ed_prop_nsh_tlv {
> >      uint8_t data[0];
> >  };
> > 
> > +struct ofp_ed_prop_mpls_ethertype {
> > +    struct ofp_ed_prop_header header;
> > +    uint16_t ether_type;         /* MPLS ethertype .*/
> > +    uint8_t pad[2];          /* Padding to 8 bytes. */
> > +};
> > +
> > +
> >  /*
> >   * Internal representation of encap/decap properties
> >   */
> > @@ -96,6 +108,12 @@ struct ofpact_ed_prop_nsh_tlv {
> >      /* tlv_len octets of metadata value, padded to a multiple of 8
> > bytes. */
> >      uint8_t data[0];
> >  };
> > +
> > +struct ofpact_ed_prop_mpls_ethertype {
> > +    struct ofpact_ed_prop header;
> > +    uint16_t ether_type;         /* MPLS ethertype .*/
> > +    uint8_t pad[2];          /* Padding to 8 bytes. */
> > +};
> >  enum ofperr decode_ed_prop(const struct ofp_ed_prop_header **ofp_prop,
> >                             struct ofpbuf *out, size_t *remaining);
> >  enum ofperr encode_ed_prop(const struct ofpact_ed_prop **prop,
> > diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
> > index 94cc9b80c..bbdea5603 100644
> > --- a/lib/dpif-netdev.c
> > +++ b/lib/dpif-netdev.c
> > @@ -8044,6 +8044,7 @@ dp_execute_cb(void *aux_, struct dp_packet_batch
> > *packets_,
> >      case OVS_ACTION_ATTR_CT_CLEAR:
> >      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
> >      case OVS_ACTION_ATTR_DROP:
> > +    case OVS_ACTION_ATTR_ADD_MPLS:
> >      case __OVS_ACTION_ATTR_MAX:
> >          OVS_NOT_REACHED();
> >      }
> > diff --git a/lib/dpif.c b/lib/dpif.c
> > index 56d0b4a65..bbd1296e3 100644
> > --- a/lib/dpif.c
> > +++ b/lib/dpif.c
> > @@ -1273,6 +1273,7 @@ dpif_execute_helper_cb(void *aux_, struct
> > dp_packet_batch *packets_,
> >      case OVS_ACTION_ATTR_UNSPEC:
> >      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
> >      case OVS_ACTION_ATTR_DROP:
> > +    case OVS_ACTION_ATTR_ADD_MPLS:
> >      case __OVS_ACTION_ATTR_MAX:
> >          OVS_NOT_REACHED();
> >      }
> > diff --git a/lib/odp-execute.c b/lib/odp-execute.c
> > index 6eeda2a61..2f4cdd92c 100644
> > --- a/lib/odp-execute.c
> > +++ b/lib/odp-execute.c
> > @@ -819,6 +819,7 @@ requires_datapath_assistance(const struct nlattr *a)
> >      case OVS_ACTION_ATTR_POP_NSH:
> >      case OVS_ACTION_ATTR_CT_CLEAR:
> >      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
> > +    case OVS_ACTION_ATTR_ADD_MPLS:
> >      case OVS_ACTION_ATTR_DROP:
> >          return false;
> > 
> > @@ -1061,6 +1062,17 @@ odp_execute_actions(void *dp, struct
> > dp_packet_batch *batch, bool steal,
> >              }
> >              break;
> > 
> > +        case OVS_ACTION_ATTR_ADD_MPLS: {
> > +            const struct ovs_action_add_mpls *mpls = nl_attr_get(a);
> > +            bool l3_flag =  mpls->tun_flags &
> > OVS_MPLS_L3_TUNNEL_FLAG_MASK;
> > +
> > +            DP_PACKET_BATCH_FOR_EACH (i, packet, batch) {
> > +                add_mpls(packet, mpls->mpls_ethertype, mpls->mpls_lse,
> > +                         l3_flag);
> > +            }
> > +            break;
> > +        }
> > +
> >          case OVS_ACTION_ATTR_DROP:{
> >              const enum xlate_error *drop_reason = nl_attr_get(a);
> > 
> > diff --git a/lib/odp-util.c b/lib/odp-util.c
> > index a8598d52a..f24e16d08 100644
> > --- a/lib/odp-util.c
> > +++ b/lib/odp-util.c
> > @@ -142,6 +142,8 @@ odp_action_len(uint16_t type)
> >      case OVS_ACTION_ATTR_PUSH_NSH: return ATTR_LEN_VARIABLE;
> >      case OVS_ACTION_ATTR_POP_NSH: return 0;
> >      case OVS_ACTION_ATTR_CHECK_PKT_LEN: return ATTR_LEN_VARIABLE;
> > +    case OVS_ACTION_ATTR_ADD_MPLS:
> > +         return sizeof(struct ovs_action_add_mpls);
> >      case OVS_ACTION_ATTR_DROP: return sizeof(uint32_t);
> > 
> >      case OVS_ACTION_ATTR_UNSPEC:
> > @@ -1254,6 +1256,14 @@ format_odp_action(struct ds *ds, const struct
> > nlattr *a,
> >      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
> >          format_odp_check_pkt_len_action(ds, a, portno_names);
> >          break;
> > +    case OVS_ACTION_ATTR_ADD_MPLS: {
> > +        const struct ovs_action_push_mpls *mpls = nl_attr_get(a);
> > +        ds_put_cstr(ds, "add_mpls(");
> > +        format_mpls_lse(ds, mpls->mpls_lse);
> > +        ds_put_format(ds, ",eth_type=0x%"PRIx16")",
> > +                      ntohs(mpls->mpls_ethertype));
> > +        break;
> > +    }
> >      case OVS_ACTION_ATTR_DROP:
> >          ds_put_cstr(ds, "drop");
> >          break;
> > @@ -7876,7 +7886,8 @@ commit_vlan_action(const struct flow* flow, struct
> > flow *base,
> >  /* Wildcarding already done at action translation time. */
> >  static void
> >  commit_mpls_action(const struct flow *flow, struct flow *base,
> > -                   struct ofpbuf *odp_actions)
> > +                   struct ofpbuf *odp_actions, bool pending_encap,
> > +                   bool pending_decap)
> >  {
> >      int base_n = flow_count_mpls_labels(base, NULL);
> >      int flow_n = flow_count_mpls_labels(flow, NULL);
> > @@ -7913,7 +7924,11 @@ commit_mpls_action(const struct flow *flow,
> > struct flow *base,
> >              if ((!eth_type_mpls(flow->dl_type)) && base_n > 1) {
> >                  dl_type = htons(ETH_TYPE_MPLS);
> >              } else {
> > -                dl_type = flow->dl_type;
> > +                if ((flow->packet_type == PT_ETH) && pending_decap) {
> > +                    dl_type =  htons(ETH_TYPE_TEB);
> > +                } else {
> > +                    dl_type = flow->dl_type;
> > +                }
> >              }
> >              nl_msg_put_be16(odp_actions, OVS_ACTION_ATTR_POP_MPLS,
> > dl_type);
> >              ovs_assert(flow_pop_mpls(base, base_n, flow->dl_type,
> > NULL));
> > @@ -7924,18 +7939,29 @@ commit_mpls_action(const struct flow *flow,
> > struct flow *base,
> >      /* If, after the above popping and setting, there are more LSEs in
> > flow
> >       * than base then some LSEs need to be pushed. */
> >      while (base_n < flow_n) {
> > -        struct ovs_action_push_mpls *mpls;
> > 
> > -        mpls = nl_msg_put_unspec_zero(odp_actions,
> > -                                      OVS_ACTION_ATTR_PUSH_MPLS,
> > -                                      sizeof *mpls);
> > -        mpls->mpls_ethertype = flow->dl_type;
> > -        mpls->mpls_lse = flow->mpls_lse[flow_n - base_n - 1];
> > +        if (pending_encap) {
> > +             struct ovs_action_add_mpls *mpls;
> > +
> > +             mpls = nl_msg_put_unspec_zero(odp_actions,
> > +                                           OVS_ACTION_ATTR_ADD_MPLS,
> > +                                           sizeof *mpls);
> > +             mpls->mpls_ethertype = flow->dl_type;
> > +             mpls->mpls_lse = flow->mpls_lse[flow_n - base_n - 1];
> > +        } else {
> > +             struct ovs_action_push_mpls *mpls;
> > +
> > +             mpls = nl_msg_put_unspec_zero(odp_actions,
> > +                                           OVS_ACTION_ATTR_PUSH_MPLS,
> > +                                           sizeof *mpls);
> > +             mpls->mpls_ethertype = flow->dl_type;
> > +             mpls->mpls_lse = flow->mpls_lse[flow_n - base_n - 1];
> > +        }
> >          /* Update base flow's MPLS stack, but do not clear L3.  We need
> > the L3
> >           * headers if the flow is restored later due to returning from
> > a patch
> >           * port or group bucket. */
> > -        flow_push_mpls(base, base_n, mpls->mpls_ethertype, NULL,
> > false);
> > -        flow_set_mpls_lse(base, 0, mpls->mpls_lse);
> > +        flow_push_mpls(base, base_n, flow->dl_type, NULL, false);
> > +        flow_set_mpls_lse(base, 0, flow->mpls_lse[flow_n - base_n -
> > 1]);
> >          base_n++;
> >      }
> >  }
> > @@ -8586,6 +8612,10 @@ commit_encap_decap_action(const struct flow
> > *flow,
> >              memcpy(&base_flow->dl_dst, &flow->dl_dst,
> >                     sizeof(*flow) - offsetof(struct flow, dl_dst));
> >              break;
> > +        case PT_MPLS:
> > +            commit_mpls_action(flow, base_flow, odp_actions,
> > pending_encap,
> > +                               pending_decap);
> > +            break;
> >          default:
> >              /* Only the above protocols are supported for encap.
> >               * The check is done at action translation. */
> > @@ -8608,6 +8638,10 @@ commit_encap_decap_action(const struct flow
> > *flow,
> >                  /* pop_nsh. */
> >                  odp_put_pop_nsh_action(odp_actions);
> >                  break;
> > +            case PT_MPLS:
> > +                commit_mpls_action(flow, base_flow, odp_actions,
> > pending_encap,
> > +                                   pending_decap);
> > +                break;
> >              default:
> >                  /* Checks are done during translation. */
> >                  OVS_NOT_REACHED();
> > @@ -8653,7 +8687,7 @@ commit_odp_actions(const struct flow *flow, struct
> > flow *base,
> >      /* Make packet a non-MPLS packet before committing L3/4 actions,
> >       * which would otherwise do nothing. */
> >      if (eth_type_mpls(base->dl_type) && !eth_type_mpls(flow->dl_type))
> > {
> > -        commit_mpls_action(flow, base, odp_actions);
> > +        commit_mpls_action(flow, base, odp_actions, false, false);
> >          mpls_done = true;
> >      }
> >      commit_set_nsh_action(flow, base, odp_actions, wc, use_masked);
> > @@ -8661,7 +8695,7 @@ commit_odp_actions(const struct flow *flow, struct
> > flow *base,
> >      commit_set_port_action(flow, base, odp_actions, wc, use_masked);
> >      slow2 = commit_set_icmp_action(flow, base, odp_actions, wc);
> >      if (!mpls_done) {
> > -        commit_mpls_action(flow, base, odp_actions);
> > +        commit_mpls_action(flow, base, odp_actions, false, false);
> >      }
> >      commit_vlan_action(flow, base, odp_actions, wc);
> >      commit_set_priority_action(flow, base, odp_actions, wc,
> > use_masked);
> > diff --git a/lib/ofp-actions.c b/lib/ofp-actions.c
> > index 0342a228b..28a12a569 100644
> > --- a/lib/ofp-actions.c
> > +++ b/lib/ofp-actions.c
> > @@ -4441,6 +4441,7 @@ decode_NXAST_RAW_ENCAP(const struct
> > nx_action_encap *nae,
> >      switch (ntohl(nae->new_pkt_type)) {
> >      case PT_ETH:
> >      case PT_NSH:
> > +    case PT_MPLS:
> >          /* Add supported encap header types here. */
> >          break;
> >      default:
> > @@ -4492,6 +4493,8 @@ parse_encap_header(const char *hdr, ovs_be32
> > *packet_type)
> >          *packet_type = htonl(PT_ETH);
> >      } else if (strcmp(hdr, "nsh") == 0) {
> >          *packet_type = htonl(PT_NSH);
> > +    } else if (strcmp(hdr, "mpls") == 0) {
> > +        *packet_type = htonl(PT_MPLS);
> >      } else {
> >          return false;
> >      }
> > @@ -4573,6 +4576,8 @@ format_encap_pkt_type(const ovs_be32 pkt_type)
> >          return "ethernet";
> >      case PT_NSH:
> >          return "nsh";
> > +    case PT_MPLS:
> > +        return "mpls";
> >      default:
> >          return "UNKNOWN";
> >      }
> > diff --git a/lib/ofp-ed-props.c b/lib/ofp-ed-props.c
> > index 02a9235d5..fc261e4c6 100644
> > --- a/lib/ofp-ed-props.c
> > +++ b/lib/ofp-ed-props.c
> > @@ -79,6 +79,27 @@ decode_ed_prop(const struct ofp_ed_prop_header
> > **ofp_prop,
> >          }
> >          break;
> >      }
> > +    case OFPPPC_MPLS: {
> > +       switch (prop_type) {
> > +        case OFPPPT_PROP_MPLS_ETHERTYPE: {
> > +            struct ofp_ed_prop_mpls_ethertype *opnmt =
> > +                ALIGNED_CAST(struct ofp_ed_prop_mpls_ethertype *,
> > *ofp_prop);
> > +            if (len > sizeof(*opnmt) || len > *remaining) {
> > +                return OFPERR_NXBAC_BAD_ED_PROP;
> > +            }
> > +            struct ofpact_ed_prop_mpls_ethertype *pnmt =
> > +                    ofpbuf_put_uninit(out, sizeof(*pnmt));
> > +            pnmt->header.prop_class = prop_class;
> > +            pnmt->header.type = prop_type;
> > +            pnmt->header.len = len;
> > +            pnmt->ether_type = opnmt->ether_type;
> > +            break;
> > +        }
> > +        default:
> > +            return OFPERR_NXBAC_UNKNOWN_ED_PROP;
> > +        }
> > +        break;
> > +    }
> >      default:
> >          return OFPERR_NXBAC_UNKNOWN_ED_PROP;
> >      }
> > @@ -134,6 +155,27 @@ encode_ed_prop(const struct ofpact_ed_prop **prop,
> >          }
> >          break;
> >      }
> > +    case OFPPPC_MPLS: {
> > +       switch ((*prop)->type) {
> > +       case OFPPPT_PROP_MPLS_ETHERTYPE: {
> > +           struct ofpact_ed_prop_mpls_ethertype *pnmt =
> > +                ALIGNED_CAST(struct ofpact_ed_prop_mpls_ethertype *,
> > *prop);
> > +            struct ofp_ed_prop_mpls_ethertype *opnmt =
> > +                    ofpbuf_put_uninit(out, sizeof(*opnmt));
> > +            opnmt->header.prop_class = htons((*prop)->prop_class);
> > +            opnmt->header.type = (*prop)->type;
> > +            opnmt->header.len =
> > +                    offsetof(struct ofpact_ed_prop_mpls_ethertype,
> > pad);
> > +            opnmt->ether_type = pnmt->ether_type;
> > +            prop_len = sizeof(*pnmt);
> > +            break;
> > +
> > +       }
> > +       default:
> > +            return OFPERR_OFPBAC_BAD_ARGUMENT;
> > +       }
> > +       break;
> > +    }
> >      default:
> >          return OFPERR_OFPBAC_BAD_ARGUMENT;
> >      }
> > @@ -181,6 +223,13 @@ parse_ed_prop_type(uint16_t prop_class,
> >          } else {
> >              return false;
> >          }
> > +    case OFPPPC_MPLS:
> > +        if (!strcmp(str, "ether_type")) {
> > +            *type = OFPPPT_PROP_MPLS_ETHERTYPE;
> > +            return true;
> > +        } else {
> > +            return false;
> > +        }
> >      default:
> >          return false;
> >      }
> > @@ -259,6 +308,28 @@ parse_ed_prop_value(uint16_t prop_class, uint8_t
> > prop_type OVS_UNUSED,
> >              OVS_NOT_REACHED();
> >          }
> >          break;
> > +    case OFPPPC_MPLS:
> > +        switch (prop_type) {
> > +        case OFPPPT_PROP_MPLS_ETHERTYPE: {
> > +            uint16_t ethertype;
> > +            error = str_to_u16(value, "ether_type", &ethertype);
> > +            if (error != NULL) {
> > +                return error;
> > +            }
> > +            struct ofpact_ed_prop_mpls_ethertype *pnmt =
> > +                    ofpbuf_put_uninit(out, sizeof(*pnmt));
> > +            pnmt->header.prop_class = prop_class;
> > +            pnmt->header.type = prop_type;
> > +            pnmt->header.len =
> > +                    offsetof(struct ofpact_ed_prop_mpls_ethertype,
> > pad);
> > +            pnmt->ether_type = ethertype;
> > +
> > +            break;
> > +        }
> > +        default:
> > +            break;
> > +      }
> > +      break;
> >      default:
> >          /* Unsupported property classes rejected before. */
> >          OVS_NOT_REACHED();
> > @@ -300,6 +371,14 @@ format_ed_prop_type(const struct ofpact_ed_prop
> > *prop)
> >              OVS_NOT_REACHED();
> >          }
> >          break;
> > +    case OFPPPC_MPLS:
> > +         switch (prop->type) {
> > +         case OFPPPT_PROP_MPLS_ETHERTYPE:
> > +              return "ether_type";
> > +         default:
> > +               OVS_NOT_REACHED();
> > +         }
> > +         break;
> >      default:
> >          OVS_NOT_REACHED();
> >      }
> > @@ -332,6 +411,18 @@ format_ed_prop(struct ds *s OVS_UNUSED,
> >          default:
> >              OVS_NOT_REACHED();
> >          }
> > +     case OFPPPC_MPLS:
> > +        switch (prop->type) {
> > +        case OFPPPT_PROP_MPLS_ETHERTYPE: {
> > +          struct ofpact_ed_prop_mpls_ethertype *pnmt =
> > +                ALIGNED_CAST(struct ofpact_ed_prop_mpls_ethertype *,
> > prop);
> > +            ds_put_format(s, "%s=%d", format_ed_prop_type(prop),
> > +                          pnmt->ether_type);
> > +            return;
> > +        }
> > +        default:
> > +            OVS_NOT_REACHED();
> > +        }
> >      default:
> >          OVS_NOT_REACHED();
> >      }
> > diff --git a/lib/ovs-actions.xml b/lib/ovs-actions.xml
> > index a2778de4b..e97f818d9 100644
> > --- a/lib/ovs-actions.xml
> > +++ b/lib/ovs-actions.xml
> > @@ -265,13 +265,13 @@
> >        </p>
> > 
> >        <p>
> > -        When a <code>decap</code> action decapsulates a packet, Open
> > vSwitch
> > -        raises this error if it does not support the type of inner
> > packet.
> > -        <code>decap</code> of an Ethernet header raises this error if a
> > VLAN
> > -        header is present, <code>decap</code> of a NSH packet raises
> > this error
> > -        if the NSH inner packet is not Ethernet, IPv4, IPv6, or NSH,
> > and
> > -        <code>decap</code> of other types of packets is unsupported and
> > also
> > -        raises this error.
> > +        The <code>decap</code> action is supported only for packet
> > types
> > +        ethernet, NSH and MPLS. Openvswitch raises this error for other
> > +        packet types. When a <code>decap</code> action decapsulates a
> > packet,
> > +        Open vSwitch raises this error if it does not support the type
> > of inner
> > +        packet. <code>decap</code> of an Ethernet header raises this
> > error if a
> > +        VLAN header is present, <code>decap</code> of a NSH packet
> > raises this
> > +        error if the NSH inner packet is not Ethernet, IPv4, IPv6, or
> > NSH.
> >        </p>
> > 
> >        <p>
> > @@ -1097,6 +1097,8 @@ for <var>i</var> in [1,<var>n_members</var>]:
> >        <h2>The <code>encap</code> action</h2>
> >        <syntax><code>encap(nsh(</code>[<code>md_type=<var>md_type</var></code>]<code>, </code>[<code>tlv(<var>class</var>,<var>type</var>,<var>value</var>)</code>]...<code>))</code></syntax>
> >        <syntax><code>encap(ethernet)</code></syntax>
> > +
> > <syntax><code>encap(mpls(ether_type=<var>ether_type</var>))</code>
> > +      </syntax>
> > 
> >        <p>
> >          The <code>encap</code> action encapsulates a packet with a
> > specified
> > @@ -1135,6 +1137,12 @@ for <var>i</var> in [1,<var>n_members</var>]:
> >          source and destination are initially zeroed.
> >        </p>
> > 
> > +      <p>
> > +        The <code>encap(mpls(ethertype=....))</code> variant
> > encapsulates an
> > +        ethernet or L3 packet with a MPLS header. The
> > <var>ethertype</var>
> > +        could be MPLS unicast (0x8847) or multicast (0x8848)
> > ethertypes.
> > +      </p>
> > +
> >        <conformance>
> >          This action is an Open vSwitch extension to OpenFlow 1.3 and
> > later,
> >          introduced in Open vSwitch 2.8.
> > @@ -1144,6 +1152,9 @@ for <var>i</var> in [1,<var>n_members</var>]:
> >      <action name="DECAP">
> >        <h2>The <code>decap</code> action</h2>
> >        <syntax><code>decap</code></syntax>
> > +      <syntax><code>decap(packet_type(ns=<var>name_space</var>,
> > +      type=<var>ethertype</var>))</code></syntax> for decapsulating
> > MPLS
> > +      packets.
> > 
> >        <p>
> >          Removes an outermost encapsulation from the packet:
> > @@ -1164,6 +1175,12 @@ for <var>i</var> in [1,<var>n_members</var>]:
> >            packet type errors.
> >          </li>
> > 
> > +        <li>
> > +          Otherwise, if the packet is a MPLS packet, removes the MPLS
> > header
> > +          and classifies the inner packet as mentioned in the packet
> > type
> > +          argument of the decap.
> > +        </li>
> > +
> >          <li>
> >            Otherwise, raises an unsupported packet type error.
> >          </li>
> > diff --git a/lib/packets.c b/lib/packets.c
> > index 4a7643c5d..5e3c3900f 100644
> > --- a/lib/packets.c
> > +++ b/lib/packets.c
> > @@ -418,6 +418,38 @@ push_mpls(struct dp_packet *packet, ovs_be16
> > ethtype, ovs_be32 lse)
> >      pkt_metadata_init_conn(&packet->md);
> >  }
> > 
> > +void
> > +add_mpls(struct dp_packet *packet, ovs_be16 ethtype, ovs_be32 lse, bool
> > l3)
> > +{
> > +    char * header;
> > +
> > +    if (!eth_type_mpls(ethtype)) {
> > +        return;
> > +    }
> > +
> > +    if (!l3) {
> > +        header =  dp_packet_push_uninit(packet, MPLS_HLEN);
> > +        memcpy(header, &lse, sizeof lse);
> > +        packet->l2_5_ofs = 0;
> > +        packet->packet_type = htonl(PT_MPLS);
> > +    } else {
> > +        size_t len;
> > +
> > +        if (!is_mpls(packet)) {
> > +            /* Set MPLS label stack offset. */
> > +            packet->l2_5_ofs = packet->l3_ofs;
> > +        }
> > +        set_ethertype(packet, ethtype);
> > +
> > +        /* Push new MPLS shim header onto packet. */
> > +        len = packet->l2_5_ofs;
> > +        header = dp_packet_resize_l2_5(packet, MPLS_HLEN);
> > +        memmove(header, header + MPLS_HLEN, len);
> > +        memcpy(header + len, &lse, sizeof lse);
> > +    }
> > +    pkt_metadata_init_conn(&packet->md);
> > +}
> > +
> >  /* If 'packet' is an MPLS packet, removes its outermost MPLS label
> > stack entry.
> >   * If the label that was removed was the only MPLS label, changes
> > 'packet''s
> >   * Ethertype to 'ethtype' (which ordinarily should not be an MPLS
> > @@ -429,6 +461,10 @@ pop_mpls(struct dp_packet *packet, ovs_be16
> > ethtype)
> >          struct mpls_hdr *mh = dp_packet_l2_5(packet);
> >          size_t len = packet->l2_5_ofs;
> > 
> > +        if (ethtype == htons(ETH_TYPE_TEB)) {
> > +             packet->packet_type = htonl(PT_ETH);
> > +        }
> > +
> >          set_ethertype(packet, ethtype);
> >          if (get_16aligned_be32(&mh->mpls_lse) & htonl(MPLS_BOS_MASK)) {
> >              dp_packet_set_l2_5(packet, NULL);
> > diff --git a/lib/packets.h b/lib/packets.h
> > index 481bc22fa..3f5862e08 100644
> > --- a/lib/packets.h
> > +++ b/lib/packets.h
> > @@ -350,6 +350,8 @@ void set_mpls_lse_label(ovs_be32 *lse, ovs_be32
> > label);
> >  void set_mpls_lse_bos(ovs_be32 *lse, uint8_t bos);
> >  ovs_be32 set_mpls_lse_values(uint8_t ttl, uint8_t tc, uint8_t bos,
> >                               ovs_be32 label);
> > +void add_mpls(struct dp_packet *packet, ovs_be16 ethtype, ovs_be32 lse,
> > +              bool l3_flag);
> > 
> >  /* Example:
> >   *
> > diff --git a/ofproto/ofproto-dpif-ipfix.c b/ofproto/ofproto-dpif-ipfix.c
> > index 796eb6f88..9280e008e 100644
> > --- a/ofproto/ofproto-dpif-ipfix.c
> > +++ b/ofproto/ofproto-dpif-ipfix.c
> > @@ -3018,6 +3018,7 @@ dpif_ipfix_read_actions(const struct flow *flow,
> >          case OVS_ACTION_ATTR_CHECK_PKT_LEN:
> >          case OVS_ACTION_ATTR_UNSPEC:
> >          case OVS_ACTION_ATTR_DROP:
> > +        case OVS_ACTION_ATTR_ADD_MPLS:
> >          case __OVS_ACTION_ATTR_MAX:
> >          default:
> >              break;
> > diff --git a/ofproto/ofproto-dpif-sflow.c b/ofproto/ofproto-dpif-sflow.c
> > index fdcb9eabb..ca46a9bc4 100644
> > --- a/ofproto/ofproto-dpif-sflow.c
> > +++ b/ofproto/ofproto-dpif-sflow.c
> > @@ -1226,6 +1226,7 @@ dpif_sflow_read_actions(const struct flow *flow,
> >          case OVS_ACTION_ATTR_UNSPEC:
> >          case OVS_ACTION_ATTR_CHECK_PKT_LEN:
> >          case OVS_ACTION_ATTR_DROP:
> > +        case OVS_ACTION_ATTR_ADD_MPLS:
> >          case __OVS_ACTION_ATTR_MAX:
> >          default:
> >              break;
> > diff --git a/ofproto/ofproto-dpif-xlate.c b/ofproto/ofproto-dpif-xlate.c
> > index 7108c8a30..a97534233 100644
> > --- a/ofproto/ofproto-dpif-xlate.c
> > +++ b/ofproto/ofproto-dpif-xlate.c
> > @@ -6381,6 +6381,45 @@ rewrite_flow_encap_ethernet(struct xlate_ctx
> > *ctx,
> >          ctx->error = XLATE_UNSUPPORTED_PACKET_TYPE;
> >      }
> >  }
> > +static void
> > +rewrite_flow_encap_mpls(struct xlate_ctx *ctx,
> > +                        const struct ofpact_encap *encap,
> > +                        struct flow *flow,
> > +                        struct flow_wildcards *wc)
> > +{
> > +    int n;
> > +    uint32_t i;
> > +    uint16_t ether_type;
> > +    const char *ptr = (char *) encap->props;
> > +
> > +     for (i = 0; i < encap->n_props; i++) {
> > +        struct ofpact_ed_prop *prop_ptr =
> > +            ALIGNED_CAST(struct ofpact_ed_prop *, ptr);
> > +        if (prop_ptr->prop_class == OFPPPC_MPLS) {
> > +            switch (prop_ptr->type) {
> > +                case OFPPPT_PROP_MPLS_ETHERTYPE: {
> > +                     struct ofpact_ed_prop_mpls_ethertype
> > *prop_ether_type =
> > +                        ALIGNED_CAST(struct
> > ofpact_ed_prop_mpls_ethertype *,
> > +                                     prop_ptr);
> > +                    ether_type = prop_ether_type->ether_type;
> > +                    break;
> > +                 }
> > +            }
> > +        }
> > +     }
> > +
> > +    wc->masks.packet_type = OVS_BE32_MAX;
> > +    if (flow->packet_type != htonl(PT_MPLS)) {
> > +        memset(&ctx->wc->masks.mpls_lse, 0x0,
> > +               sizeof *wc->masks.mpls_lse * FLOW_MAX_MPLS_LABELS);
> > +        memset(&flow->mpls_lse, 0x0, sizeof *flow->mpls_lse *
> > +               FLOW_MAX_MPLS_LABELS);
> > +    }
> > +    flow->packet_type = htonl(PT_MPLS);
> > +    n = flow_count_mpls_labels(flow, ctx->wc);
> > +    flow_push_mpls(flow, n, htons(ether_type), ctx->wc, true);
> > +}
> > +
> > 
> >  /* For an MD2 NSH header returns a pointer to an ofpbuf with the
> > encoded
> >   * MD2 TLVs provided as encap properties to the encap operation. This
> > @@ -6513,6 +6552,12 @@ xlate_generic_encap_action(struct xlate_ctx *ctx,
> >          case PT_NSH:
> >              encap_data = rewrite_flow_push_nsh(ctx, encap, flow, wc);
> >              break;
> > +        case PT_MPLS:
> > +            rewrite_flow_encap_mpls(ctx, encap,  flow, wc);
> > +            if (!ctx->xbridge->support.add_mpls) {
> > +                ctx->xout->slow |= SLOW_ACTION;
> > +            }
> > +            break;
> >          default:
> >              /* New packet type was checked during decoding. */
> >              OVS_NOT_REACHED();
> > @@ -6582,6 +6627,21 @@ xlate_generic_decap_action(struct xlate_ctx *ctx,
> >              ctx->pending_decap = true;
> >              /* Trigger recirculation. */
> >              return true;
> > +        case PT_MPLS: {
> > +             int n;
> > +             ovs_be16 ethertype;
> > +
> > +             flow->packet_type = decap->new_pkt_type;
> > +             ethertype = pt_ns_type_be(flow->packet_type);
> > +
> > +             n = flow_count_mpls_labels(flow, ctx->wc);
> > +             flow_pop_mpls(flow, n, ethertype, ctx->wc);
> > +             if (!ctx->xbridge->support.add_mpls) {
> > +                ctx->xout->slow |= SLOW_ACTION;
> > +             }
> > +             ctx->pending_decap = true;
> > +             return true;
> > +        }
> >          default:
> >              /* Error handling: drop packet. */
> >              xlate_report_debug(
> > diff --git a/ofproto/ofproto-dpif.c b/ofproto/ofproto-dpif.c
> > index fd0b2fdea..d9a2922e7 100644
> > --- a/ofproto/ofproto-dpif.c
> > +++ b/ofproto/ofproto-dpif.c
> > @@ -1520,6 +1520,44 @@ check_nd_extensions(struct dpif_backer *backer)
> > 
> >      return !error;
> >  }
> > +/* Tests whether 'backer''s datapath supports the
> > + * OVS_ACTION_ATTR_ADD_MPLS action. */
> > +static bool
> > +check_add_mpls(struct dpif_backer *backer)
> > +{
> > +    struct odputil_keybuf keybuf;
> > +    struct ofpbuf actions;
> > +    struct ofpbuf key;
> > +    struct flow flow;
> > +    bool supported;
> > +
> > +    struct odp_flow_key_parms odp_parms = {
> > +        .flow = &flow,
> > +        .probe = true,
> > +    };
> > +
> > +    memset(&flow, 0, sizeof flow);
> > +    ofpbuf_use_stack(&key, &keybuf, sizeof keybuf);
> > +    odp_flow_key_from_flow(&odp_parms, &key);
> > +    ofpbuf_init(&actions, 64);
> > +
> > +    struct ovs_action_add_mpls *mpls;
> > +
> > +    mpls = nl_msg_put_unspec_zero(&actions,
> > +                                  OVS_ACTION_ATTR_ADD_MPLS,
> > +                                  sizeof *mpls);
> > +    mpls->mpls_ethertype = htons(ETH_TYPE_MPLS);
> > +
> > +    supported = dpif_probe_feature(backer->dpif, "add_mpls", &key,
> > +                                   &actions, NULL);
> > +    ofpbuf_uninit(&actions);
> > +    VLOG_INFO("%s: Datapath %s add_mpls action",
> > +              dpif_name(backer->dpif), supported ? "supports"
> > +                                                 : "does not support");
> > +    return supported;
> > +
> > +}
> > +
> > 
> >  #define CHECK_FEATURE__(NAME, SUPPORT, FIELD, VALUE, ETHTYPE)
> > \
> >  static bool
> > \
> > @@ -1590,6 +1628,7 @@ check_support(struct dpif_backer *backer)
> >          dpif_supports_explicit_drop_action(backer->dpif);
> >      backer->rt_support.lb_output_action=
> >          dpif_supports_lb_output_action(backer->dpif);
> > +    backer->rt_support.add_mpls = check_add_mpls(backer);
> > 
> >      /* Flow fields. */
> >      backer->rt_support.odp.ct_state = check_ct_state(backer);
> > diff --git a/ofproto/ofproto-dpif.h b/ofproto/ofproto-dpif.h
> > index b41c3d82a..c04bfff8d 100644
> > --- a/ofproto/ofproto-dpif.h
> > +++ b/ofproto/ofproto-dpif.h
> > @@ -204,7 +204,10 @@ struct group_dpif *group_dpif_lookup(struct
> > ofproto_dpif *,
> >      DPIF_SUPPORT_FIELD(bool, explicit_drop_action, "Explicit Drop
> > action")  \
> >                                                                              \
> >      /* True if the datapath supports balance_tcp optimization */
> > \
> > -    DPIF_SUPPORT_FIELD(bool, lb_output_action, "Optimized Balance TCP
> > mode")
> > +    DPIF_SUPPORT_FIELD(bool, lb_output_action, "Optimized Balance TCP
> > mode")\
> > +
> > \
> > +    /* True if the datapath supports layer 2 MPLS tunnelling */
> > \
> > +    DPIF_SUPPORT_FIELD(bool, add_mpls, "l2 MPLS tunnelling")
> > 
> > 
> >  /* Stores the various features which the corresponding backer supports.
> > */
> > diff --git a/tests/system-traffic.at b/tests/system-traffic.at
> > index fb5b9a36d..b865b5210 100644
> > --- a/tests/system-traffic.at
> > +++ b/tests/system-traffic.at
> > @@ -1027,9 +1027,45 @@ NS_CHECK_EXEC([at_ns0], [ping -q -c 3 -i 0.3 -w 2
> > 10.1.1.2 | FORMAT_PING], [0],
> >  3 packets transmitted, 3 received, 0% packet loss, time 0ms
> >  ])
> > 
> > +OVS_TRAFFIC_VSWITCHD_STOP
> > +AT_CLEANUP
> > +
> > +
> > +AT_SETUP([datapath - ptap mpls actions])
> > +OVS_TRAFFIC_VSWITCHD_START([_ADD_BR([br1])])
> > +
> > +ADD_NAMESPACES(at_ns0, at_ns1)
> > +
> > +ADD_VETH(p0, at_ns0, br0, "10.1.1.1/24")
> > +ADD_VETH(p1, at_ns1, br1, "10.1.1.2/24")
> > +
> > +AT_CHECK([ip link add patch0 type veth peer name patch1])
> > +on_exit 'ip link del patch0'
> > +
> > +AT_CHECK([ip link set dev patch0 up])
> > +AT_CHECK([ip link set dev patch1 up])
> > +AT_CHECK([ovs-vsctl add-port br0 patch0 -- set interface patch0
> > ofport_request=100])
> > +AT_CHECK([ovs-vsctl add-port br1 patch1 -- set interface patch1
> > ofport_request=100])
> > +
> > +AT_DATA([flows.txt], [dnl
> > +table=0,priority=100,dl_type=0x0800 actions=encap(mpls(ether_type=0x8847)),set_mpls_label:2,encap(ethernet),output:100
> > +table=0,priority=100,dl_type=0x8847,mpls_label=2
> > actions=decap(),decap(packet_type(ns=0,type=0)),resubmit(,3)
> > +table=0,priority=10 actions=resubmit(,3)
> > +table=3,priority=10 actions=normal
> > +])
> > +
> > +AT_CHECK([ovs-ofctl  -Oopenflow13 add-flows br0 flows.txt])
> > +AT_CHECK([ovs-ofctl  -Oopenflow13 add-flows br1 flows.txt])
> > +
> > +NS_CHECK_EXEC([at_ns0], [ping -q -c 3 -i 0.3 -w 2 10.1.1.2 |
> > FORMAT_PING], [0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +
> >  NS_CHECK_EXEC([at_ns1], [ping -q -c 3 -i 0.3 -w 2 10.1.1.1 |
> > FORMAT_PING], [0], [dnl
> >  3 packets transmitted, 3 received, 0% packet loss, time 0ms
> >  ])
> > +
> >  OVS_TRAFFIC_VSWITCHD_STOP
> >  AT_CLEANUP
> > 
> > -- 
> > 2.18.4
>

On 1 Apr 2021, at 6:10, Martin Varghese wrote:

> On Wed, Mar 31, 2021 at 03:59:40PM +0200, Eelco Chaudron wrote:
>>
>>
>> On 26 Mar 2021, at 7:21, Martin Varghese wrote:
>>
>>> From: Martin Varghese <martin.varghese@nokia.com>
>>>
>>> The encap & decap actions are extended to support MPLS packet type.
>>> Encap & decap actions adds and removes MPLS header at start of the
>>> packet.
>>
>> Hi Martin,
>>
>> I’m trying to do some real-life testing, and I’m running into 
>> issues. This
>> might be me setting it up wrongly but just wanting to confirm…
>>
>> I’m sending an MPLS packet that contains an ARP packet into a 
>> physical port.
>> This is the packet:
>>
>> Frame 4: 64 bytes on wire (512 bits), 64 bytes captured (512 bits)
>>     Encapsulation type: Ethernet (1)
>>     [Protocols in frame: eth:ethertype:mpls:data]
>> Ethernet II, Src: 00:00:00_00:00:01 (00:00:00:00:00:01), Dst:
>> 00:00:00_00:00:02 (00:00:00:00:00:02)
>>     Destination: 00:00:00_00:00:02 (00:00:00:00:00:02)
>>         Address: 00:00:00_00:00:02 (00:00:00:00:00:02)
>>         .... ..0. .... .... .... .... = LG bit: Globally unique 
>> address
>> (factory default)
>>         .... ...0 .... .... .... .... = IG bit: Individual address 
>> (unicast)
>>     Source: 00:00:00_00:00:01 (00:00:00:00:00:01)
>>         Address: 00:00:00_00:00:01 (00:00:00:00:00:01)
>>         .... ..0. .... .... .... .... = LG bit: Globally unique 
>> address
>> (factory default)
>>         .... ...0 .... .... .... .... = IG bit: Individual address 
>> (unicast)
>>     Type: MPLS label switched packet (0x8847)
>> MultiProtocol Label Switching Header, Label: 100, Exp: 0, S: 1, TTL: 
>> 64
>>     0000 0000 0000 0110 0100 .... .... .... = MPLS Label: 100
>>     .... .... .... .... .... 000. .... .... = MPLS Experimental Bits: 
>> 0
>>     .... .... .... .... .... ...1 .... .... = MPLS Bottom Of Label 
>> Stack: 1
>>     .... .... .... .... .... .... 0100 0000 = MPLS TTL: 64
>> Data (46 bytes)
>>
>> 0000  ff ff ff ff ff ff 52 54 00 88 51 38 08 06 00 01   
>> ......RT..Q8....
>> 0010  08 00 06 04 00 01 52 54 00 88 51 38 01 01 01 65   
>> ......RT..Q8...e
>> 0020  00 00 00 00 00 00 01 01 01 64 27 98 a0 47         
>> .........d'..G
>>     Data:
>> ffffffffffff525400885138080600010800060400015254008851380101016500000000?
>>
>>
>> I’m trying to use the following rules:
>>
>>   ovs-ofctl del-flows ovs_pvp_br0
>>   ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0
>> "priority=100,dl_type=0x8847,mpls_label=100
>> actions=decap(),decap(packet_type(ns=0,type=0x806)),resubmit(,3)"
>>   ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0 "table=3,priority=10
>> actions=normal"
>>
>> With these, I expect the packet to be sent to vnet0, but it’s not. 
>> Actually,
>> the datapath rule looks odd, while the userspace rules seem to match:
>>
>>   $ ovs-dpctl dump-flows
>>   recirc_id(0),in_port(1),eth(),eth_type(0x8847),mpls(label=100/0xfffff,tc=0/0,ttl=0/0x0,bos=1/1),
>> packets:13, bytes:1118, used:0.322s,
>> actions:pop_eth,pop_mpls(eth_type=0x806),recirc(0x19a)
>>   recirc_id(0x19a),in_port(1),eth_type(0x0806), packets:13, 
>> bytes:884,
>> used:0.322s, actions:drop
>>
>>   $ ovs-ofctl dump-flows ovs_pvp_br0 -O OpenFlow13
>>   cookie=0x0, duration=85.007s, table=0, n_packets=51, n_bytes=4386,
>> priority=100,mpls,mpls_label=100
>> actions=decap(),decap(packet_type(ns=0,type=0x806)),resubmit(,3)
>>   cookie=0x0, duration=84.990s, table=3, n_packets=51, n_bytes=3468,
>> priority=10 actions=NORMAL
>>
> The inner packet is ethernet. So the packet type should be 
> (ns=0,type=0)
> ?

Forgot to add that I already tried that to start with, based on the 
example, but as that did not work I tried 0x806.

PS: I have this as a remark in my review notes, i.e., to explain the ns 
and type usage here.


This resulted in packets being counted at the open flow level, but it 
results in NO data path rules. Do get an error though:

2021-04-01T06:53:36.056Z|00141|dpif(handler37)|WARN|system@ovs-system: 
failed to put[create] (Invalid argument) 
ufid:3d2d6f6d-5a66-4ace-8b09-7cdcfa5efc8e 
recirc_id(0),dp_hash(0/0),skb_priority(0/0),in_port(1),skb_mark(0/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),eth(src=00:00:00:00:00:01/00:00:00:00:00:00,dst=00:00:00:00:00:02/00:00:00:00:00:00),eth_type(0x8847),mpls(label=100/0xfffff,tc=0/0,ttl=64/0x0,bos=1/1), 
actions:pop_eth,pop_mpls(eth_type=0x6558),set(eth()),recirc(0x4c)
2021-04-01T06:53:36.056Z|00142|dpif(handler37)|WARN|system@ovs-system: 
execute pop_eth,pop_mpls(eth_type=0x6558),set(eth()),recirc(0x4c) failed 
(Invalid argument) on packet 
mpls,vlan_tci=0x0000,dl_src=00:00:00:00:00:01,dl_dst=00:00:00:00:00:02,mpls_label=100,mpls_tc=0,mpls_ttl=64,mpls_bos=1
  with metadata skb_priority(0),skb_mark(0),in_port(1) mtu 0

Are there missing parts in my kernel that do not get properly detected 
by the feature detection?

$ ovs-appctl dpif/show-dp-features ovs_pvp_br0
Masked set action: Yes
Tunnel push pop: No
Ufid: Yes
Truncate action: Yes
Clone action: Yes
Sample nesting: 10
Conntrack eventmask: Yes
Conntrack clear: Yes
Max dp_hash algorithm: 0
Check pkt length action: Yes
Conntrack timeout policy: Yes
Explicit Drop action: No
Optimized Balance TCP mode: No
l2 MPLS tunnelling: Yes
Max VLAN headers: 2
Max MPLS depth: 3
Recirc: Yes
CT state: Yes
CT zone: Yes
CT mark: Yes
CT label: Yes
CT state NAT: Yes
CT orig tuple: Yes
CT orig tuple for IPv6: Yes
IPv6 ND Extension: No

>>
>> If I use the old way, doing pop_mpls, it works fine:
>>
>>
>> ovs-ofctl del-flows ovs_pvp_br0
>> ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0
>> "priority=100,dl_type=0x8847,mpls_label=100
>> actions=pop_mpls:0x0806,resubmit(,3)"
>> ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0 "table=3,priority=10
>> actions=normal"
>>
>>
>> I also noticed (despite the test example) to make encap work, I had 
>> to set
>> the ethernet MAC addresses, or else the packets were not getting out.
>> So something like:
>>
>>   ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0 
>> "priority=100,in_port=vnet0,actions=encap(mpls(ether_type=0x8847)),set_mpls_label:100,encap(ethernet),,set_field:00:00:00:00:00:02->dl_dst,set_field:00:00:00:00:00:01->dl_src,output:enp5s0f0
>>
>
> The packets are not going out because you are sending the packet on a
> real nic and not on a virtual inerface (veth pair) ?

So for a real NIC we need to set the MAC addresses, maybe some where in 
the documentation we should add an example on how to use this feature?

>> Maybe the test case can be made more realistic? Once I understand the
>> failure, I can continue with the review.
>>
>>
>> Cheers,
>>
>> Eelco
>>
>>
>>
>>> Signed-off-by: Martin Varghese <martin.varghese@nokia.com>
>>> ---
>>>  NEWS                                          |  2 +-
>>>  .../linux/compat/include/linux/openvswitch.h  | 35 ++++++-
>>>  include/openvswitch/ofp-ed-props.h            | 18 ++++
>>>  lib/dpif-netdev.c                             |  1 +
>>>  lib/dpif.c                                    |  1 +
>>>  lib/odp-execute.c                             | 12 +++
>>>  lib/odp-util.c                                | 58 +++++++++---
>>>  lib/ofp-actions.c                             |  5 +
>>>  lib/ofp-ed-props.c                            | 91 
>>> +++++++++++++++++++
>>>  lib/ovs-actions.xml                           | 31 +++++--
>>>  lib/packets.c                                 | 36 ++++++++
>>>  lib/packets.h                                 |  2 +
>>>  ofproto/ofproto-dpif-ipfix.c                  |  1 +
>>>  ofproto/ofproto-dpif-sflow.c                  |  1 +
>>>  ofproto/ofproto-dpif-xlate.c                  | 60 ++++++++++++
>>>  ofproto/ofproto-dpif.c                        | 39 ++++++++
>>>  ofproto/ofproto-dpif.h                        |  5 +-
>>>  tests/system-traffic.at                       | 36 ++++++++
>>>  18 files changed, 410 insertions(+), 24 deletions(-)
>>>
>>> diff --git a/NEWS b/NEWS
>>> index 95cf922aa..4bf4e9e7b 100644
>>> --- a/NEWS
>>> +++ b/NEWS
>>> @@ -120,7 +120,7 @@ v2.14.0 - 17 Aug 2020
>>>     - GTP-U Tunnel Protocol
>>>       * Add two new fields: tun_gtpu_flags, tun_gtpu_msgtype.
>>>       * Only support for userspace datapath.
>>> -
>>> +   - Encap & Decap action support for MPLS packet type.
>>>
>>>  v2.13.0 - 14 Feb 2020
>>>  ---------------------
>>> diff --git a/datapath/linux/compat/include/linux/openvswitch.h
>>> b/datapath/linux/compat/include/linux/openvswitch.h
>>> index 875de2025..8feea7dd4 100644
>>> --- a/datapath/linux/compat/include/linux/openvswitch.h
>>> +++ b/datapath/linux/compat/include/linux/openvswitch.h
>>> @@ -810,8 +810,32 @@ struct ovs_action_push_tnl {
>>>  };
>>>  #endif
>>>
>>> -/**
>>> - * enum ovs_ct_attr - Attributes for %OVS_ACTION_ATTR_CT action.
>>> +/* struct ovs_action_add_mpls - %OVS_ACTION_ATTR_ADD_MPLS action
>>> + * argument.
>>> + * @mpls_lse: MPLS label stack entry to push.
>>> + * @mpls_ethertype: Ethertype to set in the encapsulating ethernet
>>> frame.
>>> + * @tun_flags: MPLS tunnel attributes.
>>> + *
>>> + * The only values @mpls_ethertype should ever be given are
>>> %ETH_P_MPLS_UC and
>>> + * %ETH_P_MPLS_MC, indicating MPLS unicast or multicast. Other are
>>> rejected.
>>> + */
>>> +struct ovs_action_add_mpls {
>>> +	__be32 mpls_lse;
>>> +	__be16 mpls_ethertype; /* Either %ETH_P_MPLS_UC or %ETH_P_MPLS_MC 
>>> */
>>> +	__u16 tun_flags;
>>> +};
>>> +
>>> +#define OVS_MPLS_L3_TUNNEL_FLAG_MASK  (1 << 0) /* Flag to specify 
>>> the
>>> place of
>>> +						* insertion of MPLS header.
>>> +						* When false, the MPLS header
>>> +						* will be inserted at the start
>>> +						* of the packet.
>>> +						* When true, the MPLS header
>>> +						* will be inserted at the start
>>> +						* of the l3 header.
>>> +						*/
>>> +
>>> +/* enum ovs_ct_attr - Attributes for %OVS_ACTION_ATTR_CT action.
>>>   * @OVS_CT_ATTR_COMMIT: If present, commits the connection to the
>>> conntrack
>>>   * table. This allows future packets for the same connection to be
>>> identified
>>>   * as 'established' or 'related'. The flow key for the current 
>>> packet
>>> will
>>> @@ -1001,7 +1025,11 @@ struct check_pkt_len_arg {
>>>   * @OVS_ACTION_ATTR_CHECK_PKT_LEN: Check the packet length and 
>>> execute
>>> a set
>>>   * of actions if greater than the specified packet length, else 
>>> execute
>>>   * another set of actions.
>>> - * @OVS_ACTION_ATTR_DROP: Explicit drop action.
>>> + * @OVS_ACTION_ATTR_ADD_MPLS: Push a new MPLS label stack entry at 
>>> the
>>> + * start of the packet or at the start of the l3 header depending 
>>> on
>>> the value
>>> + * of l3 tunnel flag in the tun_flags field of 
>>> OVS_ACTION_ATTR_ADD_MPLS
>>> + * argument.
>>> +  * @OVS_ACTION_ATTR_DROP: Explicit drop action.
>>>   */
>>>
>>>  enum ovs_action_attr {
>>> @@ -1030,6 +1058,7 @@ enum ovs_action_attr {
>>>  	OVS_ACTION_ATTR_METER,        /* u32 meter number. */
>>>  	OVS_ACTION_ATTR_CLONE,        /* Nested OVS_CLONE_ATTR_*.  */
>>>  	OVS_ACTION_ATTR_CHECK_PKT_LEN, /* Nested OVS_CHECK_PKT_LEN_ATTR_*. 
>>> */
>>> +	OVS_ACTION_ATTR_ADD_MPLS,     /* struct ovs_action_add_mpls. */
>>>
>>>  #ifndef __KERNEL__
>>>  	OVS_ACTION_ATTR_TUNNEL_PUSH,   /* struct ovs_action_push_tnl*/
>>> diff --git a/include/openvswitch/ofp-ed-props.h
>>> b/include/openvswitch/ofp-ed-props.h
>>> index 306c6fe73..c85f3c283 100644
>>> --- a/include/openvswitch/ofp-ed-props.h
>>> +++ b/include/openvswitch/ofp-ed-props.h
>>> @@ -46,6 +46,11 @@ enum ofp_ed_nsh_prop_type {
>>>      OFPPPT_PROP_NSH_TLV = 2,     /* property TLV in NSH */
>>>  };
>>>
>>> +enum ofp_ed_mpls_prop_type {
>>> +    OFPPPT_PROP_MPLS_NONE = 0,    /* unused */
>>> +    OFPPPT_PROP_MPLS_ETHERTYPE = 1,  /* MPLS Ethertype */
>>> +};
>>> +
>>>  /*
>>>   * External representation of encap/decap properties.
>>>   * These must be padded to a multiple of 8 bytes.
>>> @@ -72,6 +77,13 @@ struct ofp_ed_prop_nsh_tlv {
>>>      uint8_t data[0];
>>>  };
>>>
>>> +struct ofp_ed_prop_mpls_ethertype {
>>> +    struct ofp_ed_prop_header header;
>>> +    uint16_t ether_type;         /* MPLS ethertype .*/
>>> +    uint8_t pad[2];          /* Padding to 8 bytes. */
>>> +};
>>> +
>>> +
>>>  /*
>>>   * Internal representation of encap/decap properties
>>>   */
>>> @@ -96,6 +108,12 @@ struct ofpact_ed_prop_nsh_tlv {
>>>      /* tlv_len octets of metadata value, padded to a multiple of 8
>>> bytes. */
>>>      uint8_t data[0];
>>>  };
>>> +
>>> +struct ofpact_ed_prop_mpls_ethertype {
>>> +    struct ofpact_ed_prop header;
>>> +    uint16_t ether_type;         /* MPLS ethertype .*/
>>> +    uint8_t pad[2];          /* Padding to 8 bytes. */
>>> +};
>>>  enum ofperr decode_ed_prop(const struct ofp_ed_prop_header 
>>> **ofp_prop,
>>>                             struct ofpbuf *out, size_t *remaining);
>>>  enum ofperr encode_ed_prop(const struct ofpact_ed_prop **prop,
>>> diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
>>> index 94cc9b80c..bbdea5603 100644
>>> --- a/lib/dpif-netdev.c
>>> +++ b/lib/dpif-netdev.c
>>> @@ -8044,6 +8044,7 @@ dp_execute_cb(void *aux_, struct 
>>> dp_packet_batch
>>> *packets_,
>>>      case OVS_ACTION_ATTR_CT_CLEAR:
>>>      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
>>>      case OVS_ACTION_ATTR_DROP:
>>> +    case OVS_ACTION_ATTR_ADD_MPLS:
>>>      case __OVS_ACTION_ATTR_MAX:
>>>          OVS_NOT_REACHED();
>>>      }
>>> diff --git a/lib/dpif.c b/lib/dpif.c
>>> index 56d0b4a65..bbd1296e3 100644
>>> --- a/lib/dpif.c
>>> +++ b/lib/dpif.c
>>> @@ -1273,6 +1273,7 @@ dpif_execute_helper_cb(void *aux_, struct
>>> dp_packet_batch *packets_,
>>>      case OVS_ACTION_ATTR_UNSPEC:
>>>      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
>>>      case OVS_ACTION_ATTR_DROP:
>>> +    case OVS_ACTION_ATTR_ADD_MPLS:
>>>      case __OVS_ACTION_ATTR_MAX:
>>>          OVS_NOT_REACHED();
>>>      }
>>> diff --git a/lib/odp-execute.c b/lib/odp-execute.c
>>> index 6eeda2a61..2f4cdd92c 100644
>>> --- a/lib/odp-execute.c
>>> +++ b/lib/odp-execute.c
>>> @@ -819,6 +819,7 @@ requires_datapath_assistance(const struct nlattr 
>>> *a)
>>>      case OVS_ACTION_ATTR_POP_NSH:
>>>      case OVS_ACTION_ATTR_CT_CLEAR:
>>>      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
>>> +    case OVS_ACTION_ATTR_ADD_MPLS:
>>>      case OVS_ACTION_ATTR_DROP:
>>>          return false;
>>>
>>> @@ -1061,6 +1062,17 @@ odp_execute_actions(void *dp, struct
>>> dp_packet_batch *batch, bool steal,
>>>              }
>>>              break;
>>>
>>> +        case OVS_ACTION_ATTR_ADD_MPLS: {
>>> +            const struct ovs_action_add_mpls *mpls = 
>>> nl_attr_get(a);
>>> +            bool l3_flag =  mpls->tun_flags &
>>> OVS_MPLS_L3_TUNNEL_FLAG_MASK;
>>> +
>>> +            DP_PACKET_BATCH_FOR_EACH (i, packet, batch) {
>>> +                add_mpls(packet, mpls->mpls_ethertype, 
>>> mpls->mpls_lse,
>>> +                         l3_flag);
>>> +            }
>>> +            break;
>>> +        }
>>> +
>>>          case OVS_ACTION_ATTR_DROP:{
>>>              const enum xlate_error *drop_reason = nl_attr_get(a);
>>>
>>> diff --git a/lib/odp-util.c b/lib/odp-util.c
>>> index a8598d52a..f24e16d08 100644
>>> --- a/lib/odp-util.c
>>> +++ b/lib/odp-util.c
>>> @@ -142,6 +142,8 @@ odp_action_len(uint16_t type)
>>>      case OVS_ACTION_ATTR_PUSH_NSH: return ATTR_LEN_VARIABLE;
>>>      case OVS_ACTION_ATTR_POP_NSH: return 0;
>>>      case OVS_ACTION_ATTR_CHECK_PKT_LEN: return ATTR_LEN_VARIABLE;
>>> +    case OVS_ACTION_ATTR_ADD_MPLS:
>>> +         return sizeof(struct ovs_action_add_mpls);
>>>      case OVS_ACTION_ATTR_DROP: return sizeof(uint32_t);
>>>
>>>      case OVS_ACTION_ATTR_UNSPEC:
>>> @@ -1254,6 +1256,14 @@ format_odp_action(struct ds *ds, const struct
>>> nlattr *a,
>>>      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
>>>          format_odp_check_pkt_len_action(ds, a, portno_names);
>>>          break;
>>> +    case OVS_ACTION_ATTR_ADD_MPLS: {
>>> +        const struct ovs_action_push_mpls *mpls = nl_attr_get(a);
>>> +        ds_put_cstr(ds, "add_mpls(");
>>> +        format_mpls_lse(ds, mpls->mpls_lse);
>>> +        ds_put_format(ds, ",eth_type=0x%"PRIx16")",
>>> +                      ntohs(mpls->mpls_ethertype));
>>> +        break;
>>> +    }
>>>      case OVS_ACTION_ATTR_DROP:
>>>          ds_put_cstr(ds, "drop");
>>>          break;
>>> @@ -7876,7 +7886,8 @@ commit_vlan_action(const struct flow* flow, 
>>> struct
>>> flow *base,
>>>  /* Wildcarding already done at action translation time. */
>>>  static void
>>>  commit_mpls_action(const struct flow *flow, struct flow *base,
>>> -                   struct ofpbuf *odp_actions)
>>> +                   struct ofpbuf *odp_actions, bool pending_encap,
>>> +                   bool pending_decap)
>>>  {
>>>      int base_n = flow_count_mpls_labels(base, NULL);
>>>      int flow_n = flow_count_mpls_labels(flow, NULL);
>>> @@ -7913,7 +7924,11 @@ commit_mpls_action(const struct flow *flow,
>>> struct flow *base,
>>>              if ((!eth_type_mpls(flow->dl_type)) && base_n > 1) {
>>>                  dl_type = htons(ETH_TYPE_MPLS);
>>>              } else {
>>> -                dl_type = flow->dl_type;
>>> +                if ((flow->packet_type == PT_ETH) && pending_decap) 
>>> {
>>> +                    dl_type =  htons(ETH_TYPE_TEB);
>>> +                } else {
>>> +                    dl_type = flow->dl_type;
>>> +                }
>>>              }
>>>              nl_msg_put_be16(odp_actions, OVS_ACTION_ATTR_POP_MPLS,
>>> dl_type);
>>>              ovs_assert(flow_pop_mpls(base, base_n, flow->dl_type,
>>> NULL));
>>> @@ -7924,18 +7939,29 @@ commit_mpls_action(const struct flow *flow,
>>> struct flow *base,
>>>      /* If, after the above popping and setting, there are more LSEs 
>>> in
>>> flow
>>>       * than base then some LSEs need to be pushed. */
>>>      while (base_n < flow_n) {
>>> -        struct ovs_action_push_mpls *mpls;
>>>
>>> -        mpls = nl_msg_put_unspec_zero(odp_actions,
>>> -                                      OVS_ACTION_ATTR_PUSH_MPLS,
>>> -                                      sizeof *mpls);
>>> -        mpls->mpls_ethertype = flow->dl_type;
>>> -        mpls->mpls_lse = flow->mpls_lse[flow_n - base_n - 1];
>>> +        if (pending_encap) {
>>> +             struct ovs_action_add_mpls *mpls;
>>> +
>>> +             mpls = nl_msg_put_unspec_zero(odp_actions,
>>> +                                           
>>> OVS_ACTION_ATTR_ADD_MPLS,
>>> +                                           sizeof *mpls);
>>> +             mpls->mpls_ethertype = flow->dl_type;
>>> +             mpls->mpls_lse = flow->mpls_lse[flow_n - base_n - 1];
>>> +        } else {
>>> +             struct ovs_action_push_mpls *mpls;
>>> +
>>> +             mpls = nl_msg_put_unspec_zero(odp_actions,
>>> +                                           
>>> OVS_ACTION_ATTR_PUSH_MPLS,
>>> +                                           sizeof *mpls);
>>> +             mpls->mpls_ethertype = flow->dl_type;
>>> +             mpls->mpls_lse = flow->mpls_lse[flow_n - base_n - 1];
>>> +        }
>>>          /* Update base flow's MPLS stack, but do not clear L3.  We 
>>> need
>>> the L3
>>>           * headers if the flow is restored later due to returning 
>>> from
>>> a patch
>>>           * port or group bucket. */
>>> -        flow_push_mpls(base, base_n, mpls->mpls_ethertype, NULL,
>>> false);
>>> -        flow_set_mpls_lse(base, 0, mpls->mpls_lse);
>>> +        flow_push_mpls(base, base_n, flow->dl_type, NULL, false);
>>> +        flow_set_mpls_lse(base, 0, flow->mpls_lse[flow_n - base_n -
>>> 1]);
>>>          base_n++;
>>>      }
>>>  }
>>> @@ -8586,6 +8612,10 @@ commit_encap_decap_action(const struct flow
>>> *flow,
>>>              memcpy(&base_flow->dl_dst, &flow->dl_dst,
>>>                     sizeof(*flow) - offsetof(struct flow, dl_dst));
>>>              break;
>>> +        case PT_MPLS:
>>> +            commit_mpls_action(flow, base_flow, odp_actions,
>>> pending_encap,
>>> +                               pending_decap);
>>> +            break;
>>>          default:
>>>              /* Only the above protocols are supported for encap.
>>>               * The check is done at action translation. */
>>> @@ -8608,6 +8638,10 @@ commit_encap_decap_action(const struct flow
>>> *flow,
>>>                  /* pop_nsh. */
>>>                  odp_put_pop_nsh_action(odp_actions);
>>>                  break;
>>> +            case PT_MPLS:
>>> +                commit_mpls_action(flow, base_flow, odp_actions,
>>> pending_encap,
>>> +                                   pending_decap);
>>> +                break;
>>>              default:
>>>                  /* Checks are done during translation. */
>>>                  OVS_NOT_REACHED();
>>> @@ -8653,7 +8687,7 @@ commit_odp_actions(const struct flow *flow, 
>>> struct
>>> flow *base,
>>>      /* Make packet a non-MPLS packet before committing L3/4 
>>> actions,
>>>       * which would otherwise do nothing. */
>>>      if (eth_type_mpls(base->dl_type) && 
>>> !eth_type_mpls(flow->dl_type))
>>> {
>>> -        commit_mpls_action(flow, base, odp_actions);
>>> +        commit_mpls_action(flow, base, odp_actions, false, false);
>>>          mpls_done = true;
>>>      }
>>>      commit_set_nsh_action(flow, base, odp_actions, wc, use_masked);
>>> @@ -8661,7 +8695,7 @@ commit_odp_actions(const struct flow *flow, 
>>> struct
>>> flow *base,
>>>      commit_set_port_action(flow, base, odp_actions, wc, 
>>> use_masked);
>>>      slow2 = commit_set_icmp_action(flow, base, odp_actions, wc);
>>>      if (!mpls_done) {
>>> -        commit_mpls_action(flow, base, odp_actions);
>>> +        commit_mpls_action(flow, base, odp_actions, false, false);
>>>      }
>>>      commit_vlan_action(flow, base, odp_actions, wc);
>>>      commit_set_priority_action(flow, base, odp_actions, wc,
>>> use_masked);
>>> diff --git a/lib/ofp-actions.c b/lib/ofp-actions.c
>>> index 0342a228b..28a12a569 100644
>>> --- a/lib/ofp-actions.c
>>> +++ b/lib/ofp-actions.c
>>> @@ -4441,6 +4441,7 @@ decode_NXAST_RAW_ENCAP(const struct
>>> nx_action_encap *nae,
>>>      switch (ntohl(nae->new_pkt_type)) {
>>>      case PT_ETH:
>>>      case PT_NSH:
>>> +    case PT_MPLS:
>>>          /* Add supported encap header types here. */
>>>          break;
>>>      default:
>>> @@ -4492,6 +4493,8 @@ parse_encap_header(const char *hdr, ovs_be32
>>> *packet_type)
>>>          *packet_type = htonl(PT_ETH);
>>>      } else if (strcmp(hdr, "nsh") == 0) {
>>>          *packet_type = htonl(PT_NSH);
>>> +    } else if (strcmp(hdr, "mpls") == 0) {
>>> +        *packet_type = htonl(PT_MPLS);
>>>      } else {
>>>          return false;
>>>      }
>>> @@ -4573,6 +4576,8 @@ format_encap_pkt_type(const ovs_be32 pkt_type)
>>>          return "ethernet";
>>>      case PT_NSH:
>>>          return "nsh";
>>> +    case PT_MPLS:
>>> +        return "mpls";
>>>      default:
>>>          return "UNKNOWN";
>>>      }
>>> diff --git a/lib/ofp-ed-props.c b/lib/ofp-ed-props.c
>>> index 02a9235d5..fc261e4c6 100644
>>> --- a/lib/ofp-ed-props.c
>>> +++ b/lib/ofp-ed-props.c
>>> @@ -79,6 +79,27 @@ decode_ed_prop(const struct ofp_ed_prop_header
>>> **ofp_prop,
>>>          }
>>>          break;
>>>      }
>>> +    case OFPPPC_MPLS: {
>>> +       switch (prop_type) {
>>> +        case OFPPPT_PROP_MPLS_ETHERTYPE: {
>>> +            struct ofp_ed_prop_mpls_ethertype *opnmt =
>>> +                ALIGNED_CAST(struct ofp_ed_prop_mpls_ethertype *,
>>> *ofp_prop);
>>> +            if (len > sizeof(*opnmt) || len > *remaining) {
>>> +                return OFPERR_NXBAC_BAD_ED_PROP;
>>> +            }
>>> +            struct ofpact_ed_prop_mpls_ethertype *pnmt =
>>> +                    ofpbuf_put_uninit(out, sizeof(*pnmt));
>>> +            pnmt->header.prop_class = prop_class;
>>> +            pnmt->header.type = prop_type;
>>> +            pnmt->header.len = len;
>>> +            pnmt->ether_type = opnmt->ether_type;
>>> +            break;
>>> +        }
>>> +        default:
>>> +            return OFPERR_NXBAC_UNKNOWN_ED_PROP;
>>> +        }
>>> +        break;
>>> +    }
>>>      default:
>>>          return OFPERR_NXBAC_UNKNOWN_ED_PROP;
>>>      }
>>> @@ -134,6 +155,27 @@ encode_ed_prop(const struct ofpact_ed_prop 
>>> **prop,
>>>          }
>>>          break;
>>>      }
>>> +    case OFPPPC_MPLS: {
>>> +       switch ((*prop)->type) {
>>> +       case OFPPPT_PROP_MPLS_ETHERTYPE: {
>>> +           struct ofpact_ed_prop_mpls_ethertype *pnmt =
>>> +                ALIGNED_CAST(struct ofpact_ed_prop_mpls_ethertype 
>>> *,
>>> *prop);
>>> +            struct ofp_ed_prop_mpls_ethertype *opnmt =
>>> +                    ofpbuf_put_uninit(out, sizeof(*opnmt));
>>> +            opnmt->header.prop_class = htons((*prop)->prop_class);
>>> +            opnmt->header.type = (*prop)->type;
>>> +            opnmt->header.len =
>>> +                    offsetof(struct ofpact_ed_prop_mpls_ethertype,
>>> pad);
>>> +            opnmt->ether_type = pnmt->ether_type;
>>> +            prop_len = sizeof(*pnmt);
>>> +            break;
>>> +
>>> +       }
>>> +       default:
>>> +            return OFPERR_OFPBAC_BAD_ARGUMENT;
>>> +       }
>>> +       break;
>>> +    }
>>>      default:
>>>          return OFPERR_OFPBAC_BAD_ARGUMENT;
>>>      }
>>> @@ -181,6 +223,13 @@ parse_ed_prop_type(uint16_t prop_class,
>>>          } else {
>>>              return false;
>>>          }
>>> +    case OFPPPC_MPLS:
>>> +        if (!strcmp(str, "ether_type")) {
>>> +            *type = OFPPPT_PROP_MPLS_ETHERTYPE;
>>> +            return true;
>>> +        } else {
>>> +            return false;
>>> +        }
>>>      default:
>>>          return false;
>>>      }
>>> @@ -259,6 +308,28 @@ parse_ed_prop_value(uint16_t prop_class, 
>>> uint8_t
>>> prop_type OVS_UNUSED,
>>>              OVS_NOT_REACHED();
>>>          }
>>>          break;
>>> +    case OFPPPC_MPLS:
>>> +        switch (prop_type) {
>>> +        case OFPPPT_PROP_MPLS_ETHERTYPE: {
>>> +            uint16_t ethertype;
>>> +            error = str_to_u16(value, "ether_type", &ethertype);
>>> +            if (error != NULL) {
>>> +                return error;
>>> +            }
>>> +            struct ofpact_ed_prop_mpls_ethertype *pnmt =
>>> +                    ofpbuf_put_uninit(out, sizeof(*pnmt));
>>> +            pnmt->header.prop_class = prop_class;
>>> +            pnmt->header.type = prop_type;
>>> +            pnmt->header.len =
>>> +                    offsetof(struct ofpact_ed_prop_mpls_ethertype,
>>> pad);
>>> +            pnmt->ether_type = ethertype;
>>> +
>>> +            break;
>>> +        }
>>> +        default:
>>> +            break;
>>> +      }
>>> +      break;
>>>      default:
>>>          /* Unsupported property classes rejected before. */
>>>          OVS_NOT_REACHED();
>>> @@ -300,6 +371,14 @@ format_ed_prop_type(const struct ofpact_ed_prop
>>> *prop)
>>>              OVS_NOT_REACHED();
>>>          }
>>>          break;
>>> +    case OFPPPC_MPLS:
>>> +         switch (prop->type) {
>>> +         case OFPPPT_PROP_MPLS_ETHERTYPE:
>>> +              return "ether_type";
>>> +         default:
>>> +               OVS_NOT_REACHED();
>>> +         }
>>> +         break;
>>>      default:
>>>          OVS_NOT_REACHED();
>>>      }
>>> @@ -332,6 +411,18 @@ format_ed_prop(struct ds *s OVS_UNUSED,
>>>          default:
>>>              OVS_NOT_REACHED();
>>>          }
>>> +     case OFPPPC_MPLS:
>>> +        switch (prop->type) {
>>> +        case OFPPPT_PROP_MPLS_ETHERTYPE: {
>>> +          struct ofpact_ed_prop_mpls_ethertype *pnmt =
>>> +                ALIGNED_CAST(struct ofpact_ed_prop_mpls_ethertype 
>>> *,
>>> prop);
>>> +            ds_put_format(s, "%s=%d", format_ed_prop_type(prop),
>>> +                          pnmt->ether_type);
>>> +            return;
>>> +        }
>>> +        default:
>>> +            OVS_NOT_REACHED();
>>> +        }
>>>      default:
>>>          OVS_NOT_REACHED();
>>>      }
>>> diff --git a/lib/ovs-actions.xml b/lib/ovs-actions.xml
>>> index a2778de4b..e97f818d9 100644
>>> --- a/lib/ovs-actions.xml
>>> +++ b/lib/ovs-actions.xml
>>> @@ -265,13 +265,13 @@
>>>        </p>
>>>
>>>        <p>
>>> -        When a <code>decap</code> action decapsulates a packet, 
>>> Open
>>> vSwitch
>>> -        raises this error if it does not support the type of inner
>>> packet.
>>> -        <code>decap</code> of an Ethernet header raises this error 
>>> if a
>>> VLAN
>>> -        header is present, <code>decap</code> of a NSH packet 
>>> raises
>>> this error
>>> -        if the NSH inner packet is not Ethernet, IPv4, IPv6, or 
>>> NSH,
>>> and
>>> -        <code>decap</code> of other types of packets is unsupported 
>>> and
>>> also
>>> -        raises this error.
>>> +        The <code>decap</code> action is supported only for packet
>>> types
>>> +        ethernet, NSH and MPLS. Openvswitch raises this error for 
>>> other
>>> +        packet types. When a <code>decap</code> action decapsulates 
>>> a
>>> packet,
>>> +        Open vSwitch raises this error if it does not support the 
>>> type
>>> of inner
>>> +        packet. <code>decap</code> of an Ethernet header raises 
>>> this
>>> error if a
>>> +        VLAN header is present, <code>decap</code> of a NSH packet
>>> raises this
>>> +        error if the NSH inner packet is not Ethernet, IPv4, IPv6, 
>>> or
>>> NSH.
>>>        </p>
>>>
>>>        <p>
>>> @@ -1097,6 +1097,8 @@ for <var>i</var> in [1,<var>n_members</var>]:
>>>        <h2>The <code>encap</code> action</h2>
>>>        <syntax><code>encap(nsh(</code>[<code>md_type=<var>md_type</var></code>]<code>, 
>>> </code>[<code>tlv(<var>class</var>,<var>type</var>,<var>value</var>)</code>]...<code>))</code></syntax>
>>>        <syntax><code>encap(ethernet)</code></syntax>
>>> +
>>> <syntax><code>encap(mpls(ether_type=<var>ether_type</var>))</code>
>>> +      </syntax>
>>>
>>>        <p>
>>>          The <code>encap</code> action encapsulates a packet with a
>>> specified
>>> @@ -1135,6 +1137,12 @@ for <var>i</var> in [1,<var>n_members</var>]:
>>>          source and destination are initially zeroed.
>>>        </p>
>>>
>>> +      <p>
>>> +        The <code>encap(mpls(ethertype=....))</code> variant
>>> encapsulates an
>>> +        ethernet or L3 packet with a MPLS header. The
>>> <var>ethertype</var>
>>> +        could be MPLS unicast (0x8847) or multicast (0x8848)
>>> ethertypes.
>>> +      </p>
>>> +
>>>        <conformance>
>>>          This action is an Open vSwitch extension to OpenFlow 1.3 
>>> and
>>> later,
>>>          introduced in Open vSwitch 2.8.
>>> @@ -1144,6 +1152,9 @@ for <var>i</var> in [1,<var>n_members</var>]:
>>>      <action name="DECAP">
>>>        <h2>The <code>decap</code> action</h2>
>>>        <syntax><code>decap</code></syntax>
>>> +      <syntax><code>decap(packet_type(ns=<var>name_space</var>,
>>> +      type=<var>ethertype</var>))</code></syntax> for decapsulating
>>> MPLS
>>> +      packets.
>>>
>>>        <p>
>>>          Removes an outermost encapsulation from the packet:
>>> @@ -1164,6 +1175,12 @@ for <var>i</var> in [1,<var>n_members</var>]:
>>>            packet type errors.
>>>          </li>
>>>
>>> +        <li>
>>> +          Otherwise, if the packet is a MPLS packet, removes the 
>>> MPLS
>>> header
>>> +          and classifies the inner packet as mentioned in the 
>>> packet
>>> type
>>> +          argument of the decap.
>>> +        </li>
>>> +
>>>          <li>
>>>            Otherwise, raises an unsupported packet type error.
>>>          </li>
>>> diff --git a/lib/packets.c b/lib/packets.c
>>> index 4a7643c5d..5e3c3900f 100644
>>> --- a/lib/packets.c
>>> +++ b/lib/packets.c
>>> @@ -418,6 +418,38 @@ push_mpls(struct dp_packet *packet, ovs_be16
>>> ethtype, ovs_be32 lse)
>>>      pkt_metadata_init_conn(&packet->md);
>>>  }
>>>
>>> +void
>>> +add_mpls(struct dp_packet *packet, ovs_be16 ethtype, ovs_be32 lse, 
>>> bool
>>> l3)
>>> +{
>>> +    char * header;
>>> +
>>> +    if (!eth_type_mpls(ethtype)) {
>>> +        return;
>>> +    }
>>> +
>>> +    if (!l3) {
>>> +        header =  dp_packet_push_uninit(packet, MPLS_HLEN);
>>> +        memcpy(header, &lse, sizeof lse);
>>> +        packet->l2_5_ofs = 0;
>>> +        packet->packet_type = htonl(PT_MPLS);
>>> +    } else {
>>> +        size_t len;
>>> +
>>> +        if (!is_mpls(packet)) {
>>> +            /* Set MPLS label stack offset. */
>>> +            packet->l2_5_ofs = packet->l3_ofs;
>>> +        }
>>> +        set_ethertype(packet, ethtype);
>>> +
>>> +        /* Push new MPLS shim header onto packet. */
>>> +        len = packet->l2_5_ofs;
>>> +        header = dp_packet_resize_l2_5(packet, MPLS_HLEN);
>>> +        memmove(header, header + MPLS_HLEN, len);
>>> +        memcpy(header + len, &lse, sizeof lse);
>>> +    }
>>> +    pkt_metadata_init_conn(&packet->md);
>>> +}
>>> +
>>>  /* If 'packet' is an MPLS packet, removes its outermost MPLS label
>>> stack entry.
>>>   * If the label that was removed was the only MPLS label, changes
>>> 'packet''s
>>>   * Ethertype to 'ethtype' (which ordinarily should not be an MPLS
>>> @@ -429,6 +461,10 @@ pop_mpls(struct dp_packet *packet, ovs_be16
>>> ethtype)
>>>          struct mpls_hdr *mh = dp_packet_l2_5(packet);
>>>          size_t len = packet->l2_5_ofs;
>>>
>>> +        if (ethtype == htons(ETH_TYPE_TEB)) {
>>> +             packet->packet_type = htonl(PT_ETH);
>>> +        }
>>> +
>>>          set_ethertype(packet, ethtype);
>>>          if (get_16aligned_be32(&mh->mpls_lse) & 
>>> htonl(MPLS_BOS_MASK)) {
>>>              dp_packet_set_l2_5(packet, NULL);
>>> diff --git a/lib/packets.h b/lib/packets.h
>>> index 481bc22fa..3f5862e08 100644
>>> --- a/lib/packets.h
>>> +++ b/lib/packets.h
>>> @@ -350,6 +350,8 @@ void set_mpls_lse_label(ovs_be32 *lse, ovs_be32
>>> label);
>>>  void set_mpls_lse_bos(ovs_be32 *lse, uint8_t bos);
>>>  ovs_be32 set_mpls_lse_values(uint8_t ttl, uint8_t tc, uint8_t bos,
>>>                               ovs_be32 label);
>>> +void add_mpls(struct dp_packet *packet, ovs_be16 ethtype, ovs_be32 
>>> lse,
>>> +              bool l3_flag);
>>>
>>>  /* Example:
>>>   *
>>> diff --git a/ofproto/ofproto-dpif-ipfix.c 
>>> b/ofproto/ofproto-dpif-ipfix.c
>>> index 796eb6f88..9280e008e 100644
>>> --- a/ofproto/ofproto-dpif-ipfix.c
>>> +++ b/ofproto/ofproto-dpif-ipfix.c
>>> @@ -3018,6 +3018,7 @@ dpif_ipfix_read_actions(const struct flow 
>>> *flow,
>>>          case OVS_ACTION_ATTR_CHECK_PKT_LEN:
>>>          case OVS_ACTION_ATTR_UNSPEC:
>>>          case OVS_ACTION_ATTR_DROP:
>>> +        case OVS_ACTION_ATTR_ADD_MPLS:
>>>          case __OVS_ACTION_ATTR_MAX:
>>>          default:
>>>              break;
>>> diff --git a/ofproto/ofproto-dpif-sflow.c 
>>> b/ofproto/ofproto-dpif-sflow.c
>>> index fdcb9eabb..ca46a9bc4 100644
>>> --- a/ofproto/ofproto-dpif-sflow.c
>>> +++ b/ofproto/ofproto-dpif-sflow.c
>>> @@ -1226,6 +1226,7 @@ dpif_sflow_read_actions(const struct flow 
>>> *flow,
>>>          case OVS_ACTION_ATTR_UNSPEC:
>>>          case OVS_ACTION_ATTR_CHECK_PKT_LEN:
>>>          case OVS_ACTION_ATTR_DROP:
>>> +        case OVS_ACTION_ATTR_ADD_MPLS:
>>>          case __OVS_ACTION_ATTR_MAX:
>>>          default:
>>>              break;
>>> diff --git a/ofproto/ofproto-dpif-xlate.c 
>>> b/ofproto/ofproto-dpif-xlate.c
>>> index 7108c8a30..a97534233 100644
>>> --- a/ofproto/ofproto-dpif-xlate.c
>>> +++ b/ofproto/ofproto-dpif-xlate.c
>>> @@ -6381,6 +6381,45 @@ rewrite_flow_encap_ethernet(struct xlate_ctx
>>> *ctx,
>>>          ctx->error = XLATE_UNSUPPORTED_PACKET_TYPE;
>>>      }
>>>  }
>>> +static void
>>> +rewrite_flow_encap_mpls(struct xlate_ctx *ctx,
>>> +                        const struct ofpact_encap *encap,
>>> +                        struct flow *flow,
>>> +                        struct flow_wildcards *wc)
>>> +{
>>> +    int n;
>>> +    uint32_t i;
>>> +    uint16_t ether_type;
>>> +    const char *ptr = (char *) encap->props;
>>> +
>>> +     for (i = 0; i < encap->n_props; i++) {
>>> +        struct ofpact_ed_prop *prop_ptr =
>>> +            ALIGNED_CAST(struct ofpact_ed_prop *, ptr);
>>> +        if (prop_ptr->prop_class == OFPPPC_MPLS) {
>>> +            switch (prop_ptr->type) {
>>> +                case OFPPPT_PROP_MPLS_ETHERTYPE: {
>>> +                     struct ofpact_ed_prop_mpls_ethertype
>>> *prop_ether_type =
>>> +                        ALIGNED_CAST(struct
>>> ofpact_ed_prop_mpls_ethertype *,
>>> +                                     prop_ptr);
>>> +                    ether_type = prop_ether_type->ether_type;
>>> +                    break;
>>> +                 }
>>> +            }
>>> +        }
>>> +     }
>>> +
>>> +    wc->masks.packet_type = OVS_BE32_MAX;
>>> +    if (flow->packet_type != htonl(PT_MPLS)) {
>>> +        memset(&ctx->wc->masks.mpls_lse, 0x0,
>>> +               sizeof *wc->masks.mpls_lse * FLOW_MAX_MPLS_LABELS);
>>> +        memset(&flow->mpls_lse, 0x0, sizeof *flow->mpls_lse *
>>> +               FLOW_MAX_MPLS_LABELS);
>>> +    }
>>> +    flow->packet_type = htonl(PT_MPLS);
>>> +    n = flow_count_mpls_labels(flow, ctx->wc);
>>> +    flow_push_mpls(flow, n, htons(ether_type), ctx->wc, true);
>>> +}
>>> +
>>>
>>>  /* For an MD2 NSH header returns a pointer to an ofpbuf with the
>>> encoded
>>>   * MD2 TLVs provided as encap properties to the encap operation. 
>>> This
>>> @@ -6513,6 +6552,12 @@ xlate_generic_encap_action(struct xlate_ctx 
>>> *ctx,
>>>          case PT_NSH:
>>>              encap_data = rewrite_flow_push_nsh(ctx, encap, flow, 
>>> wc);
>>>              break;
>>> +        case PT_MPLS:
>>> +            rewrite_flow_encap_mpls(ctx, encap,  flow, wc);
>>> +            if (!ctx->xbridge->support.add_mpls) {
>>> +                ctx->xout->slow |= SLOW_ACTION;
>>> +            }
>>> +            break;
>>>          default:
>>>              /* New packet type was checked during decoding. */
>>>              OVS_NOT_REACHED();
>>> @@ -6582,6 +6627,21 @@ xlate_generic_decap_action(struct xlate_ctx 
>>> *ctx,
>>>              ctx->pending_decap = true;
>>>              /* Trigger recirculation. */
>>>              return true;
>>> +        case PT_MPLS: {
>>> +             int n;
>>> +             ovs_be16 ethertype;
>>> +
>>> +             flow->packet_type = decap->new_pkt_type;
>>> +             ethertype = pt_ns_type_be(flow->packet_type);
>>> +
>>> +             n = flow_count_mpls_labels(flow, ctx->wc);
>>> +             flow_pop_mpls(flow, n, ethertype, ctx->wc);
>>> +             if (!ctx->xbridge->support.add_mpls) {
>>> +                ctx->xout->slow |= SLOW_ACTION;
>>> +             }
>>> +             ctx->pending_decap = true;
>>> +             return true;
>>> +        }
>>>          default:
>>>              /* Error handling: drop packet. */
>>>              xlate_report_debug(
>>> diff --git a/ofproto/ofproto-dpif.c b/ofproto/ofproto-dpif.c
>>> index fd0b2fdea..d9a2922e7 100644
>>> --- a/ofproto/ofproto-dpif.c
>>> +++ b/ofproto/ofproto-dpif.c
>>> @@ -1520,6 +1520,44 @@ check_nd_extensions(struct dpif_backer 
>>> *backer)
>>>
>>>      return !error;
>>>  }
>>> +/* Tests whether 'backer''s datapath supports the
>>> + * OVS_ACTION_ATTR_ADD_MPLS action. */
>>> +static bool
>>> +check_add_mpls(struct dpif_backer *backer)
>>> +{
>>> +    struct odputil_keybuf keybuf;
>>> +    struct ofpbuf actions;
>>> +    struct ofpbuf key;
>>> +    struct flow flow;
>>> +    bool supported;
>>> +
>>> +    struct odp_flow_key_parms odp_parms = {
>>> +        .flow = &flow,
>>> +        .probe = true,
>>> +    };
>>> +
>>> +    memset(&flow, 0, sizeof flow);
>>> +    ofpbuf_use_stack(&key, &keybuf, sizeof keybuf);
>>> +    odp_flow_key_from_flow(&odp_parms, &key);
>>> +    ofpbuf_init(&actions, 64);
>>> +
>>> +    struct ovs_action_add_mpls *mpls;
>>> +
>>> +    mpls = nl_msg_put_unspec_zero(&actions,
>>> +                                  OVS_ACTION_ATTR_ADD_MPLS,
>>> +                                  sizeof *mpls);
>>> +    mpls->mpls_ethertype = htons(ETH_TYPE_MPLS);
>>> +
>>> +    supported = dpif_probe_feature(backer->dpif, "add_mpls", &key,
>>> +                                   &actions, NULL);
>>> +    ofpbuf_uninit(&actions);
>>> +    VLOG_INFO("%s: Datapath %s add_mpls action",
>>> +              dpif_name(backer->dpif), supported ? "supports"
>>> +                                                 : "does not 
>>> support");
>>> +    return supported;
>>> +
>>> +}
>>> +
>>>
>>>  #define CHECK_FEATURE__(NAME, SUPPORT, FIELD, VALUE, ETHTYPE)
>>> \
>>>  static bool
>>> \
>>> @@ -1590,6 +1628,7 @@ check_support(struct dpif_backer *backer)
>>>          dpif_supports_explicit_drop_action(backer->dpif);
>>>      backer->rt_support.lb_output_action=
>>>          dpif_supports_lb_output_action(backer->dpif);
>>> +    backer->rt_support.add_mpls = check_add_mpls(backer);
>>>
>>>      /* Flow fields. */
>>>      backer->rt_support.odp.ct_state = check_ct_state(backer);
>>> diff --git a/ofproto/ofproto-dpif.h b/ofproto/ofproto-dpif.h
>>> index b41c3d82a..c04bfff8d 100644
>>> --- a/ofproto/ofproto-dpif.h
>>> +++ b/ofproto/ofproto-dpif.h
>>> @@ -204,7 +204,10 @@ struct group_dpif *group_dpif_lookup(struct
>>> ofproto_dpif *,
>>>      DPIF_SUPPORT_FIELD(bool, explicit_drop_action, "Explicit Drop
>>> action")  \
>>>                                                                              \
>>>      /* True if the datapath supports balance_tcp optimization */
>>> \
>>> -    DPIF_SUPPORT_FIELD(bool, lb_output_action, "Optimized Balance 
>>> TCP
>>> mode")
>>> +    DPIF_SUPPORT_FIELD(bool, lb_output_action, "Optimized Balance 
>>> TCP
>>> mode")\
>>> +
>>> \
>>> +    /* True if the datapath supports layer 2 MPLS tunnelling */
>>> \
>>> +    DPIF_SUPPORT_FIELD(bool, add_mpls, "l2 MPLS tunnelling")
>>>
>>>
>>>  /* Stores the various features which the corresponding backer 
>>> supports.
>>> */
>>> diff --git a/tests/system-traffic.at b/tests/system-traffic.at
>>> index fb5b9a36d..b865b5210 100644
>>> --- a/tests/system-traffic.at
>>> +++ b/tests/system-traffic.at
>>> @@ -1027,9 +1027,45 @@ NS_CHECK_EXEC([at_ns0], [ping -q -c 3 -i 0.3 
>>> -w 2
>>> 10.1.1.2 | FORMAT_PING], [0],
>>>  3 packets transmitted, 3 received, 0% packet loss, time 0ms
>>>  ])
>>>
>>> +OVS_TRAFFIC_VSWITCHD_STOP
>>> +AT_CLEANUP
>>> +
>>> +
>>> +AT_SETUP([datapath - ptap mpls actions])
>>> +OVS_TRAFFIC_VSWITCHD_START([_ADD_BR([br1])])
>>> +
>>> +ADD_NAMESPACES(at_ns0, at_ns1)
>>> +
>>> +ADD_VETH(p0, at_ns0, br0, "10.1.1.1/24")
>>> +ADD_VETH(p1, at_ns1, br1, "10.1.1.2/24")
>>> +
>>> +AT_CHECK([ip link add patch0 type veth peer name patch1])
>>> +on_exit 'ip link del patch0'
>>> +
>>> +AT_CHECK([ip link set dev patch0 up])
>>> +AT_CHECK([ip link set dev patch1 up])
>>> +AT_CHECK([ovs-vsctl add-port br0 patch0 -- set interface patch0
>>> ofport_request=100])
>>> +AT_CHECK([ovs-vsctl add-port br1 patch1 -- set interface patch1
>>> ofport_request=100])
>>> +
>>> +AT_DATA([flows.txt], [dnl
>>> +table=0,priority=100,dl_type=0x0800 
>>> actions=encap(mpls(ether_type=0x8847)),set_mpls_label:2,encap(ethernet),output:100
>>> +table=0,priority=100,dl_type=0x8847,mpls_label=2
>>> actions=decap(),decap(packet_type(ns=0,type=0)),resubmit(,3)
>>> +table=0,priority=10 actions=resubmit(,3)
>>> +table=3,priority=10 actions=normal
>>> +])
>>> +
>>> +AT_CHECK([ovs-ofctl  -Oopenflow13 add-flows br0 flows.txt])
>>> +AT_CHECK([ovs-ofctl  -Oopenflow13 add-flows br1 flows.txt])
>>> +
>>> +NS_CHECK_EXEC([at_ns0], [ping -q -c 3 -i 0.3 -w 2 10.1.1.2 |
>>> FORMAT_PING], [0], [dnl
>>> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
>>> +])
>>> +
>>> +
>>>  NS_CHECK_EXEC([at_ns1], [ping -q -c 3 -i 0.3 -w 2 10.1.1.1 |
>>> FORMAT_PING], [0], [dnl
>>>  3 packets transmitted, 3 received, 0% packet loss, time 0ms
>>>  ])
>>> +
>>>  OVS_TRAFFIC_VSWITCHD_STOP
>>>  AT_CLEANUP
>>>
>>> -- 
>>> 2.18.4
>>

On 1 Apr 2021, at 8:59, Eelco Chaudron wrote:

> On 1 Apr 2021, at 6:10, Martin Varghese wrote:
>
>> On Wed, Mar 31, 2021 at 03:59:40PM +0200, Eelco Chaudron wrote:
>>>
>>>
>>> On 26 Mar 2021, at 7:21, Martin Varghese wrote:
>>>
>>>> From: Martin Varghese <martin.varghese@nokia.com>
>>>>
>>>> The encap & decap actions are extended to support MPLS packet type.
>>>> Encap & decap actions adds and removes MPLS header at start of the
>>>> packet.
>>>
>>> Hi Martin,
>>>
>>> I’m trying to do some real-life testing, and I’m running into 
>>> issues. This
>>> might be me setting it up wrongly but just wanting to confirm…
>>>
>>> I’m sending an MPLS packet that contains an ARP packet into a 
>>> physical port.
>>> This is the packet:
>>>
>>> Frame 4: 64 bytes on wire (512 bits), 64 bytes captured (512 bits)
>>>     Encapsulation type: Ethernet (1)
>>>     [Protocols in frame: eth:ethertype:mpls:data]
>>> Ethernet II, Src: 00:00:00_00:00:01 (00:00:00:00:00:01), Dst:
>>> 00:00:00_00:00:02 (00:00:00:00:00:02)
>>>     Destination: 00:00:00_00:00:02 (00:00:00:00:00:02)
>>>         Address: 00:00:00_00:00:02 (00:00:00:00:00:02)
>>>         .... ..0. .... .... .... .... = LG bit: Globally unique 
>>> address
>>> (factory default)
>>>         .... ...0 .... .... .... .... = IG bit: Individual address 
>>> (unicast)
>>>     Source: 00:00:00_00:00:01 (00:00:00:00:00:01)
>>>         Address: 00:00:00_00:00:01 (00:00:00:00:00:01)
>>>         .... ..0. .... .... .... .... = LG bit: Globally unique 
>>> address
>>> (factory default)
>>>         .... ...0 .... .... .... .... = IG bit: Individual address 
>>> (unicast)
>>>     Type: MPLS label switched packet (0x8847)
>>> MultiProtocol Label Switching Header, Label: 100, Exp: 0, S: 1, TTL: 
>>> 64
>>>     0000 0000 0000 0110 0100 .... .... .... = MPLS Label: 100
>>>     .... .... .... .... .... 000. .... .... = MPLS Experimental 
>>> Bits: 0
>>>     .... .... .... .... .... ...1 .... .... = MPLS Bottom Of Label 
>>> Stack: 1
>>>     .... .... .... .... .... .... 0100 0000 = MPLS TTL: 64
>>> Data (46 bytes)
>>>
>>> 0000  ff ff ff ff ff ff 52 54 00 88 51 38 08 06 00 01   
>>> ......RT..Q8....
>>> 0010  08 00 06 04 00 01 52 54 00 88 51 38 01 01 01 65   
>>> ......RT..Q8...e
>>> 0020  00 00 00 00 00 00 01 01 01 64 27 98 a0 47         
>>> .........d'..G
>>>     Data:
>>> ffffffffffff525400885138080600010800060400015254008851380101016500000000?
>>>
>>>
>>> I’m trying to use the following rules:
>>>
>>>   ovs-ofctl del-flows ovs_pvp_br0
>>>   ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0
>>> "priority=100,dl_type=0x8847,mpls_label=100
>>> actions=decap(),decap(packet_type(ns=0,type=0x806)),resubmit(,3)"
>>>   ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0 "table=3,priority=10
>>> actions=normal"
>>>
>>> With these, I expect the packet to be sent to vnet0, but it’s not. 
>>> Actually,
>>> the datapath rule looks odd, while the userspace rules seem to 
>>> match:
>>>
>>>   $ ovs-dpctl dump-flows
>>>   recirc_id(0),in_port(1),eth(),eth_type(0x8847),mpls(label=100/0xfffff,tc=0/0,ttl=0/0x0,bos=1/1),
>>> packets:13, bytes:1118, used:0.322s,
>>> actions:pop_eth,pop_mpls(eth_type=0x806),recirc(0x19a)
>>>   recirc_id(0x19a),in_port(1),eth_type(0x0806), packets:13, 
>>> bytes:884,
>>> used:0.322s, actions:drop
>>>
>>>   $ ovs-ofctl dump-flows ovs_pvp_br0 -O OpenFlow13
>>>   cookie=0x0, duration=85.007s, table=0, n_packets=51, n_bytes=4386,
>>> priority=100,mpls,mpls_label=100
>>> actions=decap(),decap(packet_type(ns=0,type=0x806)),resubmit(,3)
>>>   cookie=0x0, duration=84.990s, table=3, n_packets=51, n_bytes=3468,
>>> priority=10 actions=NORMAL
>>>
>> The inner packet is ethernet. So the packet type should be 
>> (ns=0,type=0)
>> ?
>
> Forgot to add that I already tried that to start with, based on the 
> example, but as that did not work I tried 0x806.
>
> PS: I have this as a remark in my review notes, i.e., to explain the 
> ns and type usage here.
>
>
> This resulted in packets being counted at the open flow level, but it 
> results in NO data path rules. Do get an error though:
>
> 2021-04-01T06:53:36.056Z|00141|dpif(handler37)|WARN|system@ovs-system: 
> failed to put[create] (Invalid argument) 
> ufid:3d2d6f6d-5a66-4ace-8b09-7cdcfa5efc8e 
> recirc_id(0),dp_hash(0/0),skb_priority(0/0),in_port(1),skb_mark(0/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),eth(src=00:00:00:00:00:01/00:00:00:00:00:00,dst=00:00:00:00:00:02/00:00:00:00:00:00),eth_type(0x8847),mpls(label=100/0xfffff,tc=0/0,ttl=64/0x0,bos=1/1), 
> actions:pop_eth,pop_mpls(eth_type=0x6558),set(eth()),recirc(0x4c)
> 2021-04-01T06:53:36.056Z|00142|dpif(handler37)|WARN|system@ovs-system: 
> execute pop_eth,pop_mpls(eth_type=0x6558),set(eth()),recirc(0x4c) 
> failed (Invalid argument) on packet 
> mpls,vlan_tci=0x0000,dl_src=00:00:00:00:00:01,dl_dst=00:00:00:00:00:02,mpls_label=100,mpls_tc=0,mpls_ttl=64,mpls_bos=1
>  with metadata skb_priority(0),skb_mark(0),in_port(1) mtu 0
>
> Are there missing parts in my kernel that do not get properly detected 
> by the feature detection?

Just to be sure I build the “lastest” net kernel, e43accba9b07, 
5.12.0-rc2+, and I see the same problem.

> $ ovs-appctl dpif/show-dp-features ovs_pvp_br0
> Masked set action: Yes
> Tunnel push pop: No
> Ufid: Yes
> Truncate action: Yes
> Clone action: Yes
> Sample nesting: 10
> Conntrack eventmask: Yes
> Conntrack clear: Yes
> Max dp_hash algorithm: 0
> Check pkt length action: Yes
> Conntrack timeout policy: Yes
> Explicit Drop action: No
> Optimized Balance TCP mode: No
> l2 MPLS tunnelling: Yes
> Max VLAN headers: 2
> Max MPLS depth: 3
> Recirc: Yes
> CT state: Yes
> CT zone: Yes
> CT mark: Yes
> CT label: Yes
> CT state NAT: Yes
> CT orig tuple: Yes
> CT orig tuple for IPv6: Yes
> IPv6 ND Extension: No
>
>>>
>>> If I use the old way, doing pop_mpls, it works fine:
>>>
>>>
>>> ovs-ofctl del-flows ovs_pvp_br0
>>> ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0
>>> "priority=100,dl_type=0x8847,mpls_label=100
>>> actions=pop_mpls:0x0806,resubmit(,3)"
>>> ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0 "table=3,priority=10
>>> actions=normal"
>>>
>>>
>>> I also noticed (despite the test example) to make encap work, I had 
>>> to set
>>> the ethernet MAC addresses, or else the packets were not getting 
>>> out.
>>> So something like:
>>>
>>>   ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0 
>>> "priority=100,in_port=vnet0,actions=encap(mpls(ether_type=0x8847)),set_mpls_label:100,encap(ethernet),,set_field:00:00:00:00:00:02->dl_dst,set_field:00:00:00:00:00:01->dl_src,output:enp5s0f0
>>>
>>
>> The packets are not going out because you are sending the packet on a
>> real nic and not on a virtual inerface (veth pair) ?
>
> So for a real NIC we need to set the MAC addresses, maybe some where 
> in the documentation we should add an example on how to use this 
> feature?
>
>>> Maybe the test case can be made more realistic? Once I understand 
>>> the
>>> failure, I can continue with the review.
>>>
>>>
>>> Cheers,
>>>
>>> Eelco
>>>
>>>
>>>
>>>> Signed-off-by: Martin Varghese <martin.varghese@nokia.com>
>>>> ---
>>>>  NEWS                                          |  2 +-
>>>>  .../linux/compat/include/linux/openvswitch.h  | 35 ++++++-
>>>>  include/openvswitch/ofp-ed-props.h            | 18 ++++
>>>>  lib/dpif-netdev.c                             |  1 +
>>>>  lib/dpif.c                                    |  1 +
>>>>  lib/odp-execute.c                             | 12 +++
>>>>  lib/odp-util.c                                | 58 +++++++++---
>>>>  lib/ofp-actions.c                             |  5 +
>>>>  lib/ofp-ed-props.c                            | 91 
>>>> +++++++++++++++++++
>>>>  lib/ovs-actions.xml                           | 31 +++++--
>>>>  lib/packets.c                                 | 36 ++++++++
>>>>  lib/packets.h                                 |  2 +
>>>>  ofproto/ofproto-dpif-ipfix.c                  |  1 +
>>>>  ofproto/ofproto-dpif-sflow.c                  |  1 +
>>>>  ofproto/ofproto-dpif-xlate.c                  | 60 ++++++++++++
>>>>  ofproto/ofproto-dpif.c                        | 39 ++++++++
>>>>  ofproto/ofproto-dpif.h                        |  5 +-
>>>>  tests/system-traffic.at                       | 36 ++++++++
>>>>  18 files changed, 410 insertions(+), 24 deletions(-)
>>>>
>>>> diff --git a/NEWS b/NEWS
>>>> index 95cf922aa..4bf4e9e7b 100644
>>>> --- a/NEWS
>>>> +++ b/NEWS
>>>> @@ -120,7 +120,7 @@ v2.14.0 - 17 Aug 2020
>>>>     - GTP-U Tunnel Protocol
>>>>       * Add two new fields: tun_gtpu_flags, tun_gtpu_msgtype.
>>>>       * Only support for userspace datapath.
>>>> -
>>>> +   - Encap & Decap action support for MPLS packet type.
>>>>
>>>>  v2.13.0 - 14 Feb 2020
>>>>  ---------------------
>>>> diff --git a/datapath/linux/compat/include/linux/openvswitch.h
>>>> b/datapath/linux/compat/include/linux/openvswitch.h
>>>> index 875de2025..8feea7dd4 100644
>>>> --- a/datapath/linux/compat/include/linux/openvswitch.h
>>>> +++ b/datapath/linux/compat/include/linux/openvswitch.h
>>>> @@ -810,8 +810,32 @@ struct ovs_action_push_tnl {
>>>>  };
>>>>  #endif
>>>>
>>>> -/**
>>>> - * enum ovs_ct_attr - Attributes for %OVS_ACTION_ATTR_CT action.
>>>> +/* struct ovs_action_add_mpls - %OVS_ACTION_ATTR_ADD_MPLS action
>>>> + * argument.
>>>> + * @mpls_lse: MPLS label stack entry to push.
>>>> + * @mpls_ethertype: Ethertype to set in the encapsulating ethernet
>>>> frame.
>>>> + * @tun_flags: MPLS tunnel attributes.
>>>> + *
>>>> + * The only values @mpls_ethertype should ever be given are
>>>> %ETH_P_MPLS_UC and
>>>> + * %ETH_P_MPLS_MC, indicating MPLS unicast or multicast. Other are
>>>> rejected.
>>>> + */
>>>> +struct ovs_action_add_mpls {
>>>> +	__be32 mpls_lse;
>>>> +	__be16 mpls_ethertype; /* Either %ETH_P_MPLS_UC or %ETH_P_MPLS_MC 
>>>> */
>>>> +	__u16 tun_flags;
>>>> +};
>>>> +
>>>> +#define OVS_MPLS_L3_TUNNEL_FLAG_MASK  (1 << 0) /* Flag to specify 
>>>> the
>>>> place of
>>>> +						* insertion of MPLS header.
>>>> +						* When false, the MPLS header
>>>> +						* will be inserted at the start
>>>> +						* of the packet.
>>>> +						* When true, the MPLS header
>>>> +						* will be inserted at the start
>>>> +						* of the l3 header.
>>>> +						*/
>>>> +
>>>> +/* enum ovs_ct_attr - Attributes for %OVS_ACTION_ATTR_CT action.
>>>>   * @OVS_CT_ATTR_COMMIT: If present, commits the connection to the
>>>> conntrack
>>>>   * table. This allows future packets for the same connection to be
>>>> identified
>>>>   * as 'established' or 'related'. The flow key for the current 
>>>> packet
>>>> will
>>>> @@ -1001,7 +1025,11 @@ struct check_pkt_len_arg {
>>>>   * @OVS_ACTION_ATTR_CHECK_PKT_LEN: Check the packet length and 
>>>> execute
>>>> a set
>>>>   * of actions if greater than the specified packet length, else 
>>>> execute
>>>>   * another set of actions.
>>>> - * @OVS_ACTION_ATTR_DROP: Explicit drop action.
>>>> + * @OVS_ACTION_ATTR_ADD_MPLS: Push a new MPLS label stack entry at 
>>>> the
>>>> + * start of the packet or at the start of the l3 header depending 
>>>> on
>>>> the value
>>>> + * of l3 tunnel flag in the tun_flags field of 
>>>> OVS_ACTION_ATTR_ADD_MPLS
>>>> + * argument.
>>>> +  * @OVS_ACTION_ATTR_DROP: Explicit drop action.
>>>>   */
>>>>
>>>>  enum ovs_action_attr {
>>>> @@ -1030,6 +1058,7 @@ enum ovs_action_attr {
>>>>  	OVS_ACTION_ATTR_METER,        /* u32 meter number. */
>>>>  	OVS_ACTION_ATTR_CLONE,        /* Nested OVS_CLONE_ATTR_*.  */
>>>>  	OVS_ACTION_ATTR_CHECK_PKT_LEN, /* Nested 
>>>> OVS_CHECK_PKT_LEN_ATTR_*. */
>>>> +	OVS_ACTION_ATTR_ADD_MPLS,     /* struct ovs_action_add_mpls. */
>>>>
>>>>  #ifndef __KERNEL__
>>>>  	OVS_ACTION_ATTR_TUNNEL_PUSH,   /* struct ovs_action_push_tnl*/
>>>> diff --git a/include/openvswitch/ofp-ed-props.h
>>>> b/include/openvswitch/ofp-ed-props.h
>>>> index 306c6fe73..c85f3c283 100644
>>>> --- a/include/openvswitch/ofp-ed-props.h
>>>> +++ b/include/openvswitch/ofp-ed-props.h
>>>> @@ -46,6 +46,11 @@ enum ofp_ed_nsh_prop_type {
>>>>      OFPPPT_PROP_NSH_TLV = 2,     /* property TLV in NSH */
>>>>  };
>>>>
>>>> +enum ofp_ed_mpls_prop_type {
>>>> +    OFPPPT_PROP_MPLS_NONE = 0,    /* unused */
>>>> +    OFPPPT_PROP_MPLS_ETHERTYPE = 1,  /* MPLS Ethertype */
>>>> +};
>>>> +
>>>>  /*
>>>>   * External representation of encap/decap properties.
>>>>   * These must be padded to a multiple of 8 bytes.
>>>> @@ -72,6 +77,13 @@ struct ofp_ed_prop_nsh_tlv {
>>>>      uint8_t data[0];
>>>>  };
>>>>
>>>> +struct ofp_ed_prop_mpls_ethertype {
>>>> +    struct ofp_ed_prop_header header;
>>>> +    uint16_t ether_type;         /* MPLS ethertype .*/
>>>> +    uint8_t pad[2];          /* Padding to 8 bytes. */
>>>> +};
>>>> +
>>>> +
>>>>  /*
>>>>   * Internal representation of encap/decap properties
>>>>   */
>>>> @@ -96,6 +108,12 @@ struct ofpact_ed_prop_nsh_tlv {
>>>>      /* tlv_len octets of metadata value, padded to a multiple of 8
>>>> bytes. */
>>>>      uint8_t data[0];
>>>>  };
>>>> +
>>>> +struct ofpact_ed_prop_mpls_ethertype {
>>>> +    struct ofpact_ed_prop header;
>>>> +    uint16_t ether_type;         /* MPLS ethertype .*/
>>>> +    uint8_t pad[2];          /* Padding to 8 bytes. */
>>>> +};
>>>>  enum ofperr decode_ed_prop(const struct ofp_ed_prop_header 
>>>> **ofp_prop,
>>>>                             struct ofpbuf *out, size_t *remaining);
>>>>  enum ofperr encode_ed_prop(const struct ofpact_ed_prop **prop,
>>>> diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
>>>> index 94cc9b80c..bbdea5603 100644
>>>> --- a/lib/dpif-netdev.c
>>>> +++ b/lib/dpif-netdev.c
>>>> @@ -8044,6 +8044,7 @@ dp_execute_cb(void *aux_, struct 
>>>> dp_packet_batch
>>>> *packets_,
>>>>      case OVS_ACTION_ATTR_CT_CLEAR:
>>>>      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
>>>>      case OVS_ACTION_ATTR_DROP:
>>>> +    case OVS_ACTION_ATTR_ADD_MPLS:
>>>>      case __OVS_ACTION_ATTR_MAX:
>>>>          OVS_NOT_REACHED();
>>>>      }
>>>> diff --git a/lib/dpif.c b/lib/dpif.c
>>>> index 56d0b4a65..bbd1296e3 100644
>>>> --- a/lib/dpif.c
>>>> +++ b/lib/dpif.c
>>>> @@ -1273,6 +1273,7 @@ dpif_execute_helper_cb(void *aux_, struct
>>>> dp_packet_batch *packets_,
>>>>      case OVS_ACTION_ATTR_UNSPEC:
>>>>      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
>>>>      case OVS_ACTION_ATTR_DROP:
>>>> +    case OVS_ACTION_ATTR_ADD_MPLS:
>>>>      case __OVS_ACTION_ATTR_MAX:
>>>>          OVS_NOT_REACHED();
>>>>      }
>>>> diff --git a/lib/odp-execute.c b/lib/odp-execute.c
>>>> index 6eeda2a61..2f4cdd92c 100644
>>>> --- a/lib/odp-execute.c
>>>> +++ b/lib/odp-execute.c
>>>> @@ -819,6 +819,7 @@ requires_datapath_assistance(const struct 
>>>> nlattr *a)
>>>>      case OVS_ACTION_ATTR_POP_NSH:
>>>>      case OVS_ACTION_ATTR_CT_CLEAR:
>>>>      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
>>>> +    case OVS_ACTION_ATTR_ADD_MPLS:
>>>>      case OVS_ACTION_ATTR_DROP:
>>>>          return false;
>>>>
>>>> @@ -1061,6 +1062,17 @@ odp_execute_actions(void *dp, struct
>>>> dp_packet_batch *batch, bool steal,
>>>>              }
>>>>              break;
>>>>
>>>> +        case OVS_ACTION_ATTR_ADD_MPLS: {
>>>> +            const struct ovs_action_add_mpls *mpls = 
>>>> nl_attr_get(a);
>>>> +            bool l3_flag =  mpls->tun_flags &
>>>> OVS_MPLS_L3_TUNNEL_FLAG_MASK;
>>>> +
>>>> +            DP_PACKET_BATCH_FOR_EACH (i, packet, batch) {
>>>> +                add_mpls(packet, mpls->mpls_ethertype, 
>>>> mpls->mpls_lse,
>>>> +                         l3_flag);
>>>> +            }
>>>> +            break;
>>>> +        }
>>>> +
>>>>          case OVS_ACTION_ATTR_DROP:{
>>>>              const enum xlate_error *drop_reason = nl_attr_get(a);
>>>>
>>>> diff --git a/lib/odp-util.c b/lib/odp-util.c
>>>> index a8598d52a..f24e16d08 100644
>>>> --- a/lib/odp-util.c
>>>> +++ b/lib/odp-util.c
>>>> @@ -142,6 +142,8 @@ odp_action_len(uint16_t type)
>>>>      case OVS_ACTION_ATTR_PUSH_NSH: return ATTR_LEN_VARIABLE;
>>>>      case OVS_ACTION_ATTR_POP_NSH: return 0;
>>>>      case OVS_ACTION_ATTR_CHECK_PKT_LEN: return ATTR_LEN_VARIABLE;
>>>> +    case OVS_ACTION_ATTR_ADD_MPLS:
>>>> +         return sizeof(struct ovs_action_add_mpls);
>>>>      case OVS_ACTION_ATTR_DROP: return sizeof(uint32_t);
>>>>
>>>>      case OVS_ACTION_ATTR_UNSPEC:
>>>> @@ -1254,6 +1256,14 @@ format_odp_action(struct ds *ds, const 
>>>> struct
>>>> nlattr *a,
>>>>      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
>>>>          format_odp_check_pkt_len_action(ds, a, portno_names);
>>>>          break;
>>>> +    case OVS_ACTION_ATTR_ADD_MPLS: {
>>>> +        const struct ovs_action_push_mpls *mpls = nl_attr_get(a);
>>>> +        ds_put_cstr(ds, "add_mpls(");
>>>> +        format_mpls_lse(ds, mpls->mpls_lse);
>>>> +        ds_put_format(ds, ",eth_type=0x%"PRIx16")",
>>>> +                      ntohs(mpls->mpls_ethertype));
>>>> +        break;
>>>> +    }
>>>>      case OVS_ACTION_ATTR_DROP:
>>>>          ds_put_cstr(ds, "drop");
>>>>          break;
>>>> @@ -7876,7 +7886,8 @@ commit_vlan_action(const struct flow* flow, 
>>>> struct
>>>> flow *base,
>>>>  /* Wildcarding already done at action translation time. */
>>>>  static void
>>>>  commit_mpls_action(const struct flow *flow, struct flow *base,
>>>> -                   struct ofpbuf *odp_actions)
>>>> +                   struct ofpbuf *odp_actions, bool pending_encap,
>>>> +                   bool pending_decap)
>>>>  {
>>>>      int base_n = flow_count_mpls_labels(base, NULL);
>>>>      int flow_n = flow_count_mpls_labels(flow, NULL);
>>>> @@ -7913,7 +7924,11 @@ commit_mpls_action(const struct flow *flow,
>>>> struct flow *base,
>>>>              if ((!eth_type_mpls(flow->dl_type)) && base_n > 1) {
>>>>                  dl_type = htons(ETH_TYPE_MPLS);
>>>>              } else {
>>>> -                dl_type = flow->dl_type;
>>>> +                if ((flow->packet_type == PT_ETH) && 
>>>> pending_decap) {
>>>> +                    dl_type =  htons(ETH_TYPE_TEB);
>>>> +                } else {
>>>> +                    dl_type = flow->dl_type;
>>>> +                }
>>>>              }
>>>>              nl_msg_put_be16(odp_actions, OVS_ACTION_ATTR_POP_MPLS,
>>>> dl_type);
>>>>              ovs_assert(flow_pop_mpls(base, base_n, flow->dl_type,
>>>> NULL));
>>>> @@ -7924,18 +7939,29 @@ commit_mpls_action(const struct flow *flow,
>>>> struct flow *base,
>>>>      /* If, after the above popping and setting, there are more 
>>>> LSEs in
>>>> flow
>>>>       * than base then some LSEs need to be pushed. */
>>>>      while (base_n < flow_n) {
>>>> -        struct ovs_action_push_mpls *mpls;
>>>>
>>>> -        mpls = nl_msg_put_unspec_zero(odp_actions,
>>>> -                                      OVS_ACTION_ATTR_PUSH_MPLS,
>>>> -                                      sizeof *mpls);
>>>> -        mpls->mpls_ethertype = flow->dl_type;
>>>> -        mpls->mpls_lse = flow->mpls_lse[flow_n - base_n - 1];
>>>> +        if (pending_encap) {
>>>> +             struct ovs_action_add_mpls *mpls;
>>>> +
>>>> +             mpls = nl_msg_put_unspec_zero(odp_actions,
>>>> +                                           
>>>> OVS_ACTION_ATTR_ADD_MPLS,
>>>> +                                           sizeof *mpls);
>>>> +             mpls->mpls_ethertype = flow->dl_type;
>>>> +             mpls->mpls_lse = flow->mpls_lse[flow_n - base_n - 1];
>>>> +        } else {
>>>> +             struct ovs_action_push_mpls *mpls;
>>>> +
>>>> +             mpls = nl_msg_put_unspec_zero(odp_actions,
>>>> +                                           
>>>> OVS_ACTION_ATTR_PUSH_MPLS,
>>>> +                                           sizeof *mpls);
>>>> +             mpls->mpls_ethertype = flow->dl_type;
>>>> +             mpls->mpls_lse = flow->mpls_lse[flow_n - base_n - 1];
>>>> +        }
>>>>          /* Update base flow's MPLS stack, but do not clear L3.  We 
>>>> need
>>>> the L3
>>>>           * headers if the flow is restored later due to returning 
>>>> from
>>>> a patch
>>>>           * port or group bucket. */
>>>> -        flow_push_mpls(base, base_n, mpls->mpls_ethertype, NULL,
>>>> false);
>>>> -        flow_set_mpls_lse(base, 0, mpls->mpls_lse);
>>>> +        flow_push_mpls(base, base_n, flow->dl_type, NULL, false);
>>>> +        flow_set_mpls_lse(base, 0, flow->mpls_lse[flow_n - base_n 
>>>> -
>>>> 1]);
>>>>          base_n++;
>>>>      }
>>>>  }
>>>> @@ -8586,6 +8612,10 @@ commit_encap_decap_action(const struct flow
>>>> *flow,
>>>>              memcpy(&base_flow->dl_dst, &flow->dl_dst,
>>>>                     sizeof(*flow) - offsetof(struct flow, dl_dst));
>>>>              break;
>>>> +        case PT_MPLS:
>>>> +            commit_mpls_action(flow, base_flow, odp_actions,
>>>> pending_encap,
>>>> +                               pending_decap);
>>>> +            break;
>>>>          default:
>>>>              /* Only the above protocols are supported for encap.
>>>>               * The check is done at action translation. */
>>>> @@ -8608,6 +8638,10 @@ commit_encap_decap_action(const struct flow
>>>> *flow,
>>>>                  /* pop_nsh. */
>>>>                  odp_put_pop_nsh_action(odp_actions);
>>>>                  break;
>>>> +            case PT_MPLS:
>>>> +                commit_mpls_action(flow, base_flow, odp_actions,
>>>> pending_encap,
>>>> +                                   pending_decap);
>>>> +                break;
>>>>              default:
>>>>                  /* Checks are done during translation. */
>>>>                  OVS_NOT_REACHED();
>>>> @@ -8653,7 +8687,7 @@ commit_odp_actions(const struct flow *flow, 
>>>> struct
>>>> flow *base,
>>>>      /* Make packet a non-MPLS packet before committing L3/4 
>>>> actions,
>>>>       * which would otherwise do nothing. */
>>>>      if (eth_type_mpls(base->dl_type) && 
>>>> !eth_type_mpls(flow->dl_type))
>>>> {
>>>> -        commit_mpls_action(flow, base, odp_actions);
>>>> +        commit_mpls_action(flow, base, odp_actions, false, false);
>>>>          mpls_done = true;
>>>>      }
>>>>      commit_set_nsh_action(flow, base, odp_actions, wc, 
>>>> use_masked);
>>>> @@ -8661,7 +8695,7 @@ commit_odp_actions(const struct flow *flow, 
>>>> struct
>>>> flow *base,
>>>>      commit_set_port_action(flow, base, odp_actions, wc, 
>>>> use_masked);
>>>>      slow2 = commit_set_icmp_action(flow, base, odp_actions, wc);
>>>>      if (!mpls_done) {
>>>> -        commit_mpls_action(flow, base, odp_actions);
>>>> +        commit_mpls_action(flow, base, odp_actions, false, false);
>>>>      }
>>>>      commit_vlan_action(flow, base, odp_actions, wc);
>>>>      commit_set_priority_action(flow, base, odp_actions, wc,
>>>> use_masked);
>>>> diff --git a/lib/ofp-actions.c b/lib/ofp-actions.c
>>>> index 0342a228b..28a12a569 100644
>>>> --- a/lib/ofp-actions.c
>>>> +++ b/lib/ofp-actions.c
>>>> @@ -4441,6 +4441,7 @@ decode_NXAST_RAW_ENCAP(const struct
>>>> nx_action_encap *nae,
>>>>      switch (ntohl(nae->new_pkt_type)) {
>>>>      case PT_ETH:
>>>>      case PT_NSH:
>>>> +    case PT_MPLS:
>>>>          /* Add supported encap header types here. */
>>>>          break;
>>>>      default:
>>>> @@ -4492,6 +4493,8 @@ parse_encap_header(const char *hdr, ovs_be32
>>>> *packet_type)
>>>>          *packet_type = htonl(PT_ETH);
>>>>      } else if (strcmp(hdr, "nsh") == 0) {
>>>>          *packet_type = htonl(PT_NSH);
>>>> +    } else if (strcmp(hdr, "mpls") == 0) {
>>>> +        *packet_type = htonl(PT_MPLS);
>>>>      } else {
>>>>          return false;
>>>>      }
>>>> @@ -4573,6 +4576,8 @@ format_encap_pkt_type(const ovs_be32 
>>>> pkt_type)
>>>>          return "ethernet";
>>>>      case PT_NSH:
>>>>          return "nsh";
>>>> +    case PT_MPLS:
>>>> +        return "mpls";
>>>>      default:
>>>>          return "UNKNOWN";
>>>>      }
>>>> diff --git a/lib/ofp-ed-props.c b/lib/ofp-ed-props.c
>>>> index 02a9235d5..fc261e4c6 100644
>>>> --- a/lib/ofp-ed-props.c
>>>> +++ b/lib/ofp-ed-props.c
>>>> @@ -79,6 +79,27 @@ decode_ed_prop(const struct ofp_ed_prop_header
>>>> **ofp_prop,
>>>>          }
>>>>          break;
>>>>      }
>>>> +    case OFPPPC_MPLS: {
>>>> +       switch (prop_type) {
>>>> +        case OFPPPT_PROP_MPLS_ETHERTYPE: {
>>>> +            struct ofp_ed_prop_mpls_ethertype *opnmt =
>>>> +                ALIGNED_CAST(struct ofp_ed_prop_mpls_ethertype *,
>>>> *ofp_prop);
>>>> +            if (len > sizeof(*opnmt) || len > *remaining) {
>>>> +                return OFPERR_NXBAC_BAD_ED_PROP;
>>>> +            }
>>>> +            struct ofpact_ed_prop_mpls_ethertype *pnmt =
>>>> +                    ofpbuf_put_uninit(out, sizeof(*pnmt));
>>>> +            pnmt->header.prop_class = prop_class;
>>>> +            pnmt->header.type = prop_type;
>>>> +            pnmt->header.len = len;
>>>> +            pnmt->ether_type = opnmt->ether_type;
>>>> +            break;
>>>> +        }
>>>> +        default:
>>>> +            return OFPERR_NXBAC_UNKNOWN_ED_PROP;
>>>> +        }
>>>> +        break;
>>>> +    }
>>>>      default:
>>>>          return OFPERR_NXBAC_UNKNOWN_ED_PROP;
>>>>      }
>>>> @@ -134,6 +155,27 @@ encode_ed_prop(const struct ofpact_ed_prop 
>>>> **prop,
>>>>          }
>>>>          break;
>>>>      }
>>>> +    case OFPPPC_MPLS: {
>>>> +       switch ((*prop)->type) {
>>>> +       case OFPPPT_PROP_MPLS_ETHERTYPE: {
>>>> +           struct ofpact_ed_prop_mpls_ethertype *pnmt =
>>>> +                ALIGNED_CAST(struct ofpact_ed_prop_mpls_ethertype 
>>>> *,
>>>> *prop);
>>>> +            struct ofp_ed_prop_mpls_ethertype *opnmt =
>>>> +                    ofpbuf_put_uninit(out, sizeof(*opnmt));
>>>> +            opnmt->header.prop_class = htons((*prop)->prop_class);
>>>> +            opnmt->header.type = (*prop)->type;
>>>> +            opnmt->header.len =
>>>> +                    offsetof(struct ofpact_ed_prop_mpls_ethertype,
>>>> pad);
>>>> +            opnmt->ether_type = pnmt->ether_type;
>>>> +            prop_len = sizeof(*pnmt);
>>>> +            break;
>>>> +
>>>> +       }
>>>> +       default:
>>>> +            return OFPERR_OFPBAC_BAD_ARGUMENT;
>>>> +       }
>>>> +       break;
>>>> +    }
>>>>      default:
>>>>          return OFPERR_OFPBAC_BAD_ARGUMENT;
>>>>      }
>>>> @@ -181,6 +223,13 @@ parse_ed_prop_type(uint16_t prop_class,
>>>>          } else {
>>>>              return false;
>>>>          }
>>>> +    case OFPPPC_MPLS:
>>>> +        if (!strcmp(str, "ether_type")) {
>>>> +            *type = OFPPPT_PROP_MPLS_ETHERTYPE;
>>>> +            return true;
>>>> +        } else {
>>>> +            return false;
>>>> +        }
>>>>      default:
>>>>          return false;
>>>>      }
>>>> @@ -259,6 +308,28 @@ parse_ed_prop_value(uint16_t prop_class, 
>>>> uint8_t
>>>> prop_type OVS_UNUSED,
>>>>              OVS_NOT_REACHED();
>>>>          }
>>>>          break;
>>>> +    case OFPPPC_MPLS:
>>>> +        switch (prop_type) {
>>>> +        case OFPPPT_PROP_MPLS_ETHERTYPE: {
>>>> +            uint16_t ethertype;
>>>> +            error = str_to_u16(value, "ether_type", &ethertype);
>>>> +            if (error != NULL) {
>>>> +                return error;
>>>> +            }
>>>> +            struct ofpact_ed_prop_mpls_ethertype *pnmt =
>>>> +                    ofpbuf_put_uninit(out, sizeof(*pnmt));
>>>> +            pnmt->header.prop_class = prop_class;
>>>> +            pnmt->header.type = prop_type;
>>>> +            pnmt->header.len =
>>>> +                    offsetof(struct ofpact_ed_prop_mpls_ethertype,
>>>> pad);
>>>> +            pnmt->ether_type = ethertype;
>>>> +
>>>> +            break;
>>>> +        }
>>>> +        default:
>>>> +            break;
>>>> +      }
>>>> +      break;
>>>>      default:
>>>>          /* Unsupported property classes rejected before. */
>>>>          OVS_NOT_REACHED();
>>>> @@ -300,6 +371,14 @@ format_ed_prop_type(const struct 
>>>> ofpact_ed_prop
>>>> *prop)
>>>>              OVS_NOT_REACHED();
>>>>          }
>>>>          break;
>>>> +    case OFPPPC_MPLS:
>>>> +         switch (prop->type) {
>>>> +         case OFPPPT_PROP_MPLS_ETHERTYPE:
>>>> +              return "ether_type";
>>>> +         default:
>>>> +               OVS_NOT_REACHED();
>>>> +         }
>>>> +         break;
>>>>      default:
>>>>          OVS_NOT_REACHED();
>>>>      }
>>>> @@ -332,6 +411,18 @@ format_ed_prop(struct ds *s OVS_UNUSED,
>>>>          default:
>>>>              OVS_NOT_REACHED();
>>>>          }
>>>> +     case OFPPPC_MPLS:
>>>> +        switch (prop->type) {
>>>> +        case OFPPPT_PROP_MPLS_ETHERTYPE: {
>>>> +          struct ofpact_ed_prop_mpls_ethertype *pnmt =
>>>> +                ALIGNED_CAST(struct ofpact_ed_prop_mpls_ethertype 
>>>> *,
>>>> prop);
>>>> +            ds_put_format(s, "%s=%d", format_ed_prop_type(prop),
>>>> +                          pnmt->ether_type);
>>>> +            return;
>>>> +        }
>>>> +        default:
>>>> +            OVS_NOT_REACHED();
>>>> +        }
>>>>      default:
>>>>          OVS_NOT_REACHED();
>>>>      }
>>>> diff --git a/lib/ovs-actions.xml b/lib/ovs-actions.xml
>>>> index a2778de4b..e97f818d9 100644
>>>> --- a/lib/ovs-actions.xml
>>>> +++ b/lib/ovs-actions.xml
>>>> @@ -265,13 +265,13 @@
>>>>        </p>
>>>>
>>>>        <p>
>>>> -        When a <code>decap</code> action decapsulates a packet, 
>>>> Open
>>>> vSwitch
>>>> -        raises this error if it does not support the type of inner
>>>> packet.
>>>> -        <code>decap</code> of an Ethernet header raises this error 
>>>> if a
>>>> VLAN
>>>> -        header is present, <code>decap</code> of a NSH packet 
>>>> raises
>>>> this error
>>>> -        if the NSH inner packet is not Ethernet, IPv4, IPv6, or 
>>>> NSH,
>>>> and
>>>> -        <code>decap</code> of other types of packets is 
>>>> unsupported and
>>>> also
>>>> -        raises this error.
>>>> +        The <code>decap</code> action is supported only for packet
>>>> types
>>>> +        ethernet, NSH and MPLS. Openvswitch raises this error for 
>>>> other
>>>> +        packet types. When a <code>decap</code> action 
>>>> decapsulates a
>>>> packet,
>>>> +        Open vSwitch raises this error if it does not support the 
>>>> type
>>>> of inner
>>>> +        packet. <code>decap</code> of an Ethernet header raises 
>>>> this
>>>> error if a
>>>> +        VLAN header is present, <code>decap</code> of a NSH packet
>>>> raises this
>>>> +        error if the NSH inner packet is not Ethernet, IPv4, IPv6, 
>>>> or
>>>> NSH.
>>>>        </p>
>>>>
>>>>        <p>
>>>> @@ -1097,6 +1097,8 @@ for <var>i</var> in [1,<var>n_members</var>]:
>>>>        <h2>The <code>encap</code> action</h2>
>>>>        <syntax><code>encap(nsh(</code>[<code>md_type=<var>md_type</var></code>]<code>, 
>>>> </code>[<code>tlv(<var>class</var>,<var>type</var>,<var>value</var>)</code>]...<code>))</code></syntax>
>>>>        <syntax><code>encap(ethernet)</code></syntax>
>>>> +
>>>> <syntax><code>encap(mpls(ether_type=<var>ether_type</var>))</code>
>>>> +      </syntax>
>>>>
>>>>        <p>
>>>>          The <code>encap</code> action encapsulates a packet with a
>>>> specified
>>>> @@ -1135,6 +1137,12 @@ for <var>i</var> in 
>>>> [1,<var>n_members</var>]:
>>>>          source and destination are initially zeroed.
>>>>        </p>
>>>>
>>>> +      <p>
>>>> +        The <code>encap(mpls(ethertype=....))</code> variant
>>>> encapsulates an
>>>> +        ethernet or L3 packet with a MPLS header. The
>>>> <var>ethertype</var>
>>>> +        could be MPLS unicast (0x8847) or multicast (0x8848)
>>>> ethertypes.
>>>> +      </p>
>>>> +
>>>>        <conformance>
>>>>          This action is an Open vSwitch extension to OpenFlow 1.3 
>>>> and
>>>> later,
>>>>          introduced in Open vSwitch 2.8.
>>>> @@ -1144,6 +1152,9 @@ for <var>i</var> in [1,<var>n_members</var>]:
>>>>      <action name="DECAP">
>>>>        <h2>The <code>decap</code> action</h2>
>>>>        <syntax><code>decap</code></syntax>
>>>> +      <syntax><code>decap(packet_type(ns=<var>name_space</var>,
>>>> +      type=<var>ethertype</var>))</code></syntax> for 
>>>> decapsulating
>>>> MPLS
>>>> +      packets.
>>>>
>>>>        <p>
>>>>          Removes an outermost encapsulation from the packet:
>>>> @@ -1164,6 +1175,12 @@ for <var>i</var> in 
>>>> [1,<var>n_members</var>]:
>>>>            packet type errors.
>>>>          </li>
>>>>
>>>> +        <li>
>>>> +          Otherwise, if the packet is a MPLS packet, removes the 
>>>> MPLS
>>>> header
>>>> +          and classifies the inner packet as mentioned in the 
>>>> packet
>>>> type
>>>> +          argument of the decap.
>>>> +        </li>
>>>> +
>>>>          <li>
>>>>            Otherwise, raises an unsupported packet type error.
>>>>          </li>
>>>> diff --git a/lib/packets.c b/lib/packets.c
>>>> index 4a7643c5d..5e3c3900f 100644
>>>> --- a/lib/packets.c
>>>> +++ b/lib/packets.c
>>>> @@ -418,6 +418,38 @@ push_mpls(struct dp_packet *packet, ovs_be16
>>>> ethtype, ovs_be32 lse)
>>>>      pkt_metadata_init_conn(&packet->md);
>>>>  }
>>>>
>>>> +void
>>>> +add_mpls(struct dp_packet *packet, ovs_be16 ethtype, ovs_be32 lse, 
>>>> bool
>>>> l3)
>>>> +{
>>>> +    char * header;
>>>> +
>>>> +    if (!eth_type_mpls(ethtype)) {
>>>> +        return;
>>>> +    }
>>>> +
>>>> +    if (!l3) {
>>>> +        header =  dp_packet_push_uninit(packet, MPLS_HLEN);
>>>> +        memcpy(header, &lse, sizeof lse);
>>>> +        packet->l2_5_ofs = 0;
>>>> +        packet->packet_type = htonl(PT_MPLS);
>>>> +    } else {
>>>> +        size_t len;
>>>> +
>>>> +        if (!is_mpls(packet)) {
>>>> +            /* Set MPLS label stack offset. */
>>>> +            packet->l2_5_ofs = packet->l3_ofs;
>>>> +        }
>>>> +        set_ethertype(packet, ethtype);
>>>> +
>>>> +        /* Push new MPLS shim header onto packet. */
>>>> +        len = packet->l2_5_ofs;
>>>> +        header = dp_packet_resize_l2_5(packet, MPLS_HLEN);
>>>> +        memmove(header, header + MPLS_HLEN, len);
>>>> +        memcpy(header + len, &lse, sizeof lse);
>>>> +    }
>>>> +    pkt_metadata_init_conn(&packet->md);
>>>> +}
>>>> +
>>>>  /* If 'packet' is an MPLS packet, removes its outermost MPLS label
>>>> stack entry.
>>>>   * If the label that was removed was the only MPLS label, changes
>>>> 'packet''s
>>>>   * Ethertype to 'ethtype' (which ordinarily should not be an MPLS
>>>> @@ -429,6 +461,10 @@ pop_mpls(struct dp_packet *packet, ovs_be16
>>>> ethtype)
>>>>          struct mpls_hdr *mh = dp_packet_l2_5(packet);
>>>>          size_t len = packet->l2_5_ofs;
>>>>
>>>> +        if (ethtype == htons(ETH_TYPE_TEB)) {
>>>> +             packet->packet_type = htonl(PT_ETH);
>>>> +        }
>>>> +
>>>>          set_ethertype(packet, ethtype);
>>>>          if (get_16aligned_be32(&mh->mpls_lse) & 
>>>> htonl(MPLS_BOS_MASK)) {
>>>>              dp_packet_set_l2_5(packet, NULL);
>>>> diff --git a/lib/packets.h b/lib/packets.h
>>>> index 481bc22fa..3f5862e08 100644
>>>> --- a/lib/packets.h
>>>> +++ b/lib/packets.h
>>>> @@ -350,6 +350,8 @@ void set_mpls_lse_label(ovs_be32 *lse, ovs_be32
>>>> label);
>>>>  void set_mpls_lse_bos(ovs_be32 *lse, uint8_t bos);
>>>>  ovs_be32 set_mpls_lse_values(uint8_t ttl, uint8_t tc, uint8_t bos,
>>>>                               ovs_be32 label);
>>>> +void add_mpls(struct dp_packet *packet, ovs_be16 ethtype, ovs_be32 
>>>> lse,
>>>> +              bool l3_flag);
>>>>
>>>>  /* Example:
>>>>   *
>>>> diff --git a/ofproto/ofproto-dpif-ipfix.c 
>>>> b/ofproto/ofproto-dpif-ipfix.c
>>>> index 796eb6f88..9280e008e 100644
>>>> --- a/ofproto/ofproto-dpif-ipfix.c
>>>> +++ b/ofproto/ofproto-dpif-ipfix.c
>>>> @@ -3018,6 +3018,7 @@ dpif_ipfix_read_actions(const struct flow 
>>>> *flow,
>>>>          case OVS_ACTION_ATTR_CHECK_PKT_LEN:
>>>>          case OVS_ACTION_ATTR_UNSPEC:
>>>>          case OVS_ACTION_ATTR_DROP:
>>>> +        case OVS_ACTION_ATTR_ADD_MPLS:
>>>>          case __OVS_ACTION_ATTR_MAX:
>>>>          default:
>>>>              break;
>>>> diff --git a/ofproto/ofproto-dpif-sflow.c 
>>>> b/ofproto/ofproto-dpif-sflow.c
>>>> index fdcb9eabb..ca46a9bc4 100644
>>>> --- a/ofproto/ofproto-dpif-sflow.c
>>>> +++ b/ofproto/ofproto-dpif-sflow.c
>>>> @@ -1226,6 +1226,7 @@ dpif_sflow_read_actions(const struct flow 
>>>> *flow,
>>>>          case OVS_ACTION_ATTR_UNSPEC:
>>>>          case OVS_ACTION_ATTR_CHECK_PKT_LEN:
>>>>          case OVS_ACTION_ATTR_DROP:
>>>> +        case OVS_ACTION_ATTR_ADD_MPLS:
>>>>          case __OVS_ACTION_ATTR_MAX:
>>>>          default:
>>>>              break;
>>>> diff --git a/ofproto/ofproto-dpif-xlate.c 
>>>> b/ofproto/ofproto-dpif-xlate.c
>>>> index 7108c8a30..a97534233 100644
>>>> --- a/ofproto/ofproto-dpif-xlate.c
>>>> +++ b/ofproto/ofproto-dpif-xlate.c
>>>> @@ -6381,6 +6381,45 @@ rewrite_flow_encap_ethernet(struct xlate_ctx
>>>> *ctx,
>>>>          ctx->error = XLATE_UNSUPPORTED_PACKET_TYPE;
>>>>      }
>>>>  }
>>>> +static void
>>>> +rewrite_flow_encap_mpls(struct xlate_ctx *ctx,
>>>> +                        const struct ofpact_encap *encap,
>>>> +                        struct flow *flow,
>>>> +                        struct flow_wildcards *wc)
>>>> +{
>>>> +    int n;
>>>> +    uint32_t i;
>>>> +    uint16_t ether_type;
>>>> +    const char *ptr = (char *) encap->props;
>>>> +
>>>> +     for (i = 0; i < encap->n_props; i++) {
>>>> +        struct ofpact_ed_prop *prop_ptr =
>>>> +            ALIGNED_CAST(struct ofpact_ed_prop *, ptr);
>>>> +        if (prop_ptr->prop_class == OFPPPC_MPLS) {
>>>> +            switch (prop_ptr->type) {
>>>> +                case OFPPPT_PROP_MPLS_ETHERTYPE: {
>>>> +                     struct ofpact_ed_prop_mpls_ethertype
>>>> *prop_ether_type =
>>>> +                        ALIGNED_CAST(struct
>>>> ofpact_ed_prop_mpls_ethertype *,
>>>> +                                     prop_ptr);
>>>> +                    ether_type = prop_ether_type->ether_type;
>>>> +                    break;
>>>> +                 }
>>>> +            }
>>>> +        }
>>>> +     }
>>>> +
>>>> +    wc->masks.packet_type = OVS_BE32_MAX;
>>>> +    if (flow->packet_type != htonl(PT_MPLS)) {
>>>> +        memset(&ctx->wc->masks.mpls_lse, 0x0,
>>>> +               sizeof *wc->masks.mpls_lse * FLOW_MAX_MPLS_LABELS);
>>>> +        memset(&flow->mpls_lse, 0x0, sizeof *flow->mpls_lse *
>>>> +               FLOW_MAX_MPLS_LABELS);
>>>> +    }
>>>> +    flow->packet_type = htonl(PT_MPLS);
>>>> +    n = flow_count_mpls_labels(flow, ctx->wc);
>>>> +    flow_push_mpls(flow, n, htons(ether_type), ctx->wc, true);
>>>> +}
>>>> +
>>>>
>>>>  /* For an MD2 NSH header returns a pointer to an ofpbuf with the
>>>> encoded
>>>>   * MD2 TLVs provided as encap properties to the encap operation. 
>>>> This
>>>> @@ -6513,6 +6552,12 @@ xlate_generic_encap_action(struct xlate_ctx 
>>>> *ctx,
>>>>          case PT_NSH:
>>>>              encap_data = rewrite_flow_push_nsh(ctx, encap, flow, 
>>>> wc);
>>>>              break;
>>>> +        case PT_MPLS:
>>>> +            rewrite_flow_encap_mpls(ctx, encap,  flow, wc);
>>>> +            if (!ctx->xbridge->support.add_mpls) {
>>>> +                ctx->xout->slow |= SLOW_ACTION;
>>>> +            }
>>>> +            break;
>>>>          default:
>>>>              /* New packet type was checked during decoding. */
>>>>              OVS_NOT_REACHED();
>>>> @@ -6582,6 +6627,21 @@ xlate_generic_decap_action(struct xlate_ctx 
>>>> *ctx,
>>>>              ctx->pending_decap = true;
>>>>              /* Trigger recirculation. */
>>>>              return true;
>>>> +        case PT_MPLS: {
>>>> +             int n;
>>>> +             ovs_be16 ethertype;
>>>> +
>>>> +             flow->packet_type = decap->new_pkt_type;
>>>> +             ethertype = pt_ns_type_be(flow->packet_type);
>>>> +
>>>> +             n = flow_count_mpls_labels(flow, ctx->wc);
>>>> +             flow_pop_mpls(flow, n, ethertype, ctx->wc);
>>>> +             if (!ctx->xbridge->support.add_mpls) {
>>>> +                ctx->xout->slow |= SLOW_ACTION;
>>>> +             }
>>>> +             ctx->pending_decap = true;
>>>> +             return true;
>>>> +        }
>>>>          default:
>>>>              /* Error handling: drop packet. */
>>>>              xlate_report_debug(
>>>> diff --git a/ofproto/ofproto-dpif.c b/ofproto/ofproto-dpif.c
>>>> index fd0b2fdea..d9a2922e7 100644
>>>> --- a/ofproto/ofproto-dpif.c
>>>> +++ b/ofproto/ofproto-dpif.c
>>>> @@ -1520,6 +1520,44 @@ check_nd_extensions(struct dpif_backer 
>>>> *backer)
>>>>
>>>>      return !error;
>>>>  }
>>>> +/* Tests whether 'backer''s datapath supports the
>>>> + * OVS_ACTION_ATTR_ADD_MPLS action. */
>>>> +static bool
>>>> +check_add_mpls(struct dpif_backer *backer)
>>>> +{
>>>> +    struct odputil_keybuf keybuf;
>>>> +    struct ofpbuf actions;
>>>> +    struct ofpbuf key;
>>>> +    struct flow flow;
>>>> +    bool supported;
>>>> +
>>>> +    struct odp_flow_key_parms odp_parms = {
>>>> +        .flow = &flow,
>>>> +        .probe = true,
>>>> +    };
>>>> +
>>>> +    memset(&flow, 0, sizeof flow);
>>>> +    ofpbuf_use_stack(&key, &keybuf, sizeof keybuf);
>>>> +    odp_flow_key_from_flow(&odp_parms, &key);
>>>> +    ofpbuf_init(&actions, 64);
>>>> +
>>>> +    struct ovs_action_add_mpls *mpls;
>>>> +
>>>> +    mpls = nl_msg_put_unspec_zero(&actions,
>>>> +                                  OVS_ACTION_ATTR_ADD_MPLS,
>>>> +                                  sizeof *mpls);
>>>> +    mpls->mpls_ethertype = htons(ETH_TYPE_MPLS);
>>>> +
>>>> +    supported = dpif_probe_feature(backer->dpif, "add_mpls", &key,
>>>> +                                   &actions, NULL);
>>>> +    ofpbuf_uninit(&actions);
>>>> +    VLOG_INFO("%s: Datapath %s add_mpls action",
>>>> +              dpif_name(backer->dpif), supported ? "supports"
>>>> +                                                 : "does not 
>>>> support");
>>>> +    return supported;
>>>> +
>>>> +}
>>>> +
>>>>
>>>>  #define CHECK_FEATURE__(NAME, SUPPORT, FIELD, VALUE, ETHTYPE)
>>>> \
>>>>  static bool
>>>> \
>>>> @@ -1590,6 +1628,7 @@ check_support(struct dpif_backer *backer)
>>>>          dpif_supports_explicit_drop_action(backer->dpif);
>>>>      backer->rt_support.lb_output_action=
>>>>          dpif_supports_lb_output_action(backer->dpif);
>>>> +    backer->rt_support.add_mpls = check_add_mpls(backer);
>>>>
>>>>      /* Flow fields. */
>>>>      backer->rt_support.odp.ct_state = check_ct_state(backer);
>>>> diff --git a/ofproto/ofproto-dpif.h b/ofproto/ofproto-dpif.h
>>>> index b41c3d82a..c04bfff8d 100644
>>>> --- a/ofproto/ofproto-dpif.h
>>>> +++ b/ofproto/ofproto-dpif.h
>>>> @@ -204,7 +204,10 @@ struct group_dpif *group_dpif_lookup(struct
>>>> ofproto_dpif *,
>>>>      DPIF_SUPPORT_FIELD(bool, explicit_drop_action, "Explicit Drop
>>>> action")  \
>>>>                                                                              \
>>>>      /* True if the datapath supports balance_tcp optimization */
>>>> \
>>>> -    DPIF_SUPPORT_FIELD(bool, lb_output_action, "Optimized Balance 
>>>> TCP
>>>> mode")
>>>> +    DPIF_SUPPORT_FIELD(bool, lb_output_action, "Optimized Balance 
>>>> TCP
>>>> mode")\
>>>> +
>>>> \
>>>> +    /* True if the datapath supports layer 2 MPLS tunnelling */
>>>> \
>>>> +    DPIF_SUPPORT_FIELD(bool, add_mpls, "l2 MPLS tunnelling")
>>>>
>>>>
>>>>  /* Stores the various features which the corresponding backer 
>>>> supports.
>>>> */
>>>> diff --git a/tests/system-traffic.at b/tests/system-traffic.at
>>>> index fb5b9a36d..b865b5210 100644
>>>> --- a/tests/system-traffic.at
>>>> +++ b/tests/system-traffic.at
>>>> @@ -1027,9 +1027,45 @@ NS_CHECK_EXEC([at_ns0], [ping -q -c 3 -i 0.3 
>>>> -w 2
>>>> 10.1.1.2 | FORMAT_PING], [0],
>>>>  3 packets transmitted, 3 received, 0% packet loss, time 0ms
>>>>  ])
>>>>
>>>> +OVS_TRAFFIC_VSWITCHD_STOP
>>>> +AT_CLEANUP
>>>> +
>>>> +
>>>> +AT_SETUP([datapath - ptap mpls actions])
>>>> +OVS_TRAFFIC_VSWITCHD_START([_ADD_BR([br1])])
>>>> +
>>>> +ADD_NAMESPACES(at_ns0, at_ns1)
>>>> +
>>>> +ADD_VETH(p0, at_ns0, br0, "10.1.1.1/24")
>>>> +ADD_VETH(p1, at_ns1, br1, "10.1.1.2/24")
>>>> +
>>>> +AT_CHECK([ip link add patch0 type veth peer name patch1])
>>>> +on_exit 'ip link del patch0'
>>>> +
>>>> +AT_CHECK([ip link set dev patch0 up])
>>>> +AT_CHECK([ip link set dev patch1 up])
>>>> +AT_CHECK([ovs-vsctl add-port br0 patch0 -- set interface patch0
>>>> ofport_request=100])
>>>> +AT_CHECK([ovs-vsctl add-port br1 patch1 -- set interface patch1
>>>> ofport_request=100])
>>>> +
>>>> +AT_DATA([flows.txt], [dnl
>>>> +table=0,priority=100,dl_type=0x0800 
>>>> actions=encap(mpls(ether_type=0x8847)),set_mpls_label:2,encap(ethernet),output:100
>>>> +table=0,priority=100,dl_type=0x8847,mpls_label=2
>>>> actions=decap(),decap(packet_type(ns=0,type=0)),resubmit(,3)
>>>> +table=0,priority=10 actions=resubmit(,3)
>>>> +table=3,priority=10 actions=normal
>>>> +])
>>>> +
>>>> +AT_CHECK([ovs-ofctl  -Oopenflow13 add-flows br0 flows.txt])
>>>> +AT_CHECK([ovs-ofctl  -Oopenflow13 add-flows br1 flows.txt])
>>>> +
>>>> +NS_CHECK_EXEC([at_ns0], [ping -q -c 3 -i 0.3 -w 2 10.1.1.2 |
>>>> FORMAT_PING], [0], [dnl
>>>> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
>>>> +])
>>>> +
>>>> +
>>>>  NS_CHECK_EXEC([at_ns1], [ping -q -c 3 -i 0.3 -w 2 10.1.1.1 |
>>>> FORMAT_PING], [0], [dnl
>>>>  3 packets transmitted, 3 received, 0% packet loss, time 0ms
>>>>  ])
>>>> +
>>>>  OVS_TRAFFIC_VSWITCHD_STOP
>>>>  AT_CLEANUP
>>>>
>>>> -- 
>>>> 2.18.4
>>>

On Thu, Apr 01, 2021 at 08:59:27AM +0200, Eelco Chaudron wrote:
> 
> 
> On 1 Apr 2021, at 6:10, Martin Varghese wrote:
> 
> > On Wed, Mar 31, 2021 at 03:59:40PM +0200, Eelco Chaudron wrote:
> > > 
> > > 
> > > On 26 Mar 2021, at 7:21, Martin Varghese wrote:
> > > 
> > > > From: Martin Varghese <martin.varghese@nokia.com>
> > > > 
> > > > The encap & decap actions are extended to support MPLS packet type.
> > > > Encap & decap actions adds and removes MPLS header at start of the
> > > > packet.
> > > 
> > > Hi Martin,
> > > 
> > > I’m trying to do some real-life testing, and I’m running into
> > > issues. This
> > > might be me setting it up wrongly but just wanting to confirm…
> > > 
> > > I’m sending an MPLS packet that contains an ARP packet into a
> > > physical port.
> > > This is the packet:
> > > 
> > > Frame 4: 64 bytes on wire (512 bits), 64 bytes captured (512 bits)
> > >     Encapsulation type: Ethernet (1)
> > >     [Protocols in frame: eth:ethertype:mpls:data]
> > > Ethernet II, Src: 00:00:00_00:00:01 (00:00:00:00:00:01), Dst:
> > > 00:00:00_00:00:02 (00:00:00:00:00:02)
> > >     Destination: 00:00:00_00:00:02 (00:00:00:00:00:02)
> > >         Address: 00:00:00_00:00:02 (00:00:00:00:00:02)
> > >         .... ..0. .... .... .... .... = LG bit: Globally unique
> > > address
> > > (factory default)
> > >         .... ...0 .... .... .... .... = IG bit: Individual address
> > > (unicast)
> > >     Source: 00:00:00_00:00:01 (00:00:00:00:00:01)
> > >         Address: 00:00:00_00:00:01 (00:00:00:00:00:01)
> > >         .... ..0. .... .... .... .... = LG bit: Globally unique
> > > address
> > > (factory default)
> > >         .... ...0 .... .... .... .... = IG bit: Individual address
> > > (unicast)
> > >     Type: MPLS label switched packet (0x8847)
> > > MultiProtocol Label Switching Header, Label: 100, Exp: 0, S: 1, TTL:
> > > 64
> > >     0000 0000 0000 0110 0100 .... .... .... = MPLS Label: 100
> > >     .... .... .... .... .... 000. .... .... = MPLS Experimental
> > > Bits: 0
> > >     .... .... .... .... .... ...1 .... .... = MPLS Bottom Of Label
> > > Stack: 1
> > >     .... .... .... .... .... .... 0100 0000 = MPLS TTL: 64
> > > Data (46 bytes)
> > > 
> > > 0000  ff ff ff ff ff ff 52 54 00 88 51 38 08 06 00 01
> > > ......RT..Q8....
> > > 0010  08 00 06 04 00 01 52 54 00 88 51 38 01 01 01 65
> > > ......RT..Q8...e
> > > 0020  00 00 00 00 00 00 01 01 01 64 27 98 a0 47
> > > .........d'..G
> > >     Data:
> > > ffffffffffff525400885138080600010800060400015254008851380101016500000000?
> > > 
> > > 
> > > I’m trying to use the following rules:
> > > 
> > >   ovs-ofctl del-flows ovs_pvp_br0
> > >   ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0
> > > "priority=100,dl_type=0x8847,mpls_label=100
> > > actions=decap(),decap(packet_type(ns=0,type=0x806)),resubmit(,3)"
> > >   ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0 "table=3,priority=10
> > > actions=normal"
> > > 
> > > With these, I expect the packet to be sent to vnet0, but it’s not.
> > > Actually,
> > > the datapath rule looks odd, while the userspace rules seem to match:
> > > 
> > >   $ ovs-dpctl dump-flows
> > >   recirc_id(0),in_port(1),eth(),eth_type(0x8847),mpls(label=100/0xfffff,tc=0/0,ttl=0/0x0,bos=1/1),
> > > packets:13, bytes:1118, used:0.322s,
> > > actions:pop_eth,pop_mpls(eth_type=0x806),recirc(0x19a)
> > >   recirc_id(0x19a),in_port(1),eth_type(0x0806), packets:13,
> > > bytes:884,
> > > used:0.322s, actions:drop
> > > 
> > >   $ ovs-ofctl dump-flows ovs_pvp_br0 -O OpenFlow13
> > >   cookie=0x0, duration=85.007s, table=0, n_packets=51, n_bytes=4386,
> > > priority=100,mpls,mpls_label=100
> > > actions=decap(),decap(packet_type(ns=0,type=0x806)),resubmit(,3)
> > >   cookie=0x0, duration=84.990s, table=3, n_packets=51, n_bytes=3468,
> > > priority=10 actions=NORMAL
> > > 
> > The inner packet is ethernet. So the packet type should be (ns=0,type=0)
> > ?
> 
> Forgot to add that I already tried that to start with, based on the example,
> but as that did not work I tried 0x806.
> 
> PS: I have this as a remark in my review notes, i.e., to explain the ns and
> type usage here.
> 
> 
> This resulted in packets being counted at the open flow level, but it
> results in NO data path rules. Do get an error though:
> 
> 2021-04-01T06:53:36.056Z|00141|dpif(handler37)|WARN|system@ovs-system:
> failed to put[create] (Invalid argument)
> ufid:3d2d6f6d-5a66-4ace-8b09-7cdcfa5efc8e recirc_id(0),dp_hash(0/0),skb_priority(0/0),in_port(1),skb_mark(0/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),eth(src=00:00:00:00:00:01/00:00:00:00:00:00,dst=00:00:00:00:00:02/00:00:00:00:00:00),eth_type(0x8847),mpls(label=100/0xfffff,tc=0/0,ttl=64/0x0,bos=1/1),
> actions:pop_eth,pop_mpls(eth_type=0x6558),set(eth()),recirc(0x4c)

This set(eth) before the recirc is the problem i guesss. I need to check
> 2021-04-01T06:53:36.056Z|00142|dpif(handler37)|WARN|system@ovs-system:
> execute pop_eth,pop_mpls(eth_type=0x6558),set(eth()),recirc(0x4c) failed
> (Invalid argument) on packet mpls,vlan_tci=0x0000,dl_src=00:00:00:00:00:01,dl_dst=00:00:00:00:00:02,mpls_label=100,mpls_tc=0,mpls_ttl=64,mpls_bos=1
>  with metadata skb_priority(0),skb_mark(0),in_port(1) mtu 0
> 
> Are there missing parts in my kernel that do not get properly detected by
> the feature detection?
> 
> $ ovs-appctl dpif/show-dp-features ovs_pvp_br0
> Masked set action: Yes
> Tunnel push pop: No
> Ufid: Yes
> Truncate action: Yes
> Clone action: Yes
> Sample nesting: 10
> Conntrack eventmask: Yes
> Conntrack clear: Yes
> Max dp_hash algorithm: 0
> Check pkt length action: Yes
> Conntrack timeout policy: Yes
> Explicit Drop action: No
> Optimized Balance TCP mode: No
> l2 MPLS tunnelling: Yes
> Max VLAN headers: 2
> Max MPLS depth: 3
> Recirc: Yes
> CT state: Yes
> CT zone: Yes
> CT mark: Yes
> CT label: Yes
> CT state NAT: Yes
> CT orig tuple: Yes
> CT orig tuple for IPv6: Yes
> IPv6 ND Extension: No
> 
You are good

I am not sure what is going wrong. Your test case looks same as the unit
test i added.

I tried myself again and this is i get

ovs-ofctl -O OpenFlow13 add-flow br_mpls2 "in_port=$egress_port,dl_type=0x8847
+actions=decap(),decap(packet_type(ns=0,type=0),goto_table:1"
ovs-ofctl -O OpenFlow13 add-flow br_mpls2
+"table=1,in_port=$egress_port,dl_type=0x0800,nw_dst=1.1.1.2
+actions=set_field:00:00:00:00:00:02->dl_dst,set_field:00:00:00:00:00:01->dl_sr
+c output:$ingress_port"

recirc_id(0x3),in_port(6),eth(src=36:b1:ee:7c:01:03,dst=36:b1:ee:7c:01:02),eth_
+type(0x0800),ipv4(dst=1.1.1.2,frag=no), packets:3, bytes:294, used:0.837s,
+actions:set(eth(src=00:00:00:00:00:01,dst=00:00:00:00:00:02)),4
recirc_id(0),in_port(6),eth(),eth_type(0x8847),mpls(label=0/0x0,tc=0/0,ttl=0/0x
+0,bos=1/1), packets:3, bytes:348, used:0.837s,
+actions:pop_eth,pop_mpls(eth_type=0x6558),recirc(0x3)

The packet to the ovs is
ETH|MPLS|ETH|IP ?
How it is differnt from you test case?

Thanks for your time.



> > > 
> > > If I use the old way, doing pop_mpls, it works fine:
> > > 
> > > 
> > > ovs-ofctl del-flows ovs_pvp_br0
> > > ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0
> > > "priority=100,dl_type=0x8847,mpls_label=100
> > > actions=pop_mpls:0x0806,resubmit(,3)"
> > > ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0 "table=3,priority=10
> > > actions=normal"
> > > 
> > > 
> > > I also noticed (despite the test example) to make encap work, I had
> > > to set
> > > the ethernet MAC addresses, or else the packets were not getting out.
> > > So something like:
> > > 
> > >   ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0 "priority=100,in_port=vnet0,actions=encap(mpls(ether_type=0x8847)),set_mpls_label:100,encap(ethernet),,set_field:00:00:00:00:00:02->dl_dst,set_field:00:00:00:00:00:01->dl_src,output:enp5s0f0
> > > 

> > 
> > The packets are not going out because you are sending the packet on a
> > real nic and not on a virtual inerface (veth pair) ?
> 
> So for a real NIC we need to set the MAC addresses, maybe some where in the
> documentation we should add an example on how to use this feature?
> 
> > > Maybe the test case can be made more realistic? Once I understand the
> > > failure, I can continue with the review.
> > > 
> > > 
> > > Cheers,
> > > 
> > > Eelco
> > > 
> > > 
> > > 
> > > > Signed-off-by: Martin Varghese <martin.varghese@nokia.com>
> > > > ---
> > > >  NEWS                                          |  2 +-
> > > >  .../linux/compat/include/linux/openvswitch.h  | 35 ++++++-
> > > >  include/openvswitch/ofp-ed-props.h            | 18 ++++
> > > >  lib/dpif-netdev.c                             |  1 +
> > > >  lib/dpif.c                                    |  1 +
> > > >  lib/odp-execute.c                             | 12 +++
> > > >  lib/odp-util.c                                | 58 +++++++++---
> > > >  lib/ofp-actions.c                             |  5 +
> > > >  lib/ofp-ed-props.c                            | 91
> > > > +++++++++++++++++++
> > > >  lib/ovs-actions.xml                           | 31 +++++--
> > > >  lib/packets.c                                 | 36 ++++++++
> > > >  lib/packets.h                                 |  2 +
> > > >  ofproto/ofproto-dpif-ipfix.c                  |  1 +
> > > >  ofproto/ofproto-dpif-sflow.c                  |  1 +
> > > >  ofproto/ofproto-dpif-xlate.c                  | 60 ++++++++++++
> > > >  ofproto/ofproto-dpif.c                        | 39 ++++++++
> > > >  ofproto/ofproto-dpif.h                        |  5 +-
> > > >  tests/system-traffic.at                       | 36 ++++++++
> > > >  18 files changed, 410 insertions(+), 24 deletions(-)
> > > > 
> > > > diff --git a/NEWS b/NEWS
> > > > index 95cf922aa..4bf4e9e7b 100644
> > > > --- a/NEWS
> > > > +++ b/NEWS
> > > > @@ -120,7 +120,7 @@ v2.14.0 - 17 Aug 2020
> > > >     - GTP-U Tunnel Protocol
> > > >       * Add two new fields: tun_gtpu_flags, tun_gtpu_msgtype.
> > > >       * Only support for userspace datapath.
> > > > -
> > > > +   - Encap & Decap action support for MPLS packet type.
> > > > 
> > > >  v2.13.0 - 14 Feb 2020
> > > >  ---------------------
> > > > diff --git a/datapath/linux/compat/include/linux/openvswitch.h
> > > > b/datapath/linux/compat/include/linux/openvswitch.h
> > > > index 875de2025..8feea7dd4 100644
> > > > --- a/datapath/linux/compat/include/linux/openvswitch.h
> > > > +++ b/datapath/linux/compat/include/linux/openvswitch.h
> > > > @@ -810,8 +810,32 @@ struct ovs_action_push_tnl {
> > > >  };
> > > >  #endif
> > > > 
> > > > -/**
> > > > - * enum ovs_ct_attr - Attributes for %OVS_ACTION_ATTR_CT action.
> > > > +/* struct ovs_action_add_mpls - %OVS_ACTION_ATTR_ADD_MPLS action
> > > > + * argument.
> > > > + * @mpls_lse: MPLS label stack entry to push.
> > > > + * @mpls_ethertype: Ethertype to set in the encapsulating ethernet
> > > > frame.
> > > > + * @tun_flags: MPLS tunnel attributes.
> > > > + *
> > > > + * The only values @mpls_ethertype should ever be given are
> > > > %ETH_P_MPLS_UC and
> > > > + * %ETH_P_MPLS_MC, indicating MPLS unicast or multicast. Other are
> > > > rejected.
> > > > + */
> > > > +struct ovs_action_add_mpls {
> > > > +	__be32 mpls_lse;
> > > > +	__be16 mpls_ethertype; /* Either %ETH_P_MPLS_UC or
> > > > %ETH_P_MPLS_MC */
> > > > +	__u16 tun_flags;
> > > > +};
> > > > +
> > > > +#define OVS_MPLS_L3_TUNNEL_FLAG_MASK  (1 << 0) /* Flag to
> > > > specify the
> > > > place of
> > > > +						* insertion of MPLS header.
> > > > +						* When false, the MPLS header
> > > > +						* will be inserted at the start
> > > > +						* of the packet.
> > > > +						* When true, the MPLS header
> > > > +						* will be inserted at the start
> > > > +						* of the l3 header.
> > > > +						*/
> > > > +
> > > > +/* enum ovs_ct_attr - Attributes for %OVS_ACTION_ATTR_CT action.
> > > >   * @OVS_CT_ATTR_COMMIT: If present, commits the connection to the
> > > > conntrack
> > > >   * table. This allows future packets for the same connection to be
> > > > identified
> > > >   * as 'established' or 'related'. The flow key for the current
> > > > packet
> > > > will
> > > > @@ -1001,7 +1025,11 @@ struct check_pkt_len_arg {
> > > >   * @OVS_ACTION_ATTR_CHECK_PKT_LEN: Check the packet length and
> > > > execute
> > > > a set
> > > >   * of actions if greater than the specified packet length, else
> > > > execute
> > > >   * another set of actions.
> > > > - * @OVS_ACTION_ATTR_DROP: Explicit drop action.
> > > > + * @OVS_ACTION_ATTR_ADD_MPLS: Push a new MPLS label stack entry
> > > > at the
> > > > + * start of the packet or at the start of the l3 header
> > > > depending on
> > > > the value
> > > > + * of l3 tunnel flag in the tun_flags field of
> > > > OVS_ACTION_ATTR_ADD_MPLS
> > > > + * argument.
> > > > +  * @OVS_ACTION_ATTR_DROP: Explicit drop action.
> > > >   */
> > > > 
> > > >  enum ovs_action_attr {
> > > > @@ -1030,6 +1058,7 @@ enum ovs_action_attr {
> > > >  	OVS_ACTION_ATTR_METER,        /* u32 meter number. */
> > > >  	OVS_ACTION_ATTR_CLONE,        /* Nested OVS_CLONE_ATTR_*.  */
> > > >  	OVS_ACTION_ATTR_CHECK_PKT_LEN, /* Nested
> > > > OVS_CHECK_PKT_LEN_ATTR_*. */
> > > > +	OVS_ACTION_ATTR_ADD_MPLS,     /* struct ovs_action_add_mpls. */
> > > > 
> > > >  #ifndef __KERNEL__
> > > >  	OVS_ACTION_ATTR_TUNNEL_PUSH,   /* struct ovs_action_push_tnl*/
> > > > diff --git a/include/openvswitch/ofp-ed-props.h
> > > > b/include/openvswitch/ofp-ed-props.h
> > > > index 306c6fe73..c85f3c283 100644
> > > > --- a/include/openvswitch/ofp-ed-props.h
> > > > +++ b/include/openvswitch/ofp-ed-props.h
> > > > @@ -46,6 +46,11 @@ enum ofp_ed_nsh_prop_type {
> > > >      OFPPPT_PROP_NSH_TLV = 2,     /* property TLV in NSH */
> > > >  };
> > > > 
> > > > +enum ofp_ed_mpls_prop_type {
> > > > +    OFPPPT_PROP_MPLS_NONE = 0,    /* unused */
> > > > +    OFPPPT_PROP_MPLS_ETHERTYPE = 1,  /* MPLS Ethertype */
> > > > +};
> > > > +
> > > >  /*
> > > >   * External representation of encap/decap properties.
> > > >   * These must be padded to a multiple of 8 bytes.
> > > > @@ -72,6 +77,13 @@ struct ofp_ed_prop_nsh_tlv {
> > > >      uint8_t data[0];
> > > >  };
> > > > 
> > > > +struct ofp_ed_prop_mpls_ethertype {
> > > > +    struct ofp_ed_prop_header header;
> > > > +    uint16_t ether_type;         /* MPLS ethertype .*/
> > > > +    uint8_t pad[2];          /* Padding to 8 bytes. */
> > > > +};
> > > > +
> > > > +
> > > >  /*
> > > >   * Internal representation of encap/decap properties
> > > >   */
> > > > @@ -96,6 +108,12 @@ struct ofpact_ed_prop_nsh_tlv {
> > > >      /* tlv_len octets of metadata value, padded to a multiple of 8
> > > > bytes. */
> > > >      uint8_t data[0];
> > > >  };
> > > > +
> > > > +struct ofpact_ed_prop_mpls_ethertype {
> > > > +    struct ofpact_ed_prop header;
> > > > +    uint16_t ether_type;         /* MPLS ethertype .*/
> > > > +    uint8_t pad[2];          /* Padding to 8 bytes. */
> > > > +};
> > > >  enum ofperr decode_ed_prop(const struct ofp_ed_prop_header
> > > > **ofp_prop,
> > > >                             struct ofpbuf *out, size_t *remaining);
> > > >  enum ofperr encode_ed_prop(const struct ofpact_ed_prop **prop,
> > > > diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
> > > > index 94cc9b80c..bbdea5603 100644
> > > > --- a/lib/dpif-netdev.c
> > > > +++ b/lib/dpif-netdev.c
> > > > @@ -8044,6 +8044,7 @@ dp_execute_cb(void *aux_, struct
> > > > dp_packet_batch
> > > > *packets_,
> > > >      case OVS_ACTION_ATTR_CT_CLEAR:
> > > >      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
> > > >      case OVS_ACTION_ATTR_DROP:
> > > > +    case OVS_ACTION_ATTR_ADD_MPLS:
> > > >      case __OVS_ACTION_ATTR_MAX:
> > > >          OVS_NOT_REACHED();
> > > >      }
> > > > diff --git a/lib/dpif.c b/lib/dpif.c
> > > > index 56d0b4a65..bbd1296e3 100644
> > > > --- a/lib/dpif.c
> > > > +++ b/lib/dpif.c
> > > > @@ -1273,6 +1273,7 @@ dpif_execute_helper_cb(void *aux_, struct
> > > > dp_packet_batch *packets_,
> > > >      case OVS_ACTION_ATTR_UNSPEC:
> > > >      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
> > > >      case OVS_ACTION_ATTR_DROP:
> > > > +    case OVS_ACTION_ATTR_ADD_MPLS:
> > > >      case __OVS_ACTION_ATTR_MAX:
> > > >          OVS_NOT_REACHED();
> > > >      }
> > > > diff --git a/lib/odp-execute.c b/lib/odp-execute.c
> > > > index 6eeda2a61..2f4cdd92c 100644
> > > > --- a/lib/odp-execute.c
> > > > +++ b/lib/odp-execute.c
> > > > @@ -819,6 +819,7 @@ requires_datapath_assistance(const struct
> > > > nlattr *a)
> > > >      case OVS_ACTION_ATTR_POP_NSH:
> > > >      case OVS_ACTION_ATTR_CT_CLEAR:
> > > >      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
> > > > +    case OVS_ACTION_ATTR_ADD_MPLS:
> > > >      case OVS_ACTION_ATTR_DROP:
> > > >          return false;
> > > > 
> > > > @@ -1061,6 +1062,17 @@ odp_execute_actions(void *dp, struct
> > > > dp_packet_batch *batch, bool steal,
> > > >              }
> > > >              break;
> > > > 
> > > > +        case OVS_ACTION_ATTR_ADD_MPLS: {
> > > > +            const struct ovs_action_add_mpls *mpls =
> > > > nl_attr_get(a);
> > > > +            bool l3_flag =  mpls->tun_flags &
> > > > OVS_MPLS_L3_TUNNEL_FLAG_MASK;
> > > > +
> > > > +            DP_PACKET_BATCH_FOR_EACH (i, packet, batch) {
> > > > +                add_mpls(packet, mpls->mpls_ethertype,
> > > > mpls->mpls_lse,
> > > > +                         l3_flag);
> > > > +            }
> > > > +            break;
> > > > +        }
> > > > +
> > > >          case OVS_ACTION_ATTR_DROP:{
> > > >              const enum xlate_error *drop_reason = nl_attr_get(a);
> > > > 
> > > > diff --git a/lib/odp-util.c b/lib/odp-util.c
> > > > index a8598d52a..f24e16d08 100644
> > > > --- a/lib/odp-util.c
> > > > +++ b/lib/odp-util.c
> > > > @@ -142,6 +142,8 @@ odp_action_len(uint16_t type)
> > > >      case OVS_ACTION_ATTR_PUSH_NSH: return ATTR_LEN_VARIABLE;
> > > >      case OVS_ACTION_ATTR_POP_NSH: return 0;
> > > >      case OVS_ACTION_ATTR_CHECK_PKT_LEN: return ATTR_LEN_VARIABLE;
> > > > +    case OVS_ACTION_ATTR_ADD_MPLS:
> > > > +         return sizeof(struct ovs_action_add_mpls);
> > > >      case OVS_ACTION_ATTR_DROP: return sizeof(uint32_t);
> > > > 
> > > >      case OVS_ACTION_ATTR_UNSPEC:
> > > > @@ -1254,6 +1256,14 @@ format_odp_action(struct ds *ds, const struct
> > > > nlattr *a,
> > > >      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
> > > >          format_odp_check_pkt_len_action(ds, a, portno_names);
> > > >          break;
> > > > +    case OVS_ACTION_ATTR_ADD_MPLS: {
> > > > +        const struct ovs_action_push_mpls *mpls = nl_attr_get(a);
> > > > +        ds_put_cstr(ds, "add_mpls(");
> > > > +        format_mpls_lse(ds, mpls->mpls_lse);
> > > > +        ds_put_format(ds, ",eth_type=0x%"PRIx16")",
> > > > +                      ntohs(mpls->mpls_ethertype));
> > > > +        break;
> > > > +    }
> > > >      case OVS_ACTION_ATTR_DROP:
> > > >          ds_put_cstr(ds, "drop");
> > > >          break;
> > > > @@ -7876,7 +7886,8 @@ commit_vlan_action(const struct flow*
> > > > flow, struct
> > > > flow *base,
> > > >  /* Wildcarding already done at action translation time. */
> > > >  static void
> > > >  commit_mpls_action(const struct flow *flow, struct flow *base,
> > > > -                   struct ofpbuf *odp_actions)
> > > > +                   struct ofpbuf *odp_actions, bool pending_encap,
> > > > +                   bool pending_decap)
> > > >  {
> > > >      int base_n = flow_count_mpls_labels(base, NULL);
> > > >      int flow_n = flow_count_mpls_labels(flow, NULL);
> > > > @@ -7913,7 +7924,11 @@ commit_mpls_action(const struct flow *flow,
> > > > struct flow *base,
> > > >              if ((!eth_type_mpls(flow->dl_type)) && base_n > 1) {
> > > >                  dl_type = htons(ETH_TYPE_MPLS);
> > > >              } else {
> > > > -                dl_type = flow->dl_type;
> > > > +                if ((flow->packet_type == PT_ETH) &&
> > > > pending_decap) {
> > > > +                    dl_type =  htons(ETH_TYPE_TEB);
> > > > +                } else {
> > > > +                    dl_type = flow->dl_type;
> > > > +                }
> > > >              }
> > > >              nl_msg_put_be16(odp_actions, OVS_ACTION_ATTR_POP_MPLS,
> > > > dl_type);
> > > >              ovs_assert(flow_pop_mpls(base, base_n, flow->dl_type,
> > > > NULL));
> > > > @@ -7924,18 +7939,29 @@ commit_mpls_action(const struct flow *flow,
> > > > struct flow *base,
> > > >      /* If, after the above popping and setting, there are more
> > > > LSEs in
> > > > flow
> > > >       * than base then some LSEs need to be pushed. */
> > > >      while (base_n < flow_n) {
> > > > -        struct ovs_action_push_mpls *mpls;
> > > > 
> > > > -        mpls = nl_msg_put_unspec_zero(odp_actions,
> > > > -                                      OVS_ACTION_ATTR_PUSH_MPLS,
> > > > -                                      sizeof *mpls);
> > > > -        mpls->mpls_ethertype = flow->dl_type;
> > > > -        mpls->mpls_lse = flow->mpls_lse[flow_n - base_n - 1];
> > > > +        if (pending_encap) {
> > > > +             struct ovs_action_add_mpls *mpls;
> > > > +
> > > > +             mpls = nl_msg_put_unspec_zero(odp_actions,
> > > > +
> > > > OVS_ACTION_ATTR_ADD_MPLS,
> > > > +                                           sizeof *mpls);
> > > > +             mpls->mpls_ethertype = flow->dl_type;
> > > > +             mpls->mpls_lse = flow->mpls_lse[flow_n - base_n - 1];
> > > > +        } else {
> > > > +             struct ovs_action_push_mpls *mpls;
> > > > +
> > > > +             mpls = nl_msg_put_unspec_zero(odp_actions,
> > > > +
> > > > OVS_ACTION_ATTR_PUSH_MPLS,
> > > > +                                           sizeof *mpls);
> > > > +             mpls->mpls_ethertype = flow->dl_type;
> > > > +             mpls->mpls_lse = flow->mpls_lse[flow_n - base_n - 1];
> > > > +        }
> > > >          /* Update base flow's MPLS stack, but do not clear L3.
> > > > We need
> > > > the L3
> > > >           * headers if the flow is restored later due to
> > > > returning from
> > > > a patch
> > > >           * port or group bucket. */
> > > > -        flow_push_mpls(base, base_n, mpls->mpls_ethertype, NULL,
> > > > false);
> > > > -        flow_set_mpls_lse(base, 0, mpls->mpls_lse);
> > > > +        flow_push_mpls(base, base_n, flow->dl_type, NULL, false);
> > > > +        flow_set_mpls_lse(base, 0, flow->mpls_lse[flow_n - base_n -
> > > > 1]);
> > > >          base_n++;
> > > >      }
> > > >  }
> > > > @@ -8586,6 +8612,10 @@ commit_encap_decap_action(const struct flow
> > > > *flow,
> > > >              memcpy(&base_flow->dl_dst, &flow->dl_dst,
> > > >                     sizeof(*flow) - offsetof(struct flow, dl_dst));
> > > >              break;
> > > > +        case PT_MPLS:
> > > > +            commit_mpls_action(flow, base_flow, odp_actions,
> > > > pending_encap,
> > > > +                               pending_decap);
> > > > +            break;
> > > >          default:
> > > >              /* Only the above protocols are supported for encap.
> > > >               * The check is done at action translation. */
> > > > @@ -8608,6 +8638,10 @@ commit_encap_decap_action(const struct flow
> > > > *flow,
> > > >                  /* pop_nsh. */
> > > >                  odp_put_pop_nsh_action(odp_actions);
> > > >                  break;
> > > > +            case PT_MPLS:
> > > > +                commit_mpls_action(flow, base_flow, odp_actions,
> > > > pending_encap,
> > > > +                                   pending_decap);
> > > > +                break;
> > > >              default:
> > > >                  /* Checks are done during translation. */
> > > >                  OVS_NOT_REACHED();
> > > > @@ -8653,7 +8687,7 @@ commit_odp_actions(const struct flow
> > > > *flow, struct
> > > > flow *base,
> > > >      /* Make packet a non-MPLS packet before committing L3/4
> > > > actions,
> > > >       * which would otherwise do nothing. */
> > > >      if (eth_type_mpls(base->dl_type) &&
> > > > !eth_type_mpls(flow->dl_type))
> > > > {
> > > > -        commit_mpls_action(flow, base, odp_actions);
> > > > +        commit_mpls_action(flow, base, odp_actions, false, false);
> > > >          mpls_done = true;
> > > >      }
> > > >      commit_set_nsh_action(flow, base, odp_actions, wc, use_masked);
> > > > @@ -8661,7 +8695,7 @@ commit_odp_actions(const struct flow
> > > > *flow, struct
> > > > flow *base,
> > > >      commit_set_port_action(flow, base, odp_actions, wc,
> > > > use_masked);
> > > >      slow2 = commit_set_icmp_action(flow, base, odp_actions, wc);
> > > >      if (!mpls_done) {
> > > > -        commit_mpls_action(flow, base, odp_actions);
> > > > +        commit_mpls_action(flow, base, odp_actions, false, false);
> > > >      }
> > > >      commit_vlan_action(flow, base, odp_actions, wc);
> > > >      commit_set_priority_action(flow, base, odp_actions, wc,
> > > > use_masked);
> > > > diff --git a/lib/ofp-actions.c b/lib/ofp-actions.c
> > > > index 0342a228b..28a12a569 100644
> > > > --- a/lib/ofp-actions.c
> > > > +++ b/lib/ofp-actions.c
> > > > @@ -4441,6 +4441,7 @@ decode_NXAST_RAW_ENCAP(const struct
> > > > nx_action_encap *nae,
> > > >      switch (ntohl(nae->new_pkt_type)) {
> > > >      case PT_ETH:
> > > >      case PT_NSH:
> > > > +    case PT_MPLS:
> > > >          /* Add supported encap header types here. */
> > > >          break;
> > > >      default:
> > > > @@ -4492,6 +4493,8 @@ parse_encap_header(const char *hdr, ovs_be32
> > > > *packet_type)
> > > >          *packet_type = htonl(PT_ETH);
> > > >      } else if (strcmp(hdr, "nsh") == 0) {
> > > >          *packet_type = htonl(PT_NSH);
> > > > +    } else if (strcmp(hdr, "mpls") == 0) {
> > > > +        *packet_type = htonl(PT_MPLS);
> > > >      } else {
> > > >          return false;
> > > >      }
> > > > @@ -4573,6 +4576,8 @@ format_encap_pkt_type(const ovs_be32 pkt_type)
> > > >          return "ethernet";
> > > >      case PT_NSH:
> > > >          return "nsh";
> > > > +    case PT_MPLS:
> > > > +        return "mpls";
> > > >      default:
> > > >          return "UNKNOWN";
> > > >      }
> > > > diff --git a/lib/ofp-ed-props.c b/lib/ofp-ed-props.c
> > > > index 02a9235d5..fc261e4c6 100644
> > > > --- a/lib/ofp-ed-props.c
> > > > +++ b/lib/ofp-ed-props.c
> > > > @@ -79,6 +79,27 @@ decode_ed_prop(const struct ofp_ed_prop_header
> > > > **ofp_prop,
> > > >          }
> > > >          break;
> > > >      }
> > > > +    case OFPPPC_MPLS: {
> > > > +       switch (prop_type) {
> > > > +        case OFPPPT_PROP_MPLS_ETHERTYPE: {
> > > > +            struct ofp_ed_prop_mpls_ethertype *opnmt =
> > > > +                ALIGNED_CAST(struct ofp_ed_prop_mpls_ethertype *,
> > > > *ofp_prop);
> > > > +            if (len > sizeof(*opnmt) || len > *remaining) {
> > > > +                return OFPERR_NXBAC_BAD_ED_PROP;
> > > > +            }
> > > > +            struct ofpact_ed_prop_mpls_ethertype *pnmt =
> > > > +                    ofpbuf_put_uninit(out, sizeof(*pnmt));
> > > > +            pnmt->header.prop_class = prop_class;
> > > > +            pnmt->header.type = prop_type;
> > > > +            pnmt->header.len = len;
> > > > +            pnmt->ether_type = opnmt->ether_type;
> > > > +            break;
> > > > +        }
> > > > +        default:
> > > > +            return OFPERR_NXBAC_UNKNOWN_ED_PROP;
> > > > +        }
> > > > +        break;
> > > > +    }
> > > >      default:
> > > >          return OFPERR_NXBAC_UNKNOWN_ED_PROP;
> > > >      }
> > > > @@ -134,6 +155,27 @@ encode_ed_prop(const struct ofpact_ed_prop
> > > > **prop,
> > > >          }
> > > >          break;
> > > >      }
> > > > +    case OFPPPC_MPLS: {
> > > > +       switch ((*prop)->type) {
> > > > +       case OFPPPT_PROP_MPLS_ETHERTYPE: {
> > > > +           struct ofpact_ed_prop_mpls_ethertype *pnmt =
> > > > +                ALIGNED_CAST(struct
> > > > ofpact_ed_prop_mpls_ethertype *,
> > > > *prop);
> > > > +            struct ofp_ed_prop_mpls_ethertype *opnmt =
> > > > +                    ofpbuf_put_uninit(out, sizeof(*opnmt));
> > > > +            opnmt->header.prop_class = htons((*prop)->prop_class);
> > > > +            opnmt->header.type = (*prop)->type;
> > > > +            opnmt->header.len =
> > > > +                    offsetof(struct ofpact_ed_prop_mpls_ethertype,
> > > > pad);
> > > > +            opnmt->ether_type = pnmt->ether_type;
> > > > +            prop_len = sizeof(*pnmt);
> > > > +            break;
> > > > +
> > > > +       }
> > > > +       default:
> > > > +            return OFPERR_OFPBAC_BAD_ARGUMENT;
> > > > +       }
> > > > +       break;
> > > > +    }
> > > >      default:
> > > >          return OFPERR_OFPBAC_BAD_ARGUMENT;
> > > >      }
> > > > @@ -181,6 +223,13 @@ parse_ed_prop_type(uint16_t prop_class,
> > > >          } else {
> > > >              return false;
> > > >          }
> > > > +    case OFPPPC_MPLS:
> > > > +        if (!strcmp(str, "ether_type")) {
> > > > +            *type = OFPPPT_PROP_MPLS_ETHERTYPE;
> > > > +            return true;
> > > > +        } else {
> > > > +            return false;
> > > > +        }
> > > >      default:
> > > >          return false;
> > > >      }
> > > > @@ -259,6 +308,28 @@ parse_ed_prop_value(uint16_t prop_class,
> > > > uint8_t
> > > > prop_type OVS_UNUSED,
> > > >              OVS_NOT_REACHED();
> > > >          }
> > > >          break;
> > > > +    case OFPPPC_MPLS:
> > > > +        switch (prop_type) {
> > > > +        case OFPPPT_PROP_MPLS_ETHERTYPE: {
> > > > +            uint16_t ethertype;
> > > > +            error = str_to_u16(value, "ether_type", &ethertype);
> > > > +            if (error != NULL) {
> > > > +                return error;
> > > > +            }
> > > > +            struct ofpact_ed_prop_mpls_ethertype *pnmt =
> > > > +                    ofpbuf_put_uninit(out, sizeof(*pnmt));
> > > > +            pnmt->header.prop_class = prop_class;
> > > > +            pnmt->header.type = prop_type;
> > > > +            pnmt->header.len =
> > > > +                    offsetof(struct ofpact_ed_prop_mpls_ethertype,
> > > > pad);
> > > > +            pnmt->ether_type = ethertype;
> > > > +
> > > > +            break;
> > > > +        }
> > > > +        default:
> > > > +            break;
> > > > +      }
> > > > +      break;
> > > >      default:
> > > >          /* Unsupported property classes rejected before. */
> > > >          OVS_NOT_REACHED();
> > > > @@ -300,6 +371,14 @@ format_ed_prop_type(const struct ofpact_ed_prop
> > > > *prop)
> > > >              OVS_NOT_REACHED();
> > > >          }
> > > >          break;
> > > > +    case OFPPPC_MPLS:
> > > > +         switch (prop->type) {
> > > > +         case OFPPPT_PROP_MPLS_ETHERTYPE:
> > > > +              return "ether_type";
> > > > +         default:
> > > > +               OVS_NOT_REACHED();
> > > > +         }
> > > > +         break;
> > > >      default:
> > > >          OVS_NOT_REACHED();
> > > >      }
> > > > @@ -332,6 +411,18 @@ format_ed_prop(struct ds *s OVS_UNUSED,
> > > >          default:
> > > >              OVS_NOT_REACHED();
> > > >          }
> > > > +     case OFPPPC_MPLS:
> > > > +        switch (prop->type) {
> > > > +        case OFPPPT_PROP_MPLS_ETHERTYPE: {
> > > > +          struct ofpact_ed_prop_mpls_ethertype *pnmt =
> > > > +                ALIGNED_CAST(struct
> > > > ofpact_ed_prop_mpls_ethertype *,
> > > > prop);
> > > > +            ds_put_format(s, "%s=%d", format_ed_prop_type(prop),
> > > > +                          pnmt->ether_type);
> > > > +            return;
> > > > +        }
> > > > +        default:
> > > > +            OVS_NOT_REACHED();
> > > > +        }
> > > >      default:
> > > >          OVS_NOT_REACHED();
> > > >      }
> > > > diff --git a/lib/ovs-actions.xml b/lib/ovs-actions.xml
> > > > index a2778de4b..e97f818d9 100644
> > > > --- a/lib/ovs-actions.xml
> > > > +++ b/lib/ovs-actions.xml
> > > > @@ -265,13 +265,13 @@
> > > >        </p>
> > > > 
> > > >        <p>
> > > > -        When a <code>decap</code> action decapsulates a packet,
> > > > Open
> > > > vSwitch
> > > > -        raises this error if it does not support the type of inner
> > > > packet.
> > > > -        <code>decap</code> of an Ethernet header raises this
> > > > error if a
> > > > VLAN
> > > > -        header is present, <code>decap</code> of a NSH packet
> > > > raises
> > > > this error
> > > > -        if the NSH inner packet is not Ethernet, IPv4, IPv6, or
> > > > NSH,
> > > > and
> > > > -        <code>decap</code> of other types of packets is
> > > > unsupported and
> > > > also
> > > > -        raises this error.
> > > > +        The <code>decap</code> action is supported only for packet
> > > > types
> > > > +        ethernet, NSH and MPLS. Openvswitch raises this error
> > > > for other
> > > > +        packet types. When a <code>decap</code> action
> > > > decapsulates a
> > > > packet,
> > > > +        Open vSwitch raises this error if it does not support
> > > > the type
> > > > of inner
> > > > +        packet. <code>decap</code> of an Ethernet header raises
> > > > this
> > > > error if a
> > > > +        VLAN header is present, <code>decap</code> of a NSH packet
> > > > raises this
> > > > +        error if the NSH inner packet is not Ethernet, IPv4,
> > > > IPv6, or
> > > > NSH.
> > > >        </p>
> > > > 
> > > >        <p>
> > > > @@ -1097,6 +1097,8 @@ for <var>i</var> in [1,<var>n_members</var>]:
> > > >        <h2>The <code>encap</code> action</h2>
> > > >        <syntax><code>encap(nsh(</code>[<code>md_type=<var>md_type</var></code>]<code>, </code>[<code>tlv(<var>class</var>,<var>type</var>,<var>value</var>)</code>]...<code>))</code></syntax>
> > > >        <syntax><code>encap(ethernet)</code></syntax>
> > > > +
> > > > <syntax><code>encap(mpls(ether_type=<var>ether_type</var>))</code>
> > > > +      </syntax>
> > > > 
> > > >        <p>
> > > >          The <code>encap</code> action encapsulates a packet with a
> > > > specified
> > > > @@ -1135,6 +1137,12 @@ for <var>i</var> in [1,<var>n_members</var>]:
> > > >          source and destination are initially zeroed.
> > > >        </p>
> > > > 
> > > > +      <p>
> > > > +        The <code>encap(mpls(ethertype=....))</code> variant
> > > > encapsulates an
> > > > +        ethernet or L3 packet with a MPLS header. The
> > > > <var>ethertype</var>
> > > > +        could be MPLS unicast (0x8847) or multicast (0x8848)
> > > > ethertypes.
> > > > +      </p>
> > > > +
> > > >        <conformance>
> > > >          This action is an Open vSwitch extension to OpenFlow
> > > > 1.3 and
> > > > later,
> > > >          introduced in Open vSwitch 2.8.
> > > > @@ -1144,6 +1152,9 @@ for <var>i</var> in [1,<var>n_members</var>]:
> > > >      <action name="DECAP">
> > > >        <h2>The <code>decap</code> action</h2>
> > > >        <syntax><code>decap</code></syntax>
> > > > +      <syntax><code>decap(packet_type(ns=<var>name_space</var>,
> > > > +      type=<var>ethertype</var>))</code></syntax> for decapsulating
> > > > MPLS
> > > > +      packets.
> > > > 
> > > >        <p>
> > > >          Removes an outermost encapsulation from the packet:
> > > > @@ -1164,6 +1175,12 @@ for <var>i</var> in [1,<var>n_members</var>]:
> > > >            packet type errors.
> > > >          </li>
> > > > 
> > > > +        <li>
> > > > +          Otherwise, if the packet is a MPLS packet, removes
> > > > the MPLS
> > > > header
> > > > +          and classifies the inner packet as mentioned in the
> > > > packet
> > > > type
> > > > +          argument of the decap.
> > > > +        </li>
> > > > +
> > > >          <li>
> > > >            Otherwise, raises an unsupported packet type error.
> > > >          </li>
> > > > diff --git a/lib/packets.c b/lib/packets.c
> > > > index 4a7643c5d..5e3c3900f 100644
> > > > --- a/lib/packets.c
> > > > +++ b/lib/packets.c
> > > > @@ -418,6 +418,38 @@ push_mpls(struct dp_packet *packet, ovs_be16
> > > > ethtype, ovs_be32 lse)
> > > >      pkt_metadata_init_conn(&packet->md);
> > > >  }
> > > > 
> > > > +void
> > > > +add_mpls(struct dp_packet *packet, ovs_be16 ethtype, ovs_be32
> > > > lse, bool
> > > > l3)
> > > > +{
> > > > +    char * header;
> > > > +
> > > > +    if (!eth_type_mpls(ethtype)) {
> > > > +        return;
> > > > +    }
> > > > +
> > > > +    if (!l3) {
> > > > +        header =  dp_packet_push_uninit(packet, MPLS_HLEN);
> > > > +        memcpy(header, &lse, sizeof lse);
> > > > +        packet->l2_5_ofs = 0;
> > > > +        packet->packet_type = htonl(PT_MPLS);
> > > > +    } else {
> > > > +        size_t len;
> > > > +
> > > > +        if (!is_mpls(packet)) {
> > > > +            /* Set MPLS label stack offset. */
> > > > +            packet->l2_5_ofs = packet->l3_ofs;
> > > > +        }
> > > > +        set_ethertype(packet, ethtype);
> > > > +
> > > > +        /* Push new MPLS shim header onto packet. */
> > > > +        len = packet->l2_5_ofs;
> > > > +        header = dp_packet_resize_l2_5(packet, MPLS_HLEN);
> > > > +        memmove(header, header + MPLS_HLEN, len);
> > > > +        memcpy(header + len, &lse, sizeof lse);
> > > > +    }
> > > > +    pkt_metadata_init_conn(&packet->md);
> > > > +}
> > > > +
> > > >  /* If 'packet' is an MPLS packet, removes its outermost MPLS label
> > > > stack entry.
> > > >   * If the label that was removed was the only MPLS label, changes
> > > > 'packet''s
> > > >   * Ethertype to 'ethtype' (which ordinarily should not be an MPLS
> > > > @@ -429,6 +461,10 @@ pop_mpls(struct dp_packet *packet, ovs_be16
> > > > ethtype)
> > > >          struct mpls_hdr *mh = dp_packet_l2_5(packet);
> > > >          size_t len = packet->l2_5_ofs;
> > > > 
> > > > +        if (ethtype == htons(ETH_TYPE_TEB)) {
> > > > +             packet->packet_type = htonl(PT_ETH);
> > > > +        }
> > > > +
> > > >          set_ethertype(packet, ethtype);
> > > >          if (get_16aligned_be32(&mh->mpls_lse) &
> > > > htonl(MPLS_BOS_MASK)) {
> > > >              dp_packet_set_l2_5(packet, NULL);
> > > > diff --git a/lib/packets.h b/lib/packets.h
> > > > index 481bc22fa..3f5862e08 100644
> > > > --- a/lib/packets.h
> > > > +++ b/lib/packets.h
> > > > @@ -350,6 +350,8 @@ void set_mpls_lse_label(ovs_be32 *lse, ovs_be32
> > > > label);
> > > >  void set_mpls_lse_bos(ovs_be32 *lse, uint8_t bos);
> > > >  ovs_be32 set_mpls_lse_values(uint8_t ttl, uint8_t tc, uint8_t bos,
> > > >                               ovs_be32 label);
> > > > +void add_mpls(struct dp_packet *packet, ovs_be16 ethtype,
> > > > ovs_be32 lse,
> > > > +              bool l3_flag);
> > > > 
> > > >  /* Example:
> > > >   *
> > > > diff --git a/ofproto/ofproto-dpif-ipfix.c
> > > > b/ofproto/ofproto-dpif-ipfix.c
> > > > index 796eb6f88..9280e008e 100644
> > > > --- a/ofproto/ofproto-dpif-ipfix.c
> > > > +++ b/ofproto/ofproto-dpif-ipfix.c
> > > > @@ -3018,6 +3018,7 @@ dpif_ipfix_read_actions(const struct flow
> > > > *flow,
> > > >          case OVS_ACTION_ATTR_CHECK_PKT_LEN:
> > > >          case OVS_ACTION_ATTR_UNSPEC:
> > > >          case OVS_ACTION_ATTR_DROP:
> > > > +        case OVS_ACTION_ATTR_ADD_MPLS:
> > > >          case __OVS_ACTION_ATTR_MAX:
> > > >          default:
> > > >              break;
> > > > diff --git a/ofproto/ofproto-dpif-sflow.c
> > > > b/ofproto/ofproto-dpif-sflow.c
> > > > index fdcb9eabb..ca46a9bc4 100644
> > > > --- a/ofproto/ofproto-dpif-sflow.c
> > > > +++ b/ofproto/ofproto-dpif-sflow.c
> > > > @@ -1226,6 +1226,7 @@ dpif_sflow_read_actions(const struct flow
> > > > *flow,
> > > >          case OVS_ACTION_ATTR_UNSPEC:
> > > >          case OVS_ACTION_ATTR_CHECK_PKT_LEN:
> > > >          case OVS_ACTION_ATTR_DROP:
> > > > +        case OVS_ACTION_ATTR_ADD_MPLS:
> > > >          case __OVS_ACTION_ATTR_MAX:
> > > >          default:
> > > >              break;
> > > > diff --git a/ofproto/ofproto-dpif-xlate.c
> > > > b/ofproto/ofproto-dpif-xlate.c
> > > > index 7108c8a30..a97534233 100644
> > > > --- a/ofproto/ofproto-dpif-xlate.c
> > > > +++ b/ofproto/ofproto-dpif-xlate.c
> > > > @@ -6381,6 +6381,45 @@ rewrite_flow_encap_ethernet(struct xlate_ctx
> > > > *ctx,
> > > >          ctx->error = XLATE_UNSUPPORTED_PACKET_TYPE;
> > > >      }
> > > >  }
> > > > +static void
> > > > +rewrite_flow_encap_mpls(struct xlate_ctx *ctx,
> > > > +                        const struct ofpact_encap *encap,
> > > > +                        struct flow *flow,
> > > > +                        struct flow_wildcards *wc)
> > > > +{
> > > > +    int n;
> > > > +    uint32_t i;
> > > > +    uint16_t ether_type;
> > > > +    const char *ptr = (char *) encap->props;
> > > > +
> > > > +     for (i = 0; i < encap->n_props; i++) {
> > > > +        struct ofpact_ed_prop *prop_ptr =
> > > > +            ALIGNED_CAST(struct ofpact_ed_prop *, ptr);
> > > > +        if (prop_ptr->prop_class == OFPPPC_MPLS) {
> > > > +            switch (prop_ptr->type) {
> > > > +                case OFPPPT_PROP_MPLS_ETHERTYPE: {
> > > > +                     struct ofpact_ed_prop_mpls_ethertype
> > > > *prop_ether_type =
> > > > +                        ALIGNED_CAST(struct
> > > > ofpact_ed_prop_mpls_ethertype *,
> > > > +                                     prop_ptr);
> > > > +                    ether_type = prop_ether_type->ether_type;
> > > > +                    break;
> > > > +                 }
> > > > +            }
> > > > +        }
> > > > +     }
> > > > +
> > > > +    wc->masks.packet_type = OVS_BE32_MAX;
> > > > +    if (flow->packet_type != htonl(PT_MPLS)) {
> > > > +        memset(&ctx->wc->masks.mpls_lse, 0x0,
> > > > +               sizeof *wc->masks.mpls_lse * FLOW_MAX_MPLS_LABELS);
> > > > +        memset(&flow->mpls_lse, 0x0, sizeof *flow->mpls_lse *
> > > > +               FLOW_MAX_MPLS_LABELS);
> > > > +    }
> > > > +    flow->packet_type = htonl(PT_MPLS);
> > > > +    n = flow_count_mpls_labels(flow, ctx->wc);
> > > > +    flow_push_mpls(flow, n, htons(ether_type), ctx->wc, true);
> > > > +}
> > > > +
> > > > 
> > > >  /* For an MD2 NSH header returns a pointer to an ofpbuf with the
> > > > encoded
> > > >   * MD2 TLVs provided as encap properties to the encap
> > > > operation. This
> > > > @@ -6513,6 +6552,12 @@ xlate_generic_encap_action(struct
> > > > xlate_ctx *ctx,
> > > >          case PT_NSH:
> > > >              encap_data = rewrite_flow_push_nsh(ctx, encap,
> > > > flow, wc);
> > > >              break;
> > > > +        case PT_MPLS:
> > > > +            rewrite_flow_encap_mpls(ctx, encap,  flow, wc);
> > > > +            if (!ctx->xbridge->support.add_mpls) {
> > > > +                ctx->xout->slow |= SLOW_ACTION;
> > > > +            }
> > > > +            break;
> > > >          default:
> > > >              /* New packet type was checked during decoding. */
> > > >              OVS_NOT_REACHED();
> > > > @@ -6582,6 +6627,21 @@ xlate_generic_decap_action(struct
> > > > xlate_ctx *ctx,
> > > >              ctx->pending_decap = true;
> > > >              /* Trigger recirculation. */
> > > >              return true;
> > > > +        case PT_MPLS: {
> > > > +             int n;
> > > > +             ovs_be16 ethertype;
> > > > +
> > > > +             flow->packet_type = decap->new_pkt_type;
> > > > +             ethertype = pt_ns_type_be(flow->packet_type);
> > > > +
> > > > +             n = flow_count_mpls_labels(flow, ctx->wc);
> > > > +             flow_pop_mpls(flow, n, ethertype, ctx->wc);
> > > > +             if (!ctx->xbridge->support.add_mpls) {
> > > > +                ctx->xout->slow |= SLOW_ACTION;
> > > > +             }
> > > > +             ctx->pending_decap = true;
> > > > +             return true;
> > > > +        }
> > > >          default:
> > > >              /* Error handling: drop packet. */
> > > >              xlate_report_debug(
> > > > diff --git a/ofproto/ofproto-dpif.c b/ofproto/ofproto-dpif.c
> > > > index fd0b2fdea..d9a2922e7 100644
> > > > --- a/ofproto/ofproto-dpif.c
> > > > +++ b/ofproto/ofproto-dpif.c
> > > > @@ -1520,6 +1520,44 @@ check_nd_extensions(struct dpif_backer
> > > > *backer)
> > > > 
> > > >      return !error;
> > > >  }
> > > > +/* Tests whether 'backer''s datapath supports the
> > > > + * OVS_ACTION_ATTR_ADD_MPLS action. */
> > > > +static bool
> > > > +check_add_mpls(struct dpif_backer *backer)
> > > > +{
> > > > +    struct odputil_keybuf keybuf;
> > > > +    struct ofpbuf actions;
> > > > +    struct ofpbuf key;
> > > > +    struct flow flow;
> > > > +    bool supported;
> > > > +
> > > > +    struct odp_flow_key_parms odp_parms = {
> > > > +        .flow = &flow,
> > > > +        .probe = true,
> > > > +    };
> > > > +
> > > > +    memset(&flow, 0, sizeof flow);
> > > > +    ofpbuf_use_stack(&key, &keybuf, sizeof keybuf);
> > > > +    odp_flow_key_from_flow(&odp_parms, &key);
> > > > +    ofpbuf_init(&actions, 64);
> > > > +
> > > > +    struct ovs_action_add_mpls *mpls;
> > > > +
> > > > +    mpls = nl_msg_put_unspec_zero(&actions,
> > > > +                                  OVS_ACTION_ATTR_ADD_MPLS,
> > > > +                                  sizeof *mpls);
> > > > +    mpls->mpls_ethertype = htons(ETH_TYPE_MPLS);
> > > > +
> > > > +    supported = dpif_probe_feature(backer->dpif, "add_mpls", &key,
> > > > +                                   &actions, NULL);
> > > > +    ofpbuf_uninit(&actions);
> > > > +    VLOG_INFO("%s: Datapath %s add_mpls action",
> > > > +              dpif_name(backer->dpif), supported ? "supports"
> > > > +                                                 : "does not
> > > > support");
> > > > +    return supported;
> > > > +
> > > > +}
> > > > +
> > > > 
> > > >  #define CHECK_FEATURE__(NAME, SUPPORT, FIELD, VALUE, ETHTYPE)
> > > > \
> > > >  static bool
> > > > \
> > > > @@ -1590,6 +1628,7 @@ check_support(struct dpif_backer *backer)
> > > >          dpif_supports_explicit_drop_action(backer->dpif);
> > > >      backer->rt_support.lb_output_action=
> > > >          dpif_supports_lb_output_action(backer->dpif);
> > > > +    backer->rt_support.add_mpls = check_add_mpls(backer);
> > > > 
> > > >      /* Flow fields. */
> > > >      backer->rt_support.odp.ct_state = check_ct_state(backer);
> > > > diff --git a/ofproto/ofproto-dpif.h b/ofproto/ofproto-dpif.h
> > > > index b41c3d82a..c04bfff8d 100644
> > > > --- a/ofproto/ofproto-dpif.h
> > > > +++ b/ofproto/ofproto-dpif.h
> > > > @@ -204,7 +204,10 @@ struct group_dpif *group_dpif_lookup(struct
> > > > ofproto_dpif *,
> > > >      DPIF_SUPPORT_FIELD(bool, explicit_drop_action, "Explicit Drop
> > > > action")  \
> > > >                                                                              \
> > > >      /* True if the datapath supports balance_tcp optimization */
> > > > \
> > > > -    DPIF_SUPPORT_FIELD(bool, lb_output_action, "Optimized
> > > > Balance TCP
> > > > mode")
> > > > +    DPIF_SUPPORT_FIELD(bool, lb_output_action, "Optimized
> > > > Balance TCP
> > > > mode")\
> > > > +
> > > > \
> > > > +    /* True if the datapath supports layer 2 MPLS tunnelling */
> > > > \
> > > > +    DPIF_SUPPORT_FIELD(bool, add_mpls, "l2 MPLS tunnelling")
> > > > 
> > > > 
> > > >  /* Stores the various features which the corresponding backer
> > > > supports.
> > > > */
> > > > diff --git a/tests/system-traffic.at b/tests/system-traffic.at
> > > > index fb5b9a36d..b865b5210 100644
> > > > --- a/tests/system-traffic.at
> > > > +++ b/tests/system-traffic.at
> > > > @@ -1027,9 +1027,45 @@ NS_CHECK_EXEC([at_ns0], [ping -q -c 3 -i
> > > > 0.3 -w 2
> > > > 10.1.1.2 | FORMAT_PING], [0],
> > > >  3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > >  ])
> > > > 
> > > > +OVS_TRAFFIC_VSWITCHD_STOP
> > > > +AT_CLEANUP
> > > > +
> > > > +
> > > > +AT_SETUP([datapath - ptap mpls actions])
> > > > +OVS_TRAFFIC_VSWITCHD_START([_ADD_BR([br1])])
> > > > +
> > > > +ADD_NAMESPACES(at_ns0, at_ns1)
> > > > +
> > > > +ADD_VETH(p0, at_ns0, br0, "10.1.1.1/24")
> > > > +ADD_VETH(p1, at_ns1, br1, "10.1.1.2/24")
> > > > +
> > > > +AT_CHECK([ip link add patch0 type veth peer name patch1])
> > > > +on_exit 'ip link del patch0'
> > > > +
> > > > +AT_CHECK([ip link set dev patch0 up])
> > > > +AT_CHECK([ip link set dev patch1 up])
> > > > +AT_CHECK([ovs-vsctl add-port br0 patch0 -- set interface patch0
> > > > ofport_request=100])
> > > > +AT_CHECK([ovs-vsctl add-port br1 patch1 -- set interface patch1
> > > > ofport_request=100])
> > > > +
> > > > +AT_DATA([flows.txt], [dnl
> > > > +table=0,priority=100,dl_type=0x0800 actions=encap(mpls(ether_type=0x8847)),set_mpls_label:2,encap(ethernet),output:100
> > > > +table=0,priority=100,dl_type=0x8847,mpls_label=2
> > > > actions=decap(),decap(packet_type(ns=0,type=0)),resubmit(,3)
> > > > +table=0,priority=10 actions=resubmit(,3)
> > > > +table=3,priority=10 actions=normal
> > > > +])
> > > > +
> > > > +AT_CHECK([ovs-ofctl  -Oopenflow13 add-flows br0 flows.txt])
> > > > +AT_CHECK([ovs-ofctl  -Oopenflow13 add-flows br1 flows.txt])
> > > > +
> > > > +NS_CHECK_EXEC([at_ns0], [ping -q -c 3 -i 0.3 -w 2 10.1.1.2 |
> > > > FORMAT_PING], [0], [dnl
> > > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > > +])
> > > > +
> > > > +
> > > >  NS_CHECK_EXEC([at_ns1], [ping -q -c 3 -i 0.3 -w 2 10.1.1.1 |
> > > > FORMAT_PING], [0], [dnl
> > > >  3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > >  ])
> > > > +
> > > >  OVS_TRAFFIC_VSWITCHD_STOP
> > > >  AT_CLEANUP
> > > > 
> > > > -- 
> > > > 2.18.4
> > > 
>

On 1 Apr 2021, at 10:35, Martin Varghese wrote:

> On Thu, Apr 01, 2021 at 08:59:27AM +0200, Eelco Chaudron wrote:
>>
>>
>> On 1 Apr 2021, at 6:10, Martin Varghese wrote:
>>
>>> On Wed, Mar 31, 2021 at 03:59:40PM +0200, Eelco Chaudron wrote:
>>>>
>>>>
>>>> On 26 Mar 2021, at 7:21, Martin Varghese wrote:
>>>>
>>>>> From: Martin Varghese <martin.varghese@nokia.com>
>>>>>
>>>>> The encap & decap actions are extended to support MPLS packet 
>>>>> type.
>>>>> Encap & decap actions adds and removes MPLS header at start of the
>>>>> packet.
>>>>
>>>> Hi Martin,
>>>>
>>>> I’m trying to do some real-life testing, and I’m running into
>>>> issues. This
>>>> might be me setting it up wrongly but just wanting to confirm…
>>>>
>>>> I’m sending an MPLS packet that contains an ARP packet into a
>>>> physical port.
>>>> This is the packet:
>>>>
>>>> Frame 4: 64 bytes on wire (512 bits), 64 bytes captured (512 bits)
>>>>     Encapsulation type: Ethernet (1)
>>>>     [Protocols in frame: eth:ethertype:mpls:data]
>>>> Ethernet II, Src: 00:00:00_00:00:01 (00:00:00:00:00:01), Dst:
>>>> 00:00:00_00:00:02 (00:00:00:00:00:02)
>>>>     Destination: 00:00:00_00:00:02 (00:00:00:00:00:02)
>>>>         Address: 00:00:00_00:00:02 (00:00:00:00:00:02)
>>>>         .... ..0. .... .... .... .... = LG bit: Globally unique
>>>> address
>>>> (factory default)
>>>>         .... ...0 .... .... .... .... = IG bit: Individual address
>>>> (unicast)
>>>>     Source: 00:00:00_00:00:01 (00:00:00:00:00:01)
>>>>         Address: 00:00:00_00:00:01 (00:00:00:00:00:01)
>>>>         .... ..0. .... .... .... .... = LG bit: Globally unique
>>>> address
>>>> (factory default)
>>>>         .... ...0 .... .... .... .... = IG bit: Individual address
>>>> (unicast)
>>>>     Type: MPLS label switched packet (0x8847)
>>>> MultiProtocol Label Switching Header, Label: 100, Exp: 0, S: 1, 
>>>> TTL:
>>>> 64
>>>>     0000 0000 0000 0110 0100 .... .... .... = MPLS Label: 100
>>>>     .... .... .... .... .... 000. .... .... = MPLS Experimental
>>>> Bits: 0
>>>>     .... .... .... .... .... ...1 .... .... = MPLS Bottom Of Label
>>>> Stack: 1
>>>>     .... .... .... .... .... .... 0100 0000 = MPLS TTL: 64
>>>> Data (46 bytes)
>>>>
>>>> 0000  ff ff ff ff ff ff 52 54 00 88 51 38 08 06 00 01
>>>> ......RT..Q8....
>>>> 0010  08 00 06 04 00 01 52 54 00 88 51 38 01 01 01 65
>>>> ......RT..Q8...e
>>>> 0020  00 00 00 00 00 00 01 01 01 64 27 98 a0 47
>>>> .........d'..G
>>>>     Data:
>>>> ffffffffffff525400885138080600010800060400015254008851380101016500000000?
>>>>
>>>>
>>>> I’m trying to use the following rules:
>>>>
>>>>   ovs-ofctl del-flows ovs_pvp_br0
>>>>   ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0
>>>> "priority=100,dl_type=0x8847,mpls_label=100
>>>> actions=decap(),decap(packet_type(ns=0,type=0x806)),resubmit(,3)"
>>>>   ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0 "table=3,priority=10
>>>> actions=normal"
>>>>
>>>> With these, I expect the packet to be sent to vnet0, but it’s 
>>>> not.
>>>> Actually,
>>>> the datapath rule looks odd, while the userspace rules seem to 
>>>> match:
>>>>
>>>>   $ ovs-dpctl dump-flows
>>>>   recirc_id(0),in_port(1),eth(),eth_type(0x8847),mpls(label=100/0xfffff,tc=0/0,ttl=0/0x0,bos=1/1),
>>>> packets:13, bytes:1118, used:0.322s,
>>>> actions:pop_eth,pop_mpls(eth_type=0x806),recirc(0x19a)
>>>>   recirc_id(0x19a),in_port(1),eth_type(0x0806), packets:13,
>>>> bytes:884,
>>>> used:0.322s, actions:drop
>>>>
>>>>   $ ovs-ofctl dump-flows ovs_pvp_br0 -O OpenFlow13
>>>>   cookie=0x0, duration=85.007s, table=0, n_packets=51, 
>>>> n_bytes=4386,
>>>> priority=100,mpls,mpls_label=100
>>>> actions=decap(),decap(packet_type(ns=0,type=0x806)),resubmit(,3)
>>>>   cookie=0x0, duration=84.990s, table=3, n_packets=51, 
>>>> n_bytes=3468,
>>>> priority=10 actions=NORMAL
>>>>
>>> The inner packet is ethernet. So the packet type should be 
>>> (ns=0,type=0)
>>> ?
>>
>> Forgot to add that I already tried that to start with, based on the 
>> example,
>> but as that did not work I tried 0x806.
>>
>> PS: I have this as a remark in my review notes, i.e., to explain the 
>> ns and
>> type usage here.
>>
>>
>> This resulted in packets being counted at the open flow level, but it
>> results in NO data path rules. Do get an error though:
>>
>> 2021-04-01T06:53:36.056Z|00141|dpif(handler37)|WARN|system@ovs-system:
>> failed to put[create] (Invalid argument)
>> ufid:3d2d6f6d-5a66-4ace-8b09-7cdcfa5efc8e 
>> recirc_id(0),dp_hash(0/0),skb_priority(0/0),in_port(1),skb_mark(0/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),eth(src=00:00:00:00:00:01/00:00:00:00:00:00,dst=00:00:00:00:00:02/00:00:00:00:00:00),eth_type(0x8847),mpls(label=100/0xfffff,tc=0/0,ttl=64/0x0,bos=1/1),
>> actions:pop_eth,pop_mpls(eth_type=0x6558),set(eth()),recirc(0x4c)
>
> This set(eth) before the recirc is the problem i guesss. I need to 
> check
>> 2021-04-01T06:53:36.056Z|00142|dpif(handler37)|WARN|system@ovs-system:
>> execute pop_eth,pop_mpls(eth_type=0x6558),set(eth()),recirc(0x4c) 
>> failed
>> (Invalid argument) on packet 
>> mpls,vlan_tci=0x0000,dl_src=00:00:00:00:00:01,dl_dst=00:00:00:00:00:02,mpls_label=100,mpls_tc=0,mpls_ttl=64,mpls_bos=1
>>  with metadata skb_priority(0),skb_mark(0),in_port(1) mtu 0
>>
>> Are there missing parts in my kernel that do not get properly 
>> detected by
>> the feature detection?
>>
>> $ ovs-appctl dpif/show-dp-features ovs_pvp_br0
>> Masked set action: Yes
>> Tunnel push pop: No
>> Ufid: Yes
>> Truncate action: Yes
>> Clone action: Yes
>> Sample nesting: 10
>> Conntrack eventmask: Yes
>> Conntrack clear: Yes
>> Max dp_hash algorithm: 0
>> Check pkt length action: Yes
>> Conntrack timeout policy: Yes
>> Explicit Drop action: No
>> Optimized Balance TCP mode: No
>> l2 MPLS tunnelling: Yes
>> Max VLAN headers: 2
>> Max MPLS depth: 3
>> Recirc: Yes
>> CT state: Yes
>> CT zone: Yes
>> CT mark: Yes
>> CT label: Yes
>> CT state NAT: Yes
>> CT orig tuple: Yes
>> CT orig tuple for IPv6: Yes
>> IPv6 ND Extension: No
>>
> You are good
>
> I am not sure what is going wrong. Your test case looks same as the 
> unit
> test i added.
>
> I tried myself again and this is i get
>
> ovs-ofctl -O OpenFlow13 add-flow br_mpls2 
> "in_port=$egress_port,dl_type=0x8847
> +actions=decap(),decap(packet_type(ns=0,type=0),goto_table:1"
> ovs-ofctl -O OpenFlow13 add-flow br_mpls2
> +"table=1,in_port=$egress_port,dl_type=0x0800,nw_dst=1.1.1.2
> +actions=set_field:00:00:00:00:00:02->dl_dst,set_field:00:00:00:00:00:01->dl_sr
> +c output:$ingress_port"
>
> recirc_id(0x3),in_port(6),eth(src=36:b1:ee:7c:01:03,dst=36:b1:ee:7c:01:02),eth_
> +type(0x0800),ipv4(dst=1.1.1.2,frag=no), packets:3, bytes:294, 
> used:0.837s,
> +actions:set(eth(src=00:00:00:00:00:01,dst=00:00:00:00:00:02)),4
> recirc_id(0),in_port(6),eth(),eth_type(0x8847),mpls(label=0/0x0,tc=0/0,ttl=0/0x
> +0,bos=1/1), packets:3, bytes:348, used:0.837s,
> +actions:pop_eth,pop_mpls(eth_type=0x6558),recirc(0x3)
>
> The packet to the ovs is
> ETH|MPLS|ETH|IP ?
> How it is differnt from you test case?

Mine is ETH|MPLS|ETH|ARP, which works fine with pop_mpls

> Thanks for your time.

Your welcome

>>>>
>>>> If I use the old way, doing pop_mpls, it works fine:
>>>>
>>>>
>>>> ovs-ofctl del-flows ovs_pvp_br0
>>>> ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0
>>>> "priority=100,dl_type=0x8847,mpls_label=100
>>>> actions=pop_mpls:0x0806,resubmit(,3)"
>>>> ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0 "table=3,priority=10
>>>> actions=normal"
>>>>
>>>>
>>>> I also noticed (despite the test example) to make encap work, I had
>>>> to set
>>>> the ethernet MAC addresses, or else the packets were not getting 
>>>> out.
>>>> So something like:
>>>>
>>>>   ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0 
>>>> "priority=100,in_port=vnet0,actions=encap(mpls(ether_type=0x8847)),set_mpls_label:100,encap(ethernet),,set_field:00:00:00:00:00:02->dl_dst,set_field:00:00:00:00:00:01->dl_src,output:enp5s0f0
>>>>
>
>>>
>>> The packets are not going out because you are sending the packet on 
>>> a
>>> real nic and not on a virtual inerface (veth pair) ?
>>
>> So for a real NIC we need to set the MAC addresses, maybe some where 
>> in the
>> documentation we should add an example on how to use this feature?
>>
>>>> Maybe the test case can be made more realistic? Once I understand 
>>>> the
>>>> failure, I can continue with the review.
>>>>
>>>>
>>>> Cheers,
>>>>
>>>> Eelco
>>>>
>>>>
>>>>
>>>>> Signed-off-by: Martin Varghese <martin.varghese@nokia.com>
>>>>> ---
>>>>>  NEWS                                          |  2 +-
>>>>>  .../linux/compat/include/linux/openvswitch.h  | 35 ++++++-
>>>>>  include/openvswitch/ofp-ed-props.h            | 18 ++++
>>>>>  lib/dpif-netdev.c                             |  1 +
>>>>>  lib/dpif.c                                    |  1 +
>>>>>  lib/odp-execute.c                             | 12 +++
>>>>>  lib/odp-util.c                                | 58 +++++++++---
>>>>>  lib/ofp-actions.c                             |  5 +
>>>>>  lib/ofp-ed-props.c                            | 91
>>>>> +++++++++++++++++++
>>>>>  lib/ovs-actions.xml                           | 31 +++++--
>>>>>  lib/packets.c                                 | 36 ++++++++
>>>>>  lib/packets.h                                 |  2 +
>>>>>  ofproto/ofproto-dpif-ipfix.c                  |  1 +
>>>>>  ofproto/ofproto-dpif-sflow.c                  |  1 +
>>>>>  ofproto/ofproto-dpif-xlate.c                  | 60 ++++++++++++
>>>>>  ofproto/ofproto-dpif.c                        | 39 ++++++++
>>>>>  ofproto/ofproto-dpif.h                        |  5 +-
>>>>>  tests/system-traffic.at                       | 36 ++++++++
>>>>>  18 files changed, 410 insertions(+), 24 deletions(-)
>>>>>
>>>>> diff --git a/NEWS b/NEWS
>>>>> index 95cf922aa..4bf4e9e7b 100644
>>>>> --- a/NEWS
>>>>> +++ b/NEWS
>>>>> @@ -120,7 +120,7 @@ v2.14.0 - 17 Aug 2020
>>>>>     - GTP-U Tunnel Protocol
>>>>>       * Add two new fields: tun_gtpu_flags, tun_gtpu_msgtype.
>>>>>       * Only support for userspace datapath.
>>>>> -
>>>>> +   - Encap & Decap action support for MPLS packet type.
>>>>>
>>>>>  v2.13.0 - 14 Feb 2020
>>>>>  ---------------------
>>>>> diff --git a/datapath/linux/compat/include/linux/openvswitch.h
>>>>> b/datapath/linux/compat/include/linux/openvswitch.h
>>>>> index 875de2025..8feea7dd4 100644
>>>>> --- a/datapath/linux/compat/include/linux/openvswitch.h
>>>>> +++ b/datapath/linux/compat/include/linux/openvswitch.h
>>>>> @@ -810,8 +810,32 @@ struct ovs_action_push_tnl {
>>>>>  };
>>>>>  #endif
>>>>>
>>>>> -/**
>>>>> - * enum ovs_ct_attr - Attributes for %OVS_ACTION_ATTR_CT action.
>>>>> +/* struct ovs_action_add_mpls - %OVS_ACTION_ATTR_ADD_MPLS action
>>>>> + * argument.
>>>>> + * @mpls_lse: MPLS label stack entry to push.
>>>>> + * @mpls_ethertype: Ethertype to set in the encapsulating 
>>>>> ethernet
>>>>> frame.
>>>>> + * @tun_flags: MPLS tunnel attributes.
>>>>> + *
>>>>> + * The only values @mpls_ethertype should ever be given are
>>>>> %ETH_P_MPLS_UC and
>>>>> + * %ETH_P_MPLS_MC, indicating MPLS unicast or multicast. Other 
>>>>> are
>>>>> rejected.
>>>>> + */
>>>>> +struct ovs_action_add_mpls {
>>>>> +	__be32 mpls_lse;
>>>>> +	__be16 mpls_ethertype; /* Either %ETH_P_MPLS_UC or
>>>>> %ETH_P_MPLS_MC */
>>>>> +	__u16 tun_flags;
>>>>> +};
>>>>> +
>>>>> +#define OVS_MPLS_L3_TUNNEL_FLAG_MASK  (1 << 0) /* Flag to
>>>>> specify the
>>>>> place of
>>>>> +						* insertion of MPLS header.
>>>>> +						* When false, the MPLS header
>>>>> +						* will be inserted at the start
>>>>> +						* of the packet.
>>>>> +						* When true, the MPLS header
>>>>> +						* will be inserted at the start
>>>>> +						* of the l3 header.
>>>>> +						*/
>>>>> +
>>>>> +/* enum ovs_ct_attr - Attributes for %OVS_ACTION_ATTR_CT action.
>>>>>   * @OVS_CT_ATTR_COMMIT: If present, commits the connection to the
>>>>> conntrack
>>>>>   * table. This allows future packets for the same connection to 
>>>>> be
>>>>> identified
>>>>>   * as 'established' or 'related'. The flow key for the current
>>>>> packet
>>>>> will
>>>>> @@ -1001,7 +1025,11 @@ struct check_pkt_len_arg {
>>>>>   * @OVS_ACTION_ATTR_CHECK_PKT_LEN: Check the packet length and
>>>>> execute
>>>>> a set
>>>>>   * of actions if greater than the specified packet length, else
>>>>> execute
>>>>>   * another set of actions.
>>>>> - * @OVS_ACTION_ATTR_DROP: Explicit drop action.
>>>>> + * @OVS_ACTION_ATTR_ADD_MPLS: Push a new MPLS label stack entry
>>>>> at the
>>>>> + * start of the packet or at the start of the l3 header
>>>>> depending on
>>>>> the value
>>>>> + * of l3 tunnel flag in the tun_flags field of
>>>>> OVS_ACTION_ATTR_ADD_MPLS
>>>>> + * argument.
>>>>> +  * @OVS_ACTION_ATTR_DROP: Explicit drop action.
>>>>>   */
>>>>>
>>>>>  enum ovs_action_attr {
>>>>> @@ -1030,6 +1058,7 @@ enum ovs_action_attr {
>>>>>  	OVS_ACTION_ATTR_METER,        /* u32 meter number. */
>>>>>  	OVS_ACTION_ATTR_CLONE,        /* Nested OVS_CLONE_ATTR_*.  */
>>>>>  	OVS_ACTION_ATTR_CHECK_PKT_LEN, /* Nested
>>>>> OVS_CHECK_PKT_LEN_ATTR_*. */
>>>>> +	OVS_ACTION_ATTR_ADD_MPLS,     /* struct ovs_action_add_mpls. */
>>>>>
>>>>>  #ifndef __KERNEL__
>>>>>  	OVS_ACTION_ATTR_TUNNEL_PUSH,   /* struct ovs_action_push_tnl*/
>>>>> diff --git a/include/openvswitch/ofp-ed-props.h
>>>>> b/include/openvswitch/ofp-ed-props.h
>>>>> index 306c6fe73..c85f3c283 100644
>>>>> --- a/include/openvswitch/ofp-ed-props.h
>>>>> +++ b/include/openvswitch/ofp-ed-props.h
>>>>> @@ -46,6 +46,11 @@ enum ofp_ed_nsh_prop_type {
>>>>>      OFPPPT_PROP_NSH_TLV = 2,     /* property TLV in NSH */
>>>>>  };
>>>>>
>>>>> +enum ofp_ed_mpls_prop_type {
>>>>> +    OFPPPT_PROP_MPLS_NONE = 0,    /* unused */
>>>>> +    OFPPPT_PROP_MPLS_ETHERTYPE = 1,  /* MPLS Ethertype */
>>>>> +};
>>>>> +
>>>>>  /*
>>>>>   * External representation of encap/decap properties.
>>>>>   * These must be padded to a multiple of 8 bytes.
>>>>> @@ -72,6 +77,13 @@ struct ofp_ed_prop_nsh_tlv {
>>>>>      uint8_t data[0];
>>>>>  };
>>>>>
>>>>> +struct ofp_ed_prop_mpls_ethertype {
>>>>> +    struct ofp_ed_prop_header header;
>>>>> +    uint16_t ether_type;         /* MPLS ethertype .*/
>>>>> +    uint8_t pad[2];          /* Padding to 8 bytes. */
>>>>> +};
>>>>> +
>>>>> +
>>>>>  /*
>>>>>   * Internal representation of encap/decap properties
>>>>>   */
>>>>> @@ -96,6 +108,12 @@ struct ofpact_ed_prop_nsh_tlv {
>>>>>      /* tlv_len octets of metadata value, padded to a multiple of 
>>>>> 8
>>>>> bytes. */
>>>>>      uint8_t data[0];
>>>>>  };
>>>>> +
>>>>> +struct ofpact_ed_prop_mpls_ethertype {
>>>>> +    struct ofpact_ed_prop header;
>>>>> +    uint16_t ether_type;         /* MPLS ethertype .*/
>>>>> +    uint8_t pad[2];          /* Padding to 8 bytes. */
>>>>> +};
>>>>>  enum ofperr decode_ed_prop(const struct ofp_ed_prop_header
>>>>> **ofp_prop,
>>>>>                             struct ofpbuf *out, size_t 
>>>>> *remaining);
>>>>>  enum ofperr encode_ed_prop(const struct ofpact_ed_prop **prop,
>>>>> diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
>>>>> index 94cc9b80c..bbdea5603 100644
>>>>> --- a/lib/dpif-netdev.c
>>>>> +++ b/lib/dpif-netdev.c
>>>>> @@ -8044,6 +8044,7 @@ dp_execute_cb(void *aux_, struct
>>>>> dp_packet_batch
>>>>> *packets_,
>>>>>      case OVS_ACTION_ATTR_CT_CLEAR:
>>>>>      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
>>>>>      case OVS_ACTION_ATTR_DROP:
>>>>> +    case OVS_ACTION_ATTR_ADD_MPLS:
>>>>>      case __OVS_ACTION_ATTR_MAX:
>>>>>          OVS_NOT_REACHED();
>>>>>      }
>>>>> diff --git a/lib/dpif.c b/lib/dpif.c
>>>>> index 56d0b4a65..bbd1296e3 100644
>>>>> --- a/lib/dpif.c
>>>>> +++ b/lib/dpif.c
>>>>> @@ -1273,6 +1273,7 @@ dpif_execute_helper_cb(void *aux_, struct
>>>>> dp_packet_batch *packets_,
>>>>>      case OVS_ACTION_ATTR_UNSPEC:
>>>>>      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
>>>>>      case OVS_ACTION_ATTR_DROP:
>>>>> +    case OVS_ACTION_ATTR_ADD_MPLS:
>>>>>      case __OVS_ACTION_ATTR_MAX:
>>>>>          OVS_NOT_REACHED();
>>>>>      }
>>>>> diff --git a/lib/odp-execute.c b/lib/odp-execute.c
>>>>> index 6eeda2a61..2f4cdd92c 100644
>>>>> --- a/lib/odp-execute.c
>>>>> +++ b/lib/odp-execute.c
>>>>> @@ -819,6 +819,7 @@ requires_datapath_assistance(const struct
>>>>> nlattr *a)
>>>>>      case OVS_ACTION_ATTR_POP_NSH:
>>>>>      case OVS_ACTION_ATTR_CT_CLEAR:
>>>>>      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
>>>>> +    case OVS_ACTION_ATTR_ADD_MPLS:
>>>>>      case OVS_ACTION_ATTR_DROP:
>>>>>          return false;
>>>>>
>>>>> @@ -1061,6 +1062,17 @@ odp_execute_actions(void *dp, struct
>>>>> dp_packet_batch *batch, bool steal,
>>>>>              }
>>>>>              break;
>>>>>
>>>>> +        case OVS_ACTION_ATTR_ADD_MPLS: {
>>>>> +            const struct ovs_action_add_mpls *mpls =
>>>>> nl_attr_get(a);
>>>>> +            bool l3_flag =  mpls->tun_flags &
>>>>> OVS_MPLS_L3_TUNNEL_FLAG_MASK;
>>>>> +
>>>>> +            DP_PACKET_BATCH_FOR_EACH (i, packet, batch) {
>>>>> +                add_mpls(packet, mpls->mpls_ethertype,
>>>>> mpls->mpls_lse,
>>>>> +                         l3_flag);
>>>>> +            }
>>>>> +            break;
>>>>> +        }
>>>>> +
>>>>>          case OVS_ACTION_ATTR_DROP:{
>>>>>              const enum xlate_error *drop_reason = nl_attr_get(a);
>>>>>
>>>>> diff --git a/lib/odp-util.c b/lib/odp-util.c
>>>>> index a8598d52a..f24e16d08 100644
>>>>> --- a/lib/odp-util.c
>>>>> +++ b/lib/odp-util.c
>>>>> @@ -142,6 +142,8 @@ odp_action_len(uint16_t type)
>>>>>      case OVS_ACTION_ATTR_PUSH_NSH: return ATTR_LEN_VARIABLE;
>>>>>      case OVS_ACTION_ATTR_POP_NSH: return 0;
>>>>>      case OVS_ACTION_ATTR_CHECK_PKT_LEN: return ATTR_LEN_VARIABLE;
>>>>> +    case OVS_ACTION_ATTR_ADD_MPLS:
>>>>> +         return sizeof(struct ovs_action_add_mpls);
>>>>>      case OVS_ACTION_ATTR_DROP: return sizeof(uint32_t);
>>>>>
>>>>>      case OVS_ACTION_ATTR_UNSPEC:
>>>>> @@ -1254,6 +1256,14 @@ format_odp_action(struct ds *ds, const 
>>>>> struct
>>>>> nlattr *a,
>>>>>      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
>>>>>          format_odp_check_pkt_len_action(ds, a, portno_names);
>>>>>          break;
>>>>> +    case OVS_ACTION_ATTR_ADD_MPLS: {
>>>>> +        const struct ovs_action_push_mpls *mpls = nl_attr_get(a);
>>>>> +        ds_put_cstr(ds, "add_mpls(");
>>>>> +        format_mpls_lse(ds, mpls->mpls_lse);
>>>>> +        ds_put_format(ds, ",eth_type=0x%"PRIx16")",
>>>>> +                      ntohs(mpls->mpls_ethertype));
>>>>> +        break;
>>>>> +    }
>>>>>      case OVS_ACTION_ATTR_DROP:
>>>>>          ds_put_cstr(ds, "drop");
>>>>>          break;
>>>>> @@ -7876,7 +7886,8 @@ commit_vlan_action(const struct flow*
>>>>> flow, struct
>>>>> flow *base,
>>>>>  /* Wildcarding already done at action translation time. */
>>>>>  static void
>>>>>  commit_mpls_action(const struct flow *flow, struct flow *base,
>>>>> -                   struct ofpbuf *odp_actions)
>>>>> +                   struct ofpbuf *odp_actions, bool 
>>>>> pending_encap,
>>>>> +                   bool pending_decap)
>>>>>  {
>>>>>      int base_n = flow_count_mpls_labels(base, NULL);
>>>>>      int flow_n = flow_count_mpls_labels(flow, NULL);
>>>>> @@ -7913,7 +7924,11 @@ commit_mpls_action(const struct flow *flow,
>>>>> struct flow *base,
>>>>>              if ((!eth_type_mpls(flow->dl_type)) && base_n > 1) {
>>>>>                  dl_type = htons(ETH_TYPE_MPLS);
>>>>>              } else {
>>>>> -                dl_type = flow->dl_type;
>>>>> +                if ((flow->packet_type == PT_ETH) &&
>>>>> pending_decap) {
>>>>> +                    dl_type =  htons(ETH_TYPE_TEB);
>>>>> +                } else {
>>>>> +                    dl_type = flow->dl_type;
>>>>> +                }
>>>>>              }
>>>>>              nl_msg_put_be16(odp_actions, 
>>>>> OVS_ACTION_ATTR_POP_MPLS,
>>>>> dl_type);
>>>>>              ovs_assert(flow_pop_mpls(base, base_n, flow->dl_type,
>>>>> NULL));
>>>>> @@ -7924,18 +7939,29 @@ commit_mpls_action(const struct flow 
>>>>> *flow,
>>>>> struct flow *base,
>>>>>      /* If, after the above popping and setting, there are more
>>>>> LSEs in
>>>>> flow
>>>>>       * than base then some LSEs need to be pushed. */
>>>>>      while (base_n < flow_n) {
>>>>> -        struct ovs_action_push_mpls *mpls;
>>>>>
>>>>> -        mpls = nl_msg_put_unspec_zero(odp_actions,
>>>>> -                                      OVS_ACTION_ATTR_PUSH_MPLS,
>>>>> -                                      sizeof *mpls);
>>>>> -        mpls->mpls_ethertype = flow->dl_type;
>>>>> -        mpls->mpls_lse = flow->mpls_lse[flow_n - base_n - 1];
>>>>> +        if (pending_encap) {
>>>>> +             struct ovs_action_add_mpls *mpls;
>>>>> +
>>>>> +             mpls = nl_msg_put_unspec_zero(odp_actions,
>>>>> +
>>>>> OVS_ACTION_ATTR_ADD_MPLS,
>>>>> +                                           sizeof *mpls);
>>>>> +             mpls->mpls_ethertype = flow->dl_type;
>>>>> +             mpls->mpls_lse = flow->mpls_lse[flow_n - base_n - 
>>>>> 1];
>>>>> +        } else {
>>>>> +             struct ovs_action_push_mpls *mpls;
>>>>> +
>>>>> +             mpls = nl_msg_put_unspec_zero(odp_actions,
>>>>> +
>>>>> OVS_ACTION_ATTR_PUSH_MPLS,
>>>>> +                                           sizeof *mpls);
>>>>> +             mpls->mpls_ethertype = flow->dl_type;
>>>>> +             mpls->mpls_lse = flow->mpls_lse[flow_n - base_n - 
>>>>> 1];
>>>>> +        }
>>>>>          /* Update base flow's MPLS stack, but do not clear L3.
>>>>> We need
>>>>> the L3
>>>>>           * headers if the flow is restored later due to
>>>>> returning from
>>>>> a patch
>>>>>           * port or group bucket. */
>>>>> -        flow_push_mpls(base, base_n, mpls->mpls_ethertype, NULL,
>>>>> false);
>>>>> -        flow_set_mpls_lse(base, 0, mpls->mpls_lse);
>>>>> +        flow_push_mpls(base, base_n, flow->dl_type, NULL, false);
>>>>> +        flow_set_mpls_lse(base, 0, flow->mpls_lse[flow_n - base_n 
>>>>> -
>>>>> 1]);
>>>>>          base_n++;
>>>>>      }
>>>>>  }
>>>>> @@ -8586,6 +8612,10 @@ commit_encap_decap_action(const struct flow
>>>>> *flow,
>>>>>              memcpy(&base_flow->dl_dst, &flow->dl_dst,
>>>>>                     sizeof(*flow) - offsetof(struct flow, 
>>>>> dl_dst));
>>>>>              break;
>>>>> +        case PT_MPLS:
>>>>> +            commit_mpls_action(flow, base_flow, odp_actions,
>>>>> pending_encap,
>>>>> +                               pending_decap);
>>>>> +            break;
>>>>>          default:
>>>>>              /* Only the above protocols are supported for encap.
>>>>>               * The check is done at action translation. */
>>>>> @@ -8608,6 +8638,10 @@ commit_encap_decap_action(const struct flow
>>>>> *flow,
>>>>>                  /* pop_nsh. */
>>>>>                  odp_put_pop_nsh_action(odp_actions);
>>>>>                  break;
>>>>> +            case PT_MPLS:
>>>>> +                commit_mpls_action(flow, base_flow, odp_actions,
>>>>> pending_encap,
>>>>> +                                   pending_decap);
>>>>> +                break;
>>>>>              default:
>>>>>                  /* Checks are done during translation. */
>>>>>                  OVS_NOT_REACHED();
>>>>> @@ -8653,7 +8687,7 @@ commit_odp_actions(const struct flow
>>>>> *flow, struct
>>>>> flow *base,
>>>>>      /* Make packet a non-MPLS packet before committing L3/4
>>>>> actions,
>>>>>       * which would otherwise do nothing. */
>>>>>      if (eth_type_mpls(base->dl_type) &&
>>>>> !eth_type_mpls(flow->dl_type))
>>>>> {
>>>>> -        commit_mpls_action(flow, base, odp_actions);
>>>>> +        commit_mpls_action(flow, base, odp_actions, false, 
>>>>> false);
>>>>>          mpls_done = true;
>>>>>      }
>>>>>      commit_set_nsh_action(flow, base, odp_actions, wc, 
>>>>> use_masked);
>>>>> @@ -8661,7 +8695,7 @@ commit_odp_actions(const struct flow
>>>>> *flow, struct
>>>>> flow *base,
>>>>>      commit_set_port_action(flow, base, odp_actions, wc,
>>>>> use_masked);
>>>>>      slow2 = commit_set_icmp_action(flow, base, odp_actions, wc);
>>>>>      if (!mpls_done) {
>>>>> -        commit_mpls_action(flow, base, odp_actions);
>>>>> +        commit_mpls_action(flow, base, odp_actions, false, 
>>>>> false);
>>>>>      }
>>>>>      commit_vlan_action(flow, base, odp_actions, wc);
>>>>>      commit_set_priority_action(flow, base, odp_actions, wc,
>>>>> use_masked);
>>>>> diff --git a/lib/ofp-actions.c b/lib/ofp-actions.c
>>>>> index 0342a228b..28a12a569 100644
>>>>> --- a/lib/ofp-actions.c
>>>>> +++ b/lib/ofp-actions.c
>>>>> @@ -4441,6 +4441,7 @@ decode_NXAST_RAW_ENCAP(const struct
>>>>> nx_action_encap *nae,
>>>>>      switch (ntohl(nae->new_pkt_type)) {
>>>>>      case PT_ETH:
>>>>>      case PT_NSH:
>>>>> +    case PT_MPLS:
>>>>>          /* Add supported encap header types here. */
>>>>>          break;
>>>>>      default:
>>>>> @@ -4492,6 +4493,8 @@ parse_encap_header(const char *hdr, ovs_be32
>>>>> *packet_type)
>>>>>          *packet_type = htonl(PT_ETH);
>>>>>      } else if (strcmp(hdr, "nsh") == 0) {
>>>>>          *packet_type = htonl(PT_NSH);
>>>>> +    } else if (strcmp(hdr, "mpls") == 0) {
>>>>> +        *packet_type = htonl(PT_MPLS);
>>>>>      } else {
>>>>>          return false;
>>>>>      }
>>>>> @@ -4573,6 +4576,8 @@ format_encap_pkt_type(const ovs_be32 
>>>>> pkt_type)
>>>>>          return "ethernet";
>>>>>      case PT_NSH:
>>>>>          return "nsh";
>>>>> +    case PT_MPLS:
>>>>> +        return "mpls";
>>>>>      default:
>>>>>          return "UNKNOWN";
>>>>>      }
>>>>> diff --git a/lib/ofp-ed-props.c b/lib/ofp-ed-props.c
>>>>> index 02a9235d5..fc261e4c6 100644
>>>>> --- a/lib/ofp-ed-props.c
>>>>> +++ b/lib/ofp-ed-props.c
>>>>> @@ -79,6 +79,27 @@ decode_ed_prop(const struct ofp_ed_prop_header
>>>>> **ofp_prop,
>>>>>          }
>>>>>          break;
>>>>>      }
>>>>> +    case OFPPPC_MPLS: {
>>>>> +       switch (prop_type) {
>>>>> +        case OFPPPT_PROP_MPLS_ETHERTYPE: {
>>>>> +            struct ofp_ed_prop_mpls_ethertype *opnmt =
>>>>> +                ALIGNED_CAST(struct ofp_ed_prop_mpls_ethertype *,
>>>>> *ofp_prop);
>>>>> +            if (len > sizeof(*opnmt) || len > *remaining) {
>>>>> +                return OFPERR_NXBAC_BAD_ED_PROP;
>>>>> +            }
>>>>> +            struct ofpact_ed_prop_mpls_ethertype *pnmt =
>>>>> +                    ofpbuf_put_uninit(out, sizeof(*pnmt));
>>>>> +            pnmt->header.prop_class = prop_class;
>>>>> +            pnmt->header.type = prop_type;
>>>>> +            pnmt->header.len = len;
>>>>> +            pnmt->ether_type = opnmt->ether_type;
>>>>> +            break;
>>>>> +        }
>>>>> +        default:
>>>>> +            return OFPERR_NXBAC_UNKNOWN_ED_PROP;
>>>>> +        }
>>>>> +        break;
>>>>> +    }
>>>>>      default:
>>>>>          return OFPERR_NXBAC_UNKNOWN_ED_PROP;
>>>>>      }
>>>>> @@ -134,6 +155,27 @@ encode_ed_prop(const struct ofpact_ed_prop
>>>>> **prop,
>>>>>          }
>>>>>          break;
>>>>>      }
>>>>> +    case OFPPPC_MPLS: {
>>>>> +       switch ((*prop)->type) {
>>>>> +       case OFPPPT_PROP_MPLS_ETHERTYPE: {
>>>>> +           struct ofpact_ed_prop_mpls_ethertype *pnmt =
>>>>> +                ALIGNED_CAST(struct
>>>>> ofpact_ed_prop_mpls_ethertype *,
>>>>> *prop);
>>>>> +            struct ofp_ed_prop_mpls_ethertype *opnmt =
>>>>> +                    ofpbuf_put_uninit(out, sizeof(*opnmt));
>>>>> +            opnmt->header.prop_class = 
>>>>> htons((*prop)->prop_class);
>>>>> +            opnmt->header.type = (*prop)->type;
>>>>> +            opnmt->header.len =
>>>>> +                    offsetof(struct 
>>>>> ofpact_ed_prop_mpls_ethertype,
>>>>> pad);
>>>>> +            opnmt->ether_type = pnmt->ether_type;
>>>>> +            prop_len = sizeof(*pnmt);
>>>>> +            break;
>>>>> +
>>>>> +       }
>>>>> +       default:
>>>>> +            return OFPERR_OFPBAC_BAD_ARGUMENT;
>>>>> +       }
>>>>> +       break;
>>>>> +    }
>>>>>      default:
>>>>>          return OFPERR_OFPBAC_BAD_ARGUMENT;
>>>>>      }
>>>>> @@ -181,6 +223,13 @@ parse_ed_prop_type(uint16_t prop_class,
>>>>>          } else {
>>>>>              return false;
>>>>>          }
>>>>> +    case OFPPPC_MPLS:
>>>>> +        if (!strcmp(str, "ether_type")) {
>>>>> +            *type = OFPPPT_PROP_MPLS_ETHERTYPE;
>>>>> +            return true;
>>>>> +        } else {
>>>>> +            return false;
>>>>> +        }
>>>>>      default:
>>>>>          return false;
>>>>>      }
>>>>> @@ -259,6 +308,28 @@ parse_ed_prop_value(uint16_t prop_class,
>>>>> uint8_t
>>>>> prop_type OVS_UNUSED,
>>>>>              OVS_NOT_REACHED();
>>>>>          }
>>>>>          break;
>>>>> +    case OFPPPC_MPLS:
>>>>> +        switch (prop_type) {
>>>>> +        case OFPPPT_PROP_MPLS_ETHERTYPE: {
>>>>> +            uint16_t ethertype;
>>>>> +            error = str_to_u16(value, "ether_type", &ethertype);
>>>>> +            if (error != NULL) {
>>>>> +                return error;
>>>>> +            }
>>>>> +            struct ofpact_ed_prop_mpls_ethertype *pnmt =
>>>>> +                    ofpbuf_put_uninit(out, sizeof(*pnmt));
>>>>> +            pnmt->header.prop_class = prop_class;
>>>>> +            pnmt->header.type = prop_type;
>>>>> +            pnmt->header.len =
>>>>> +                    offsetof(struct 
>>>>> ofpact_ed_prop_mpls_ethertype,
>>>>> pad);
>>>>> +            pnmt->ether_type = ethertype;
>>>>> +
>>>>> +            break;
>>>>> +        }
>>>>> +        default:
>>>>> +            break;
>>>>> +      }
>>>>> +      break;
>>>>>      default:
>>>>>          /* Unsupported property classes rejected before. */
>>>>>          OVS_NOT_REACHED();
>>>>> @@ -300,6 +371,14 @@ format_ed_prop_type(const struct 
>>>>> ofpact_ed_prop
>>>>> *prop)
>>>>>              OVS_NOT_REACHED();
>>>>>          }
>>>>>          break;
>>>>> +    case OFPPPC_MPLS:
>>>>> +         switch (prop->type) {
>>>>> +         case OFPPPT_PROP_MPLS_ETHERTYPE:
>>>>> +              return "ether_type";
>>>>> +         default:
>>>>> +               OVS_NOT_REACHED();
>>>>> +         }
>>>>> +         break;
>>>>>      default:
>>>>>          OVS_NOT_REACHED();
>>>>>      }
>>>>> @@ -332,6 +411,18 @@ format_ed_prop(struct ds *s OVS_UNUSED,
>>>>>          default:
>>>>>              OVS_NOT_REACHED();
>>>>>          }
>>>>> +     case OFPPPC_MPLS:
>>>>> +        switch (prop->type) {
>>>>> +        case OFPPPT_PROP_MPLS_ETHERTYPE: {
>>>>> +          struct ofpact_ed_prop_mpls_ethertype *pnmt =
>>>>> +                ALIGNED_CAST(struct
>>>>> ofpact_ed_prop_mpls_ethertype *,
>>>>> prop);
>>>>> +            ds_put_format(s, "%s=%d", format_ed_prop_type(prop),
>>>>> +                          pnmt->ether_type);
>>>>> +            return;
>>>>> +        }
>>>>> +        default:
>>>>> +            OVS_NOT_REACHED();
>>>>> +        }
>>>>>      default:
>>>>>          OVS_NOT_REACHED();
>>>>>      }
>>>>> diff --git a/lib/ovs-actions.xml b/lib/ovs-actions.xml
>>>>> index a2778de4b..e97f818d9 100644
>>>>> --- a/lib/ovs-actions.xml
>>>>> +++ b/lib/ovs-actions.xml
>>>>> @@ -265,13 +265,13 @@
>>>>>        </p>
>>>>>
>>>>>        <p>
>>>>> -        When a <code>decap</code> action decapsulates a packet,
>>>>> Open
>>>>> vSwitch
>>>>> -        raises this error if it does not support the type of 
>>>>> inner
>>>>> packet.
>>>>> -        <code>decap</code> of an Ethernet header raises this
>>>>> error if a
>>>>> VLAN
>>>>> -        header is present, <code>decap</code> of a NSH packet
>>>>> raises
>>>>> this error
>>>>> -        if the NSH inner packet is not Ethernet, IPv4, IPv6, or
>>>>> NSH,
>>>>> and
>>>>> -        <code>decap</code> of other types of packets is
>>>>> unsupported and
>>>>> also
>>>>> -        raises this error.
>>>>> +        The <code>decap</code> action is supported only for 
>>>>> packet
>>>>> types
>>>>> +        ethernet, NSH and MPLS. Openvswitch raises this error
>>>>> for other
>>>>> +        packet types. When a <code>decap</code> action
>>>>> decapsulates a
>>>>> packet,
>>>>> +        Open vSwitch raises this error if it does not support
>>>>> the type
>>>>> of inner
>>>>> +        packet. <code>decap</code> of an Ethernet header raises
>>>>> this
>>>>> error if a
>>>>> +        VLAN header is present, <code>decap</code> of a NSH 
>>>>> packet
>>>>> raises this
>>>>> +        error if the NSH inner packet is not Ethernet, IPv4,
>>>>> IPv6, or
>>>>> NSH.
>>>>>        </p>
>>>>>
>>>>>        <p>
>>>>> @@ -1097,6 +1097,8 @@ for <var>i</var> in 
>>>>> [1,<var>n_members</var>]:
>>>>>        <h2>The <code>encap</code> action</h2>
>>>>>        <syntax><code>encap(nsh(</code>[<code>md_type=<var>md_type</var></code>]<code>, 
>>>>> </code>[<code>tlv(<var>class</var>,<var>type</var>,<var>value</var>)</code>]...<code>))</code></syntax>
>>>>>        <syntax><code>encap(ethernet)</code></syntax>
>>>>> +
>>>>> <syntax><code>encap(mpls(ether_type=<var>ether_type</var>))</code>
>>>>> +      </syntax>
>>>>>
>>>>>        <p>
>>>>>          The <code>encap</code> action encapsulates a packet with 
>>>>> a
>>>>> specified
>>>>> @@ -1135,6 +1137,12 @@ for <var>i</var> in 
>>>>> [1,<var>n_members</var>]:
>>>>>          source and destination are initially zeroed.
>>>>>        </p>
>>>>>
>>>>> +      <p>
>>>>> +        The <code>encap(mpls(ethertype=....))</code> variant
>>>>> encapsulates an
>>>>> +        ethernet or L3 packet with a MPLS header. The
>>>>> <var>ethertype</var>
>>>>> +        could be MPLS unicast (0x8847) or multicast (0x8848)
>>>>> ethertypes.
>>>>> +      </p>
>>>>> +
>>>>>        <conformance>
>>>>>          This action is an Open vSwitch extension to OpenFlow
>>>>> 1.3 and
>>>>> later,
>>>>>          introduced in Open vSwitch 2.8.
>>>>> @@ -1144,6 +1152,9 @@ for <var>i</var> in 
>>>>> [1,<var>n_members</var>]:
>>>>>      <action name="DECAP">
>>>>>        <h2>The <code>decap</code> action</h2>
>>>>>        <syntax><code>decap</code></syntax>
>>>>> +      <syntax><code>decap(packet_type(ns=<var>name_space</var>,
>>>>> +      type=<var>ethertype</var>))</code></syntax> for 
>>>>> decapsulating
>>>>> MPLS
>>>>> +      packets.
>>>>>
>>>>>        <p>
>>>>>          Removes an outermost encapsulation from the packet:
>>>>> @@ -1164,6 +1175,12 @@ for <var>i</var> in 
>>>>> [1,<var>n_members</var>]:
>>>>>            packet type errors.
>>>>>          </li>
>>>>>
>>>>> +        <li>
>>>>> +          Otherwise, if the packet is a MPLS packet, removes
>>>>> the MPLS
>>>>> header
>>>>> +          and classifies the inner packet as mentioned in the
>>>>> packet
>>>>> type
>>>>> +          argument of the decap.
>>>>> +        </li>
>>>>> +
>>>>>          <li>
>>>>>            Otherwise, raises an unsupported packet type error.
>>>>>          </li>
>>>>> diff --git a/lib/packets.c b/lib/packets.c
>>>>> index 4a7643c5d..5e3c3900f 100644
>>>>> --- a/lib/packets.c
>>>>> +++ b/lib/packets.c
>>>>> @@ -418,6 +418,38 @@ push_mpls(struct dp_packet *packet, ovs_be16
>>>>> ethtype, ovs_be32 lse)
>>>>>      pkt_metadata_init_conn(&packet->md);
>>>>>  }
>>>>>
>>>>> +void
>>>>> +add_mpls(struct dp_packet *packet, ovs_be16 ethtype, ovs_be32
>>>>> lse, bool
>>>>> l3)
>>>>> +{
>>>>> +    char * header;
>>>>> +
>>>>> +    if (!eth_type_mpls(ethtype)) {
>>>>> +        return;
>>>>> +    }
>>>>> +
>>>>> +    if (!l3) {
>>>>> +        header =  dp_packet_push_uninit(packet, MPLS_HLEN);
>>>>> +        memcpy(header, &lse, sizeof lse);
>>>>> +        packet->l2_5_ofs = 0;
>>>>> +        packet->packet_type = htonl(PT_MPLS);
>>>>> +    } else {
>>>>> +        size_t len;
>>>>> +
>>>>> +        if (!is_mpls(packet)) {
>>>>> +            /* Set MPLS label stack offset. */
>>>>> +            packet->l2_5_ofs = packet->l3_ofs;
>>>>> +        }
>>>>> +        set_ethertype(packet, ethtype);
>>>>> +
>>>>> +        /* Push new MPLS shim header onto packet. */
>>>>> +        len = packet->l2_5_ofs;
>>>>> +        header = dp_packet_resize_l2_5(packet, MPLS_HLEN);
>>>>> +        memmove(header, header + MPLS_HLEN, len);
>>>>> +        memcpy(header + len, &lse, sizeof lse);
>>>>> +    }
>>>>> +    pkt_metadata_init_conn(&packet->md);
>>>>> +}
>>>>> +
>>>>>  /* If 'packet' is an MPLS packet, removes its outermost MPLS 
>>>>> label
>>>>> stack entry.
>>>>>   * If the label that was removed was the only MPLS label, changes
>>>>> 'packet''s
>>>>>   * Ethertype to 'ethtype' (which ordinarily should not be an MPLS
>>>>> @@ -429,6 +461,10 @@ pop_mpls(struct dp_packet *packet, ovs_be16
>>>>> ethtype)
>>>>>          struct mpls_hdr *mh = dp_packet_l2_5(packet);
>>>>>          size_t len = packet->l2_5_ofs;
>>>>>
>>>>> +        if (ethtype == htons(ETH_TYPE_TEB)) {
>>>>> +             packet->packet_type = htonl(PT_ETH);
>>>>> +        }
>>>>> +
>>>>>          set_ethertype(packet, ethtype);
>>>>>          if (get_16aligned_be32(&mh->mpls_lse) &
>>>>> htonl(MPLS_BOS_MASK)) {
>>>>>              dp_packet_set_l2_5(packet, NULL);
>>>>> diff --git a/lib/packets.h b/lib/packets.h
>>>>> index 481bc22fa..3f5862e08 100644
>>>>> --- a/lib/packets.h
>>>>> +++ b/lib/packets.h
>>>>> @@ -350,6 +350,8 @@ void set_mpls_lse_label(ovs_be32 *lse, 
>>>>> ovs_be32
>>>>> label);
>>>>>  void set_mpls_lse_bos(ovs_be32 *lse, uint8_t bos);
>>>>>  ovs_be32 set_mpls_lse_values(uint8_t ttl, uint8_t tc, uint8_t 
>>>>> bos,
>>>>>                               ovs_be32 label);
>>>>> +void add_mpls(struct dp_packet *packet, ovs_be16 ethtype,
>>>>> ovs_be32 lse,
>>>>> +              bool l3_flag);
>>>>>
>>>>>  /* Example:
>>>>>   *
>>>>> diff --git a/ofproto/ofproto-dpif-ipfix.c
>>>>> b/ofproto/ofproto-dpif-ipfix.c
>>>>> index 796eb6f88..9280e008e 100644
>>>>> --- a/ofproto/ofproto-dpif-ipfix.c
>>>>> +++ b/ofproto/ofproto-dpif-ipfix.c
>>>>> @@ -3018,6 +3018,7 @@ dpif_ipfix_read_actions(const struct flow
>>>>> *flow,
>>>>>          case OVS_ACTION_ATTR_CHECK_PKT_LEN:
>>>>>          case OVS_ACTION_ATTR_UNSPEC:
>>>>>          case OVS_ACTION_ATTR_DROP:
>>>>> +        case OVS_ACTION_ATTR_ADD_MPLS:
>>>>>          case __OVS_ACTION_ATTR_MAX:
>>>>>          default:
>>>>>              break;
>>>>> diff --git a/ofproto/ofproto-dpif-sflow.c
>>>>> b/ofproto/ofproto-dpif-sflow.c
>>>>> index fdcb9eabb..ca46a9bc4 100644
>>>>> --- a/ofproto/ofproto-dpif-sflow.c
>>>>> +++ b/ofproto/ofproto-dpif-sflow.c
>>>>> @@ -1226,6 +1226,7 @@ dpif_sflow_read_actions(const struct flow
>>>>> *flow,
>>>>>          case OVS_ACTION_ATTR_UNSPEC:
>>>>>          case OVS_ACTION_ATTR_CHECK_PKT_LEN:
>>>>>          case OVS_ACTION_ATTR_DROP:
>>>>> +        case OVS_ACTION_ATTR_ADD_MPLS:
>>>>>          case __OVS_ACTION_ATTR_MAX:
>>>>>          default:
>>>>>              break;
>>>>> diff --git a/ofproto/ofproto-dpif-xlate.c
>>>>> b/ofproto/ofproto-dpif-xlate.c
>>>>> index 7108c8a30..a97534233 100644
>>>>> --- a/ofproto/ofproto-dpif-xlate.c
>>>>> +++ b/ofproto/ofproto-dpif-xlate.c
>>>>> @@ -6381,6 +6381,45 @@ rewrite_flow_encap_ethernet(struct 
>>>>> xlate_ctx
>>>>> *ctx,
>>>>>          ctx->error = XLATE_UNSUPPORTED_PACKET_TYPE;
>>>>>      }
>>>>>  }
>>>>> +static void
>>>>> +rewrite_flow_encap_mpls(struct xlate_ctx *ctx,
>>>>> +                        const struct ofpact_encap *encap,
>>>>> +                        struct flow *flow,
>>>>> +                        struct flow_wildcards *wc)
>>>>> +{
>>>>> +    int n;
>>>>> +    uint32_t i;
>>>>> +    uint16_t ether_type;
>>>>> +    const char *ptr = (char *) encap->props;
>>>>> +
>>>>> +     for (i = 0; i < encap->n_props; i++) {
>>>>> +        struct ofpact_ed_prop *prop_ptr =
>>>>> +            ALIGNED_CAST(struct ofpact_ed_prop *, ptr);
>>>>> +        if (prop_ptr->prop_class == OFPPPC_MPLS) {
>>>>> +            switch (prop_ptr->type) {
>>>>> +                case OFPPPT_PROP_MPLS_ETHERTYPE: {
>>>>> +                     struct ofpact_ed_prop_mpls_ethertype
>>>>> *prop_ether_type =
>>>>> +                        ALIGNED_CAST(struct
>>>>> ofpact_ed_prop_mpls_ethertype *,
>>>>> +                                     prop_ptr);
>>>>> +                    ether_type = prop_ether_type->ether_type;
>>>>> +                    break;
>>>>> +                 }
>>>>> +            }
>>>>> +        }
>>>>> +     }
>>>>> +
>>>>> +    wc->masks.packet_type = OVS_BE32_MAX;
>>>>> +    if (flow->packet_type != htonl(PT_MPLS)) {
>>>>> +        memset(&ctx->wc->masks.mpls_lse, 0x0,
>>>>> +               sizeof *wc->masks.mpls_lse * 
>>>>> FLOW_MAX_MPLS_LABELS);
>>>>> +        memset(&flow->mpls_lse, 0x0, sizeof *flow->mpls_lse *
>>>>> +               FLOW_MAX_MPLS_LABELS);
>>>>> +    }
>>>>> +    flow->packet_type = htonl(PT_MPLS);
>>>>> +    n = flow_count_mpls_labels(flow, ctx->wc);
>>>>> +    flow_push_mpls(flow, n, htons(ether_type), ctx->wc, true);
>>>>> +}
>>>>> +
>>>>>
>>>>>  /* For an MD2 NSH header returns a pointer to an ofpbuf with the
>>>>> encoded
>>>>>   * MD2 TLVs provided as encap properties to the encap
>>>>> operation. This
>>>>> @@ -6513,6 +6552,12 @@ xlate_generic_encap_action(struct
>>>>> xlate_ctx *ctx,
>>>>>          case PT_NSH:
>>>>>              encap_data = rewrite_flow_push_nsh(ctx, encap,
>>>>> flow, wc);
>>>>>              break;
>>>>> +        case PT_MPLS:
>>>>> +            rewrite_flow_encap_mpls(ctx, encap,  flow, wc);
>>>>> +            if (!ctx->xbridge->support.add_mpls) {
>>>>> +                ctx->xout->slow |= SLOW_ACTION;
>>>>> +            }
>>>>> +            break;
>>>>>          default:
>>>>>              /* New packet type was checked during decoding. */
>>>>>              OVS_NOT_REACHED();
>>>>> @@ -6582,6 +6627,21 @@ xlate_generic_decap_action(struct
>>>>> xlate_ctx *ctx,
>>>>>              ctx->pending_decap = true;
>>>>>              /* Trigger recirculation. */
>>>>>              return true;
>>>>> +        case PT_MPLS: {
>>>>> +             int n;
>>>>> +             ovs_be16 ethertype;
>>>>> +
>>>>> +             flow->packet_type = decap->new_pkt_type;
>>>>> +             ethertype = pt_ns_type_be(flow->packet_type);
>>>>> +
>>>>> +             n = flow_count_mpls_labels(flow, ctx->wc);
>>>>> +             flow_pop_mpls(flow, n, ethertype, ctx->wc);
>>>>> +             if (!ctx->xbridge->support.add_mpls) {
>>>>> +                ctx->xout->slow |= SLOW_ACTION;
>>>>> +             }
>>>>> +             ctx->pending_decap = true;
>>>>> +             return true;
>>>>> +        }
>>>>>          default:
>>>>>              /* Error handling: drop packet. */
>>>>>              xlate_report_debug(
>>>>> diff --git a/ofproto/ofproto-dpif.c b/ofproto/ofproto-dpif.c
>>>>> index fd0b2fdea..d9a2922e7 100644
>>>>> --- a/ofproto/ofproto-dpif.c
>>>>> +++ b/ofproto/ofproto-dpif.c
>>>>> @@ -1520,6 +1520,44 @@ check_nd_extensions(struct dpif_backer
>>>>> *backer)
>>>>>
>>>>>      return !error;
>>>>>  }
>>>>> +/* Tests whether 'backer''s datapath supports the
>>>>> + * OVS_ACTION_ATTR_ADD_MPLS action. */
>>>>> +static bool
>>>>> +check_add_mpls(struct dpif_backer *backer)
>>>>> +{
>>>>> +    struct odputil_keybuf keybuf;
>>>>> +    struct ofpbuf actions;
>>>>> +    struct ofpbuf key;
>>>>> +    struct flow flow;
>>>>> +    bool supported;
>>>>> +
>>>>> +    struct odp_flow_key_parms odp_parms = {
>>>>> +        .flow = &flow,
>>>>> +        .probe = true,
>>>>> +    };
>>>>> +
>>>>> +    memset(&flow, 0, sizeof flow);
>>>>> +    ofpbuf_use_stack(&key, &keybuf, sizeof keybuf);
>>>>> +    odp_flow_key_from_flow(&odp_parms, &key);
>>>>> +    ofpbuf_init(&actions, 64);
>>>>> +
>>>>> +    struct ovs_action_add_mpls *mpls;
>>>>> +
>>>>> +    mpls = nl_msg_put_unspec_zero(&actions,
>>>>> +                                  OVS_ACTION_ATTR_ADD_MPLS,
>>>>> +                                  sizeof *mpls);
>>>>> +    mpls->mpls_ethertype = htons(ETH_TYPE_MPLS);
>>>>> +
>>>>> +    supported = dpif_probe_feature(backer->dpif, "add_mpls", 
>>>>> &key,
>>>>> +                                   &actions, NULL);
>>>>> +    ofpbuf_uninit(&actions);
>>>>> +    VLOG_INFO("%s: Datapath %s add_mpls action",
>>>>> +              dpif_name(backer->dpif), supported ? "supports"
>>>>> +                                                 : "does not
>>>>> support");
>>>>> +    return supported;
>>>>> +
>>>>> +}
>>>>> +
>>>>>
>>>>>  #define CHECK_FEATURE__(NAME, SUPPORT, FIELD, VALUE, ETHTYPE)
>>>>> \
>>>>>  static bool
>>>>> \
>>>>> @@ -1590,6 +1628,7 @@ check_support(struct dpif_backer *backer)
>>>>>          dpif_supports_explicit_drop_action(backer->dpif);
>>>>>      backer->rt_support.lb_output_action=
>>>>>          dpif_supports_lb_output_action(backer->dpif);
>>>>> +    backer->rt_support.add_mpls = check_add_mpls(backer);
>>>>>
>>>>>      /* Flow fields. */
>>>>>      backer->rt_support.odp.ct_state = check_ct_state(backer);
>>>>> diff --git a/ofproto/ofproto-dpif.h b/ofproto/ofproto-dpif.h
>>>>> index b41c3d82a..c04bfff8d 100644
>>>>> --- a/ofproto/ofproto-dpif.h
>>>>> +++ b/ofproto/ofproto-dpif.h
>>>>> @@ -204,7 +204,10 @@ struct group_dpif *group_dpif_lookup(struct
>>>>> ofproto_dpif *,
>>>>>      DPIF_SUPPORT_FIELD(bool, explicit_drop_action, "Explicit Drop
>>>>> action")  \
>>>>>                                                                              \
>>>>>      /* True if the datapath supports balance_tcp optimization */
>>>>> \
>>>>> -    DPIF_SUPPORT_FIELD(bool, lb_output_action, "Optimized
>>>>> Balance TCP
>>>>> mode")
>>>>> +    DPIF_SUPPORT_FIELD(bool, lb_output_action, "Optimized
>>>>> Balance TCP
>>>>> mode")\
>>>>> +
>>>>> \
>>>>> +    /* True if the datapath supports layer 2 MPLS tunnelling */
>>>>> \
>>>>> +    DPIF_SUPPORT_FIELD(bool, add_mpls, "l2 MPLS tunnelling")
>>>>>
>>>>>
>>>>>  /* Stores the various features which the corresponding backer
>>>>> supports.
>>>>> */
>>>>> diff --git a/tests/system-traffic.at b/tests/system-traffic.at
>>>>> index fb5b9a36d..b865b5210 100644
>>>>> --- a/tests/system-traffic.at
>>>>> +++ b/tests/system-traffic.at
>>>>> @@ -1027,9 +1027,45 @@ NS_CHECK_EXEC([at_ns0], [ping -q -c 3 -i
>>>>> 0.3 -w 2
>>>>> 10.1.1.2 | FORMAT_PING], [0],
>>>>>  3 packets transmitted, 3 received, 0% packet loss, time 0ms
>>>>>  ])
>>>>>
>>>>> +OVS_TRAFFIC_VSWITCHD_STOP
>>>>> +AT_CLEANUP
>>>>> +
>>>>> +
>>>>> +AT_SETUP([datapath - ptap mpls actions])
>>>>> +OVS_TRAFFIC_VSWITCHD_START([_ADD_BR([br1])])
>>>>> +
>>>>> +ADD_NAMESPACES(at_ns0, at_ns1)
>>>>> +
>>>>> +ADD_VETH(p0, at_ns0, br0, "10.1.1.1/24")
>>>>> +ADD_VETH(p1, at_ns1, br1, "10.1.1.2/24")
>>>>> +
>>>>> +AT_CHECK([ip link add patch0 type veth peer name patch1])
>>>>> +on_exit 'ip link del patch0'
>>>>> +
>>>>> +AT_CHECK([ip link set dev patch0 up])
>>>>> +AT_CHECK([ip link set dev patch1 up])
>>>>> +AT_CHECK([ovs-vsctl add-port br0 patch0 -- set interface patch0
>>>>> ofport_request=100])
>>>>> +AT_CHECK([ovs-vsctl add-port br1 patch1 -- set interface patch1
>>>>> ofport_request=100])
>>>>> +
>>>>> +AT_DATA([flows.txt], [dnl
>>>>> +table=0,priority=100,dl_type=0x0800 
>>>>> actions=encap(mpls(ether_type=0x8847)),set_mpls_label:2,encap(ethernet),output:100
>>>>> +table=0,priority=100,dl_type=0x8847,mpls_label=2
>>>>> actions=decap(),decap(packet_type(ns=0,type=0)),resubmit(,3)
>>>>> +table=0,priority=10 actions=resubmit(,3)
>>>>> +table=3,priority=10 actions=normal
>>>>> +])
>>>>> +
>>>>> +AT_CHECK([ovs-ofctl  -Oopenflow13 add-flows br0 flows.txt])
>>>>> +AT_CHECK([ovs-ofctl  -Oopenflow13 add-flows br1 flows.txt])
>>>>> +
>>>>> +NS_CHECK_EXEC([at_ns0], [ping -q -c 3 -i 0.3 -w 2 10.1.1.2 |
>>>>> FORMAT_PING], [0], [dnl
>>>>> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
>>>>> +])
>>>>> +
>>>>> +
>>>>>  NS_CHECK_EXEC([at_ns1], [ping -q -c 3 -i 0.3 -w 2 10.1.1.1 |
>>>>> FORMAT_PING], [0], [dnl
>>>>>  3 packets transmitted, 3 received, 0% packet loss, time 0ms
>>>>>  ])
>>>>> +
>>>>>  OVS_TRAFFIC_VSWITCHD_STOP
>>>>>  AT_CLEANUP
>>>>>
>>>>> -- 
>>>>> 2.18.4
>>>>
>>

On Thu, Apr 01, 2021 at 10:54:42AM +0200, Eelco Chaudron wrote:
> 
> 
> On 1 Apr 2021, at 10:35, Martin Varghese wrote:
> 
> > On Thu, Apr 01, 2021 at 08:59:27AM +0200, Eelco Chaudron wrote:
> > > 
> > > 
> > > On 1 Apr 2021, at 6:10, Martin Varghese wrote:
> > > 
> > > > On Wed, Mar 31, 2021 at 03:59:40PM +0200, Eelco Chaudron wrote:
> > > > > 
> > > > > 
> > > > > On 26 Mar 2021, at 7:21, Martin Varghese wrote:
> > > > > 
> > > > > > From: Martin Varghese <martin.varghese@nokia.com>
> > > > > > 
> > > > > > The encap & decap actions are extended to support MPLS
> > > > > > packet type.
> > > > > > Encap & decap actions adds and removes MPLS header at start of the
> > > > > > packet.
> > > > > 
> > > > > Hi Martin,
> > > > > 
> > > > > I’m trying to do some real-life testing, and I’m running into
> > > > > issues. This
> > > > > might be me setting it up wrongly but just wanting to confirm…
> > > > > 
> > > > > I’m sending an MPLS packet that contains an ARP packet into a
> > > > > physical port.
> > > > > This is the packet:
> > > > > 
> > > > > Frame 4: 64 bytes on wire (512 bits), 64 bytes captured (512 bits)
> > > > >     Encapsulation type: Ethernet (1)
> > > > >     [Protocols in frame: eth:ethertype:mpls:data]
> > > > > Ethernet II, Src: 00:00:00_00:00:01 (00:00:00:00:00:01), Dst:
> > > > > 00:00:00_00:00:02 (00:00:00:00:00:02)
> > > > >     Destination: 00:00:00_00:00:02 (00:00:00:00:00:02)
> > > > >         Address: 00:00:00_00:00:02 (00:00:00:00:00:02)
> > > > >         .... ..0. .... .... .... .... = LG bit: Globally unique
> > > > > address
> > > > > (factory default)
> > > > >         .... ...0 .... .... .... .... = IG bit: Individual address
> > > > > (unicast)
> > > > >     Source: 00:00:00_00:00:01 (00:00:00:00:00:01)
> > > > >         Address: 00:00:00_00:00:01 (00:00:00:00:00:01)
> > > > >         .... ..0. .... .... .... .... = LG bit: Globally unique
> > > > > address
> > > > > (factory default)
> > > > >         .... ...0 .... .... .... .... = IG bit: Individual address
> > > > > (unicast)
> > > > >     Type: MPLS label switched packet (0x8847)
> > > > > MultiProtocol Label Switching Header, Label: 100, Exp: 0, S:
> > > > > 1, TTL:
> > > > > 64
> > > > >     0000 0000 0000 0110 0100 .... .... .... = MPLS Label: 100
> > > > >     .... .... .... .... .... 000. .... .... = MPLS Experimental
> > > > > Bits: 0
> > > > >     .... .... .... .... .... ...1 .... .... = MPLS Bottom Of Label
> > > > > Stack: 1
> > > > >     .... .... .... .... .... .... 0100 0000 = MPLS TTL: 64
> > > > > Data (46 bytes)
> > > > > 
> > > > > 0000  ff ff ff ff ff ff 52 54 00 88 51 38 08 06 00 01
> > > > > ......RT..Q8....
> > > > > 0010  08 00 06 04 00 01 52 54 00 88 51 38 01 01 01 65
> > > > > ......RT..Q8...e
> > > > > 0020  00 00 00 00 00 00 01 01 01 64 27 98 a0 47
> > > > > .........d'..G
> > > > >     Data:
> > > > > ffffffffffff525400885138080600010800060400015254008851380101016500000000?
> > > > > 
> > > > > 
> > > > > I’m trying to use the following rules:
> > > > > 
> > > > >   ovs-ofctl del-flows ovs_pvp_br0
> > > > >   ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0
> > > > > "priority=100,dl_type=0x8847,mpls_label=100
> > > > > actions=decap(),decap(packet_type(ns=0,type=0x806)),resubmit(,3)"
> > > > >   ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0 "table=3,priority=10
> > > > > actions=normal"
> > > > > 
> > > > > With these, I expect the packet to be sent to vnet0, but
> > > > > it’s not.
> > > > > Actually,
> > > > > the datapath rule looks odd, while the userspace rules seem
> > > > > to match:
> > > > > 
> > > > >   $ ovs-dpctl dump-flows
> > > > >   recirc_id(0),in_port(1),eth(),eth_type(0x8847),mpls(label=100/0xfffff,tc=0/0,ttl=0/0x0,bos=1/1),
> > > > > packets:13, bytes:1118, used:0.322s,
> > > > > actions:pop_eth,pop_mpls(eth_type=0x806),recirc(0x19a)
> > > > >   recirc_id(0x19a),in_port(1),eth_type(0x0806), packets:13,
> > > > > bytes:884,
> > > > > used:0.322s, actions:drop
> > > > > 
> > > > >   $ ovs-ofctl dump-flows ovs_pvp_br0 -O OpenFlow13
> > > > >   cookie=0x0, duration=85.007s, table=0, n_packets=51,
> > > > > n_bytes=4386,
> > > > > priority=100,mpls,mpls_label=100
> > > > > actions=decap(),decap(packet_type(ns=0,type=0x806)),resubmit(,3)
> > > > >   cookie=0x0, duration=84.990s, table=3, n_packets=51,
> > > > > n_bytes=3468,
> > > > > priority=10 actions=NORMAL
> > > > > 
> > > > The inner packet is ethernet. So the packet type should be
> > > > (ns=0,type=0)
> > > > ?
> > > 
> > > Forgot to add that I already tried that to start with, based on the
> > > example,
> > > but as that did not work I tried 0x806.
> > > 
> > > PS: I have this as a remark in my review notes, i.e., to explain the
> > > ns and
> > > type usage here.
> > > 
> > > 
> > > This resulted in packets being counted at the open flow level, but it
> > > results in NO data path rules. Do get an error though:
> > > 
> > > 2021-04-01T06:53:36.056Z|00141|dpif(handler37)|WARN|system@ovs-system:
> > > failed to put[create] (Invalid argument)
> > > ufid:3d2d6f6d-5a66-4ace-8b09-7cdcfa5efc8e recirc_id(0),dp_hash(0/0),skb_priority(0/0),in_port(1),skb_mark(0/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),eth(src=00:00:00:00:00:01/00:00:00:00:00:00,dst=00:00:00:00:00:02/00:00:00:00:00:00),eth_type(0x8847),mpls(label=100/0xfffff,tc=0/0,ttl=64/0x0,bos=1/1),
> > > actions:pop_eth,pop_mpls(eth_type=0x6558),set(eth()),recirc(0x4c)
> > 
> > This set(eth) before the recirc is the problem i guesss. I need to check
> > > 2021-04-01T06:53:36.056Z|00142|dpif(handler37)|WARN|system@ovs-system:
> > > execute pop_eth,pop_mpls(eth_type=0x6558),set(eth()),recirc(0x4c)
> > > failed
> > > (Invalid argument) on packet mpls,vlan_tci=0x0000,dl_src=00:00:00:00:00:01,dl_dst=00:00:00:00:00:02,mpls_label=100,mpls_tc=0,mpls_ttl=64,mpls_bos=1
> > >  with metadata skb_priority(0),skb_mark(0),in_port(1) mtu 0
> > > 
> > > Are there missing parts in my kernel that do not get properly
> > > detected by
> > > the feature detection?
> > > 
> > > $ ovs-appctl dpif/show-dp-features ovs_pvp_br0
> > > Masked set action: Yes
> > > Tunnel push pop: No
> > > Ufid: Yes
> > > Truncate action: Yes
> > > Clone action: Yes
> > > Sample nesting: 10
> > > Conntrack eventmask: Yes
> > > Conntrack clear: Yes
> > > Max dp_hash algorithm: 0
> > > Check pkt length action: Yes
> > > Conntrack timeout policy: Yes
> > > Explicit Drop action: No
> > > Optimized Balance TCP mode: No
> > > l2 MPLS tunnelling: Yes
> > > Max VLAN headers: 2
> > > Max MPLS depth: 3
> > > Recirc: Yes
> > > CT state: Yes
> > > CT zone: Yes
> > > CT mark: Yes
> > > CT label: Yes
> > > CT state NAT: Yes
> > > CT orig tuple: Yes
> > > CT orig tuple for IPv6: Yes
> > > IPv6 ND Extension: No
> > > 
> > You are good
> > 
> > I am not sure what is going wrong. Your test case looks same as the unit
> > test i added.
> > 
> > I tried myself again and this is i get
> > 
> > ovs-ofctl -O OpenFlow13 add-flow br_mpls2
> > "in_port=$egress_port,dl_type=0x8847
> > +actions=decap(),decap(packet_type(ns=0,type=0),goto_table:1"
> > ovs-ofctl -O OpenFlow13 add-flow br_mpls2
> > +"table=1,in_port=$egress_port,dl_type=0x0800,nw_dst=1.1.1.2
> > +actions=set_field:00:00:00:00:00:02->dl_dst,set_field:00:00:00:00:00:01->dl_sr
> > +c output:$ingress_port"
> > 
> > recirc_id(0x3),in_port(6),eth(src=36:b1:ee:7c:01:03,dst=36:b1:ee:7c:01:02),eth_
> > +type(0x0800),ipv4(dst=1.1.1.2,frag=no), packets:3, bytes:294,
> > used:0.837s,
> > +actions:set(eth(src=00:00:00:00:00:01,dst=00:00:00:00:00:02)),4
> > recirc_id(0),in_port(6),eth(),eth_type(0x8847),mpls(label=0/0x0,tc=0/0,ttl=0/0x
> > +0,bos=1/1), packets:3, bytes:348, used:0.837s,
> > +actions:pop_eth,pop_mpls(eth_type=0x6558),recirc(0x3)
> > 
> > The packet to the ovs is
> > ETH|MPLS|ETH|IP ?
> > How it is differnt from you test case?
> 
> Mine is ETH|MPLS|ETH|ARP, which works fine with pop_mpls
>
I am wondering how old mpls pop  action works
could you please put down the userspace and datapath rules when you used
pop_mpls.

In my understanding it can never work as what you have after MPLS is
ethernet and not ARP.


> > Thanks for your time.
> 
> Your welcome
> 
> > > > > 
> > > > > If I use the old way, doing pop_mpls, it works fine:
> > > > > 
> > > > > 
> > > > > ovs-ofctl del-flows ovs_pvp_br0
> > > > > ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0
> > > > > "priority=100,dl_type=0x8847,mpls_label=100
> > > > > actions=pop_mpls:0x0806,resubmit(,3)"
> > > > > ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0 "table=3,priority=10
> > > > > actions=normal"
> > > > > 
> > > > > 
> > > > > I also noticed (despite the test example) to make encap work, I had
> > > > > to set
> > > > > the ethernet MAC addresses, or else the packets were not
> > > > > getting out.
> > > > > So something like:
> > > > > 
> > > > >   ovs-ofctl add-flow -O OpenFlow13 ovs_pvp_br0 "priority=100,in_port=vnet0,actions=encap(mpls(ether_type=0x8847)),set_mpls_label:100,encap(ethernet),,set_field:00:00:00:00:00:02->dl_dst,set_field:00:00:00:00:00:01->dl_src,output:enp5s0f0
> > > > > 
> > 
> > > > 
> > > > The packets are not going out because you are sending the packet
> > > > on a
> > > > real nic and not on a virtual inerface (veth pair) ?
> > > 
> > > So for a real NIC we need to set the MAC addresses, maybe some where
> > > in the
> > > documentation we should add an example on how to use this feature?
> > > 
> > > > > Maybe the test case can be made more realistic? Once I
> > > > > understand the
> > > > > failure, I can continue with the review.
> > > > > 
> > > > > 
> > > > > Cheers,
> > > > > 
> > > > > Eelco
> > > > > 
> > > > > 
> > > > > 
> > > > > > Signed-off-by: Martin Varghese <martin.varghese@nokia.com>
> > > > > > ---
> > > > > >  NEWS                                          |  2 +-
> > > > > >  .../linux/compat/include/linux/openvswitch.h  | 35 ++++++-
> > > > > >  include/openvswitch/ofp-ed-props.h            | 18 ++++
> > > > > >  lib/dpif-netdev.c                             |  1 +
> > > > > >  lib/dpif.c                                    |  1 +
> > > > > >  lib/odp-execute.c                             | 12 +++
> > > > > >  lib/odp-util.c                                | 58 +++++++++---
> > > > > >  lib/ofp-actions.c                             |  5 +
> > > > > >  lib/ofp-ed-props.c                            | 91
> > > > > > +++++++++++++++++++
> > > > > >  lib/ovs-actions.xml                           | 31 +++++--
> > > > > >  lib/packets.c                                 | 36 ++++++++
> > > > > >  lib/packets.h                                 |  2 +
> > > > > >  ofproto/ofproto-dpif-ipfix.c                  |  1 +
> > > > > >  ofproto/ofproto-dpif-sflow.c                  |  1 +
> > > > > >  ofproto/ofproto-dpif-xlate.c                  | 60 ++++++++++++
> > > > > >  ofproto/ofproto-dpif.c                        | 39 ++++++++
> > > > > >  ofproto/ofproto-dpif.h                        |  5 +-
> > > > > >  tests/system-traffic.at                       | 36 ++++++++
> > > > > >  18 files changed, 410 insertions(+), 24 deletions(-)
> > > > > > 
> > > > > > diff --git a/NEWS b/NEWS
> > > > > > index 95cf922aa..4bf4e9e7b 100644
> > > > > > --- a/NEWS
> > > > > > +++ b/NEWS
> > > > > > @@ -120,7 +120,7 @@ v2.14.0 - 17 Aug 2020
> > > > > >     - GTP-U Tunnel Protocol
> > > > > >       * Add two new fields: tun_gtpu_flags, tun_gtpu_msgtype.
> > > > > >       * Only support for userspace datapath.
> > > > > > -
> > > > > > +   - Encap & Decap action support for MPLS packet type.
> > > > > > 
> > > > > >  v2.13.0 - 14 Feb 2020
> > > > > >  ---------------------
> > > > > > diff --git a/datapath/linux/compat/include/linux/openvswitch.h
> > > > > > b/datapath/linux/compat/include/linux/openvswitch.h
> > > > > > index 875de2025..8feea7dd4 100644
> > > > > > --- a/datapath/linux/compat/include/linux/openvswitch.h
> > > > > > +++ b/datapath/linux/compat/include/linux/openvswitch.h
> > > > > > @@ -810,8 +810,32 @@ struct ovs_action_push_tnl {
> > > > > >  };
> > > > > >  #endif
> > > > > > 
> > > > > > -/**
> > > > > > - * enum ovs_ct_attr - Attributes for %OVS_ACTION_ATTR_CT action.
> > > > > > +/* struct ovs_action_add_mpls - %OVS_ACTION_ATTR_ADD_MPLS action
> > > > > > + * argument.
> > > > > > + * @mpls_lse: MPLS label stack entry to push.
> > > > > > + * @mpls_ethertype: Ethertype to set in the
> > > > > > encapsulating ethernet
> > > > > > frame.
> > > > > > + * @tun_flags: MPLS tunnel attributes.
> > > > > > + *
> > > > > > + * The only values @mpls_ethertype should ever be given are
> > > > > > %ETH_P_MPLS_UC and
> > > > > > + * %ETH_P_MPLS_MC, indicating MPLS unicast or
> > > > > > multicast. Other are
> > > > > > rejected.
> > > > > > + */
> > > > > > +struct ovs_action_add_mpls {
> > > > > > +	__be32 mpls_lse;
> > > > > > +	__be16 mpls_ethertype; /* Either %ETH_P_MPLS_UC or
> > > > > > %ETH_P_MPLS_MC */
> > > > > > +	__u16 tun_flags;
> > > > > > +};
> > > > > > +
> > > > > > +#define OVS_MPLS_L3_TUNNEL_FLAG_MASK  (1 << 0) /* Flag to
> > > > > > specify the
> > > > > > place of
> > > > > > +						* insertion of MPLS header.
> > > > > > +						* When false, the MPLS header
> > > > > > +						* will be inserted at the start
> > > > > > +						* of the packet.
> > > > > > +						* When true, the MPLS header
> > > > > > +						* will be inserted at the start
> > > > > > +						* of the l3 header.
> > > > > > +						*/
> > > > > > +
> > > > > > +/* enum ovs_ct_attr - Attributes for %OVS_ACTION_ATTR_CT action.
> > > > > >   * @OVS_CT_ATTR_COMMIT: If present, commits the connection to the
> > > > > > conntrack
> > > > > >   * table. This allows future packets for the same
> > > > > > connection to be
> > > > > > identified
> > > > > >   * as 'established' or 'related'. The flow key for the current
> > > > > > packet
> > > > > > will
> > > > > > @@ -1001,7 +1025,11 @@ struct check_pkt_len_arg {
> > > > > >   * @OVS_ACTION_ATTR_CHECK_PKT_LEN: Check the packet length and
> > > > > > execute
> > > > > > a set
> > > > > >   * of actions if greater than the specified packet length, else
> > > > > > execute
> > > > > >   * another set of actions.
> > > > > > - * @OVS_ACTION_ATTR_DROP: Explicit drop action.
> > > > > > + * @OVS_ACTION_ATTR_ADD_MPLS: Push a new MPLS label stack entry
> > > > > > at the
> > > > > > + * start of the packet or at the start of the l3 header
> > > > > > depending on
> > > > > > the value
> > > > > > + * of l3 tunnel flag in the tun_flags field of
> > > > > > OVS_ACTION_ATTR_ADD_MPLS
> > > > > > + * argument.
> > > > > > +  * @OVS_ACTION_ATTR_DROP: Explicit drop action.
> > > > > >   */
> > > > > > 
> > > > > >  enum ovs_action_attr {
> > > > > > @@ -1030,6 +1058,7 @@ enum ovs_action_attr {
> > > > > >  	OVS_ACTION_ATTR_METER,        /* u32 meter number. */
> > > > > >  	OVS_ACTION_ATTR_CLONE,        /* Nested OVS_CLONE_ATTR_*.  */
> > > > > >  	OVS_ACTION_ATTR_CHECK_PKT_LEN, /* Nested
> > > > > > OVS_CHECK_PKT_LEN_ATTR_*. */
> > > > > > +	OVS_ACTION_ATTR_ADD_MPLS,     /* struct ovs_action_add_mpls. */
> > > > > > 
> > > > > >  #ifndef __KERNEL__
> > > > > >  	OVS_ACTION_ATTR_TUNNEL_PUSH,   /* struct ovs_action_push_tnl*/
> > > > > > diff --git a/include/openvswitch/ofp-ed-props.h
> > > > > > b/include/openvswitch/ofp-ed-props.h
> > > > > > index 306c6fe73..c85f3c283 100644
> > > > > > --- a/include/openvswitch/ofp-ed-props.h
> > > > > > +++ b/include/openvswitch/ofp-ed-props.h
> > > > > > @@ -46,6 +46,11 @@ enum ofp_ed_nsh_prop_type {
> > > > > >      OFPPPT_PROP_NSH_TLV = 2,     /* property TLV in NSH */
> > > > > >  };
> > > > > > 
> > > > > > +enum ofp_ed_mpls_prop_type {
> > > > > > +    OFPPPT_PROP_MPLS_NONE = 0,    /* unused */
> > > > > > +    OFPPPT_PROP_MPLS_ETHERTYPE = 1,  /* MPLS Ethertype */
> > > > > > +};
> > > > > > +
> > > > > >  /*
> > > > > >   * External representation of encap/decap properties.
> > > > > >   * These must be padded to a multiple of 8 bytes.
> > > > > > @@ -72,6 +77,13 @@ struct ofp_ed_prop_nsh_tlv {
> > > > > >      uint8_t data[0];
> > > > > >  };
> > > > > > 
> > > > > > +struct ofp_ed_prop_mpls_ethertype {
> > > > > > +    struct ofp_ed_prop_header header;
> > > > > > +    uint16_t ether_type;         /* MPLS ethertype .*/
> > > > > > +    uint8_t pad[2];          /* Padding to 8 bytes. */
> > > > > > +};
> > > > > > +
> > > > > > +
> > > > > >  /*
> > > > > >   * Internal representation of encap/decap properties
> > > > > >   */
> > > > > > @@ -96,6 +108,12 @@ struct ofpact_ed_prop_nsh_tlv {
> > > > > >      /* tlv_len octets of metadata value, padded to a
> > > > > > multiple of 8
> > > > > > bytes. */
> > > > > >      uint8_t data[0];
> > > > > >  };
> > > > > > +
> > > > > > +struct ofpact_ed_prop_mpls_ethertype {
> > > > > > +    struct ofpact_ed_prop header;
> > > > > > +    uint16_t ether_type;         /* MPLS ethertype .*/
> > > > > > +    uint8_t pad[2];          /* Padding to 8 bytes. */
> > > > > > +};
> > > > > >  enum ofperr decode_ed_prop(const struct ofp_ed_prop_header
> > > > > > **ofp_prop,
> > > > > >                             struct ofpbuf *out, size_t
> > > > > > *remaining);
> > > > > >  enum ofperr encode_ed_prop(const struct ofpact_ed_prop **prop,
> > > > > > diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
> > > > > > index 94cc9b80c..bbdea5603 100644
> > > > > > --- a/lib/dpif-netdev.c
> > > > > > +++ b/lib/dpif-netdev.c
> > > > > > @@ -8044,6 +8044,7 @@ dp_execute_cb(void *aux_, struct
> > > > > > dp_packet_batch
> > > > > > *packets_,
> > > > > >      case OVS_ACTION_ATTR_CT_CLEAR:
> > > > > >      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
> > > > > >      case OVS_ACTION_ATTR_DROP:
> > > > > > +    case OVS_ACTION_ATTR_ADD_MPLS:
> > > > > >      case __OVS_ACTION_ATTR_MAX:
> > > > > >          OVS_NOT_REACHED();
> > > > > >      }
> > > > > > diff --git a/lib/dpif.c b/lib/dpif.c
> > > > > > index 56d0b4a65..bbd1296e3 100644
> > > > > > --- a/lib/dpif.c
> > > > > > +++ b/lib/dpif.c
> > > > > > @@ -1273,6 +1273,7 @@ dpif_execute_helper_cb(void *aux_, struct
> > > > > > dp_packet_batch *packets_,
> > > > > >      case OVS_ACTION_ATTR_UNSPEC:
> > > > > >      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
> > > > > >      case OVS_ACTION_ATTR_DROP:
> > > > > > +    case OVS_ACTION_ATTR_ADD_MPLS:
> > > > > >      case __OVS_ACTION_ATTR_MAX:
> > > > > >          OVS_NOT_REACHED();
> > > > > >      }
> > > > > > diff --git a/lib/odp-execute.c b/lib/odp-execute.c
> > > > > > index 6eeda2a61..2f4cdd92c 100644
> > > > > > --- a/lib/odp-execute.c
> > > > > > +++ b/lib/odp-execute.c
> > > > > > @@ -819,6 +819,7 @@ requires_datapath_assistance(const struct
> > > > > > nlattr *a)
> > > > > >      case OVS_ACTION_ATTR_POP_NSH:
> > > > > >      case OVS_ACTION_ATTR_CT_CLEAR:
> > > > > >      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
> > > > > > +    case OVS_ACTION_ATTR_ADD_MPLS:
> > > > > >      case OVS_ACTION_ATTR_DROP:
> > > > > >          return false;
> > > > > > 
> > > > > > @@ -1061,6 +1062,17 @@ odp_execute_actions(void *dp, struct
> > > > > > dp_packet_batch *batch, bool steal,
> > > > > >              }
> > > > > >              break;
> > > > > > 
> > > > > > +        case OVS_ACTION_ATTR_ADD_MPLS: {
> > > > > > +            const struct ovs_action_add_mpls *mpls =
> > > > > > nl_attr_get(a);
> > > > > > +            bool l3_flag =  mpls->tun_flags &
> > > > > > OVS_MPLS_L3_TUNNEL_FLAG_MASK;
> > > > > > +
> > > > > > +            DP_PACKET_BATCH_FOR_EACH (i, packet, batch) {
> > > > > > +                add_mpls(packet, mpls->mpls_ethertype,
> > > > > > mpls->mpls_lse,
> > > > > > +                         l3_flag);
> > > > > > +            }
> > > > > > +            break;
> > > > > > +        }
> > > > > > +
> > > > > >          case OVS_ACTION_ATTR_DROP:{
> > > > > >              const enum xlate_error *drop_reason = nl_attr_get(a);
> > > > > > 
> > > > > > diff --git a/lib/odp-util.c b/lib/odp-util.c
> > > > > > index a8598d52a..f24e16d08 100644
> > > > > > --- a/lib/odp-util.c
> > > > > > +++ b/lib/odp-util.c
> > > > > > @@ -142,6 +142,8 @@ odp_action_len(uint16_t type)
> > > > > >      case OVS_ACTION_ATTR_PUSH_NSH: return ATTR_LEN_VARIABLE;
> > > > > >      case OVS_ACTION_ATTR_POP_NSH: return 0;
> > > > > >      case OVS_ACTION_ATTR_CHECK_PKT_LEN: return ATTR_LEN_VARIABLE;
> > > > > > +    case OVS_ACTION_ATTR_ADD_MPLS:
> > > > > > +         return sizeof(struct ovs_action_add_mpls);
> > > > > >      case OVS_ACTION_ATTR_DROP: return sizeof(uint32_t);
> > > > > > 
> > > > > >      case OVS_ACTION_ATTR_UNSPEC:
> > > > > > @@ -1254,6 +1256,14 @@ format_odp_action(struct ds *ds,
> > > > > > const struct
> > > > > > nlattr *a,
> > > > > >      case OVS_ACTION_ATTR_CHECK_PKT_LEN:
> > > > > >          format_odp_check_pkt_len_action(ds, a, portno_names);
> > > > > >          break;
> > > > > > +    case OVS_ACTION_ATTR_ADD_MPLS: {
> > > > > > +        const struct ovs_action_push_mpls *mpls = nl_attr_get(a);
> > > > > > +        ds_put_cstr(ds, "add_mpls(");
> > > > > > +        format_mpls_lse(ds, mpls->mpls_lse);
> > > > > > +        ds_put_format(ds, ",eth_type=0x%"PRIx16")",
> > > > > > +                      ntohs(mpls->mpls_ethertype));
> > > > > > +        break;
> > > > > > +    }
> > > > > >      case OVS_ACTION_ATTR_DROP:
> > > > > >          ds_put_cstr(ds, "drop");
> > > > > >          break;
> > > > > > @@ -7876,7 +7886,8 @@ commit_vlan_action(const struct flow*
> > > > > > flow, struct
> > > > > > flow *base,
> > > > > >  /* Wildcarding already done at action translation time. */
> > > > > >  static void
> > > > > >  commit_mpls_action(const struct flow *flow, struct flow *base,
> > > > > > -                   struct ofpbuf *odp_actions)
> > > > > > +                   struct ofpbuf *odp_actions, bool
> > > > > > pending_encap,
> > > > > > +                   bool pending_decap)
> > > > > >  {
> > > > > >      int base_n = flow_count_mpls_labels(base, NULL);
> > > > > >      int flow_n = flow_count_mpls_labels(flow, NULL);
> > > > > > @@ -7913,7 +7924,11 @@ commit_mpls_action(const struct flow *flow,
> > > > > > struct flow *base,
> > > > > >              if ((!eth_type_mpls(flow->dl_type)) && base_n > 1) {
> > > > > >                  dl_type = htons(ETH_TYPE_MPLS);
> > > > > >              } else {
> > > > > > -                dl_type = flow->dl_type;
> > > > > > +                if ((flow->packet_type == PT_ETH) &&
> > > > > > pending_decap) {
> > > > > > +                    dl_type =  htons(ETH_TYPE_TEB);
> > > > > > +                } else {
> > > > > > +                    dl_type = flow->dl_type;
> > > > > > +                }
> > > > > >              }
> > > > > >              nl_msg_put_be16(odp_actions,
> > > > > > OVS_ACTION_ATTR_POP_MPLS,
> > > > > > dl_type);
> > > > > >              ovs_assert(flow_pop_mpls(base, base_n, flow->dl_type,
> > > > > > NULL));
> > > > > > @@ -7924,18 +7939,29 @@ commit_mpls_action(const struct
> > > > > > flow *flow,
> > > > > > struct flow *base,
> > > > > >      /* If, after the above popping and setting, there are more
> > > > > > LSEs in
> > > > > > flow
> > > > > >       * than base then some LSEs need to be pushed. */
> > > > > >      while (base_n < flow_n) {
> > > > > > -        struct ovs_action_push_mpls *mpls;
> > > > > > 
> > > > > > -        mpls = nl_msg_put_unspec_zero(odp_actions,
> > > > > > -                                      OVS_ACTION_ATTR_PUSH_MPLS,
> > > > > > -                                      sizeof *mpls);
> > > > > > -        mpls->mpls_ethertype = flow->dl_type;
> > > > > > -        mpls->mpls_lse = flow->mpls_lse[flow_n - base_n - 1];
> > > > > > +        if (pending_encap) {
> > > > > > +             struct ovs_action_add_mpls *mpls;
> > > > > > +
> > > > > > +             mpls = nl_msg_put_unspec_zero(odp_actions,
> > > > > > +
> > > > > > OVS_ACTION_ATTR_ADD_MPLS,
> > > > > > +                                           sizeof *mpls);
> > > > > > +             mpls->mpls_ethertype = flow->dl_type;
> > > > > > +             mpls->mpls_lse = flow->mpls_lse[flow_n -
> > > > > > base_n - 1];
> > > > > > +        } else {
> > > > > > +             struct ovs_action_push_mpls *mpls;
> > > > > > +
> > > > > > +             mpls = nl_msg_put_unspec_zero(odp_actions,
> > > > > > +
> > > > > > OVS_ACTION_ATTR_PUSH_MPLS,
> > > > > > +                                           sizeof *mpls);
> > > > > > +             mpls->mpls_ethertype = flow->dl_type;
> > > > > > +             mpls->mpls_lse = flow->mpls_lse[flow_n -
> > > > > > base_n - 1];
> > > > > > +        }
> > > > > >          /* Update base flow's MPLS stack, but do not clear L3.
> > > > > > We need
> > > > > > the L3
> > > > > >           * headers if the flow is restored later due to
> > > > > > returning from
> > > > > > a patch
> > > > > >           * port or group bucket. */
> > > > > > -        flow_push_mpls(base, base_n, mpls->mpls_ethertype, NULL,
> > > > > > false);
> > > > > > -        flow_set_mpls_lse(base, 0, mpls->mpls_lse);
> > > > > > +        flow_push_mpls(base, base_n, flow->dl_type, NULL, false);
> > > > > > +        flow_set_mpls_lse(base, 0,
> > > > > > flow->mpls_lse[flow_n - base_n -
> > > > > > 1]);
> > > > > >          base_n++;
> > > > > >      }
> > > > > >  }
> > > > > > @@ -8586,6 +8612,10 @@ commit_encap_decap_action(const struct flow
> > > > > > *flow,
> > > > > >              memcpy(&base_flow->dl_dst, &flow->dl_dst,
> > > > > >                     sizeof(*flow) - offsetof(struct
> > > > > > flow, dl_dst));
> > > > > >              break;
> > > > > > +        case PT_MPLS:
> > > > > > +            commit_mpls_action(flow, base_flow, odp_actions,
> > > > > > pending_encap,
> > > > > > +                               pending_decap);
> > > > > > +            break;
> > > > > >          default:
> > > > > >              /* Only the above protocols are supported for encap.
> > > > > >               * The check is done at action translation. */
> > > > > > @@ -8608,6 +8638,10 @@ commit_encap_decap_action(const struct flow
> > > > > > *flow,
> > > > > >                  /* pop_nsh. */
> > > > > >                  odp_put_pop_nsh_action(odp_actions);
> > > > > >                  break;
> > > > > > +            case PT_MPLS:
> > > > > > +                commit_mpls_action(flow, base_flow, odp_actions,
> > > > > > pending_encap,
> > > > > > +                                   pending_decap);
> > > > > > +                break;
> > > > > >              default:
> > > > > >                  /* Checks are done during translation. */
> > > > > >                  OVS_NOT_REACHED();
> > > > > > @@ -8653,7 +8687,7 @@ commit_odp_actions(const struct flow
> > > > > > *flow, struct
> > > > > > flow *base,
> > > > > >      /* Make packet a non-MPLS packet before committing L3/4
> > > > > > actions,
> > > > > >       * which would otherwise do nothing. */
> > > > > >      if (eth_type_mpls(base->dl_type) &&
> > > > > > !eth_type_mpls(flow->dl_type))
> > > > > > {
> > > > > > -        commit_mpls_action(flow, base, odp_actions);
> > > > > > +        commit_mpls_action(flow, base, odp_actions,
> > > > > > false, false);
> > > > > >          mpls_done = true;
> > > > > >      }
> > > > > >      commit_set_nsh_action(flow, base, odp_actions, wc,
> > > > > > use_masked);
> > > > > > @@ -8661,7 +8695,7 @@ commit_odp_actions(const struct flow
> > > > > > *flow, struct
> > > > > > flow *base,
> > > > > >      commit_set_port_action(flow, base, odp_actions, wc,
> > > > > > use_masked);
> > > > > >      slow2 = commit_set_icmp_action(flow, base, odp_actions, wc);
> > > > > >      if (!mpls_done) {
> > > > > > -        commit_mpls_action(flow, base, odp_actions);
> > > > > > +        commit_mpls_action(flow, base, odp_actions,
> > > > > > false, false);
> > > > > >      }
> > > > > >      commit_vlan_action(flow, base, odp_actions, wc);
> > > > > >      commit_set_priority_action(flow, base, odp_actions, wc,
> > > > > > use_masked);
> > > > > > diff --git a/lib/ofp-actions.c b/lib/ofp-actions.c
> > > > > > index 0342a228b..28a12a569 100644
> > > > > > --- a/lib/ofp-actions.c
> > > > > > +++ b/lib/ofp-actions.c
> > > > > > @@ -4441,6 +4441,7 @@ decode_NXAST_RAW_ENCAP(const struct
> > > > > > nx_action_encap *nae,
> > > > > >      switch (ntohl(nae->new_pkt_type)) {
> > > > > >      case PT_ETH:
> > > > > >      case PT_NSH:
> > > > > > +    case PT_MPLS:
> > > > > >          /* Add supported encap header types here. */
> > > > > >          break;
> > > > > >      default:
> > > > > > @@ -4492,6 +4493,8 @@ parse_encap_header(const char *hdr, ovs_be32
> > > > > > *packet_type)
> > > > > >          *packet_type = htonl(PT_ETH);
> > > > > >      } else if (strcmp(hdr, "nsh") == 0) {
> > > > > >          *packet_type = htonl(PT_NSH);
> > > > > > +    } else if (strcmp(hdr, "mpls") == 0) {
> > > > > > +        *packet_type = htonl(PT_MPLS);
> > > > > >      } else {
> > > > > >          return false;
> > > > > >      }
> > > > > > @@ -4573,6 +4576,8 @@ format_encap_pkt_type(const
> > > > > > ovs_be32 pkt_type)
> > > > > >          return "ethernet";
> > > > > >      case PT_NSH:
> > > > > >          return "nsh";
> > > > > > +    case PT_MPLS:
> > > > > > +        return "mpls";
> > > > > >      default:
> > > > > >          return "UNKNOWN";
> > > > > >      }
> > > > > > diff --git a/lib/ofp-ed-props.c b/lib/ofp-ed-props.c
> > > > > > index 02a9235d5..fc261e4c6 100644
> > > > > > --- a/lib/ofp-ed-props.c
> > > > > > +++ b/lib/ofp-ed-props.c
> > > > > > @@ -79,6 +79,27 @@ decode_ed_prop(const struct ofp_ed_prop_header
> > > > > > **ofp_prop,
> > > > > >          }
> > > > > >          break;
> > > > > >      }
> > > > > > +    case OFPPPC_MPLS: {
> > > > > > +       switch (prop_type) {
> > > > > > +        case OFPPPT_PROP_MPLS_ETHERTYPE: {
> > > > > > +            struct ofp_ed_prop_mpls_ethertype *opnmt =
> > > > > > +                ALIGNED_CAST(struct ofp_ed_prop_mpls_ethertype *,
> > > > > > *ofp_prop);
> > > > > > +            if (len > sizeof(*opnmt) || len > *remaining) {
> > > > > > +                return OFPERR_NXBAC_BAD_ED_PROP;
> > > > > > +            }
> > > > > > +            struct ofpact_ed_prop_mpls_ethertype *pnmt =
> > > > > > +                    ofpbuf_put_uninit(out, sizeof(*pnmt));
> > > > > > +            pnmt->header.prop_class = prop_class;
> > > > > > +            pnmt->header.type = prop_type;
> > > > > > +            pnmt->header.len = len;
> > > > > > +            pnmt->ether_type = opnmt->ether_type;
> > > > > > +            break;
> > > > > > +        }
> > > > > > +        default:
> > > > > > +            return OFPERR_NXBAC_UNKNOWN_ED_PROP;
> > > > > > +        }
> > > > > > +        break;
> > > > > > +    }
> > > > > >      default:
> > > > > >          return OFPERR_NXBAC_UNKNOWN_ED_PROP;
> > > > > >      }
> > > > > > @@ -134,6 +155,27 @@ encode_ed_prop(const struct ofpact_ed_prop
> > > > > > **prop,
> > > > > >          }
> > > > > >          break;
> > > > > >      }
> > > > > > +    case OFPPPC_MPLS: {
> > > > > > +       switch ((*prop)->type) {
> > > > > > +       case OFPPPT_PROP_MPLS_ETHERTYPE: {
> > > > > > +           struct ofpact_ed_prop_mpls_ethertype *pnmt =
> > > > > > +                ALIGNED_CAST(struct
> > > > > > ofpact_ed_prop_mpls_ethertype *,
> > > > > > *prop);
> > > > > > +            struct ofp_ed_prop_mpls_ethertype *opnmt =
> > > > > > +                    ofpbuf_put_uninit(out, sizeof(*opnmt));
> > > > > > +            opnmt->header.prop_class =
> > > > > > htons((*prop)->prop_class);
> > > > > > +            opnmt->header.type = (*prop)->type;
> > > > > > +            opnmt->header.len =
> > > > > > +                    offsetof(struct
> > > > > > ofpact_ed_prop_mpls_ethertype,
> > > > > > pad);
> > > > > > +            opnmt->ether_type = pnmt->ether_type;
> > > > > > +            prop_len = sizeof(*pnmt);
> > > > > > +            break;
> > > > > > +
> > > > > > +       }
> > > > > > +       default:
> > > > > > +            return OFPERR_OFPBAC_BAD_ARGUMENT;
> > > > > > +       }
> > > > > > +       break;
> > > > > > +    }
> > > > > >      default:
> > > > > >          return OFPERR_OFPBAC_BAD_ARGUMENT;
> > > > > >      }
> > > > > > @@ -181,6 +223,13 @@ parse_ed_prop_type(uint16_t prop_class,
> > > > > >          } else {
> > > > > >              return false;
> > > > > >          }
> > > > > > +    case OFPPPC_MPLS:
> > > > > > +        if (!strcmp(str, "ether_type")) {
> > > > > > +            *type = OFPPPT_PROP_MPLS_ETHERTYPE;
> > > > > > +            return true;
> > > > > > +        } else {
> > > > > > +            return false;
> > > > > > +        }
> > > > > >      default:
> > > > > >          return false;
> > > > > >      }
> > > > > > @@ -259,6 +308,28 @@ parse_ed_prop_value(uint16_t prop_class,
> > > > > > uint8_t
> > > > > > prop_type OVS_UNUSED,
> > > > > >              OVS_NOT_REACHED();
> > > > > >          }
> > > > > >          break;
> > > > > > +    case OFPPPC_MPLS:
> > > > > > +        switch (prop_type) {
> > > > > > +        case OFPPPT_PROP_MPLS_ETHERTYPE: {
> > > > > > +            uint16_t ethertype;
> > > > > > +            error = str_to_u16(value, "ether_type", &ethertype);
> > > > > > +            if (error != NULL) {
> > > > > > +                return error;
> > > > > > +            }
> > > > > > +            struct ofpact_ed_prop_mpls_ethertype *pnmt =
> > > > > > +                    ofpbuf_put_uninit(out, sizeof(*pnmt));
> > > > > > +            pnmt->header.prop_class = prop_class;
> > > > > > +            pnmt->header.type = prop_type;
> > > > > > +            pnmt->header.len =
> > > > > > +                    offsetof(struct
> > > > > > ofpact_ed_prop_mpls_ethertype,
> > > > > > pad);
> > > > > > +            pnmt->ether_type = ethertype;
> > > > > > +
> > > > > > +            break;
> > > > > > +        }
> > > > > > +        default:
> > > > > > +            break;
> > > > > > +      }
> > > > > > +      break;
> > > > > >      default:
> > > > > >          /* Unsupported property classes rejected before. */
> > > > > >          OVS_NOT_REACHED();
> > > > > > @@ -300,6 +371,14 @@ format_ed_prop_type(const struct
> > > > > > ofpact_ed_prop
> > > > > > *prop)
> > > > > >              OVS_NOT_REACHED();
> > > > > >          }
> > > > > >          break;
> > > > > > +    case OFPPPC_MPLS:
> > > > > > +         switch (prop->type) {
> > > > > > +         case OFPPPT_PROP_MPLS_ETHERTYPE:
> > > > > > +              return "ether_type";
> > > > > > +         default:
> > > > > > +               OVS_NOT_REACHED();
> > > > > > +         }
> > > > > > +         break;
> > > > > >      default:
> > > > > >          OVS_NOT_REACHED();
> > > > > >      }
> > > > > > @@ -332,6 +411,18 @@ format_ed_prop(struct ds *s OVS_UNUSED,
> > > > > >          default:
> > > > > >              OVS_NOT_REACHED();
> > > > > >          }
> > > > > > +     case OFPPPC_MPLS:
> > > > > > +        switch (prop->type) {
> > > > > > +        case OFPPPT_PROP_MPLS_ETHERTYPE: {
> > > > > > +          struct ofpact_ed_prop_mpls_ethertype *pnmt =
> > > > > > +                ALIGNED_CAST(struct
> > > > > > ofpact_ed_prop_mpls_ethertype *,
> > > > > > prop);
> > > > > > +            ds_put_format(s, "%s=%d", format_ed_prop_type(prop),
> > > > > > +                          pnmt->ether_type);
> > > > > > +            return;
> > > > > > +        }
> > > > > > +        default:
> > > > > > +            OVS_NOT_REACHED();
> > > > > > +        }
> > > > > >      default:
> > > > > >          OVS_NOT_REACHED();
> > > > > >      }
> > > > > > diff --git a/lib/ovs-actions.xml b/lib/ovs-actions.xml
> > > > > > index a2778de4b..e97f818d9 100644
> > > > > > --- a/lib/ovs-actions.xml
> > > > > > +++ b/lib/ovs-actions.xml
> > > > > > @@ -265,13 +265,13 @@
> > > > > >        </p>
> > > > > > 
> > > > > >        <p>
> > > > > > -        When a <code>decap</code> action decapsulates a packet,
> > > > > > Open
> > > > > > vSwitch
> > > > > > -        raises this error if it does not support the
> > > > > > type of inner
> > > > > > packet.
> > > > > > -        <code>decap</code> of an Ethernet header raises this
> > > > > > error if a
> > > > > > VLAN
> > > > > > -        header is present, <code>decap</code> of a NSH packet
> > > > > > raises
> > > > > > this error
> > > > > > -        if the NSH inner packet is not Ethernet, IPv4, IPv6, or
> > > > > > NSH,
> > > > > > and
> > > > > > -        <code>decap</code> of other types of packets is
> > > > > > unsupported and
> > > > > > also
> > > > > > -        raises this error.
> > > > > > +        The <code>decap</code> action is supported only
> > > > > > for packet
> > > > > > types
> > > > > > +        ethernet, NSH and MPLS. Openvswitch raises this error
> > > > > > for other
> > > > > > +        packet types. When a <code>decap</code> action
> > > > > > decapsulates a
> > > > > > packet,
> > > > > > +        Open vSwitch raises this error if it does not support
> > > > > > the type
> > > > > > of inner
> > > > > > +        packet. <code>decap</code> of an Ethernet header raises
> > > > > > this
> > > > > > error if a
> > > > > > +        VLAN header is present, <code>decap</code> of a
> > > > > > NSH packet
> > > > > > raises this
> > > > > > +        error if the NSH inner packet is not Ethernet, IPv4,
> > > > > > IPv6, or
> > > > > > NSH.
> > > > > >        </p>
> > > > > > 
> > > > > >        <p>
> > > > > > @@ -1097,6 +1097,8 @@ for <var>i</var> in
> > > > > > [1,<var>n_members</var>]:
> > > > > >        <h2>The <code>encap</code> action</h2>
> > > > > >        <syntax><code>encap(nsh(</code>[<code>md_type=<var>md_type</var></code>]<code>, </code>[<code>tlv(<var>class</var>,<var>type</var>,<var>value</var>)</code>]...<code>))</code></syntax>
> > > > > >        <syntax><code>encap(ethernet)</code></syntax>
> > > > > > +
> > > > > > <syntax><code>encap(mpls(ether_type=<var>ether_type</var>))</code>
> > > > > > +      </syntax>
> > > > > > 
> > > > > >        <p>
> > > > > >          The <code>encap</code> action encapsulates a
> > > > > > packet with a
> > > > > > specified
> > > > > > @@ -1135,6 +1137,12 @@ for <var>i</var> in
> > > > > > [1,<var>n_members</var>]:
> > > > > >          source and destination are initially zeroed.
> > > > > >        </p>
> > > > > > 
> > > > > > +      <p>
> > > > > > +        The <code>encap(mpls(ethertype=....))</code> variant
> > > > > > encapsulates an
> > > > > > +        ethernet or L3 packet with a MPLS header. The
> > > > > > <var>ethertype</var>
> > > > > > +        could be MPLS unicast (0x8847) or multicast (0x8848)
> > > > > > ethertypes.
> > > > > > +      </p>
> > > > > > +
> > > > > >        <conformance>
> > > > > >          This action is an Open vSwitch extension to OpenFlow
> > > > > > 1.3 and
> > > > > > later,
> > > > > >          introduced in Open vSwitch 2.8.
> > > > > > @@ -1144,6 +1152,9 @@ for <var>i</var> in
> > > > > > [1,<var>n_members</var>]:
> > > > > >      <action name="DECAP">
> > > > > >        <h2>The <code>decap</code> action</h2>
> > > > > >        <syntax><code>decap</code></syntax>
> > > > > > +      <syntax><code>decap(packet_type(ns=<var>name_space</var>,
> > > > > > +      type=<var>ethertype</var>))</code></syntax> for
> > > > > > decapsulating
> > > > > > MPLS
> > > > > > +      packets.
> > > > > > 
> > > > > >        <p>
> > > > > >          Removes an outermost encapsulation from the packet:
> > > > > > @@ -1164,6 +1175,12 @@ for <var>i</var> in
> > > > > > [1,<var>n_members</var>]:
> > > > > >            packet type errors.
> > > > > >          </li>
> > > > > > 
> > > > > > +        <li>
> > > > > > +          Otherwise, if the packet is a MPLS packet, removes
> > > > > > the MPLS
> > > > > > header
> > > > > > +          and classifies the inner packet as mentioned in the
> > > > > > packet
> > > > > > type
> > > > > > +          argument of the decap.
> > > > > > +        </li>
> > > > > > +
> > > > > >          <li>
> > > > > >            Otherwise, raises an unsupported packet type error.
> > > > > >          </li>
> > > > > > diff --git a/lib/packets.c b/lib/packets.c
> > > > > > index 4a7643c5d..5e3c3900f 100644
> > > > > > --- a/lib/packets.c
> > > > > > +++ b/lib/packets.c
> > > > > > @@ -418,6 +418,38 @@ push_mpls(struct dp_packet *packet, ovs_be16
> > > > > > ethtype, ovs_be32 lse)
> > > > > >      pkt_metadata_init_conn(&packet->md);
> > > > > >  }
> > > > > > 
> > > > > > +void
> > > > > > +add_mpls(struct dp_packet *packet, ovs_be16 ethtype, ovs_be32
> > > > > > lse, bool
> > > > > > l3)
> > > > > > +{
> > > > > > +    char * header;
> > > > > > +
> > > > > > +    if (!eth_type_mpls(ethtype)) {
> > > > > > +        return;
> > > > > > +    }
> > > > > > +
> > > > > > +    if (!l3) {
> > > > > > +        header =  dp_packet_push_uninit(packet, MPLS_HLEN);
> > > > > > +        memcpy(header, &lse, sizeof lse);
> > > > > > +        packet->l2_5_ofs = 0;
> > > > > > +        packet->packet_type = htonl(PT_MPLS);
> > > > > > +    } else {
> > > > > > +        size_t len;
> > > > > > +
> > > > > > +        if (!is_mpls(packet)) {
> > > > > > +            /* Set MPLS label stack offset. */
> > > > > > +            packet->l2_5_ofs = packet->l3_ofs;
> > > > > > +        }
> > > > > > +        set_ethertype(packet, ethtype);
> > > > > > +
> > > > > > +        /* Push new MPLS shim header onto packet. */
> > > > > > +        len = packet->l2_5_ofs;
> > > > > > +        header = dp_packet_resize_l2_5(packet, MPLS_HLEN);
> > > > > > +        memmove(header, header + MPLS_HLEN, len);
> > > > > > +        memcpy(header + len, &lse, sizeof lse);
> > > > > > +    }
> > > > > > +    pkt_metadata_init_conn(&packet->md);
> > > > > > +}
> > > > > > +
> > > > > >  /* If 'packet' is an MPLS packet, removes its outermost
> > > > > > MPLS label
> > > > > > stack entry.
> > > > > >   * If the label that was removed was the only MPLS label, changes
> > > > > > 'packet''s
> > > > > >   * Ethertype to 'ethtype' (which ordinarily should not be an MPLS
> > > > > > @@ -429,6 +461,10 @@ pop_mpls(struct dp_packet *packet, ovs_be16
> > > > > > ethtype)
> > > > > >          struct mpls_hdr *mh = dp_packet_l2_5(packet);
> > > > > >          size_t len = packet->l2_5_ofs;
> > > > > > 
> > > > > > +        if (ethtype == htons(ETH_TYPE_TEB)) {
> > > > > > +             packet->packet_type = htonl(PT_ETH);
> > > > > > +        }
> > > > > > +
> > > > > >          set_ethertype(packet, ethtype);
> > > > > >          if (get_16aligned_be32(&mh->mpls_lse) &
> > > > > > htonl(MPLS_BOS_MASK)) {
> > > > > >              dp_packet_set_l2_5(packet, NULL);
> > > > > > diff --git a/lib/packets.h b/lib/packets.h
> > > > > > index 481bc22fa..3f5862e08 100644
> > > > > > --- a/lib/packets.h
> > > > > > +++ b/lib/packets.h
> > > > > > @@ -350,6 +350,8 @@ void set_mpls_lse_label(ovs_be32
> > > > > > *lse, ovs_be32
> > > > > > label);
> > > > > >  void set_mpls_lse_bos(ovs_be32 *lse, uint8_t bos);
> > > > > >  ovs_be32 set_mpls_lse_values(uint8_t ttl, uint8_t tc,
> > > > > > uint8_t bos,
> > > > > >                               ovs_be32 label);
> > > > > > +void add_mpls(struct dp_packet *packet, ovs_be16 ethtype,
> > > > > > ovs_be32 lse,
> > > > > > +              bool l3_flag);
> > > > > > 
> > > > > >  /* Example:
> > > > > >   *
> > > > > > diff --git a/ofproto/ofproto-dpif-ipfix.c
> > > > > > b/ofproto/ofproto-dpif-ipfix.c
> > > > > > index 796eb6f88..9280e008e 100644
> > > > > > --- a/ofproto/ofproto-dpif-ipfix.c
> > > > > > +++ b/ofproto/ofproto-dpif-ipfix.c
> > > > > > @@ -3018,6 +3018,7 @@ dpif_ipfix_read_actions(const struct flow
> > > > > > *flow,
> > > > > >          case OVS_ACTION_ATTR_CHECK_PKT_LEN:
> > > > > >          case OVS_ACTION_ATTR_UNSPEC:
> > > > > >          case OVS_ACTION_ATTR_DROP:
> > > > > > +        case OVS_ACTION_ATTR_ADD_MPLS:
> > > > > >          case __OVS_ACTION_ATTR_MAX:
> > > > > >          default:
> > > > > >              break;
> > > > > > diff --git a/ofproto/ofproto-dpif-sflow.c
> > > > > > b/ofproto/ofproto-dpif-sflow.c
> > > > > > index fdcb9eabb..ca46a9bc4 100644
> > > > > > --- a/ofproto/ofproto-dpif-sflow.c
> > > > > > +++ b/ofproto/ofproto-dpif-sflow.c
> > > > > > @@ -1226,6 +1226,7 @@ dpif_sflow_read_actions(const struct flow
> > > > > > *flow,
> > > > > >          case OVS_ACTION_ATTR_UNSPEC:
> > > > > >          case OVS_ACTION_ATTR_CHECK_PKT_LEN:
> > > > > >          case OVS_ACTION_ATTR_DROP:
> > > > > > +        case OVS_ACTION_ATTR_ADD_MPLS:
> > > > > >          case __OVS_ACTION_ATTR_MAX:
> > > > > >          default:
> > > > > >              break;
> > > > > > diff --git a/ofproto/ofproto-dpif-xlate.c
> > > > > > b/ofproto/ofproto-dpif-xlate.c
> > > > > > index 7108c8a30..a97534233 100644
> > > > > > --- a/ofproto/ofproto-dpif-xlate.c
> > > > > > +++ b/ofproto/ofproto-dpif-xlate.c
> > > > > > @@ -6381,6 +6381,45 @@
> > > > > > rewrite_flow_encap_ethernet(struct xlate_ctx
> > > > > > *ctx,
> > > > > >          ctx->error = XLATE_UNSUPPORTED_PACKET_TYPE;
> > > > > >      }
> > > > > >  }
> > > > > > +static void
> > > > > > +rewrite_flow_encap_mpls(struct xlate_ctx *ctx,
> > > > > > +                        const struct ofpact_encap *encap,
> > > > > > +                        struct flow *flow,
> > > > > > +                        struct flow_wildcards *wc)
> > > > > > +{
> > > > > > +    int n;
> > > > > > +    uint32_t i;
> > > > > > +    uint16_t ether_type;
> > > > > > +    const char *ptr = (char *) encap->props;
> > > > > > +
> > > > > > +     for (i = 0; i < encap->n_props; i++) {
> > > > > > +        struct ofpact_ed_prop *prop_ptr =
> > > > > > +            ALIGNED_CAST(struct ofpact_ed_prop *, ptr);
> > > > > > +        if (prop_ptr->prop_class == OFPPPC_MPLS) {
> > > > > > +            switch (prop_ptr->type) {
> > > > > > +                case OFPPPT_PROP_MPLS_ETHERTYPE: {
> > > > > > +                     struct ofpact_ed_prop_mpls_ethertype
> > > > > > *prop_ether_type =
> > > > > > +                        ALIGNED_CAST(struct
> > > > > > ofpact_ed_prop_mpls_ethertype *,
> > > > > > +                                     prop_ptr);
> > > > > > +                    ether_type = prop_ether_type->ether_type;
> > > > > > +                    break;
> > > > > > +                 }
> > > > > > +            }
> > > > > > +        }
> > > > > > +     }
> > > > > > +
> > > > > > +    wc->masks.packet_type = OVS_BE32_MAX;
> > > > > > +    if (flow->packet_type != htonl(PT_MPLS)) {
> > > > > > +        memset(&ctx->wc->masks.mpls_lse, 0x0,
> > > > > > +               sizeof *wc->masks.mpls_lse *
> > > > > > FLOW_MAX_MPLS_LABELS);
> > > > > > +        memset(&flow->mpls_lse, 0x0, sizeof *flow->mpls_lse *
> > > > > > +               FLOW_MAX_MPLS_LABELS);
> > > > > > +    }
> > > > > > +    flow->packet_type = htonl(PT_MPLS);
> > > > > > +    n = flow_count_mpls_labels(flow, ctx->wc);
> > > > > > +    flow_push_mpls(flow, n, htons(ether_type), ctx->wc, true);
> > > > > > +}
> > > > > > +
> > > > > > 
> > > > > >  /* For an MD2 NSH header returns a pointer to an ofpbuf with the
> > > > > > encoded
> > > > > >   * MD2 TLVs provided as encap properties to the encap
> > > > > > operation. This
> > > > > > @@ -6513,6 +6552,12 @@ xlate_generic_encap_action(struct
> > > > > > xlate_ctx *ctx,
> > > > > >          case PT_NSH:
> > > > > >              encap_data = rewrite_flow_push_nsh(ctx, encap,
> > > > > > flow, wc);
> > > > > >              break;
> > > > > > +        case PT_MPLS:
> > > > > > +            rewrite_flow_encap_mpls(ctx, encap,  flow, wc);
> > > > > > +            if (!ctx->xbridge->support.add_mpls) {
> > > > > > +                ctx->xout->slow |= SLOW_ACTION;
> > > > > > +            }
> > > > > > +            break;
> > > > > >          default:
> > > > > >              /* New packet type was checked during decoding. */
> > > > > >              OVS_NOT_REACHED();
> > > > > > @@ -6582,6 +6627,21 @@ xlate_generic_decap_action(struct
> > > > > > xlate_ctx *ctx,
> > > > > >              ctx->pending_decap = true;
> > > > > >              /* Trigger recirculation. */
> > > > > >              return true;
> > > > > > +        case PT_MPLS: {
> > > > > > +             int n;
> > > > > > +             ovs_be16 ethertype;
> > > > > > +
> > > > > > +             flow->packet_type = decap->new_pkt_type;
> > > > > > +             ethertype = pt_ns_type_be(flow->packet_type);
> > > > > > +
> > > > > > +             n = flow_count_mpls_labels(flow, ctx->wc);
> > > > > > +             flow_pop_mpls(flow, n, ethertype, ctx->wc);
> > > > > > +             if (!ctx->xbridge->support.add_mpls) {
> > > > > > +                ctx->xout->slow |= SLOW_ACTION;
> > > > > > +             }
> > > > > > +             ctx->pending_decap = true;
> > > > > > +             return true;
> > > > > > +        }
> > > > > >          default:
> > > > > >              /* Error handling: drop packet. */
> > > > > >              xlate_report_debug(
> > > > > > diff --git a/ofproto/ofproto-dpif.c b/ofproto/ofproto-dpif.c
> > > > > > index fd0b2fdea..d9a2922e7 100644
> > > > > > --- a/ofproto/ofproto-dpif.c
> > > > > > +++ b/ofproto/ofproto-dpif.c
> > > > > > @@ -1520,6 +1520,44 @@ check_nd_extensions(struct dpif_backer
> > > > > > *backer)
> > > > > > 
> > > > > >      return !error;
> > > > > >  }
> > > > > > +/* Tests whether 'backer''s datapath supports the
> > > > > > + * OVS_ACTION_ATTR_ADD_MPLS action. */
> > > > > > +static bool
> > > > > > +check_add_mpls(struct dpif_backer *backer)
> > > > > > +{
> > > > > > +    struct odputil_keybuf keybuf;
> > > > > > +    struct ofpbuf actions;
> > > > > > +    struct ofpbuf key;
> > > > > > +    struct flow flow;
> > > > > > +    bool supported;
> > > > > > +
> > > > > > +    struct odp_flow_key_parms odp_parms = {
> > > > > > +        .flow = &flow,
> > > > > > +        .probe = true,
> > > > > > +    };
> > > > > > +
> > > > > > +    memset(&flow, 0, sizeof flow);
> > > > > > +    ofpbuf_use_stack(&key, &keybuf, sizeof keybuf);
> > > > > > +    odp_flow_key_from_flow(&odp_parms, &key);
> > > > > > +    ofpbuf_init(&actions, 64);
> > > > > > +
> > > > > > +    struct ovs_action_add_mpls *mpls;
> > > > > > +
> > > > > > +    mpls = nl_msg_put_unspec_zero(&actions,
> > > > > > +                                  OVS_ACTION_ATTR_ADD_MPLS,
> > > > > > +                                  sizeof *mpls);
> > > > > > +    mpls->mpls_ethertype = htons(ETH_TYPE_MPLS);
> > > > > > +
> > > > > > +    supported = dpif_probe_feature(backer->dpif,
> > > > > > "add_mpls", &key,
> > > > > > +                                   &actions, NULL);
> > > > > > +    ofpbuf_uninit(&actions);
> > > > > > +    VLOG_INFO("%s: Datapath %s add_mpls action",
> > > > > > +              dpif_name(backer->dpif), supported ? "supports"
> > > > > > +                                                 : "does not
> > > > > > support");
> > > > > > +    return supported;
> > > > > > +
> > > > > > +}
> > > > > > +
> > > > > > 
> > > > > >  #define CHECK_FEATURE__(NAME, SUPPORT, FIELD, VALUE, ETHTYPE)
> > > > > > \
> > > > > >  static bool
> > > > > > \
> > > > > > @@ -1590,6 +1628,7 @@ check_support(struct dpif_backer *backer)
> > > > > >          dpif_supports_explicit_drop_action(backer->dpif);
> > > > > >      backer->rt_support.lb_output_action=
> > > > > >          dpif_supports_lb_output_action(backer->dpif);
> > > > > > +    backer->rt_support.add_mpls = check_add_mpls(backer);
> > > > > > 
> > > > > >      /* Flow fields. */
> > > > > >      backer->rt_support.odp.ct_state = check_ct_state(backer);
> > > > > > diff --git a/ofproto/ofproto-dpif.h b/ofproto/ofproto-dpif.h
> > > > > > index b41c3d82a..c04bfff8d 100644
> > > > > > --- a/ofproto/ofproto-dpif.h
> > > > > > +++ b/ofproto/ofproto-dpif.h
> > > > > > @@ -204,7 +204,10 @@ struct group_dpif *group_dpif_lookup(struct
> > > > > > ofproto_dpif *,
> > > > > >      DPIF_SUPPORT_FIELD(bool, explicit_drop_action, "Explicit Drop
> > > > > > action")  \
> > > > > >                                                                              \
> > > > > >      /* True if the datapath supports balance_tcp optimization */
> > > > > > \
> > > > > > -    DPIF_SUPPORT_FIELD(bool, lb_output_action, "Optimized
> > > > > > Balance TCP
> > > > > > mode")
> > > > > > +    DPIF_SUPPORT_FIELD(bool, lb_output_action, "Optimized
> > > > > > Balance TCP
> > > > > > mode")\
> > > > > > +
> > > > > > \
> > > > > > +    /* True if the datapath supports layer 2 MPLS tunnelling */
> > > > > > \
> > > > > > +    DPIF_SUPPORT_FIELD(bool, add_mpls, "l2 MPLS tunnelling")
> > > > > > 
> > > > > > 
> > > > > >  /* Stores the various features which the corresponding backer
> > > > > > supports.
> > > > > > */
> > > > > > diff --git a/tests/system-traffic.at b/tests/system-traffic.at
> > > > > > index fb5b9a36d..b865b5210 100644
> > > > > > --- a/tests/system-traffic.at
> > > > > > +++ b/tests/system-traffic.at
> > > > > > @@ -1027,9 +1027,45 @@ NS_CHECK_EXEC([at_ns0], [ping -q -c 3 -i
> > > > > > 0.3 -w 2
> > > > > > 10.1.1.2 | FORMAT_PING], [0],
> > > > > >  3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > > > >  ])
> > > > > > 
> > > > > > +OVS_TRAFFIC_VSWITCHD_STOP
> > > > > > +AT_CLEANUP
> > > > > > +
> > > > > > +
> > > > > > +AT_SETUP([datapath - ptap mpls actions])
> > > > > > +OVS_TRAFFIC_VSWITCHD_START([_ADD_BR([br1])])
> > > > > > +
> > > > > > +ADD_NAMESPACES(at_ns0, at_ns1)
> > > > > > +
> > > > > > +ADD_VETH(p0, at_ns0, br0, "10.1.1.1/24")
> > > > > > +ADD_VETH(p1, at_ns1, br1, "10.1.1.2/24")
> > > > > > +
> > > > > > +AT_CHECK([ip link add patch0 type veth peer name patch1])
> > > > > > +on_exit 'ip link del patch0'
> > > > > > +
> > > > > > +AT_CHECK([ip link set dev patch0 up])
> > > > > > +AT_CHECK([ip link set dev patch1 up])
> > > > > > +AT_CHECK([ovs-vsctl add-port br0 patch0 -- set interface patch0
> > > > > > ofport_request=100])
> > > > > > +AT_CHECK([ovs-vsctl add-port br1 patch1 -- set interface patch1
> > > > > > ofport_request=100])
> > > > > > +
> > > > > > +AT_DATA([flows.txt], [dnl
> > > > > > +table=0,priority=100,dl_type=0x0800 actions=encap(mpls(ether_type=0x8847)),set_mpls_label:2,encap(ethernet),output:100
> > > > > > +table=0,priority=100,dl_type=0x8847,mpls_label=2
> > > > > > actions=decap(),decap(packet_type(ns=0,type=0)),resubmit(,3)
> > > > > > +table=0,priority=10 actions=resubmit(,3)
> > > > > > +table=3,priority=10 actions=normal
> > > > > > +])
> > > > > > +
> > > > > > +AT_CHECK([ovs-ofctl  -Oopenflow13 add-flows br0 flows.txt])
> > > > > > +AT_CHECK([ovs-ofctl  -Oopenflow13 add-flows br1 flows.txt])
> > > > > > +
> > > > > > +NS_CHECK_EXEC([at_ns0], [ping -q -c 3 -i 0.3 -w 2 10.1.1.2 |
> > > > > > FORMAT_PING], [0], [dnl
> > > > > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > > > > +])
> > > > > > +
> > > > > > +
> > > > > >  NS_CHECK_EXEC([at_ns1], [ping -q -c 3 -i 0.3 -w 2 10.1.1.1 |
> > > > > > FORMAT_PING], [0], [dnl
> > > > > >  3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > > > >  ])
> > > > > > +
> > > > > >  OVS_TRAFFIC_VSWITCHD_STOP
> > > > > >  AT_CLEANUP
> > > > > > 
> > > > > > -- 
> > > > > > 2.18.4
> > > > > 
> > > 
>