[ovs-dev,v2,10/17] userspace: add layer 3 flow and switching support

This commit relaxes the assumption that all packets have an Ethernet
header, and adds support for layer 3 flows.  For each packet received on
the Linux kernel datapath the l2 and l3 members of struct ofpbuf are
intialized appropriately, and some functions now expect this (notably
flow_extract()), in order to differentiate between layer 2 and layer 3
packets.  struct flow has now a new 'base_layer' member, because we
cannot assume that a flow has no Ethernet header when eth_src and
eth_dst are 0.  For layer 3 packets, the protocol type is still stored
in the eth_type member.

Switching L2->L3 and L3->L2 are both implemented by adding the pop_eth
and push_eth actions respectively when a transition is detected.  The
push_eth action puts 0s on both source and destination MACs.  These
addresses can be modified with mod_dl_dst and mod_dl_src actions.

Added new prerequisite MFP_ETHERNET for fields MFF_ETH_SRC, MFF_ETH_DST,
MFF_VLAN_TCI, MFF_DL_VLAN, MFF_VLAN_VID and MFF_DL_VLAN_PCP.

L3 packets are differentiated from L2 packets by the
absence of the OVS_KEY_ATTR_ETHERNET attribute in the flow key.

Signed-off-by: Lorand Jakab <lojakab@cisco.com>
Signed-off-by: Simon Horman <simon.horman@netronome.com>
Signed-off-by: Jiri Benc <jbenc@redhat.com>
Signed-off-by: Yi Yang <yi.y.yang@intel.com>
---
 build-aux/extract-ofp-fields    |   1 +
 include/openvswitch/flow.h      |  17 +++++-
 include/openvswitch/match.h     |   1 +
 include/openvswitch/meta-flow.h |   9 +--
 include/openvswitch/ofp-print.h |   8 ++-
 lib/dp-packet.h                 |  14 ++++-
 lib/dpif-netdev.c               |   4 +-
 lib/dpif-netlink.c              |   4 ++
 lib/dpif.c                      |   7 +--
 lib/flow.c                      | 114 ++++++++++++++++++++++++-------------
 lib/flow.h                      |  11 ++++
 lib/match.c                     |   9 ++-
 lib/meta-flow.c                 |   2 +
 lib/netdev-bsd.c                |   2 +
 lib/netdev-dummy.c              |   1 +
 lib/netdev-linux.c              |   2 +
 lib/nx-match.c                  |   2 +-
 lib/odp-util.c                  | 122 +++++++++++++++++++++++++++-------------
 lib/odp-util.h                  |   2 +-
 lib/ofp-print.c                 |  27 +++++++--
 lib/ofp-util.c                  |   2 +-
 lib/packets.c                   |  15 ++++-
 lib/packets.h                   |   2 +
 ofproto/ofproto-dpif-rid.h      |   2 +-
 ofproto/ofproto-dpif-xlate.c    |  30 +++++++---
 ofproto/ofproto-dpif-xlate.h    |   2 +-
 ofproto/ofproto-dpif.c          |   2 +-
 tests/ofproto-dpif.at           |   6 +-
 tests/tunnel-push-pop-ipv6.at   |  10 ++--
 tests/tunnel-push-pop.at        |  10 ++--
 tests/tunnel.at                 |  10 ++--
 31 files changed, 312 insertions(+), 138 deletions(-)

This patch is not in line with the ongoing work to support L3 tunnels on 
legacy (non packet type-aware) OVS bridges as specified in
https://docs.google.com/document/d/1oWMYUH8sjZJzWa72o2q9kU0N6pNE-rwZcLH3-kbbDR8/edit?usp=sharing

To avoid extensive rework, we suggest to replace the patch with the 
final solution based on explicit packet_type field in struct flow.

Regards, Jan

On 2016-12-28 13:26, Yi Yang wrote:
> This commit relaxes the assumption that all packets have an Ethernet
> header, and adds support for layer 3 flows.  For each packet received on
> the Linux kernel datapath the l2 and l3 members of struct ofpbuf are
> intialized appropriately, and some functions now expect this (notably
> flow_extract()), in order to differentiate between layer 2 and layer 3
> packets.  struct flow has now a new 'base_layer' member, because we
> cannot assume that a flow has no Ethernet header when eth_src and
> eth_dst are 0.  For layer 3 packets, the protocol type is still stored
> in the eth_type member.
>
> Switching L2->L3 and L3->L2 are both implemented by adding the pop_eth
> and push_eth actions respectively when a transition is detected.  The
> push_eth action puts 0s on both source and destination MACs.  These
> addresses can be modified with mod_dl_dst and mod_dl_src actions.
>
> Added new prerequisite MFP_ETHERNET for fields MFF_ETH_SRC, MFF_ETH_DST,
> MFF_VLAN_TCI, MFF_DL_VLAN, MFF_VLAN_VID and MFF_DL_VLAN_PCP.
>
> L3 packets are differentiated from L2 packets by the
> absence of the OVS_KEY_ATTR_ETHERNET attribute in the flow key.
>
> Signed-off-by: Lorand Jakab <lojakab@cisco.com>
> Signed-off-by: Simon Horman <simon.horman@netronome.com>
> Signed-off-by: Jiri Benc <jbenc@redhat.com>
> Signed-off-by: Yi Yang <yi.y.yang@intel.com>
> ---
>   build-aux/extract-ofp-fields    |   1 +
>   include/openvswitch/flow.h      |  17 +++++-
>   include/openvswitch/match.h     |   1 +
>   include/openvswitch/meta-flow.h |   9 +--
>   include/openvswitch/ofp-print.h |   8 ++-
>   lib/dp-packet.h                 |  14 ++++-
>   lib/dpif-netdev.c               |   4 +-
>   lib/dpif-netlink.c              |   4 ++
>   lib/dpif.c                      |   7 +--
>   lib/flow.c                      | 114 ++++++++++++++++++++++++-------------
>   lib/flow.h                      |  11 ++++
>   lib/match.c                     |   9 ++-
>   lib/meta-flow.c                 |   2 +
>   lib/netdev-bsd.c                |   2 +
>   lib/netdev-dummy.c              |   1 +
>   lib/netdev-linux.c              |   2 +
>   lib/nx-match.c                  |   2 +-
>   lib/odp-util.c                  | 122 +++++++++++++++++++++++++++-------------
>   lib/odp-util.h                  |   2 +-
>   lib/ofp-print.c                 |  27 +++++++--
>   lib/ofp-util.c                  |   2 +-
>   lib/packets.c                   |  15 ++++-
>   lib/packets.h                   |   2 +
>   ofproto/ofproto-dpif-rid.h      |   2 +-
>   ofproto/ofproto-dpif-xlate.c    |  30 +++++++---
>   ofproto/ofproto-dpif-xlate.h    |   2 +-
>   ofproto/ofproto-dpif.c          |   2 +-
>   tests/ofproto-dpif.at           |   6 +-
>   tests/tunnel-push-pop-ipv6.at   |  10 ++--
>   tests/tunnel-push-pop.at        |  10 ++--
>   tests/tunnel.at                 |  10 ++--
>   31 files changed, 312 insertions(+), 138 deletions(-)
>
> diff --git a/build-aux/extract-ofp-fields b/build-aux/extract-ofp-fields
> index 8d43e4b..7f58788 100755
> --- a/build-aux/extract-ofp-fields
> +++ b/build-aux/extract-ofp-fields
> @@ -35,6 +35,7 @@ FORMATTING = {"decimal":            ("MFS_DECIMAL",      1,   8),
>                 "TCP flags":          ("MFS_TCP_FLAGS",    2,   2)}
>   
>   PREREQS = {"none": "MFP_NONE",
> +	   "Ethernet": "MFP_ETHERNET",
>              "ARP": "MFP_ARP",
>              "VLAN VID": "MFP_VLAN_VID",
>              "IPv4": "MFP_IPV4",
> diff --git a/include/openvswitch/flow.h b/include/openvswitch/flow.h
> index df80dfe..93ed37e 100644
> --- a/include/openvswitch/flow.h
> +++ b/include/openvswitch/flow.h
> @@ -23,7 +23,7 @@
>   /* This sequence number should be incremented whenever anything involving flows
>    * or the wildcarding of flows changes.  This will cause build assertion
>    * failures in places which likely need to be updated. */
> -#define FLOW_WC_SEQ 36
> +#define FLOW_WC_SEQ 37
>   
>   /* Number of Open vSwitch extension 32-bit registers. */
>   #define FLOW_N_REGS 16
> @@ -58,6 +58,11 @@ BUILD_ASSERT_DECL(FLOW_TNL_F_OAM == NX_TUN_FLAG_OAM);
>   
>   const char *flow_tun_flag_to_string(uint32_t flags);
>   
> +enum base_layer {
> +    LAYER_2 = 0,
> +    LAYER_3 = 1
> +};
> +
>   /* Maximum number of supported MPLS labels. */
>   #define FLOW_MAX_MPLS_LABELS 3
>   
> @@ -77,6 +82,10 @@ const char *flow_tun_flag_to_string(uint32_t flags);
>    * lower layer fields are first used to determine if the later fields need to
>    * be looked at.  This enables better wildcarding for datapath flows.
>    *
> + * The starting layer is specified by 'base_layer'.  When 'base_layer' is
> + * LAYER_3, dl_src, dl_tci, and vlan_tci are not used for matching. The
> + * dl_type field is still used to specify the layer 3 protocol.
> + *
>    * NOTE: Order of the fields is significant, any change in the order must be
>    * reflected in miniflow_extract()!
>    */
> @@ -98,6 +107,8 @@ struct flow {
>       ovs_u128 ct_label;          /* Connection label. */
>       uint32_t conj_id;           /* Conjunction ID. */
>       ofp_port_t actset_output;   /* Output port in action set. */
> +    uint8_t base_layer;         /* Fields start at this layer */
> +    uint8_t pad2[7];            /* Pad to 64 bits. */
>   
>       /* L2, Order the same as in the Ethernet header! (64-bit aligned) */
>       struct eth_addr dl_dst;     /* Ethernet destination address. */
> @@ -135,8 +146,8 @@ BUILD_ASSERT_DECL(sizeof(struct flow_tnl) % sizeof(uint64_t) == 0);
>   
>   /* Remember to update FLOW_WC_SEQ when changing 'struct flow'. */
>   BUILD_ASSERT_DECL(offsetof(struct flow, igmp_group_ip4) + sizeof(uint32_t)
> -                  == sizeof(struct flow_tnl) + 248
> -                  && FLOW_WC_SEQ == 36);
> +                  == sizeof(struct flow_tnl) + 256
> +                  && FLOW_WC_SEQ == 37);
>   
>   /* Incremental points at which flow classification may be performed in
>    * segments.
> diff --git a/include/openvswitch/match.h b/include/openvswitch/match.h
> index 0b5f050..28cc3b2 100644
> --- a/include/openvswitch/match.h
> +++ b/include/openvswitch/match.h
> @@ -99,6 +99,7 @@ void match_set_ct_mark(struct match *, uint32_t ct_mark);
>   void match_set_ct_mark_masked(struct match *, uint32_t ct_mark, uint32_t mask);
>   void match_set_ct_label(struct match *, ovs_u128 ct_label);
>   void match_set_ct_label_masked(struct match *, ovs_u128 ct_label, ovs_u128 mask);
> +void match_set_base_layer(struct match *, uint8_t base_layer);
>   void match_set_skb_priority(struct match *, uint32_t skb_priority);
>   void match_set_dl_type(struct match *, ovs_be16);
>   void match_set_dl_src(struct match *, const struct eth_addr );
> diff --git a/include/openvswitch/meta-flow.h b/include/openvswitch/meta-flow.h
> index b091c1b..83033d3 100644
> --- a/include/openvswitch/meta-flow.h
> +++ b/include/openvswitch/meta-flow.h
> @@ -1073,7 +1073,7 @@ enum OVS_PACKED_ENUM mf_field_id {
>        * Type: be16.
>        * Maskable: bitwise.
>        * Formatting: hexadecimal.
> -     * Prerequisites: none.
> +     * Prerequisites: Ethernet.
>        * Access: read/write.
>        * NXM: NXM_OF_VLAN_TCI(4) since v1.1.
>        * OXM: none.
> @@ -1089,7 +1089,7 @@ enum OVS_PACKED_ENUM mf_field_id {
>        * Type: be16 (low 12 bits).
>        * Maskable: no.
>        * Formatting: decimal.
> -     * Prerequisites: none.
> +     * Prerequisites: Ethernet.
>        * Access: read/write.
>        * NXM: none.
>        * OXM: none.
> @@ -1107,7 +1107,7 @@ enum OVS_PACKED_ENUM mf_field_id {
>        * Type: be16 (low 12 bits).
>        * Maskable: bitwise.
>        * Formatting: decimal.
> -     * Prerequisites: none.
> +     * Prerequisites: Ethernet.
>        * Access: read/write.
>        * NXM: none.
>        * OXM: OXM_OF_VLAN_VID(6) since OF1.2 and v1.7.
> @@ -1123,7 +1123,7 @@ enum OVS_PACKED_ENUM mf_field_id {
>        * Type: u8 (low 3 bits).
>        * Maskable: no.
>        * Formatting: decimal.
> -     * Prerequisites: none.
> +     * Prerequisites: Ethernet.
>        * Access: read/write.
>        * NXM: none.
>        * OXM: none.
> @@ -1866,6 +1866,7 @@ enum OVS_PACKED_ENUM mf_prereqs {
>       MFP_NONE,
>   
>       /* L2 requirements. */
> +    MFP_ETHERNET,
>       MFP_ARP,
>       MFP_VLAN_VID,
>       MFP_IPV4,
> diff --git a/include/openvswitch/ofp-print.h b/include/openvswitch/ofp-print.h
> index 58fd403..dce80a7 100644
> --- a/include/openvswitch/ofp-print.h
> +++ b/include/openvswitch/ofp-print.h
> @@ -21,6 +21,9 @@
>   
>   #include <stdint.h>
>   #include <stdio.h>
> +#include <stdbool.h>
> +
> +#include <openvswitch/types.h>
>   
>   struct ds;
>   struct ofp10_match;
> @@ -29,6 +32,7 @@ struct ofp_header;
>   struct ofputil_flow_stats;
>   struct ofputil_table_features;
>   struct ofputil_table_stats;
> +struct dp_packet;
>   
>   #ifdef  __cplusplus
>   extern "C" {
> @@ -41,7 +45,9 @@ void ofp10_match_print(struct ds *, const struct ofp10_match *, int verbosity);
>   
>   char *ofp_to_string(const void *, size_t, int verbosity);
>   char *ofp10_match_to_string(const struct ofp10_match *, int verbosity);
> -char *ofp_packet_to_string(const void *data, size_t len);
> +char *ofp_packet_to_string(const void *data, size_t len,
> +			   ovs_be16 packet_ethertype);
> +char *ofp_dp_packet_to_string(const struct dp_packet *);
>   
>   void ofp_print_flow_stats(struct ds *, struct ofputil_flow_stats *);
>   void ofp_print_version(const struct ofp_header *, struct ds *);
> diff --git a/lib/dp-packet.h b/lib/dp-packet.h
> index 1469864..8a43fca 100644
> --- a/lib/dp-packet.h
> +++ b/lib/dp-packet.h
> @@ -262,12 +262,20 @@ dp_packet_equal(const struct dp_packet *a, const struct dp_packet *b)
>              !memcmp(dp_packet_data(a), dp_packet_data(b), dp_packet_size(a));
>   }
>   
> -/* Get the start of the Ethernet frame.  'l3_ofs' marks the end of the l2
> - * headers, so return NULL if it is not set. */
> +static inline bool
> +dp_packet_is_l3(const struct dp_packet *b)
> +{
> +    return b->l3_ofs == 0 || b->l2_5_ofs == 0;
> +}
> +
> +/* Get the start of the Ethernet frame.  Return NULL if 'b' is an l3 packet
> + * or if 'l3_ofs', which marks the end of the l2 headers, is not set. */
>   static inline void *
>   dp_packet_l2(const struct dp_packet *b)
>   {
> -    return (b->l3_ofs != UINT16_MAX) ? dp_packet_data(b) : NULL;
> +    return (b->l3_ofs != UINT16_MAX && !dp_packet_is_l3(b))
> +        ?  dp_packet_data(b)
> +        : NULL;
>   }
>   
>   /* Resets all layer offsets.  'l3' offset must be set before 'l2' can be
> diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
> index 488712b..9e08105 100644
> --- a/lib/dpif-netdev.c
> +++ b/lib/dpif-netdev.c
> @@ -3840,9 +3840,7 @@ dp_netdev_upcall(struct dp_netdev_pmd_thread *pmd, struct dp_packet *packet_,
>   
>           ofpbuf_init(&key, 0);
>           odp_flow_key_from_flow(&odp_parms, &key);
> -        packet_str = ofp_packet_to_string(dp_packet_data(packet_),
> -                                          dp_packet_size(packet_));
> -
> +        packet_str = ofp_dp_packet_to_string(packet_);
>           odp_flow_key_format(key.data, key.size, &ds);
>   
>           VLOG_DBG("%s: %s upcall:\n%s\n%s", dp->name,
> diff --git a/lib/dpif-netlink.c b/lib/dpif-netlink.c
> index c8b0e37..ddeaeca 100644
> --- a/lib/dpif-netlink.c
> +++ b/lib/dpif-netlink.c
> @@ -2034,6 +2034,10 @@ parse_odp_packet(const struct dpif_netlink *dpif, struct ofpbuf *buf,
>                       (char *)dp_packet_data(&upcall->packet) + sizeof(struct nlattr));
>       dp_packet_set_size(&upcall->packet, nl_attr_get_size(a[OVS_PACKET_ATTR_PACKET]));
>   
> +    if (!nl_attr_find__(upcall->key, upcall->key_len, OVS_KEY_ATTR_ETHERNET)) {
> +        dp_packet_set_l3(&upcall->packet, dp_packet_data(&upcall->packet));
> +    }
> +
>       *dp_ifindex = ovs_header->dp_ifindex;
>   
>       return 0;
> diff --git a/lib/dpif.c b/lib/dpif.c
> index 9634d44..a104d16 100644
> --- a/lib/dpif.c
> +++ b/lib/dpif.c
> @@ -1430,9 +1430,7 @@ dpif_print_packet(struct dpif *dpif, struct dpif_upcall *upcall)
>           struct ds flow;
>           char *packet;
>   
> -        packet = ofp_packet_to_string(dp_packet_data(&upcall->packet),
> -                                      dp_packet_size(&upcall->packet));
> -
> +        packet = ofp_dp_packet_to_string(&upcall->packet);
>           ds_init(&flow);
>           odp_flow_key_format(upcall->key, upcall->key_len, &flow);
>   
> @@ -1725,8 +1723,7 @@ log_execute_message(struct dpif *dpif, const struct dpif_execute *execute,
>           struct ds ds = DS_EMPTY_INITIALIZER;
>           char *packet;
>   
> -        packet = ofp_packet_to_string(dp_packet_data(execute->packet),
> -                                      dp_packet_size(execute->packet));
> +        packet = ofp_dp_packet_to_string(execute->packet);
>           ds_put_format(&ds, "%s: %sexecute ",
>                         dpif_name(dpif),
>                         (subexecute ? "sub-"
> diff --git a/lib/flow.c b/lib/flow.c
> index b6d0d15..ac22d55 100644
> --- a/lib/flow.c
> +++ b/lib/flow.c
> @@ -125,7 +125,7 @@ struct mf_ctx {
>    * away.  Some GCC versions gave warnings on ALWAYS_INLINE, so these are
>    * defined as macros. */
>   
> -#if (FLOW_WC_SEQ != 36)
> +#if (FLOW_WC_SEQ != 37)
>   #define MINIFLOW_ASSERT(X) ovs_assert(X)
>   BUILD_MESSAGE("FLOW_WC_SEQ changed: miniflow_extract() will have runtime "
>                  "assertions enabled. Consider updating FLOW_WC_SEQ after "
> @@ -516,18 +516,18 @@ parse_ipv6_ext_hdrs(const void **datap, size_t *sizep, uint8_t *nw_proto,
>       return parse_ipv6_ext_hdrs__(datap, sizep, nw_proto, nw_frag);
>   }
>   
> -/* Initializes 'flow' members from 'packet' and 'md'
> +/* Initializes 'flow' members from 'packet' and 'md'.
> + * Expects packet->l3_ofs to be set to 0 for layer 3 packets.
>    *
> - * Initializes 'packet' header l2 pointer to the start of the Ethernet
> - * header, and the layer offsets as follows:
> + * Initializes the layer offsets as follows:
>    *
>    *    - packet->l2_5_ofs to the start of the MPLS shim header, or UINT16_MAX
> - *      when there is no MPLS shim header.
> + *      when there is no MPLS shim header, or Ethernet header
>    *
> - *    - packet->l3_ofs to just past the Ethernet header, or just past the
> - *      vlan_header if one is present, to the first byte of the payload of the
> - *      Ethernet frame.  UINT16_MAX if the frame is too short to contain an
> - *      Ethernet header.
> + *    - packet->l3_ofs (if not 0) to just past the Ethernet header, or just
> + *      past the vlan_header if one is present, to the first byte of the
> + *      payload of the Ethernet frame.  UINT16_MAX if the frame is too short to
> + *      contain an Ethernet header.
>    *
>    *    - packet->l4_ofs to just past the IPv4 header, if one is present and
>    *      has at least the content used for the fields of interest for the flow,
> @@ -558,9 +558,10 @@ miniflow_extract(struct dp_packet *packet, struct miniflow *dst)
>       uint64_t *values = miniflow_values(dst);
>       struct mf_ctx mf = { FLOWMAP_EMPTY_INITIALIZER, values,
>                            values + FLOW_U64S };
> -    const char *l2;
> +    const char *frame;
>       ovs_be16 dl_type;
>       uint8_t nw_frag, nw_tos, nw_ttl, nw_proto;
> +    bool is_l3 = dp_packet_is_l3(packet);
>   
>       /* Metadata. */
>       if (flow_tnl_dst_is_set(&md->tunnel)) {
> @@ -608,23 +609,47 @@ miniflow_extract(struct dp_packet *packet, struct miniflow *dst)
>       }
>   
>       /* Initialize packet's layer pointer and offsets. */
> -    l2 = data;
> +    frame = data;
>       dp_packet_reset_offsets(packet);
>   
> -    /* Must have full Ethernet header to proceed. */
> -    if (OVS_UNLIKELY(size < sizeof(struct eth_header))) {
> -        goto out;
> -    } else {
> -        ovs_be16 vlan_tci;
> +    if (!is_l3) {
> +        /* No need to store a zero value for base_layer in the miniflow
> +         * which would cost an extra word of storage. */
> +        BUILD_ASSERT(LAYER_2 == 0);
>   
> -        /* Link layer. */
> -        ASSERT_SEQUENTIAL(dl_dst, dl_src);
> -        miniflow_push_macs(mf, dl_dst, data);
> -        /* dl_type, vlan_tci. */
> -        vlan_tci = parse_vlan(&data, &size);
> -        dl_type = parse_ethertype(&data, &size);
> +        /* Must have full Ethernet header to proceed. */
> +        if (OVS_UNLIKELY(size < sizeof(struct eth_header))) {
> +            goto out;
> +        } else {
> +            ovs_be16 vlan_tci;
> +
> +            /* Link layer. */
> +            ASSERT_SEQUENTIAL(dl_dst, dl_src);
> +            miniflow_push_macs(mf, dl_dst, data);
> +            /* dl_type, vlan_tci. */
> +            vlan_tci = parse_vlan(&data, &size);
> +            dl_type = parse_ethertype(&data, &size);
> +            miniflow_push_be16(mf, dl_type, dl_type);
> +            miniflow_push_be16(mf, vlan_tci, vlan_tci);
> +        }
> +    } else {
> +        packet->l3_ofs = 0;
> +        /* miniflow_pad_from_64(mf, base_layer);
> +         * ^^^^ This is weird. It was present in the original Simon's
> +         * patch but gives assertion failures on (offset % 8 != 0) after
> +         * rebase. If I'm reading the macros correctly, the call should not
> +         * be here now, as the structure fields shifted after rebase and
> +         * this one happens to start on 8 bytes boundary now. This means
> +         * that after another rebase this can break silently. The
> +         * miniflow_pad_from_64 call should really evaluate to nothing if
> +         * the offset is divisible by 8 instead of crashing. */
> +        miniflow_push_uint8(mf, base_layer, LAYER_3);
> +        miniflow_pad_to_64(mf, base_layer);
> +
> +        dl_type = packet->md.packet_ethertype;
> +        miniflow_pad_from_64(mf, dl_type);
>           miniflow_push_be16(mf, dl_type, dl_type);
> -        miniflow_push_be16(mf, vlan_tci, vlan_tci);
> +        miniflow_push_be16(mf, vlan_tci, 0);
>       }
>   
>       /* Parse mpls. */
> @@ -632,13 +657,13 @@ miniflow_extract(struct dp_packet *packet, struct miniflow *dst)
>           int count;
>           const void *mpls = data;
>   
> -        packet->l2_5_ofs = (char *)data - l2;
> +        packet->l2_5_ofs = (char *)data - frame;
>           count = parse_mpls(&data, &size);
>           miniflow_push_words_32(mf, mpls_lse, mpls, count);
>       }
>   
>       /* Network layer. */
> -    packet->l3_ofs = (char *)data - l2;
> +    packet->l3_ofs = (char *)data - frame;
>   
>       nw_frag = 0;
>       if (OVS_LIKELY(dl_type == htons(ETH_TYPE_IP))) {
> @@ -755,7 +780,7 @@ miniflow_extract(struct dp_packet *packet, struct miniflow *dst)
>           goto out;
>       }
>   
> -    packet->l4_ofs = (char *)data - l2;
> +    packet->l4_ofs = (char *)data - frame;
>       miniflow_push_be32(mf, nw_frag,
>                          BYTES_TO_BE32(nw_frag, nw_tos, nw_ttl, nw_proto));
>   
> @@ -869,7 +894,7 @@ flow_get_metadata(const struct flow *flow, struct match *flow_metadata)
>   {
>       int i;
>   
> -    BUILD_ASSERT_DECL(FLOW_WC_SEQ == 36);
> +    BUILD_ASSERT_DECL(FLOW_WC_SEQ == 37);
>   
>       match_init_catchall(flow_metadata);
>       if (flow->tunnel.tun_id != htonll(0)) {
> @@ -925,6 +950,10 @@ flow_get_metadata(const struct flow *flow, struct match *flow_metadata)
>       if (!ovs_u128_is_zero(flow->ct_label)) {
>           match_set_ct_label(flow_metadata, flow->ct_label);
>       }
> +
> +    if (flow->base_layer != LAYER_2) {
> +        match_set_base_layer(flow_metadata, flow->base_layer);
> +    }
>   }
>   
>   const char *ct_state_to_string(uint32_t state)
> @@ -1275,7 +1304,7 @@ void flow_wildcards_init_for_packet(struct flow_wildcards *wc,
>       memset(&wc->masks, 0x0, sizeof wc->masks);
>   
>       /* Update this function whenever struct flow changes. */
> -    BUILD_ASSERT_DECL(FLOW_WC_SEQ == 36);
> +    BUILD_ASSERT_DECL(FLOW_WC_SEQ == 37);
>   
>       if (flow_tnl_dst_is_set(&flow->tunnel)) {
>           if (flow->tunnel.flags & FLOW_TNL_F_KEY) {
> @@ -1323,10 +1352,13 @@ void flow_wildcards_init_for_packet(struct flow_wildcards *wc,
>   
>       /* actset_output wildcarded. */
>   
> -    WC_MASK_FIELD(wc, dl_dst);
> -    WC_MASK_FIELD(wc, dl_src);
> +    if (flow->base_layer == LAYER_2) {
> +        WC_MASK_FIELD(wc, dl_dst);
> +        WC_MASK_FIELD(wc, dl_src);
> +        WC_MASK_FIELD(wc, vlan_tci);
> +    }
> +
>       WC_MASK_FIELD(wc, dl_type);
> -    WC_MASK_FIELD(wc, vlan_tci);
>   
>       if (flow->dl_type == htons(ETH_TYPE_IP)) {
>           WC_MASK_FIELD(wc, nw_src);
> @@ -1393,7 +1425,7 @@ void
>   flow_wc_map(const struct flow *flow, struct flowmap *map)
>   {
>       /* Update this function whenever struct flow changes. */
> -    BUILD_ASSERT_DECL(FLOW_WC_SEQ == 36);
> +    BUILD_ASSERT_DECL(FLOW_WC_SEQ == 37);
>   
>       flowmap_init(map);
>   
> @@ -1416,15 +1448,18 @@ flow_wc_map(const struct flow *flow, struct flowmap *map)
>       FLOWMAP_SET(map, recirc_id);
>       FLOWMAP_SET(map, dp_hash);
>       FLOWMAP_SET(map, in_port);
> -    FLOWMAP_SET(map, dl_dst);
> -    FLOWMAP_SET(map, dl_src);
>       FLOWMAP_SET(map, dl_type);
> -    FLOWMAP_SET(map, vlan_tci);
>       FLOWMAP_SET(map, ct_state);
>       FLOWMAP_SET(map, ct_zone);
>       FLOWMAP_SET(map, ct_mark);
>       FLOWMAP_SET(map, ct_label);
>   
> +    if (flow->base_layer == LAYER_2) {
> +        FLOWMAP_SET(map, dl_dst);
> +        FLOWMAP_SET(map, dl_src);
> +        FLOWMAP_SET(map, vlan_tci);
> +    }
> +
>       /* Ethertype-dependent fields. */
>       if (OVS_LIKELY(flow->dl_type == htons(ETH_TYPE_IP))) {
>           FLOWMAP_SET(map, nw_src);
> @@ -1477,12 +1512,13 @@ void
>   flow_wildcards_clear_non_packet_fields(struct flow_wildcards *wc)
>   {
>       /* Update this function whenever struct flow changes. */
> -    BUILD_ASSERT_DECL(FLOW_WC_SEQ == 36);
> +    BUILD_ASSERT_DECL(FLOW_WC_SEQ == 37);
>   
>       memset(&wc->masks.metadata, 0, sizeof wc->masks.metadata);
>       memset(&wc->masks.regs, 0, sizeof wc->masks.regs);
>       wc->masks.actset_output = 0;
>       wc->masks.conj_id = 0;
> +    wc->masks.base_layer = 0;
>   }
>   
>   /* Returns true if 'wc' matches every packet, false if 'wc' fixes any bits or
> @@ -1621,7 +1657,7 @@ flow_wildcards_set_xxreg_mask(struct flow_wildcards *wc, int idx,
>   uint32_t
>   miniflow_hash_5tuple(const struct miniflow *flow, uint32_t basis)
>   {
> -    BUILD_ASSERT_DECL(FLOW_WC_SEQ == 36);
> +    BUILD_ASSERT_DECL(FLOW_WC_SEQ == 37);
>       uint32_t hash = basis;
>   
>       if (flow) {
> @@ -1668,7 +1704,7 @@ ASSERT_SEQUENTIAL(ipv6_src, ipv6_dst);
>   uint32_t
>   flow_hash_5tuple(const struct flow *flow, uint32_t basis)
>   {
> -    BUILD_ASSERT_DECL(FLOW_WC_SEQ == 36);
> +    BUILD_ASSERT_DECL(FLOW_WC_SEQ == 37);
>       uint32_t hash = basis;
>   
>       if (flow) {
> @@ -2136,7 +2172,7 @@ flow_push_mpls(struct flow *flow, int n, ovs_be16 mpls_eth_type,
>   
>           if (clear_flow_L3) {
>               /* Clear all L3 and L4 fields and dp_hash. */
> -            BUILD_ASSERT(FLOW_WC_SEQ == 36);
> +            BUILD_ASSERT(FLOW_WC_SEQ == 37);
>               memset((char *) flow + FLOW_SEGMENT_2_ENDS_AT, 0,
>                      sizeof(struct flow) - FLOW_SEGMENT_2_ENDS_AT);
>               flow->dp_hash = 0;
> diff --git a/lib/flow.h b/lib/flow.h
> index 62315bc..fa7f274 100644
> --- a/lib/flow.h
> +++ b/lib/flow.h
> @@ -875,6 +875,8 @@ pkt_metadata_from_flow(struct pkt_metadata *md, const struct flow *flow)
>       md->ct_zone = flow->ct_zone;
>       md->ct_mark = flow->ct_mark;
>       md->ct_label = flow->ct_label;
> +    md->base_layer = flow->base_layer;
> +    md->packet_ethertype = flow->dl_type;
>   }
>   
>   /* Often, during translation we need to read a value from a flow('FLOW') and
> @@ -884,6 +886,15 @@ pkt_metadata_from_flow(struct pkt_metadata *md, const struct flow *flow)
>   #define FLOW_WC_GET_AND_MASK_WC(FLOW, WC, FIELD) \
>       (((WC) ? WC_MASK_FIELD(WC, FIELD) : NULL), ((FLOW)->FIELD))
>   
> +static inline bool is_ethernet(const struct flow *flow,
> +                               struct flow_wildcards *wc)
> +{
> +    if (wc) {
> +        WC_MASK_FIELD(wc, base_layer);
> +    }
> +    return flow->base_layer == LAYER_2;
> +}
> +
>   static inline bool is_vlan(const struct flow *flow,
>                              struct flow_wildcards *wc)
>   {
> diff --git a/lib/match.c b/lib/match.c
> index 3fcaec5..c551e57 100644
> --- a/lib/match.c
> +++ b/lib/match.c
> @@ -384,6 +384,13 @@ match_set_ct_label_masked(struct match *match, ovs_u128 value, ovs_u128 mask)
>   }
>   
>   void
> +match_set_base_layer(struct match *match, uint8_t base_layer)
> +{
> +    match->flow.base_layer = base_layer;
> +    match->wc.masks.base_layer = UINT8_MAX;
> +}
> +
> +void
>   match_set_dl_type(struct match *match, ovs_be16 dl_type)
>   {
>       match->wc.masks.dl_type = OVS_BE16_MAX;
> @@ -1075,7 +1082,7 @@ match_format(const struct match *match, struct ds *s, int priority)
>   
>       int i;
>   
> -    BUILD_ASSERT_DECL(FLOW_WC_SEQ == 36);
> +    BUILD_ASSERT_DECL(FLOW_WC_SEQ == 37);
>   
>       if (priority != OFP_DEFAULT_PRIORITY) {
>           ds_put_format(s, "%spriority=%s%d,",
> diff --git a/lib/meta-flow.c b/lib/meta-flow.c
> index e25adec..487c12e 100644
> --- a/lib/meta-flow.c
> +++ b/lib/meta-flow.c
> @@ -374,6 +374,8 @@ mf_are_prereqs_ok(const struct mf_field *mf, const struct flow *flow,
>       switch (mf->prereqs) {
>       case MFP_NONE:
>           return true;
> +    case MFP_ETHERNET:
> +        return is_ethernet(flow, wc);
>       case MFP_ARP:
>           return (flow->dl_type == htons(ETH_TYPE_ARP) ||
>                   flow->dl_type == htons(ETH_TYPE_RARP));
> diff --git a/lib/netdev-bsd.c b/lib/netdev-bsd.c
> index 94c515d..7855463 100644
> --- a/lib/netdev-bsd.c
> +++ b/lib/netdev-bsd.c
> @@ -580,6 +580,7 @@ netdev_rxq_bsd_recv_pcap(struct netdev_rxq_bsd *rxq, struct dp_packet *buffer)
>   
>           if (ret > 0) {
>               dp_packet_set_size(buffer, dp_packet_size(buffer) + arg.retval);
> +            dp_packet_reset_offsets(buffer);
>               return 0;
>           }
>           if (ret == -1) {
> @@ -606,6 +607,7 @@ netdev_rxq_bsd_recv_tap(struct netdev_rxq_bsd *rxq, struct dp_packet *buffer)
>           ssize_t retval = read(rxq->fd, dp_packet_data(buffer), size);
>           if (retval >= 0) {
>               dp_packet_set_size(buffer, dp_packet_size(buffer) + retval);
> +            dp_packet_reset_offsets(buffer);
>               return 0;
>           } else if (errno != EINTR) {
>               if (errno != EAGAIN) {
> diff --git a/lib/netdev-dummy.c b/lib/netdev-dummy.c
> index d406cbc..8e66717 100644
> --- a/lib/netdev-dummy.c
> +++ b/lib/netdev-dummy.c
> @@ -273,6 +273,7 @@ dummy_packet_stream_run(struct netdev_dummy *dev, struct dummy_packet_stream *s)
>                                             dp_packet_clone(&s->rxbuf), 0);
>                   dp_packet_clear(&s->rxbuf);
>               }
> +            dp_packet_reset_offsets(&s->rxbuf);
>           } else if (retval != -EAGAIN) {
>               error = (retval < 0 ? -retval
>                        : dp_packet_size(&s->rxbuf) ? EPROTO
> diff --git a/lib/netdev-linux.c b/lib/netdev-linux.c
> index a5a9ec1..30189b4 100644
> --- a/lib/netdev-linux.c
> +++ b/lib/netdev-linux.c
> @@ -1050,6 +1050,7 @@ netdev_linux_rxq_recv_sock(int fd, struct dp_packet *buffer)
>       }
>   
>       dp_packet_set_size(buffer, dp_packet_size(buffer) + retval);
> +    dp_packet_reset_offsets(buffer);
>   
>       for (cmsg = CMSG_FIRSTHDR(&msgh); cmsg; cmsg = CMSG_NXTHDR(&msgh, cmsg)) {
>           const struct tpacket_auxdata *aux;
> @@ -1096,6 +1097,7 @@ netdev_linux_rxq_recv_tap(int fd, struct dp_packet *buffer)
>       }
>   
>       dp_packet_set_size(buffer, dp_packet_size(buffer) + retval);
> +    dp_packet_reset_offsets(buffer);
>       return 0;
>   }
>   
> diff --git a/lib/nx-match.c b/lib/nx-match.c
> index 9201aae..da2919f 100644
> --- a/lib/nx-match.c
> +++ b/lib/nx-match.c
> @@ -930,7 +930,7 @@ nx_put_raw(struct ofpbuf *b, enum ofp_version oxm, const struct match *match,
>       int match_len;
>       int i;
>   
> -    BUILD_ASSERT_DECL(FLOW_WC_SEQ == 36);
> +    BUILD_ASSERT_DECL(FLOW_WC_SEQ == 37);
>   
>       /* Metadata. */
>       if (match->wc.masks.dp_hash) {
> diff --git a/lib/odp-util.c b/lib/odp-util.c
> index a50e76e..6725294 100644
> --- a/lib/odp-util.c
> +++ b/lib/odp-util.c
> @@ -4330,7 +4330,7 @@ odp_flow_key_from_flow__(const struct odp_flow_key_parms *parms,
>                            bool export_mask, struct ofpbuf *buf)
>   {
>       struct ovs_key_ethernet *eth_key;
> -    size_t encap;
> +    size_t encap = 0;
>       const struct flow *flow = parms->flow;
>       const struct flow *data = export_mask ? parms->mask : parms->flow;
>   
> @@ -4368,41 +4368,43 @@ odp_flow_key_from_flow__(const struct odp_flow_key_parms *parms,
>           nl_msg_put_odp_port(buf, OVS_KEY_ATTR_IN_PORT, data->in_port.odp_port);
>       }
>   
> -    eth_key = nl_msg_put_unspec_uninit(buf, OVS_KEY_ATTR_ETHERNET,
> -                                       sizeof *eth_key);
> -    get_ethernet_key(data, eth_key);
> +    if (flow->base_layer == LAYER_2) {
> +        eth_key = nl_msg_put_unspec_uninit(buf, OVS_KEY_ATTR_ETHERNET,
> +                                           sizeof *eth_key);
> +        get_ethernet_key(data, eth_key);
>   
> -    if (flow->vlan_tci != htons(0) || flow->dl_type == htons(ETH_TYPE_VLAN)) {
> -        if (export_mask) {
> -            nl_msg_put_be16(buf, OVS_KEY_ATTR_ETHERTYPE, OVS_BE16_MAX);
> -        } else {
> -            nl_msg_put_be16(buf, OVS_KEY_ATTR_ETHERTYPE, htons(ETH_TYPE_VLAN));
> -        }
> -        nl_msg_put_be16(buf, OVS_KEY_ATTR_VLAN, data->vlan_tci);
> -        encap = nl_msg_start_nested(buf, OVS_KEY_ATTR_ENCAP);
> -        if (flow->vlan_tci == htons(0)) {
> -            goto unencap;
> +        if (flow->vlan_tci != htons(0) ||
> +            flow->dl_type == htons(ETH_TYPE_VLAN)) {
> +            if (export_mask) {
> +                nl_msg_put_be16(buf, OVS_KEY_ATTR_ETHERTYPE, OVS_BE16_MAX);
> +            } else {
> +                nl_msg_put_be16(buf, OVS_KEY_ATTR_ETHERTYPE,
> +                                htons(ETH_TYPE_VLAN));
> +            }
> +            nl_msg_put_be16(buf, OVS_KEY_ATTR_VLAN, data->vlan_tci);
> +            encap = nl_msg_start_nested(buf, OVS_KEY_ATTR_ENCAP);
> +            if (flow->vlan_tci == htons(0)) {
> +                goto unencap;
> +            }
>           }
> -    } else {
> -        encap = 0;
> -    }
>   
> -    if (ntohs(flow->dl_type) < ETH_TYPE_MIN) {
> -        /* For backwards compatibility with kernels that don't support
> -         * wildcarding, the following convention is used to encode the
> -         * OVS_KEY_ATTR_ETHERTYPE for key and mask:
> -         *
> -         *   key      mask    matches
> -         * -------- --------  -------
> -         *  >0x5ff   0xffff   Specified Ethernet II Ethertype.
> -         *  >0x5ff      0     Any Ethernet II or non-Ethernet II frame.
> -         *  <none>   0xffff   Any non-Ethernet II frame (except valid
> -         *                    802.3 SNAP packet with valid eth_type).
> -         */
> -        if (export_mask) {
> -            nl_msg_put_be16(buf, OVS_KEY_ATTR_ETHERTYPE, OVS_BE16_MAX);
> +        if (ntohs(flow->dl_type) < ETH_TYPE_MIN) {
> +            /* For backwards compatibility with kernels that don't support
> +             * wildcarding, the following convention is used to encode the
> +             * OVS_KEY_ATTR_ETHERTYPE for key and mask:
> +             *
> +             *   key      mask    matches
> +             * -------- --------  -------
> +             *  >0x5ff   0xffff   Specified Ethernet II Ethertype.
> +             *  >0x5ff      0     Any Ethernet II or non-Ethernet II frame.
> +             *  <none>   0xffff   Any non-Ethernet II frame (except valid
> +             *                    802.3 SNAP packet with valid eth_type).
> +             */
> +            if (export_mask) {
> +                nl_msg_put_be16(buf, OVS_KEY_ATTR_ETHERTYPE, OVS_BE16_MAX);
> +            }
> +            goto unencap;
>           }
> -        goto unencap;
>       }
>   
>       nl_msg_put_be16(buf, OVS_KEY_ATTR_ETHERTYPE, data->dl_type);
> @@ -4561,6 +4563,10 @@ odp_key_from_pkt_metadata(struct ofpbuf *buf, const struct pkt_metadata *md)
>       if (md->in_port.odp_port != ODPP_NONE) {
>           nl_msg_put_odp_port(buf, OVS_KEY_ATTR_IN_PORT, md->in_port.odp_port);
>       }
> +
> +    if (md->base_layer == LAYER_3) {
> +        nl_msg_put_be16(buf, OVS_KEY_ATTR_ETHERTYPE, md->packet_ethertype);
> +    }
>   }
>   
>   /* Generate packet metadata from the given ODP flow key. */
> @@ -4569,10 +4575,13 @@ odp_key_to_pkt_metadata(const struct nlattr *key, size_t key_len,
>                           struct pkt_metadata *md)
>   {
>       const struct nlattr *nla;
> +    ovs_be16 ethertype = 0;
>       size_t left;
>       uint32_t wanted_attrs = 1u << OVS_KEY_ATTR_PRIORITY |
>           1u << OVS_KEY_ATTR_SKB_MARK | 1u << OVS_KEY_ATTR_TUNNEL |
> -        1u << OVS_KEY_ATTR_IN_PORT;
> +        1u << OVS_KEY_ATTR_IN_PORT | 1u << OVS_KEY_ATTR_ETHERTYPE |
> +        1u << OVS_KEY_ATTR_ETHERNET | 1u << OVS_KEY_ATTR_IPV4 |
> +        1u << OVS_KEY_ATTR_IPV6;
>   
>       pkt_metadata_init(md, ODPP_NONE);
>   
> @@ -4637,14 +4646,38 @@ odp_key_to_pkt_metadata(const struct nlattr *key, size_t key_len,
>               md->in_port.odp_port = nl_attr_get_odp_port(nla);
>               wanted_attrs &= ~(1u << OVS_KEY_ATTR_IN_PORT);
>               break;
> +        case OVS_KEY_ATTR_ETHERNET:
> +            wanted_attrs &= ~(1u << OVS_KEY_ATTR_ETHERNET);
> +            break;
> +        case OVS_KEY_ATTR_ETHERTYPE:
> +            ethertype = nl_attr_get_be16(nla);
> +            wanted_attrs &= ~(1u << OVS_KEY_ATTR_ETHERTYPE);
> +            break;
> +        case OVS_KEY_ATTR_IPV4:
> +            wanted_attrs &= ~(1u << OVS_KEY_ATTR_IPV4);
> +            break;
> +        case OVS_KEY_ATTR_IPV6:
> +            wanted_attrs &= ~(1u << OVS_KEY_ATTR_IPV6);
> +            break;
>           default:
>               break;
>           }
>   
>           if (!wanted_attrs) {
> -            return; /* Have everything. */
> +            break; /* Have everything. */
>           }
>       }
> +
> +    /* OVS_KEY_ATTR_ETHERTYPE present and OVS_KEY_ATTR_ETHERNET absent
> +     * indicates Layer 3. */
> +    if (!(wanted_attrs & (1u << OVS_KEY_ATTR_ETHERTYPE)) &&
> +        wanted_attrs & (1u << OVS_KEY_ATTR_ETHERNET)) {
> +        md->base_layer = LAYER_3;
> +        md->packet_ethertype = ethertype;
> +    } else {
> +        md->base_layer = LAYER_2;
> +    }
> +
>   }
>   
>   uint32_t
> @@ -4808,7 +4841,15 @@ parse_ethertype(const struct nlattr *attrs[OVS_KEY_ATTR_MAX + 1],
>           *expected_attrs |= UINT64_C(1) << OVS_KEY_ATTR_ETHERTYPE;
>       } else {
>           if (!is_mask) {
> -            flow->dl_type = htons(FLOW_DL_TYPE_NONE);
> +            if (present_attrs & (UINT64_C(1) << OVS_KEY_ATTR_IPV4)) {
> +                flow->dl_type = htons(ETH_TYPE_IP);
> +            } else if (present_attrs & (UINT64_C(1) << OVS_KEY_ATTR_IPV6)) {
> +                flow->dl_type = htons(ETH_TYPE_IPV6);
> +            } else {
> +                flow->dl_type = htons(FLOW_DL_TYPE_NONE);
> +            }
> +        } else if (src_flow->base_layer == LAYER_3) {
> +            flow->dl_type = htons(0xffff);
>           } else if (ntohs(src_flow->dl_type) < ETH_TYPE_MIN) {
>               /* See comments in odp_flow_key_from_flow__(). */
>               VLOG_ERR_RL(&rl, "mask expected for non-Ethernet II frame");
> @@ -5223,12 +5264,13 @@ odp_flow_key_to_flow__(const struct nlattr *key, size_t key_len,
>   
>           eth_key = nl_attr_get(attrs[OVS_KEY_ATTR_ETHERNET]);
>           put_ethernet_key(eth_key, flow);
> -        if (is_mask) {
> -            expected_attrs |= UINT64_C(1) << OVS_KEY_ATTR_ETHERNET;
> -        }
> -    }
> -    if (!is_mask) {
> +        flow->base_layer = LAYER_2;
>           expected_attrs |= UINT64_C(1) << OVS_KEY_ATTR_ETHERNET;
> +    } else if (present_attrs & (UINT64_C(1) << OVS_KEY_ATTR_ETHERTYPE)) {
> +        flow->base_layer = LAYER_3;
> +        expected_attrs |= UINT64_C(1) << OVS_KEY_ATTR_ETHERTYPE;
> +    } else if (is_mask && src_flow->base_layer == LAYER_3) {
> +        flow->base_layer = LAYER_3;
>       }
>   
>       /* Get Ethertype or 802.1Q TPID or FLOW_DL_TYPE_NONE. */
> diff --git a/lib/odp-util.h b/lib/odp-util.h
> index 42011bc..f391e2a 100644
> --- a/lib/odp-util.h
> +++ b/lib/odp-util.h
> @@ -142,7 +142,7 @@ void odp_portno_names_destroy(struct hmap *portno_names);
>    * add another field and forget to adjust this value.
>    */
>   #define ODPUTIL_FLOW_KEY_BYTES 640
> -BUILD_ASSERT_DECL(FLOW_WC_SEQ == 36);
> +BUILD_ASSERT_DECL(FLOW_WC_SEQ == 37);
>   
>   /* A buffer with sufficient size and alignment to hold an nlattr-formatted flow
>    * key.  An array of "struct nlattr" might not, in theory, be sufficiently
> diff --git a/lib/ofp-print.c b/lib/ofp-print.c
> index 7b7c430..d5d3403 100644
> --- a/lib/ofp-print.c
> +++ b/lib/ofp-print.c
> @@ -55,10 +55,10 @@
>   static void ofp_print_queue_name(struct ds *string, uint32_t port);
>   static void ofp_print_error(struct ds *, enum ofperr);
>   
> -/* Returns a string that represents the contents of the Ethernet frame in the
> +/* Returns a string that represents the contents of the packet in the
>    * 'len' bytes starting at 'data'.  The caller must free the returned string.*/
>   char *
> -ofp_packet_to_string(const void *data, size_t len)
> +ofp_packet_to_string(const void *data, size_t len, ovs_be16 packet_ethertype)
>   {
>       struct ds ds = DS_EMPTY_INITIALIZER;
>       struct dp_packet buf;
> @@ -66,6 +66,11 @@ ofp_packet_to_string(const void *data, size_t len)
>       size_t l4_size;
>   
>       dp_packet_use_const(&buf, data, len);
> +    if (packet_ethertype) {
> +        /* This is a layer 3 packet */
> +        buf.md.packet_ethertype = packet_ethertype;
> +        buf.l3_ofs = 0;
> +    }
>       flow_extract(&buf, &flow);
>       flow_format(&ds, &flow);
>   
> @@ -96,6 +101,17 @@ ofp_packet_to_string(const void *data, size_t len)
>       return ds_cstr(&ds);
>   }
>   
> +/* Returns a string that represents the contents of the packet in the
> + * 'len' bytes starting at 'data'.  The caller must free the returned string.*/
> +char *
> +ofp_dp_packet_to_string(const struct dp_packet *p)
> +{
> +    ovs_assert(!dp_packet_is_l3(p) || ntohs(p->md.packet_ethertype));
> +    return ofp_packet_to_string(dp_packet_data(p), dp_packet_size(p),
> +                                dp_packet_is_l3(p) ? p->md.packet_ethertype
> +                                : htons(0));
> +}
> +
>   static void
>   format_hex_arg(struct ds *s, const uint8_t *data, size_t len)
>   {
> @@ -200,7 +216,7 @@ ofp_print_packet_in(struct ds *string, const struct ofp_header *oh,
>   
>       if (verbosity > 0) {
>           char *packet = ofp_packet_to_string(public->packet,
> -                                            public->packet_len);
> +                                            public->packet_len, htons(0));
>           ds_put_cstr(string, packet);
>           free(packet);
>       }
> @@ -236,7 +252,8 @@ ofp_print_packet_out(struct ds *string, const struct ofp_header *oh,
>       if (po.buffer_id == UINT32_MAX) {
>           ds_put_format(string, " data_len=%"PRIuSIZE, po.packet_len);
>           if (verbosity > 0 && po.packet_len > 0) {
> -            char *packet = ofp_packet_to_string(po.packet, po.packet_len);
> +            char *packet = ofp_packet_to_string(po.packet, po.packet_len,
> +                                                htons(0));
>               ds_put_char(string, '\n');
>               ds_put_cstr(string, packet);
>               free(packet);
> @@ -3719,5 +3736,5 @@ ofp_print(FILE *stream, const void *oh, size_t len, int verbosity)
>   void
>   ofp_print_packet(FILE *stream, const void *data, size_t len)
>   {
> -    print_and_free(stream, ofp_packet_to_string(data, len));
> +    print_and_free(stream, ofp_packet_to_string(data, len, htons(0)));
>   }
> diff --git a/lib/ofp-util.c b/lib/ofp-util.c
> index b9efd32..d5d4b7d 100644
> --- a/lib/ofp-util.c
> +++ b/lib/ofp-util.c
> @@ -101,7 +101,7 @@ ofputil_netmask_to_wcbits(ovs_be32 netmask)
>   void
>   ofputil_wildcard_from_ofpfw10(uint32_t ofpfw, struct flow_wildcards *wc)
>   {
> -    BUILD_ASSERT_DECL(FLOW_WC_SEQ == 36);
> +    BUILD_ASSERT_DECL(FLOW_WC_SEQ == 37);
>   
>       /* Initialize most of wc. */
>       flow_wildcards_init_catchall(wc);
> diff --git a/lib/packets.c b/lib/packets.c
> index 19d3bfa..97a2eee 100644
> --- a/lib/packets.c
> +++ b/lib/packets.c
> @@ -237,16 +237,24 @@ push_eth(struct dp_packet *packet, const struct eth_addr *dst,
>       eh->eth_src = *src;
>   }
>   
> -/* Removes Ethernet header, including all VLAN and MPLS headers, from 'packet'.
> +/* Removes Ethernet header, including VLAN header, from 'packet'.
>    *
>    * Previous to calling this function, 'ofpbuf_l3(packet)' must not be NULL */
>   void
>   pop_eth(struct dp_packet *packet)
>   {
> +    char *l2_5 = dp_packet_l2_5(packet);;
> +    int increment;
> +
>       ovs_assert(dp_packet_l3(packet) != NULL);
>   
> -    dp_packet_resize_l2_5(packet, -packet->l3_ofs);
> -    dp_packet_set_l2_5(packet, NULL);
> +    if (l2_5) {
> +        increment = packet->l2_5_ofs;
> +    } else {
> +        increment = packet->l3_ofs;
> +    }
> +
> +    dp_packet_resize_l2(packet, -increment);
>   }
>   
>   /* Set ethertype of the packet. */
> @@ -256,6 +264,7 @@ set_ethertype(struct dp_packet *packet, ovs_be16 eth_type)
>       struct eth_header *eh = dp_packet_l2(packet);
>   
>       if (!eh) {
> +        packet->md.packet_ethertype = eth_type;
>           return;
>       }
>   
> diff --git a/lib/packets.h b/lib/packets.h
> index 245f651..1993a3a 100644
> --- a/lib/packets.h
> +++ b/lib/packets.h
> @@ -104,6 +104,8 @@ struct pkt_metadata {
>       uint32_t ct_mark;           /* Connection mark. */
>       ovs_u128 ct_label;          /* Connection label. */
>       union flow_in_port in_port; /* Input port. */
> +    ovs_be16 packet_ethertype;  /* Ethertype of the packet */
> +    uint8_t base_layer;         /* Packet starts at this layer */
>       struct flow_tnl tunnel;     /* Encapsulating tunnel parameters. Note that
>                                    * if 'ip_dst' == 0, the rest of the fields may
>                                    * be uninitialized. */
> diff --git a/ofproto/ofproto-dpif-rid.h b/ofproto/ofproto-dpif-rid.h
> index 3bca817..f622278 100644
> --- a/ofproto/ofproto-dpif-rid.h
> +++ b/ofproto/ofproto-dpif-rid.h
> @@ -99,7 +99,7 @@ struct rule;
>   /* Metadata for restoring pipeline context after recirculation.  Helpers
>    * are inlined below to keep them together with the definition for easier
>    * updates. */
> -BUILD_ASSERT_DECL(FLOW_WC_SEQ == 36);
> +BUILD_ASSERT_DECL(FLOW_WC_SEQ == 37);
>   
>   struct frozen_metadata {
>       /* Metadata in struct flow. */
> diff --git a/ofproto/ofproto-dpif-xlate.c b/ofproto/ofproto-dpif-xlate.c
> index 2977be5..1e0bcea 100644
> --- a/ofproto/ofproto-dpif-xlate.c
> +++ b/ofproto/ofproto-dpif-xlate.c
> @@ -160,6 +160,7 @@ struct xport {
>   
>       bool may_enable;                 /* May be enabled in bonds. */
>       bool is_tunnel;                  /* Is a tunnel port. */
> +    bool is_layer3;                  /* Is a layer 3 port. */
>   
>       struct cfm *cfm;                 /* CFM handle or null. */
>       struct bfd *bfd;                 /* BFD handle or null. */
> @@ -543,7 +544,7 @@ static void xlate_xport_set(struct xport *xport, odp_port_t odp_port,
>                               int stp_port_no, const struct rstp_port *rstp_port,
>                               enum ofputil_port_config config,
>                               enum ofputil_port_state state, bool is_tunnel,
> -                            bool may_enable);
> +                            bool may_enable, bool is_layer3);
>   static void xlate_xbridge_remove(struct xlate_cfg *, struct xbridge *);
>   static void xlate_xbundle_remove(struct xlate_cfg *, struct xbundle *);
>   static void xlate_xport_remove(struct xlate_cfg *, struct xport *);
> @@ -713,12 +714,13 @@ xlate_xport_set(struct xport *xport, odp_port_t odp_port,
>                   const struct bfd *bfd, const struct lldp *lldp, int stp_port_no,
>                   const struct rstp_port* rstp_port,
>                   enum ofputil_port_config config, enum ofputil_port_state state,
> -                bool is_tunnel, bool may_enable)
> +                bool is_tunnel, bool may_enable, bool is_layer3)
>   {
>       xport->config = config;
>       xport->state = state;
>       xport->stp_port_no = stp_port_no;
>       xport->is_tunnel = is_tunnel;
> +    xport->is_layer3 = is_layer3;
>       xport->may_enable = may_enable;
>       xport->odp_port = odp_port;
>   
> @@ -809,7 +811,7 @@ xlate_xport_copy(struct xbridge *xbridge, struct xbundle *xbundle,
>       xlate_xport_set(new_xport, xport->odp_port, xport->netdev, xport->cfm,
>                       xport->bfd, xport->lldp, xport->stp_port_no,
>                       xport->rstp_port, xport->config, xport->state,
> -                    xport->is_tunnel, xport->may_enable);
> +                    xport->is_tunnel, xport->may_enable, xport->is_layer3);
>   
>       if (xport->peer) {
>           struct xport *peer = xport_lookup(new_xcfg, xport->peer->ofport);
> @@ -1047,7 +1049,7 @@ xlate_ofport_set(struct ofproto_dpif *ofproto, struct ofbundle *ofbundle,
>                    const struct ofproto_port_queue *qdscp_list, size_t n_qdscp,
>                    enum ofputil_port_config config,
>                    enum ofputil_port_state state, bool is_tunnel,
> -                 bool may_enable)
> +                 bool may_enable, bool is_layer3)
>   {
>       size_t i;
>       struct xport *xport;
> @@ -1068,7 +1070,7 @@ xlate_ofport_set(struct ofproto_dpif *ofproto, struct ofbundle *ofbundle,
>   
>       xlate_xport_set(xport, odp_port, netdev, cfm, bfd, lldp,
>                       stp_port_no, rstp_port, config, state, is_tunnel,
> -                    may_enable);
> +                    may_enable, is_layer3);
>   
>       if (xport->peer) {
>           xport->peer->peer = NULL;
> @@ -2359,7 +2361,7 @@ xlate_normal(struct xlate_ctx *ctx)
>   
>       /* Learn source MAC. */
>       bool is_grat_arp = is_gratuitous_arp(flow, wc);
> -    if (ctx->xin->allow_side_effects) {
> +    if (ctx->xin->allow_side_effects && !in_port->is_layer3) {
>           update_learning_table(ctx->xbridge, in_xbundle, flow->dl_src, vlan,
>                                 is_grat_arp);
>       }
> @@ -2921,7 +2923,7 @@ compose_output_action__(struct xlate_ctx *ctx, ofp_port_t ofp_port,
>   
>       /* If 'struct flow' gets additional metadata, we'll need to zero it out
>        * before traversing a patch port. */
> -    BUILD_ASSERT_DECL(FLOW_WC_SEQ == 36);
> +    BUILD_ASSERT_DECL(FLOW_WC_SEQ == 37);
>       memset(&flow_tnl, 0, sizeof flow_tnl);
>   
>       if (!xport) {
> @@ -2962,6 +2964,14 @@ compose_output_action__(struct xlate_ctx *ctx, ofp_port_t ofp_port,
>           }
>       }
>   
> +    if (xport->is_layer3) {
> +        if (flow->base_layer == LAYER_2) {
> +            flow->base_layer = LAYER_3;
> +        }
> +    } else if (flow->base_layer == LAYER_3) {
> +        flow->base_layer = LAYER_2;
> +    }
> +
>       if (xport->peer) {
>           const struct xport *peer = xport->peer;
>           struct flow old_flow = ctx->xin->flow;
> @@ -3648,9 +3658,14 @@ execute_controller_action(struct xlate_ctx *ctx, int len,
>                             uint16_t controller_id,
>                             const uint8_t *userdata, size_t userdata_len)
>   {
> +    struct flow *flow = &ctx->xin->flow;
>       struct dp_packet_batch batch;
>       struct dp_packet *packet;
>   
> +    if (flow->base_layer == LAYER_3) {
> +        flow->base_layer = LAYER_2;
> +    }
> +
>       ctx->xout->slow |= SLOW_CONTROLLER;
>       xlate_commit_actions(ctx);
>       if (!ctx->xin->packet) {
> @@ -3662,6 +3677,7 @@ execute_controller_action(struct xlate_ctx *ctx, int len,
>       }
>   
>       packet = dp_packet_clone(ctx->xin->packet);
> +
>       packet_batch_init_packet(&batch, packet);
>       odp_execute_actions(NULL, &batch, false,
>                           ctx->odp_actions->data, ctx->odp_actions->size, NULL);
> diff --git a/ofproto/ofproto-dpif-xlate.h b/ofproto/ofproto-dpif-xlate.h
> index 51c05ac..40e9724 100644
> --- a/ofproto/ofproto-dpif-xlate.h
> +++ b/ofproto/ofproto-dpif-xlate.h
> @@ -181,7 +181,7 @@ void xlate_ofport_set(struct ofproto_dpif *, struct ofbundle *,
>                         const struct ofproto_port_queue *qdscp,
>                         size_t n_qdscp, enum ofputil_port_config,
>                         enum ofputil_port_state, bool is_tunnel,
> -                      bool may_enable);
> +                      bool may_enable, bool is_l3);
>   void xlate_ofport_remove(struct ofport_dpif *);
>   
>   struct ofproto_dpif * xlate_lookup_ofproto(const struct dpif_backer *,
> diff --git a/ofproto/ofproto-dpif.c b/ofproto/ofproto-dpif.c
> index aa36462..a70a491 100644
> --- a/ofproto/ofproto-dpif.c
> +++ b/ofproto/ofproto-dpif.c
> @@ -636,7 +636,7 @@ type_run(const char *type)
>                                    ofport->rstp_port, ofport->qdscp,
>                                    ofport->n_qdscp, ofport->up.pp.config,
>                                    ofport->up.pp.state, ofport->is_tunnel,
> -                                 ofport->may_enable);
> +                                 ofport->may_enable, ofport->is_layer3);
>               }
>               xlate_txn_commit();
>           }
> diff --git a/tests/ofproto-dpif.at b/tests/ofproto-dpif.at
> index e35a806..5ed45e3 100644
> --- a/tests/ofproto-dpif.at
> +++ b/tests/ofproto-dpif.at
> @@ -4773,15 +4773,15 @@ in_port=2 actions=output:1
>   ])
>   AT_CHECK([ovs-ofctl add-flows br0 flows.txt])
>   
> -odp_flow="in_port(p1)"
> -br_flow="in_port=1"
> +odp_flow="in_port(p1),eth(src=00:00:00:00:00:00,dst=00:00:00:00:00:00)"
> +br_flow="in_port=1,dl_dst=00:00:00:00:00:00"
>   # Test command: ofproto/trace odp_flow with in_port as a name.
>   AT_CHECK([ovs-appctl ofproto/trace "$odp_flow"], [0], [stdout])
>   AT_CHECK([tail -1 stdout], [0], [dnl
>   Datapath actions: 2
>   ])
>   
> -odp_flow="in_port(1)"
> +odp_flow="in_port(1),eth(src=00:00:00:00:00:00,dst=00:00:00:00:00:00)"
>   # Test command: ofproto/trace odp_flow
>   AT_CHECK([ovs-appctl ofproto/trace "$odp_flow"], [0], [stdout])
>   AT_CHECK([tail -1 stdout], [0], [dnl
> diff --git a/tests/tunnel-push-pop-ipv6.at b/tests/tunnel-push-pop-ipv6.at
> index 16dc571..3f3d5ee 100644
> --- a/tests/tunnel-push-pop-ipv6.at
> +++ b/tests/tunnel-push-pop-ipv6.at
> @@ -88,28 +88,28 @@ AT_CHECK([tail -1 stdout], [0],
>   
>   dnl Check VXLAN tunnel push
>   AT_CHECK([ovs-ofctl add-flow int-br action=2])
> -AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'in_port(2),eth_type(0x0800),ipv4(src=1.1.3.88,dst=1.1.3.112,proto=47,tos=0,ttl=64,frag=no)'], [0], [stdout])
> +AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'in_port(2),eth(src=f8:bc:12:44:34:b6,dst=aa:55:aa:55:00:00),eth_type(0x0800),ipv4(src=1.1.3.88,dst=1.1.3.112,proto=47,tos=0,ttl=64,frag=no)'], [0], [stdout])
>   AT_CHECK([tail -1 stdout], [0],
>     [Datapath actions: tnl_push(tnl_port(4789),header(size=70,type=4,eth(dst=f8:bc:12:44:34:b6,src=aa:55:aa:55:00:00,dl_type=0x86dd),ipv6(src=2001:cafe::88,dst=2001:cafe::92,label=0,proto=17,tclass=0x0,hlimit=64),udp(src=0,dst=4789,csum=0xffff),vxlan(flags=0x8000000,vni=0x7b)),out_port(100))
>   ])
>   
>   dnl Check VXLAN tunnel push set tunnel id by flow and checksum
>   AT_CHECK([ovs-ofctl add-flow int-br "actions=set_tunnel:124,4"])
> -AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'in_port(2),eth_type(0x0800),ipv4(src=1.1.3.88,dst=1.1.3.112,proto=47,tos=0,ttl=64,frag=no)'], [0], [stdout])
> +AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'in_port(2),eth(src=f8:bc:12:44:34:b6,dst=aa:55:aa:55:00:00),eth_type(0x0800),ipv4(src=1.1.3.88,dst=1.1.3.112,proto=47,tos=0,ttl=64,frag=no)'], [0], [stdout])
>   AT_CHECK([tail -1 stdout], [0],
>     [Datapath actions: tnl_push(tnl_port(4789),header(size=70,type=4,eth(dst=f8:bc:12:44:34:b7,src=aa:55:aa:55:00:00,dl_type=0x86dd),ipv6(src=2001:cafe::88,dst=2001:cafe::93,label=0,proto=17,tclass=0x0,hlimit=64),udp(src=0,dst=4789,csum=0xffff),vxlan(flags=0x8000000,vni=0x7c)),out_port(100))
>   ])
>   
>   dnl Check GRE tunnel push
>   AT_CHECK([ovs-ofctl add-flow int-br action=3])
> -AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'in_port(2),eth_type(0x0800),ipv4(src=1.1.3.88,dst=1.1.3.112,proto=47,tos=0,ttl=64,frag=no)'], [0], [stdout])
> +AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'in_port(2),eth(src=f8:bc:12:44:34:b6,dst=aa:55:aa:55:00:00),eth_type(0x0800),ipv4(src=1.1.3.88,dst=1.1.3.112,proto=47,tos=0,ttl=64,frag=no)'], [0], [stdout])
>   AT_CHECK([tail -1 stdout], [0],
>     [Datapath actions: tnl_push(tnl_port(3),header(size=62,type=3,eth(dst=f8:bc:12:44:34:b6,src=aa:55:aa:55:00:00,dl_type=0x86dd),ipv6(src=2001:cafe::88,dst=2001:cafe::92,label=0,proto=47,tclass=0x0,hlimit=64),gre((flags=0x2000,proto=0x6558),key=0x1c8)),out_port(100))
>   ])
>   
>   dnl Check Geneve tunnel push
>   AT_CHECK([ovs-ofctl add-flow int-br "actions=set_field:2001:cafe::92->tun_ipv6_dst,5"])
> -AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'in_port(2),eth_type(0x0800),ipv4(src=1.1.3.88,dst=1.1.3.112,proto=47,tos=0,ttl=64,frag=no)'], [0], [stdout])
> +AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'in_port(2),eth(src=f8:bc:12:44:34:b6,dst=aa:55:aa:55:00:00),eth_type(0x0800),ipv4(src=1.1.3.88,dst=1.1.3.112,proto=47,tos=0,ttl=64,frag=no)'], [0], [stdout])
>   AT_CHECK([tail -1 stdout], [0],
>     [Datapath actions: tnl_push(tnl_port(6081),header(size=70,type=5,eth(dst=f8:bc:12:44:34:b6,src=aa:55:aa:55:00:00,dl_type=0x86dd),ipv6(src=2001:cafe::88,dst=2001:cafe::92,label=0,proto=17,tclass=0x0,hlimit=64),udp(src=0,dst=6081,csum=0xffff),geneve(vni=0x7b)),out_port(100))
>   ])
> @@ -117,7 +117,7 @@ AT_CHECK([tail -1 stdout], [0],
>   dnl Check Geneve tunnel push with options
>   AT_CHECK([ovs-ofctl add-tlv-map int-br "{class=0xffff,type=0x80,len=4}->tun_metadata0"])
>   AT_CHECK([ovs-ofctl add-flow int-br "actions=set_field:2001:cafe::92->tun_ipv6_dst,set_field:0xa->tun_metadata0,5"])
> -AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'in_port(2),eth_type(0x0800),ipv4(src=1.1.3.88,dst=1.1.3.112,proto=47,tos=0,ttl=64,frag=no)'], [0], [stdout])
> +AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'in_port(2),eth(src=f8:bc:12:44:34:b6,dst=aa:55:aa:55:00:00),eth_type(0x0800),ipv4(src=1.1.3.88,dst=1.1.3.112,proto=47,tos=0,ttl=64,frag=no)'], [0], [stdout])
>   AT_CHECK([tail -1 stdout], [0],
>     [Datapath actions: tnl_push(tnl_port(6081),header(size=78,type=5,eth(dst=f8:bc:12:44:34:b6,src=aa:55:aa:55:00:00,dl_type=0x86dd),ipv6(src=2001:cafe::88,dst=2001:cafe::92,label=0,proto=17,tclass=0x0,hlimit=64),udp(src=0,dst=6081,csum=0xffff),geneve(crit,vni=0x7b,options({class=0xffff,type=0x80,len=4,0xa}))),out_port(100))
>   ])
> diff --git a/tests/tunnel-push-pop.at b/tests/tunnel-push-pop.at
> index 700ef55..0e596f2 100644
> --- a/tests/tunnel-push-pop.at
> +++ b/tests/tunnel-push-pop.at
> @@ -93,28 +93,28 @@ AT_CHECK([tail -1 stdout], [0],
>   
>   dnl Check VXLAN tunnel push
>   AT_CHECK([ovs-ofctl add-flow int-br action=2])
> -AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'in_port(2),eth_type(0x0800),ipv4(src=1.1.3.88,dst=1.1.3.112,proto=47,tos=0,ttl=64,frag=no)'], [0], [stdout])
> +AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'in_port(2),eth(src=f8:bc:12:44:34:b6,dst=aa:55:aa:55:00:00),eth_type(0x0800),ipv4(src=1.1.3.88,dst=1.1.3.112,proto=47,tos=0,ttl=64,frag=no)'], [0], [stdout])
>   AT_CHECK([tail -1 stdout], [0],
>     [Datapath actions: tnl_push(tnl_port(4789),header(size=50,type=4,eth(dst=f8:bc:12:44:34:b6,src=aa:55:aa:55:00:00,dl_type=0x0800),ipv4(src=1.1.2.88,dst=1.1.2.92,proto=17,tos=0,ttl=64,frag=0x4000),udp(src=0,dst=4789,csum=0x0),vxlan(flags=0x8000000,vni=0x7b)),out_port(100))
>   ])
>   
>   dnl Check VXLAN tunnel push set tunnel id by flow and checksum
>   AT_CHECK([ovs-ofctl add-flow int-br "actions=set_tunnel:124,4"])
> -AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'in_port(2),eth_type(0x0800),ipv4(src=1.1.3.88,dst=1.1.3.112,proto=47,tos=0,ttl=64,frag=no)'], [0], [stdout])
> +AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'in_port(2),eth(src=f8:bc:12:44:34:b6,dst=aa:55:aa:55:00:00),eth_type(0x0800),ipv4(src=1.1.3.88,dst=1.1.3.112,proto=47,tos=0,ttl=64,frag=no)'], [0], [stdout])
>   AT_CHECK([tail -1 stdout], [0],
>     [Datapath actions: tnl_push(tnl_port(4789),header(size=50,type=4,eth(dst=f8:bc:12:44:34:b7,src=aa:55:aa:55:00:00,dl_type=0x0800),ipv4(src=1.1.2.88,dst=1.1.2.93,proto=17,tos=0,ttl=64,frag=0x4000),udp(src=0,dst=4789,csum=0xffff),vxlan(flags=0x8000000,vni=0x7c)),out_port(100))
>   ])
>   
>   dnl Check GRE tunnel push
>   AT_CHECK([ovs-ofctl add-flow int-br action=3])
> -AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'in_port(2),eth_type(0x0800),ipv4(src=1.1.3.88,dst=1.1.3.112,proto=47,tos=0,ttl=64,frag=no)'], [0], [stdout])
> +AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'in_port(2),eth(src=f8:bc:12:44:34:b6,dst=aa:55:aa:55:00:00),eth_type(0x0800),ipv4(src=1.1.3.88,dst=1.1.3.112,proto=47,tos=0,ttl=64,frag=no)'], [0], [stdout])
>   AT_CHECK([tail -1 stdout], [0],
>     [Datapath actions: tnl_push(tnl_port(3),header(size=42,type=3,eth(dst=f8:bc:12:44:34:b6,src=aa:55:aa:55:00:00,dl_type=0x0800),ipv4(src=1.1.2.88,dst=1.1.2.92,proto=47,tos=0,ttl=64,frag=0x4000),gre((flags=0x2000,proto=0x6558),key=0x1c8)),out_port(100))
>   ])
>   
>   dnl Check Geneve tunnel push
>   AT_CHECK([ovs-ofctl add-flow int-br "actions=set_field:1.1.2.92->tun_dst,5"])
> -AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'in_port(2),eth_type(0x0800),ipv4(src=1.1.3.88,dst=1.1.3.112,proto=47,tos=0,ttl=64,frag=no)'], [0], [stdout])
> +AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'in_port(2),eth(src=f8:bc:12:44:34:b6,dst=aa:55:aa:55:00:00),eth_type(0x0800),ipv4(src=1.1.3.88,dst=1.1.3.112,proto=47,tos=0,ttl=64,frag=no)'], [0], [stdout])
>   AT_CHECK([tail -1 stdout], [0],
>     [Datapath actions: tnl_push(tnl_port(6081),header(size=50,type=5,eth(dst=f8:bc:12:44:34:b6,src=aa:55:aa:55:00:00,dl_type=0x0800),ipv4(src=1.1.2.88,dst=1.1.2.92,proto=17,tos=0,ttl=64,frag=0x4000),udp(src=0,dst=6081,csum=0x0),geneve(vni=0x7b)),out_port(100))
>   ])
> @@ -122,7 +122,7 @@ AT_CHECK([tail -1 stdout], [0],
>   dnl Check Geneve tunnel push with options
>   AT_CHECK([ovs-ofctl add-tlv-map int-br "{class=0xffff,type=0x80,len=4}->tun_metadata0"])
>   AT_CHECK([ovs-ofctl add-flow int-br "actions=set_field:1.1.2.92->tun_dst,set_field:0xa->tun_metadata0,5"])
> -AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'in_port(2),eth_type(0x0800),ipv4(src=1.1.3.88,dst=1.1.3.112,proto=47,tos=0,ttl=64,frag=no)'], [0], [stdout])
> +AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'in_port(2),eth(src=f8:bc:12:44:34:b6,dst=aa:55:aa:55:00:00),eth_type(0x0800),ipv4(src=1.1.3.88,dst=1.1.3.112,proto=47,tos=0,ttl=64,frag=no)'], [0], [stdout])
>   AT_CHECK([tail -1 stdout], [0],
>     [Datapath actions: tnl_push(tnl_port(6081),header(size=58,type=5,eth(dst=f8:bc:12:44:34:b6,src=aa:55:aa:55:00:00,dl_type=0x0800),ipv4(src=1.1.2.88,dst=1.1.2.92,proto=17,tos=0,ttl=64,frag=0x4000),udp(src=0,dst=6081,csum=0x0),geneve(crit,vni=0x7b,options({class=0xffff,type=0x80,len=4,0xa}))),out_port(100))
>   ])
> diff --git a/tests/tunnel.at b/tests/tunnel.at
> index 1ba209d..881d6ba 100644
> --- a/tests/tunnel.at
> +++ b/tests/tunnel.at
> @@ -488,14 +488,14 @@ AT_CHECK([tail -1 stdout], [0],
>   ])
>   
>   dnl Option match
> -AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'recirc_id(0),tunnel(tun_id=0x0,src=1.1.1.1,dst=1.1.1.2,ttl=64,geneve({class=0xffff,type=0,len=4,0xb}),flags(df|key)),in_port(6081),skb_mark(0),eth_type(0x0800),ipv4(frag=no)'], [0], [stdout])
> +AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'recirc_id(0),tunnel(tun_id=0x0,src=1.1.1.1,dst=1.1.1.2,ttl=64,geneve({class=0xffff,type=0,len=4,0xb}),flags(df|key)),in_port(6081),skb_mark(0),eth(src=50:54:00:00:00:05,dst=50:54:00:00:00:07),eth_type(0x0800),ipv4(frag=no)'], [0], [stdout])
>   AT_CHECK([tail -2 stdout], [0],
>     [Megaflow: recirc_id=0,ip,tun_id=0,tun_src=1.1.1.1,tun_dst=1.1.1.2,tun_tos=0,tun_flags=+df-csum+key,tun_metadata0=0xb/0xf,in_port=1,nw_frag=no
>   Datapath actions: 2
>   ])
>   
>   dnl Skip unknown option
> -AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'recirc_id(0),tunnel(tun_id=0x0,src=1.1.1.1,dst=1.1.1.2,ttl=64,geneve({class=0xffff,type=0,len=4,0xb}{class=0xffff,type=2,len=4,0xc}),flags(df|key)),in_port(6081),skb_mark(0),eth_type(0x0800),ipv4(frag=no)'], [0], [stdout])
> +AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'recirc_id(0),tunnel(tun_id=0x0,src=1.1.1.1,dst=1.1.1.2,ttl=64,geneve({class=0xffff,type=0,len=4,0xb}{class=0xffff,type=2,len=4,0xc}),flags(df|key)),in_port(6081),skb_mark(0),eth(src=50:54:00:00:00:05,dst=50:54:00:00:00:07),eth_type(0x0800),ipv4(frag=no)'], [0], [stdout])
>   AT_CHECK([tail -2 stdout], [0],
>     [Megaflow: recirc_id=0,ip,tun_id=0,tun_src=1.1.1.1,tun_dst=1.1.1.2,tun_tos=0,tun_flags=+df-csum+key,tun_metadata0=0xb/0xf,in_port=1,nw_frag=no
>   Datapath actions: 2
> @@ -529,7 +529,7 @@ AT_CHECK([ovs-ofctl del-tlv-map br0 "{class=0xffff,type=3,len=4}->tun_metadata3"
>   AT_CHECK([ovs-ofctl add-tlv-map br0 "{class=0xffff,type=3,len=8}->tun_metadata3"])
>   
>   AT_CHECK([ovs-ofctl add-flow br0 tun_metadata3=0x1234567890abcdef,actions=2])
> -AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'recirc_id(0),tunnel(tun_id=0x0,src=1.1.1.1,dst=1.1.1.2,ttl=64,geneve({class=0xffff,type=3,len=8,0x1234567890abcdef}),flags(df|key)),in_port(6081),skb_mark(0),eth_type(0x0800),ipv4(frag=no)'], [0], [stdout])
> +AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'recirc_id(0),tunnel(tun_id=0x0,src=1.1.1.1,dst=1.1.1.2,ttl=64,geneve({class=0xffff,type=3,len=8,0x1234567890abcdef}),flags(df|key)),in_port(6081),skb_mark(0),eth(src=50:54:00:00:00:05,dst=50:54:00:00:00:07),eth_type(0x0800),ipv4(frag=no)'], [0], [stdout])
>   AT_CHECK([tail -2 stdout], [0],
>     [Megaflow: recirc_id=0,ip,tun_id=0,tun_src=1.1.1.1,tun_dst=1.1.1.2,tun_tos=0,tun_flags=+df-csum+key,tun_metadata3=0x1234567890abcdef,in_port=1,nw_frag=no
>   Datapath actions: 2
> @@ -564,13 +564,13 @@ AT_CHECK([ovs-ofctl dump-flows br0 | ofctl_strip | sort],
>   NXST_FLOW reply:
>   ])
>   
> -AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'recirc_id(0),tunnel(tun_id=0x0,src=1.1.1.1,dst=1.1.1.2,ttl=64,geneve({class=0xffff,type=0,len=4,0x12345678}),flags(df|key)),in_port(6081),skb_mark(0),eth_type(0x0800),ipv4(frag=no)'], [0], [stdout])
> +AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'recirc_id(0),tunnel(tun_id=0x0,src=1.1.1.1,dst=1.1.1.2,ttl=64,geneve({class=0xffff,type=0,len=4,0x12345678}),flags(df|key)),in_port(6081),skb_mark(0),eth(src=50:54:00:00:00:05,dst=50:54:00:00:00:07),eth_type(0x0800),ipv4(frag=no)'], [0], [stdout])
>   AT_CHECK([tail -2 stdout], [0],
>     [Megaflow: recirc_id=0,ip,tun_id=0,tun_src=1.1.1.1,tun_dst=1.1.1.2,tun_tos=0,tun_flags=+df-csum+key,tun_metadata0,tun_metadata1=NP,tun_metadata2=NP,in_port=1,nw_frag=no
>   Datapath actions: 2
>   ])
>   
> -AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'recirc_id(0),tunnel(tun_id=0x0,src=1.1.1.1,dst=1.1.1.2,ttl=64,geneve({class=0xffff,type=1,len=0}),flags(df|key)),in_port(6081),skb_mark(0),eth_type(0x0800),ipv4(frag=no)'], [0], [stdout])
> +AT_CHECK([ovs-appctl ofproto/trace ovs-dummy 'recirc_id(0),tunnel(tun_id=0x0,src=1.1.1.1,dst=1.1.1.2,ttl=64,geneve({class=0xffff,type=1,len=0}),flags(df|key)),in_port(6081),skb_mark(0),eth(src=50:54:00:00:00:05,dst=50:54:00:00:00:07),eth_type(0x0800),ipv4(frag=no)'], [0], [stdout])
>   AT_CHECK([tail -2 stdout], [0],
>     [Megaflow: recirc_id=0,ip,tun_id=0,tun_src=1.1.1.1,tun_dst=1.1.1.2,tun_tos=0,tun_flags=+df-csum+key,tun_metadata1,tun_metadata2=NP,in_port=1,nw_ecn=0,nw_frag=no
>   Datapath actions: set(tunnel(tun_id=0x0,dst=1.1.1.1,ttl=64,tp_dst=6081,geneve({class=0xffff,type=0x1,len=0}),flags(df|key))),6081