[ovs-dev,PATCHv2,2/6] Add connection tracking mark support.
diff mbox

Message ID 1442531068-41423-3-git-send-email-joestringer@nicira.com
State Changes Requested
Headers show

Commit Message

Joe Stringer Sept. 17, 2015, 11:04 p.m. UTC
This patch adds a new 32-bit metadata field to the connection tracking
interface. When a mark is specified as part of the ct action and the
connection is committed, the value is saved with the current connection.
Subsequent ct lookups with the table specified will expose this metadata
as the "ct_mark" field in the flow.

For example, to allow new connections from port 1->2 and only allow
established connections from port 2->1, and to associate a mark with those
connections:

    priority=1,action=drop
    priority=10,arp,action=normal
    priority=10,icmp,action=normal
    in_port=1,tcp,action=ct(commit,exec(set_field:1->ct_mark)),2
    in_port=2,ct_state=-trk,tcp,action=ct(table=1)
    table=1,in_port=2,ct_state=+trk,ct_mark=1,tcp,action=1

Signed-off-by: Joe Stringer <joestringer@nicira.com>
Acked-by: Jarno Rajahalme <jrajahalme@nicira.com>
---
v2: Only make ct_mark visible after recirculation
    Require COMMIT flag for nested actions
    Address feedback from v1
---
 NEWS                                              |   4 +-
 datapath/flow_netlink.c                           |   2 +-
 datapath/linux/compat/include/linux/openvswitch.h |   5 +
 lib/dpif-netdev.c                                 |   2 +-
 lib/flow.c                                        |  13 +++
 lib/flow.h                                        |   9 +-
 lib/match.c                                       |  18 +++
 lib/match.h                                       |   2 +
 lib/meta-flow.c                                   |  24 ++++
 lib/meta-flow.h                                   |  19 ++++
 lib/nx-match.c                                    |   6 +-
 lib/odp-execute.c                                 |   2 +
 lib/odp-util.c                                    |  57 +++++++++-
 lib/odp-util.h                                    |   6 +-
 lib/ofp-actions.c                                 | 110 +++++++++++++++++-
 lib/ofp-actions.h                                 |  39 ++++++-
 lib/packets.h                                     |   1 +
 ofproto/ofproto-dpif-sflow.c                      |   1 +
 ofproto/ofproto-dpif-xlate.c                      |  30 +++++
 ofproto/ofproto-dpif.c                            |   9 +-
 ofproto/ofproto-unixctl.man                       |   2 +
 tests/dpif-netdev.at                              |   2 +-
 tests/odp.at                                      |   8 +-
 tests/ofproto-dpif.at                             |   4 +-
 tests/ofproto.at                                  |   3 +-
 tests/system-traffic.at                           | 132 ++++++++++++++++++++++
 tests/test-odp.c                                  |   1 +
 utilities/ovs-ofctl.8.in                          |  16 +++
 28 files changed, 497 insertions(+), 30 deletions(-)

Comments

Ben Pfaff Sept. 18, 2015, 4:58 p.m. UTC | #1
On Thu, Sep 17, 2015 at 04:04:24PM -0700, Joe Stringer wrote:
> This patch adds a new 32-bit metadata field to the connection tracking
> interface. When a mark is specified as part of the ct action and the
> connection is committed, the value is saved with the current connection.
> Subsequent ct lookups with the table specified will expose this metadata
> as the "ct_mark" field in the flow.
> 
> For example, to allow new connections from port 1->2 and only allow
> established connections from port 2->1, and to associate a mark with those
> connections:
> 
>     priority=1,action=drop
>     priority=10,arp,action=normal
>     priority=10,icmp,action=normal
>     in_port=1,tcp,action=ct(commit,exec(set_field:1->ct_mark)),2
>     in_port=2,ct_state=-trk,tcp,action=ct(table=1)
>     table=1,in_port=2,ct_state=+trk,ct_mark=1,tcp,action=1
> 
> Signed-off-by: Joe Stringer <joestringer@nicira.com>
> Acked-by: Jarno Rajahalme <jrajahalme@nicira.com>

Thanks a lot for writing this!

Without the explanation and the example in the commit log above, the
power and the restrictions and the intent are hard to understand.  Do
you think you could add any or all of the above to the documentation
somewhere?

decode_NXAST_RAW_CT() assumes that the nested actions are always
OpenFlow 1.0 actions.  I think that's a bad idea--I think that the
nested actions should use the same OpenFlow version as NXAST_RAW_CT
itself.  I don't think that information is readily available to decode
functions, so you might have to add a new parameter to all the decode
functions (in a separate patch please to avoid making this one hard to
review).

It seems slightly inconsistent for decode_NXAST_RAW_CT() to have just
one case where it returns without pushing the actions back on.  I'd
change the
        return OFPERR_OFPBAC_BAD_ARGUMENT;
to
        error = OFPERR_OFPBAC_BAD_ARGUMENT;
unless there's some reason for the difference that I didn't spot.

ofpacts_verify_nested() seems odd.  First it seems strange to verify
'outer_action' at all like this:
+    if (outer_action != OFPACT_CT && outer_action != OFPACT_WRITE_ACTIONS) {
+        return unsupported_nesting(a->type, outer_action);
+    }
because aren't the possibilities known at compile time?  If I'm right
about that, then if we need to check 'outer_action' then I'd guess an
ovs_assert would be the right thing since we'd be checking an invariant.

Second, the check at the end of ofpacts_verify_nested() seems like
something we'd want to check for actions that aren't inside an outer
action at all, not just something to check inside OFPACT_WRITE_ACTIONS:
+    if (field && outer_action != OFPACT_CT && field->id == MFF_CT_MARK) {
+        VLOG_WARN("cannot set CT fields outside of \"ct\" action");
+        return OFPERR_OFPBAC_BAD_SET_ARGUMENT;
     }

This comment on CT_MEMBERS strikes me a bit oddly.  There's ordinarily
no need to worry about alignment in a struct unless you want tight
packing or to exactly match some protocol definition.  Neither one comes
up here as far as I can tell.  Also I don't know why ct_pad is there
when there's an automatically sized pad[] member in struct
ofpact_conntrack:
+/* Even though zone_src contains a pointer, it should be 64-bit aligned due
+ * to the initial ofpact,flags,zone_imm members in struct ofpact_conntrack. */
 #define CT_MEMBERS                      \
     uint16_t flags;                     \
     uint16_t zone_imm;                  \
     struct mf_subfield zone_src;        \
-    uint8_t recirc_table;
+    uint8_t recirc_table;               \
+    uint16_t ct_pad;

The documentation seems a little incomplete, because it says that
certain actions can only be taken inside ct(), but it leaves out the
severe restrictions on what is allowed to be done inside ct().

I'm not certain that verification actually recurses into OFPACT_CT
nested actions.  I don't see a test case that provokes a nested action
error, which would probably be easy to write.

Thanks,

Ben.
Joe Stringer Sept. 22, 2015, 6:03 p.m. UTC | #2
On 18 September 2015 at 09:58, Ben Pfaff <blp@nicira.com> wrote:
> On Thu, Sep 17, 2015 at 04:04:24PM -0700, Joe Stringer wrote:
>> This patch adds a new 32-bit metadata field to the connection tracking
>> interface. When a mark is specified as part of the ct action and the
>> connection is committed, the value is saved with the current connection.
>> Subsequent ct lookups with the table specified will expose this metadata
>> as the "ct_mark" field in the flow.
>>
>> For example, to allow new connections from port 1->2 and only allow
>> established connections from port 2->1, and to associate a mark with those
>> connections:
>>
>>     priority=1,action=drop
>>     priority=10,arp,action=normal
>>     priority=10,icmp,action=normal
>>     in_port=1,tcp,action=ct(commit,exec(set_field:1->ct_mark)),2
>>     in_port=2,ct_state=-trk,tcp,action=ct(table=1)
>>     table=1,in_port=2,ct_state=+trk,ct_mark=1,tcp,action=1
>>
>> Signed-off-by: Joe Stringer <joestringer@nicira.com>
>> Acked-by: Jarno Rajahalme <jrajahalme@nicira.com>
>
> Thanks a lot for writing this!
>
> Without the explanation and the example in the commit log above, the
> power and the restrictions and the intent are hard to understand.  Do
> you think you could add any or all of the above to the documentation
> somewhere?

A fair request. I see that the match fields for
ct_zone/ct_mark/ct_label are a bit lacking in ovs-ofctl, and I'll do
another iteration across the ovs-ofctl documentation in general to try
to make it flow better. I can also add a basic example of using
connection tracking after the action documentation.

> decode_NXAST_RAW_CT() assumes that the nested actions are always
> OpenFlow 1.0 actions.  I think that's a bad idea--I think that the
> nested actions should use the same OpenFlow version as NXAST_RAW_CT
> itself.  I don't think that information is readily available to decode
> functions, so you might have to add a new parameter to all the decode
> functions (in a separate patch please to avoid making this one hard to
> review).

Agree, next series will have this patch.

> It seems slightly inconsistent for decode_NXAST_RAW_CT() to have just
> one case where it returns without pushing the actions back on.  I'd
> change the
>         return OFPERR_OFPBAC_BAD_ARGUMENT;
> to
>         error = OFPERR_OFPBAC_BAD_ARGUMENT;
> unless there's some reason for the difference that I didn't spot.

Nope, that was unintended. Good catch.

> ofpacts_verify_nested() seems odd.  First it seems strange to verify
> 'outer_action' at all like this:
> +    if (outer_action != OFPACT_CT && outer_action != OFPACT_WRITE_ACTIONS) {
> +        return unsupported_nesting(a->type, outer_action);
> +    }
> because aren't the possibilities known at compile time?  If I'm right
> about that, then if we need to check 'outer_action' then I'd guess an
> ovs_assert would be the right thing since we'd be checking an invariant.

OK.

> Second, the check at the end of ofpacts_verify_nested() seems like
> something we'd want to check for actions that aren't inside an outer
> action at all, not just something to check inside OFPACT_WRITE_ACTIONS:
> +    if (field && outer_action != OFPACT_CT && field->id == MFF_CT_MARK) {
> +        VLOG_WARN("cannot set CT fields outside of \"ct\" action");
> +        return OFPERR_OFPBAC_BAD_SET_ARGUMENT;
>      }

True, the function originally handled both cases but this behaviour
was dropped by mistake. I'll update it so that ofpacts_verify_nested()
handles all constraints related to nesting, and is called
unconditionally. I should be able to write a simple test to ensure
this behaviour, too.

> This comment on CT_MEMBERS strikes me a bit oddly.  There's ordinarily
> no need to worry about alignment in a struct unless you want tight
> packing or to exactly match some protocol definition.  Neither one comes
> up here as far as I can tell.  Also I don't know why ct_pad is there
> when there's an automatically sized pad[] member in struct
> ofpact_conntrack:
> +/* Even though zone_src contains a pointer, it should be 64-bit aligned due
> + * to the initial ofpact,flags,zone_imm members in struct ofpact_conntrack. */
>  #define CT_MEMBERS                      \
>      uint16_t flags;                     \
>      uint16_t zone_imm;                  \
>      struct mf_subfield zone_src;        \
> -    uint8_t recirc_table;
> +    uint8_t recirc_table;               \
> +    uint16_t ct_pad;

This is one of the pieces that changed between the version I sent to
you separately vs. the email above, sorry about that. The ct_pad
member is no longer present, CT_MEMBERS still exists but there's a
different explanation:

/* We want to determine the size of these elements at compile time to ensure
 * actions alignment, but we also want to allow ofpact_conntrack to have
 * basic _put(), _get(), etc accessors defined below which access these
 * members directly from ofpact_conntrack. An anonymous struct will serve
 * both of these purposes. */
#define CT_MEMBERS                      \
struct {                                \
    struct ofpact ofpact;               \
    uint16_t flags;                     \
    uint16_t zone_imm;                  \
    struct mf_subfield zone_src;        \
    uint16_t alg;                       \
    uint8_t recirc_table;               \
}

/* This is (and should only be) used for calculating action alignment. */
struct ofpact_ct_pad {
    CT_MEMBERS;
};

/* OFPACT_CT.
 *
 * Used for NXAST_CT. */
struct ofpact_conntrack {
    union {
        CT_MEMBERS;
        uint8_t pad[OFPACT_ALIGN(sizeof(struct ofpact_ct_pad))];
    };
    struct ofpact actions[];
};
BUILD_ASSERT_DECL(offsetof(struct ofpact_conntrack, actions)
                  % OFPACT_ALIGNTO == 0);
BUILD_ASSERT_DECL(offsetof(struct ofpact_conntrack, actions)
                  == sizeof(struct ofpact_conntrack));

> The documentation seems a little incomplete, because it says that
> certain actions can only be taken inside ct(), but it leaves out the
> severe restrictions on what is allowed to be done inside ct().

I'll update it.

> I'm not certain that verification actually recurses into OFPACT_CT
> nested actions.  I don't see a test case that provokes a nested action
> error, which would probably be easy to write.

Sure, I'll add it.

Patch
diff mbox

diff --git a/NEWS b/NEWS
index 45ae059..6eeccdc 100644
--- a/NEWS
+++ b/NEWS
@@ -22,8 +22,8 @@  Post-v2.4.0
      a Vagrant box.  See INSTALL.md for details
    - Dropped support for GRE64 tunnel.
    - Add support for connection tracking through the new "ct" action
-     and "ct_state"/"ct_zone" match fields.  Only available on Linux kernels
-     with the connection tracking module loaded.
+     and "ct_state"/"ct_zone"/"ct_mark" match fields.  Only available on
+     Linux kernels with the connection tracking module loaded.
 
 
 v2.4.0 - 20 Aug 2015
diff --git a/datapath/flow_netlink.c b/datapath/flow_netlink.c
index fd57b33..4e9547a 100644
--- a/datapath/flow_netlink.c
+++ b/datapath/flow_netlink.c
@@ -281,7 +281,7 @@  size_t ovs_key_attr_size(void)
 	/* Whenever adding new OVS_KEY_ FIELDS, we should consider
 	 * updating this function.
 	 */
-	BUILD_BUG_ON(OVS_KEY_ATTR_TUNNEL_INFO != 24);
+	BUILD_BUG_ON(OVS_KEY_ATTR_TUNNEL_INFO != 25);
 
 	return    nla_total_size(4)   /* OVS_KEY_ATTR_PRIORITY */
 		+ nla_total_size(0)   /* OVS_KEY_ATTR_TUNNEL */
diff --git a/datapath/linux/compat/include/linux/openvswitch.h b/datapath/linux/compat/include/linux/openvswitch.h
index 43db224..f6fb3a1 100644
--- a/datapath/linux/compat/include/linux/openvswitch.h
+++ b/datapath/linux/compat/include/linux/openvswitch.h
@@ -345,6 +345,7 @@  enum ovs_key_attr {
 				 * the accepted length of the array. */
 	OVS_KEY_ATTR_CT_STATE,	/* u8 bitmask of OVS_CS_F_* */
 	OVS_KEY_ATTR_CT_ZONE,	/* u16 connection tracking zone. */
+	OVS_KEY_ATTR_CT_MARK,	/* u32 connection tracking mark */
 
 #ifdef __KERNEL__
 	/* Only used within kernel data path. */
@@ -656,11 +657,15 @@  struct ovs_action_push_tnl {
  * enum ovs_ct_attr - Attributes for %OVS_ACTION_ATTR_CT action.
  * @OVS_CT_ATTR_FLAGS: u32 connection tracking flags.
  * @OVS_CT_ATTR_ZONE: u16 connection tracking zone.
+ * @OVS_CT_ATTR_MARK: u32 value followed by u32 mask. For each bit set in the
+ * mask, the corresponding bit in the value is copied to the connection
+ * tracking mark field in the connection.
  */
 enum ovs_ct_attr {
 	OVS_CT_ATTR_UNSPEC,
 	OVS_CT_ATTR_FLAGS,      /* u32 bitmask of OVS_CT_F_*. */
 	OVS_CT_ATTR_ZONE,       /* u16 zone id. */
+	OVS_CT_ATTR_MARK,       /* mark to associate with this connection. */
 	__OVS_CT_ATTR_MAX
 };
 
diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
index d7b28e1..37608bd 100644
--- a/lib/dpif-netdev.c
+++ b/lib/dpif-netdev.c
@@ -1921,7 +1921,7 @@  dpif_netdev_flow_from_nlattrs(const struct nlattr *key, uint32_t key_len,
     }
 
     /* Userspace datapath doesn't support conntrack. */
-    if (flow->ct_state || flow->ct_zone) {
+    if (flow->ct_state || flow->ct_zone || flow->ct_mark) {
         return EINVAL;
     }
 
diff --git a/lib/flow.c b/lib/flow.c
index a826a1b..3cbf35b 100644
--- a/lib/flow.c
+++ b/lib/flow.c
@@ -483,6 +483,11 @@  miniflow_extract(struct dp_packet *packet, struct miniflow *dst)
         miniflow_push_uint16(mf, ct_zone, md->ct_zone);
     }
 
+    if (md->ct_state) {
+        miniflow_push_uint32(mf, ct_mark, md->ct_mark);
+        miniflow_pad_to_64(mf, pad1);
+    }
+
     /* Initialize packet's layer pointer and offsets. */
     l2 = data;
     dp_packet_reset_offsets(packet);
@@ -839,6 +844,9 @@  flow_get_metadata(const struct flow *flow, struct match *flow_metadata)
     if (flow->ct_zone != 0) {
         match_set_ct_zone(flow_metadata, flow->ct_zone);
     }
+    if (flow->ct_mark != 0) {
+        match_set_ct_mark(flow_metadata, flow->ct_mark);
+    }
 }
 
 char *
@@ -1120,6 +1128,9 @@  flow_format(struct ds *ds, const struct flow *flow)
     if (!flow->ct_zone) {
         WC_UNMASK_FIELD(wc, ct_zone);
     }
+    if (!flow->ct_mark) {
+        WC_UNMASK_FIELD(wc, ct_mark);
+    }
     for (int i = 0; i < FLOW_N_REGS; i++) {
         if (!flow->regs[i]) {
             WC_UNMASK_FIELD(wc, regs[i]);
@@ -1196,6 +1207,7 @@  void flow_wildcards_init_for_packet(struct flow_wildcards *wc,
     WC_MASK_FIELD(wc, pkt_mark);
     WC_MASK_FIELD(wc, ct_state);
     WC_MASK_FIELD(wc, ct_zone);
+    WC_MASK_FIELD(wc, ct_mark);
     WC_MASK_FIELD(wc, recirc_id);
     WC_MASK_FIELD(wc, dp_hash);
     WC_MASK_FIELD(wc, in_port);
@@ -1301,6 +1313,7 @@  flow_wc_map(const struct flow *flow, struct flowmap *map)
     FLOWMAP_SET(map, vlan_tci);
     FLOWMAP_SET(map, ct_state);
     FLOWMAP_SET(map, ct_zone);
+    FLOWMAP_SET(map, ct_mark);
 
     /* Ethertype-dependent fields. */
     if (OVS_LIKELY(flow->dl_type == htons(ETH_TYPE_IP))) {
diff --git a/lib/flow.h b/lib/flow.h
index 3965335..ceda11b 100644
--- a/lib/flow.h
+++ b/lib/flow.h
@@ -104,9 +104,11 @@  struct flow {
     uint32_t recirc_id;         /* Must be exact match. */
     uint16_t ct_state;          /* Connection tracking state. */
     uint16_t ct_zone;           /* Connection tracking zone. */
+    uint32_t ct_mark;           /* Connection mark.*/
+    uint8_t pad1[4];            /* Pad to 64 bits. */
     uint32_t conj_id;           /* Conjunction ID. */
     ofp_port_t actset_output;   /* Output port in action set. */
-    uint8_t pad1[2];            /* Pad to 64 bits. */
+    uint8_t pad2[2];            /* Pad to 64 bits. */
 
     /* L2, Order the same as in the Ethernet header! (64-bit aligned) */
     struct eth_addr dl_dst;     /* Ethernet destination address. */
@@ -129,7 +131,7 @@  struct flow {
     struct eth_addr arp_sha;    /* ARP/ND source hardware address. */
     struct eth_addr arp_tha;    /* ARP/ND target hardware address. */
     ovs_be16 tcp_flags;         /* TCP flags. With L3 to avoid matching L4. */
-    ovs_be16 pad2;              /* Pad to 64 bits. */
+    ovs_be16 pad3;              /* Pad to 64 bits. */
 
     /* L4 (64-bit aligned) */
     ovs_be16 tp_src;            /* TCP/UDP/SCTP source port. */
@@ -155,7 +157,7 @@  BUILD_ASSERT_DECL(sizeof(struct flow_tnl) % sizeof(uint64_t) == 0);
 
 /* Remember to update FLOW_WC_SEQ when changing 'struct flow'. */
 BUILD_ASSERT_DECL(offsetof(struct flow, igmp_group_ip4) + sizeof(uint32_t)
-                  == sizeof(struct flow_tnl) + 192
+                  == sizeof(struct flow_tnl) + 200
                   && FLOW_WC_SEQ == 34);
 
 /* Incremental points at which flow classification may be performed in
@@ -984,6 +986,7 @@  pkt_metadata_from_flow(struct pkt_metadata *md, const struct flow *flow)
     md->in_port = flow->in_port;
     md->ct_state = flow->ct_state;
     md->ct_zone = flow->ct_zone;
+    md->ct_mark = flow->ct_mark;
 }
 
 static inline bool is_ip_any(const struct flow *flow)
diff --git a/lib/match.c b/lib/match.c
index 1a7d441..bab0a9a 100644
--- a/lib/match.c
+++ b/lib/match.c
@@ -305,6 +305,20 @@  match_set_ct_zone(struct match *match, uint16_t ct_zone)
 }
 
 void
+match_set_ct_mark(struct match *match, uint32_t ct_mark)
+{
+    match_set_ct_mark_masked(match, ct_mark, UINT32_MAX);
+}
+
+void
+match_set_ct_mark_masked(struct match *match, uint32_t ct_mark,
+                           uint32_t mask)
+{
+    match->flow.ct_mark = ct_mark & mask;
+    match->wc.masks.ct_mark = mask;
+}
+
+void
 match_set_dl_type(struct match *match, ovs_be16 dl_type)
 {
     match->wc.masks.dl_type = OVS_BE16_MAX;
@@ -1008,6 +1022,10 @@  match_format(const struct match *match, struct ds *s, int priority)
         format_uint16_masked(s, "ct_zone", f->ct_zone, wc->masks.ct_zone);
     }
 
+    if (wc->masks.ct_mark) {
+        format_uint32_masked(s, "ct_mark", f->ct_mark, wc->masks.ct_mark);
+    }
+
     if (wc->masks.dl_type) {
         skip_type = true;
         if (f->dl_type == htons(ETH_TYPE_IP)) {
diff --git a/lib/match.h b/lib/match.h
index c9cd718..2b969c4 100644
--- a/lib/match.h
+++ b/lib/match.h
@@ -86,6 +86,8 @@  void match_set_pkt_mark_masked(struct match *, uint32_t pkt_mark, uint32_t mask)
 void match_set_ct_state(struct match *, uint16_t ct_state);
 void match_set_ct_state_masked(struct match *, uint16_t ct_state, uint16_t mask);
 void match_set_ct_zone(struct match *, uint16_t ct_zone);
+void match_set_ct_mark(struct match *, uint32_t ct_mark);
+void match_set_ct_mark_masked(struct match *, uint32_t ct_mark, uint32_t mask);
 void match_set_skb_priority(struct match *, uint32_t skb_priority);
 void match_set_dl_type(struct match *, ovs_be16);
 void match_set_dl_src(struct match *, const struct eth_addr );
diff --git a/lib/meta-flow.c b/lib/meta-flow.c
index 2195161..455b14f 100644
--- a/lib/meta-flow.c
+++ b/lib/meta-flow.c
@@ -218,6 +218,8 @@  mf_is_all_wild(const struct mf_field *mf, const struct flow_wildcards *wc)
         return !wc->masks.ct_state;
     case MFF_CT_ZONE:
         return !wc->masks.ct_zone;
+    case MFF_CT_MARK:
+        return !wc->masks.ct_mark;
     CASE_MFF_REGS:
         return !wc->masks.regs[mf->id - MFF_REG0];
     CASE_MFF_XREGS:
@@ -503,6 +505,7 @@  mf_is_value_valid(const struct mf_field *mf, const union mf_value *value)
     case MFF_PKT_MARK:
     case MFF_CT_STATE:
     case MFF_CT_ZONE:
+    case MFF_CT_MARK:
     CASE_MFF_REGS:
     CASE_MFF_XREGS:
     case MFF_ETH_SRC:
@@ -658,6 +661,10 @@  mf_get_value(const struct mf_field *mf, const struct flow *flow,
         value->be16 = htons(flow->ct_zone);
         break;
 
+    case MFF_CT_MARK:
+        value->be32 = htonl(flow->ct_mark);
+        break;
+
     CASE_MFF_REGS:
         value->be32 = htonl(flow->regs[mf->id - MFF_REG0]);
         break;
@@ -898,6 +905,10 @@  mf_set_value(const struct mf_field *mf,
         match_set_ct_zone(match, ntohs(value->be16));
         break;
 
+    case MFF_CT_MARK:
+        match_set_ct_mark(match, ntohl(value->be32));
+        break;
+
     CASE_MFF_REGS:
         match_set_reg(match, mf->id - MFF_REG0, ntohl(value->be32));
         break;
@@ -1190,6 +1201,10 @@  mf_set_flow_value(const struct mf_field *mf,
         flow->ct_zone = ntohs(value->be16);
         break;
 
+    case MFF_CT_MARK:
+        flow->ct_mark = ntohl(value->be32);
+        break;
+
     CASE_MFF_REGS:
         flow->regs[mf->id - MFF_REG0] = ntohl(value->be32);
         break;
@@ -1489,6 +1504,11 @@  mf_set_wild(const struct mf_field *mf, struct match *match, char **err_str)
         match->wc.masks.ct_zone = 0;
         break;
 
+    case MFF_CT_MARK:
+        match->flow.ct_mark = 0;
+        match->wc.masks.ct_mark = 0;
+        break;
+
     CASE_MFF_REGS:
         match_set_reg_masked(match, mf->id - MFF_REG0, 0, 0);
         break;
@@ -1756,6 +1776,10 @@  mf_set(const struct mf_field *mf,
         match_set_ct_state_masked(match, ntohs(value->be16), ntohs(mask->be16));
         break;
 
+    case MFF_CT_MARK:
+        match_set_ct_mark_masked(match, ntohl(value->be32), ntohl(mask->be32));
+        break;
+
     case MFF_ETH_DST:
         match_set_dl_dst_masked(match, value->mac, mask->mac);
         break;
diff --git a/lib/meta-flow.h b/lib/meta-flow.h
index 82f9e40..4423dd1 100644
--- a/lib/meta-flow.h
+++ b/lib/meta-flow.h
@@ -768,6 +768,25 @@  enum OVS_PACKED_ENUM mf_field_id {
      */
     MFF_CT_ZONE,
 
+    /* "ct_mark".
+     *
+     * Connection tracking mark.  The mark is carried with the
+     * connection tracking state.  On Linux this corresponds to the
+     * nf_conn's "mark" member but the exact implementation is
+     * platform-dependent.
+     *
+     * Writable only from nested actions within the NXAST_CT action.
+     *
+     * Type: be32.
+     * Maskable: bitwise.
+     * Formatting: hexadecimal.
+     * Prerequisites: none.
+     * Access: read/write.
+     * NXM: NXM_NX_CT_MARK(107) since v2.5.
+     * OXM: none.
+     */
+    MFF_CT_MARK,
+
 #if FLOW_N_REGS == 8
     /* "reg<N>".
      *
diff --git a/lib/nx-match.c b/lib/nx-match.c
index 09e0de0..4b384de 100644
--- a/lib/nx-match.c
+++ b/lib/nx-match.c
@@ -1032,7 +1032,7 @@  nx_put_raw(struct ofpbuf *b, enum ofp_version oxm, const struct match *match,
         }
     }
 
-    /* Mark. */
+    /* Packet mark. */
     nxm_put_32m(b, MFF_PKT_MARK, oxm, htonl(flow->pkt_mark),
                 htonl(match->wc.masks.pkt_mark));
 
@@ -1045,6 +1045,10 @@  nx_put_raw(struct ofpbuf *b, enum ofp_version oxm, const struct match *match,
         nxm_put_16m(b, MFF_CT_ZONE, oxm, htons(flow->ct_zone),
                     htons(match->wc.masks.ct_zone));
     }
+    if (match->wc.masks.ct_mark) {
+        nxm_put_32m(b, MFF_CT_MARK, oxm, htonl(flow->ct_mark),
+                    htonl(match->wc.masks.ct_mark));
+    }
 
     /* OpenFlow 1.1+ Metadata. */
     nxm_put_64m(b, MFF_METADATA, oxm,
diff --git a/lib/odp-execute.c b/lib/odp-execute.c
index 575631c..4d6384e 100644
--- a/lib/odp-execute.c
+++ b/lib/odp-execute.c
@@ -328,6 +328,7 @@  odp_execute_set_action(struct dp_packet *packet, const struct nlattr *a)
     case OVS_KEY_ATTR_TCP_FLAGS:
     case OVS_KEY_ATTR_CT_STATE:
     case OVS_KEY_ATTR_CT_ZONE:
+    case OVS_KEY_ATTR_CT_MARK:
     case __OVS_KEY_ATTR_MAX:
     default:
         OVS_NOT_REACHED();
@@ -418,6 +419,7 @@  odp_execute_masked_set_action(struct dp_packet *packet,
     case OVS_KEY_ATTR_UNSPEC:
     case OVS_KEY_ATTR_CT_STATE:
     case OVS_KEY_ATTR_CT_ZONE:
+    case OVS_KEY_ATTR_CT_MARK:
     case OVS_KEY_ATTR_ENCAP:
     case OVS_KEY_ATTR_ETHERTYPE:
     case OVS_KEY_ATTR_IN_PORT:
diff --git a/lib/odp-util.c b/lib/odp-util.c
index 1ce6258..3784b0f 100644
--- a/lib/odp-util.c
+++ b/lib/odp-util.c
@@ -137,6 +137,7 @@  ovs_key_attr_to_string(enum ovs_key_attr attr, char *namebuf, size_t bufsize)
     case OVS_KEY_ATTR_SKB_MARK: return "skb_mark";
     case OVS_KEY_ATTR_CT_STATE: return "ct_state";
     case OVS_KEY_ATTR_CT_ZONE: return "ct_zone";
+    case OVS_KEY_ATTR_CT_MARK: return "ct_mark";
     case OVS_KEY_ATTR_TUNNEL: return "tunnel";
     case OVS_KEY_ATTR_IN_PORT: return "in_port";
     case OVS_KEY_ATTR_ETHERNET: return "eth";
@@ -540,12 +541,15 @@  static const struct nl_policy ovs_conntrack_policy[] = {
                             .min_len = sizeof(uint32_t) },
     [OVS_CT_ATTR_ZONE] = { .type = NL_A_U16, .optional = true,
                             .min_len = sizeof(uint16_t)},
+    [OVS_CT_ATTR_MARK] = { .type = NL_A_UNSPEC, .optional = true,
+                           .min_len = sizeof(uint32_t) * 2 },
 };
 
 static void
 format_odp_conntrack_action(struct ds *ds, const struct nlattr *attr)
 {
     struct nlattr *a[ARRAY_SIZE(ovs_conntrack_policy)];
+    const uint32_t *mark;
     uint32_t flags;
     uint16_t zone;
 
@@ -556,9 +560,10 @@  format_odp_conntrack_action(struct ds *ds, const struct nlattr *attr)
 
     flags = a[OVS_CT_ATTR_FLAGS] ? nl_attr_get_u32(a[OVS_CT_ATTR_FLAGS]) : 0;
     zone = a[OVS_CT_ATTR_ZONE] ? nl_attr_get_u16(a[OVS_CT_ATTR_ZONE]) : 0;
+    mark = a[OVS_CT_ATTR_MARK] ? nl_attr_get(a[OVS_CT_ATTR_MARK]) : NULL;
 
     ds_put_format(ds, "ct");
-    if (flags || zone) {
+    if (flags || zone || mark) {
         ds_put_cstr(ds, "(");
         if (flags & OVS_CT_F_COMMIT) {
             ds_put_format(ds, "commit");
@@ -569,6 +574,13 @@  format_odp_conntrack_action(struct ds *ds, const struct nlattr *attr)
             }
             ds_put_format(ds, "zone=%"PRIu16, zone);
         }
+        if (mark) {
+            if (ds_last(ds) != '(') {
+                ds_put_char(ds, ',');
+            }
+            ds_put_format(ds, "mark=%"PRIx32"/%"PRIx32, *mark,
+                          *(mark + 1));
+        }
         ds_put_cstr(ds, ")");
     }
 }
@@ -1011,6 +1023,10 @@  parse_conntrack_action(const char *s_, struct ofpbuf *actions)
     if (ovs_scan(s, "ct")) {
         uint32_t flags = 0;
         uint16_t zone = 0;
+        struct {
+            uint32_t value;
+            uint32_t mask;
+        } ct_mark = { 0, 0 };
         size_t start;
         char *end;
 
@@ -1035,6 +1051,16 @@  parse_conntrack_action(const char *s_, struct ofpbuf *actions)
                     s += n;
                     continue;
                 }
+                if (ovs_scan(s, "mark=%"SCNu32"%n", &ct_mark.value, &n)) {
+                    s += n;
+                    n = -1;
+                    if (ovs_scan(s, "/%"SCNu32"%n", &ct_mark.mask, &n)) {
+                        s += n;
+                    } else {
+                        ct_mark.mask = UINT32_MAX;
+                    }
+                    continue;
+                }
 
                 if (n < 0) {
                     return -EINVAL;
@@ -1050,6 +1076,10 @@  parse_conntrack_action(const char *s_, struct ofpbuf *actions)
         if (zone) {
             nl_msg_put_u16(actions, OVS_CT_ATTR_ZONE, zone);
         }
+        if (ct_mark.mask) {
+            nl_msg_put_unspec(actions, OVS_CT_ATTR_MARK, &ct_mark,
+                              sizeof(ct_mark));
+        }
         nl_msg_end_nested(actions, start);
     }
 
@@ -1319,6 +1349,7 @@  static const struct attr_len_tbl ovs_flow_key_attr_lens[OVS_KEY_ATTR_MAX + 1] =
     [OVS_KEY_ATTR_ND]        = { .len = sizeof(struct ovs_key_nd) },
     [OVS_KEY_ATTR_CT_STATE]  = { .len = 1 },
     [OVS_KEY_ATTR_CT_ZONE]   = { .len = 2 },
+    [OVS_KEY_ATTR_CT_MARK]   = { .len = 4 },
 };
 
 /* Returns the correct length of the payload for a flow key attribute of the
@@ -2180,6 +2211,15 @@  format_odp_key_attr(const struct nlattr *a, const struct nlattr *ma,
         }
         break;
 
+    case OVS_KEY_ATTR_CT_MARK:
+        if (verbose || !mask_empty(ma)) {
+            ds_put_format(ds, "%#"PRIx32, nl_attr_get_u32(a));
+            if (!is_exact) {
+                ds_put_format(ds, "/%#"PRIx32, nl_attr_get_u32(ma));
+            }
+        }
+        break;
+
     case OVS_KEY_ATTR_CT_STATE:
         if (!is_exact) {
             format_flags_masked(ds, NULL, packet_ct_state_to_string,
@@ -3309,6 +3349,7 @@  parse_odp_key_mask_attr(const char *s, const struct simap *port_names,
 
     SCAN_SINGLE("ct_state(", uint8_t, ct_state, OVS_KEY_ATTR_CT_STATE);
     SCAN_SINGLE("ct_zone(", uint16_t, u16, OVS_KEY_ATTR_CT_ZONE);
+    SCAN_SINGLE("ct_mark(", uint32_t, u32, OVS_KEY_ATTR_CT_MARK);
 
     SCAN_BEGIN_NESTED("tunnel(", OVS_KEY_ATTR_TUNNEL) {
         SCAN_FIELD_NESTED("tun_id=", ovs_be64, be64, OVS_TUNNEL_KEY_ATTR_ID);
@@ -3550,6 +3591,9 @@  odp_flow_key_from_flow__(const struct odp_flow_key_parms *parms,
     if (parms->support.ct_zone) {
         nl_msg_put_u16(buf, OVS_KEY_ATTR_CT_ZONE, data->ct_zone);
     }
+    if (parms->support.ct_mark) {
+        nl_msg_put_u32(buf, OVS_KEY_ATTR_CT_MARK, data->ct_mark);
+    }
     if (parms->support.recirc) {
         nl_msg_put_u32(buf, OVS_KEY_ATTR_RECIRC_ID, data->recirc_id);
         nl_msg_put_u32(buf, OVS_KEY_ATTR_DP_HASH, data->dp_hash);
@@ -3737,6 +3781,9 @@  odp_key_from_pkt_metadata(struct ofpbuf *buf, const struct pkt_metadata *md)
         if (md->ct_zone) {
             nl_msg_put_u16(buf, OVS_KEY_ATTR_CT_ZONE, md->ct_zone);
         }
+        if (md->ct_mark) {
+            nl_msg_put_u32(buf, OVS_KEY_ATTR_CT_MARK, md->ct_mark);
+        }
     }
 
     /* Add an ingress port attribute if 'odp_in_port' is not the magical
@@ -3794,6 +3841,10 @@  odp_key_to_pkt_metadata(const struct nlattr *key, size_t key_len,
             md->ct_zone = nl_attr_get_u16(nla);
             wanted_attrs &= ~(1u << OVS_KEY_ATTR_CT_ZONE);
             break;
+        case OVS_KEY_ATTR_CT_MARK:
+            md->ct_mark = nl_attr_get_u32(nla);
+            wanted_attrs &= ~(1u << OVS_KEY_ATTR_CT_MARK);
+            break;
         case OVS_KEY_ATTR_TUNNEL: {
             enum odp_key_fitness res;
 
@@ -4356,6 +4407,10 @@  odp_flow_key_to_flow__(const struct nlattr *key, size_t key_len,
         flow->ct_zone = nl_attr_get_u16(attrs[OVS_KEY_ATTR_CT_ZONE]);
         expected_attrs |= UINT64_C(1) << OVS_KEY_ATTR_CT_ZONE;
     }
+    if (present_attrs & (UINT64_C(1) << OVS_KEY_ATTR_CT_MARK)) {
+        flow->ct_mark = nl_attr_get_u32(attrs[OVS_KEY_ATTR_CT_MARK]);
+        expected_attrs |= UINT64_C(1) << OVS_KEY_ATTR_CT_MARK;
+    }
 
     if (present_attrs & (UINT64_C(1) << OVS_KEY_ATTR_TUNNEL)) {
         enum odp_key_fitness res;
diff --git a/lib/odp-util.h b/lib/odp-util.h
index 428b87c..74b07d8 100644
--- a/lib/odp-util.h
+++ b/lib/odp-util.h
@@ -122,6 +122,7 @@  void odp_portno_names_destroy(struct hmap *portno_names);
  *  OVS_KEY_ATTR_RECIRC_ID               4    --     4      8
  *  OVS_KEY_ATTR_CONN_STATE              2     2     4      8
  *  OVS_KEY_ATTR_CONN_ZONE               2     2     4      8
+ *  OVS_KEY_ATTR_CONN_MARK               4    --     4      8
  *  OVS_KEY_ATTR_ETHERNET               12    --     4     16
  *  OVS_KEY_ATTR_ETHERTYPE               2     2     4      8  (outer VLAN ethertype)
  *  OVS_KEY_ATTR_VLAN                    2     2     4      8
@@ -131,12 +132,12 @@  void odp_portno_names_destroy(struct hmap *portno_names);
  *  OVS_KEY_ATTR_ICMPV6                  2     2     4      8
  *  OVS_KEY_ATTR_ND                     28    --     4     32
  *  ----------------------------------------------------------
- *  total                                                 504
+ *  total                                                 512
  *
  * We include some slack space in case the calculation isn't quite right or we
  * add another field and forget to adjust this value.
  */
-#define ODPUTIL_FLOW_KEY_BYTES 512
+#define ODPUTIL_FLOW_KEY_BYTES 576
 BUILD_ASSERT_DECL(FLOW_WC_SEQ == 34);
 
 /* A buffer with sufficient size and alignment to hold an nlattr-formatted flow
@@ -172,6 +173,7 @@  struct odp_support {
     /* If true, serialise the corresponding OVS_KEY_ATTR_CONN_* field. */
     bool ct_state;
     bool ct_zone;
+    bool ct_mark;
 };
 
 struct odp_flow_key_parms {
diff --git a/lib/ofp-actions.c b/lib/ofp-actions.c
index 17bbd2c..21d900d 100644
--- a/lib/ofp-actions.c
+++ b/lib/ofp-actions.c
@@ -353,6 +353,10 @@  static enum ofperr ofpacts_pull_openflow_actions__(
     struct ofpbuf *openflow, unsigned int actions_len,
     enum ofp_version version, uint32_t allowed_ovsinsts,
     struct ofpbuf *ofpacts, enum ofpact_type outer_action);
+static char * OVS_WARN_UNUSED_RESULT ofpacts_parse_copy(
+    const char *s_, struct ofpbuf *ofpacts,
+    enum ofputil_protocol *usable_protocols,
+    bool allow_instructions, enum ofpact_type outer_action);
 
 /* Pull off existing actions or instructions. Used by nesting actions to keep
  * ofpacts_parse() oblivious of actions nesting.
@@ -4541,6 +4545,10 @@  format_DEBUG_RECIRC(const struct ofpact_null *a OVS_UNUSED, struct ds *s)
  *
  *      It is strongly recommended that this table is later than the current
  *      table, to prevent loops.
+ *
+ * Zero or more actions may immediately follow this action. These actions will
+ * be executed within the context of the connection tracker, and they require
+ * the NX_CT_F_COMMIT flag to be set.
  */
 struct nx_action_conntrack {
     ovs_be16 type;              /* OFPAT_VENDOR. */
@@ -4557,6 +4565,8 @@  struct nx_action_conntrack {
     uint8_t recirc_table;       /* Recirculate to a specific table, or
                                    NX_CT_RECIRC_NONE for no recirculation. */
     uint8_t pad[5];             /* Zeroes */
+    /* Followed by a sequence of ofpact elements, until the end of the action
+     * is reached. */
 };
 OFP_ASSERT(sizeof(struct nx_action_conntrack) == 24);
 
@@ -4591,7 +4601,9 @@  decode_ct_zone(const struct nx_action_conntrack *nac,
 static enum ofperr
 decode_NXAST_RAW_CT(const struct nx_action_conntrack *nac, struct ofpbuf *out)
 {
+    const size_t ct_offset = ofpacts_pull(out);
     struct ofpact_conntrack *conntrack;
+    struct ofpbuf openflow;
     int error = 0;
 
     conntrack = ofpact_put_CT(out);
@@ -4602,15 +4614,40 @@  decode_NXAST_RAW_CT(const struct nx_action_conntrack *nac, struct ofpbuf *out)
     }
     conntrack->recirc_table = nac->recirc_table;
 
+    ofpbuf_pull(out, sizeof(*conntrack));
+
+    ofpbuf_use_const(&openflow, nac + 1, ntohs(nac->len) - sizeof(*nac));
+    error = ofpacts_pull_openflow_actions__(&openflow, openflow.size,
+                                            OFP10_VERSION,
+                                            1u << OVSINST_OFPIT11_APPLY_ACTIONS,
+                                            out, OFPACT_CT);
+    if (error) {
+        goto out;
+    }
+
+    conntrack = ofpbuf_push_uninit(out, sizeof(*conntrack));
+    out->header = &conntrack->ofpact;
+    ofpact_update_len(out, &conntrack->ofpact);
+
+    if (conntrack->ofpact.len > sizeof(*conntrack)
+        && !(conntrack->flags & NX_CT_F_COMMIT)) {
+        VLOG_WARN_RL(&rl, "CT action requires commit flag if actions are "
+                     "specified.");
+        return OFPERR_OFPBAC_BAD_ARGUMENT;
+    }
+
 out:
+    ofpbuf_push_uninit(out, ct_offset);
     return error;
 }
 
 static void
 encode_CT(const struct ofpact_conntrack *conntrack,
-          enum ofp_version ofp_version OVS_UNUSED, struct ofpbuf *out)
+          enum ofp_version ofp_version, struct ofpbuf *out)
 {
     struct nx_action_conntrack *nac;
+    const size_t ofs = out->size;
+    size_t len;
 
     nac = put_NXAST_CT(out);
     nac->flags = htons(conntrack->flags);
@@ -4623,6 +4660,13 @@  encode_CT(const struct ofpact_conntrack *conntrack,
         nac->zone_imm = htons(conntrack->zone_imm);
     }
     nac->recirc_table = conntrack->recirc_table;
+
+    len = ofpacts_put_openflow_actions(conntrack->actions,
+                                       ofpact_ct_get_action_len(conntrack),
+                                       out, ofp_version);
+    len += sizeof(*nac);
+    nac = ofpbuf_at(out, ofs, sizeof(*nac));
+    nac->len = htons(len);
 }
 
 /* Parses 'arg' as the argument to a "ct" action, and appends such an
@@ -4632,8 +4676,9 @@  encode_CT(const struct ofpact_conntrack *conntrack,
  * error.  The caller is responsible for freeing the returned string. */
 static char * OVS_WARN_UNUSED_RESULT
 parse_CT(char *arg, struct ofpbuf *ofpacts,
-         enum ofputil_protocol *usable_protocols OVS_UNUSED)
+         enum ofputil_protocol *usable_protocols)
 {
+    const size_t ct_offset = ofpacts_pull(ofpacts);
     struct ofpact_conntrack *oc;
     char *error = NULL;
     char *key, *value;
@@ -4656,6 +4701,16 @@  parse_CT(char *arg, struct ofpbuf *ofpacts,
                     return error;
                 }
             }
+        } else if (!strcmp(key, "exec")) {
+            /* Hide existing actions from ofpacts_parse_copy(), so the
+             * nesting can be handled transparently. */
+            ofpbuf_pull(ofpacts, sizeof(*oc));
+            error = ofpacts_parse_copy(value, ofpacts, usable_protocols, false,
+                                       OFPACT_CT);
+
+            /* Put the CT action back on and update its length. */
+            oc = ofpbuf_push_uninit(ofpacts, sizeof(*oc));
+            oc->ofpact.len = ofpacts->size;
         } else {
             error = xasprintf("invalid argument to \"ct\" action: `%s'", key);
         }
@@ -4664,6 +4719,7 @@  parse_CT(char *arg, struct ofpbuf *ofpacts,
         }
     }
 
+    ofpbuf_push_uninit(ofpacts, ct_offset);
     return error;
 }
 
@@ -4699,6 +4755,12 @@  format_CT(const struct ofpact_conntrack *a, struct ds *s)
         append_comma(s, &first);
         ds_put_format(s, "zone=%"PRIu16, a->zone_imm);
     }
+    if (ofpact_ct_get_action_len(a)) {
+        append_comma(s, &first);
+        ds_put_cstr(s, "exec(");
+        ofpacts_format(a->actions, ofpact_ct_get_action_len(a), s);
+        ds_put_char(s, ')');
+    }
     ds_put_char(s, ')');
 }
 
@@ -6056,13 +6118,49 @@  ofpacts_check_consistency(struct ofpact ofpacts[], size_t ofpacts_len,
             : 0);
 }
 
+static const struct mf_field *
+ofpact_get_mf_field(enum ofpact_type type, const void *ofpact)
+{
+    if (type == OFPACT_SET_FIELD) {
+        const struct ofpact_set_field *orl = ofpact;
+
+        return orl->field;
+    } else if (type == OFPACT_REG_MOVE) {
+        const struct ofpact_reg_move *orm = ofpact;
+
+        return orm->dst.field;
+    }
+
+    return NULL;
+}
+
+static enum ofperr
+unsupported_nesting(enum ofpact_type action, enum ofpact_type outer_action)
+{
+    VLOG_WARN("\"%s\" action doesn't support nested action \"%s\"",
+              ofpact_name(outer_action), ofpact_name(action));
+    return OFPERR_OFPBAC_BAD_ARGUMENT;
+}
+
 static enum ofperr
 ofpacts_verify_nested(const struct ofpact *a, enum ofpact_type outer_action)
 {
-    if (outer_action != OFPACT_WRITE_ACTIONS) {
-        VLOG_WARN("\"%s\" action doesn't support nested action \"%s\"",
-                  ofpact_name(outer_action), ofpact_name(a->type));
-        return OFPERR_OFPBAC_BAD_ARGUMENT;
+    const struct mf_field *field;
+
+    if (outer_action != OFPACT_CT && outer_action != OFPACT_WRITE_ACTIONS) {
+        return unsupported_nesting(a->type, outer_action);
+    }
+
+    field = ofpact_get_mf_field(a->type, a);
+    if (outer_action == OFPACT_CT
+        && (!field
+            || (field && field->id != MFF_CT_MARK))) {
+        return unsupported_nesting(a->type, outer_action);
+    }
+
+    if (field && outer_action != OFPACT_CT && field->id == MFF_CT_MARK) {
+        VLOG_WARN("cannot set CT fields outside of \"ct\" action");
+        return OFPERR_OFPBAC_BAD_SET_ARGUMENT;
     }
 
     return 0;
diff --git a/lib/ofp-actions.h b/lib/ofp-actions.h
index abe2469..591d8a2 100644
--- a/lib/ofp-actions.h
+++ b/lib/ofp-actions.h
@@ -485,16 +485,45 @@  enum nx_conntrack_flags {
  * used in conjunction with the NX_CT_F_COMMIT flag above. */
 #define NX_CT_RECIRC_NONE OFPTT_ALL
 
+/* We want to determine the size of these elements at compile time to ensure
+ * actions alignment, but we also want to allow ofpact_conntrack to have
+ * basic _put(), _get(), etc accessors defined below which access these
+ * members directly from ofpact_conntrack. An anonymous struct will serve
+ * both of these purposes. */
+#define CT_MEMBERS                      \
+struct {                                \
+    struct ofpact ofpact;               \
+    uint16_t flags;                     \
+    uint16_t zone_imm;                  \
+    struct mf_subfield zone_src;        \
+    uint8_t recirc_table;               \
+}
+
+/* This is (and should only be) used for calculating action alignment. */
+struct ofpact_ct_pad {
+    CT_MEMBERS;
+};
+
 /* OFPACT_CT.
  *
  * Used for NXAST_CT. */
 struct ofpact_conntrack {
-    struct ofpact ofpact;
-    uint16_t flags;
-    uint16_t zone_imm;
-    struct mf_subfield zone_src;
-    uint8_t recirc_table;
+    union {
+        CT_MEMBERS;
+        uint8_t pad[OFPACT_ALIGN(sizeof(struct ofpact_ct_pad))];
+    };
+    struct ofpact actions[];
 };
+BUILD_ASSERT_DECL(offsetof(struct ofpact_conntrack, actions)
+                  % OFPACT_ALIGNTO == 0);
+BUILD_ASSERT_DECL(offsetof(struct ofpact_conntrack, actions)
+                  == sizeof(struct ofpact_conntrack));
+
+static inline size_t
+ofpact_ct_get_action_len(const struct ofpact_conntrack *oc)
+{
+    return oc->ofpact.len - offsetof(struct ofpact_conntrack, actions);
+}
 
 static inline size_t
 ofpact_nest_get_action_len(const struct ofpact_nest *on)
diff --git a/lib/packets.h b/lib/packets.h
index 6e71033..1061ee9 100644
--- a/lib/packets.h
+++ b/lib/packets.h
@@ -129,6 +129,7 @@  struct pkt_metadata {
     union flow_in_port in_port; /* Input port. */
     uint16_t ct_state;          /* Connection state. */
     uint16_t ct_zone;           /* Connection zone. */
+    uint32_t ct_mark;           /* Connection mark. */
     struct flow_tnl tunnel;     /* Encapsulating tunnel parameters. Note that
                                  * if 'ip_dst' == 0, the rest of the fields may
                                  * be uninitialized. */
diff --git a/ofproto/ofproto-dpif-sflow.c b/ofproto/ofproto-dpif-sflow.c
index b57c84b..310bbc5 100644
--- a/ofproto/ofproto-dpif-sflow.c
+++ b/ofproto/ofproto-dpif-sflow.c
@@ -1031,6 +1031,7 @@  sflow_read_set_action(const struct nlattr *attr,
     case OVS_KEY_ATTR_ND:
     case OVS_KEY_ATTR_CT_STATE:
     case OVS_KEY_ATTR_CT_ZONE:
+    case OVS_KEY_ATTR_CT_MARK:
     case OVS_KEY_ATTR_UNSPEC:
     case __OVS_KEY_ATTR_MAX:
     default:
diff --git a/ofproto/ofproto-dpif-xlate.c b/ofproto/ofproto-dpif-xlate.c
index 3d1958e..6027ffd 100644
--- a/ofproto/ofproto-dpif-xlate.c
+++ b/ofproto/ofproto-dpif-xlate.c
@@ -2807,6 +2807,7 @@  clear_conntrack(struct flow *flow)
 {
     flow->ct_state = 0;
     flow->ct_zone = 0;
+    flow->ct_mark = 0;
 }
 
 static void
@@ -4146,8 +4147,29 @@  recirc_unroll_actions(const struct ofpact *ofpacts, size_t ofpacts_len,
     }
 
 static void
+put_ct_mark(const struct flow *flow, struct flow *base_flow,
+            struct ofpbuf *odp_actions, struct flow_wildcards *wc)
+{
+    uint32_t base;
+    struct {
+        uint32_t key;
+        uint32_t mask;
+    } odp_attr;
+
+    base = base_flow->ct_mark;
+    odp_attr.key = flow->ct_mark;
+    odp_attr.mask = wc->masks.ct_mark;
+
+    if (odp_attr.mask && odp_attr.key != base) {
+        nl_msg_put_unspec(odp_actions, OVS_CT_ATTR_MARK, &odp_attr,
+                          sizeof(odp_attr));
+    }
+}
+
+static void
 compose_conntrack_action(struct xlate_ctx *ctx, struct ofpact_conntrack *ofc)
 {
+    uint32_t old_ct_mark = ctx->base_flow.ct_mark;
     uint32_t flags = 0;
     size_t ct_offset;
     uint16_t zone;
@@ -4156,6 +4178,9 @@  compose_conntrack_action(struct xlate_ctx *ctx, struct ofpact_conntrack *ofc)
      * conntrack action. */
     xlate_commit_actions(ctx);
 
+    /* Process nested actions first, to populate the key. */
+    do_xlate_actions(ofc->actions, ofpact_ct_get_action_len(ofc), ctx);
+
     if (ofc->flags & NX_CT_F_COMMIT) {
         flags |= OVS_CT_F_COMMIT;
     }
@@ -4168,8 +4193,13 @@  compose_conntrack_action(struct xlate_ctx *ctx, struct ofpact_conntrack *ofc)
     ct_offset = nl_msg_start_nested(ctx->odp_actions, OVS_ACTION_ATTR_CT);
     nl_msg_put_u32(ctx->odp_actions, OVS_CT_ATTR_FLAGS, flags);
     nl_msg_put_u16(ctx->odp_actions, OVS_CT_ATTR_ZONE, zone);
+    put_ct_mark(&ctx->xin->flow, &ctx->base_flow, ctx->odp_actions, ctx->wc);
     nl_msg_end_nested(ctx->odp_actions, ct_offset);
 
+    /* Restore the original ct fields in the key. These should only be exposed
+     * after recirculation to another table. */
+    ctx->base_flow.ct_mark = old_ct_mark;
+
     if (ofc->recirc_table == NX_CT_RECIRC_NONE) {
         /* If we do not recirculate as part of this action, hide the results of
          * connection tracking from subsequent recirculations. */
diff --git a/ofproto/ofproto-dpif.c b/ofproto/ofproto-dpif.c
index ba85727..f837543 100644
--- a/ofproto/ofproto-dpif.c
+++ b/ofproto/ofproto-dpif.c
@@ -1263,6 +1263,7 @@  check_##NAME(struct dpif_backer *backer)                                    \
 
 CHECK_FEATURE(ct_state)
 CHECK_FEATURE(ct_zone)
+CHECK_FEATURE(ct_mark)
 
 #undef CHECK_FEATURE
 #undef CHECK_FEATURE__
@@ -1281,6 +1282,7 @@  check_support(struct dpif_backer *backer)
 
     backer->support.odp.ct_state = check_ct_state(backer);
     backer->support.odp.ct_zone = check_ct_zone(backer);
+    backer->support.odp.ct_mark = check_ct_mark(backer);
 }
 
 static int
@@ -3980,16 +3982,19 @@  static enum ofperr
 rule_check(struct rule *rule)
 {
     uint16_t ct_state_mask, ct_zone_mask;
+    uint32_t ct_mark_mask;
 
     ct_state_mask = MINIFLOW_GET_U16(&rule->cr.match.mask->masks, ct_state);
     ct_zone_mask = MINIFLOW_GET_U16(&rule->cr.match.mask->masks, ct_zone);
+    ct_mark_mask = MINIFLOW_GET_U32(&rule->cr.match.mask->masks, ct_mark);
 
-    if (ct_state_mask || ct_zone_mask) {
+    if (ct_state_mask || ct_zone_mask || ct_mark_mask) {
         struct ofproto_dpif *ofproto = ofproto_dpif_cast(rule->ofproto);
         const struct odp_support *support = &ofproto_dpif_get_support(ofproto)->odp;
 
         if ((ct_state_mask && !support->ct_state)
-            || (ct_zone_mask && !support->ct_zone)) {
+            || (ct_zone_mask && !support->ct_zone)
+            || (ct_mark_mask && !support->ct_mark)) {
             return OFPERR_OFPBMC_BAD_FIELD;
         }
         if (ct_state_mask & CS_UNSUPPORTED_MASK) {
diff --git a/ofproto/ofproto-unixctl.man b/ofproto/ofproto-unixctl.man
index 892f6fc..87ef80d 100644
--- a/ofproto/ofproto-unixctl.man
+++ b/ofproto/ofproto-unixctl.man
@@ -107,6 +107,8 @@  Mark of the packet.
 Connection state of the packet.
 .IP \fIct_zone\fR
 Connection tracking zone for packet.
+.IP \fIct_mark\fR
+Connection mark of the packet.
 .IP \fItun_id\fR
 The tunnel ID on which the packet arrived.
 .IP \fIin_port\fR
diff --git a/tests/dpif-netdev.at b/tests/dpif-netdev.at
index 43108e7..502416f 100644
--- a/tests/dpif-netdev.at
+++ b/tests/dpif-netdev.at
@@ -82,7 +82,7 @@  AT_CHECK([cat ovs-vswitchd.log | grep -A 1 'miss upcall' | tail -n 1], [0], [dnl
 skb_priority(0),skb_mark(0),recirc_id(0),dp_hash(0),in_port(1),eth(src=50:54:00:00:00:09,dst=50:54:00:00:00:0a),eth_type(0x0800),ipv4(src=10.0.0.2,dst=10.0.0.1,proto=1,tos=0,ttl=64,frag=no),icmp(type=8,code=0)
 ])
 AT_CHECK([cat ovs-vswitchd.log | FILTER_FLOW_INSTALL | STRIP_XOUT], [0], [dnl
-pkt_mark=0,recirc_id=0,dp_hash=0,skb_priority=0,ct_state=0,ct_zone=0,icmp,in_port=1,vlan_tci=0x0000,dl_src=50:54:00:00:00:09,dl_dst=50:54:00:00:00:0a,nw_src=10.0.0.2,nw_dst=10.0.0.1,nw_tos=0,nw_ecn=0,nw_ttl=64,icmp_type=8,icmp_code=0, actions: <del>
+pkt_mark=0,recirc_id=0,dp_hash=0,skb_priority=0,ct_state=0,ct_zone=0,ct_mark=0,icmp,in_port=1,vlan_tci=0x0000,dl_src=50:54:00:00:00:09,dl_dst=50:54:00:00:00:0a,nw_src=10.0.0.2,nw_dst=10.0.0.1,nw_tos=0,nw_ecn=0,nw_ttl=64,icmp_type=8,icmp_code=0, actions: <del>
 recirc_id=0,ip,in_port=1,vlan_tci=0x0000/0x1fff,dl_src=50:54:00:00:00:09,dl_dst=50:54:00:00:00:0a,nw_frag=no, actions: <del>
 ])
 
diff --git a/tests/odp.at b/tests/odp.at
index 74d44ff..bf3fa8a 100644
--- a/tests/odp.at
+++ b/tests/odp.at
@@ -70,6 +70,10 @@  s/\(eth([[^)]]*)\),*/\1,eth_type(0x8100),vlan(vid=99,pcp=7),encap(/
 s/$/)/' odp-base.txt
 
  echo
+ echo '# Valid forms with conntrack fields.'
+ sed 's/^/skb_priority(0),skb_mark(0),ct_mark(0x12345678),recirc_id(0),dp_hash(0),/' odp-base.txt
+
+ echo
  echo '# Valid forms with IP first fragment.'
 sed 's/^/skb_priority(0),skb_mark(0),recirc_id(0),dp_hash(0),/' odp-base.txt | sed -n 's/,frag=no),/,frag=first),/p'
 
@@ -89,7 +93,7 @@  s/^/ODP_FIT_TOO_LITTLE: /
 dnl Some fields are always printed for this test, because wildcards aren't
 dnl specified. We can skip these.
 sed -i 's/\(skb_mark(0)\),\(ct\)/\1,ct_state(0),ct_zone(0),\2/' odp-out.txt
-sed -i 's/\(skb_mark([[^)]]*)\),\(recirc\)/\1,ct_state(0),ct_zone(0),\2/' odp-out.txt
+sed -i 's/\(skb_mark([[^)]]*)\),\(recirc\)/\1,ct_state(0),ct_zone(0),ct_mark(0),\2/' odp-out.txt
 
 AT_CHECK_UNQUOTED([ovstest test-odp parse-keys < odp-in.txt], [0], [`cat odp-out.txt`
 ])
@@ -159,7 +163,7 @@  s/$/)/' odp-base.txt
 
  echo
  echo '# Valid forms with conntrack fields.'
- sed 's/\(eth([[^)]]*),?\)/\1,ct_state(+trk),/' odp-base.txt
+ sed 's/\(eth([[^)]]*),?\)/\1,ct_state(+trk),ct_mark(0x12345678\/0xFOFOFOFO),/' odp-base.txt
 
  echo
  echo '# Valid forms with IP first fragment.'
diff --git a/tests/ofproto-dpif.at b/tests/ofproto-dpif.at
index bee3ae2..a6a6ee9 100644
--- a/tests/ofproto-dpif.at
+++ b/tests/ofproto-dpif.at
@@ -6523,8 +6523,8 @@  for i in 1 2 3 4; do
 done
 sleep 1
 AT_CHECK([cat ovs-vswitchd.log | STRIP_UFID | FILTER_FLOW_INSTALL | STRIP_USED], [0], [dnl
-pkt_mark=0,recirc_id=0,dp_hash=0,skb_priority=0,ct_state=0,ct_zone=0,icmp,in_port=1,vlan_tci=0x0000,dl_src=50:54:00:00:00:09,dl_dst=50:54:00:00:00:0a,nw_src=10.0.0.2,nw_dst=10.0.0.1,nw_tos=0,nw_ecn=0,nw_ttl=64,icmp_type=8,icmp_code=0, actions:2
-pkt_mark=0,recirc_id=0,dp_hash=0,skb_priority=0,ct_state=0,ct_zone=0,icmp,in_port=1,vlan_tci=0x0000,dl_src=50:54:00:00:00:0b,dl_dst=50:54:00:00:00:0c,nw_src=10.0.0.4,nw_dst=10.0.0.3,nw_tos=0,nw_ecn=0,nw_ttl=64,icmp_type=8,icmp_code=0, actions:drop
+pkt_mark=0,recirc_id=0,dp_hash=0,skb_priority=0,ct_state=0,ct_zone=0,ct_mark=0,icmp,in_port=1,vlan_tci=0x0000,dl_src=50:54:00:00:00:09,dl_dst=50:54:00:00:00:0a,nw_src=10.0.0.2,nw_dst=10.0.0.1,nw_tos=0,nw_ecn=0,nw_ttl=64,icmp_type=8,icmp_code=0, actions:2
+pkt_mark=0,recirc_id=0,dp_hash=0,skb_priority=0,ct_state=0,ct_zone=0,ct_mark=0,icmp,in_port=1,vlan_tci=0x0000,dl_src=50:54:00:00:00:0b,dl_dst=50:54:00:00:00:0c,nw_src=10.0.0.4,nw_dst=10.0.0.3,nw_tos=0,nw_ecn=0,nw_ttl=64,icmp_type=8,icmp_code=0, actions:drop
 ])
 AT_CHECK([cat ovs-vswitchd.log | STRIP_UFID | FILTER_FLOW_DUMP | grep 'packets:3'], [0], [dnl
 skb_priority(0),skb_mark(0),recirc_id(0),dp_hash(0),in_port(1),eth(src=50:54:00:00:00:09,dst=50:54:00:00:00:0a),eth_type(0x0800),ipv4(src=10.0.0.2,dst=10.0.0.1,proto=1,tos=0,ttl=64,frag=no),icmp(type=8,code=0), packets:3, bytes:180, used:0.0s, actions:2
diff --git a/tests/ofproto.at b/tests/ofproto.at
index 3386ae0..8cdca9c 100644
--- a/tests/ofproto.at
+++ b/tests/ofproto.at
@@ -1538,7 +1538,7 @@  head_table () {
         actions: output group set_field strip_vlan push_vlan mod_nw_ttl dec_ttl set_mpls_ttl dec_mpls_ttl push_mpls pop_mpls set_queue
         supported on Set-Field: tun_id tun_src tun_dst tun_flags tun_gbp_id tun_gbp_flags
 tun_metadata0 tun_metadata1 tun_metadata2 tun_metadata3 tun_metadata4 tun_metadata5 tun_metadata6 tun_metadata7 tun_metadata8 tun_metadata9 tun_metadata10 tun_metadata11 tun_metadata12 tun_metadata13 tun_metadata14 tun_metadata15 tun_metadata16 tun_metadata17 tun_metadata18 tun_metadata19 tun_metadata20 tun_metadata21 tun_metadata22 tun_metadata23 tun_metadata24 tun_metadata25 tun_metadata26 tun_metadata27 tun_metadata28 tun_metadata29 tun_metadata30 tun_metadata31 tun_metadata32 tun_metadata33 tun_metadata34 tun_metadata35 tun_metadata36 tun_metadata37 tun_metadata38 tun_metadata39 tun_metadata40 tun_metadata41 tun_metadata42 tun_metadata43 tun_metadata44 tun_metadata45 tun_metadata46 tun_metadata47 tun_metadata48 tun_metadata49 tun_metadata50 tun_metadata51 tun_metadata52 tun_metadata53 tun_metadata54 tun_metadata55 tun_metadata56 tun_metadata57 tun_metadata58 tun_metadata59 tun_metadata60 tun_metadata61 tun_metadata62 tun_metadata63
-metadata in_port in_port_oxm pkt_mark reg0 reg1 reg2 reg3 reg4 reg5 reg6 reg7 xreg0 xreg1 xreg2 xreg3 eth_src eth_dst vlan_tci vlan_vid vlan_pcp mpls_label mpls_tc ip_src ip_dst ipv6_src ipv6_dst ipv6_label nw_tos ip_dscp nw_ecn nw_ttl arp_op arp_spa arp_tpa arp_sha arp_tha tcp_src tcp_dst udp_src udp_dst sctp_src sctp_dst nd_target nd_sll nd_tll
+metadata in_port in_port_oxm pkt_mark ct_mark reg0 reg1 reg2 reg3 reg4 reg5 reg6 reg7 xreg0 xreg1 xreg2 xreg3 eth_src eth_dst vlan_tci vlan_vid vlan_pcp mpls_label mpls_tc ip_src ip_dst ipv6_src ipv6_dst ipv6_label nw_tos ip_dscp nw_ecn nw_ttl arp_op arp_spa arp_tpa arp_sha arp_tha tcp_src tcp_dst udp_src udp_dst sctp_src sctp_dst nd_target nd_sll nd_tll
     matching:
       dp_hash: arbitrary mask
       recirc_id: exact match or wildcard
@@ -1620,6 +1620,7 @@  metadata in_port in_port_oxm pkt_mark reg0 reg1 reg2 reg3 reg4 reg5 reg6 reg7 xr
       pkt_mark: arbitrary mask
       ct_state: arbitrary mask
       ct_zone: exact match or wildcard
+      ct_mark: arbitrary mask
       reg0: arbitrary mask
       reg1: arbitrary mask
       reg2: arbitrary mask
diff --git a/tests/system-traffic.at b/tests/system-traffic.at
index 57283b7..ccdaa2a 100644
--- a/tests/system-traffic.at
+++ b/tests/system-traffic.at
@@ -566,6 +566,138 @@  TIME_WAIT src=10.1.1.1 dst=10.1.1.2 sport=<cleared> dport=<cleared> src=10.1.1.2
 OVS_TRAFFIC_VSWITCHD_STOP
 AT_CLEANUP
 
+AT_SETUP([conntrack - ct_mark])
+CHECK_CONNTRACK()
+OVS_TRAFFIC_VSWITCHD_START(
+   [set-fail-mode br0 standalone -- ])
+
+ADD_NAMESPACES(at_ns0, at_ns1, at_ns2, at_ns3)
+
+ADD_VETH(p0, at_ns0, br0, "10.1.1.1/24")
+ADD_VETH(p1, at_ns1, br0, "10.1.1.2/24")
+ADD_VETH(p2, at_ns2, br0, "10.1.1.3/24")
+ADD_VETH(p3, at_ns3, br0, "10.1.1.4/24")
+
+dnl Allow traffic between ns0<->ns1 using the ct_mark.
+dnl Check that different marks do not match for traffic between ns2<->ns3.
+AT_DATA([flows.txt], [dnl
+priority=1,action=drop
+priority=10,arp,action=normal
+priority=10,icmp,action=normal
+priority=100,in_port=1,tcp,action=ct(commit,exec(set_field:1->ct_mark)),2
+priority=100,in_port=2,ct_state=-trk,tcp,action=ct(table=0)
+priority=100,in_port=2,ct_state=+trk,ct_mark=1,tcp,action=1
+priority=100,in_port=3,tcp,action=ct(commit,exec(set_field:2->ct_mark)),4
+priority=100,in_port=4,ct_state=-trk,tcp,action=ct(table=0)
+priority=100,in_port=4,ct_state=+trk,ct_mark=1,tcp,action=3
+])
+
+AT_CHECK([ovs-ofctl add-flows br0 flows.txt])
+
+dnl HTTP requests from p0->p1 should work fine.
+NETNS_DAEMONIZE([at_ns1], [[$PYTHON $srcdir/test-l7.py]], [http0.pid])
+NS_CHECK_EXEC([at_ns0], [wget 10.1.1.2 -t 3 -T 1 --retry-connrefused -v -o wget0.log])
+
+AT_CHECK([conntrack -L 2>&1 | FORMAT_CT(10.1.1.2) | grep TIME], [0], [dnl
+TIME_WAIT src=10.1.1.1 dst=10.1.1.2 sport=<cleared> dport=<cleared> src=10.1.1.2 dst=10.1.1.1 sport=<cleared> dport=<cleared> [[ASSURED]] mark=1 use=1
+])
+
+dnl HTTP requests from p2->p3 should fail due to network failure.
+dnl Try 3 times, in 1 second intervals.
+NETNS_DAEMONIZE([at_ns3], [[$PYTHON $srcdir/test-l7.py]], [http1.pid])
+NS_CHECK_EXEC([at_ns2], [wget 10.1.1.4 -t 3 -T 1 -v -o wget1.log], [4])
+
+AT_CHECK([conntrack -L 2>&1 | FORMAT_CT(10.1.1.4)], [0], [dnl
+SYN_RECV src=10.1.1.3 dst=10.1.1.4 sport=<cleared> dport=<cleared> src=10.1.1.4 dst=10.1.1.3 sport=<cleared> dport=<cleared> mark=2 use=1
+])
+
+OVS_TRAFFIC_VSWITCHD_STOP
+AT_CLEANUP
+
+AT_SETUP([conntrack - ct_mark from register])
+CHECK_CONNTRACK()
+OVS_TRAFFIC_VSWITCHD_START(
+   [set-fail-mode br0 standalone -- ])
+
+ADD_NAMESPACES(at_ns0, at_ns1, at_ns2, at_ns3)
+
+ADD_VETH(p0, at_ns0, br0, "10.1.1.1/24")
+ADD_VETH(p1, at_ns1, br0, "10.1.1.2/24")
+ADD_VETH(p2, at_ns2, br0, "10.1.1.3/24")
+ADD_VETH(p3, at_ns3, br0, "10.1.1.4/24")
+
+dnl Allow any traffic from ns0->ns1. Only allow nd, return traffic from ns1->ns0.
+AT_DATA([flows.txt], [dnl
+priority=1,action=drop
+priority=10,arp,action=normal
+priority=10,icmp,action=normal
+priority=100,in_port=1,tcp,action=load:1->NXM_NX_REG0[[0..31]],ct(commit,exec(move:NXM_NX_REG0[[0..31]]->NXM_NX_CT_MARK[[]])),2
+priority=100,in_port=2,ct_state=-trk,tcp,action=ct(table=0)
+priority=100,in_port=2,ct_state=+trk,ct_mark=1,tcp,action=1
+priority=100,in_port=3,tcp,action=load:2->NXM_NX_REG0[[0..31]],ct(commit,exec(move:NXM_NX_REG0[[0..31]]->NXM_NX_CT_MARK[[]])),4
+priority=100,in_port=4,ct_state=-trk,tcp,action=ct(table=0)
+priority=100,in_port=4,ct_state=+trk,ct_mark=1,tcp,action=3
+])
+
+AT_CHECK([ovs-ofctl add-flows br0 flows.txt])
+
+dnl HTTP requests from p0->p1 should work fine.
+NETNS_DAEMONIZE([at_ns1], [[$PYTHON $srcdir/test-l7.py]], [http0.pid])
+NS_CHECK_EXEC([at_ns0], [wget 10.1.1.2 -t 3 -T 1 --retry-connrefused -v -o wget0.log])
+
+AT_CHECK([conntrack -L 2>&1 | FORMAT_CT(10.1.1.2) | grep TIME], [0], [dnl
+TIME_WAIT src=10.1.1.1 dst=10.1.1.2 sport=<cleared> dport=<cleared> src=10.1.1.2 dst=10.1.1.1 sport=<cleared> dport=<cleared> [[ASSURED]] mark=1 use=1
+])
+
+dnl HTTP requests from p2->p3 should fail due to network failure.
+dnl Try 3 times, in 1 second intervals.
+NETNS_DAEMONIZE([at_ns3], [[$PYTHON $srcdir/test-l7.py]], [http1.pid])
+NS_CHECK_EXEC([at_ns2], [wget 10.1.1.4 -t 3 -T 1 -v -o wget1.log], [4])
+
+AT_CHECK([conntrack -L 2>&1 | FORMAT_CT(10.1.1.4)], [0], [dnl
+SYN_RECV src=10.1.1.3 dst=10.1.1.4 sport=<cleared> dport=<cleared> src=10.1.1.4 dst=10.1.1.3 sport=<cleared> dport=<cleared> mark=2 use=1
+])
+
+OVS_TRAFFIC_VSWITCHD_STOP
+AT_CLEANUP
+
+AT_SETUP([conntrack - ICMP related])
+CHECK_CONNTRACK()
+OVS_TRAFFIC_VSWITCHD_START(
+   [set-fail-mode br0 secure -- ])
+
+ADD_NAMESPACES(at_ns0, at_ns1)
+
+ADD_VETH(p0, at_ns0, br0, "10.1.1.1/24")
+ADD_VETH(p1, at_ns1, br0, "10.1.1.2/24")
+
+dnl Allow UDP traffic from ns0->ns1. Only allow related ICMP responses back.
+AT_DATA([flows.txt], [dnl
+priority=1,action=drop
+priority=10,arp,action=normal
+priority=100,in_port=1,udp,action=ct(commit,exec(set_field:1->ct_mark)),2
+priority=100,in_port=2,icmp,ct_state=-trk,action=ct(table=0)
+priority=100,in_port=2,icmp,ct_state=+trk+rel,ct_mark=1,action=1
+])
+
+AT_CHECK([ovs-ofctl add-flows br0 flows.txt])
+
+dnl UDP packets from ns0->ns1 should solicit "destination unreachable" response.
+dnl We pass "-q 1" here to handle openbsd-style nc that can't quit immediately.
+NS_CHECK_EXEC([at_ns0], [bash -c "echo a | nc -q 1 -u 10.1.1.2 10000"])
+
+AT_CHECK([ovs-appctl revalidator/purge], [0])
+AT_CHECK([ovs-ofctl dump-flows br0 | ofctl_strip | sort | grep -v drop], [0], [dnl
+ n_packets=1, n_bytes=44, priority=100,udp,in_port=1 actions=ct(commit,exec(load:0x1->NXM_NX_CT_MARK[[]])),output:2
+ n_packets=1, n_bytes=72, priority=100,ct_state=+rel+trk,ct_mark=0x1,icmp,in_port=2 actions=output:1
+ n_packets=1, n_bytes=72, priority=100,ct_state=-trk,icmp,in_port=2 actions=ct(table=0)
+ n_packets=2, n_bytes=84, priority=10,arp actions=NORMAL
+NXST_FLOW reply:
+])
+
+OVS_TRAFFIC_VSWITCHD_STOP
+AT_CLEANUP
+
 AT_SETUP([conntrack - ICMP related 2])
 CHECK_CONNTRACK()
 OVS_TRAFFIC_VSWITCHD_START(
diff --git a/tests/test-odp.c b/tests/test-odp.c
index 0e65ced..245e1f9 100644
--- a/tests/test-odp.c
+++ b/tests/test-odp.c
@@ -60,6 +60,7 @@  parse_keys(bool wc_keys)
                     .recirc = true,
                     .ct_state = true,
                     .ct_zone = true,
+                    .ct_mark = true,
                 },
             };
 
diff --git a/utilities/ovs-ofctl.8.in b/utilities/ovs-ofctl.8.in
index 91708f7..afeeaeb 100644
--- a/utilities/ovs-ofctl.8.in
+++ b/utilities/ovs-ofctl.8.in
@@ -1364,6 +1364,10 @@  by using the \fBct\fR action.
 .IP \fBct_zone=\fIvalue
 Matches connection zone \fIvalue\fR exactly.
 .
+.IP \fBct_mark=\fIvalue\fR[\fB/\fImask\fR]
+Matches connection mark \fIvalue\fR either exactly or with optional
+\fImask\fR.
+.
 .PP
 Defining IPv6 flows (those with \fBdl_type\fR equal to 0x86dd) requires
 support for NXM.  The following shorthand notations are available for
@@ -1627,6 +1631,18 @@  is not provided, then the default is to use zone zero. The \fBzone\fR may be
 specified either as an immediate 16-bit \fIvalue\fR, or may be provided from an
 NXM field \fIsrc\fR. The \fIstart\fR and \fIend\fR pair are inclusive, and must
 specify a 16-bit range within the field.
+.IP \fBexec\fB(\fR[\fIaction\fR][\fB,\fIaction\fR...]\fB)\fR
+Perform actions within the context of connection tracking. Some actions may
+only be performed in this context, for instance modifying the ct_mark
+field:
+.
+.RS
+.IP \fBset_field:\fIvalue\fR->ct_mark\fR
+Store a 32-bit metadata value with the connection. This will populate the
+\fBct_mark\fR flow field if the \fBtable\fR is specified in the \fBct\fR
+action.
+.RE
+.
 .RE
 .IP
 Currently, connection tracking is only available on Linux kernels with the