Patchwork ulogd2: raw2packet_BASE changes

login
register
mail settings
Submitter Mr Dash Four
Date March 5, 2013, 12:48 p.m.
Message ID <5135E98B.20409@googlemail.com>
Download mbox | patch
Permalink /patch/225003/
State Not Applicable
Headers show

Comments

Mr Dash Four - March 5, 2013, 12:48 p.m.
This patch makes it possible in the event of icmp destination-unreachable
type messages, all "related" connection properties to be made available
to the BASE raw2packet ulogd2 module for use as ulogd2 keys to registered
plugins.

Such keys are named as "o.X.Y", where "X" is the protocol used (tcp, udp
and so on), and "Y" is the property name in question.

All such properties vary depending on the protocol used for the "related"
connection.

Signed-off-by: Mr Dash Four <mr.dash.four@googlemail.com>
---
  filter/raw2packet/ulogd_raw2packet_BASE.c |  457 +++++++++++++++++++++++++----
  1 file changed, 396 insertions(+), 61 deletions(-)




--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Eric Leblond - March 7, 2013, 7:58 a.m.
Hello,

Thanks a lot for this patch.

On Tue, 2013-03-05 at 12:48 +0000, Mr Dash Four wrote:
> This patch makes it possible in the event of icmp destination-unreachable
> type messages, all "related" connection properties to be made available
> to the BASE raw2packet ulogd2 module for use as ulogd2 keys to registered
> plugins.
> 
> Such keys are named as "o.X.Y", where "X" is the protocol used (tcp, udp
> and so on), and "Y" is the property name in question.
> 
> All such properties vary depending on the protocol used for the "related"
> connection.

It is an interesting work. The information contained in these messages
are really meaningful and they need to be contained somewhere. I'm
wondering if it is not more convenient to simply pass a field with raw
data to the stack and have it used as binary in output plugins.
This way, we will treat all cases (I mean all protocols, ipsec or gre
are not supported in your patch) and the impact on the rest of the
plugins will be smaller.
I really fear changes on all output plugins will be huge without
handling all cases.

This will put work on interface and maybe it is a bit too much for them.
If we consider the problem from an admin point of view the most
meaningful information is the tuple (IP src and dest, proto, src and dst
port) the rest is a bit too much.

Considering this, I will not accept this patch but I will be happy with
a version exporting the binary information and some selected fields.

BR,

> 
> Signed-off-by: Mr Dash Four <mr.dash.four@googlemail.com>
> ---
>   filter/raw2packet/ulogd_raw2packet_BASE.c |  457 +++++++++++++++++++++++++----
>   1 file changed, 396 insertions(+), 61 deletions(-)
> 
> diff --git a/filter/raw2packet/ulogd_raw2packet_BASE.c b/filter/raw2packet/ulogd_raw2packet_BASE.c
> index 8dfe38e..071fd5a 100644
> --- a/filter/raw2packet/ulogd_raw2packet_BASE.c
> +++ b/filter/raw2packet/ulogd_raw2packet_BASE.c
> @@ -114,6 +114,47 @@ enum output_keys {
>   	KEY_SCTP_SPORT,
>   	KEY_SCTP_DPORT,
>   	KEY_SCTP_CSUM,
> +	KEY_O_IP_SADDR,
> +	KEY_O_IP_DADDR,
> +	KEY_O_IP_PROTOCOL,
> +	KEY_O_IP_TOS,
> +	KEY_O_IP_TTL,
> +	KEY_O_IP_TOTLEN,
> +	KEY_O_IP_IHL,
> +	KEY_O_IP_CSUM,
> +	KEY_O_IP_ID,
> +	KEY_O_IP_FRAGOFF,
> +	KEY_O_TCP_SPORT,
> +	KEY_O_TCP_DPORT,
> +	KEY_O_TCP_SEQ,
> +	KEY_O_TCP_ACKSEQ,
> +	KEY_O_TCP_WINDOW,
> +	KEY_O_TCP_OFFSET,
> +	KEY_O_TCP_RESERVED,
> +	KEY_O_TCP_URG,
> +	KEY_O_TCP_URGP,
> +	KEY_O_TCP_ACK,
> +	KEY_O_TCP_PSH,
> +	KEY_O_TCP_RST,
> +	KEY_O_TCP_SYN,
> +	KEY_O_TCP_FIN,
> +	KEY_O_TCP_RES1,
> +	KEY_O_TCP_RES2,
> +	KEY_O_TCP_CSUM,
> +	KEY_O_UDP_SPORT,
> +	KEY_O_UDP_DPORT,
> +	KEY_O_UDP_LEN,
> +	KEY_O_UDP_CSUM,
> +	KEY_O_ICMP_TYPE,
> +	KEY_O_ICMP_CODE,
> +	KEY_O_ICMP_ECHOID,
> +	KEY_O_ICMP_ECHOSEQ,
> +	KEY_O_ICMP_GATEWAY,
> +	KEY_O_ICMP_FRAGMTU,
> +	KEY_O_ICMP_CSUM,
> +	KEY_O_SCTP_SPORT,
> +	KEY_O_SCTP_DPORT,
> +	KEY_O_SCTP_CSUM,
> 
>   };
> 
> @@ -524,39 +565,322 @@ static struct ulogd_key iphdr_rets[] = {
>   		.flags = ULOGD_RETF_NONE,
>   		.name = "sctp.csum",
>   	},
> +	[KEY_O_IP_SADDR] = {
> +		.type = ULOGD_RET_IPADDR,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.ip.saddr",
> +		.ipfix = {
> +			.vendor = IPFIX_VENDOR_IETF,
> +			.field_id = IPFIX_sourceIPv4Address,
> +		},
> +	},
> +	[KEY_O_IP_DADDR] = {
> +		.type = ULOGD_RET_IPADDR,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.ip.daddr",
> +		.ipfix = {
> +			.vendor = IPFIX_VENDOR_IETF,
> +			.field_id = IPFIX_destinationIPv4Address,
> +		},
> +	},
> +	[KEY_O_IP_PROTOCOL] = {
> +		.type = ULOGD_RET_UINT8,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.ip.protocol",
> +		.ipfix = {
> +			.vendor = IPFIX_VENDOR_IETF,
> +			.field_id = IPFIX_protocolIdentifier,
> +		},
> +	},
> +	[KEY_O_IP_TOS] = {
> +		.type = ULOGD_RET_UINT8,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.ip.tos",
> +		.ipfix = {
> +			.vendor = IPFIX_VENDOR_IETF,
> +			.field_id = IPFIX_classOfServiceIPv4,
> +		},
> +	},
> +	[KEY_O_IP_TTL] = {
> +		.type = ULOGD_RET_UINT8,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.ip.ttl",
> +		.ipfix = {
> +			.vendor = IPFIX_VENDOR_IETF,
> +			.field_id = IPFIX_ipTimeToLive,
> +		},
> +	},
> +	[KEY_O_IP_TOTLEN] = {
> +		.type = ULOGD_RET_UINT16,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.ip.totlen",
> +		.ipfix = {
> +			.vendor = IPFIX_VENDOR_IETF,
> +			.field_id = IPFIX_totalLengthIPv4,
> +		},
> +	},
> +	[KEY_O_IP_IHL] = {
> +		.type = ULOGD_RET_UINT8,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.ip.ihl",
> +		.ipfix = {
> +			.vendor = IPFIX_VENDOR_IETF,
> +			.field_id = IPFIX_internetHeaderLengthIPv4,
> +		},
> +	},
> +	[KEY_O_IP_CSUM] = {
> +		.type = ULOGD_RET_UINT16,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.ip.csum",
> +	},
> +	[KEY_O_IP_ID] = {
> +		.type = ULOGD_RET_UINT16,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.ip.id",
> +		.ipfix = {
> +			.vendor = IPFIX_VENDOR_IETF,
> +			.field_id = IPFIX_identificationIPv4,
> +		},
> +	},
> +	[KEY_O_IP_FRAGOFF] = {
> +		.type = ULOGD_RET_UINT16,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.ip.fragoff",
> +		.ipfix = {
> +			.vendor = IPFIX_VENDOR_IETF,
> +			.field_id = IPFIX_fragmentOffsetIPv4,
> +		},
> +	},
> +	[KEY_O_TCP_SPORT] = {
> +		.type = ULOGD_RET_UINT16,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.tcp.sport",
> +		.ipfix = {
> +			.vendor = IPFIX_VENDOR_IETF,
> +			.field_id = IPFIX_tcpSourcePort,
> +		},
> +	},
> +	[KEY_O_TCP_DPORT] = {
> +		.type = ULOGD_RET_UINT16,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.tcp.dport",
> +		.ipfix = {
> +			.vendor = IPFIX_VENDOR_IETF,
> +			.field_id = IPFIX_tcpDestinationPort,
> +		},
> +	},
> +	[KEY_O_TCP_SEQ] = {
> +		.type = ULOGD_RET_UINT32,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.tcp.seq",
> +		.ipfix = {
> +			.vendor = IPFIX_VENDOR_IETF,
> +			.field_id = IPFIX_tcpSequenceNumber,
> +		},
> +	},
> +	[KEY_O_TCP_ACKSEQ] = {
> +		.type = ULOGD_RET_UINT32,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.tcp.ackseq",
> +		.ipfix = {
> +			.vendor = IPFIX_VENDOR_IETF,
> +			.field_id = IPFIX_tcpAcknowledgementNumber,
> +		},
> +	},
> +	[KEY_O_TCP_OFFSET] = {
> +		.type = ULOGD_RET_UINT8,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.tcp.offset",
> +	},
> +	[KEY_O_TCP_RESERVED] = {
> +		.type = ULOGD_RET_UINT8,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.tcp.reserved",
> +	},
> +	[KEY_O_TCP_WINDOW] = {
> +		.type = ULOGD_RET_UINT16,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.tcp.window",
> +		.ipfix = {
> +			.vendor = IPFIX_VENDOR_IETF,
> +			.field_id = IPFIX_tcpWindowSize,
> +		},
> +	},
> +	[KEY_O_TCP_URG] = {
> +		.type = ULOGD_RET_BOOL,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.tcp.urg",
> +	},
> +	[KEY_O_TCP_URGP] = {
> +		.type = ULOGD_RET_UINT16,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.tcp.urgp",
> +		.ipfix = {
> +			.vendor = IPFIX_VENDOR_IETF,
> +			.field_id = IPFIX_tcpUrgentPointer,
> +		},
> +	},
> +	[KEY_O_TCP_ACK] = {
> +		.type = ULOGD_RET_BOOL,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.tcp.ack",
> +	},
> +	[KEY_O_TCP_PSH] = {
> +		.type = ULOGD_RET_BOOL,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.tcp.psh",
> +	},
> +	[KEY_O_TCP_RST] = {
> +		.type = ULOGD_RET_BOOL,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.tcp.rst",
> +	},
> +	[KEY_O_TCP_SYN] = {
> +		.type = ULOGD_RET_BOOL,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.tcp.syn",
> +	},
> +	[KEY_O_TCP_FIN] = {
> +		.type = ULOGD_RET_BOOL,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.tcp.fin",
> +	},
> +	[KEY_O_TCP_RES1] = {
> +		.type = ULOGD_RET_BOOL,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.tcp.res1",
> +	},
> +	[KEY_O_TCP_RES2] = {
> +		.type = ULOGD_RET_BOOL,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.tcp.res2",
> +	},
> +	[KEY_O_TCP_CSUM] = {
> +		.type = ULOGD_RET_UINT16,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.tcp.csum",
> +	},
> +	[KEY_O_UDP_SPORT] = {
> +		.type = ULOGD_RET_UINT16,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.udp.sport",
> +		.ipfix = {
> +			.vendor = IPFIX_VENDOR_IETF,
> +			.field_id = IPFIX_udpSourcePort,
> +		},
> +	},
> +	[KEY_O_UDP_DPORT] = {
> +		.type = ULOGD_RET_UINT16,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.udp.dport",
> +		.ipfix = {
> +			.vendor = IPFIX_VENDOR_IETF,
> +			.field_id = IPFIX_udpDestinationPort,
> +		},
> +	},
> +	[KEY_O_UDP_LEN] = {
> +		.type = ULOGD_RET_UINT16,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.udp.len",
> +	},
> +	[KEY_O_UDP_CSUM] = {
> +		.type = ULOGD_RET_UINT16,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.udp.csum",
> +	},
> +	[KEY_O_ICMP_TYPE] = {
> +		.type = ULOGD_RET_UINT8,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.icmp.type",
> +		.ipfix = {
> +			.vendor = IPFIX_VENDOR_IETF,
> +			.field_id = IPFIX_icmpTypeIPv4,
> +		},
> +	},
> +	[KEY_O_ICMP_CODE] = {
> +		.type = ULOGD_RET_UINT8,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.icmp.code",
> +		.ipfix = {
> +			.vendor = IPFIX_VENDOR_IETF,
> +			.field_id = IPFIX_icmpCodeIPv4,
> +		},
> +	},
> +	[KEY_O_ICMP_ECHOID] = {
> +		.type = ULOGD_RET_UINT16,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.icmp.echoid",
> +	},
> +	[KEY_O_ICMP_ECHOSEQ] = {
> +		.type = ULOGD_RET_UINT16,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.icmp.echoseq",
> +	},
> +	[KEY_O_ICMP_GATEWAY] = {
> +		.type = ULOGD_RET_IPADDR,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.icmp.gateway",
> +	},
> +	[KEY_O_ICMP_FRAGMTU] = {
> +		.type = ULOGD_RET_UINT16,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.icmp.fragmtu",
> +	},
> +	[KEY_O_ICMP_CSUM] = {
> +		.type = ULOGD_RET_UINT16,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.icmp.csum",
> +	},
> +	[KEY_O_SCTP_SPORT] = {
> +		.type = ULOGD_RET_UINT16,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.sctp.sport",
> +	},
> +	[KEY_O_SCTP_DPORT] = {
> +		.type = ULOGD_RET_UINT16,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.sctp.dport",
> +	},
> +	[KEY_O_SCTP_CSUM] = {
> +		.type = ULOGD_RET_UINT32,
> +		.flags = ULOGD_RETF_NONE,
> +		.name = "o.sctp.csum",
> +	},
>   };
> 
> +#define recurse_key(key) recurse ? KEY_##key : KEY_O_##key
> +
>   /***********************************************************************
>    * 			TCP HEADER
>    ***********************************************************************/
> 
>   static int _interp_tcp(struct ulogd_pluginstance *pi, struct tcphdr *tcph,
> -		       u_int32_t len)
> +		       u_int32_t len, u_int32_t recurse)
>   {
>   	struct ulogd_key *ret = pi->output.keys;
> 
>   	if (len < sizeof(struct tcphdr))
>   		return ULOGD_IRET_OK;
>   	
> -	okey_set_u16(&ret[KEY_TCP_SPORT], ntohs(tcph->source));
> -	okey_set_u16(&ret[KEY_TCP_DPORT], ntohs(tcph->dest));
> -	okey_set_u32(&ret[KEY_TCP_SEQ], ntohl(tcph->seq));
> -	okey_set_u32(&ret[KEY_TCP_ACKSEQ], ntohl(tcph->ack_seq));
> -	okey_set_u8(&ret[KEY_TCP_OFFSET], ntohs(tcph->doff));
> -	okey_set_u8(&ret[KEY_TCP_RESERVED], ntohs(tcph->res1));
> -	okey_set_u16(&ret[KEY_TCP_WINDOW], ntohs(tcph->window));
> -
> -	okey_set_b(&ret[KEY_TCP_URG], tcph->urg);
> +	okey_set_u16(&ret[recurse_key(TCP_SPORT)], ntohs(tcph->source));
> +	okey_set_u16(&ret[recurse_key(TCP_DPORT)], ntohs(tcph->dest));
> +	okey_set_u32(&ret[recurse_key(TCP_SEQ)], ntohl(tcph->seq));
> +	okey_set_u32(&ret[recurse_key(TCP_ACKSEQ)], ntohl(tcph->ack_seq));
> +	okey_set_u8(&ret[recurse_key(TCP_OFFSET)], ntohs(tcph->doff));
> +	okey_set_u8(&ret[recurse_key(TCP_RESERVED)], ntohs(tcph->res1));
> +	okey_set_u16(&ret[recurse_key(TCP_WINDOW)], ntohs(tcph->window));
> +
> +	okey_set_b(&ret[recurse_key(TCP_URG)], tcph->urg);
>   	if (tcph->urg)
> -		okey_set_u16(&ret[KEY_TCP_URGP], ntohs(tcph->urg_ptr));
> -	okey_set_b(&ret[KEY_TCP_ACK], tcph->ack);
> -	okey_set_b(&ret[KEY_TCP_PSH], tcph->psh);
> -	okey_set_b(&ret[KEY_TCP_RST], tcph->rst);
> -	okey_set_b(&ret[KEY_TCP_SYN], tcph->syn);
> -	okey_set_b(&ret[KEY_TCP_FIN], tcph->fin);
> -	okey_set_b(&ret[KEY_TCP_RES1], tcph->res1);
> -	okey_set_b(&ret[KEY_TCP_RES2], tcph->res2);
> -	okey_set_u16(&ret[KEY_TCP_CSUM], ntohs(tcph->check));
> +		okey_set_u16(&ret[recurse_key(TCP_URGP)], ntohs(tcph->urg_ptr));
> +	okey_set_b(&ret[recurse_key(TCP_ACK)], tcph->ack);
> +	okey_set_b(&ret[recurse_key(TCP_PSH)], tcph->psh);
> +	okey_set_b(&ret[recurse_key(TCP_RST)], tcph->rst);
> +	okey_set_b(&ret[recurse_key(TCP_SYN)], tcph->syn);
> +	okey_set_b(&ret[recurse_key(TCP_FIN)], tcph->fin);
> +	okey_set_b(&ret[recurse_key(TCP_RES1)], tcph->res1);
> +	okey_set_b(&ret[recurse_key(TCP_RES2)], tcph->res2);
> +	okey_set_u16(&ret[recurse_key(TCP_CSUM)], ntohs(tcph->check));
>   	
>   	return ULOGD_IRET_OK;
>   }
> @@ -566,7 +890,7 @@ static int _interp_tcp(struct ulogd_pluginstance *pi, struct tcphdr *tcph,
>    ***********************************************************************/
> 
>   static int _interp_udp(struct ulogd_pluginstance *pi, struct udphdr *udph,
> -		       u_int32_t len)
> +		       u_int32_t len, u_int32_t recurse)
>   		
>   {
>   	struct ulogd_key *ret = pi->output.keys;
> @@ -574,10 +898,10 @@ static int _interp_udp(struct ulogd_pluginstance *pi, struct udphdr *udph,
>   	if (len < sizeof(struct udphdr))
>   		return ULOGD_IRET_OK;
> 
> -	okey_set_u16(&ret[KEY_UDP_SPORT], ntohs(udph->source));
> -	okey_set_u16(&ret[KEY_UDP_DPORT], ntohs(udph->dest));
> -	okey_set_u16(&ret[KEY_UDP_LEN], ntohs(udph->len));
> -	okey_set_u16(&ret[KEY_UDP_CSUM], ntohs(udph->check));
> +	okey_set_u16(&ret[recurse_key(UDP_SPORT)], ntohs(udph->source));
> +	okey_set_u16(&ret[recurse_key(UDP_DPORT)], ntohs(udph->dest));
> +	okey_set_u16(&ret[recurse_key(UDP_LEN)], ntohs(udph->len));
> +	okey_set_u16(&ret[recurse_key(UDP_CSUM)], ntohs(udph->check));
>   	
>   	return ULOGD_IRET_OK;
>   }
> @@ -595,7 +919,7 @@ typedef struct sctphdr {
>   } __attribute__((packed)) sctp_sctphdr_t;
> 
>   static int _interp_sctp(struct ulogd_pluginstance *pi, struct sctphdr *sctph,
> -		       u_int32_t len)
> +		       u_int32_t len, u_int32_t recurse)
>   		
>   {
>   	struct ulogd_key *ret = pi->output.keys;
> @@ -603,50 +927,60 @@ static int _interp_sctp(struct ulogd_pluginstance *pi, struct sctphdr *sctph,
>   	if (len < sizeof(struct sctphdr))
>   		return ULOGD_IRET_OK;
> 
> -	ret[KEY_SCTP_SPORT].u.value.ui16 = ntohs(sctph->source);
> -	ret[KEY_SCTP_SPORT].flags |= ULOGD_RETF_VALID;
> -	ret[KEY_SCTP_DPORT].u.value.ui16 = ntohs(sctph->dest);
> -	ret[KEY_SCTP_DPORT].flags |= ULOGD_RETF_VALID;
> -	ret[KEY_SCTP_CSUM].u.value.ui32 = ntohl(sctph->checksum);
> -	ret[KEY_SCTP_CSUM].flags |= ULOGD_RETF_VALID;
> +	ret[recurse_key(SCTP_SPORT)].u.value.ui16 = ntohs(sctph->source);
> +	ret[recurse_key(SCTP_SPORT)].flags |= ULOGD_RETF_VALID;
> +	ret[recurse_key(SCTP_DPORT)].u.value.ui16 = ntohs(sctph->dest);
> +	ret[recurse_key(SCTP_DPORT)].flags |= ULOGD_RETF_VALID;
> +	ret[recurse_key(SCTP_CSUM)].u.value.ui32 = ntohl(sctph->checksum);
> +	ret[recurse_key(SCTP_CSUM)].flags |= ULOGD_RETF_VALID;
>   	
>   	return ULOGD_IRET_OK;
>   }
> 
> +static int _interp_iphdr(struct ulogd_pluginstance *pi, struct iphdr *iph, u_int32_t len,
> +			 u_int32_t recurse);
> +
>   /***********************************************************************
>    * 			ICMP HEADER
>    ***********************************************************************/
> 
>   static int _interp_icmp(struct ulogd_pluginstance *pi, struct icmphdr *icmph,
> -			u_int32_t len)
> +			u_int32_t len, u_int32_t recurse)
>   {
>   	struct ulogd_key *ret = pi->output.keys;
> 
>   	if (len < sizeof(struct icmphdr))
>   		return ULOGD_IRET_OK;
> 
> -	okey_set_u8(&ret[KEY_ICMP_TYPE], icmph->type);
> -	okey_set_u8(&ret[KEY_ICMP_CODE], icmph->code);
> +	okey_set_u8(&ret[recurse_key(ICMP_TYPE)], icmph->type);
> +	okey_set_u8(&ret[recurse_key(ICMP_CODE)], icmph->code);
> 
>   	switch (icmph->type) {
>   	case ICMP_ECHO:
>   	case ICMP_ECHOREPLY:
> -		okey_set_u16(&ret[KEY_ICMP_ECHOID], ntohs(icmph->un.echo.id));
> -		okey_set_u16(&ret[KEY_ICMP_ECHOSEQ],
> +			okey_set_u16(&ret[recurse_key(ICMP_ECHOID)], ntohs(icmph->un.echo.id));
> +			okey_set_u16(&ret[recurse_key(ICMP_ECHOSEQ)],
>   			     ntohs(icmph->un.echo.sequence));
>   		break;
>   	case ICMP_REDIRECT:
>   	case ICMP_PARAMETERPROB:
> -		okey_set_u32(&ret[KEY_ICMP_GATEWAY], ntohl(icmph->un.gateway));
> +			okey_set_u32(&ret[recurse_key(ICMP_GATEWAY)], ntohl(icmph->un.gateway));
>   		break;
>   	case ICMP_DEST_UNREACH:
> -		if (icmph->code == ICMP_FRAG_NEEDED) {
> -			okey_set_u16(&ret[KEY_ICMP_FRAGMTU],
> +		case ICMP_SOURCE_QUENCH:
> +		case ICMP_TIME_EXCEEDED: {
> +			void *nexthdr = (u_int32_t *)icmph + 2;
> +			len -= sizeof(struct icmphdr);
> +			_interp_iphdr(pi, nexthdr, len, 0);
> +			if (icmph->type == ICMP_DEST_UNREACH &&
> +			    icmph->code == ICMP_FRAG_NEEDED) {
> +				okey_set_u16(&ret[recurse_key(ICMP_FRAGMTU)],
>   				     ntohs(icmph->un.frag.mtu));
>   		}
>   		break;
>   	}
> -	okey_set_u16(&ret[KEY_ICMP_CSUM], icmph->checksum);
> +	}
> +	okey_set_u16(&ret[recurse_key(ICMP_CSUM)], icmph->checksum);
> 
>   	return ULOGD_IRET_OK;
>   }
> @@ -704,40 +1038,39 @@ static int _interp_ahesp(struct ulogd_pluginstance *pi, void *protoh,
>    * 			IP HEADER
>    ***********************************************************************/
> 
> -static int _interp_iphdr(struct ulogd_pluginstance *pi, u_int32_t len)
> +static int _interp_iphdr(struct ulogd_pluginstance *pi, struct iphdr *iph, u_int32_t len,
> +			 u_int32_t recurse)
>   {
>   	struct ulogd_key *ret = pi->output.keys;
> -	struct iphdr *iph =
> -		ikey_get_ptr(&pi->input.keys[INKEY_RAW_PCKT]);
>   	void *nexthdr = (u_int32_t *)iph + iph->ihl;
> 
>   	if (len < sizeof(struct iphdr) || len <= (u_int32_t)(iph->ihl * 4))
>   		return ULOGD_IRET_OK;
>   	len -= iph->ihl * 4;
> 
> -	okey_set_u32(&ret[KEY_IP_SADDR], iph->saddr);
> -	okey_set_u32(&ret[KEY_IP_DADDR], iph->daddr);
> -	okey_set_u8(&ret[KEY_IP_PROTOCOL], iph->protocol);
> -	okey_set_u8(&ret[KEY_IP_TOS], iph->tos);
> -	okey_set_u8(&ret[KEY_IP_TTL], iph->ttl);
> -	okey_set_u16(&ret[KEY_IP_TOTLEN], ntohs(iph->tot_len));
> -	okey_set_u8(&ret[KEY_IP_IHL], iph->ihl);
> -	okey_set_u16(&ret[KEY_IP_CSUM], ntohs(iph->check));
> -	okey_set_u16(&ret[KEY_IP_ID], ntohs(iph->id));
> -	okey_set_u16(&ret[KEY_IP_FRAGOFF], ntohs(iph->frag_off));
> +	okey_set_u32(&ret[recurse_key(IP_SADDR)], iph->saddr);
> +	okey_set_u32(&ret[recurse_key(IP_DADDR)], iph->daddr);
> +	okey_set_u8(&ret[recurse_key(IP_PROTOCOL)], iph->protocol);
> +	okey_set_u8(&ret[recurse_key(IP_TOS)], iph->tos);
> +	okey_set_u8(&ret[recurse_key(IP_TTL)], iph->ttl);
> +	okey_set_u16(&ret[recurse_key(IP_TOTLEN)], ntohs(iph->tot_len));
> +	okey_set_u8(&ret[recurse_key(IP_IHL)], iph->ihl);
> +	okey_set_u16(&ret[recurse_key(IP_CSUM)], ntohs(iph->check));
> +	okey_set_u16(&ret[recurse_key(IP_ID)], ntohs(iph->id));
> +	okey_set_u16(&ret[recurse_key(IP_FRAGOFF)], ntohs(iph->frag_off));
> 
>   	switch (iph->protocol) {
>   	case IPPROTO_TCP:
> -		_interp_tcp(pi, nexthdr, len);
> +		_interp_tcp(pi, nexthdr, len, recurse);
>   		break;
>   	case IPPROTO_UDP:
> -		_interp_udp(pi, nexthdr, len);
> +		_interp_udp(pi, nexthdr, len, recurse);
>   		break;
>   	case IPPROTO_ICMP:
> -		_interp_icmp(pi, nexthdr, len);
> +		_interp_icmp(pi, nexthdr, len, recurse);
>   		break;
>   	case IPPROTO_SCTP:
> -		_interp_sctp(pi, nexthdr, len);
> +		_interp_sctp(pi, nexthdr, len, recurse);
>   		break;
>   	case IPPROTO_AH:
>   	case IPPROTO_ESP:
> @@ -864,10 +1197,10 @@ static int _interp_ipv6hdr(struct ulogd_pluginstance *pi, u_int32_t len)
> 
>   	switch (curhdr) {
>   	case IPPROTO_TCP:
> -		_interp_tcp(pi, (void *)ipv6h + ptr, len);
> +		_interp_tcp(pi, (void *)ipv6h + ptr, len, 0);
>   		break;
>   	case IPPROTO_UDP:
> -		_interp_udp(pi, (void *)ipv6h + ptr, len);
> +		_interp_udp(pi, (void *)ipv6h + ptr, len, 0);
>   		break;
>   	case IPPROTO_ICMPV6:
>   		_interp_icmpv6(pi, (void *)ipv6h + ptr, len);
> @@ -914,7 +1247,8 @@ static int _interp_bridge(struct ulogd_pluginstance *pi, u_int32_t len)
> 
>   	switch (proto) {
>   	case ETH_P_IP:
> -		_interp_iphdr(pi, len);
> +		_interp_iphdr(pi, ikey_get_ptr(&pi->input.keys[INKEY_RAW_PCKT]),
> +			      len, 1);
>   		break;
>   	case ETH_P_IPV6:
>   		_interp_ipv6hdr(pi, len);
> @@ -940,7 +1274,8 @@ static int _interp_pkt(struct ulogd_pluginstance *pi)
> 
>   	switch (family) {
>   	case AF_INET:
> -		return _interp_iphdr(pi, len);
> +		return _interp_iphdr(pi, ikey_get_ptr(&pi->input.keys[INKEY_RAW_PCKT]),
> +				     len, 1);
>   	case AF_INET6:
>   		return _interp_ipv6hdr(pi, len);
>   	case AF_BRIDGE:
> 
> 
>
Mr Dash Four - March 16, 2013, 2:14 p.m.
> I'm wondering if it is not more convenient to simply pass a field with raw
> data to the stack and have it used as binary in output plugins.
> This way, we will treat all cases (I mean all protocols, ipsec or gre
> are not supported in your patch) and the impact on the rest of the
> plugins will be smaller.
>   
I don't think so.

The very nature of the information contained within strictly depends on 
the protocol used in the original connection. If that was udp for 
example, then the only "additional information" (in other words, what my 
patch can extract from the secondary header) available to the plugins 
would be the KEY_O_UDP_* keys and nothing else.

Besides, if a binary data is passed to the plugins, that data also needs 
to be processed - by *all* plugins connected to the chain, so I don't 
see this as a benefit at all.

> I really fear changes on all output plugins will be huge without
> handling all cases.
>   
Have you done a performance analysis to make that assumption? If so, 
could you post the results for all of us to see what that impact really is?

> This will put work on interface and maybe it is a bit too much for them.
>   
Same question as above?

> If we consider the problem from an admin point of view the most
> meaningful information is the tuple (IP src and dest, proto, src and dst
> port) the rest is a bit too much.
>   
I have been using this addition for over 7 months now (mainly as an 
extension to the PGSQL and GPRINT plugins as they are most-widely 
deployed on my servers here) and from experience I can tell you that, at 
least for the TCP part of the original connection, I need the TCP flags 
(in addition to everything else), simply because a connection can be 
rejected with icmp unreachable because of illegal flags combination. So 
in that particular case, I need to have all TCP flags available as keys.

Besides, I don't see how cutting a few extra keys will add, 
significantly, to the processing time, but if you have done the analysis 
and have it available, please post it here for all of us to see and 
prove me wrong.

> Considering this, I will not accept this patch but I will be happy with
> a version exporting the binary information and some selected fields.
>   
I am not convinced making binary data available (which will be different 
depending on the underlying protocol of the original connection used), 
which needs to be processed by the plugins in the whole chain, as oppose 
to having the *relevant* keys already available to them, will add any 
benefit - both performance-wise, as well as feature-wise.

On a separate note, care to answer the questions I asked you on this 
list over 4 months ago (which are still to be addressed by yourself) [1] 
with regards to the patch I submitted over 6 months ago [2] and do you 
think that it is acceptable for you as a current maintainer of the 
ulogd2 project to ignore addressing such requests? Thanks.

[1] - http://www.spinics.net/lists/netfilter-devel/msg23982.html
[2] - http://www.spinics.net/lists/netfilter-devel/msg23120.html
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Eric Leblond - March 17, 2013, 10:06 a.m.
Hello,

On Sat, 2013-03-16 at 14:14 +0000, Mr Dash Four wrote:
> > I'm wondering if it is not more convenient to simply pass a field with raw
> > data to the stack and have it used as binary in output plugins.
> > This way, we will treat all cases (I mean all protocols, ipsec or gre
> > are not supported in your patch) and the impact on the rest of the
> > plugins will be smaller.
> >   
> I don't think so.
> 
> The very nature of the information contained within strictly depends on 
> the protocol used in the original connection. If that was udp for 
> example, then the only "additional information" (in other words, what my 
> patch can extract from the secondary header) available to the plugins 
> would be the KEY_O_UDP_* keys and nothing else.
> 
> Besides, if a binary data is passed to the plugins, that data also needs 
> to be processed - by *all* plugins connected to the chain, so I don't 
> see this as a benefit at all.
> 
> > I really fear changes on all output plugins will be huge without
> > handling all cases.
> >   
> Have you done a performance analysis to make that assumption? If so, 
> could you post the results for all of us to see what that impact really is?

Do I ever use the word "performance" ? I'm talking about developing
work 

> > This will put work on interface and maybe it is a bit too much for them.
> >   
> Same question as above?

And same answer, I'm talking here about user interface that use ulogd
provided data to give information to user. They will have to decode all
these fields.

> > If we consider the problem from an admin point of view the most
> > meaningful information is the tuple (IP src and dest, proto, src and dst
> > port) the rest is a bit too much.
> >   
> I have been using this addition for over 7 months now (mainly as an 
> extension to the PGSQL and GPRINT plugins as they are most-widely 
> deployed on my servers here) and from experience I can tell you that, at 
> least for the TCP part of the original connection, I need the TCP flags 
> (in addition to everything else), simply because a connection can be 
> rejected with icmp unreachable because of illegal flags combination. So 
> in that particular case, I need to have all TCP flags available as keys.

This information is also available in the binary data I suggest to add.
And you could get it from there if needed.

> > Considering this, I will not accept this patch but I will be happy with
> > a version exporting the binary information and some selected fields.
> >   
> I am not convinced making binary data available (which will be different 
> depending on the underlying protocol of the original connection used), 
> which needs to be processed by the plugins in the whole chain, as oppose 
> to having the *relevant* keys already available to them, will add any 
> benefit - both performance-wise, as well as feature-wise.

I'm talking about information loss. Storing binary format allows you to
get back to the correct information at any time.

> On a separate note, care to answer the questions I asked you on this 
> list over 4 months ago (which are still to be addressed by yourself) [1] 
> with regards to the patch I submitted over 6 months ago [2] and do you 
> think that it is acceptable for you as a current maintainer of the 
> ulogd2 project to ignore addressing such requests? Thanks.
> 
> [1] - http://www.spinics.net/lists/netfilter-devel/msg23982.html

Can you rebase it to current tree and resubmit with white space fixed as
discussed before ?

> [2] - http://www.spinics.net/lists/netfilter-devel/msg23120.html

Do you have provided a patchset containing the Null dereference fix and
the SSL feature as asked by Pablo Neira Ayuso ?

BR,
Mr Dash Four - March 22, 2013, 6:40 p.m.
Eric Leblond wrote:
> Hello,
>
> On Sat, 2013-03-16 at 14:14 +0000, Mr Dash Four wrote:
>   
>>> I'm wondering if it is not more convenient to simply pass a field with raw
>>> data to the stack and have it used as binary in output plugins.
>>> This way, we will treat all cases (I mean all protocols, ipsec or gre
>>> are not supported in your patch) and the impact on the rest of the
>>> plugins will be smaller.
>>>   
>>>       
>> I don't think so.
>>
>> The very nature of the information contained within strictly depends on 
>> the protocol used in the original connection. If that was udp for 
>> example, then the only "additional information" (in other words, what my 
>> patch can extract from the secondary header) available to the plugins 
>> would be the KEY_O_UDP_* keys and nothing else.
>>
>> Besides, if a binary data is passed to the plugins, that data also needs 
>> to be processed - by *all* plugins connected to the chain, so I don't 
>> see this as a benefit at all.
>>
>>     
>>> I really fear changes on all output plugins will be huge without
>>> handling all cases.
>>>   
>>>       
>> Have you done a performance analysis to make that assumption? If so, 
>> could you post the results for all of us to see what that impact really is?
>>     
>
> Do I ever use the word "performance" ? I'm talking about developing
> work 
>
>   
>>> This will put work on interface and maybe it is a bit too much for them.
>>>   
>>>       
>> Same question as above?
>>     
>
> And same answer, I'm talking here about user interface that use ulogd
> provided data to give information to user. They will have to decode all
> these fields.
>   
The way I read your response, it was clear to me that by "this will put 
work on interface and maybe it is a bit too much for them" you had some 
performance concerns with regards to my patch, hence why I asked the 
questions I did in the previous post. The clue for me was in the "too 
much for them" bit.

If I understood you now correctly, you've explained that you have 
concerns over the re-development of the rest of the plugins necessary to 
accommodate these changes, is that right?

If so, again, from experience, I know that at least 2 plugins will 
require light-to-no-work - the GPRINT (which works well without *any* 
modifications) and the PGSQL plugin. I suspect for the other db plugins 
the situation won't be any different.

Admittedly, I am using a heavily-modified version of the PGSQL plugin 
where it took me about 10 minutes to accommodate these changes, as the 
existing PGSQL plugin (and accompanying sql script) in the ulogd2 
repository is ... well, lets just say that I won't recommend this to my 
competitors, let alone use this myself.

I can't really vouch for the rest of the plugins though, but if I go 
ahead with your suggestion above and distribute the new properties I 
listed in my patch as a binary object, then there will be needed the 
same amount of effort required at the very least, if not more, to modify 
the existing plugins to accommodate these changes, as well as be able to 
process the binary data into something more meaningful, so I don't see 
how this would be any different from what is already done in my patch?

>> I have been using this addition for over 7 months now (mainly as an 
>> extension to the PGSQL and GPRINT plugins as they are most-widely 
>> deployed on my servers here) and from experience I can tell you that, at 
>> least for the TCP part of the original connection, I need the TCP flags 
>> (in addition to everything else), simply because a connection can be 
>> rejected with icmp unreachable because of illegal flags combination. So 
>> in that particular case, I need to have all TCP flags available as keys.
>>     
>
> This information is also available in the binary data I suggest to add.
> And you could get it from there if needed.
>   
Except that you didn't. You suggested that I just include the src/dst ip 
addresses and ports, protocol and nothing more.To quote your previous 
response:

> If we consider the problem from an admin point of view the most
> meaningful information is the tuple (IP src and dest, proto, src and dst
> port) the rest is a bit too much.
>   

Care to highlight where in the above you mention TCP flags?

>> [1] - http://www.spinics.net/lists/netfilter-devel/msg23982.html
>>     
>
> Can you rebase it to current tree and resubmit with white space fixed as
> discussed before ?
>   
Care to address the questions I asked you in that post?

>> [2] - http://www.spinics.net/lists/netfilter-devel/msg23120.html
>>     
>
> Do you have provided a patchset containing the Null dereference fix and
> the SSL feature as asked by Pablo Neira Ayuso ?
>   
I'll do that when I get some answers. Besides, the main purpose of this 
patch wasn't fixing the null dereference bug, but to introduce the SSL 
feature and I am yet to hear from you whether you are happy with that 
part of the patch I provided a couple of months ago.
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Patch

diff --git a/filter/raw2packet/ulogd_raw2packet_BASE.c b/filter/raw2packet/ulogd_raw2packet_BASE.c
index 8dfe38e..071fd5a 100644
--- a/filter/raw2packet/ulogd_raw2packet_BASE.c
+++ b/filter/raw2packet/ulogd_raw2packet_BASE.c
@@ -114,6 +114,47 @@  enum output_keys {
  	KEY_SCTP_SPORT,
  	KEY_SCTP_DPORT,
  	KEY_SCTP_CSUM,
+	KEY_O_IP_SADDR,
+	KEY_O_IP_DADDR,
+	KEY_O_IP_PROTOCOL,
+	KEY_O_IP_TOS,
+	KEY_O_IP_TTL,
+	KEY_O_IP_TOTLEN,
+	KEY_O_IP_IHL,
+	KEY_O_IP_CSUM,
+	KEY_O_IP_ID,
+	KEY_O_IP_FRAGOFF,
+	KEY_O_TCP_SPORT,
+	KEY_O_TCP_DPORT,
+	KEY_O_TCP_SEQ,
+	KEY_O_TCP_ACKSEQ,
+	KEY_O_TCP_WINDOW,
+	KEY_O_TCP_OFFSET,
+	KEY_O_TCP_RESERVED,
+	KEY_O_TCP_URG,
+	KEY_O_TCP_URGP,
+	KEY_O_TCP_ACK,
+	KEY_O_TCP_PSH,
+	KEY_O_TCP_RST,
+	KEY_O_TCP_SYN,
+	KEY_O_TCP_FIN,
+	KEY_O_TCP_RES1,
+	KEY_O_TCP_RES2,
+	KEY_O_TCP_CSUM,
+	KEY_O_UDP_SPORT,
+	KEY_O_UDP_DPORT,
+	KEY_O_UDP_LEN,
+	KEY_O_UDP_CSUM,
+	KEY_O_ICMP_TYPE,
+	KEY_O_ICMP_CODE,
+	KEY_O_ICMP_ECHOID,
+	KEY_O_ICMP_ECHOSEQ,
+	KEY_O_ICMP_GATEWAY,
+	KEY_O_ICMP_FRAGMTU,
+	KEY_O_ICMP_CSUM,
+	KEY_O_SCTP_SPORT,
+	KEY_O_SCTP_DPORT,
+	KEY_O_SCTP_CSUM,

  };

@@ -524,39 +565,322 @@  static struct ulogd_key iphdr_rets[] = {
  		.flags = ULOGD_RETF_NONE,
  		.name = "sctp.csum",
  	},
+	[KEY_O_IP_SADDR] = {
+		.type = ULOGD_RET_IPADDR,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.ip.saddr",
+		.ipfix = {
+			.vendor = IPFIX_VENDOR_IETF,
+			.field_id = IPFIX_sourceIPv4Address,
+		},
+	},
+	[KEY_O_IP_DADDR] = {
+		.type = ULOGD_RET_IPADDR,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.ip.daddr",
+		.ipfix = {
+			.vendor = IPFIX_VENDOR_IETF,
+			.field_id = IPFIX_destinationIPv4Address,
+		},
+	},
+	[KEY_O_IP_PROTOCOL] = {
+		.type = ULOGD_RET_UINT8,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.ip.protocol",
+		.ipfix = {
+			.vendor = IPFIX_VENDOR_IETF,
+			.field_id = IPFIX_protocolIdentifier,
+		},
+	},
+	[KEY_O_IP_TOS] = {
+		.type = ULOGD_RET_UINT8,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.ip.tos",
+		.ipfix = {
+			.vendor = IPFIX_VENDOR_IETF,
+			.field_id = IPFIX_classOfServiceIPv4,
+		},
+	},
+	[KEY_O_IP_TTL] = {
+		.type = ULOGD_RET_UINT8,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.ip.ttl",
+		.ipfix = {
+			.vendor = IPFIX_VENDOR_IETF,
+			.field_id = IPFIX_ipTimeToLive,
+		},
+	},
+	[KEY_O_IP_TOTLEN] = {
+		.type = ULOGD_RET_UINT16,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.ip.totlen",
+		.ipfix = {
+			.vendor = IPFIX_VENDOR_IETF,
+			.field_id = IPFIX_totalLengthIPv4,
+		},
+	},
+	[KEY_O_IP_IHL] = {
+		.type = ULOGD_RET_UINT8,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.ip.ihl",
+		.ipfix = {
+			.vendor = IPFIX_VENDOR_IETF,
+			.field_id = IPFIX_internetHeaderLengthIPv4,
+		},
+	},
+	[KEY_O_IP_CSUM] = {
+		.type = ULOGD_RET_UINT16,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.ip.csum",
+	},
+	[KEY_O_IP_ID] = {
+		.type = ULOGD_RET_UINT16,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.ip.id",
+		.ipfix = {
+			.vendor = IPFIX_VENDOR_IETF,
+			.field_id = IPFIX_identificationIPv4,
+		},
+	},
+	[KEY_O_IP_FRAGOFF] = {
+		.type = ULOGD_RET_UINT16,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.ip.fragoff",
+		.ipfix = {
+			.vendor = IPFIX_VENDOR_IETF,
+			.field_id = IPFIX_fragmentOffsetIPv4,
+		},
+	},
+	[KEY_O_TCP_SPORT] = {
+		.type = ULOGD_RET_UINT16,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.tcp.sport",
+		.ipfix = {
+			.vendor = IPFIX_VENDOR_IETF,
+			.field_id = IPFIX_tcpSourcePort,
+		},
+	},
+	[KEY_O_TCP_DPORT] = {
+		.type = ULOGD_RET_UINT16,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.tcp.dport",
+		.ipfix = {
+			.vendor = IPFIX_VENDOR_IETF,
+			.field_id = IPFIX_tcpDestinationPort,
+		},
+	},
+	[KEY_O_TCP_SEQ] = {
+		.type = ULOGD_RET_UINT32,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.tcp.seq",
+		.ipfix = {
+			.vendor = IPFIX_VENDOR_IETF,
+			.field_id = IPFIX_tcpSequenceNumber,
+		},
+	},
+	[KEY_O_TCP_ACKSEQ] = {
+		.type = ULOGD_RET_UINT32,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.tcp.ackseq",
+		.ipfix = {
+			.vendor = IPFIX_VENDOR_IETF,
+			.field_id = IPFIX_tcpAcknowledgementNumber,
+		},
+	},
+	[KEY_O_TCP_OFFSET] = {
+		.type = ULOGD_RET_UINT8,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.tcp.offset",
+	},
+	[KEY_O_TCP_RESERVED] = {
+		.type = ULOGD_RET_UINT8,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.tcp.reserved",
+	},
+	[KEY_O_TCP_WINDOW] = {
+		.type = ULOGD_RET_UINT16,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.tcp.window",
+		.ipfix = {
+			.vendor = IPFIX_VENDOR_IETF,
+			.field_id = IPFIX_tcpWindowSize,
+		},
+	},
+	[KEY_O_TCP_URG] = {
+		.type = ULOGD_RET_BOOL,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.tcp.urg",
+	},
+	[KEY_O_TCP_URGP] = {
+		.type = ULOGD_RET_UINT16,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.tcp.urgp",
+		.ipfix = {
+			.vendor = IPFIX_VENDOR_IETF,
+			.field_id = IPFIX_tcpUrgentPointer,
+		},
+	},
+	[KEY_O_TCP_ACK] = {
+		.type = ULOGD_RET_BOOL,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.tcp.ack",
+	},
+	[KEY_O_TCP_PSH] = {
+		.type = ULOGD_RET_BOOL,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.tcp.psh",
+	},
+	[KEY_O_TCP_RST] = {
+		.type = ULOGD_RET_BOOL,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.tcp.rst",
+	},
+	[KEY_O_TCP_SYN] = {
+		.type = ULOGD_RET_BOOL,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.tcp.syn",
+	},
+	[KEY_O_TCP_FIN] = {
+		.type = ULOGD_RET_BOOL,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.tcp.fin",
+	},
+	[KEY_O_TCP_RES1] = {
+		.type = ULOGD_RET_BOOL,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.tcp.res1",
+	},
+	[KEY_O_TCP_RES2] = {
+		.type = ULOGD_RET_BOOL,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.tcp.res2",
+	},
+	[KEY_O_TCP_CSUM] = {
+		.type = ULOGD_RET_UINT16,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.tcp.csum",
+	},
+	[KEY_O_UDP_SPORT] = {
+		.type = ULOGD_RET_UINT16,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.udp.sport",
+		.ipfix = {
+			.vendor = IPFIX_VENDOR_IETF,
+			.field_id = IPFIX_udpSourcePort,
+		},
+	},
+	[KEY_O_UDP_DPORT] = {
+		.type = ULOGD_RET_UINT16,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.udp.dport",
+		.ipfix = {
+			.vendor = IPFIX_VENDOR_IETF,
+			.field_id = IPFIX_udpDestinationPort,
+		},
+	},
+	[KEY_O_UDP_LEN] = {
+		.type = ULOGD_RET_UINT16,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.udp.len",
+	},
+	[KEY_O_UDP_CSUM] = {
+		.type = ULOGD_RET_UINT16,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.udp.csum",
+	},
+	[KEY_O_ICMP_TYPE] = {
+		.type = ULOGD_RET_UINT8,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.icmp.type",
+		.ipfix = {
+			.vendor = IPFIX_VENDOR_IETF,
+			.field_id = IPFIX_icmpTypeIPv4,
+		},
+	},
+	[KEY_O_ICMP_CODE] = {
+		.type = ULOGD_RET_UINT8,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.icmp.code",
+		.ipfix = {
+			.vendor = IPFIX_VENDOR_IETF,
+			.field_id = IPFIX_icmpCodeIPv4,
+		},
+	},
+	[KEY_O_ICMP_ECHOID] = {
+		.type = ULOGD_RET_UINT16,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.icmp.echoid",
+	},
+	[KEY_O_ICMP_ECHOSEQ] = {
+		.type = ULOGD_RET_UINT16,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.icmp.echoseq",
+	},
+	[KEY_O_ICMP_GATEWAY] = {
+		.type = ULOGD_RET_IPADDR,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.icmp.gateway",
+	},
+	[KEY_O_ICMP_FRAGMTU] = {
+		.type = ULOGD_RET_UINT16,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.icmp.fragmtu",
+	},
+	[KEY_O_ICMP_CSUM] = {
+		.type = ULOGD_RET_UINT16,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.icmp.csum",
+	},
+	[KEY_O_SCTP_SPORT] = {
+		.type = ULOGD_RET_UINT16,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.sctp.sport",
+	},
+	[KEY_O_SCTP_DPORT] = {
+		.type = ULOGD_RET_UINT16,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.sctp.dport",
+	},
+	[KEY_O_SCTP_CSUM] = {
+		.type = ULOGD_RET_UINT32,
+		.flags = ULOGD_RETF_NONE,
+		.name = "o.sctp.csum",
+	},
  };

+#define recurse_key(key) recurse ? KEY_##key : KEY_O_##key
+
  /***********************************************************************
   * 			TCP HEADER
   ***********************************************************************/

  static int _interp_tcp(struct ulogd_pluginstance *pi, struct tcphdr *tcph,
-		       u_int32_t len)
+		       u_int32_t len, u_int32_t recurse)
  {
  	struct ulogd_key *ret = pi->output.keys;

  	if (len < sizeof(struct tcphdr))
  		return ULOGD_IRET_OK;
  	
-	okey_set_u16(&ret[KEY_TCP_SPORT], ntohs(tcph->source));
-	okey_set_u16(&ret[KEY_TCP_DPORT], ntohs(tcph->dest));
-	okey_set_u32(&ret[KEY_TCP_SEQ], ntohl(tcph->seq));
-	okey_set_u32(&ret[KEY_TCP_ACKSEQ], ntohl(tcph->ack_seq));
-	okey_set_u8(&ret[KEY_TCP_OFFSET], ntohs(tcph->doff));
-	okey_set_u8(&ret[KEY_TCP_RESERVED], ntohs(tcph->res1));
-	okey_set_u16(&ret[KEY_TCP_WINDOW], ntohs(tcph->window));
-
-	okey_set_b(&ret[KEY_TCP_URG], tcph->urg);
+	okey_set_u16(&ret[recurse_key(TCP_SPORT)], ntohs(tcph->source));
+	okey_set_u16(&ret[recurse_key(TCP_DPORT)], ntohs(tcph->dest));
+	okey_set_u32(&ret[recurse_key(TCP_SEQ)], ntohl(tcph->seq));
+	okey_set_u32(&ret[recurse_key(TCP_ACKSEQ)], ntohl(tcph->ack_seq));
+	okey_set_u8(&ret[recurse_key(TCP_OFFSET)], ntohs(tcph->doff));
+	okey_set_u8(&ret[recurse_key(TCP_RESERVED)], ntohs(tcph->res1));
+	okey_set_u16(&ret[recurse_key(TCP_WINDOW)], ntohs(tcph->window));
+
+	okey_set_b(&ret[recurse_key(TCP_URG)], tcph->urg);
  	if (tcph->urg)
-		okey_set_u16(&ret[KEY_TCP_URGP], ntohs(tcph->urg_ptr));
-	okey_set_b(&ret[KEY_TCP_ACK], tcph->ack);
-	okey_set_b(&ret[KEY_TCP_PSH], tcph->psh);
-	okey_set_b(&ret[KEY_TCP_RST], tcph->rst);
-	okey_set_b(&ret[KEY_TCP_SYN], tcph->syn);
-	okey_set_b(&ret[KEY_TCP_FIN], tcph->fin);
-	okey_set_b(&ret[KEY_TCP_RES1], tcph->res1);
-	okey_set_b(&ret[KEY_TCP_RES2], tcph->res2);
-	okey_set_u16(&ret[KEY_TCP_CSUM], ntohs(tcph->check));
+		okey_set_u16(&ret[recurse_key(TCP_URGP)], ntohs(tcph->urg_ptr));
+	okey_set_b(&ret[recurse_key(TCP_ACK)], tcph->ack);
+	okey_set_b(&ret[recurse_key(TCP_PSH)], tcph->psh);
+	okey_set_b(&ret[recurse_key(TCP_RST)], tcph->rst);
+	okey_set_b(&ret[recurse_key(TCP_SYN)], tcph->syn);
+	okey_set_b(&ret[recurse_key(TCP_FIN)], tcph->fin);
+	okey_set_b(&ret[recurse_key(TCP_RES1)], tcph->res1);
+	okey_set_b(&ret[recurse_key(TCP_RES2)], tcph->res2);
+	okey_set_u16(&ret[recurse_key(TCP_CSUM)], ntohs(tcph->check));
  	
  	return ULOGD_IRET_OK;
  }
@@ -566,7 +890,7 @@  static int _interp_tcp(struct ulogd_pluginstance *pi, struct tcphdr *tcph,
   ***********************************************************************/

  static int _interp_udp(struct ulogd_pluginstance *pi, struct udphdr *udph,
-		       u_int32_t len)
+		       u_int32_t len, u_int32_t recurse)
  		
  {
  	struct ulogd_key *ret = pi->output.keys;
@@ -574,10 +898,10 @@  static int _interp_udp(struct ulogd_pluginstance *pi, struct udphdr *udph,
  	if (len < sizeof(struct udphdr))
  		return ULOGD_IRET_OK;

-	okey_set_u16(&ret[KEY_UDP_SPORT], ntohs(udph->source));
-	okey_set_u16(&ret[KEY_UDP_DPORT], ntohs(udph->dest));
-	okey_set_u16(&ret[KEY_UDP_LEN], ntohs(udph->len));
-	okey_set_u16(&ret[KEY_UDP_CSUM], ntohs(udph->check));
+	okey_set_u16(&ret[recurse_key(UDP_SPORT)], ntohs(udph->source));
+	okey_set_u16(&ret[recurse_key(UDP_DPORT)], ntohs(udph->dest));
+	okey_set_u16(&ret[recurse_key(UDP_LEN)], ntohs(udph->len));
+	okey_set_u16(&ret[recurse_key(UDP_CSUM)], ntohs(udph->check));
  	
  	return ULOGD_IRET_OK;
  }
@@ -595,7 +919,7 @@  typedef struct sctphdr {
  } __attribute__((packed)) sctp_sctphdr_t;

  static int _interp_sctp(struct ulogd_pluginstance *pi, struct sctphdr *sctph,
-		       u_int32_t len)
+		       u_int32_t len, u_int32_t recurse)
  		
  {
  	struct ulogd_key *ret = pi->output.keys;
@@ -603,50 +927,60 @@  static int _interp_sctp(struct ulogd_pluginstance *pi, struct sctphdr *sctph,
  	if (len < sizeof(struct sctphdr))
  		return ULOGD_IRET_OK;

-	ret[KEY_SCTP_SPORT].u.value.ui16 = ntohs(sctph->source);
-	ret[KEY_SCTP_SPORT].flags |= ULOGD_RETF_VALID;
-	ret[KEY_SCTP_DPORT].u.value.ui16 = ntohs(sctph->dest);
-	ret[KEY_SCTP_DPORT].flags |= ULOGD_RETF_VALID;
-	ret[KEY_SCTP_CSUM].u.value.ui32 = ntohl(sctph->checksum);
-	ret[KEY_SCTP_CSUM].flags |= ULOGD_RETF_VALID;
+	ret[recurse_key(SCTP_SPORT)].u.value.ui16 = ntohs(sctph->source);
+	ret[recurse_key(SCTP_SPORT)].flags |= ULOGD_RETF_VALID;
+	ret[recurse_key(SCTP_DPORT)].u.value.ui16 = ntohs(sctph->dest);
+	ret[recurse_key(SCTP_DPORT)].flags |= ULOGD_RETF_VALID;
+	ret[recurse_key(SCTP_CSUM)].u.value.ui32 = ntohl(sctph->checksum);
+	ret[recurse_key(SCTP_CSUM)].flags |= ULOGD_RETF_VALID;
  	
  	return ULOGD_IRET_OK;
  }

+static int _interp_iphdr(struct ulogd_pluginstance *pi, struct iphdr *iph, u_int32_t len,
+			 u_int32_t recurse);
+
  /***********************************************************************
   * 			ICMP HEADER
   ***********************************************************************/

  static int _interp_icmp(struct ulogd_pluginstance *pi, struct icmphdr *icmph,
-			u_int32_t len)
+			u_int32_t len, u_int32_t recurse)
  {
  	struct ulogd_key *ret = pi->output.keys;

  	if (len < sizeof(struct icmphdr))
  		return ULOGD_IRET_OK;

-	okey_set_u8(&ret[KEY_ICMP_TYPE], icmph->type);
-	okey_set_u8(&ret[KEY_ICMP_CODE], icmph->code);
+	okey_set_u8(&ret[recurse_key(ICMP_TYPE)], icmph->type);
+	okey_set_u8(&ret[recurse_key(ICMP_CODE)], icmph->code);

  	switch (icmph->type) {
  	case ICMP_ECHO:
  	case ICMP_ECHOREPLY:
-		okey_set_u16(&ret[KEY_ICMP_ECHOID], ntohs(icmph->un.echo.id));
-		okey_set_u16(&ret[KEY_ICMP_ECHOSEQ],
+			okey_set_u16(&ret[recurse_key(ICMP_ECHOID)], ntohs(icmph->un.echo.id));
+			okey_set_u16(&ret[recurse_key(ICMP_ECHOSEQ)],
  			     ntohs(icmph->un.echo.sequence));
  		break;
  	case ICMP_REDIRECT:
  	case ICMP_PARAMETERPROB:
-		okey_set_u32(&ret[KEY_ICMP_GATEWAY], ntohl(icmph->un.gateway));
+			okey_set_u32(&ret[recurse_key(ICMP_GATEWAY)], ntohl(icmph->un.gateway));
  		break;
  	case ICMP_DEST_UNREACH:
-		if (icmph->code == ICMP_FRAG_NEEDED) {
-			okey_set_u16(&ret[KEY_ICMP_FRAGMTU],
+		case ICMP_SOURCE_QUENCH:
+		case ICMP_TIME_EXCEEDED: {
+			void *nexthdr = (u_int32_t *)icmph + 2;
+			len -= sizeof(struct icmphdr);
+			_interp_iphdr(pi, nexthdr, len, 0);
+			if (icmph->type == ICMP_DEST_UNREACH &&
+			    icmph->code == ICMP_FRAG_NEEDED) {
+				okey_set_u16(&ret[recurse_key(ICMP_FRAGMTU)],
  				     ntohs(icmph->un.frag.mtu));
  		}
  		break;
  	}
-	okey_set_u16(&ret[KEY_ICMP_CSUM], icmph->checksum);
+	}
+	okey_set_u16(&ret[recurse_key(ICMP_CSUM)], icmph->checksum);

  	return ULOGD_IRET_OK;
  }
@@ -704,40 +1038,39 @@  static int _interp_ahesp(struct ulogd_pluginstance *pi, void *protoh,
   * 			IP HEADER
   ***********************************************************************/

-static int _interp_iphdr(struct ulogd_pluginstance *pi, u_int32_t len)
+static int _interp_iphdr(struct ulogd_pluginstance *pi, struct iphdr *iph, u_int32_t len,
+			 u_int32_t recurse)
  {
  	struct ulogd_key *ret = pi->output.keys;
-	struct iphdr *iph =
-		ikey_get_ptr(&pi->input.keys[INKEY_RAW_PCKT]);
  	void *nexthdr = (u_int32_t *)iph + iph->ihl;

  	if (len < sizeof(struct iphdr) || len <= (u_int32_t)(iph->ihl * 4))
  		return ULOGD_IRET_OK;
  	len -= iph->ihl * 4;

-	okey_set_u32(&ret[KEY_IP_SADDR], iph->saddr);
-	okey_set_u32(&ret[KEY_IP_DADDR], iph->daddr);
-	okey_set_u8(&ret[KEY_IP_PROTOCOL], iph->protocol);
-	okey_set_u8(&ret[KEY_IP_TOS], iph->tos);
-	okey_set_u8(&ret[KEY_IP_TTL], iph->ttl);
-	okey_set_u16(&ret[KEY_IP_TOTLEN], ntohs(iph->tot_len));
-	okey_set_u8(&ret[KEY_IP_IHL], iph->ihl);
-	okey_set_u16(&ret[KEY_IP_CSUM], ntohs(iph->check));
-	okey_set_u16(&ret[KEY_IP_ID], ntohs(iph->id));
-	okey_set_u16(&ret[KEY_IP_FRAGOFF], ntohs(iph->frag_off));
+	okey_set_u32(&ret[recurse_key(IP_SADDR)], iph->saddr);
+	okey_set_u32(&ret[recurse_key(IP_DADDR)], iph->daddr);
+	okey_set_u8(&ret[recurse_key(IP_PROTOCOL)], iph->protocol);
+	okey_set_u8(&ret[recurse_key(IP_TOS)], iph->tos);
+	okey_set_u8(&ret[recurse_key(IP_TTL)], iph->ttl);
+	okey_set_u16(&ret[recurse_key(IP_TOTLEN)], ntohs(iph->tot_len));
+	okey_set_u8(&ret[recurse_key(IP_IHL)], iph->ihl);
+	okey_set_u16(&ret[recurse_key(IP_CSUM)], ntohs(iph->check));
+	okey_set_u16(&ret[recurse_key(IP_ID)], ntohs(iph->id));
+	okey_set_u16(&ret[recurse_key(IP_FRAGOFF)], ntohs(iph->frag_off));

  	switch (iph->protocol) {
  	case IPPROTO_TCP:
-		_interp_tcp(pi, nexthdr, len);
+		_interp_tcp(pi, nexthdr, len, recurse);
  		break;
  	case IPPROTO_UDP:
-		_interp_udp(pi, nexthdr, len);
+		_interp_udp(pi, nexthdr, len, recurse);
  		break;
  	case IPPROTO_ICMP:
-		_interp_icmp(pi, nexthdr, len);
+		_interp_icmp(pi, nexthdr, len, recurse);
  		break;
  	case IPPROTO_SCTP:
-		_interp_sctp(pi, nexthdr, len);
+		_interp_sctp(pi, nexthdr, len, recurse);
  		break;
  	case IPPROTO_AH:
  	case IPPROTO_ESP:
@@ -864,10 +1197,10 @@  static int _interp_ipv6hdr(struct ulogd_pluginstance *pi, u_int32_t len)

  	switch (curhdr) {
  	case IPPROTO_TCP:
-		_interp_tcp(pi, (void *)ipv6h + ptr, len);
+		_interp_tcp(pi, (void *)ipv6h + ptr, len, 0);
  		break;
  	case IPPROTO_UDP:
-		_interp_udp(pi, (void *)ipv6h + ptr, len);
+		_interp_udp(pi, (void *)ipv6h + ptr, len, 0);
  		break;
  	case IPPROTO_ICMPV6:
  		_interp_icmpv6(pi, (void *)ipv6h + ptr, len);
@@ -914,7 +1247,8 @@  static int _interp_bridge(struct ulogd_pluginstance *pi, u_int32_t len)

  	switch (proto) {
  	case ETH_P_IP:
-		_interp_iphdr(pi, len);
+		_interp_iphdr(pi, ikey_get_ptr(&pi->input.keys[INKEY_RAW_PCKT]),
+			      len, 1);
  		break;
  	case ETH_P_IPV6:
  		_interp_ipv6hdr(pi, len);
@@ -940,7 +1274,8 @@  static int _interp_pkt(struct ulogd_pluginstance *pi)

  	switch (family) {
  	case AF_INET:
-		return _interp_iphdr(pi, len);
+		return _interp_iphdr(pi, ikey_get_ptr(&pi->input.keys[INKEY_RAW_PCKT]),
+				     len, 1);
  	case AF_INET6:
  		return _interp_ipv6hdr(pi, len);
  	case AF_BRIDGE: