diff mbox

[01/10] Add GTP hub stub, as simplistic UDP forwarder.

Message ID 1444227508-26608-2-git-send-email-nhofmeyr@sysmocom.de
State Changes Requested
Headers show

Commit Message

Neels Hofmeyr Oct. 7, 2015, 2:18 p.m. UTC
First steps towards a new GTP hub. The aim is to mux GTP connections, so that
multiple SGSN <--> GGSN links can pass through a single point. Background:
allow having more than one SGSN, possibly in various remote locations.

The recent addition of OAP to GSUP is related to the same background idea.

Sponsored-by: On-Waves ehf
---
 openbsc/include/openbsc/debug.h |   1 +
 openbsc/src/gprs/gtphub_main.c  | 296 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 297 insertions(+)
 create mode 100644 openbsc/src/gprs/gtphub_main.c

Comments

Holger Freyther Oct. 8, 2015, 8:48 a.m. UTC | #1
> On 07 Oct 2015, at 16:18, Neels Hofmeyr <nhofmeyr@sysmocom.de> wrote:
> 
> 
> +/* TODO move to osmocom/core/socket.c ? */

where did you take this from?



> +/* TODO move to osmocom/core/socket.c ? */


> +int osmo_sockaddr_init(struct sockaddr_storage *addr, socklen_t *addr_len,
> +		       uint16_t family, uint16_t type, uint8_t proto,
> +		       const char *host, uint16_t port)
> +{

ah yeah. the by hostname is a good idea. But this should only be used at the
beginning of an application as we don't want to block on a DNS query.

> 
> +static struct log_info_cat gtphub_categories[] = {

> +static const struct log_info gtphub_log_info = {
> +	.filter_fn = gtphub_log_filter_fn,
> +	.cat = gtphub_categories,
> +	.num_cat = ARRAY_SIZE(gtphub_categories),
> +};

nothing else will be used as debug log? Depending on the answer it might be
better to just always initialize the GTPHUB area..



> 
> +int gtp_relay(struct osmo_fd *from, struct sockaddr_storage *from_addr, socklen_t *from_addr_len,
> +	      struct osmo_fd *to, struct sockaddr_storage *to_addr, socklen_t to_addr_len)
> +{
> +	static uint8_t buf[4096];
> +
> +	errno = 0;
> +	ssize_t received = recvfrom(from->fd, buf, sizeof(buf), 0,
> +				    (struct sockaddr*)from_addr, from_addr_len);
> +

When the first gbproxy was built our buffer was smaller and we truncated messages. Now 4k
is pretty big and bigger than our expectation for a UDP message. If you use recvmsg then the
msg_flags will have MSG_TRUNC set or not. Not sure if you want to use it.



> +	rc = osmo_sock_init_ofd(&clients_ofd, AF_UNSPEC, SOCK_DGRAM, IPPROTO_UDP, clients_addr_str, clients_port, OSMO_SOCK_F_BIND);
> +	rc = osmo_sock_init_ofd(&server_ofd, AF_UNSPEC, SOCK_DGRAM, IPPROTO_UDP, server_rx_addr_str, server_rx_port, OSMO_SOCK_F_BIND);


Who is the server? who is the client? Maybe give it names to what we expect on the side?
One is the SGSN, the other is the GGSN?
Neels Hofmeyr Oct. 8, 2015, 11:24 a.m. UTC | #2
On Thu, Oct 08, 2015 at 10:48:07AM +0200, Holger Freyther wrote:
> 
> > On 07 Oct 2015, at 16:18, Neels Hofmeyr <nhofmeyr@sysmocom.de> wrote:
> > 
> > 
> > +/* TODO move to osmocom/core/socket.c ? */
> 
> where did you take this from?

basically the beginning of osmo_sock_init()

> > +/* TODO move to osmocom/core/socket.c ? */
> 
> 
> > +int osmo_sockaddr_init(struct sockaddr_storage *addr, socklen_t *addr_len,
> > +		       uint16_t family, uint16_t type, uint8_t proto,
> > +		       const char *host, uint16_t port)
> > +{
> 
> ah yeah. the by hostname is a good idea. But this should only be used at the
> beginning of an application as we don't want to block on a DNS query.

In fact, I use this function to hardcode the GTP server. We will in future
fetch the GTP server by DNS, and I'm not entirely clear on whether
getaddrinfo() can resolve that. If we need a full res_query() instead, the
gtphub will no longer need this function, and its use is reduced to a
courtesy to future API users...

> > +static struct log_info_cat gtphub_categories[] = {
> 
> > +static const struct log_info gtphub_log_info = {
> > +	.filter_fn = gtphub_log_filter_fn,
> > +	.cat = gtphub_categories,
> > +	.num_cat = ARRAY_SIZE(gtphub_categories),
> > +};
> 
> nothing else will be used as debug log? Depending on the answer it might be
> better to just always initialize the GTPHUB area..

I don't fully see thru the debug setup yet. Other binaries around seem to
re-create very similar category lists, apparently just listing all that is
defined. I tried to find the minimum first. Others may be added as needed.

What do you mean by
> better to just always initialize the GTPHUB area..
? IIUC that's what the patch is doing right now?

> If you use recvmsg then the msg_flags will have MSG_TRUNC set or not.
> Not sure if you want to use it.

I'll put a TODO comment there. The 4kb buf should suffice for now. Thanks
for the hint! Wasn't aware.

> > +	rc = osmo_sock_init_ofd(&clients_ofd, AF_UNSPEC, SOCK_DGRAM, IPPROTO_UDP, clients_addr_str, clients_port, OSMO_SOCK_F_BIND);
> > +	rc = osmo_sock_init_ofd(&server_ofd, AF_UNSPEC, SOCK_DGRAM, IPPROTO_UDP, server_rx_addr_str, server_rx_port, OSMO_SOCK_F_BIND);
> 
> 
> Who is the server? who is the client? Maybe give it names to what we expect on the side?
> One is the SGSN, the other is the GGSN?

I thought of a GTP server == GGSN and GTP client == SGSN. If that's
ambiguous at all I'll be happy to change it, e.g. to to_sgsns, to_ggsns.

  [SGSN] <---> [to_sgsns  GTPHUB  to_ggsns] <---> [GGSN]

With the "to_" prefix to avoid this ambiguity: if the SGSN contacts the
gtphub, the gtphub looks like a GGSN, so a confused reader could think of
the "to_sgsns" port as a GGSN. Or rather a "from_" prefix? :P

~Neels
Harald Welte Oct. 8, 2015, 3 p.m. UTC | #3
Hi Neels,

On Wed, Oct 07, 2015 at 04:18:19PM +0200, Neels Hofmeyr wrote:
> +#define LOGERR(fmt, args...) \
> +	LOGP(DGTPHUB, LOGL_ERROR, fmt, ##args)
> +
> +#define LOG(fmt, args...) \
> +	LOGP(DGTPHUB, LOGL_NOTICE, fmt, ##args)

we typically don't do this in the Osmo* codebase,bt use the full
statement.  However, it is your choice.

The problem I see with this is that you reduce/remove the flexibility of
using multiplelog subsstems and levels.  You more or less assume that
there will always only be two log levels used, and all messages will use
the same subsystem.  Is that really the case?

When the full statement is used everytime, it is also easier to later
change / tune the logging levels of certain statements, which is
typically what happens after you run it in production with thousands of
messages flowing through and you see what kind of granularity makes
sense.

Also, I think in patch 10/10 you state 'add debug messages' but then
use the LOG() macro.  So in fact you are adding NOTICE messages, not
DEBUG messages.  So either of the two statements is wrong.
Neels Hofmeyr Oct. 9, 2015, 12:01 a.m. UTC | #4
On Thu, Oct 08, 2015 at 05:00:08PM +0200, Harald Welte wrote:
> Hi Neels,
> 
> On Wed, Oct 07, 2015 at 04:18:19PM +0200, Neels Hofmeyr wrote:
> > +#define LOGERR(fmt, args...) \
> > +	LOGP(DGTPHUB, LOGL_ERROR, fmt, ##args)
> > +
> > +#define LOG(fmt, args...) \
> > +	LOGP(DGTPHUB, LOGL_NOTICE, fmt, ##args)
> 
> we typically don't do this in the Osmo* codebase,bt use the full
> statement.  However, it is your choice.
> 
> The problem I see with this is that you reduce/remove the flexibility of
> using multiplelog subsstems and levels.  You more or less assume that
> there will always only be two log levels used, and all messages will use
> the same subsystem.  Is that really the case?

It's intended as a per-C-file local definition just to reduce repetition.
If a file needs another log level (more than once), I intend to create
another LOGxxx #define; but never in an h. LOGP() is also available.

> When the full statement is used everytime, it is also easier to later
> change / tune the logging levels of certain statements, which is
> typically what happens after you run it in production with thousands of
> messages flowing through and you see what kind of granularity makes
> sense.

ok, I see. I'll drop them as I go, then.

> Also, I think in patch 10/10 you state 'add debug messages' but then
> use the LOG() macro.  So in fact you are adding NOTICE messages, not
> DEBUG messages.  So either of the two statements is wrong.

That's right. Most current LOG()s are actually not intended to live past
the commit that concludes GTP transcoding, fairly soon. That's what I
meant by debug, really.

Thanks!

~Neels
diff mbox

Patch

diff --git a/openbsc/include/openbsc/debug.h b/openbsc/include/openbsc/debug.h
index 19d8fc2..189ca47 100644
--- a/openbsc/include/openbsc/debug.h
+++ b/openbsc/include/openbsc/debug.h
@@ -33,6 +33,7 @@  enum {
 	DCTRL,
 	DSMPP,
 	DFILTER,
+	DGTPHUB,
 	Debug_LastEntry,
 };
 
diff --git a/openbsc/src/gprs/gtphub_main.c b/openbsc/src/gprs/gtphub_main.c
new file mode 100644
index 0000000..aa35952
--- /dev/null
+++ b/openbsc/src/gprs/gtphub_main.c
@@ -0,0 +1,296 @@ 
+/* GTP Hub main program */
+
+/* (C) 2015 by sysmocom s.f.m.c. GmbH <info@sysmocom.de>
+ * All Rights Reserved
+ *
+ * Author: Neels Hofmeyr
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as published by
+ * the Free Software Foundation; either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <string.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <errno.h>
+
+#include <sys/socket.h>
+#include <netinet/in.h>
+
+#define _GNU_SOURCE
+#include <getopt.h>
+
+#include <osmocom/core/application.h>
+#include <osmocom/core/utils.h>
+#include <osmocom/core/socket.h>
+#include <osmocom/core/select.h>
+#include <osmocom/core/logging.h>
+
+#include <openbsc/debug.h>
+
+#include <gtp.h>
+
+#include <unistd.h>
+
+#define LOGERR(fmt, args...) \
+	LOGP(DGTPHUB, LOGL_ERROR, fmt, ##args)
+
+#define LOG(fmt, args...) \
+	LOGP(DGTPHUB, LOGL_NOTICE, fmt, ##args)
+
+extern void *osmo_gtphub_ctx;
+
+/* TODO move to osmocom/core/socket.c ? */
+#include <netdb.h>
+/* The caller is required to call freeaddrinfo(*result), iff zero is returned. */
+/* use this in osmo_sock_init() to remove dup. */
+static int _osmo_getaddrinfo(struct addrinfo **result,
+			     uint16_t family, uint16_t type, uint8_t proto,
+			     const char *host, uint16_t port)
+{
+	struct addrinfo hints;
+	char portbuf[16];
+
+	sprintf(portbuf, "%u", port);
+	memset(&hints, '\0', sizeof(struct addrinfo));
+	hints.ai_family = family;
+	if (type == SOCK_RAW) {
+		/* Workaround for glibc, that returns EAI_SERVICE (-8) if
+		 * SOCK_RAW and IPPROTO_GRE is used.
+		 */
+		hints.ai_socktype = SOCK_DGRAM;
+		hints.ai_protocol = IPPROTO_UDP;
+	} else {
+		hints.ai_socktype = type;
+		hints.ai_protocol = proto;
+	}
+
+	return getaddrinfo(host, portbuf, &hints, result);
+}
+
+/* TODO move to osmocom/core/socket.c ? */
+/*! \brief Initialize a sockaddr \param[out] addr valid sockaddr pointer to
+ * write result to \param[out] addr_len valid pointer to write addr length to
+ * \param[in] family Address Family like AF_INET, AF_INET6, AF_UNSPEC
+ * \param[in] type Socket type like SOCK_DGRAM, SOCK_STREAM \param[in] proto
+ * Protocol like IPPROTO_TCP, IPPROTO_UDP \param[in] host remote host name or
+ * IP address in string form \param[in] port remote port number in host byte
+ * order \returns 0 on success, otherwise an error code (from getaddrinfo()).
+ *
+ * Copy the first result from a getaddrinfo() call with the given parameters to
+ * *addr and *addr_len. On error, do not change *addr and return nonzero.
+ */
+int osmo_sockaddr_init(struct sockaddr_storage *addr, socklen_t *addr_len,
+		       uint16_t family, uint16_t type, uint8_t proto,
+		       const char *host, uint16_t port)
+{
+	struct addrinfo *res;
+	int rc;
+	rc = _osmo_getaddrinfo(&res, family, type, proto, host, port);
+
+	if (rc != 0) {
+		LOGERR("getaddrinfo returned error %d\n", (int)rc);
+		return -EINVAL;
+	}
+
+	OSMO_ASSERT(res->ai_addrlen <= sizeof(*addr));
+	memcpy(addr, res->ai_addr, res->ai_addrlen);
+	*addr_len = res->ai_addrlen;
+	freeaddrinfo(res);
+
+	return 0;
+}
+
+
+void *tall_bsc_ctx;
+
+const char *gtphub_copyright =
+	"Copyright (C) 2015 sysmocom s.m.f.c GmbH <info@sysmocom.de>\r\n"
+	"License AGPLv3+: GNU AGPL version 2 or later <http://gnu.org/licenses/agpl-3.0.html>\r\n"
+	"This is free software: you are free to change and redistribute it.\r\n"
+	"There is NO WARRANTY, to the extent permitted by law.\r\n";
+
+static struct log_info_cat gtphub_categories[] = {
+	[DGTPHUB] = {
+		.name = "DGTPHUB",
+		.description = "GTP Hub",
+		.color = "\033[1;33m",
+		.enabled = 1, .loglevel = LOGL_NOTICE,
+	},
+};
+
+int gtphub_log_filter_fn(const struct log_context *ctx,
+			 struct log_target *tar)
+{
+	return 0;
+}
+
+static const struct log_info gtphub_log_info = {
+	.filter_fn = gtphub_log_filter_fn,
+	.cat = gtphub_categories,
+	.num_cat = ARRAY_SIZE(gtphub_categories),
+};
+
+/* Recv datagram from from->fd, optionally write sender's address to *from_addr
+ * and *from_addr_len, parse datagram as GTP, and forward on to to->fd using
+ * *to_addr. to_addr may be NULL, if an address is set on to->fd. */
+int gtp_relay(struct osmo_fd *from, struct sockaddr_storage *from_addr, socklen_t *from_addr_len,
+	      struct osmo_fd *to, struct sockaddr_storage *to_addr, socklen_t to_addr_len)
+{
+	static uint8_t buf[4096];
+
+	errno = 0;
+	ssize_t received = recvfrom(from->fd, buf, sizeof(buf), 0,
+				    (struct sockaddr*)from_addr, from_addr_len);
+
+	if (received <= 0) {
+		if (errno != EAGAIN)
+			LOGERR("error: %s\n", strerror(errno));
+		return -errno;
+	}
+
+	if (from_addr) {
+		LOG("from %s\n", osmo_hexdump((uint8_t*)from_addr, *from_addr_len));
+	}
+
+	if (received <= 0) {
+		LOGERR("error: %s\n", strerror(errno));
+		return -EINVAL;
+	}
+
+	/* insert magic here */
+
+	errno = 0;
+	ssize_t sent = sendto(to->fd, buf, received, 0,
+			      (struct sockaddr*)to_addr, to_addr_len);
+
+	if (to_addr) {
+		LOG("to %s\n", osmo_hexdump((uint8_t*)to_addr, to_addr_len));
+	}
+
+	if (sent == -1) {
+		LOGERR("error: %s\n", strerror(errno));
+		return -EINVAL;
+	}
+
+	if (sent != received)
+		LOGERR("sent(%d) != received(%d)\n", (int)sent, (int)received);
+	else
+		LOG("%d b ok\n", (int)sent);
+
+	return 0;
+}
+
+struct sockaddr_storage last_client_addr;
+socklen_t last_client_addr_len;
+struct sockaddr_storage server_addr;
+socklen_t server_addr_len;
+
+int clients_read_cb(struct osmo_fd *clients_ofd, unsigned int what)
+{
+	LOG("reading from clients socket\n");
+	struct osmo_fd *server_ofd = clients_ofd->data;
+
+	if (!(what & BSC_FD_READ))
+		return 0;
+
+	last_client_addr_len = sizeof(last_client_addr);
+	return gtp_relay(clients_ofd, &last_client_addr, &last_client_addr_len,
+			 server_ofd, &server_addr, server_addr_len);
+}
+
+int server_read_cb(struct osmo_fd *server_ofd, unsigned int what)
+{
+	LOG("reading from server socket\n");
+	struct osmo_fd *clients_ofd = server_ofd->data;
+
+	if (!(what & BSC_FD_READ))
+		return 0;
+
+	return gtp_relay(server_ofd, NULL, NULL,
+			 clients_ofd, &last_client_addr, last_client_addr_len);
+}
+
+int main(int argc, char **argv)
+{
+	osmo_init_logging(&gtphub_log_info);
+
+	int rc;
+
+	const char* clients_addr_str = "localhost";
+	uint16_t clients_port = 3386;
+
+	const char* server_addr_str = "localhost";
+	uint16_t server_port = 1234;
+
+	/* Which local interface to use to listen for the GTP server's
+	 * responses */
+	const char* server_rx_addr_str = "localhost";
+	uint16_t server_rx_port = 4321;
+
+	rc = osmo_sockaddr_init(&server_addr, &server_addr_len,
+				AF_UNSPEC, SOCK_DGRAM, IPPROTO_UDP, server_addr_str, server_port);
+	if (rc != 0) {
+		LOGERR("Cannot resolve '%s port %d'\n", server_addr_str, server_port);
+		exit(-1);
+	}
+
+	struct osmo_fd clients_ofd;
+	struct osmo_fd server_ofd;
+
+	memset(&clients_ofd, 0, sizeof(clients_ofd));
+	clients_ofd.when = BSC_FD_READ;
+	clients_ofd.cb = clients_read_cb;
+	clients_ofd.data = &server_ofd;
+
+	rc = osmo_sock_init_ofd(&clients_ofd, AF_UNSPEC, SOCK_DGRAM, IPPROTO_UDP, clients_addr_str, clients_port, OSMO_SOCK_F_BIND);
+	if (rc < 1) {
+		LOGERR("Cannot bind to %s port %d\n", clients_addr_str, clients_port);
+		exit(-1);
+	}
+
+	memset(&server_ofd, 0, sizeof(server_ofd));
+	server_ofd.when = BSC_FD_READ;
+	server_ofd.cb = server_read_cb;
+	server_ofd.data = &clients_ofd;
+
+	rc = osmo_sock_init_ofd(&server_ofd, AF_UNSPEC, SOCK_DGRAM, IPPROTO_UDP, server_rx_addr_str, server_rx_port, OSMO_SOCK_F_BIND);
+	if (rc < 1) {
+		LOGERR("Cannot bind to %s port %d\n", server_rx_addr_str, server_rx_port);
+		exit(-1);
+	}
+
+	LOG("GTP server connection: %s port %d <--> %s port %d\n",
+	    server_rx_addr_str, (int)server_rx_port,
+	    server_addr_str, (int)server_port);
+	LOG("Listening for clients on %s port %d.\n", clients_addr_str, clients_port);
+
+	int daemonize = 0;
+
+	if (daemonize) {
+		rc = osmo_daemonize();
+		if (rc < 0) {
+			LOGERR("Error during daemonize");
+			exit(1);
+		}
+	}
+
+	while (1) {
+		rc = osmo_select_main(0);
+		if (rc < 0)
+			exit(3);
+	}
+
+	/* not reached */
+	exit(0);
+}