Patchwork [3/6] qtest: add C version of test infrastructure

login
register
mail settings
Submitter Anthony Liguori
Date Jan. 13, 2012, 6:32 p.m.
Message ID <1326479558-3016-3-git-send-email-aliguori@us.ibm.com>
Download mbox | patch
Permalink /patch/135958/
State New
Headers show

Comments

Anthony Liguori - Jan. 13, 2012, 6:32 p.m.
This also includes a qtest wrapper script to make it easier to launch qtest
tests directly.

Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
---
 scripts/qtest    |    5 +
 tests/Makefile   |    2 +
 tests/libqtest.c |  334 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/libqtest.h |   63 ++++++++++
 4 files changed, 404 insertions(+), 0 deletions(-)
 create mode 100755 scripts/qtest
 create mode 100644 tests/libqtest.c
 create mode 100644 tests/libqtest.h
Stefan Hajnoczi - Jan. 17, 2012, 11:33 a.m.
On Fri, Jan 13, 2012 at 6:32 PM, Anthony Liguori <aliguori@us.ibm.com> wrote:
> +    pid = fork();
> +    if (pid == 0) {
> +        command = g_strdup_printf("%s "
> +                                  "-qtest unix:%s,server,nowait "
> +                                  "-qtest-log /dev/null "
> +                                  "-pidfile %s "
> +                                  "-machine accel=qtest "
> +                                  "%s", qemu_binary, socket_path,
> +                                  pid_file,
> +                                  extra_args ?: "");
> +
> +        ret = system(command);

qtest_init() launches qemu with a pidfile so that we can send SIGTERM
later.  But we never get around to doing that if g_assert() fails - it
calls abort(3).  The result is a run-away qemu process.  I find the
qemu process consumes 100% in send_all() trying to write to the closed
qtest socket and SIGTERM no longer works since we're stuck in a tight
loop that never runs the event loop.

A simple solution is to handle SIGABRT in tests/libqtest.c and sent
SIGTERM to qemu when the test aborts.  The downside is that this only
covers the abort(3) case - a segfault or other abnormal termination
would still leave the run-away qemu process.

I was wondering about a qemu-side solution where a closed qtest socket
means we need to shut down, but am not sure if the chardev code lets
us do that.  (Really we want POLLHUP but we only seem to have
POLLIN/POLLOUT handlers.)

Thoughts?

Stefan
Paolo Bonzini - Jan. 17, 2012, 1:33 p.m.
On 01/17/2012 12:33 PM, Stefan Hajnoczi wrote:
> I was wondering about a qemu-side solution where a closed qtest socket
> means we need to shut down, but am not sure if the chardev code lets
> us do that.  (Really we want POLLHUP but we only seem to have
> POLLIN/POLLOUT handlers.)

For poll, both POLLIN and POLLOUT are always reported together with 
POLLHUP.  I think the same happens with select().  If you get a 
zero-read in the qtest chardev handler you can shut down.

Paolo
Stefan Hajnoczi - Jan. 17, 2012, 1:39 p.m.
On Tue, Jan 17, 2012 at 1:33 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
> On 01/17/2012 12:33 PM, Stefan Hajnoczi wrote:
>>
>> I was wondering about a qemu-side solution where a closed qtest socket
>> means we need to shut down, but am not sure if the chardev code lets
>> us do that.  (Really we want POLLHUP but we only seem to have
>> POLLIN/POLLOUT handlers.)
>
>
> For poll, both POLLIN and POLLOUT are always reported together with POLLHUP.
>  I think the same happens with select().  If you get a zero-read in the
> qtest chardev handler you can shut down.

There is already open/closed logic in qemu-char.c that acts on
select(2) becoming readable.  However, it isn't kicking in - we're
still ending up in send_all(), which should only be called when we
thing the socket is connected.  I'll investigate some more.

Stefan
Paolo Bonzini - Jan. 17, 2012, 4:09 p.m.
On 01/13/2012 07:32 PM, Anthony Liguori wrote:
> This also includes a qtest wrapper script to make it easier to launch qtest
> tests directly.
>
> Signed-off-by: Anthony Liguori<aliguori@us.ibm.com>

Here is a Python test harness for qtest.  I haven't tried merging them 
with the makefiles.

Feel free to add my s-o-b and include the files in your patches.

Paolo
Kevin Wolf - Jan. 18, 2012, 4 p.m.
Am 13.01.2012 19:32, schrieb Anthony Liguori:
> This also includes a qtest wrapper script to make it easier to launch qtest
> tests directly.
> 
> Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>

> +QTestState *qtest_init(const char *extra_args)
> +{
> +    QTestState *s;
> +    struct sockaddr_un addr;
> +    int sock, ret, i;
> +    gchar *socket_path;
> +    gchar *pid_file;
> +    gchar *command;
> +    const char *qemu_binary;
> +    pid_t pid;
> +
> +    qemu_binary = getenv("QTEST_QEMU_BINARY");
> +    g_assert(qemu_binary != NULL);
> +
> +    socket_path = g_strdup_printf("/tmp/qtest-%d.sock", getpid());
> +    pid_file = g_strdup_printf("/tmp/qtest-%d.pid", getpid());
> +
> +    s = g_malloc(sizeof(*s));
> +
> +    sock = socket(PF_UNIX, SOCK_STREAM, 0);
> +    g_assert_no_errno(sock);
> +
> +    addr.sun_family = AF_UNIX;
> +    snprintf(addr.sun_path, sizeof(addr.sun_path), "%s", socket_path);
> +
> +    pid = fork();
> +    if (pid == 0) {
> +        command = g_strdup_printf("%s "
> +                                  "-qtest unix:%s,server,nowait "
> +                                  "-qtest-log /dev/null "
> +                                  "-pidfile %s "
> +                                  "-machine accel=qtest "
> +                                  "%s", qemu_binary, socket_path,
> +                                  pid_file,
> +                                  extra_args ?: "");
> +
> +        ret = system(command);
> +        exit(ret);
> +        g_free(command);
> +    }
> +
> +    do {
> +        sleep(1);

This is the line that takes the greatest part of the time for make
check-qtest. Can we use some shorter delay if it's required at all?

> +        ret = connect(sock, (struct sockaddr *)&addr, sizeof(addr));
> +    } while (ret == -1);
> +    g_assert_no_errno(ret);
> +
> +    s->fd = sock;
> +    s->rx = g_string_new("");
> +    s->pid_file = pid_file;
> +    for (i = 0; i < MAX_IRQ; i++) {
> +        s->irq_level[i] = false;
> +    }
> +
> +    g_free(socket_path);
> +
> +    return s;
> +}

Kevin
Paolo Bonzini - Jan. 18, 2012, 4:02 p.m.
On 01/18/2012 05:00 PM, Kevin Wolf wrote:
>> >  +    do {
>> >  +        sleep(1);
> This is the line that takes the greatest part of the time for make
> check-qtest. Can we use some shorter delay if it's required at all?

You can use a client socket, listen before spawning QEMU and accept 
afterwards.  It's what I did in the Python version.

Paolo
Anthony Liguori - Jan. 18, 2012, 4:08 p.m.
On 01/18/2012 10:00 AM, Kevin Wolf wrote:
> Am 13.01.2012 19:32, schrieb Anthony Liguori:
>> This also includes a qtest wrapper script to make it easier to launch qtest
>> tests directly.
>>
>> Signed-off-by: Anthony Liguori<aliguori@us.ibm.com>
>
>> +QTestState *qtest_init(const char *extra_args)
>> +{
>> +    QTestState *s;
>> +    struct sockaddr_un addr;
>> +    int sock, ret, i;
>> +    gchar *socket_path;
>> +    gchar *pid_file;
>> +    gchar *command;
>> +    const char *qemu_binary;
>> +    pid_t pid;
>> +
>> +    qemu_binary = getenv("QTEST_QEMU_BINARY");
>> +    g_assert(qemu_binary != NULL);
>> +
>> +    socket_path = g_strdup_printf("/tmp/qtest-%d.sock", getpid());
>> +    pid_file = g_strdup_printf("/tmp/qtest-%d.pid", getpid());
>> +
>> +    s = g_malloc(sizeof(*s));
>> +
>> +    sock = socket(PF_UNIX, SOCK_STREAM, 0);
>> +    g_assert_no_errno(sock);
>> +
>> +    addr.sun_family = AF_UNIX;
>> +    snprintf(addr.sun_path, sizeof(addr.sun_path), "%s", socket_path);
>> +
>> +    pid = fork();
>> +    if (pid == 0) {
>> +        command = g_strdup_printf("%s "
>> +                                  "-qtest unix:%s,server,nowait "
>> +                                  "-qtest-log /dev/null "
>> +                                  "-pidfile %s "
>> +                                  "-machine accel=qtest "
>> +                                  "%s", qemu_binary, socket_path,
>> +                                  pid_file,
>> +                                  extra_args ?: "");
>> +
>> +        ret = system(command);
>> +        exit(ret);
>> +        g_free(command);
>> +    }
>> +
>> +    do {
>> +        sleep(1);
>
> This is the line that takes the greatest part of the time for make
> check-qtest. Can we use some shorter delay if it's required at all?

Ah, good catch.  It should change to a usleep().

It is needed, you can't guarantee that qemu is listening yet on the sockets when 
you try to connect.

Regards,

Anthony Liguori

>
>> +        ret = connect(sock, (struct sockaddr *)&addr, sizeof(addr));
>> +    } while (ret == -1);
>> +    g_assert_no_errno(ret);
>> +
>> +    s->fd = sock;
>> +    s->rx = g_string_new("");
>> +    s->pid_file = pid_file;
>> +    for (i = 0; i<  MAX_IRQ; i++) {
>> +        s->irq_level[i] = false;
>> +    }
>> +
>> +    g_free(socket_path);
>> +
>> +    return s;
>> +}
>
> Kevin
>

Patch

diff --git a/scripts/qtest b/scripts/qtest
new file mode 100755
index 0000000..5cff3d4
--- /dev/null
+++ b/scripts/qtest
@@ -0,0 +1,5 @@ 
+#!/bin/sh
+
+export QTEST_QEMU_BINARY=$1
+shift
+eval "$@"
diff --git a/tests/Makefile b/tests/Makefile
index efde63a..92d462a 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -34,6 +34,8 @@  test-qmp-input-visitor: test-qmp-input-visitor.o $(qobject-obj-y) $(qapi-obj-y)
 test-qmp-commands.o: $(addprefix $(qapi-dir)/, test-qapi-types.c test-qapi-types.h test-qapi-visit.c test-qapi-visit.h test-qmp-marshal.c test-qmp-commands.h) $(qapi-obj-y)
 test-qmp-commands: test-qmp-commands.o $(qobject-obj-y) $(qapi-obj-y) $(tools-obj-y) $(qapi-dir)/test-qapi-visit.o $(qapi-dir)/test-qapi-types.o $(qapi-dir)/test-qmp-marshal.o module.o
 
+tests/rtc-test: tests/rtc-test.o tests/libqtest.o
+
 .PHONY: check
 check: $(CHECKS)
 	gtester $(CHECKS)
diff --git a/tests/libqtest.c b/tests/libqtest.c
new file mode 100644
index 0000000..dd07b07
--- /dev/null
+++ b/tests/libqtest.c
@@ -0,0 +1,334 @@ 
+/*
+ * QTest
+ *
+ * Copyright IBM, Corp. 2012
+ *
+ * Authors:
+ *  Anthony Liguori   <aliguori@us.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+#include "libqtest.h"
+
+#include <glib.h>
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <sys/wait.h>
+#include <sys/un.h>
+#include <errno.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <string.h>
+
+#define MAX_IRQ 256
+
+QTestState *global_qtest;
+
+struct QTestState
+{
+    int fd;
+    bool irq_level[MAX_IRQ];
+    GString *rx;
+    gchar *pid_file;
+};
+
+#define g_assert_no_errno(ret) do { \
+    g_assert_cmpint(ret, !=, -1); \
+} while (0)
+
+QTestState *qtest_init(const char *extra_args)
+{
+    QTestState *s;
+    struct sockaddr_un addr;
+    int sock, ret, i;
+    gchar *socket_path;
+    gchar *pid_file;
+    gchar *command;
+    const char *qemu_binary;
+    pid_t pid;
+
+    qemu_binary = getenv("QTEST_QEMU_BINARY");
+    g_assert(qemu_binary != NULL);
+
+    socket_path = g_strdup_printf("/tmp/qtest-%d.sock", getpid());
+    pid_file = g_strdup_printf("/tmp/qtest-%d.pid", getpid());
+
+    s = g_malloc(sizeof(*s));
+
+    sock = socket(PF_UNIX, SOCK_STREAM, 0);
+    g_assert_no_errno(sock);
+
+    addr.sun_family = AF_UNIX;
+    snprintf(addr.sun_path, sizeof(addr.sun_path), "%s", socket_path);
+
+    pid = fork();
+    if (pid == 0) {
+        command = g_strdup_printf("%s "
+                                  "-qtest unix:%s,server,nowait "
+                                  "-qtest-log /dev/null "
+                                  "-pidfile %s "
+                                  "-machine accel=qtest "
+                                  "%s", qemu_binary, socket_path,
+                                  pid_file,
+                                  extra_args ?: "");
+
+        ret = system(command);
+        exit(ret);
+        g_free(command);
+    }
+
+    do {
+        sleep(1);
+        ret = connect(sock, (struct sockaddr *)&addr, sizeof(addr));
+    } while (ret == -1);
+    g_assert_no_errno(ret);
+
+    s->fd = sock;
+    s->rx = g_string_new("");
+    s->pid_file = pid_file;
+    for (i = 0; i < MAX_IRQ; i++) {
+        s->irq_level[i] = false;
+    }
+
+    g_free(socket_path);
+
+    return s;
+}
+
+void qtest_quit(QTestState *s)
+{
+    FILE *f;
+    char buffer[1024];
+
+    f = fopen(s->pid_file, "r");
+    if (f) {
+        if (fgets(buffer, sizeof(buffer), f)) {
+            pid_t pid = atoi(buffer);
+            int status = 0;
+
+            kill(pid, SIGTERM);
+            waitpid(pid, &status, 0);
+        }
+
+        fclose(f);
+    }
+}
+
+static void qtest_sendf(QTestState *s, const char *fmt, ...)
+{
+    va_list ap;
+    gchar *str;
+    size_t size, offset;
+
+    va_start(ap, fmt);
+    str = g_strdup_vprintf(fmt, ap);
+    va_end(ap);
+    size = strlen(str);
+
+    offset = 0;
+    while (offset < size) {
+        ssize_t len;
+
+        len = write(s->fd, str + offset, size - offset);
+        if (len == -1 && errno == EINTR) {
+            continue;
+        }
+
+        g_assert_no_errno(len);
+        g_assert_cmpint(len, >, 0);
+
+        offset += len;
+    }
+}
+
+static GString *qtest_recv_line(QTestState *s)
+{
+    GString *line;
+    size_t offset;
+    char *eol;
+
+    while ((eol = strchr(s->rx->str, '\n')) == NULL) {
+        ssize_t len;
+        char buffer[1024];
+
+        len = read(s->fd, buffer, sizeof(buffer));
+        if (len == -1 && errno == EINTR) {
+            continue;
+        }
+
+        if (len == -1 || len == 0) {
+            fprintf(stderr, "Broken pipe\n");
+            exit(1);
+        }
+
+        g_string_append_len(s->rx, buffer, len);
+    }
+
+    offset = eol - s->rx->str;
+    line = g_string_new_len(s->rx->str, offset);
+    g_string_erase(s->rx, 0, offset + 1);
+
+    return line;
+}
+
+static gchar **qtest_rsp(QTestState *s, int expected_args)
+{
+    GString *line;
+    gchar **words;
+    int i;
+
+redo:
+    line = qtest_recv_line(s);
+    words = g_strsplit(line->str, " ", 0);
+    g_string_free(line, TRUE);
+
+    if (strcmp(words[0], "IRQ") == 0) {
+        int irq;
+
+        g_assert(words[1] != NULL);
+        g_assert(words[2] != NULL);
+
+        irq = strtoul(words[2], NULL, 0);
+        g_assert_cmpint(irq, >=, 0);
+        g_assert_cmpint(irq, <, MAX_IRQ);
+
+        if (strcmp(words[1], "raise") == 0) {
+            s->irq_level[irq] = true;
+        } else {
+            s->irq_level[irq] = false;
+        }
+
+        g_strfreev(words);
+        goto redo;
+    }
+
+    g_assert(words[0] != NULL);
+    g_assert_cmpstr(words[0], ==, "OK");
+
+    if (expected_args) {
+        for (i = 0; i < expected_args; i++) {
+            g_assert(words[i] != NULL);
+        }
+    } else {
+        g_strfreev(words);
+    }
+    
+    return words;
+}
+
+const char *qtest_get_arch(void)
+{
+    const char *qemu = getenv("QTEST_QEMU_BINARY");
+    const char *end = strrchr(qemu, '/');
+
+    return end + strlen("/qemu-system-");
+}
+
+bool qtest_get_irq(QTestState *s, int num)
+{
+    /* dummy operation in order to make sure irq is up to date */
+    qtest_inb(s, 0);
+
+    return s->irq_level[num];
+}
+
+static void qtest_out(QTestState *s, const char *cmd, uint16_t addr, uint32_t value)
+{
+    qtest_sendf(s, "%s 0x%x 0x%x\n", cmd, addr, value);
+    qtest_rsp(s, 0);
+}
+
+void qtest_outb(QTestState *s, uint16_t addr, uint8_t value)
+{
+    qtest_out(s, "outb", addr, value);
+}
+
+void qtest_outw(QTestState *s, uint16_t addr, uint16_t value)
+{
+    qtest_out(s, "outw", addr, value);
+}
+
+void qtest_outl(QTestState *s, uint16_t addr, uint32_t value)
+{
+    qtest_out(s, "outl", addr, value);
+}
+
+static uint32_t qtest_in(QTestState *s, const char *cmd, uint16_t addr)
+{
+    gchar **args;
+    uint32_t value;
+
+    qtest_sendf(s, "%s 0x%x\n", cmd, addr);
+    args = qtest_rsp(s, 2);
+    value = strtoul(args[1], NULL, 0);
+    g_strfreev(args);
+
+    return value;
+}
+
+uint8_t qtest_inb(QTestState *s, uint16_t addr)
+{
+    return qtest_in(s, "inb", addr);
+}
+
+uint16_t qtest_inw(QTestState *s, uint16_t addr)
+{
+    return qtest_in(s, "inw", addr);
+}
+
+uint32_t qtest_inl(QTestState *s, uint16_t addr)
+{
+    return qtest_in(s, "inl", addr);
+}
+
+static int hex2nib(char ch)
+{
+    if (ch >= '0' && ch <= '9') {
+        return ch - '0';
+    } else if (ch >= 'a' && ch <= 'f') {
+        return 10 + (ch - 'a');
+    } else if (ch >= 'A' && ch <= 'F') {
+        return 10 + (ch - 'a');
+    } else {
+        return -1;
+    }
+}
+
+void qtest_memread(QTestState *s, uint64_t addr, void *data, size_t size)
+{
+    uint8_t *ptr = data;
+    gchar **args;
+    size_t i;
+
+    qtest_sendf(s, "read 0x%x 0x%x\n", addr, size);
+    args = qtest_rsp(s, 2);
+
+    for (i = 0; i < size; i++) {
+        ptr[i] = hex2nib(args[1][2 + (i * 2)]) << 4;
+        ptr[i] |= hex2nib(args[1][2 + (i * 2) + 1]);
+    }
+
+    g_strfreev(args);
+}
+
+void qtest_add_func(const char *str, void (*fn))
+{
+    gchar *path = g_strdup_printf("/%s/%s", qtest_get_arch(), str);
+    g_test_add_func(path, fn);
+}
+
+void qtest_memwrite(QTestState *s, uint64_t addr, const void *data, size_t size)
+{
+    const uint8_t *ptr = data;
+    size_t i;
+
+    qtest_sendf(s, "write 0x%x 0x%x 0x", addr, size);
+    for (i = 0; i < size; i++) {
+        qtest_sendf(s, "%02x", ptr[i]);
+    }
+    qtest_sendf(s, "\n");
+    qtest_rsp(s, 0);
+}
diff --git a/tests/libqtest.h b/tests/libqtest.h
new file mode 100644
index 0000000..dd82926
--- /dev/null
+++ b/tests/libqtest.h
@@ -0,0 +1,63 @@ 
+/*
+ * QTest
+ *
+ * Copyright IBM, Corp. 2012
+ *
+ * Authors:
+ *  Anthony Liguori   <aliguori@us.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+#ifndef LIBQTEST_H
+#define LIBQTEST_H
+
+#include <stdint.h>
+#include <stdbool.h>
+#include <sys/types.h>
+
+typedef struct QTestState QTestState;
+
+extern QTestState *global_qtest;
+
+QTestState *qtest_init(const char *extra_args);
+void qtest_quit(QTestState *s);
+
+bool qtest_get_irq(QTestState *s, int num);
+
+void qtest_outb(QTestState *s, uint16_t addr, uint8_t value);
+
+void qtest_outw(QTestState *s, uint16_t addr, uint16_t value);
+
+void qtest_outl(QTestState *s, uint16_t addr, uint32_t value);
+
+uint8_t qtest_inb(QTestState *s, uint16_t addr);
+
+uint16_t qtest_inw(QTestState *s, uint16_t addr);
+
+uint32_t qtest_inl(QTestState *s, uint16_t addr);
+
+void qtest_memread(QTestState *s, uint64_t addr, void *data, size_t size);
+
+void qtest_memwrite(QTestState *s, uint64_t addr, const void *data, size_t size);
+
+const char *qtest_get_arch(void);
+
+void qtest_add_func(const char *str, void (*fn));
+
+#define qtest_start(args) (            \
+    global_qtest = qtest_init((args)) \
+        )
+
+#define get_irq(num) qtest_get_irq(global_qtest, (num))
+#define outb(addr, val) qtest_outb(global_qtest, (addr), (val))
+#define outw(addr, val) qtest_outw(global_qtest, (addr), (val))
+#define outl(addr, val) qtest_outl(global_qtest, (addr), (val))
+#define inb(addr) qtest_inb(global_qtest, (addr))
+#define inw(addr) qtest_inw(global_qtest, (addr))
+#define inl(addr) qtest_inl(global_qtest, (addr))
+#define memread(addr, data, size) qtest_memread(global_qtest, (addr), (data), (size))
+#define memwrite(addr, data, size) qtest_memwrite(global_qtest, (addr), (data), (size))
+
+#endif