From patchwork Mon Aug 6 14:33:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emanuele Giuseppe Esposito X-Patchwork-Id: 953897 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="pHkke9PJ"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41kgDJ0NgXz9s0R for ; Tue, 7 Aug 2018 00:35:26 +1000 (AEST) Received: from localhost ([::1]:34344 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fmgbT-0004g9-NR for incoming@patchwork.ozlabs.org; Mon, 06 Aug 2018 10:35:23 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35438) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fmgam-0004ST-IG for qemu-devel@nongnu.org; Mon, 06 Aug 2018 10:34:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fmgaj-0000km-Fa for qemu-devel@nongnu.org; Mon, 06 Aug 2018 10:34:40 -0400 Received: from mail-ed1-x544.google.com ([2a00:1450:4864:20::544]:33042) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fmgaj-0000k7-4T for qemu-devel@nongnu.org; Mon, 06 Aug 2018 10:34:37 -0400 Received: by mail-ed1-x544.google.com with SMTP id x5-v6so5273795edr.0 for ; Mon, 06 Aug 2018 07:34:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=R+XS4Bfme85N0Y0kJMKGaSaV94xvHfCdSgfD85LFxxg=; b=pHkke9PJbYr5Ye/ZWVFimGzhjQY5XNN7iMujHimZSP4tpsia2ftBy2eHpqB8e/tBzj Q3Tz34fQhdfpmA/lF72yolVVA75S7E27N5L9n9fUoGpzVUiq69pRD6ejinax4uQEPUWi mK/rBI3TqCz76OCh/+fTnKl+2+BxpuvrV70Aa6oOAFoXED/9CC4UeFTQpvrj5/X48HAc Wmnlb0hX1OQbR8GeSnlHOrHAn8F8Qu2GrVammfBVc/GI5LjaGMGUYNWdJWdgsTg4dFrS rSySG032/ZVm0nhmCEimqkvL3TR11ruKG7Y//alCqOhkY3R70r7sd9j/5JTJ7FZpjxJd 0mKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=R+XS4Bfme85N0Y0kJMKGaSaV94xvHfCdSgfD85LFxxg=; b=RmqICkAquJY8E3IIlRK84VjigwYBv85IhHZ1lSbKfl14YOdnyK8wd5ChYcT87a7e3z DbpGleRreeqT5BUBGq0Mi47Gc+5SdOpwEib8RwD0X/xWdoFc2RLuDoS5xGIUBp81qgpj rCHZzZosK9Qxj+SywYirU4e6q+psrM0Ciz0TSDMLzAiD1DAj9iUWOi5WCGkj2ip1+CZH ccYcv14tBMfUbNjdIC71jxHEHr2xGyooWRqj7bBHALU81FenrxHeqXpJE+Hv0LpxO2Si 0thwhGEPWq/r+J3+F8eorPxd600mW7T77wcibqhzKNAPq3uo6iqBRPDGhctRm2NAI7qR Gf8g== X-Gm-Message-State: AOUpUlFCRFvfHyRsjhZhvx31ck1kmuDUDKjfeS0QnrVmNKXe+M0ZtQ0w 88wUmbCz/v1I90Aqs7VPTJA= X-Google-Smtp-Source: AAOMgpf97iyEnl4uSB1umobC8DsAIJWZDoF3oy72pTMbZby0fnH7K8dKq5pW4WPdHyJiXofeAiRkxA== X-Received: by 2002:a50:f297:: with SMTP id f23-v6mr18911361edm.40.1533566075626; Mon, 06 Aug 2018 07:34:35 -0700 (PDT) Received: from localhost.localdomain ([194.230.159.126]) by smtp.gmail.com with ESMTPSA id x13-v6sm14615157edx.17.2018.08.06.07.34.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 06 Aug 2018 07:34:35 -0700 (PDT) From: Emanuele Giuseppe Esposito To: Paolo Bonzini Date: Mon, 6 Aug 2018 16:33:38 +0200 Message-Id: <20180806143412.27722-1-e.emanuelegiuseppe@gmail.com> X-Mailer: git-send-email 2.17.1 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::544 Subject: [Qemu-devel] [PATCH v2 00/34] Qtest driver framework X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Stefan Hajnoczi , Emanuele Giuseppe Esposito , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" qgraph API for the qtest driver framework This series of patches introduce a different qtest driver organization, viewing machines, drivers and tests as node in a graph, each having one or multiple edges relations. The idea is to have a framework where each test asks for a specific driver, and the framework takes care of allocating the proper devices required and passing the correct command line arguments to QEMU. A node can be of four types: - MACHINE: for example "arm/raspi2" - DRIVER: for example "generic-sdhci" - INTERFACE: for example "sdhci" (interface for all "-sdhci" drivers) - TEST: for example "sdhci-test", consumes an interface and tests the functions provided An edge relation between two nodes (drivers or machines) X and Y can be: - X CONSUMES Y: Y can be plugged into X - X PRODUCES Y: X provides the interface Y - X CONTAINS Y: Y is part of X component Basic framework steps are the following: - All nodes and edges are created in their respective machine/driver/test files - The framework starts QEMU and asks for a list of available devices and machines - The framework walks the graph starting from the available machines and performs a Depth First Search for tests - Once a test is found, the path is walked again and all drivers are allocated accordingly and the final interface is passed to the test - The test is executed - Unused objects are cleaned and the path discovery is continued Depending on the QEMU binary used, only some drivers/machines will be available and only test that are reached by them will be executed. This work is being done as Google Summer of Code 2018 project for QEMU, my mentors are Paolo Bonzini and Laurent Vivier. Additional infos on the project can be found at: https://wiki.qemu.org/Features/qtest_driver_framework v2: - added command line arguments caching - converted all virtio tests - converted e1000e test - added two more machines (virt and ppc64) - introduced separation between edge-name (used by get_driver) and node-name (matched with QMP query result) - edge carries also additional arguments, like the pci address of destination device - test now can execute a function before and after the command line is allocated, to allocate various file and sockets needed by command line and test - better documentation with working example in qgraph.h - removed esplicit creation of interfaces Signed-off-by: Emanuele Giuseppe Esposito Emanuele Giuseppe Esposito (33): tests: qgraph API for the qtest driver framework tests/qgraph: rename qpci_init_pc functions tests/qgraph: pci-pc driver and interface nodes tests/qgraph: x86_64/pc machine node tests/qgraph: sdhci driver and interface nodes tests/qgraph: sdhci test node tests/qgraph: arm/raspi2 machine node tests/qgraph: rename qpci_init_spapr functions tests/qgraph: pci-spapr driver and interface nodes tests/qgraph: ppc64/pseries machine node test/qgraph: e1000e driver and interface nodes test/qgraph: e1000e-test node test/qgraph: virtio_start_device function test/qgraph: virtio-pci driver and interface nodes tests/qgraph: rename qvirtio_mmio_init_device functions test/qgraph: virtio-mmio driver and interface nodes test/qgraph: arm/virt machine node test/qgraph: virtio-serial driver and interface nodes test/qgraph: virtio-console test node test/qgraph: virtio-serial test node test/qgraph: virtio-9p driver and interface nodes test/qgraph: virtio-9p test node test/qgraph: virtio-balloon driver and interface nodes test/qgraph: virtio-balloon test node test/qgraph: virtio-rng driver and interface nodes test/qgraph: virtio-rng test node test/qgraph: virtio-blk driver and interface nodes test/qgraph: virtio-blk test node test/qgraph: virtio-net driver and interface nodes test/qgraph: virtio-net test node test/qgraph: virtio-scsi driver and interface nodes test/qgraph: virtio-scsi test node test/qgraph: temporarly commented vhost-user-test Paolo Bonzini (1): tests: virtio: separate ccw tests from libqos configure | 2 +- include/qemu/module.h | 2 + tests/Makefile.include | 79 +-- tests/e1000e-test.c | 354 +++---------- tests/i440fx-test.c | 2 +- tests/ide-test.c | 2 +- tests/libqos/ahci.c | 2 +- tests/libqos/e1000e.c | 262 ++++++++++ tests/libqos/e1000e.h | 53 ++ tests/libqos/libqos-pc.c | 2 +- tests/libqos/libqos-spapr.c | 2 +- tests/libqos/pci-pc.c | 41 +- tests/libqos/pci-pc.h | 22 +- tests/libqos/pci-spapr.c | 57 ++- tests/libqos/pci-spapr.h | 26 +- tests/libqos/pci.c | 48 +- tests/libqos/pci.h | 15 + tests/libqos/ppc64_pseries-machine.c | 111 +++++ tests/libqos/qgraph.c | 721 +++++++++++++++++++++++++++ tests/libqos/qgraph.h | 514 +++++++++++++++++++ tests/libqos/qgraph_extra.h | 263 ++++++++++ tests/libqos/raspi2-machine.c | 82 +++ tests/libqos/sdhci.c | 163 ++++++ tests/libqos/sdhci.h | 69 +++ tests/libqos/virt-machine.c | 90 ++++ tests/libqos/virtio-9p.c | 164 ++++++ tests/libqos/virtio-9p.h | 42 ++ tests/libqos/virtio-balloon.c | 111 +++++ tests/libqos/virtio-balloon.h | 39 ++ tests/libqos/virtio-blk.c | 126 +++++ tests/libqos/virtio-blk.h | 40 ++ tests/libqos/virtio-mmio.c | 72 ++- tests/libqos/virtio-mmio.h | 6 +- tests/libqos/virtio-net.c | 177 +++++++ tests/libqos/virtio-net.h | 41 ++ tests/libqos/virtio-pci.c | 80 ++- tests/libqos/virtio-pci.h | 12 + tests/libqos/virtio-rng.c | 108 ++++ tests/libqos/virtio-rng.h | 39 ++ tests/libqos/virtio-scsi.c | 117 +++++ tests/libqos/virtio-scsi.h | 39 ++ tests/libqos/virtio-serial.c | 109 ++++ tests/libqos/virtio-serial.h | 39 ++ tests/libqos/virtio.c | 9 + tests/libqos/virtio.h | 1 + tests/libqos/x86_64_pc-machine.c | 110 ++++ tests/q35-test.c | 4 +- tests/qos-test.c | 480 ++++++++++++++++++ tests/rtl8139-test.c | 2 +- tests/sdhci-test.c | 222 +++------ tests/tco-test.c | 2 +- tests/test-qgraph.c | 446 +++++++++++++++++ tests/usb-hcd-ehci-test.c | 2 +- tests/vhost-user-test.c | 7 +- tests/virtio-9p-test.c | 221 +++----- tests/virtio-balloon-test.c | 22 +- tests/virtio-blk-test.c | 473 +++++++----------- tests/virtio-ccw-test.c | 121 +++++ tests/virtio-console-test.c | 30 +- tests/virtio-net-test.c | 163 ++---- tests/virtio-rng-test.c | 25 +- tests/virtio-scsi-test.c | 155 +++--- tests/virtio-serial-test.c | 27 +- 63 files changed, 5624 insertions(+), 1243 deletions(-) create mode 100644 tests/libqos/e1000e.c create mode 100644 tests/libqos/e1000e.h create mode 100644 tests/libqos/ppc64_pseries-machine.c create mode 100644 tests/libqos/qgraph.c create mode 100644 tests/libqos/qgraph.h create mode 100644 tests/libqos/qgraph_extra.h create mode 100644 tests/libqos/raspi2-machine.c create mode 100644 tests/libqos/sdhci.c create mode 100644 tests/libqos/sdhci.h create mode 100644 tests/libqos/virt-machine.c create mode 100644 tests/libqos/virtio-9p.c create mode 100644 tests/libqos/virtio-9p.h create mode 100644 tests/libqos/virtio-balloon.c create mode 100644 tests/libqos/virtio-balloon.h create mode 100644 tests/libqos/virtio-blk.c create mode 100644 tests/libqos/virtio-blk.h create mode 100644 tests/libqos/virtio-net.c create mode 100644 tests/libqos/virtio-net.h create mode 100644 tests/libqos/virtio-rng.c create mode 100644 tests/libqos/virtio-rng.h create mode 100644 tests/libqos/virtio-scsi.c create mode 100644 tests/libqos/virtio-scsi.h create mode 100644 tests/libqos/virtio-serial.c create mode 100644 tests/libqos/virtio-serial.h create mode 100644 tests/libqos/x86_64_pc-machine.c create mode 100644 tests/qos-test.c create mode 100644 tests/test-qgraph.c create mode 100644 tests/virtio-ccw-test.c