From patchwork Wed Nov 28 19:36:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brendan Higgins X-Patchwork-Id: 1004802 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.infradead.org (client-ip=2607:7c80:54:e::133; helo=bombadil.infradead.org; envelope-from=linux-um-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="uG1Yx7Hq"; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.b="ZVg5n/fJ"; dkim-atps=neutral Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 434rYQ5srJz9s5c for ; Thu, 29 Nov 2018 06:38:34 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version :Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=eXBu6e9lMNXPEz8mnNVwcCzwAyghpbLW7Ya8t7fbPKg=; b=uG1Yx7Hq2QDong KA35mEn3MU9O8GKTjANzIsw7DLYRyK4zpAvis+hsI8dMYBY3m43Zhw1pFny2SE18chJj0xsHQbzP0 mFDC7RXjXONPFejmEnzTtr+punSY1iELzdigPAlechNmP0VwHByXlwiDwClgajMr5P9aXzoZHBfy1 GchjZdGB4DKXOVOlTooWgM8QgZIsWDdMeZyOY5cFNFUPAf5BADqcVfe6KeESGPLwzp6sk42TLbSno a+0hQkDVZ0h+WwUCrBiChczIHMxTTSsrK8n4ita5nH9ViFJ89krlYBDvlWc1nBLCSh3ZK5ipVzTrH NVoHKyRWxTElTOR1o9MQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1gS5fH-0003mr-IU; Wed, 28 Nov 2018 19:38:27 +0000 Received: from mail-pg1-x54a.google.com ([2607:f8b0:4864:20::54a]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1gS5fE-0003fJ-F3 for linux-um@lists.infradead.org; Wed, 28 Nov 2018 19:38:25 +0000 Received: by mail-pg1-x54a.google.com with SMTP id h9so12797168pgm.1 for ; Wed, 28 Nov 2018 11:38:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=PSWj5cFU+jrigLyNwOIkjVkSQOjGgnr+grtJvw72p2k=; b=ZVg5n/fJ90O5sJ3IAOCgBQT0Y735goRSISlmv+c+yyF4e39VXUeukE1DtCXZgiFsws 3CywqLzYt/Xhiav0fZQYXuP0MBPa2nj7xGUqjl4yl6PkgRF46WyaNehaXEstMd/0ZPco VTeAOL/V+elEzDVz8z+IOR1L3qbY93bwZ/KLsOrNx4lPYIJw6R+nr+w1Bow/TVkgE3+o BbHyFCw6lvIp1p/XeyiTcwpouOAogKZFDv1/WBmGooPKp1CAggkUUd+xFA//d4OZe+rO kNdACscRvJZ+9+nUv6754NjbTu0ZDERwEC42qfElTYMeQmt85QBlULjbj4UXeWOt1QPD RdxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=PSWj5cFU+jrigLyNwOIkjVkSQOjGgnr+grtJvw72p2k=; b=VhwHuH3zvj9Li+kkWxrh4QfppoelgMqjixrPFzYpHJpa8G1bjGYexyZVbrWnLZ8ZrM KGVvD27wPN05/8b5odTiob9fWAjROZ/hKVtcQKmj4PKk6x3xW2tCZb1Gy4jV7zqKoPs6 CtGY0Gc2hL0jY6VFvNK2Iqsk+pZzIp6PhFvm4AZ8KX/NMwsCPC4w4N0ZJUHP/CcYOVoJ 2Cub/g3YCHh3QW5ABqYHuKlD4dljym5w5MyHsoSGC/dSkYFpDAhrfumwDENEpxc7kKmw IFGnVGO2aOyGnQ4wARfkVfxnU3wTRHJheWeHGXEpVvnz4YVZx+4yXs8n0OXcXkUX7Lh9 y37Q== X-Gm-Message-State: AA+aEWYs3DM/Bo7fMfJvqNP0dQTolqs3EdSXttB60rqk9BYDHavYEWYN S1xMoQMTQOBQGRXKr1mpsaNTvG/rG01gDzPGxRV/Jg== X-Google-Smtp-Source: AFSGD/UNL8VzMXvdy/txBvVB+FEhEs2NNoQ9cq2xnih5rgsgUaCtXAFa837C43GfxD4knn2v0XS0RHUhlr4yWXZK/F1pTA== X-Received: by 2002:a63:b601:: with SMTP id j1mr2949103pgf.58.1543433893299; Wed, 28 Nov 2018 11:38:13 -0800 (PST) Date: Wed, 28 Nov 2018 11:36:29 -0800 In-Reply-To: <20181128193636.254378-1-brendanhiggins@google.com> Message-Id: <20181128193636.254378-13-brendanhiggins@google.com> Mime-Version: 1.0 References: <20181128193636.254378-1-brendanhiggins@google.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c-goog Subject: [RFC v3 12/19] kunit: add KUnit wrapper script and simple output parser From: Brendan Higgins To: gregkh@linuxfoundation.org, keescook@google.com, mcgrof@kernel.org, shuah@kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20181128_113824_507529_98B15040 X-CRM114-Status: GOOD ( 13.69 ) X-Spam-Score: -7.7 (-------) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-7.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:54a listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -0.0 SPF_PASS SPF: sender matches SPF record 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.0 DKIMWL_WL_MED DKIMwl.org - Whitelisted Medium sender X-BeenThere: linux-um@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: brakmo@fb.com, jdike@addtoit.com, dri-devel@lists.freedesktop.org, linux-kselftest@vger.kernel.org, frowand.list@gmail.com, robh@kernel.org, linux-nvdimm@lists.01.org, richard@nod.at, knut.omang@oracle.com, kieran.bingham@ideasonboard.com, Felix Guo , joel@jms.id.au, khilman@baylibre.com, Brendan Higgins , Tim.Bird@sony.com, linux-um@lists.infradead.org, rostedt@goodmis.org, julia.lawall@lip6.fr, dan.j.williams@intel.com, kunit-dev@googlegroups.com, linux-kernel@vger.kernel.org, daniel@ffwll.ch, mpe@ellerman.id.au, joe@perches.com Sender: "linux-um" Errors-To: linux-um-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org The KUnit wrapper script interfaces with the two modules (kunit_config.py and kunit_kernel.py) and provides a command line interface for running KUnit tests. This interface allows the caller to specify options like test timeouts. The script handles configuring, building and running the kernel and tests. The output parser (kunit_parser.py) simply strips out all the output from the kernel that is outputted as part of it's initialization sequence. This ensures that only the output from KUnit is displayed on the screen. A full version of the output is written to test.log, or can be seen by passing --raw_output to the wrapper script. Signed-off-by: Felix Guo Signed-off-by: Brendan Higgins --- tools/testing/kunit/kunit.py | 40 +++++++++++++++++++++++++++++ tools/testing/kunit/kunit_kernel.py | 3 +-- tools/testing/kunit/kunit_parser.py | 24 +++++++++++++++++ 3 files changed, 65 insertions(+), 2 deletions(-) create mode 100755 tools/testing/kunit/kunit.py create mode 100644 tools/testing/kunit/kunit_parser.py diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py new file mode 100755 index 0000000000000..1356be404996b --- /dev/null +++ b/tools/testing/kunit/kunit.py @@ -0,0 +1,40 @@ +#!/usr/bin/python3 +# SPDX-License-Identifier: GPL-2.0 + +# A thin wrapper on top of the KUnit Kernel + +import argparse +import sys +import os + +import kunit_config +import kunit_kernel +import kunit_parser + +parser = argparse.ArgumentParser(description='Runs KUnit tests.') + +parser.add_argument('--raw_output', help='don\'t format output from kernel', + action='store_true') + +parser.add_argument('--timeout', help='maximum number of seconds to allow for ' + 'all tests to run. This does not include time taken to ' + 'build the tests.', type=int, default=300, + metavar='timeout') + +cli_args = parser.parse_args() +linux = kunit_kernel.LinuxSourceTree() + +success = linux.build_reconfig() +if not success: + quit() + +print('Building KUnit Kernel ...') +success = linux.build_um_kernel() +if not success: + quit() + +print('Starting KUnit Kernel ...') +if cli_args.raw_output: + kunit_parser.raw_output(linux.run_kernel(timeout=cli_args.timeout)) +else: + kunit_parser.parse_run_tests(linux.run_kernel(timeout=cli_args.timeout)) diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py index bba7ea7ca1869..623f25b16f6c8 100644 --- a/tools/testing/kunit/kunit_kernel.py +++ b/tools/testing/kunit/kunit_kernel.py @@ -116,8 +116,7 @@ class LinuxSourceTree(object): return False return True - def run_kernel(self, args=[]): - timeout = None + def run_kernel(self, args=[], timeout=None): args.extend(['mem=256M']) process = self._ops.linux_bin(args, timeout) with open('test.log', 'w') as f: diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py new file mode 100644 index 0000000000000..1dff3adb73bd3 --- /dev/null +++ b/tools/testing/kunit/kunit_parser.py @@ -0,0 +1,24 @@ +# SPDX-License-Identifier: GPL-2.0 + +import re + +kunit_start_re = re.compile('console .* enabled') +kunit_end_re = re.compile('List of all partitions:') + +def isolate_kunit_output(kernel_output): + started = False + for line in kernel_output: + if kunit_start_re.match(line): + started = True + elif kunit_end_re.match(line): + break + elif started: + yield line + +def raw_output(kernel_output): + for line in kernel_output: + print(line) + +def parse_run_tests(kernel_output): + for output in isolate_kunit_output(kernel_output): + print(output)