From patchwork Fri Jun 10 10:49:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Po-Hsu Lin X-Patchwork-Id: 1641889 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=SJxFh/aQ; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LKHmb0krgz9s1l for ; Fri, 10 Jun 2022 20:50:21 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1nzcDR-00019v-Ol; Fri, 10 Jun 2022 10:50:09 +0000 Received: from smtp-relay-internal-0.internal ([10.131.114.225] helo=smtp-relay-internal-0.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1nzcDQ-00019o-Hx for kernel-team@lists.ubuntu.com; Fri, 10 Jun 2022 10:50:08 +0000 Received: from mail-pj1-f71.google.com (mail-pj1-f71.google.com [209.85.216.71]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id 40DDD3F157 for ; Fri, 10 Jun 2022 10:50:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1654858208; bh=WwuXgstxd4d7yvu7RCz4n33oscUlwcUnRxrQvQUibYQ=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=SJxFh/aQ4aMUkuvQ7l51bwwfqAZ8Yk7Y/5RxbkuFJWvA+6LVma7jR0rJFq/Ykfac7 nR8Z9fkbGr8sEpNjdct7IAt38dBLCBX1bnPOX4QrZTFbehSRsvqRTa21M9KLiJs4EK N5L3vDR5EpbTCNebc9+vPT6gPeFlbMck1rABEMv9E90TXOh/cUW+OOYJgZB7a1wvQL AafFSm9KTHLRXEy89ccdhgGImAt7Y77TwlJqDgNpcD6MlCwk6vH0mmkYRAzuOzFMiE Icl+zDJ5gX4kX2YCU0Xu3fCbp/lxOwwkqu9PQHfm60IqzIq7Jv7fl7JmhypjE1tMCA MFlWSVEStDQCw== Received: by mail-pj1-f71.google.com with SMTP id o8-20020a17090a9f8800b001dc9f554c7fso13202345pjp.4 for ; Fri, 10 Jun 2022 03:50:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=WwuXgstxd4d7yvu7RCz4n33oscUlwcUnRxrQvQUibYQ=; b=f6AyptyQN51mc7IKVaY/gd6nteycqoBbxyHq2wzoa5p/nN5A/y/YZ/7I36D4xgRJG8 A1d4iStdB4PeFB7KUrA5AeUKFlpmbfWr7HGdReeF6WImltqnMmrf74pLToRGk4dIbi7T o4uLib6v6hcnxWkddAmp9+VwE3t6OHizdeOcDktNTQ/w+vzPThw+G6ZIf4hqv/ttGZSz ugF7j5HIERsKT7cQQiQGY4SStHKo5sieFZHYQ/VJu/0Z3F/xGl7hp6ScxqwWd0TGRWVI 5wnx0sJx5ewojw0+5BkOJopKrNzUxfNiBQKH9LYzKhFqv90Z5PVbVTOLo3BlmWuGQ9oj Gu4A== X-Gm-Message-State: AOAM530kj1KHd3ub49ozK8+DlzcrTtQL11ZSoP2X4l3tWHY7ZnznNICm 2/krDYl/9OQo4E8ZhMIQ1zMNiHwadrEA7pkh3mlkVB9/VWFGwE8NvzE7UlL7WTKGfSOojd03oPC sDyesBUM5CBBvfkRF+o7l6RSXDySTBBDulkreh4fD X-Received: by 2002:a63:8a4c:0:b0:3fc:5c99:a194 with SMTP id y73-20020a638a4c000000b003fc5c99a194mr39948638pgd.313.1654858206951; Fri, 10 Jun 2022 03:50:06 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzUFlOFpaEUpN2Pf7qjTBEKUbdCpl6dgB8JG/CnRL87m2eQRRhvhSD832fa9A6jNnyG83mF4A== X-Received: by 2002:a63:8a4c:0:b0:3fc:5c99:a194 with SMTP id y73-20020a638a4c000000b003fc5c99a194mr39948619pgd.313.1654858206583; Fri, 10 Jun 2022 03:50:06 -0700 (PDT) Received: from localhost.localdomain (111-71-123-224.emome-ip.hinet.net. [111.71.123.224]) by smtp.gmail.com with ESMTPSA id a13-20020a170902710d00b0015e8d4eb1d2sm18381675pll.28.2022.06.10.03.50.03 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Jun 2022 03:50:04 -0700 (PDT) From: Po-Hsu Lin To: kernel-team@lists.ubuntu.com Subject: [autotest-client-tests][PATCH] UBUNTU: SAUCE: ubuntu_qrt_kernel_aslr_collisions: improve test granularity Date: Fri, 10 Jun 2022 18:49:44 +0800 Message-Id: <20220610104944.400217-1-po-hsu.lin@canonical.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" Improve the test granularity like how we do it for the ubuntu_qrt_kernel_security test, run sub test cases one-by-one with: python2 test-kernel-aslr-collisions.py -v TestClass.TestMethod Note that the KernelASLRCollisionsTest.test_stack_collision test will take about 8 minutes to run, set timeout to 15 minutes to guarantee it can finish in time. Patch tested with T-3.13 on bare-metal node "gonzo", and it works as expected. Signed-off-by: Po-Hsu Lin Acked-by: Sean Feole --- ubuntu_qrt_kernel_aslr_collisions/control | 19 ++++++++++++++++++- .../ubuntu_qrt_kernel_aslr_collisions.py | 2 +- 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/ubuntu_qrt_kernel_aslr_collisions/control b/ubuntu_qrt_kernel_aslr_collisions/control index 26074501..8970cee0 100644 --- a/ubuntu_qrt_kernel_aslr_collisions/control +++ b/ubuntu_qrt_kernel_aslr_collisions/control @@ -12,6 +12,23 @@ result = job.run_test_detail(NAME, test_name='setup', tag='setup', timeout=60*60 if result == 'ERROR': print("ERROR: test setup has failed, skipping all the sub tests") else: - result = job.run_test_detail(NAME, test_name='test-kernel-aslr-collisions.py', tag='test-kernel-aslr-collisions.py', timeout=60*60*2) + fn = os.path.join(job.bindir, 'tmp', NAME, 'src', 'qa-regression-testing', 'scripts', 'test-kernel-aslr-collisions.py') + cmd = 'grep -e "class .*Test.*:" -e "def test_.*:" {} | sed "s/(.*)://"'.format(fn) + output = utils.system_output(cmd, verbose=False).splitlines() + all_tests = {} + for line in output: + item = line.strip() + if item.startswith('class '): + category = item.replace('class ', '') + all_tests[category] = [] + elif item.startswith('def '): + sub_test = item.replace('def ', '') + all_tests[category].append(sub_test) + + for category in all_tests: + if all_tests[category]: + for sub_test in all_tests[category]: + test = '{}.{}'.format(category, sub_test) + job.run_test_detail(NAME, test_name=test, tag=test, timeout=60*15) # vi:set ts=4 sw=4 expandtab syntax=python: diff --git a/ubuntu_qrt_kernel_aslr_collisions/ubuntu_qrt_kernel_aslr_collisions.py b/ubuntu_qrt_kernel_aslr_collisions/ubuntu_qrt_kernel_aslr_collisions.py index 51dab5ed..288371a5 100644 --- a/ubuntu_qrt_kernel_aslr_collisions/ubuntu_qrt_kernel_aslr_collisions.py +++ b/ubuntu_qrt_kernel_aslr_collisions/ubuntu_qrt_kernel_aslr_collisions.py @@ -52,7 +52,7 @@ class ubuntu_qrt_kernel_aslr_collisions(test.test): scripts = os.path.join(self.srcdir, 'qa-regression-testing', 'scripts') os.chdir(scripts) - cmd = 'python2 ./%s -v' % test_name + cmd = 'python2 ./test-kernel-aslr-collisions.py -v %s' % test_name self.results = utils.system_output(cmd, retain_output=True)