From patchwork Tue Aug 28 11:18:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Lanig X-Patchwork-Id: 962841 X-Patchwork-Delegate: petr.vorel@gmail.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=lists.linux.it (client-ip=213.254.12.146; helo=picard.linux.it; envelope-from=ltp-bounces+incoming=patchwork.ozlabs.org@lists.linux.it; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=suse.com Received: from picard.linux.it (picard.linux.it [213.254.12.146]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4205rD1lZ5z9s1x for ; Tue, 28 Aug 2018 21:19:40 +1000 (AEST) Received: from picard.linux.it (localhost [IPv6:::1]) by picard.linux.it (Postfix) with ESMTP id AE0653E71BD for ; Tue, 28 Aug 2018 13:19:37 +0200 (CEST) X-Original-To: ltp@lists.linux.it Delivered-To: ltp@picard.linux.it Received: from in-5.smtp.seeweb.it (in-5.smtp.seeweb.it [IPv6:2001:4b78:1:20::5]) by picard.linux.it (Postfix) with ESMTP id 783FB3E6078 for ; Tue, 28 Aug 2018 13:19:35 +0200 (CEST) Received: from smtp.nue.novell.com (smtp.nue.novell.com [195.135.221.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by in-5.smtp.seeweb.it (Postfix) with ESMTPS id D6A6C600A46 for ; Tue, 28 Aug 2018 13:19:33 +0200 (CEST) Received: from emea4-mta.ukb.novell.com ([10.120.13.87]) by smtp.nue.novell.com with ESMTP (TLS encrypted); Tue, 28 Aug 2018 13:19:32 +0200 Received: from linux-zcnv.suse.de ([10.120.13.202]) by emea4-mta.ukb.novell.com with ESMTP (TLS encrypted); Tue, 28 Aug 2018 12:19:19 +0100 From: Christian Lanig To: ltp@lists.linux.it Date: Tue, 28 Aug 2018 13:18:44 +0200 Message-Id: <20180828111844.20152-2-clanig@suse.com> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20180828111844.20152-1-clanig@suse.com> References: <20180522193430.20117-2-pvorel@suse.cz> <20180828111844.20152-1-clanig@suse.com> X-Virus-Scanned: clamav-milter 0.99.2 at in-5.smtp.seeweb.it X-Virus-Status: Clean X-Spam-Status: No, score=-0.0 required=7.0 tests=SPF_PASS autolearn=disabled version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on in-5.smtp.seeweb.it Subject: [LTP] [PATCH 2/2] Add wanted output to shell lib test case X-BeenThere: ltp@lists.linux.it X-Mailman-Version: 2.1.18 Precedence: list List-Id: Linux Test Project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ltp-bounces+incoming=patchwork.ozlabs.org@lists.linux.it Sender: "ltp" --- Please do not merge! I would like to get some feedback for this one and the previous patch. It is based on Petr Vorel's [RFC,v3,2/2] lib: Add tests from May 22nd 2018 with the message ID 20180522193430.20117-2-pvorel@suse.cz. These two patches transfer Petr's test cases in a separate folder and a shell script loops over all files as suggested in the replies to the initial message. The output is verified against a comment section in the test case file itself. This section is started with the line "# output:" and it is expected that it goes to the end of the document without interception. I have run the script with various Shell implementations to make sure it's portable. Limitations: The script is currently unable to test e.g. whether TFAIL is displayed with red color. So far I have only added the correct output line in a single test case file so just one test is currently passing. Before proceeding I would like to know whether this is a good way to go. lib/newlib_tests/shell/test.TST_TEST.sh | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/lib/newlib_tests/shell/test.TST_TEST.sh b/lib/newlib_tests/shell/test.TST_TEST.sh index 42f14151b..aa4e20377 100755 --- a/lib/newlib_tests/shell/test.TST_TEST.sh +++ b/lib/newlib_tests/shell/test.TST_TEST.sh @@ -21,5 +21,12 @@ do_test() } tst_run +# output: # test 1 TINFO: Test 1 passed with no data ('') # test 1 TPASS: true returned 0 +# +# Summary: +# passed 1 +# failed 0 +# skipped 0 +# warnings 0