From patchwork Sat Sep 6 01:00:08 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 386566 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from theia.denx.de (theia.denx.de [85.214.87.163]) by ozlabs.org (Postfix) with ESMTP id 325281400E9 for ; Sat, 6 Sep 2014 11:03:52 +1000 (EST) Received: from localhost (localhost [127.0.0.1]) by theia.denx.de (Postfix) with ESMTP id 0D88FA7469; Sat, 6 Sep 2014 03:03:33 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at theia.denx.de Received: from theia.denx.de ([127.0.0.1]) by localhost (theia.denx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bVJO2hGsok-f; Sat, 6 Sep 2014 03:03:32 +0200 (CEST) Received: from theia.denx.de (localhost [127.0.0.1]) by theia.denx.de (Postfix) with ESMTP id 52022A75AE; Sat, 6 Sep 2014 03:02:28 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by theia.denx.de (Postfix) with ESMTP id B75CFA7459 for ; Sat, 6 Sep 2014 03:02:17 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at theia.denx.de Received: from theia.denx.de ([127.0.0.1]) by localhost (theia.denx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oyW5vgexnoHi for ; Sat, 6 Sep 2014 03:02:14 +0200 (CEST) X-policyd-weight: NOT_IN_SBL_XBL_SPAMHAUS=-1.5 NOT_IN_SPAMCOP=-1.5 NOT_IN_BL_NJABL=-1.5 (only DNSBL check requested) Received: from mail-vc0-f201.google.com (mail-vc0-f201.google.com [209.85.220.201]) by theia.denx.de (Postfix) with ESMTPS id 35C83A745B for ; Sat, 6 Sep 2014 03:01:58 +0200 (CEST) Received: by mail-vc0-f201.google.com with SMTP id le20so1345832vcb.4 for ; Fri, 05 Sep 2014 18:01:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=JleJOM8jUJLKkTq6Y1ZJM+Aq3DqxieYWJG994ltmnYo=; b=A+kesOzRse7JRhmpQTlZZDr3tyGa4IXi6Oue/Z68W25Yx9Wwl/RoKsdI18iwTYyi5b eyyV5C0WLymfw1bGDP8X79omjv508KWX+KZ3ik3IJj4LG2yG3vB2ZI0gtsAH1K+zdIQh /S2QQIAb9IaXrkcYN2U/Be6YSl0qbmq7B3qANnf0LoPtgBq216qMqgt9TUUeDidWJnXN UirR+OoAx+Q+wKLJKobkVT4lozfmYk3INuYAAwz6ydBeeuUl+GB/NrFFmsTziMXKZy9I yOWhkKt5szXYZGuZTVhoQsy4kvrbjq8iv9Uhoy6aSI1wldTCICq3Chae2UwiMz7OwHm2 S4pg== X-Gm-Message-State: ALoCoQnAXlar4g65kWEpP+QvuHuvL1Y9ug/Rv2DPzaQAGNCQ7C8Pk9OBFKk826oP6cRyHdB6mnEo X-Received: by 10.236.31.129 with SMTP id m1mr8756403yha.47.1409965317156; Fri, 05 Sep 2014 18:01:57 -0700 (PDT) Received: from corp2gmr1-1.hot.corp.google.com (corp2gmr1-1.hot.corp.google.com [172.24.189.92]) by gmr-mx.google.com with ESMTPS id n22si577265yhd.1.2014.09.05.18.01.57 for (version=TLSv1.1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 05 Sep 2014 18:01:57 -0700 (PDT) Received: from kaki.bld.corp.google.com (kaki.bld.corp.google.com [172.29.216.32]) by corp2gmr1-1.hot.corp.google.com (Postfix) with ESMTP id F2CD631C31B; Fri, 5 Sep 2014 18:01:56 -0700 (PDT) Received: by kaki.bld.corp.google.com (Postfix, from userid 121222) id 85FD1223B50; Fri, 5 Sep 2014 19:01:56 -0600 (MDT) From: Simon Glass To: U-Boot Mailing List Date: Fri, 5 Sep 2014 19:00:08 -0600 Message-Id: <1409965223-29863-4-git-send-email-sjg@chromium.org> X-Mailer: git-send-email 2.1.0.rc2.206.gedb03e5 In-Reply-To: <1409965223-29863-1-git-send-email-sjg@chromium.org> References: <1409965223-29863-1-git-send-email-sjg@chromium.org> Cc: Steve Rae Subject: [U-Boot] [PATCH v3 03/18] buildman: Enhance basic test to check summary output X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.11 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: u-boot-bounces@lists.denx.de Errors-To: u-boot-bounces@lists.denx.de Adjust the basic test so that it checks all console output. This will help to ensure that the builder is behaving correctly with printing summary information. Signed-off-by: Simon Glass --- Changes in v3: None Changes in v2: None tools/buildman/test.py | 101 ++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 96 insertions(+), 5 deletions(-) diff --git a/tools/buildman/test.py b/tools/buildman/test.py index a51c942..f0c4d0e 100644 --- a/tools/buildman/test.py +++ b/tools/buildman/test.py @@ -21,20 +21,21 @@ import builder import control import command import commit +import terminal import toolchain errors = [ '''main.c: In function 'main_loop': main.c:260:6: warning: unused variable 'joe' [-Wunused-variable] ''', - '''main.c: In function 'main_loop': + '''main.c: In function 'main_loop2': main.c:295:2: error: 'fred' undeclared (first use in this function) main.c:295:2: note: each undeclared identifier is reported only once for each function it appears in make[1]: *** [main.o] Error 1 make: *** [common/libcommon.o] Error 2 Make failed ''', - '''main.c: In function 'main_loop': + '''main.c: In function 'main_loop3': main.c:280:6: warning: unused variable 'mary' [-Wunused-variable] ''', '''powerpc-linux-ld: warning: dot moved backwards before `.bss' @@ -103,6 +104,10 @@ class TestBuild(unittest.TestCase): self.toolchains.Add('powerpc-linux-gcc', test=False) self.toolchains.Add('gcc', test=False) + # Avoid sending any output + terminal.SetPrintTestMode() + self._col = terminal.Color() + def Make(self, commit, brd, stage, *args, **kwargs): result = command.CommandResult() boardnum = int(brd.target[-1]) @@ -121,13 +126,26 @@ class TestBuild(unittest.TestCase): if not os.path.isdir(target_dir): os.mkdir(target_dir) - #time.sleep(.2 + boardnum * .2) result.combined = result.stdout + result.stderr return result - def testBasic(self): - """Test basic builder operation""" + def assertSummary(self, text, arch, plus, boards, ok=False): + col = self._col + expected_colour = col.GREEN if ok else col.RED + expect = '%10s: ' % arch + # TODO(sjg@chromium.org): If plus is '', we shouldn't need this + expect += col.Color(expected_colour, plus) + expect += ' ' + for board in boards: + expect += col.Color(expected_colour, ' %s' % board) + self.assertEqual(text, expect) + + def testOutput(self): + """Test basic builder operation and output + + This does a line-by-line verification of the summary output. + """ output_dir = tempfile.mkdtemp() if not os.path.isdir(output_dir): os.mkdir(output_dir) @@ -138,8 +156,81 @@ class TestBuild(unittest.TestCase): build.BuildBoards(self.commits, board_selected, keep_outputs=False, verbose=False) + lines = terminal.GetPrintTestLines() + count = 0 + for line in lines: + if line.text.strip(): + count += 1 + + # We should get one starting message, then an update for every commit + # built. + self.assertEqual(count, len(commits) * len(boards) + 1) build.SetDisplayOptions(show_errors=True); build.ShowSummary(self.commits, board_selected) + lines = terminal.GetPrintTestLines() + self.assertEqual(lines[0].text, '01: %s' % commits[0][1]) + self.assertEqual(lines[1].text, '02: %s' % commits[1][1]) + + # We expect all archs to fail + col = terminal.Color() + self.assertSummary(lines[2].text, 'sandbox', '+', ['board4']) + self.assertSummary(lines[3].text, 'arm', '+', ['board1']) + self.assertSummary(lines[4].text, 'powerpc', '+', ['board2', 'board3']) + + # Now we should have the compiler warning + self.assertEqual(lines[5].text, 'w+%s' % + errors[0].rstrip().replace('\n', '\nw+')) + self.assertEqual(lines[5].colour, col.MAGENTA) + + self.assertEqual(lines[6].text, '03: %s' % commits[2][1]) + self.assertSummary(lines[7].text, 'sandbox', '+', ['board4']) + self.assertSummary(lines[8].text, 'arm', '', ['board1'], ok=True) + self.assertSummary(lines[9].text, 'powerpc', '+', ['board2', 'board3']) + + # Compiler error + self.assertEqual(lines[10].text, '+%s' % + errors[1].rstrip().replace('\n', '\n+')) + + self.assertEqual(lines[11].text, '04: %s' % commits[3][1]) + self.assertSummary(lines[12].text, 'sandbox', '', ['board4'], ok=True) + self.assertSummary(lines[13].text, 'powerpc', '', ['board2', 'board3'], + ok=True) + + # Compile error fixed + self.assertEqual(lines[14].text, '-%s' % + errors[1].rstrip().replace('\n', '\n-')) + self.assertEqual(lines[14].colour, col.GREEN) + + self.assertEqual(lines[15].text, 'w+%s' % + errors[2].rstrip().replace('\n', '\nw+')) + self.assertEqual(lines[15].colour, col.MAGENTA) + + self.assertEqual(lines[16].text, '05: %s' % commits[4][1]) + self.assertSummary(lines[17].text, 'sandbox', '+', ['board4']) + self.assertSummary(lines[18].text, 'powerpc', '', ['board3'], ok=True) + + # The second line of errors[3] is a duplicate, so buildman will drop it + expect = errors[3].rstrip().split('\n') + expect = [expect[0]] + expect[2:] + self.assertEqual(lines[19].text, '+%s' % + '\n'.join(expect).replace('\n', '\n+')) + + self.assertEqual(lines[20].text, 'w-%s' % + errors[2].rstrip().replace('\n', '\nw-')) + + self.assertEqual(lines[21].text, '06: %s' % commits[5][1]) + self.assertSummary(lines[22].text, 'sandbox', '', ['board4'], ok=True) + + # The second line of errors[3] is a duplicate, so buildman will drop it + expect = errors[3].rstrip().split('\n') + expect = [expect[0]] + expect[2:] + self.assertEqual(lines[23].text, '-%s' % + '\n'.join(expect).replace('\n', '\n-')) + + self.assertEqual(lines[24].text, 'w-%s' % + errors[0].rstrip().replace('\n', '\nw-')) + + self.assertEqual(len(lines), 25) def _testGit(self): """Test basic builder operation by building a branch"""