From patchwork Wed Nov 14 05:44:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Khalid Elmously X-Patchwork-Id: 997478 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42vtjw0gcJz9s9m; Wed, 14 Nov 2018 16:44:52 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1gMnyl-000158-Iw; Wed, 14 Nov 2018 05:44:43 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:128) (Exim 4.86_2) (envelope-from ) id 1gMnyj-00014n-Bn for kernel-team@lists.ubuntu.com; Wed, 14 Nov 2018 05:44:41 +0000 Received: from mail-qk1-f200.google.com ([209.85.222.200]) by youngberry.canonical.com with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.76) (envelope-from ) id 1gMnyi-0008Jj-Uz for kernel-team@lists.ubuntu.com; Wed, 14 Nov 2018 05:44:41 +0000 Received: by mail-qk1-f200.google.com with SMTP id z68so11242237qkb.14 for ; Tue, 13 Nov 2018 21:44:40 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=GAx48uP2Ol9tcU0hs+cqgx6jO23FLvSieZdAqlMh9NM=; b=Sz+Y2ELV1OHH2/ZVp5zRITLxImHW6WImJNi2Fq3LwhMlVUciFsg9EgqotmUgJzbCPs +56+ptxKXEKUP8VyYZq+KgdLOnxbcUcF4ELroaLanY4jxyI/oE419ZUQEaf6oR9Vdz/N hZjZzmPqUr6IfXPtepPpgeIoaJlc/gqqZMGllavOhmHLOTO6KlYRGZuM4XcNETEZpIdF hjpUvByxOUYMaoedcWnUWeP5MSVFqM/kkmdSFhYgN3+YRWs5xvQuL5xOOPT2BJgNecLA i/1cjplG1f+MsqWj7AYaDnWck9ABXZOY9XYF9cEoY88BSVJW0cR2C8/IRUxQawLtv5Rb uxpQ== X-Gm-Message-State: AGRZ1gIKwwhWA+FRLNcp6U9VuK0evqO+rwbB2QfmpSffbHvUgESvwxbi nOxOtfBORUCPbasYpNTzWuDDNIqMdbuDrMtnncn1uQX+yLQnX8ZTD9VuQEt83vnTsXzR54YgprJ f9tJoSNBKYtqObck4N/0cZESM8OUKz+vYH0AYdd8rqg== X-Received: by 2002:ac8:3181:: with SMTP id h1mr490426qte.14.1542174279342; Tue, 13 Nov 2018 21:44:39 -0800 (PST) X-Google-Smtp-Source: AJdET5eYyXPTESPcZUyMGkmVuW1I3NNy0ddr19cjKV5+4PtXj6TErKKcMv4jlC14/t5RMVeOXLfD3Q== X-Received: by 2002:ac8:3181:: with SMTP id h1mr490403qte.14.1542174278846; Tue, 13 Nov 2018 21:44:38 -0800 (PST) Received: from kbuntu.fuzzbuzz.org (198-16-164-194.on.cable.ebox.net. [198.16.164.194]) by smtp.gmail.com with ESMTPSA id h35sm1792587qth.59.2018.11.13.21.44.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 13 Nov 2018 21:44:38 -0800 (PST) From: Khalid Elmously To: kernel-team@lists.ubuntu.com Subject: [kteam-tools][PATCH 1/1] stable: Factor out (and reuse) trello-related code Date: Wed, 14 Nov 2018 00:44:31 -0500 Message-Id: <20181114054431.592-2-khalid.elmously@canonical.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181114054431.592-1-khalid.elmously@canonical.com> References: <20181114054431.592-1-khalid.elmously@canonical.com> X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" Move the Trello-related code from create-sru-cards.py to its own libtrello.py file. Modify start-sru-cycle to re-use some of that code to create Trello cards that correspond to the tracking bugs that it already currently creates. The 'cycle' option that start-sru-cycle accepts is now optional instead of mandatory. When it's omitted, the most recent cycle from info/sru-cycle.yaml is used. start-sru-cycle also accepts a new option now: '--spin X' . This can be used to specify the respin number (default is '1' so this option can be omitted for the first spin). Signed-off-by: Khalid Elmously --- ktl/kernel_series.py | 6 ++ stable/create-sru-cards.py | 97 ++----------------------------- stable/libtrello.py | 114 +++++++++++++++++++++++++++++++++++++ stable/start-sru-cycle | 79 +++++++++++++++++++------ 4 files changed, 187 insertions(+), 109 deletions(-) create mode 100644 stable/libtrello.py diff --git a/ktl/kernel_series.py b/ktl/kernel_series.py index 06f9d9da..fe4ce7f6 100644 --- a/ktl/kernel_series.py +++ b/ktl/kernel_series.py @@ -229,6 +229,9 @@ class KernelSourceEntry: def __ne__(self, other): return not self.__eq__(other) + def __hash__(self): + return hash((self._name, self._series)) + @property def name(self): return self._name @@ -428,6 +431,9 @@ class KernelSeriesEntry: def __ne__(self, other): return not self.__eq__(other) + def __hash__(self): + return hash((self._name)) + @property def name(self): return self._name diff --git a/stable/create-sru-cards.py b/stable/create-sru-cards.py index 472f71f4..069aa218 100755 --- a/stable/create-sru-cards.py +++ b/stable/create-sru-cards.py @@ -6,7 +6,7 @@ import re from datetime import datetime, timedelta # the so-trello root dir needs to be on PYTHONPATH -from trellotool.trellotool import TrelloTool +from libtrello import TrelloBoard from ktl.kernel_series import KernelSeries from ktl.utils import run_command @@ -14,88 +14,6 @@ class TrelloError(Exception): pass -class SRUBoard: - def __init__(self, tt, name): - """ - :param tt: TrelloTool object - :param name: name of the board - """ - self.tt = tt - self.name = name - self.id = None - self.url = None - self.default_list_name = None - self.default_list_id = None - - def create(self, org): - """ - Create the Trello board - - :param org: Trello organization (team) - :return None - """ - params = { - 'name': self.name, - 'idOrganization': org, - 'prefs_permissionLevel': 'org', - 'defaultLists': 'false' - } - - self.tt.trello.board_create(**params) - self.lookup_board_id() - - def lookup_board_id(self): - for board in self.tt.trello.member_boards('me'): - if board['name'] == self.name: - self.id = board['id'] - self.url = board['url'] - return - raise TrelloError("Could not find id for board '%s'" % self.name) - - def add_lists(self, list_names, default_list): - """ - Add the lists to the board and save the default list - to be used to later add the cards to - - :param list_names: list with the list names - :param default_list: default list to add the cards to - :return: None - """ - for list_name in list_names: - params = { - 'name': list_name, - 'pos': 'bottom' - } - self.tt.trello.board_addlist(self.id, **params) - - self.default_list_name = default_list - self.lookup_list_id(default_list) - - def lookup_list_id(self, list_name): - for board_list in self.tt.trello.board_lists(self.id): - if board_list['name'] == list_name: - self.default_list_id = board_list['id'] - return - raise TrelloError("Could not find id for list '%s'" % list_name) - - def add_card(self, name, desc=None): - """ - Add the given card to the default list board - - :param name: card name - :param desc: card description - :return: None - """ - params = { - 'name': name, - 'pos': 'bottom', - } - if desc: - params['desc'] = desc - - self.tt.trello.list_addcard(self.default_list_id, **params) - - class SRUCardsCreator: def __init__(self, args): """ @@ -108,9 +26,6 @@ class SRUCardsCreator: self.config = {} self.config_load() - self.tt = TrelloTool() - self.tt.assert_authenticated() - def config_load(self): with open(self.config_file, 'r') as cfd: self.config = yaml.safe_load(cfd) @@ -144,10 +59,10 @@ class SRUCardsCreator: :return: None """ # create the board with the lists on the organization - board = SRUBoard(self.tt, self.config['board']['prefix_name'] + self.cycle) - print('Create board: %s' % (self.config['board']['prefix_name'] + self.cycle)) + board = TrelloBoard() if not self.args.dry_run: - board.create(self.config['board']['trello_organization']) + print('Create board: %s' % (self.config['board']['prefix_name'] + self.cycle)) + board.create(self.config['board']['trello_organization'], self.config['board']['prefix_name'] + self.cycle) board.add_lists(self.config['board']['lists'], self.config['board']['default_list']) # Cache the tuples (series, source) of supported sources @@ -284,8 +199,8 @@ def update_cycle_info(args): if __name__ == '__main__': retval = 0 - default_config = '%s/create-sru-cards.yaml' % os.path.dirname(__file__) - default_cycle_info = '%s/../info/sru-cycle.yaml' % os.path.dirname(__file__) + default_config = TrelloBoard.default_config + default_cycle_info = TrelloBoard.default_cycle_info description = 'Create a Trello board with cards for SRU cycles' epilog = ''' The script reads the configuration from a yaml file, updates the sru-cycle info diff --git a/stable/libtrello.py b/stable/libtrello.py new file mode 100644 index 00000000..02a332a2 --- /dev/null +++ b/stable/libtrello.py @@ -0,0 +1,114 @@ +import os +import yaml +from trellotool.trellotool import TrelloTool + +class TrelloError(Exception): + pass + +class TrelloBoard: + default_config = '%s/create-sru-cards.yaml' % os.path.dirname(__file__) + default_cycle_info = '%s/../info/sru-cycle.yaml' % os.path.dirname(__file__) + + def __init__(self): + """ + :param tt: TrelloTool object + :param name: name of the board + """ + self.tt = TrelloTool() + self.id = None + self.url = None + self.default_list_name = None + self.default_list_id = None + self.load_config() + + def load_cycle_info(self): + with open(self.default_cycle_info, 'r') as cfd: + self.cycle_info = yaml.safe_load(cfd) + + def load_config(self): + with open(self.default_config, 'r') as cfd2: + self.config = yaml.safe_load(cfd2) + + def create(self, org, name): + """ + Create the Trello board + + :param org: Trello organization (team) + :return None + """ + params = { + 'name': name, + 'idOrganization': org, + 'prefs_permissionLevel': 'org', + 'defaultLists': 'false' + } + + self.tt.trello.board_create(**params) + self.lookup_board_id(name) + + def lookup_board_id(self, name): + for board in self.tt.trello.member_boards('me'): + if board['name'] == name: + self.id = board['id'] + self.url = board['url'] + self.name = board['name'] + return + raise TrelloError("Could not find id for board '%s'" % name) + + def get_current_cycle(self): + with open(self.default_cycle_info, "r") as f: + for line in f: + # print ('length of line is %s' %(len(line))) + if line[0] != "#" and len(line) != 1: + # this should be the first non commented, non-empty line of the file + return (line[1:-3]) + + def add_lists(self, list_names, default_list): + """ + Add the lists to the board and save the default list + to be used to later add the cards to + + :param list_names: list with the list names + :param default_list: default list to add the cards to + :return: None + """ + for list_name in list_names: + params = { + 'name': list_name, + 'pos': 'bottom' + } + self.tt.trello.board_addlist(self.id, **params) + + self.default_list_name = default_list + self.lookup_list_id(default_list) + + def lookup_list_id(self, list_name): + for board_list in self.tt.trello.board_lists(self.id): + if board_list['name'] == list_name: + self.default_list_id = board_list['id'] + return + raise TrelloError("Could not find id for list '%s'" % list_name) + + def set_default_list(self, list_name): + for board_list in self.tt.trello.board_lists(self.id): + if board_list['name'] == list_name: + self.default_list_id = board_list['id'] + return + raise TrelloError("Could not find id for list '%s'" % list_name) + + def add_card(self, name, desc=None): + """ + Add the given card to the default list board + + :param name: card name + :param desc: card description + :return: None + """ + params = { + 'name': name, + 'pos': 'bottom', + } + if desc: + params['desc'] = desc + + self.tt.trello.list_addcard(self.default_list_id, **params) diff --git a/stable/start-sru-cycle b/stable/start-sru-cycle index 7e7a2626..d2445709 100755 --- a/stable/start-sru-cycle +++ b/stable/start-sru-cycle @@ -6,6 +6,7 @@ import os sys.path.append(os.path.realpath(os.path.join(os.path.dirname(__file__), '..', 'py3'))) +from libtrello import TrelloBoard from argparse import ArgumentParser, RawDescriptionHelpFormatter from ktl.kernel_series import KernelSeries from ktl.tracking_bug import TrackingBug @@ -77,10 +78,10 @@ class Crankers(): origin = s.source_get_origin(source) if not origin: if s.series_wanted(series): - retval.setdefault((series.codename, source.name), []) + retval.setdefault((series, source), []) else: if s.series_wanted(origin.series): - retval_series = retval.setdefault((origin.series.codename, origin.name), []) + retval_series = retval.setdefault((origin.series, origin), []) retval_series.append((series.codename, source.name)) return retval @@ -122,19 +123,42 @@ class Crankers(): retval = 1 try: data = s.tracked_packages() - cycle_tag = 'kernel-sru-cycle-' + s.args.cycle + board = TrelloBoard() - for (series, master_package) in sorted(data): - print('%s' % series) - print(' ' + master_package) + if s.args.cycle: + cycle_string = s.args.cycle + else: + cycle_string = board.get_current_cycle() + + print('Using SRU cycle: %s'%(cycle_string)) + cycle_tag = 'kernel-sru-cycle-' + cycle_string + '-' + str(s.args.spin) + board.lookup_board_id(board.config['board']['prefix_name'] + cycle_string) + + board.set_default_list('Backlog') + + for (series, master_package) in data: + + print(' %s:%s' % (series.codename, master_package.name)) - print(' %s:%s' % (series, master_package)) if not s.args.dryrun: if s.args.usemasterbug: master_bug = s.tb.get_bug(s.args.usemasterbug[0]) print("Using pre-existing master-bug: %s" %(master_bug.id)) else: - master_bug = s.tb.open(master_package, '', True, None, series) + master_bug = s.tb.open(master_package.name, '', True, None, series.codename) + if s.args.spin > 1: + card_name = 'Respin #' + str(s.args.spin) + ': ' + else: + card_name = '' + card_name += 'Crank %s/%s' % (series.codename, master_package.name) + + card_desc = 'https://bugs.launchpad.net/bugs/' + str(master_bug.id) + if series.esm: + card_desc += '\nESM mode: Note different git location and build PPA' + + print(' Adding trello card: %s' % (card_name)) + board.add_card(card_name, card_desc) + print(' ----\n') for task in master_bug.tasks: # Move the primary package to Confirmed, link-to-tracker and @@ -154,19 +178,37 @@ class Crankers(): print(' -> EE: Bug creation failed (does the package exist in the archive?)!') continue bug.tags.append(cycle_tag) - if series == derivative_series: + if series.codename == derivative_series: derivatives.append('bug {} ({})'.format(str(bug.id), derivative_package)) bug.tags.append('kernel-sru-derivative-of-%s' % master_bug.id) else: backports.append('bug {} ({})'.format(str(bug.id), derivative_package)) bug.tags.append('kernel-sru-backport-of-%s' % master_bug.id) - if not s.args.dryrun: - description = master_bug.description - description += '\n' - description += 'backports: %s\n' % ', '.join(backports) - description += 'derivatives: %s\n' % ', '.join(derivatives) - master_bug.description = description + if s.args.spin > 1: + card_name = 'Respin #' + str(s.args.spin) + ': ' + else: + card_name = '' + card_name += 'Crank ' + derivative_series + '/' + derivative_package + + card_desc = 'https://bugs.launchpad.net/bugs/' + str(bug.id) + if series.esm: + card_desc += '\nESM mode: Note different git location and build PPA' + if derivative_package == 'linux-euclid': + card_desc += '\nNo rebase to be done. Only needed if there are high and critical CVEs to be fixed.' + + print(' Adding trello card: %s' % (card_name)) + board.add_card(card_name, card_desc) + print(' ----\n') + + + + if not s.args.dryrun: + description = master_bug.description + description += '\n' + description += 'backports: %s\n' % ', '.join(backports) + description += 'derivatives: %s\n' % ', '.join(derivatives) + master_bug.description = description retval = 0 @@ -191,17 +233,18 @@ is being re-spun. app_epilog = ''' Examples: - start-sru-cycle 2018.01.01-1 - start-sru-cycle 2018.02.20-3 --series artful + start-sru-cycle 2018.01.01 + start-sru-cycle 2018.02.20 --series artful --spin 3 ''' parser = ArgumentParser(description=app_description, epilog=app_epilog, formatter_class=RawDescriptionHelpFormatter) - parser.add_argument('cycle', metavar='cycle', help='The sru cycle tag to be applied to the bugs that are created. The format is YYYY.MM.DD-# where the final "-#" is the spin number. If this is the initial start of a new SRU cycle the spin number will be 1 and is incremented for every re-spin. (e.g. \'2017.04.04-1\').') + parser.add_argument('cycle', nargs='?', help='The sru cycle tag to be applied to the bugs that are created. The format is YYYY.MM.DD (e.g. \'2017.04.04\'). This must match the title of a Trello SRU board for the Trello cards to be created on that board. If this option isn\'t specified, the most recent cycle from info/sru-cycle.yaml will be used.') parser.add_argument('--series', action='append', default=[], metavar='series', help='Only creates tracking bugs for the specified series. This includes the main kernel and all of it\'s derivatives. Can be used multiple times.') parser.add_argument('--source', action='append', default=[], metavar='source', help='Only create tracking bugs for the specified source package(s). Requiers exactly one series argument to be given. This option can be used multiple times.') parser.add_argument('--staging', action='store_true', default=False, help='Use the staging LP server to create the bug. This is just for testing and will go away when the staging database is reset.') parser.add_argument('--dry-run', '--dryrun', action='store_true', default=False, help='Make no permanent changes.', dest='dryrun') parser.add_argument('--usemasterbug', type=int, nargs=1, help='Use pre-existing master-bug instead of creating a new one') + parser.add_argument('--spin', type=int, help='Specify the respin number. Default is 1', default=1) args = parser.parse_args()