From patchwork Thu Nov 22 00:30:10 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Martin X-Patchwork-Id: 200921 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from hemlock.osuosl.org (hemlock.osuosl.org [140.211.166.133]) by ozlabs.org (Postfix) with ESMTP id 960CE2C0080 for ; Thu, 22 Nov 2012 11:30:28 +1100 (EST) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id C866EA01A3; Thu, 22 Nov 2012 00:30:26 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aOr22sJZAY0L; Thu, 22 Nov 2012 00:30:24 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id 46A63A0098; Thu, 22 Nov 2012 00:30:24 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (whitealder.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 04FD38F74A for ; Thu, 22 Nov 2012 00:30:28 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id A20558A349 for ; Thu, 22 Nov 2012 00:30:22 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UqdI4BN3rw71 for ; Thu, 22 Nov 2012 00:30:21 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wg0-f53.google.com (mail-wg0-f53.google.com [74.125.82.53]) by whitealder.osuosl.org (Postfix) with ESMTPS id 4141081BA4 for ; Thu, 22 Nov 2012 00:30:21 +0000 (UTC) Received: by mail-wg0-f53.google.com with SMTP id e12so1166804wge.10 for ; Wed, 21 Nov 2012 16:30:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id:x-mailer; bh=d2TuFbS5TJhr1t1HWfsZeOLovn+MQLQiieiQtjGrXKw=; b=r/tJ9ce/24+ZBctYyN0jFiXySrhI0/NC9pSVSpoomvan3FaryEfF4p+y9+wPejEYNY cA9QDysIzu++XxkgDSgPNl9iAcfs8HjuxwwrG+RfpFu0TSRis0GzJ1w6NoxeJVLZdSUu nvil1frYo2qJxeOSyxVZUuFETyXJdoHniTw2N3jRW9BAglmTFtJ9V5fRVANf98bcZ5h4 gSirHVVoewLsFGnTM8/nO1E3ZeRRq8L1rroOxSLA+tWliYvaPWieiJj8vSdhiE9td8+3 cKAPqs7Hu5VLFilMFYmqAeaAzfeueARKq543i9vuHqsPafBjKQj5rApsu/lyK5wRl3pI TeCg== Received: by 10.180.94.169 with SMTP id dd9mr1799426wib.14.1353544220302; Wed, 21 Nov 2012 16:30:20 -0800 (PST) Received: from localhost.localdomain (ivr94-4-82-229-165-48.fbx.proxad.net. [82.229.165.48]) by mx.google.com with ESMTPS id eu8sm1696268wib.1.2012.11.21.16.30.18 (version=SSLv3 cipher=OTHER); Wed, 21 Nov 2012 16:30:19 -0800 (PST) From: Samuel Martin To: buildroot@busybox.net Date: Thu, 22 Nov 2012 01:30:10 +0100 Message-Id: <1353544211-9308-1-git-send-email-s.martin49@gmail.com> X-Mailer: git-send-email 1.8.0 Subject: [Buildroot] [PATCH 1/2] manual: add support to autogenerate *-list.txt X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: buildroot-bounces@busybox.net * autogenerate the package-list.txt file without any git command * add script searching for deprecated stuff and generating the doc (support/script/deprecated.py) Signed-off-by: Samuel Martin --- docs/manual/manual.mk | 34 +++++++++++- support/scripts/deprecated.py | 117 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 150 insertions(+), 1 deletion(-) create mode 100755 support/scripts/deprecated.py diff --git a/docs/manual/manual.mk b/docs/manual/manual.mk index d664603..1daddb7 100644 --- a/docs/manual/manual.mk +++ b/docs/manual/manual.mk @@ -18,12 +18,44 @@ $(1): $(1)-$(3) $(1)-$(3): $$(O)/docs/$(1)/$(1).$(4) $$(O)/docs/$(1)/$(1).$(4): docs/$(1)/$(1).txt $$($(call UPPERCASE,$(1))_SOURCES) - @echo "Generating $(5) $(1)..." + @$(call MESSAGE,"Generating $(5) $(1)...") $(Q)mkdir -p $$(@D) $(Q)a2x $(6) -f $(2) -d book -L -r $(TOPDIR)/docs/images \ -D $$(@D) $$< endef +clean-package-lists: + -rm -f $(TOPDIR)/docs/manual/package-list.txt + -rm -f $(TOPDIR)/docs/manual/deprecated-list.txt + +$(TOPDIR)/docs/manual/package-list.txt: + @$(call MESSAGE,"Generating package list...") + @echo -en "\ + //\n\ + // Autogenerated file\n\ + //\n\n\ + [[package-list]]\n\ + Available packages\n\ + ------------------\n\n\ + // docs/manaual/pkg-list.txt is generated using the following command:\n\ + // $ git grep -E '\\((autotools|cmake|generic)-package\\)' package/ | \\\n\ + // cut -d':' -f1 | grep '\\.mk$$' | \\\n\ + // sed -e 's;.*\\?/\\(.*\\?\\).mk;* \\1;' | \\\n\ + // sort > docs/manual/pkg-list.txt\n\n\ + " > $@ + grep -rHE --color=never '\((autotools|cmake|generic)-package\)' \ + $(TOPDIR)/package/ | \ + cut -d':' -f1 | grep '\.mk$$' | \ + sed -e 's;.*\?/\(.*\?\).mk;* \1;' | \ + sort >> $@ + +$(TOPDIR)/docs/manual/deprecated-list.txt: + @$(call MESSAGE,"Generating deprecated list...") + python2 $(TOPDIR)/support/scripts/deprecated.py > $@ + +generate-doc-lists: clean-package-lists $(TOPDIR)/docs/manual/package-list.txt \ + $(TOPDIR)/docs/manual/deprecated-list.txt + ################################################################################ # GENDOC -- generates the make targets needed to build asciidoc documentation. # diff --git a/support/scripts/deprecated.py b/support/scripts/deprecated.py new file mode 100755 index 0000000..491fddf --- /dev/null +++ b/support/scripts/deprecated.py @@ -0,0 +1,117 @@ +#!/usr/bin/env python +## +## deprecated-packages.py +## +## Author(s): +## - Samuel MARTIN +## +## Copyright (C) 2012 Samuel MARTIN +## + +# Python 2.7 script searching for kconfig symbols depending on 'BR2_DEPRECATED' +# and generating (printing to the standard output) the manual file in asciidoc. + +import os +import re +import sys + + +NOT_SEARCHED = ('.git', 'configs', 'output', 'support') + +DEPR_SYMBOL = "BR2_DEPRECATED" + +_REGEX = r"config BR2_(.*?)\n" + \ + "((.*?(?!config)(prompt|bool|string|int) \"(.*?)\".*?|[^\n]+)\n)*" + \ + "(.*?(?!config )" + DEPR_SYMBOL + ".*?)\n" + \ + "((.*?(?!config)(prompt|bool|string|int) \"(.*?)\".*?|[^\n]+)\n)*" + +REGEX = re.compile(_REGEX, flags=re.MULTILINE) + +def get_dir_list(): + root = os.path.join(os.path.dirname(__file__), "..", "..") + root = os.path.abspath(root) + dirs = { 'buildroot': (root, False) } + for dir_ in os.listdir(root): + if dir_ in NOT_SEARCHED: + continue + dir__ = os.path.join(root, dir_) + if not os.path.isdir(dir__): + continue + dirs[dir_] = (dir__, True) + return dirs + +def find_deprecated(root, recursive): + deprecated = list() + for root_, _, files_ in os.walk(root): + if not recursive and root_ != root: + break + for file_ in files_: + if not file_.startswith("Config.in"): + continue + with open(os.path.join(root_, file_), "r") as f: + content = f.read() + if not DEPR_SYMBOL in content: + continue + found = REGEX.findall(content) + if found: + deprecated += found + return deprecated + +def find_all_deprecated(dirs): + deprecated = dict() + for key, search in dirs.iteritems(): + root_path, recursive_search = search + deprecated[key] = find_deprecated(root_path, recursive_search) + return deprecated + +def generate_asciidoc(deprecated): + for cat, matches in deprecated.iteritems(): + for i, match in enumerate(matches): + name = match[0].lower().replace("_", " ") + name = re.sub("^package ", "", name) + vers = re.sub(".*?(version )?([0-9].*)", r'\2', name) + if vers: + vers = re.sub(" ", ".", vers) + name = re.sub("(version )?([0-9].*)", vers, name) + symbol = match[4] + if not symbol: + symbol = match[9] + matches[i] = "\n** %-25s +[%s]+" % (name, symbol) + matches.sort() + deprecated[cat] = matches + output = """\ +// +// Autogenerated file +// + +[[deprecated]] +Deprecated list +--------------- + +The following stuff are marked as _deprecated_ in Buildroot due to +their status either too old or unmaintained. + +// Please check and sort by grepping the source running: +// +// $ git grep -EB4 'depends on BR2_DEPRECATED' +// +// and: +// +// $ git grep -EB4 'depends on BR2_DEPRECATED' | \\ +// grep -Eo '(:|-).*?(config|comment) BR2_.*' +""" + for cat, matches in deprecated.iteritems(): + if not matches: + continue + output += "\n\n* %s:\n" % cat.capitalize() + output += "".join(matches) + return output + +def main(): + search_dirs = get_dir_list() + depr_lists = find_all_deprecated(search_dirs) + output = generate_asciidoc(depr_lists) + print output + +if __name__ == "__main__": + main()