Patchwork [U-Boot] RFC: auto-generate ARM mach-types.h file from ARM machine database

login
register
mail settings
Submitter Michael Schwingen
Date May 5, 2011, 9:48 p.m.
Message ID <20110505214836.GA5313@discworld.dascon.de>
Download mbox | patch
Permalink /patch/94321/
State RFC
Delegated to: Albert ARIBAUD
Headers show

Comments

Michael Schwingen - May 5, 2011, 9:48 p.m.
Hi,

to conclude the discussion in the thread "Re: [U-Boot] Update and Cut down
mach types", I tried a short patch that demonstrates how to automatically
generate the mach-types.h file from a database dump (from
http://www.arm.linux.org.uk/developer/machines/?action=new).

This has multiple advantages:
 - pulling in new machine types is easier (drop in a new downloaded database
   dump), and produces a much smaller diff.
 - adding new machines is decoupled from the time they appear in Linux.
 - boards that are not in mainline Linux will not be break due to removal of
   their mach types from the Linux headers.

The AWK and Makefile fragment script is taken verbatim from Linux 2.6.38.3. 
I think the AWK script is simple enough that it will not require big
maintenance efforts (unless the machine database format changes).

The patch is edited down - I removed the diff for the deletion of the old
mach-types.h file, and shortened the new mach-types file to a few entries
just to show the concept - otherwise, the patch would be much too big for
the list.

Is this an acceptable solution? Should I go on and produce a full-fledged
patch?

cu
Michael


From 2cb8bfd1b387a4a49d9e0cebd96824c879000420 Mon Sep 17 00:00:00 2001
From: Michael Schwingen <michael@schwingen.org>
Date: Thu, 5 May 2011 23:04:00 +0200
Subject: [ARM: auto-generate mach-types.h 1/1] auto-generate mach-types.h include file from ARM machine database dump


Signed-off-by: Michael Schwingen <michael@schwingen.org>
based directly on Makefile/script from Linux-2.8.38.3

---
 Makefile                          |    9 +-
 arch/arm/include/asm/mach-types.h |42924 -------------------------------------
 arch/arm/tools/gen-mach-types     |   72 +
 arch/arm/tools/mach-types         | 3448 +++
 4 files changed, 3527 insertions(+), 42926 deletions(-)
 delete mode 100644 arch/arm/include/asm/mach-types.h
 create mode 100644 arch/arm/tools/gen-mach-types
 create mode 100644 arch/arm/tools/mach-types

1.7.2.5
Mike Frysinger - May 6, 2011, 5:09 a.m.
On Thu, May 5, 2011 at 17:48, Michael Schwingen wrote:
> --- a/Makefile
> +++ b/Makefile
> @@ -469,7 +469,7 @@ $(obj)System.map:   $(obj)u-boot
>  # This target actually generates 2 files; autoconf.mk and autoconf.mk.dep.
>  # the dep file is only include in this top level makefile to determine when
>  # to regenerate the autoconf.mk file.
> -$(obj)include/autoconf.mk.dep: $(obj)include/config.h include/common.h
> +$(obj)include/autoconf.mk.dep: $(obj)include/config.h include/common.h $(obj)include/asm/mach-types.h
>        @$(XECHO) Generating $@ ; \
>        set -e ; \
>        : Generate the dependancies ; \
> @@ -530,13 +530,18 @@ unconfig:
>                $(obj)board/*/config.tmp $(obj)board/*/*/config.tmp \
>                $(obj)include/autoconf.mk $(obj)include/autoconf.mk.dep
>
> -%_config::     unconfig
> +%_config::     unconfig $(obj)include/asm/mach-types.h
>        @$(MKCONFIG) -A $(@:_config=)
>
>  sinclude $(obj).boards.depend
>  $(obj).boards.depend:  boards.cfg
>        awk '(NF && $$1 !~ /^#/) { print $$1 ": " $$1 "_config; $$(MAKE)" }' $< > $@
>
> +
> +$(obj)include/asm/mach-types.h: arch/arm/tools/gen-mach-types arch/arm/tools/mach-types
> +       @mkdir -p $(obj)include/asm
> +       awk -f $^ > $@ || { rm -f $@; /bin/false; }
> +
>  #
>  # Functions to generate common board directory names
>  #

this all belongs in arch/arm/config.mk and not the toplevel makefile

also, dont hardcode full paths to things.  there's no reason for it.

might want to add an "update-mach-types" target so people can type
`make update-mach-types` and it'll automatically wget the right file
to the right place ...
-mike
Wolfgang Denk - May 6, 2011, 10:14 a.m.
Dear Michael Schwingen,

In message <20110505214836.GA5313@discworld.dascon.de> you wrote:
> 
> to conclude the discussion in the thread "Re: [U-Boot] Update and Cut down
> mach types", I tried a short patch that demonstrates how to automatically
> generate the mach-types.h file from a database dump (from
> http://www.arm.linux.org.uk/developer/machines/?action=new).
> 
> This has multiple advantages:
>  - pulling in new machine types is easier (drop in a new downloaded database
>    dump), and produces a much smaller diff.
>  - adding new machines is decoupled from the time they appear in Linux.
>  - boards that are not in mainline Linux will not be break due to removal of
>    their mach types from the Linux headers.
> 
> The AWK and Makefile fragment script is taken verbatim from Linux 2.6.38.3. 
> I think the AWK script is simple enough that it will not require big
> maintenance efforts (unless the machine database format changes).
> 
> The patch is edited down - I removed the diff for the deletion of the old
> mach-types.h file, and shortened the new mach-types file to a few entries
> just to show the concept - otherwise, the patch would be much too big for
> the list.
> 
> Is this an acceptable solution? Should I go on and produce a full-fledged
> patch?

I agree with the comments alreay made by Mike Frysinger.

In addition, I would like to point out that I consider it mandatory
thatthe normal build process does NOT involve any network download.
Instead, it must be run completely based on files checked into the git
repository only.

That means, that the "update-mach-types" target Mike mentions is
something that the ARM maintainer can use to update the in-tree copy
of the mach-types.h file, which then gets commited into the git
repository.  Also users can run this if they want to use the latest
version, but it MUST NOT be used for a regular build.

Best regards,

Wolfgang Denk
Michael Schwingen - May 7, 2011, 10:57 a.m.
Am 05/06/2011 07:09 AM, schrieb Mike Frysinger:
> On Thu, May 5, 2011 at 17:48, Michael Schwingen wrote:
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -469,7 +469,7 @@ $(obj)System.map:   $(obj)u-boot
>>  # This target actually generates 2 files; autoconf.mk and autoconf.mk.dep.
>>  # the dep file is only include in this top level makefile to determine when
>>  # to regenerate the autoconf.mk file.
>> -$(obj)include/autoconf.mk.dep: $(obj)include/config.h include/common.h
>> +$(obj)include/autoconf.mk.dep: $(obj)include/config.h include/common.h $(obj)include/asm/mach-types.h
>>        @$(XECHO) Generating $@ ; \
>>        set -e ; \
>>        : Generate the dependancies ; \
>> @@ -530,13 +530,18 @@ unconfig:
>>                $(obj)board/*/config.tmp $(obj)board/*/*/config.tmp \
>>                $(obj)include/autoconf.mk $(obj)include/autoconf.mk.dep
>>
>> -%_config::     unconfig
>> +%_config::     unconfig $(obj)include/asm/mach-types.h
>>        @$(MKCONFIG) -A $(@:_config=)
>>
>>  sinclude $(obj).boards.depend
>>  $(obj).boards.depend:  boards.cfg
>>        awk '(NF && $$1 !~ /^#/) { print $$1 ": " $$1 "_config; $$(MAKE)" }' $< > $@
>>
>> +
>> +$(obj)include/asm/mach-types.h: arch/arm/tools/gen-mach-types arch/arm/tools/mach-types
>> +       @mkdir -p $(obj)include/asm
>> +       awk -f $^ > $@ || { rm -f $@; /bin/false; }
>> +
>>  #
>>  # Functions to generate common board directory names
>>  #
> this all belongs in arch/arm/config.mk and not the toplevel makefile
Fine with me, however, I am hitting a big problem with the Makefile
structure:

mach-types.h needs to be built before autoconf.mk can be generated, and
the rules for autoconf.mk are in the top-level Makefile.

If I put rules in arch/arm/config.mk, then the first of these rules
becomes the default rule which is executed in subdir makes (like "make
-C arch"), which breaks compilation completely, since config.mk is
included before the rules are defined in the subdir Makefiles.

It seems the current scheme allows only variable definitions in
config.mk files, which is not sufficient here.

rules.mk would be fine, however, there is no provision to include rules
from lower directories, since all the building in subdirectories is
handled by recursively calling make (this is one of the problems that
arise by recursively calling make for each directory, but that is a
different topic).

Any ideas? Using the current Makefile structure, I see no other solution
than defining the mach-types.h generation rules in the toplevel Makefile.

> also, dont hardcode full paths to things.  there's no reason for it.
Which of these can be omitted?

When putting the rules in arch/arm/config.mk, I can make them trigger
only when compiling for ARM, and I can use $(ARCH) instead of  arm/, but
apart from that, I think I need to specify the paths, no?

> might want to add an "update-mach-types" target so people can type
> `make update-mach-types` and it'll automatically wget the right file
> to the right place ...
Good idea.
I agree with Wolfgang that this is intended to be used by the maintainer
mainly, so the mach-types file should be included in the source so that
a normal user does not need to download the file.

cu
Michael
Albert ARIBAUD - May 7, 2011, 10:58 a.m.
Hi Wolfgang,

Le 06/05/2011 12:14, Wolfgang Denk a écrit :
> Dear Michael Schwingen,
>
> In message<20110505214836.GA5313@discworld.dascon.de>  you wrote:
>>
>> to conclude the discussion in the thread "Re: [U-Boot] Update and Cut down
>> mach types", I tried a short patch that demonstrates how to automatically
>> generate the mach-types.h file from a database dump (from
>> http://www.arm.linux.org.uk/developer/machines/?action=new).
>>
>> This has multiple advantages:
>>   - pulling in new machine types is easier (drop in a new downloaded database
>>     dump), and produces a much smaller diff.
>>   - adding new machines is decoupled from the time they appear in Linux.
>>   - boards that are not in mainline Linux will not be break due to removal of
>>     their mach types from the Linux headers.
>>
>> The AWK and Makefile fragment script is taken verbatim from Linux 2.6.38.3.
>> I think the AWK script is simple enough that it will not require big
>> maintenance efforts (unless the machine database format changes).
>>
>> The patch is edited down - I removed the diff for the deletion of the old
>> mach-types.h file, and shortened the new mach-types file to a few entries
>> just to show the concept - otherwise, the patch would be much too big for
>> the list.
>>
>> Is this an acceptable solution? Should I go on and produce a full-fledged
>> patch?
>
> I agree with the comments alreay made by Mike Frysinger.
>
> In addition, I would like to point out that I consider it mandatory
> thatthe normal build process does NOT involve any network download.
> Instead, it must be run completely based on files checked into the git
> repository only.
>
> That means, that the "update-mach-types" target Mike mentions is
> something that the ARM maintainer can use to update the in-tree copy
> of the mach-types.h file, which then gets commited into the git
> repository.  Also users can run this if they want to use the latest
> version, but it MUST NOT be used for a regular build.

Agreed on all points.

> Best regards,
>
> Wolfgang Denk

Amicalement,
Wolfgang Denk - May 9, 2011, 5:37 a.m.
Dear Michael Schwingen,

In message <4DC5259C.7040208@discworld.dascon.de> you wrote:
>
> If I put rules in arch/arm/config.mk, then the first of these rules
> becomes the default rule which is executed in subdir makes (like "make
> -C arch"), which breaks compilation completely, since config.mk is
> included before the rules are defined in the subdir Makefiles.

You are not suppoed to put any make rules in config.mk files.

> It seems the current scheme allows only variable definitions in
> config.mk files, which is not sufficient here.

As the name implies, these files contain configuration (= variable)
settings. Nothing else.

> rules.mk would be fine, however, there is no provision to include rules
> from lower directories, since all the building in subdirectories is
> handled by recursively calling make (this is one of the problems that
> arise by recursively calling make for each directory, but that is a
> different topic).

But lower level directories inherit all settings from the top level
Makefile?

Best regards,

Wolfgang Denk
Michael Schwingen - May 9, 2011, 6:32 a.m.
Am 05/09/2011 07:37 AM, schrieb Wolfgang Denk:
> Dear Michael Schwingen,
>
> In message <4DC5259C.7040208@discworld.dascon.de> you wrote:
>> If I put rules in arch/arm/config.mk, then the first of these rules
>> becomes the default rule which is executed in subdir makes (like "make
>> -C arch"), which breaks compilation completely, since config.mk is
>> included before the rules are defined in the subdir Makefiles.
> You are not suppoed to put any make rules in config.mk files.
>
>> It seems the current scheme allows only variable definitions in
>> config.mk files, which is not sufficient here.
> As the name implies, these files contain configuration (= variable)
> settings. Nothing else.
>
>> rules.mk would be fine, however, there is no provision to include rules
>> from lower directories, since all the building in subdirectories is
>> handled by recursively calling make (this is one of the problems that
>> arise by recursively calling make for each directory, but that is a
>> different topic).
> But lower level directories inherit all settings from the top level
> Makefile?
Yes.
However, the rule to generate autoconf.mk is in the toplevel Makefile,
and I need the rule to generate mach-types.h at the same level, as
autoconf.mk depends on it. The lower-level *Makefiles* are executed too
late to generate anything that is needed for autoconf.mk.

cu
Michael
Mike Frysinger - May 9, 2011, 3:29 p.m.
On Saturday, May 07, 2011 06:57:32 Michael Schwingen wrote:
> mach-types.h needs to be built before autoconf.mk can be generated, and
> the rules for autoconf.mk are in the top-level Makefile.
> 
> If I put rules in arch/arm/config.mk, then the first of these rules
> becomes the default rule which is executed in subdir makes (like "make
> -C arch"), which breaks compilation completely, since config.mk is
> included before the rules are defined in the subdir Makefiles.
> 
> It seems the current scheme allows only variable definitions in
> config.mk files, which is not sufficient here.
> 
> rules.mk would be fine, however, there is no provision to include rules
> from lower directories, since all the building in subdirectories is
> handled by recursively calling make (this is one of the problems that
> arise by recursively calling make for each directory, but that is a
> different topic).
> 
> Any ideas? Using the current Makefile structure, I see no other solution
> than defining the mach-types.h generation rules in the toplevel Makefile.

do you need the mach-types file for anything else ?  if not, dont keep that in 
git, keep the generated header.

> > also, dont hardcode full paths to things.  there's no reason for it.
> 
> Which of these can be omitted?

look for "/bin/" or "/sbin/".  if your patch contains those, it is wrong.

> > might want to add an "update-mach-types" target so people can type
> > `make update-mach-types` and it'll automatically wget the right file
> > to the right place ...
> 
> Good idea.
> I agree with Wolfgang that this is intended to be used by the maintainer
> mainly, so the mach-types file should be included in the source so that
> a normal user does not need to download the file.

i didnt suggest this in the first place
-mike
Michael Schwingen - May 9, 2011, 3:54 p.m.
Mike Frysinger wrote:
> On Saturday, May 07, 2011 06:57:32 Michael Schwingen wrote:
>   
>> mach-types.h needs to be built before autoconf.mk can be generated, and
>> the rules for autoconf.mk are in the top-level Makefile.
>>
>> If I put rules in arch/arm/config.mk, then the first of these rules
>> becomes the default rule which is executed in subdir makes (like "make
>> -C arch"), which breaks compilation completely, since config.mk is
>> included before the rules are defined in the subdir Makefiles.
>>
>> It seems the current scheme allows only variable definitions in
>> config.mk files, which is not sufficient here.
>>
>> rules.mk would be fine, however, there is no provision to include rules
>> from lower directories, since all the building in subdirectories is
>> handled by recursively calling make (this is one of the problems that
>> arise by recursively calling make for each directory, but that is a
>> different topic).
>>
>> Any ideas? Using the current Makefile structure, I see no other solution
>> than defining the mach-types.h generation rules in the toplevel Makefile.
>>     
>
> do you need the mach-types file for anything else ?  if not, dont keep that in 
> git, keep the generated header.
>   
Then we are back at basically the current state.
If I do this, we need no Makefile support at all - we just need a script 
that is run by the maintainer that downloads the current mach-types from 
the web, and generates mach-types.h, which is then checked in.

cu
Michael
Mike Frysinger - May 9, 2011, 4:16 p.m.
On Monday, May 09, 2011 11:54:15 Michael Schwingen wrote:
> Then we are back at basically the current state.
> If I do this, we need no Makefile support at all - we just need a script
> that is run by the maintainer that downloads the current mach-types from
> the web, and generates mach-types.h, which is then checked in.

i'm ok with that, but i'm not an arm maintainer ;)
-mike
Albert ARIBAUD - May 10, 2011, 9:50 p.m.
Hi all,

Le 09/05/2011 18:16, Mike Frysinger a écrit :
> On Monday, May 09, 2011 11:54:15 Michael Schwingen wrote:
>> Then we are back at basically the current state.
>> If I do this, we need no Makefile support at all - we just need a script
>> that is run by the maintainer that downloads the current mach-types from
>> the web, and generates mach-types.h, which is then checked in.
>
> i'm ok with that, but i'm not an arm maintainer ;)
> -mike

I'm the ARM maintainer -- custodian, actually -- and I approve this 
message. :)

Amicalement,

Patch

diff --git a/Makefile b/Makefile
index 384a59e..07ab7fb 100644
--- a/Makefile
+++ b/Makefile
@@ -469,7 +469,7 @@  $(obj)System.map:	$(obj)u-boot
 # This target actually generates 2 files; autoconf.mk and autoconf.mk.dep.
 # the dep file is only include in this top level makefile to determine when
 # to regenerate the autoconf.mk file.
-$(obj)include/autoconf.mk.dep: $(obj)include/config.h include/common.h
+$(obj)include/autoconf.mk.dep: $(obj)include/config.h include/common.h $(obj)include/asm/mach-types.h
 	@$(XECHO) Generating $@ ; \
 	set -e ; \
 	: Generate the dependancies ; \
@@ -530,13 +530,18 @@  unconfig:
 		$(obj)board/*/config.tmp $(obj)board/*/*/config.tmp \
 		$(obj)include/autoconf.mk $(obj)include/autoconf.mk.dep
 
-%_config::	unconfig
+%_config::	unconfig $(obj)include/asm/mach-types.h
 	@$(MKCONFIG) -A $(@:_config=)
 
 sinclude $(obj).boards.depend
 $(obj).boards.depend:	boards.cfg
 	awk '(NF && $$1 !~ /^#/) { print $$1 ": " $$1 "_config; $$(MAKE)" }' $< > $@
 
+
+$(obj)include/asm/mach-types.h: arch/arm/tools/gen-mach-types arch/arm/tools/mach-types 
+	@mkdir -p $(obj)include/asm
+	awk -f $^ > $@ || { rm -f $@; /bin/false; }
+
 #
 # Functions to generate common board directory names
 #
diff --git a/arch/arm/include/asm/mach-types.h b/arch/arm/include/asm/mach-types.h
deleted file mode 100644


diff --git a/arch/arm/tools/gen-mach-types b/arch/arm/tools/gen-mach-types
new file mode 100644
index 0000000..04fef71
--- /dev/null
+++ b/arch/arm/tools/gen-mach-types
@@ -0,0 +1,72 @@ 
+#!/bin/awk
+#
+# Awk script to generate include/generated/mach-types.h
+#
+BEGIN	{ nr = 0 }
+/^#/	{ next }
+/^[ 	]*$/ { next }
+
+NF == 4 {
+	  machine_is[nr] = "machine_is_"$1;
+	  config[nr] = "CONFIG_"$2;
+	  mach_type[nr] = "MACH_TYPE_"$3;
+	  num[nr] = $4; nr++
+	}
+
+NF == 3 {
+	  machine_is[nr] = "machine_is_"$1;
+	  config[nr] = "CONFIG_"$2;
+	  mach_type[nr] = "MACH_TYPE_"$3;
+	  num[nr] = ""; nr++
+	}
+
+
+END	{
+	  printf("/*\n");
+	  printf(" * This was automagically generated from %s!\n", FILENAME);
+	  printf(" * Do NOT edit\n");
+	  printf(" */\n\n");
+	  printf("#ifndef __ASM_ARM_MACH_TYPE_H\n");
+	  printf("#define __ASM_ARM_MACH_TYPE_H\n\n");
+	  printf("#ifndef __ASSEMBLY__\n");
+	  printf("/* The type of machine we're running on */\n");
+	  printf("extern unsigned int __machine_arch_type;\n");
+	  printf("#endif\n\n");
+
+	  printf("/* see arch/arm/kernel/arch.c for a description of these */\n");
+	  for (i = 0; i < nr; i++)
+	    if (num[i] ~ /..*/)
+	      printf("#define %-30s %d\n", mach_type[i], num[i]);
+
+	  printf("\n");
+
+	  for (i = 0; i < nr; i++)
+	    if (num[i] ~ /..*/) {
+	      printf("#ifdef %s\n", config[i]);
+	      printf("# ifdef machine_arch_type\n");
+	      printf("#  undef machine_arch_type\n");
+	      printf("#  define machine_arch_type\t__machine_arch_type\n");
+	      printf("# else\n");
+	      printf("#  define machine_arch_type\t%s\n", mach_type[i]);
+	      printf("# endif\n");
+	      printf("# define %s()\t(machine_arch_type == %s)\n", machine_is[i], mach_type[i]);
+	      printf("#else\n");
+	      printf("# define %s()\t(0)\n", machine_is[i]);
+	      printf("#endif\n\n");
+	    }
+
+	  printf("/*\n * These have not yet been registered\n */\n");
+	  for (i = 0; i < nr; i++)
+	    if (num[i] !~ /..*/)
+	      printf("/* #define %-30s <<not registered>> */\n", mach_type[i]);
+
+	  for (i = 0; i < nr; i++)
+	    if (num[i] !~ /..*/) {
+	      printf("#define %s()\t(0)\n", machine_is[i]);
+	    }
+
+	  printf("\n#ifndef machine_arch_type\n");
+	  printf("#define machine_arch_type\t__machine_arch_type\n");
+	  printf("#endif\n\n");
+	  printf("#endif\n");
+	}
diff --git a/arch/arm/tools/mach-types b/arch/arm/tools/mach-types
new file mode 100644
index 0000000..957e4c9
--- /dev/null
+++ b/arch/arm/tools/mach-types
@@ -0,1 +1,38 @@ 
+# Database of machine macros and numbers
+#
+# This file is linux/arch/arm/tools/mach-types
+#
+# Up to date versions of this file can be obtained from:
+#
+#   http://www.arm.linux.org.uk/developer/machines/download.php
+#
+# Please do not send patches to this file; it is automatically generated!
+# To add an entry into this database, please see Documentation/arm/README,
+# or visit:
+#
+#   http://www.arm.linux.org.uk/developer/machines/?action=new
+#
+# Last update: Thu May 5 21:54:46 2011
+#
+# machine_is_xxx	CONFIG_xxxx		MACH_TYPE_xxx		number
+#
+ebsa110			ARCH_EBSA110		EBSA110			0
+riscpc			ARCH_RPC		RISCPC			1
+nexuspci		ARCH_NEXUSPCI		NEXUSPCI		3
+ebsa285			ARCH_EBSA285		EBSA285			4
+netwinder		ARCH_NETWINDER		NETWINDER		5
+cats			ARCH_CATS		CATS			6
+tbox			ARCH_TBOX		TBOX			7
+co285			ARCH_CO285		CO285			8
+clps7110		ARCH_CLPS7110		CLPS7110		9
+archimedes		ARCH_ARC		ARCHIMEDES		10
+a5k			ARCH_A5K		A5K			11
+etoile			ARCH_ETOILE		ETOILE			12
+lacie_nas		ARCH_LACIE_NAS		LACIE_NAS		13
+clps7500		ARCH_CLPS7500		CLPS7500		14
+shark			ARCH_SHARK		SHARK			15
+brutus			SA1100_BRUTUS		BRUTUS			16
+personal_server		ARCH_PERSONAL_SERVER	PERSONAL_SERVER		17
+itsy			SA1100_ITSY		ITSY			18
+l7200			ARCH_L7200		L7200			19
+pleb			SA1100_PLEB		PLEB			20
--