Patchwork [U-Boot,v2,3/5] Allow U-Boot scripts to be placed in a .env file

login
register
mail settings
Submitter Simon Glass
Date June 24, 2013, 8:46 p.m.
Message ID <1372106765-18401-4-git-send-email-sjg@chromium.org>
Download mbox | patch
Permalink /patch/253970/
State Superseded
Delegated to: Tom Rini
Headers show

Comments

Simon Glass - June 24, 2013, 8:46 p.m.
At present U-Boot environment variables, and thus scripts, are defined
by CONFIG_EXTRA_ENV_SETTINGS. It is painful to add large amounts of text
to this file and dealing with quoting and newlines is harder than it
should be. It would be better if we could just type the script into a
text file and have it included by U-Boot.

Add a feature that brings in a .env file associated with the board
config, if present. To use it, create a file in a board/<vendor>/env
directory called <board>.env (or common.env if you want the same
environment for all boards).

The environment variables should be of the form "var=value". Values can
extend to multiple lines. See the README under 'Environment Variables:'
for more information and an example.

Comments are not permitted in the environment with this commit.

Signed-off-by: Simon Glass <sjg@chromium.org>
---
Changes in v2:
- Add additional include to env_embedded to deal with its dirty trick
- Add dependency rule so that the environment is rebuilt when it changes
- Add information and updated example script to README
- Move .env file from include/configs to board/
- Use awk script to process environment since it is much easier on the brain

 Makefile                     | 30 +++++++++++++++++++++++++++++-
 README                       | 34 ++++++++++++++++++++++++++++++++++
 common/env_embedded.c        |  1 +
 config.mk                    |  2 ++
 include/env_default.h        |  2 ++
 mkconfig                     |  6 ++++++
 tools/scripts/env2string.awk | 43 +++++++++++++++++++++++++++++++++++++++++++
 7 files changed, 117 insertions(+), 1 deletion(-)
 create mode 100644 tools/scripts/env2string.awk
Stephen Warren - June 26, 2013, 7:56 p.m.
On 06/24/2013 02:46 PM, Simon Glass wrote:
> At present U-Boot environment variables, and thus scripts, are defined
> by CONFIG_EXTRA_ENV_SETTINGS. It is painful to add large amounts of text
> to this file and dealing with quoting and newlines is harder than it
> should be. It would be better if we could just type the script into a
> text file and have it included by U-Boot.
> 
> Add a feature that brings in a .env file associated with the board
> config, if present. To use it, create a file in a board/<vendor>/env
> directory called <board>.env (or common.env if you want the same
> environment for all boards).

I'm not entirely sure how useful common.env is.

Consider that many different Tegra boards, from different vendors, all
share the same environment. For example boards/compulab/trimslice shares
the core environment with boards/nvidia/*. IIUC, common.env wouldn't
work in this case, since the vendor is different. To solve that, we
could easily symlink board/compulab/env/trimslice.env ->
board/nvidia/env/something.env. But in that case, why not always rely on
creating symlinks, instead of having the common.env fallback work too?
Also consider that compulab makes a ton of boards only some of which use
Tegra, so even a common.env in board/compulab/env wouldn't be that useful.

> diff --git a/Makefile b/Makefile

> +$(obj)include/generated/environment.inc: $(obj)include/generated/environment.in
> +	@$(XECHO) Generating $@ ; \
> +	set -e ; \
> +	: Process the environment file ; \
> +	echo -n "CONFIG_EXTRA_ENV_TEXT=" >$@ ; \
> +	echo -n "#define CONFIG_EXTRA_ENV_TEXT " >$(ENV_HEADER) ; \
> +	awk -f tools/scripts/env2string.awk $< | \
> +		tee -a $(ENV_HEADER) >>$@ ; \

That generates two different files. Doesn't this confuse make a bit,
since if make needs to create $(ENV_HEADER) (which isn't $@ for that
rule) it won't know how to? I've certainly seen problems due to similar
make setups in the past. Perhaps even though it's a bit extra build-time
work, the two files should be generated separately? Or, is there some
reason here that there won't be a problem?

> diff --git a/README b/README

> +For example, for snapper9260 you would create a text file called
> +board/bluewater/env/snapper9260.env containing the environment text.
> +
> +>>>
> +bootcmd=
> +	if [ -z ${tftpserverip} ]; then
> +		echo "Use 'setenv tftpserverip a.b.c.d' to set IP address."
> +	fi
> +
> +	usb start; setenv autoload n; bootp;
> +	tftpboot ${tftpserverip}:
> +	bootm
> +failed=
> +	echo boot failed - please check your image
> +<<<

Presumably the parser looks for something like /^[A-Z]=/i to know when
to "switch" to a new environment variable. Is some form of escaping
useful if you want to include that form of text in your environment
variable's value? If the parser is somehow smarter than that, it might
be worth explaining the syntax structure a little more above.

> diff --git a/tools/scripts/env2string.awk b/tools/scripts/env2string.awk

> +# We begin and end with "

I'm not sure what the " means at the end of that line.
Simon Glass - Oct. 20, 2013, 8:47 p.m.
Hi Stephen,

On Wed, Jun 26, 2013 at 1:56 PM, Stephen Warren <swarren@wwwdotorg.org> wrote:
> On 06/24/2013 02:46 PM, Simon Glass wrote:
>> At present U-Boot environment variables, and thus scripts, are defined
>> by CONFIG_EXTRA_ENV_SETTINGS. It is painful to add large amounts of text
>> to this file and dealing with quoting and newlines is harder than it
>> should be. It would be better if we could just type the script into a
>> text file and have it included by U-Boot.
>>
>> Add a feature that brings in a .env file associated with the board
>> config, if present. To use it, create a file in a board/<vendor>/env
>> directory called <board>.env (or common.env if you want the same
>> environment for all boards).

[well perhaps it is time to get back to this]

>
> I'm not entirely sure how useful common.env is.
>
> Consider that many different Tegra boards, from different vendors, all
> share the same environment. For example boards/compulab/trimslice shares
> the core environment with boards/nvidia/*. IIUC, common.env wouldn't
> work in this case, since the vendor is different. To solve that, we
> could easily symlink board/compulab/env/trimslice.env ->
> board/nvidia/env/something.env. But in that case, why not always rely on
> creating symlinks, instead of having the common.env fallback work too?
> Also consider that compulab makes a ton of boards only some of which use
> Tegra, so even a common.env in board/compulab/env wouldn't be that useful.

I find symlinks a pain in the source tree. I think they should be the
exception rather than the rule. This feature hear mirrors the current
board/<vendor>/common directory which is fairly widely used.

$ ls board/*/common -d |wc -l
21

I feel that Compulab can do their own common file if they want it. A
file common to all Tegra perhaps belongs in arch/arm/.. somewhere?

>
>> diff --git a/Makefile b/Makefile
>
>> +$(obj)include/generated/environment.inc: $(obj)include/generated/environment.in
>> +     @$(XECHO) Generating $@ ; \
>> +     set -e ; \
>> +     : Process the environment file ; \
>> +     echo -n "CONFIG_EXTRA_ENV_TEXT=" >$@ ; \
>> +     echo -n "#define CONFIG_EXTRA_ENV_TEXT " >$(ENV_HEADER) ; \
>> +     awk -f tools/scripts/env2string.awk $< | \
>> +             tee -a $(ENV_HEADER) >>$@ ; \
>
> That generates two different files. Doesn't this confuse make a bit,
> since if make needs to create $(ENV_HEADER) (which isn't $@ for that
> rule) it won't know how to? I've certainly seen problems due to similar
> make setups in the past. Perhaps even though it's a bit extra build-time
> work, the two files should be generated separately? Or, is there some
> reason here that there won't be a problem?

Yes I will separate it. I agree it's a bit icky in that if someone
manually deletes one file they may not get the other. The speed-up is
small.

>
>> diff --git a/README b/README
>
>> +For example, for snapper9260 you would create a text file called
>> +board/bluewater/env/snapper9260.env containing the environment text.
>> +
>> +>>>
>> +bootcmd=
>> +     if [ -z ${tftpserverip} ]; then
>> +             echo "Use 'setenv tftpserverip a.b.c.d' to set IP address."
>> +     fi
>> +
>> +     usb start; setenv autoload n; bootp;
>> +     tftpboot ${tftpserverip}:
>> +     bootm
>> +failed=
>> +     echo boot failed - please check your image
>> +<<<
>
> Presumably the parser looks for something like /^[A-Z]=/i to know when
> to "switch" to a new environment variable. Is some form of escaping
> useful if you want to include that form of text in your environment
> variable's value? If the parser is somehow smarter than that, it might
> be worth explaining the syntax structure a little more above.

The match string is a little more basic than that since there are few
restrictions on environment variable names as far as I know:

"^([^ =][^ =]*)=(.*)"

I'll add some more details to the commit message - but the idea is to
indent your environment to avoid problems.

>
>> diff --git a/tools/scripts/env2string.awk b/tools/scripts/env2string.awk
>
>> +# We begin and end with "
>
> I'm not sure what the " means at the end of that line.

I'll improve that comment.

Regards,
Simon
Stephen Warren - Oct. 25, 2013, 3:34 p.m.
On 10/20/2013 09:47 PM, Simon Glass wrote:
> Hi Stephen,
> 
> On Wed, Jun 26, 2013 at 1:56 PM, Stephen Warren <swarren@wwwdotorg.org> wrote:
>> On 06/24/2013 02:46 PM, Simon Glass wrote:
>>> At present U-Boot environment variables, and thus scripts, are defined
>>> by CONFIG_EXTRA_ENV_SETTINGS. It is painful to add large amounts of text
>>> to this file and dealing with quoting and newlines is harder than it
>>> should be. It would be better if we could just type the script into a
>>> text file and have it included by U-Boot.
>>>
>>> Add a feature that brings in a .env file associated with the board
>>> config, if present. To use it, create a file in a board/<vendor>/env
>>> directory called <board>.env (or common.env if you want the same
>>> environment for all boards).
> 
> [well perhaps it is time to get back to this]
> 
>> I'm not entirely sure how useful common.env is.
>>
>> Consider that many different Tegra boards, from different vendors, all
>> share the same environment. For example boards/compulab/trimslice shares
>> the core environment with boards/nvidia/*. IIUC, common.env wouldn't
>> work in this case, since the vendor is different. To solve that, we
>> could easily symlink board/compulab/env/trimslice.env ->
>> board/nvidia/env/something.env. But in that case, why not always rely on
>> creating symlinks, instead of having the common.env fallback work too?
>> Also consider that compulab makes a ton of boards only some of which use
>> Tegra, so even a common.env in board/compulab/env wouldn't be that useful.
> 
> I find symlinks a pain in the source tree. I think they should be the
> exception rather than the rule. This feature here mirrors the current
> board/<vendor>/common directory which is fairly widely used.

Well, we use includes rather than symlinks, e.g. put the following into
boards/compulab/env/trimslice.env:

#include <boards/nvidia/env/something.env>

> $ ls board/*/common -d |wc -l
> 21
> 
> I feel that Compulab can do their own common file if they want it

I'd rather as many board worked the same way as possible; that makes it
much easier for distros, and even developers. We shouldn't go out of our
way to make that hard.

> A file common to all Tegra perhaps belongs in arch/arm/.. somewhere?

That might be a better location, yes.

Patch

diff --git a/Makefile b/Makefile
index 693b3f2..9dae750 100644
--- a/Makefile
+++ b/Makefile
@@ -695,7 +695,7 @@  $(obj)include/autoconf.mk.dep: $(obj)include/config.h include/common.h
 	$(CC) -x c -DDO_DEPS_ONLY -M $(CFLAGS) $(CPPFLAGS) \
 		-MQ $(obj)include/autoconf.mk include/common.h > $@
 
-$(obj)include/autoconf.mk: $(obj)include/config.h
+$(obj)include/generated/autoconf.mk.base: $(obj)include/config.h
 	@$(XECHO) Generating $@ ; \
 	set -e ; \
 	: Extract the config macros ; \
@@ -703,6 +703,34 @@  $(obj)include/autoconf.mk: $(obj)include/config.h
 		sed -n -f tools/scripts/define2mk.sed > $@.tmp && \
 	mv $@.tmp $@
 
+# We expect '<board>.env' but failing that will use 'common.env'
+ENV_HEADER := $(obj)include/generated/environment.h
+ENV_DIR := $(if $(VENDOR),$(VENDOR)/env,$(BOARD)/env)
+ENV_FILE_BOARD := $(src)board/${ENV_DIR}/$(BOARD).env
+ENV_FILE_COMMON := $(src)board/${ENV_DIR}/common.env
+ENV_FILE := $(if $(wildcard $(ENV_FILE_BOARD)),$(ENV_FILE_BOARD),$(ENV_FILE_COMMON))
+
+# Regenerate the environment if it changes
+$(obj)include/generated/environment.in: $(obj)include/generated/autoconf.mk.base \
+		$(wildcard $(ENV_FILE))
+	if [ -f "$(ENV_FILE)" ]; then \
+		cat $(ENV_FILE) >$@ ; \
+	else \
+		echo -n >$@ ; \
+	fi
+
+$(obj)include/generated/environment.inc: $(obj)include/generated/environment.in
+	@$(XECHO) Generating $@ ; \
+	set -e ; \
+	: Process the environment file ; \
+	echo -n "CONFIG_EXTRA_ENV_TEXT=" >$@ ; \
+	echo -n "#define CONFIG_EXTRA_ENV_TEXT " >$(ENV_HEADER) ; \
+	awk -f tools/scripts/env2string.awk $< | \
+		tee -a $(ENV_HEADER) >>$@ ; \
+
+$(obj)include/autoconf.mk: $(obj)include/generated/environment.inc
+	cat $(obj)include/generated/autoconf.mk.base $< >$@
+
 $(obj)include/generated/generic-asm-offsets.h:	$(obj)include/autoconf.mk.dep \
 	$(obj)lib/asm-offsets.s
 	@$(XECHO) Generating $@
diff --git a/README b/README
index cd0336c..2c8a8c9 100644
--- a/README
+++ b/README
@@ -4363,6 +4363,40 @@  environment. As long as you don't save the environment you are
 working with an in-memory copy. In case the Flash area containing the
 environment is erased by accident, a default environment is provided.
 
+The default environment is created in include/env_default.h, and can be
+augmented by various CONFIG defines. See that file for details. In
+particular you can define CONFIG_EXTRA_ENV_SETTINGS in your board file
+to add environment variables (see 'CONFIG_EXTRA_ENV_SETTINGS' above
+for details).
+
+It is also possible to create an environment file with the name
+board/<vendor>/env/<board>.env for your board. If that file is not present
+then U-Boot will look for board/<vendor>/env/common.env so that you can
+have a common environment for all vendor boards.
+
+This is a plain text file where you can type your environment variables in
+the form 'var=value'. Blank lines and multi-line variables are supported.
+
+For example, for snapper9260 you would create a text file called
+board/bluewater/env/snapper9260.env containing the environment text.
+
+>>>
+bootcmd=
+	if [ -z ${tftpserverip} ]; then
+		echo "Use 'setenv tftpserverip a.b.c.d' to set IP address."
+	fi
+
+	usb start; setenv autoload n; bootp;
+	tftpboot ${tftpserverip}:
+	bootm
+failed=
+	echo boot failed - please check your image
+<<<
+
+The resulting environment can be exported and importing using the
+'env export/import -t' commands.
+
+
 Some configuration options can be set using Environment Variables.
 
 List of environment variables (most likely not complete):
diff --git a/common/env_embedded.c b/common/env_embedded.c
index 52bc687..c8e7ee9 100644
--- a/common/env_embedded.c
+++ b/common/env_embedded.c
@@ -24,6 +24,7 @@ 
 #ifndef __ASSEMBLY__
 #define	__ASSEMBLY__			/* Dirty trick to get only #defines */
 #endif
+#include <generated/environment.h>
 #define	__ASM_STUB_PROCESSOR_H__	/* don't include asm/processor. */
 #include <config.h>
 #undef	__ASSEMBLY__
diff --git a/config.mk b/config.mk
index ddf350e..26385da 100644
--- a/config.mk
+++ b/config.mk
@@ -181,8 +181,10 @@  sinclude $(TOPDIR)/$(CPUDIR)/$(SOC)/config.mk	# include  SoC	specific rules
 endif
 ifdef	VENDOR
 BOARDDIR = $(VENDOR)/$(BOARD)
+ENVDIR=${vendor}/env
 else
 BOARDDIR = $(BOARD)
+ENVDIR=${board}/env
 endif
 ifdef	BOARD
 sinclude $(TOPDIR)/board/$(BOARDDIR)/config.mk	# include board specific rules
diff --git a/include/env_default.h b/include/env_default.h
index 39c5b7c..a394df5 100644
--- a/include/env_default.h
+++ b/include/env_default.h
@@ -137,6 +137,8 @@  const uchar default_environment[] = {
 #ifdef	CONFIG_EXTRA_ENV_SETTINGS
 	CONFIG_EXTRA_ENV_SETTINGS
 #endif
+	/* This is created in the Makefile */
+	CONFIG_EXTRA_ENV_TEXT
 	"\0"
 #ifdef DEFAULT_ENV_INSTANCE_EMBEDDED
 	}
diff --git a/mkconfig b/mkconfig
index 73f852e..7c286a7 100755
--- a/mkconfig
+++ b/mkconfig
@@ -141,8 +141,10 @@  fi
 # Assign board directory to BOARDIR variable
 if [ -z "${vendor}" ] ; then
     BOARDDIR=${board}
+    ENVDIR=${board}/env
 else
     BOARDDIR=${vendor}/${board}
+    ENVDIR=${vendor}/env
 fi
 
 #
@@ -171,10 +173,14 @@  echo "#define CONFIG_SYS_BOARD \"${board}\"" >> config.h
 
 cat << EOF >> config.h
 #define CONFIG_BOARDDIR board/$BOARDDIR
+#define CONFIG_ENVDIR board/$ENVDIR
 #include <config_cmd_defaults.h>
 #include <config_defaults.h>
 #include <configs/${CONFIG_NAME}.h>
 #include <asm/config.h>
+#if !defined(DO_DEPS_ONLY) && !defined(__ASSEMBLY__)
+#include <generated/environment.h>
+#endif
 #include <config_fallbacks.h>
 #include <config_uncmd_spl.h>
 EOF
diff --git a/tools/scripts/env2string.awk b/tools/scripts/env2string.awk
new file mode 100644
index 0000000..2d167c0
--- /dev/null
+++ b/tools/scripts/env2string.awk
@@ -0,0 +1,43 @@ 
+#
+# Sed script to parse a text file containing an environment and convert it
+# to a C string which can be compiled into U-Boot.
+#
+
+# We begin and end with "
+BEGIN {
+	# env holds the env variable we are currently processing
+	env = "";
+	ORS = ""
+	print "\""
+}
+
+{
+	# Quote quotes
+	gsub("\"", "\\\"")
+
+	# Is this the start of a new environment variable?
+	if (match($0, "^([^ =][^ =]*)=(.*)", arr)) {
+		if (length(env) != 0) {
+			# Record the value of the variable now completed
+			vars[var] = env
+		}
+		var = arr[1]
+		env = arr[2]
+	} else {
+		# Change newline to \n
+		env = env "\\n" $0;
+	}
+}
+
+END {
+	# Record the value of the variable now completed
+	if (length(env) != 0) {
+		vars[var] = env
+	}
+
+	# Print out all the variables
+	for (var in vars) {
+		print var "=" vars[var] "\\0";
+	}
+	print "\""
+}