Patchwork Adding support for the Xtensa architecture in crosstool-NG

login
register
mail settings
Submitter Chris Zankel
Date July 25, 2013, 12:09 a.m.
Message ID <51F06CCF.1010704@zankel.net>
Download mbox | patch
Permalink /patch/261562/
State New
Headers show

Comments

Chris Zankel - July 25, 2013, 12:09 a.m.
Hi,

It would be great if you'd consider adding the patches to support the 
Xtensa architecture.

Because it's a configurable architecture, there's a little bit of 
tweaking required if someone wants to build a toolchain for a different 
processor configuration than the default one that comes with the 
toolchain. I have combined that logic into a single function, which each 
component (gcc, bintuils, gdb) must call if building for a non-default 
Xtensa configuration (i.e. when CT_ARCH_XTENSA_CUSTOM_OVERLAY_FILE is 
empty).

Please let me know if you have any questions or feedback.

Thanks,
-Chris
# HG changeset patch
# User Chris Zankel <chris@zankel.net>
# Date 1374703847 25200
# Node ID 40864183a101b211d788ad94f569af324c74e56e
# Parent  58ca31386bfbb48a522724dca79040efd54041c3
xtensa: add support for the configurable Xtensa architecture.

The Xtensa processor architecture is a configurable, extensible,
and synthesizable 32-bit RISC processor core. Processor and SOC vendors
can select from various processor options and even create customized
instructions in addition to a base ISA to tailor the processor for
a particular application.

Because of the configurability, the build process requires one additional
step for gcc, binutils, and gdb to update the default configuration.
These configurations are packed into an 'overlay' tar image, and are
simply untarred on top of the default configuration during the build.
--
For unsubscribe information see http://sourceware.org/lists.html#faq
Yann E. MORIN - Oct. 2, 2013, 9:05 p.m.
Chris, All,

Sorry for the delay. Here are my comments:

On 2013-07-24 17:09 -0700, Chris Zankel spake thusly:
> # HG changeset patch
> # User Chris Zankel <chris@zankel.net>
> # Date 1374703847 25200
> # Node ID 40864183a101b211d788ad94f569af324c74e56e
> # Parent  58ca31386bfbb48a522724dca79040efd54041c3
> xtensa: add support for the configurable Xtensa architecture.
> 
> The Xtensa processor architecture is a configurable, extensible,
> and synthesizable 32-bit RISC processor core. Processor and SOC vendors
> can select from various processor options and even create customized
> instructions in addition to a base ISA to tailor the processor for
> a particular application.
> 
> Because of the configurability, the build process requires one additional
> step for gcc, binutils, and gdb to update the default configuration.
> These configurations are packed into an 'overlay' tar image, and are
> simply untarred on top of the default configuration during the build.

[--SNIP--]

> diff -r 58ca31386bfb -r 40864183a101 scripts/build/arch/xtensa.sh
> --- /dev/null	Thu Jan 01 00:00:00 1970 +0000
> +++ b/scripts/build/arch/xtensa.sh	Wed Jul 24 15:10:47 2013 -0700
> @@ -0,0 +1,75 @@
> +# Compute Xtensa-specific values
> +
> +CT_DoArchTupleValues() {
> +    # The architecture part of the tuple:
> +    CT_TARGET_ARCH="${CT_ARCH}${CT_ARCH_SUFFIX}"

Unneeded, that's the default.

> +    # The system part of the tuple:
> +    case "${CT_LIBC}" in
> +        *glibc)   CT_TARGET_SYS=gnu;;
> +        uClibc)   CT_TARGET_SYS=uclibc;;
> +    esac

Ditto.

> +}
> +
> +# This function updates the specified component (binutils, gcc, gdb, etc.)
> +# with the processor specific configuration.

Ouch! This one is a no-no.

Doing so means the extracted sources of the components can not be
re-used to build another toolchain (eg. another Xtensa toolchain with
another overlay, or for another architecture).

We already have a mechanism in place for the user to provide
"customised" sources, by way of the "custom location", see:
CUSTOM_LOCATION_ROOT_DIR

Basically, I do not like that an architecture has a hook to munge the
components' sources, or that do not have (complete) support in upstream.

> +CT_ConfigureXtensa() {
> +    local component="${1}"
> +    local version="${2}"
> +    local custom_overlay="${CT_ARCH_XTENSA_CUSTOM_OVERLAY_FILE}"
> +    local custom_location="${CT_ARCH_XTENSA_CUSTOM_OVERLAY_LOCATION}"
> +
> +    if [ -z "${CT_ARCH_XTENSA_CUSTOM_OVERLAY_FILE}" ]; then
> +        custom_overlay="xtensa-overlay.tar"
> +    fi
> +
> +    local full_file="${CT_TARBALLS_DIR}/${custom_overlay}"
> +    local basename="${component}-${version}"
> +    local ext
> +
> +    CT_TestAndAbort "${custom_overlay}: CT_CUSTOM_LOCATION_ROOT_DIR or CT_ARCH_XTENSA_CUSTOM_OVERLAY_LOCATION must be set." \
> +        -z "${CT_CUSTOM_LOCATION_ROOT_DIR}" -a -z "${custom_location}"
> +
> +    if [ -n "${CT_CUSTOM_LOCATION_ROOT_DIR}" \
> +         -a -z "${custom_location}" ]; then
> +             custom_location="${CT_CUSTOM_LOCATION_ROOT_DIR}"
> +    fi
> +
> +    if ! ext="$(CT_GetFileExtension "${basename}")"; then
> +        CT_DoLog WARN "'${basename}' not found in '${CT_TARBALLS_DIR}'"

There's no corresponding CT_GetFile. Who's responsible for downloading
that file?

> +        return 1
> +    fi
> +
> +    if [ -e "${CT_SRC_DIR}/.${basename}.configuring" ]; then
> +        CT_DoLog ERROR "The '${basename}' source were partially configured."
> +        CT_DoLog ERROR "Please remove first:"
> +        CT_DoLog ERROR " - the source dir for '${basename}', in '${CT_SRC_DIR}'"
> +        CT_DoLog ERROR " - the file '${CT_SRC_DIR}/.${basename}.extracted'"
> +        CT_DoLog ERROR " - the file '${CT_SRC_DIR}/.${basename}.patch'"
> +        CT_DoLog ERROR " - the file '${CT_SRC_DIR}/.${basename}.configuring'"
> +        CT_Abort
> +    fi
> +
> +    CT_DoLog EXTRA "Using '${custom_config}' from ${custom_location}"
> +    CT_DoExecLog DEBUG ln -sf "${custom_location}/${custom_config}" \
> +                              "${CT_TARBALLS_DIR}/${custom_config}"
> +
> +    CT_DoExecLog DEBUG touch "${CT_SRC_DIR}/.${basename}.configuring"
> +
> +    CT_Pushd "${CT_SRC_DIR}/${basename}"
> +
> +    tar_opts=( "--strip-components=1" )
> +    tar_opts+=( "-xv" )
> +
> +    case "${ext}" in
> +        .tar)           CT_DoExecLog FILE tar "${tar_opts[@]}" -f "${full_file}" "${component}";;
> +        .tar.gz|.tgz)   gzip -dc "${full_file}" | CT_DoExecLog FILE tar "${tar_opts[@]}" -f - "${component}";
> +        *)              CT_DoLog WARN "Don't know how to handle '${basename}${ext}': unknown extension"
> +                        return 1
> +                        ;;
> +    esac
> +
> +    CT_DoExecLog DEBUG touch "${CT_SRC_DIR}/.${basename}.configured"
> +    CT_DoExecLog DEBUG rm -f "${CT_SRC_DIR}/.${basename}.configuring"
> +
> +    CT_Popd
> +}

I'm not sure where to go from there.

Maybe just add "basic" Xtensa support that only requires what is already
in upstream gcc/binutils/gdb.

If the user requires a customised Xtensa configuration, then (s)he'd be
responsible for providing the mangled sources via the "custom location"
mechanism".

Regards,
Yann E. MORIN.
Chris Zankel - Oct. 4, 2013, 6:13 p.m.
Hi Yann,

On 10/2/13 2:05 PM, Yann E. MORIN wrote:
> Chris, All,
>
> Sorry for the delay. Here are my comments:
No worries, and thanks for the feedback.

>> +}
>> +
>> +# This function updates the specified component (binutils, gcc, gdb, etc.)
>> +# with the processor specific configuration.
> Ouch! This one is a no-no.
>
> Doing so means the extracted sources of the components can not be
> re-used to build another toolchain (eg. another Xtensa toolchain with
> another overlay, or for another architecture).
Well, the sources can be re-used for all other architectures. They would 
only need to be updated to build for a different Xtensa configuration. 
The 'overlay' is a simple 'tar' file and only overwrites a couple of key 
xtensa-specific configuration files.

These are the files in the overlay (tar) that are overwritten:

./gdb
./gdb/bfd
./gdb/bfd/xtensa-modules.c
./gdb/gdb
./gdb/gdb/regformats
./gdb/gdb/regformats/reg-xtensa.dat
./gdb/gdb/xtensa-config.c
./gdb/gdb/gdbserver
./gdb/gdb/gdbserver/xtensa-regmap.c
./gdb/include
./gdb/include/xtensa-config.h
./gcc
./gcc/include
./gcc/include/xtensa-config.h
./binutils
./binutils/bfd
./binutils/bfd/xtensa-modules.c
./binutils/ld
./binutils/include
./binutils/include/xtensa-config.h

> We already have a mechanism in place for the user to provide
> "customised" sources, by way of the "custom location", see:
> CUSTOM_LOCATION_ROOT_DIR
Wouldn't those sources extract to the same directory names 
(binutils-2.22, for example), so re-use would be even worse? It would 
also negate the benefit of having a tool to generate a configuration 
specific toolchain.
> Basically, I do not like that an architecture has a hook to munge the
> components' sources, or that do not have (complete) support in upstream.
I certainly understand your objections, but don't really see any good 
alternative. Because only Xtensa specific files are overwritten (no file 
used by other architectures is touched), no other architectures would be 
affected.

>> +CT_ConfigureXtensa() {
>> +    local component="${1}"
>> +    local version="${2}"
>> +    local custom_overlay="${CT_ARCH_XTENSA_CUSTOM_OVERLAY_FILE}"
>> +    local custom_location="${CT_ARCH_XTENSA_CUSTOM_OVERLAY_LOCATION}"
>> +        CT_DoLog WARN "'${basename}' not found in '${CT_TARBALLS_DIR}'"
> There's no corresponding CT_GetFile. Who's responsible for downloading
> that file?
The overlay file must be provided by the vendor that generated the 
specific Xtensa configuration.

> I'm not sure where to go from there.
>
> Maybe just add "basic" Xtensa support that only requires what is already
> in upstream gcc/binutils/gdb.
>
> If the user requires a customised Xtensa configuration, then (s)he'd be
> responsible for providing the mangled sources via the "custom location"
> mechanism".
I'd certainly hope you reconsider :-)  Crosstools-NG is just such a 
great way to generate a custom toolchain for a specific Xtensa 
configuration. I also can't seem to think of any good alternative (I'm 
open for ideas), and because this only affects Xtensa and no other 
architecture, it might not be so bad?


Thanks,
-Chris


--
For unsubscribe information see http://sourceware.org/lists.html#faq

Patch

diff -r 58ca31386bfb -r 40864183a101 config/arch/xtensa.in
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/config/arch/xtensa.in	Wed Jul 24 15:10:47 2013 -0700
@@ -0,0 +1,20 @@ 
+# xtensa specific configuration file
+
+## select ARCH_SUPPORTS_32
+## select ARCH_SUPPORTS_BOTH_MMU
+## select ARCH_DEFAULT_HAS_MMU
+##
+## help The xtensa architecture
+## help
+## help    Xtensa is a configurable and extensible processor architecture.
+## help    Supporting a specific configuration typically requires minor 
+## help    modifications to a small set of configuration files in various
+## help    development tools. This process is automated and only requires
+## help    a configuration specific 'overlay' file.
+## help
+## help    For a custom configuration, select the XTENSA_CUSTOM option and
+## help    provide the name of the overlay file through the
+## help    CT_ARCH_XTENSA_CUSTOM_OVERLAY_FILE option.
+## help
+## help    The default option (ARCH_xtensa_fsf) uses a built-in configuration,
+## help    which may or may not work for a particular Xtensa processor.
diff -r 58ca31386bfb -r 40864183a101 config/arch/xtensa.in.2
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/config/arch/xtensa.in.2	Wed Jul 24 15:10:47 2013 -0700
@@ -0,0 +1,30 @@ 
+choice
+	prompt "Target Architecture Variant"
+	default ARCH_xtensa_fsf
+config XTENSA_CUSTOM
+	bool "Custom Xtensa processor configuration"
+
+config ARCH_xtensa_fsf
+	bool "fsf - Default configuration"
+
+endchoice
+
+config ARCH_XTENSA_CUSTOM_OVERLAY_FILE
+	string "Custom Xtensa process configuration file name"
+	depends on XTENSA_CUSTOM
+	default ""
+	help
+	  Enter the name of the custom processor configuration
+	  overlay file or leave blank to use the default 'xtensa-overlay.tar'.
+	  For more information about this option, please also consult
+	  the 'help' section of the 'Target Architecture Variant'
+	  option above.
+	  
+config ARCH_XTENSA_CUSTOM_OVERLAY_LOCATION
+	string "Full path to custom Xtensa processor configurations"
+	depends on XTENSA_CUSTOM
+	default ""
+	help
+	  Enter the path to the directory for the custom processor
+	  configuration file or leave blank to use the default location:
+	  CT_CUSTOM_LOCATION_ROOT_DIR
diff -r 58ca31386bfb -r 40864183a101 scripts/build/arch/xtensa.sh
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/scripts/build/arch/xtensa.sh	Wed Jul 24 15:10:47 2013 -0700
@@ -0,0 +1,75 @@ 
+# Compute Xtensa-specific values
+
+CT_DoArchTupleValues() {
+    # The architecture part of the tuple:
+    CT_TARGET_ARCH="${CT_ARCH}${CT_ARCH_SUFFIX}"
+    # The system part of the tuple:
+    case "${CT_LIBC}" in
+        *glibc)   CT_TARGET_SYS=gnu;;
+        uClibc)   CT_TARGET_SYS=uclibc;;
+    esac
+}
+
+# This function updates the specified component (binutils, gcc, gdb, etc.)
+# with the processor specific configuration.
+CT_ConfigureXtensa() {
+    local component="${1}"
+    local version="${2}"
+    local custom_overlay="${CT_ARCH_XTENSA_CUSTOM_OVERLAY_FILE}"
+    local custom_location="${CT_ARCH_XTENSA_CUSTOM_OVERLAY_LOCATION}"
+
+    if [ -z "${CT_ARCH_XTENSA_CUSTOM_OVERLAY_FILE}" ]; then
+        custom_overlay="xtensa-overlay.tar"
+    fi
+
+    local full_file="${CT_TARBALLS_DIR}/${custom_overlay}"
+    local basename="${component}-${version}"
+    local ext
+
+    CT_TestAndAbort "${custom_overlay}: CT_CUSTOM_LOCATION_ROOT_DIR or CT_ARCH_XTENSA_CUSTOM_OVERLAY_LOCATION must be set." \
+        -z "${CT_CUSTOM_LOCATION_ROOT_DIR}" -a -z "${custom_location}"
+
+    if [ -n "${CT_CUSTOM_LOCATION_ROOT_DIR}" \
+         -a -z "${custom_location}" ]; then
+             custom_location="${CT_CUSTOM_LOCATION_ROOT_DIR}"
+    fi
+
+    if ! ext="$(CT_GetFileExtension "${basename}")"; then
+        CT_DoLog WARN "'${basename}' not found in '${CT_TARBALLS_DIR}'"
+        return 1
+    fi
+
+    if [ -e "${CT_SRC_DIR}/.${basename}.configuring" ]; then
+        CT_DoLog ERROR "The '${basename}' source were partially configured."
+        CT_DoLog ERROR "Please remove first:"
+        CT_DoLog ERROR " - the source dir for '${basename}', in '${CT_SRC_DIR}'"
+        CT_DoLog ERROR " - the file '${CT_SRC_DIR}/.${basename}.extracted'"
+        CT_DoLog ERROR " - the file '${CT_SRC_DIR}/.${basename}.patch'"
+        CT_DoLog ERROR " - the file '${CT_SRC_DIR}/.${basename}.configuring'"
+        CT_Abort
+    fi
+
+    CT_DoLog EXTRA "Using '${custom_config}' from ${custom_location}"
+    CT_DoExecLog DEBUG ln -sf "${custom_location}/${custom_config}" \
+                              "${CT_TARBALLS_DIR}/${custom_config}"
+
+    CT_DoExecLog DEBUG touch "${CT_SRC_DIR}/.${basename}.configuring"
+
+    CT_Pushd "${CT_SRC_DIR}/${basename}"
+
+    tar_opts=( "--strip-components=1" )
+    tar_opts+=( "-xv" )
+
+    case "${ext}" in
+        .tar)           CT_DoExecLog FILE tar "${tar_opts[@]}" -f "${full_file}" "${component}";;
+        .tar.gz|.tgz)   gzip -dc "${full_file}" | CT_DoExecLog FILE tar "${tar_opts[@]}" -f - "${component}";
+        *)              CT_DoLog WARN "Don't know how to handle '${basename}${ext}': unknown extension"
+                        return 1
+                        ;;
+    esac
+
+    CT_DoExecLog DEBUG touch "${CT_SRC_DIR}/.${basename}.configured"
+    CT_DoExecLog DEBUG rm -f "${CT_SRC_DIR}/.${basename}.configuring"
+
+    CT_Popd
+}
diff -r 58ca31386bfb -r 40864183a101 scripts/build/binutils/binutils.sh
--- a/scripts/build/binutils/binutils.sh	Wed Jul 10 12:40:21 2013 +0100
+++ b/scripts/build/binutils/binutils.sh	Wed Jul 24 15:10:47 2013 -0700
@@ -37,6 +37,10 @@ 
         CT_Extract "elf2flt-${CT_ELF2FLT_VERSION}"
         CT_Patch "elf2flt" "${CT_ELF2FLT_VERSION}"
     fi
+
+    if [ -n "${CT_ARCH_XTENSA_CUSTOM_NAME}" ]; then
+	CT_ConfigureXtensa "binutils" "${CT_BINUTILS_VERSION}"
+    fi
 }
 
 # Build binutils for build -> target
diff -r 58ca31386bfb -r 40864183a101 scripts/build/cc/gcc.sh
--- a/scripts/build/cc/gcc.sh	Wed Jul 10 12:40:21 2013 +0100
+++ b/scripts/build/cc/gcc.sh	Wed Jul 24 15:10:47 2013 -0700
@@ -58,6 +58,10 @@ 
        ]; then
         CT_DoExecLog ALL cp -v "${CT_TARBALLS_DIR}/ecj-latest.jar" "${CT_SRC_DIR}/gcc-${CT_CC_VERSION}/ecj.jar"
     fi
+
+	if [ -n "${CT_ARCH_XTENSA_CUSTOM_NAME}" ]; then
+		CT_ConfigureXtensa "gcc" "${CT_CC_VERSION}"
+	fi
 }
 
 #------------------------------------------------------------------------------
diff -r 58ca31386bfb -r 40864183a101 scripts/build/debug/300-gdb.sh
--- a/scripts/build/debug/300-gdb.sh	Wed Jul 10 12:40:21 2013 +0100
+++ b/scripts/build/debug/300-gdb.sh	Wed Jul 24 15:10:47 2013 -0700
@@ -93,6 +93,10 @@ 
         CT_Extract "expat-${CT_DEBUG_GDB_EXPAT_VERSION}"
         CT_Patch "expat" "${CT_DEBUG_GDB_EXPAT_VERSION}"
     fi
+
+    if [ -n "${CT_ARCH_XTENSA_CUSTOM_OVERLAY_FILE}" ]; then
+        CT_ConfigureXtensa "gdb" "${CT_GDB_VERSION}"
+    fi
 }
 
 do_debug_gdb_build() {