[v5,3/3] configure/Make: Build libfdt from submodule DTC

From: Peter Crosthwaite <peter.crosthwaite@xilinx.com>

Configure to use the DTC submodule to build libfdt when no system libfdt
is found. Prompt to install the DTC submodule if --enable-fdt is set but
no DTC can be found.

DTC has no out-of-tree build capability, so the configure will symlink
the Makefile and scripts directory in the out of tree build directory
and pass in all the needed DTC arguments to make out of tree build happen.
Ideally we fix the DTC make to support out of tree, but did it this way to
avoid commits to DTC.

Signed-off-by: David Holsgrove <david.holsgrove@xilinx.com>
Signed-off-by: Peter Crosthwaite <peter.crosthwaite@xilinx.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
---
changed from v4:
Use error_exit fn (PMM review)
changed from v3:
Rebased against recent devls.
Fixed some cosmetics.
changed from v2:
Fixed quoting of Make arguments (PMM review)
Fixed comment styling and typos (PMM review)

 Makefile  |   10 ++++++++++
 configure |   31 ++++++++++++++++++++++++++-----
 2 files changed, 36 insertions(+), 5 deletions(-)

On Thu, Apr 18, 2013 at 4:47 AM,  <peter.crosthwaite@xilinx.com> wrote:
> From: Peter Crosthwaite <peter.crosthwaite@xilinx.com>
>
> Configure to use the DTC submodule to build libfdt when no system libfdt
> is found. Prompt to install the DTC submodule if --enable-fdt is set but
> no DTC can be found.
>
> DTC has no out-of-tree build capability, so the configure will symlink
> the Makefile and scripts directory in the out of tree build directory
> and pass in all the needed DTC arguments to make out of tree build happen.
> Ideally we fix the DTC make to support out of tree, but did it this way to
> avoid commits to DTC.
>
> Signed-off-by: David Holsgrove <david.holsgrove@xilinx.com>
> Signed-off-by: Peter Crosthwaite <peter.crosthwaite@xilinx.com>
> Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
> ---
> changed from v4:
> Use error_exit fn (PMM review)
> changed from v3:
> Rebased against recent devls.
> Fixed some cosmetics.
> changed from v2:
> Fixed quoting of Make arguments (PMM review)
> Fixed comment styling and typos (PMM review)
>
>  Makefile  |   10 ++++++++++
>  configure |   31 ++++++++++++++++++++++++++-----
>  2 files changed, 36 insertions(+), 5 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index 67f19f2..88375dc 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -145,6 +145,15 @@ pixman/Makefile: $(SRC_PATH)/pixman/configure
>  $(SRC_PATH)/pixman/configure:
>         (cd $(SRC_PATH)/pixman; autoreconf -v --install)
>
> +DTC_MAKE_ARGS=-I$(SRC_PATH)/dtc VPATH=$(SRC_PATH)/dtc -C dtc V="$(V)" LIBFDT_srcdir=$(SRC_PATH)/dtc/libfdt
> +DTC_CFLAGS=$(CFLAGS) $(extra_cflags) -I$(BUILD_DIR)/dtc -I$(SRC_PATH)/dtc -I$(SRC_PATH)/dtc/libfdt

This is not entirely correct, I get these errors:
mingw32:
     CC libfdt/fdt.o
cc1: warnings being treated as errors
/src/qemu/dtc/libfdt/fdt.c:1: warning: -fPIC ignored for target (all
code is position independent)

Other flags, including -fPIC, is missing for Clang:
     LD libfdt/libfdt-1.3.0.so
/usr/bin/ld: libfdt/fdt.o: relocation R_X86_64_32S against `.rodata'
can not be used when making a shared object; recompile with -fPIC

My build script also passes additional CFLAGS to both.

Perhaps the fix is to use QEMU_CFLAGS instead of (or in addition to)
CFLAGS above for DTC_CFLAGS? Or add -fPIC as with pixman?

> +
> +subdir-dtc:dtc/libfdt dtc/tests
> +       $(call quiet-command,$(MAKE) $(DTC_MAKE_ARGS) CPPFLAGS="$(DTC_CFLAGS)" LDFLAGS="$(LDFLAGS)" ARFLAGS="$(ARFLAGS)" CC="$(CC)" AR="$(AR)" LD="$(LD)" $(SUBDIR_MAKEFLAGS) libfdt,)
> +
> +dtc/%:
> +       mkdir -p $@
> +
>  $(SUBDIR_RULES): libqemuutil.a libqemustub.a $(common-obj-y)
>
>  ROMSUBDIR_RULES=$(patsubst %,romsubdir-%, $(ROMS))
> @@ -262,6 +271,7 @@ distclean: clean
>         rm -rf $$d || exit 1 ; \
>          done
>         if test -f pixman/config.log; then make -C pixman distclean; fi
> +       if test -f dtc/version_gen.h; then make $(DTC_MAKE_ARGS) clean; fi
>
>  KEYMAPS=da     en-gb  et  fr     fr-ch  is  lt  modifiers  no  pt-br  sv \
>  ar      de     en-us  fi  fr-be  hr     it  lv  nl         pl  ru     th \
> diff --git a/configure b/configure
> index d5f1516..7546799 100755
> --- a/configure
> +++ b/configure
> @@ -2496,17 +2496,34 @@ if test "$fdt" != "no" ; then
>  int main(void) { return 0; }
>  EOF
>    if compile_prog "" "$fdt_libs" ; then
> +    # system DTC is good - use it
>      fdt=yes
> -    libs_softmmu="$libs_softmmu $fdt_libs"
> -  else
> -    if test "$fdt" = "yes" ; then
> -      feature_not_found "fdt"
> +  elif test -d ${source_path}/dtc/libfdt ; then
> +    # have submodule DTC - use it
> +    fdt=yes
> +    dtc_internal="yes"
> +    mkdir -p dtc
> +    if [ "$source_path" != `pwd` ] ; then
> +       symlink "$source_path/dtc/Makefile" "dtc/Makefile"
> +       symlink "$source_path/dtc/scripts" "dtc/scripts"
>      fi
> +    fdt_cflags="-I\$(SRC_PATH)/dtc/libfdt"
> +    fdt_libs="-L\$(BUILD_DIR)/dtc/libfdt $fdt_libs"
> +  elif test "$fdt" = "yes" ; then
> +    # have neither and want - prompt for system/submodule install
> +    error_exit "ERROR: DTC not present. Your options:" \
> +        "  (1) Preferred: Install the DTC devel package" \
> +        "  (2) Fetch the DTC submodule, using:" \
> +        "      git submodule update --init dtc"
> +  else
> +    # don't have and don't want
>      fdt_libs=
>      fdt=no
>    fi
>  fi
>
> +libs_softmmu="$libs_softmmu $fdt_libs"
> +
>  ##########################################
>  # GLX probe, used by milkymist-tmu2
>  if test "$glx" != "no" ; then
> @@ -3452,7 +3469,7 @@ if test "$cpu" = "ppc64" -a "$targetos" != "Darwin" ; then
>  fi
>
>  # add pixman flags after all config tests are done
> -QEMU_CFLAGS="$QEMU_CFLAGS $pixman_cflags"
> +QEMU_CFLAGS="$QEMU_CFLAGS $pixman_cflags $fdt_cflags"
>  libs_softmmu="$libs_softmmu $pixman_libs"
>
>  echo "Install prefix    $prefix"
> @@ -4515,6 +4532,10 @@ if [ "$pixman" = "internal" ]; then
>    echo "config-host.h: subdir-pixman" >> $config_host_mak
>  fi
>
> +if [ "$dtc_internal" = "yes" ]; then
> +  echo "config-host.h: subdir-dtc" >> $config_host_mak
> +fi
> +
>  # build tree in object directory in case the source is not in the current directory
>  DIRS="tests tests/tcg tests/tcg/cris tests/tcg/lm32 tests/libqos"
>  DIRS="$DIRS pc-bios/optionrom pc-bios/spapr-rtas"
> --
> 1.7.0.4
>
>

Il 20/04/2013 20:42, Blue Swirl ha scritto:
> This is not entirely correct, I get these errors:
> mingw32:
>      CC libfdt/fdt.o
> cc1: warnings being treated as errors
> /src/qemu/dtc/libfdt/fdt.c:1: warning: -fPIC ignored for target (all
> code is position independent)
> 
> Other flags, including -fPIC, is missing for Clang:
>      LD libfdt/libfdt-1.3.0.so
> /usr/bin/ld: libfdt/fdt.o: relocation R_X86_64_32S against `.rodata'
> can not be used when making a shared object; recompile with -fPIC
> 
> My build script also passes additional CFLAGS to both.
> 
> Perhaps the fix is to use QEMU_CFLAGS instead of (or in addition to)
> CFLAGS above for DTC_CFLAGS? Or add -fPIC as with pixman?

libfdt is not that portable, it doesn't use libtool to build shared
libraries.

Perhaps we can just build the static library?

Paolo

Hi Paolo, Blue,

On Sun, Apr 21, 2013 at 11:16 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
> Il 20/04/2013 20:42, Blue Swirl ha scritto:
>> This is not entirely correct, I get these errors:
>> mingw32:
>>      CC libfdt/fdt.o
>> cc1: warnings being treated as errors
>> /src/qemu/dtc/libfdt/fdt.c:1: warning: -fPIC ignored for target (all
>> code is position independent)
>>

This is coming from the dtc makefile which explicitly passes in -fPIC
via its default CFLAGS. I tried fixing by setting the Make invocation
to replace CFLAGS entirely, which in effect removes the -fPIC: But I
suffer a similar (later) failure to the one you posted below:

  LINK  arm-softmmu/qemu-system-arm
/usr/bin/ld: /home/peterc/Petalogix/Internal/plgx_src/qemu-build/dtc/libfdt/libfdt.a(fdt.o):
relocation R_X86_64_32S against `.rodata' can not be used when making
a shared object; recompile with -fPIC
/home/peterc/Petalogix/Internal/plgx_src/qemu-build/dtc/libfdt/libfdt.a:
could not read symbols: Bad value

Looks like QEMU mandates that all static objects are -fPIC. A bit of
googling suggests that this error is mingw specific in that mingw
can't handle -fPIC.

>> Other flags, including -fPIC, is missing for Clang:
>>      LD libfdt/libfdt-1.3.0.so
>> /usr/bin/ld: libfdt/fdt.o: relocation R_X86_64_32S against `.rodata'
>> can not be used when making a shared object; recompile with -fPIC
>>

Is this the only clang failure? If so, I'd like to favour Paolos
suggestion of just not building the .so and then we can ignore this
altogether.

>> My build script also passes additional CFLAGS to both.
>>
>> Perhaps the fix is to use QEMU_CFLAGS instead of (or in addition to)
>> CFLAGS above for DTC_CFLAGS?

CFLAGS + QEMU_CFLAGS works as of today, although I was hesitant to do
this at first as it exposes us to issues if QEMU specific CFLAGS are
added in the future than cause DTC build breakage. But ive put the fix
on list to pass in QEMU_CFLAGS as it should unbreak your build.

>> Or add -fPIC as with pixman?

Wont that just expose us to the same mingw werror failure above?

>
> libfdt is not that portable, it doesn't use libtool to build shared
> libraries.
>
> Perhaps we can just build the static library?
>

Seems reasonable, and got this working with fairly low effort. Fix on list.

Regards,
Peter

> Paolo
>
>

Il 22/04/2013 06:45, Peter Crosthwaite ha scritto:
> Hi Paolo, Blue,
> 
> On Sun, Apr 21, 2013 at 11:16 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
>> Il 20/04/2013 20:42, Blue Swirl ha scritto:
>>> This is not entirely correct, I get these errors:
>>> mingw32:
>>>      CC libfdt/fdt.o
>>> cc1: warnings being treated as errors
>>> /src/qemu/dtc/libfdt/fdt.c:1: warning: -fPIC ignored for target (all
>>> code is position independent)
>>>
> 
> This is coming from the dtc makefile which explicitly passes in -fPIC
> via its default CFLAGS. I tried fixing by setting the Make invocation
> to replace CFLAGS entirely, which in effect removes the -fPIC: But I
> suffer a similar (later) failure to the one you posted below:
> 
>   LINK  arm-softmmu/qemu-system-arm
> /usr/bin/ld: /home/peterc/Petalogix/Internal/plgx_src/qemu-build/dtc/libfdt/libfdt.a(fdt.o):
> relocation R_X86_64_32S against `.rodata' can not be used when making
> a shared object; recompile with -fPIC
> /home/peterc/Petalogix/Internal/plgx_src/qemu-build/dtc/libfdt/libfdt.a:
> could not read symbols: Bad value
> 
> Looks like QEMU mandates that all static objects are -fPIC. A bit of
> googling suggests that this error is mingw specific in that mingw
> can't handle -fPIC.

Right, that's because QEMU is built as a position-independent executable.

PIE requirements actually are a bit less strict than -fPIC.  Your
solution of passing the compilation flags sounds good, but do you pass
-Werror too?

Paolo

On 22 April 2013 05:45, Peter Crosthwaite <peter.crosthwaite@xilinx.com> wrote:
> On Sun, Apr 21, 2013 at 11:16 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
>> libfdt is not that portable, it doesn't use libtool to build shared
>> libraries.
>>
>> Perhaps we can just build the static library?
>>
>
> Seems reasonable, and got this working with fairly low effort.
> Fix on list.

Hi -- can you point me at the fix? Did it get committed?
I did a search of my mail archives but couldn't find anything
that looked like it.

(I'm looking at making libfdt mandatory for softmmu targets, for
which purpose we need to make sure all the libfdt build hiccups
are fixed...)

thanks
-- PMM

On 24 May 2013 15:33, Edgar E. Iglesias <edgar.iglesias@gmail.com> wrote:
> On Fri, May 24, 2013 at 03:16:00PM +0100, Peter Maydell wrote:
>> Hi -- can you point me at the fix? Did it get committed?
>> I did a search of my mail archives but couldn't find anything
>> that looked like it.
>>
>> (I'm looking at making libfdt mandatory for softmmu targets, for
>> which purpose we need to make sure all the libfdt build hiccups
>> are fixed...)

> I think it was this one:

> commit 9005a51b096272e1834638f8e184706548ee7364
> Author: Peter Crosthwaite <peter.crosthwaite@xilinx.com>
> Date:   Mon Apr 22 14:40:49 2013 +1000
>
>     Makefile: Don't build shared libfdt

Yep, that'll be it. Thanks!

-- PMM