| Message ID | 20180813063652.20534-3-joel@jms.id.au |
|---|---|
| State | Accepted |
| Headers | show |
| Series | CI updates for make check | expand |
| Context | Check | Description |
|---|---|---|
| snowpatch_ozlabs/apply_patch | success | master/apply_patch Successfully applied |
| snowpatch_ozlabs/build-multiarch | success | Test build-multiarch on branch master |
On Mon, 2018-08-13 at 16:06 +0930, Joel Stanley wrote: > This builds the software for the host architecture and runs make > check > inside a container. > > Added to the travis configuration. > > Signed-off-by: Joel Stanley <joel@jms.id.au> > --- > .travis.yml | 1 + > utils/test.sh | 33 +++++++++++++++++++++++++++++++++ > 2 files changed, 34 insertions(+) > create mode 100755 utils/test.sh > > diff --git a/.travis.yml b/.travis.yml > index 9e1a86bc3140..596b50ff4e1e 100644 > --- a/.travis.yml > +++ b/.travis.yml > @@ -3,3 +3,4 @@ services: > > script: > - ./utils/build.sh > + - ./utils/test.sh > diff --git a/utils/test.sh b/utils/test.sh > new file mode 100755 > index 000000000000..e2bd25893910 > --- /dev/null > +++ b/utils/test.sh > @@ -0,0 +1,33 @@ > +#!/bin/bash > + > +set -e > + > +CONTAINER=pdbg-check > + > +Dockerfile=$(cat << EOF > +FROM ubuntu:18.04 > +RUN apt-get update && apt-get install --no-install-recommends -yy \ > + make \ > + gcc \ > + autoconf \ > + automake \ > + libtool \ > + git \ > + device-tree-compiler > +RUN groupadd -g ${GROUPS} ${USER} && useradd -d ${HOME} -m -u ${UID} > -g ${GROUPS} ${USER} > +USER ${USER} > +ENV HOME ${HOME} > +RUN /bin/bash > +EOF > +) > + > +docker pull ubuntu:18.04 > +docker build -t ${CONTAINER} - <<< "${Dockerfile}" > + > +RUN="docker run --rm=true --user=${USER} -w ${PWD} -v > ${HOME}:${HOME} -t ${CONTAINER}" > + > +${RUN} ./bootstrap.sh > +${RUN} ./configure > +${RUN} make clean Does travis cache the older image? Or it's always created from scratch? If it's created from scratch, then we don't really need "make clean". If not, we also should add "git clean -x -d -f" before running "./bootstrap.sh" The same logic applies to build.sh. > +${RUN} make > +${RUN} make check > -- > 2.17.1 > Amitay.
Hi Amitay, On Mon, 13 Aug 2018 at 16:56, Amitay Isaacs <amitay@ozlabs.org> wrote: > > +docker pull ubuntu:18.04 > > +docker build -t ${CONTAINER} - <<< "${Dockerfile}" > > + > > +RUN="docker run --rm=true --user=${USER} -w ${PWD} -v > > ${HOME}:${HOME} -t ${CONTAINER}" > > + > > +${RUN} ./bootstrap.sh > > +${RUN} ./configure > > +${RUN} make clean > > Does travis cache the older image? Or it's always created from > scratch? If it's created from scratch, then we don't really need "make > clean". If not, we also should add "git clean -x -d -f" before running > "./bootstrap.sh" Travis doesn't appear to cache the containers. We do cache them on openpower.xyz's Jenkins. When I use them locally, it is cached. We don't want to do a git clean in the script as that might have unexpected results for people using it in their local tree. For our CI (Travis, Jenkins) the workspace will be clean each time, so the make clean is unnecessary. This these files are not just for travis, but is intended for anyone wanting to do a build. Hence the make clean which is useful if the previous build was a cross-compile. Without the clean a user might get this: CCLD pdbg /usr/bin/x86_64-linux-gnu-ld: unknown architecture of input file `fake.dtb.o' is incompatible with i386:x86-64 output collect2: error: ld returned 1 exit status Cheers, Joel > > The same logic applies to build.sh. > > > +${RUN} make > > +${RUN} make check > > -- > > 2.17.1 > > > > Amitay. > -- > > Everyone is a genius at least once a year. A real genius has his ideas > closer together. >
On Mon, 2018-08-13 at 17:11 +0930, Joel Stanley wrote: > Hi Amitay, > > On Mon, 13 Aug 2018 at 16:56, Amitay Isaacs <amitay@ozlabs.org> > wrote: > > > > +docker pull ubuntu:18.04 > > > +docker build -t ${CONTAINER} - <<< "${Dockerfile}" > > > + > > > +RUN="docker run --rm=true --user=${USER} -w ${PWD} -v > > > ${HOME}:${HOME} -t ${CONTAINER}" > > > + > > > +${RUN} ./bootstrap.sh > > > +${RUN} ./configure > > > +${RUN} make clean > > > > Does travis cache the older image? Or it's always created from > > scratch? If it's created from scratch, then we don't really need > > "make > > clean". If not, we also should add "git clean -x -d -f" before > > running > > "./bootstrap.sh" > > Travis doesn't appear to cache the containers. We do cache them on > openpower.xyz's Jenkins. When I use them locally, it is cached. > > We don't want to do a git clean in the script as that might have > unexpected results for people using it in their local tree. > > For our CI (Travis, Jenkins) the workspace will be clean each time, > so > the make clean is unnecessary. This these files are not just for > travis, but is intended for anyone wanting to do a build. Hence the > make clean which is useful if the previous build was a cross-compile. > Without the clean a user might get this: > > CCLD pdbg > /usr/bin/x86_64-linux-gnu-ld: unknown architecture of input file > `fake.dtb.o' is incompatible with i386:x86-64 output > collect2: error: ld returned 1 exit status More confusion. So docker is only for access to cross-compilers? I assumed that the build was in a directory in the docker container. Didn't realize that it's actually the directory on the host. Amitay.
On Mon, 13 Aug 2018 at 17:30, Amitay Isaacs <amitay@ozlabs.org> wrote: > > On Mon, 2018-08-13 at 17:11 +0930, Joel Stanley wrote: > > Hi Amitay, > > > > On Mon, 13 Aug 2018 at 16:56, Amitay Isaacs <amitay@ozlabs.org> > > wrote: > > > > > > +docker pull ubuntu:18.04 > > > > +docker build -t ${CONTAINER} - <<< "${Dockerfile}" > > > > + > > > > +RUN="docker run --rm=true --user=${USER} -w ${PWD} -v > > > > ${HOME}:${HOME} -t ${CONTAINER}" > > > > + > > > > +${RUN} ./bootstrap.sh > > > > +${RUN} ./configure > > > > +${RUN} make clean > > > > > > Does travis cache the older image? Or it's always created from > > > scratch? If it's created from scratch, then we don't really need > > > "make > > > clean". If not, we also should add "git clean -x -d -f" before > > > running > > > "./bootstrap.sh" > > > > Travis doesn't appear to cache the containers. We do cache them on > > openpower.xyz's Jenkins. When I use them locally, it is cached. > > > > We don't want to do a git clean in the script as that might have > > unexpected results for people using it in their local tree. > > > > For our CI (Travis, Jenkins) the workspace will be clean each time, > > so > > the make clean is unnecessary. This these files are not just for > > travis, but is intended for anyone wanting to do a build. Hence the > > make clean which is useful if the previous build was a cross-compile. > > Without the clean a user might get this: > > > > CCLD pdbg > > /usr/bin/x86_64-linux-gnu-ld: unknown architecture of input file > > `fake.dtb.o' is incompatible with i386:x86-64 output > > collect2: error: ld returned 1 exit status > > More confusion. So docker is only for access to cross-compilers? > I assumed that the build was in a directory in the docker container. > Didn't realize that it's actually the directory on the host. We use docker's "mount volume" option to mount the source directory at the same path as it exists on the host machine. This allows the user (or CI) to build the tool without necessarily having the tools installed on their host machine (or the same versions). I first used this technique when developing a systemd application for openbmc, when openbmc's libsystemd was newer than the one on my machine. You can still use your host's editor and what not, but the build process is isolated. I find it useful even for small tools without many external dependencies like pdbg, as it lets me test on a released version of the distro, and you find out when a new dependency has been added pretty fast. Cheers, Joel
diff --git a/.travis.yml b/.travis.yml index 9e1a86bc3140..596b50ff4e1e 100644 --- a/.travis.yml +++ b/.travis.yml @@ -3,3 +3,4 @@ services: script: - ./utils/build.sh + - ./utils/test.sh diff --git a/utils/test.sh b/utils/test.sh new file mode 100755 index 000000000000..e2bd25893910 --- /dev/null +++ b/utils/test.sh @@ -0,0 +1,33 @@ +#!/bin/bash + +set -e + +CONTAINER=pdbg-check + +Dockerfile=$(cat << EOF +FROM ubuntu:18.04 +RUN apt-get update && apt-get install --no-install-recommends -yy \ + make \ + gcc \ + autoconf \ + automake \ + libtool \ + git \ + device-tree-compiler +RUN groupadd -g ${GROUPS} ${USER} && useradd -d ${HOME} -m -u ${UID} -g ${GROUPS} ${USER} +USER ${USER} +ENV HOME ${HOME} +RUN /bin/bash +EOF +) + +docker pull ubuntu:18.04 +docker build -t ${CONTAINER} - <<< "${Dockerfile}" + +RUN="docker run --rm=true --user=${USER} -w ${PWD} -v ${HOME}:${HOME} -t ${CONTAINER}" + +${RUN} ./bootstrap.sh +${RUN} ./configure +${RUN} make clean +${RUN} make +${RUN} make check
This builds the software for the host architecture and runs make check inside a container. Added to the travis configuration. Signed-off-by: Joel Stanley <joel@jms.id.au> --- .travis.yml | 1 + utils/test.sh | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 34 insertions(+) create mode 100755 utils/test.sh